2024-04-26T21:05:03.797 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:05:03.797 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205 --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits} -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/orig.config.yaml 2024-04-26T21:05:03.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:05:03.929 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits} email: null first_in_suite: false job_id: '7675205' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTlg7GVPYpp/X2ihN3cdKqWyO1mj7L8/bv0qnRLPYDHPt1O3G8H40aIN04dESu2r7RGHUU6VtdKY1jZzknFESw= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJa2DV+bFbMG+wuhCXZJ10Lv7sJANC29baJSEzsosys7EtPg9eg2zg3TUiXlpLW6lkT+bpd2O3e7DP2hE4zXkOU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:05:03.929 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:05:03.930 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:05:03.930 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:05:03.932 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:05:03.957 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:05:03.957 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:05:03.957 INFO:teuthology.packaging:ref: None 2024-04-26T21:05:03.957 INFO:teuthology.packaging:tag: None 2024-04-26T21:05:03.957 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:05:03.957 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:03.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:05:04.094 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:05:04.096 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:05:04.103 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:05:04.103 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:05:04.143 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:05:04.160 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:05:04.167 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:05:04.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:00:10.667125', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTlg7GVPYpp/X2ihN3cdKqWyO1mj7L8/bv0qnRLPYDHPt1O3G8H40aIN04dESu2r7RGHUU6VtdKY1jZzknFESw='} 2024-04-26T21:05:04.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:00:10.665907', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJa2DV+bFbMG+wuhCXZJ10Lv7sJANC29baJSEzsosys7EtPg9eg2zg3TUiXlpLW6lkT+bpd2O3e7DP2hE4zXkOU='} 2024-04-26T21:05:04.205 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:05:04.212 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T21:05:04.212 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T21:05:04.212 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:05:04.278 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f86dd231160>, signals=[15]) 2024-04-26T21:05:04.278 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:05:04.285 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:05:04.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:05:04.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:04.368 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:05:04.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:04.442 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:05:04.449 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-26T21:05:04.467 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-26T21:05:04.468 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:05:04.526 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:05:04.527 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:05:04.527 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-26T21:05:04.549 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-04-26T21:05:04.566 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-04-26T21:05:04.567 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-04-26T21:05:04.623 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:05:04.624 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:05:04.625 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:05:04.625 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-04-26T21:05:04.641 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:05:04.651 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:05:04.656 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:05:04.657 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:05:04.659 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:05:04.680 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:05:04.685 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:05:04.686 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:05:04.717 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:05:04.835 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:05:04.881 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:05:04.896 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:05:04.896 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T21:05:04.896 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T21:05:04.897 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:05:04.897 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:05:04.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:05:04.914 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-04-26T21:05:04.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:05:04.915 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-26T21:05:04.971 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:05:04.972 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-437.el9.x86_64 2024-04-26T21:05:04.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-04-26T21:05:04.974 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:05:04.974 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-04-26T21:05:04.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-04-26T21:05:07.152 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:05:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-26T21:05:09.328 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T21:05:09.900 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-26T21:05:13.445 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T21:05:14.653 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-26T21:05:15.087 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-26T21:05:16.176 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-26T21:05:16.769 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T21:05:17.805 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T21:05:21.755 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-26T21:05:23.036 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T21:05:23.611 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:05:23.612 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:05:23.698 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:05:23.699 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-04-26T21:05:23.699 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:05:23.826 DEBUG:teuthology.orchestra.run.smithi184:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:05:24.326 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:05:21 PM UTC. 2024-04-26T21:05:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:05:24.459 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalling: 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:24.460 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:24.461 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 255 k 2024-04-26T21:05:24.461 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 0 2024-04-26T21:05:24.461 INFO:teuthology.orchestra.run.smithi184.stderr:Operation aborted. 2024-04-26T21:05:24.529 INFO:teuthology.orchestra.run.smithi184.stdout:Is this ok [y/N]: 2024-04-26T21:05:24.529 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum reinstall -y kernel || true 2024-04-26T21:05:24.984 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:05:24.984 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:05:25.020 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 09:05:21 PM UTC. 2024-04-26T21:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-26T21:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:05:25.136 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:05:25.136 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalling: 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 255 k 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 0 2024-04-26T21:05:25.137 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-04-26T21:05:25.176 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:05:25.699 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 09:05:23 PM UTC. 2024-04-26T21:05:25.823 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-26T21:05:25.838 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-26T21:05:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:Operation aborted. 2024-04-26T21:05:25.902 INFO:teuthology.orchestra.run.smithi040.stdout:Is this ok [y/N]: 2024-04-26T21:05:25.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2024-04-26T21:05:26.166 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-437.el9.x86_64.rpm 290 kB/s | 255 kB 00:00 2024-04-26T21:05:26.166 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:05:26.167 INFO:teuthology.orchestra.run.smithi184.stdout:Total 248 kB/s | 255 kB 00:01 2024-04-26T21:05:26.186 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-04-26T21:05:26.366 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-04-26T21:05:26.366 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-04-26T21:05:26.371 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-04-26T21:05:26.371 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-04-26T21:05:26.418 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:05:23 PM UTC. 2024-04-26T21:05:26.512 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-04-26T21:05:26.538 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:05:26.539 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:26.539 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-26T21:05:26.539 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:26.539 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-26T21:05:26.539 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-26T21:05:26.540 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-26T21:05:26.596 INFO:teuthology.orchestra.run.smithi184.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:05:26.597 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:27.080 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64.rpm 969 kB/s | 255 kB 00:00 2024-04-26T21:05:27.080 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:05:27.080 INFO:teuthology.orchestra.run.smithi040.stdout:Total 472 kB/s | 255 kB 00:00 2024-04-26T21:05:27.098 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-26T21:05:27.292 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-26T21:05:27.293 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-26T21:05:27.298 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-26T21:05:27.298 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-26T21:05:27.351 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:27.351 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:05:27.430 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-26T21:05:27.498 INFO:teuthology.orchestra.run.smithi040.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:05:27.498 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout:Reinstalled: 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:05:28.030 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:05:28.180 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:28.180 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:05:28.202 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:05:28.249 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:05:28.250 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:05:28.250 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:05:28.250 DEBUG:teuthology.parallel:result is None 2024-04-26T21:05:28.880 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:05:28.880 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:28.880 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalled: 2024-04-26T21:05:28.880 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:05:28.880 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:05:28.881 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:05:29.088 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:05:29.177 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:05:29.177 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:05:29.177 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:05:29.177 DEBUG:teuthology.parallel:result is None 2024-04-26T21:05:29.178 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:05:29.185 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:05:29.185 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:05:29.187 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:05:29.205 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:05:29.211 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:05:29.225 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:05:29.225 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:05:29.243 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:05:29.279 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:05:29.286 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:05:29.286 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:05:29.304 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:05:29.335 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:05:29.351 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:05:29.351 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:05:29.371 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:05:29.373 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:05:29.380 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:05:29.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:05:29.395 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:05:29.439 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:05:29.449 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:05:29.449 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:05:29.460 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:05:29.496 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:05:29.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:05:29.601 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:05:29.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:05:29.722 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:05:29.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:05:29.786 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:05:29.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:05:29.880 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:05:29.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:05:30.004 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:05:30.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:05:30.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-26T21:05:30.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-04-26T21:05:30.105 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:05:30.142 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:05:30.573 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:05:30.583 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:05:30.583 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:05:30.598 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:05:30.607 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:05:30.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-26T21:05:30.683 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-26T21:05:30.685 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-04-26T21:05:30.756 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-04-26T21:05:30.758 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:05:30.758 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-26T21:05:30.791 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-26T21:05:30.791 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-04-26T21:05:30.814 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-04-26T21:05:30.814 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:05:30.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:05:30.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:05:30.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:05:30.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:05:30.881 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:05:30.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:05:30.923 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:05:30.947 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:05:30.957 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T21:05:30.962 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T21:05:31.089 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:05:31.108 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:05:31.109 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:09:02.489 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-04-26T21:09:02.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-26T21:09:02.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:02.570 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-26T21:09:02.650 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-26T21:09:02.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-04-26T21:09:02.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:02.723 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-04-26T21:09:02.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-04-26T21:09:02.801 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:09:02.814 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:09:02.815 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:09:02.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:09:02.817 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:09:02.817 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:09:02.845 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:09:02.859 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:09:02.867 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:09:02.879 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:09:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-26T21:09:02.893 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-26T21:09:02.901 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-04-26T21:09:02.907 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:09:02.912 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-04-26T21:09:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:09:02.925 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:09:02.937 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:09:02.978 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:02.999 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:09:03.002 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:09:03.003 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-04-26T21:09:03.003 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:03.003 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:03.003 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:03.003 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:09:03.004 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:09:03.012 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:09:03.013 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:09:03.013 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:09:03.013 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:09:03.019 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:09:03.019 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:09:03.019 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:09:03.020 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:09:03.022 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:09:03.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:09:03.167 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:09:03.167 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:09:03.170 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:09:03.170 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:09:03.278 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:09:03.279 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:09:03.279 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:09:03.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:09:03.306 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:09:03.354 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:09:03.355 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:09:03.355 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:09:03.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:09:03.379 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:09:03.379 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:09:03.443 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:09:03.447 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:09:03.511 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-26T21:09:03.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-26T21:09:03.516 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:09:03.579 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-04-26T21:09:03.580 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-04-26T21:09:03.750 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-26T21:09:03.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:09:03.806 INFO:teuthology.orchestra.run.smithi184.stdout:48 files removed 2024-04-26T21:09:03.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:09:04.450 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 209 kB/s | 84 kB 00:00 2024-04-26T21:09:04.547 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 183 kB/s | 84 kB 00:00 2024-04-26T21:09:04.884 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-26T21:09:04.990 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-26T21:09:05.243 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-26T21:09:05.357 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-26T21:09:08.457 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-26T21:09:08.801 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-26T21:09:10.786 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T21:09:15.037 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T21:09:16.629 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-26T21:09:19.913 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-26T21:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:09:27.461 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:09:27.462 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:09:27.463 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:09:27.464 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:09:27.465 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:09:27.468 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:09:27.468 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:09:27.532 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-26T21:09:27.541 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:09:27.542 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:09:27.543 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-26T21:09:27.544 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:09:27.545 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:09:27.546 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:09:27.547 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:09:27.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:09:27.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:09:27.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:09:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:09:27.552 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 4 Packages 2024-04-26T21:09:27.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:09:27.554 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 222 M 2024-04-26T21:09:27.554 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-26T21:09:28.532 INFO:teuthology.orchestra.run.smithi040.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-26T21:09:28.732 INFO:teuthology.orchestra.run.smithi040.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.3 MB/s | 889 kB 00:00 2024-04-26T21:09:28.790 INFO:teuthology.orchestra.run.smithi040.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.4 MB/s | 145 kB 00:00 2024-04-26T21:09:29.007 INFO:teuthology.orchestra.run.smithi040.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 8.0 MB/s | 5.4 MB 00:00 2024-04-26T21:09:29.074 INFO:teuthology.orchestra.run.smithi040.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-26T21:09:29.166 INFO:teuthology.orchestra.run.smithi040.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 9.5 MB/s | 1.5 MB 00:00 2024-04-26T21:09:29.464 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 975 kB/s | 19 MB 00:19 2024-04-26T21:09:29.541 INFO:teuthology.orchestra.run.smithi040.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-26T21:09:30.485 INFO:teuthology.orchestra.run.smithi040.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 17 MB 00:01 2024-04-26T21:09:30.642 INFO:teuthology.orchestra.run.smithi040.stdout:(9/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 9.6 MB/s | 11 MB 00:01 2024-04-26T21:09:30.667 INFO:teuthology.orchestra.run.smithi040.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 139 kB/s | 25 kB 00:00 2024-04-26T21:09:30.734 INFO:teuthology.orchestra.run.smithi040.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.3 MB/s | 225 kB 00:00 2024-04-26T21:09:30.793 INFO:teuthology.orchestra.run.smithi040.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 465 kB/s | 27 kB 00:00 2024-04-26T21:09:30.843 INFO:teuthology.orchestra.run.smithi040.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 635 kB/s | 32 kB 00:00 2024-04-26T21:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 6.2 MB/s | 747 kB 00:00 2024-04-26T21:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 163 kB 00:00 2024-04-26T21:09:31.094 INFO:teuthology.orchestra.run.smithi040.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T21:09:31.177 INFO:teuthology.orchestra.run.smithi040.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.9 MB/s | 501 kB 00:00 2024-04-26T21:09:31.636 INFO:teuthology.orchestra.run.smithi040.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 12 MB/s | 5.4 MB 00:00 2024-04-26T21:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 677 kB/s | 45 kB 00:00 2024-04-26T21:09:31.762 INFO:teuthology.orchestra.run.smithi040.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.3 MB/s | 135 kB 00:00 2024-04-26T21:09:31.820 INFO:teuthology.orchestra.run.smithi040.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.7 MB/s | 161 kB 00:00 2024-04-26T21:09:31.904 INFO:teuthology.orchestra.run.smithi040.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T21:09:31.979 INFO:teuthology.orchestra.run.smithi040.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T21:09:32.046 INFO:teuthology.orchestra.run.smithi040.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T21:09:32.122 INFO:teuthology.orchestra.run.smithi040.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T21:09:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.4 MB/s | 3.1 MB 00:00 2024-04-26T21:09:32.556 INFO:teuthology.orchestra.run.smithi040.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T21:09:32.614 INFO:teuthology.orchestra.run.smithi040.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 588 kB/s | 34 kB 00:00 2024-04-26T21:09:32.682 INFO:teuthology.orchestra.run.smithi040.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T21:09:33.023 INFO:teuthology.orchestra.run.smithi040.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 10 MB/s | 3.5 MB 00:00 2024-04-26T21:09:33.649 INFO:teuthology.orchestra.run.smithi040.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-26T21:09:34.124 INFO:teuthology.orchestra.run.smithi040.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T21:09:34.149 INFO:teuthology.orchestra.run.smithi040.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 497 kB/s | 249 kB 00:00 2024-04-26T21:09:34.174 INFO:teuthology.orchestra.run.smithi040.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 986 kB/s | 49 kB 00:00 2024-04-26T21:09:34.199 INFO:teuthology.orchestra.run.smithi040.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 329 kB/s | 16 kB 00:00 2024-04-26T21:09:34.250 INFO:teuthology.orchestra.run.smithi040.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.6 MB/s | 273 kB 00:00 2024-04-26T21:09:34.305 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-26T21:09:34.309 INFO:teuthology.orchestra.run.smithi040.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-26T21:09:34.334 INFO:teuthology.orchestra.run.smithi040.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 219 kB/s | 18 kB 00:00 2024-04-26T21:09:34.584 INFO:teuthology.orchestra.run.smithi040.stdout:(39/152): ceph-common-19.0.0-3306.gbc612ffb.el9 3.4 MB/s | 21 MB 00:06 2024-04-26T21:09:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 267 kB/s | 80 kB 00:00 2024-04-26T21:09:34.718 INFO:teuthology.orchestra.run.smithi040.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T21:09:34.735 INFO:teuthology.orchestra.run.smithi040.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-26T21:09:34.769 INFO:teuthology.orchestra.run.smithi040.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-26T21:09:34.819 INFO:teuthology.orchestra.run.smithi040.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T21:09:34.844 INFO:teuthology.orchestra.run.smithi040.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 979 kB/s | 106 kB 00:00 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi040.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T21:09:34.895 INFO:teuthology.orchestra.run.smithi040.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T21:09:34.928 INFO:teuthology.orchestra.run.smithi040.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-04-26T21:09:35.054 INFO:teuthology.orchestra.run.smithi040.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-26T21:09:35.070 INFO:teuthology.orchestra.run.smithi040.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-04-26T21:09:35.104 INFO:teuthology.orchestra.run.smithi040.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-26T21:09:35.138 INFO:teuthology.orchestra.run.smithi040.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-26T21:09:35.163 INFO:teuthology.orchestra.run.smithi040.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-26T21:09:35.188 INFO:teuthology.orchestra.run.smithi040.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-26T21:09:35.214 INFO:teuthology.orchestra.run.smithi040.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 360 kB/s | 18 kB 00:00 2024-04-26T21:09:35.239 INFO:teuthology.orchestra.run.smithi040.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-26T21:09:35.264 INFO:teuthology.orchestra.run.smithi040.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:09:35.290 INFO:teuthology.orchestra.run.smithi040.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi040.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-26T21:09:35.365 INFO:teuthology.orchestra.run.smithi040.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-26T21:09:35.390 INFO:teuthology.orchestra.run.smithi040.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-26T21:09:35.416 INFO:teuthology.orchestra.run.smithi040.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-26T21:09:35.441 INFO:teuthology.orchestra.run.smithi040.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-26T21:09:35.466 INFO:teuthology.orchestra.run.smithi040.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T21:09:35.491 INFO:teuthology.orchestra.run.smithi040.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-26T21:09:35.542 INFO:teuthology.orchestra.run.smithi040.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-26T21:09:35.634 INFO:teuthology.orchestra.run.smithi040.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T21:09:35.693 INFO:teuthology.orchestra.run.smithi040.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-26T21:09:35.734 INFO:teuthology.orchestra.run.smithi040.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-26T21:09:35.759 INFO:teuthology.orchestra.run.smithi040.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T21:09:35.818 INFO:teuthology.orchestra.run.smithi040.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-26T21:09:35.860 INFO:teuthology.orchestra.run.smithi040.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-26T21:09:35.885 INFO:teuthology.orchestra.run.smithi040.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T21:09:35.952 INFO:teuthology.orchestra.run.smithi040.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-26T21:09:36.019 INFO:teuthology.orchestra.run.smithi040.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-26T21:09:36.044 INFO:teuthology.orchestra.run.smithi040.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-26T21:09:36.061 INFO:teuthology.orchestra.run.smithi040.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-04-26T21:09:36.137 INFO:teuthology.orchestra.run.smithi040.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-26T21:09:36.154 INFO:teuthology.orchestra.run.smithi040.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-26T21:09:36.179 INFO:teuthology.orchestra.run.smithi040.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-26T21:09:36.196 INFO:teuthology.orchestra.run.smithi040.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 589 kB/s | 35 kB 00:00 2024-04-26T21:09:36.229 INFO:teuthology.orchestra.run.smithi040.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-26T21:09:36.263 INFO:teuthology.orchestra.run.smithi040.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-26T21:09:36.288 INFO:teuthology.orchestra.run.smithi040.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:09:36.322 INFO:teuthology.orchestra.run.smithi040.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-04-26T21:09:36.347 INFO:teuthology.orchestra.run.smithi040.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-26T21:09:36.373 INFO:teuthology.orchestra.run.smithi040.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T21:09:36.439 INFO:teuthology.orchestra.run.smithi040.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-04-26T21:09:36.465 INFO:teuthology.orchestra.run.smithi040.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-26T21:09:36.490 INFO:teuthology.orchestra.run.smithi040.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T21:09:36.498 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-26T21:09:36.515 INFO:teuthology.orchestra.run.smithi040.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T21:09:36.549 INFO:teuthology.orchestra.run.smithi040.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-04-26T21:09:36.574 INFO:teuthology.orchestra.run.smithi040.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-26T21:09:36.600 INFO:teuthology.orchestra.run.smithi040.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T21:09:36.625 INFO:teuthology.orchestra.run.smithi040.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-04-26T21:09:36.650 INFO:teuthology.orchestra.run.smithi040.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-26T21:09:36.675 INFO:teuthology.orchestra.run.smithi040.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T21:09:36.700 INFO:teuthology.orchestra.run.smithi040.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:09:36.726 INFO:teuthology.orchestra.run.smithi040.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T21:09:36.751 INFO:teuthology.orchestra.run.smithi040.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi040.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-26T21:09:36.818 INFO:teuthology.orchestra.run.smithi040.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-04-26T21:09:36.944 INFO:teuthology.orchestra.run.smithi040.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-04-26T21:09:37.152 INFO:teuthology.orchestra.run.smithi040.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-26T21:09:37.269 INFO:teuthology.orchestra.run.smithi040.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-26T21:09:37.294 INFO:teuthology.orchestra.run.smithi040.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-04-26T21:09:37.378 INFO:teuthology.orchestra.run.smithi040.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-26T21:09:37.428 INFO:teuthology.orchestra.run.smithi040.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 638 kB/s | 309 kB 00:00 2024-04-26T21:09:37.478 INFO:teuthology.orchestra.run.smithi040.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-26T21:09:37.537 INFO:teuthology.orchestra.run.smithi040.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-26T21:09:37.563 INFO:teuthology.orchestra.run.smithi040.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 503 kB/s | 67 kB 00:00 2024-04-26T21:09:37.613 INFO:teuthology.orchestra.run.smithi040.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-26T21:09:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T21:09:37.663 INFO:teuthology.orchestra.run.smithi040.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-04-26T21:09:37.691 INFO:teuthology.orchestra.run.smithi040.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 531 kB/s | 32 kB 00:00 2024-04-26T21:09:37.724 INFO:teuthology.orchestra.run.smithi040.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 248 kB/s | 15 kB 00:00 2024-04-26T21:09:37.791 INFO:teuthology.orchestra.run.smithi040.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-26T21:09:37.816 INFO:teuthology.orchestra.run.smithi040.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-26T21:09:37.884 INFO:teuthology.orchestra.run.smithi040.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 530 kB/s | 35 kB 00:00 2024-04-26T21:09:37.909 INFO:teuthology.orchestra.run.smithi040.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-26T21:09:37.934 INFO:teuthology.orchestra.run.smithi040.stdout:(121/152): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-04-26T21:09:37.959 INFO:teuthology.orchestra.run.smithi040.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T21:09:38.018 INFO:teuthology.orchestra.run.smithi040.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-26T21:09:38.085 INFO:teuthology.orchestra.run.smithi040.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 961 kB/s | 144 kB 00:00 2024-04-26T21:09:38.110 INFO:teuthology.orchestra.run.smithi040.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-26T21:09:38.169 INFO:teuthology.orchestra.run.smithi040.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-04-26T21:09:38.194 INFO:teuthology.orchestra.run.smithi040.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-26T21:09:38.219 INFO:teuthology.orchestra.run.smithi040.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-26T21:09:38.244 INFO:teuthology.orchestra.run.smithi040.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-26T21:09:38.294 INFO:teuthology.orchestra.run.smithi040.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-26T21:09:38.319 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T21:09:38.320 INFO:teuthology.orchestra.run.smithi040.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-26T21:09:38.345 INFO:teuthology.orchestra.run.smithi040.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-04-26T21:09:38.370 INFO:teuthology.orchestra.run.smithi040.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-04-26T21:09:38.404 INFO:teuthology.orchestra.run.smithi040.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-26T21:09:38.429 INFO:teuthology.orchestra.run.smithi040.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T21:09:38.463 INFO:teuthology.orchestra.run.smithi040.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-04-26T21:09:38.488 INFO:teuthology.orchestra.run.smithi040.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-04-26T21:09:38.513 INFO:teuthology.orchestra.run.smithi040.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-26T21:09:38.539 INFO:teuthology.orchestra.run.smithi040.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T21:09:38.564 INFO:teuthology.orchestra.run.smithi040.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-04-26T21:09:38.589 INFO:teuthology.orchestra.run.smithi040.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-26T21:09:38.614 INFO:teuthology.orchestra.run.smithi040.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-26T21:09:38.640 INFO:teuthology.orchestra.run.smithi040.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:09:38.673 INFO:teuthology.orchestra.run.smithi040.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-26T21:09:38.707 INFO:teuthology.orchestra.run.smithi040.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 762 kB/s | 90 kB 00:00 2024-04-26T21:09:38.724 INFO:teuthology.orchestra.run.smithi040.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-26T21:09:38.749 INFO:teuthology.orchestra.run.smithi040.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T21:09:38.816 INFO:teuthology.orchestra.run.smithi040.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T21:09:39.125 INFO:teuthology.orchestra.run.smithi040.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 8.5 MB/s | 3.4 MB 00:00 2024-04-26T21:09:39.158 INFO:teuthology.orchestra.run.smithi040.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-26T21:09:39.209 INFO:teuthology.orchestra.run.smithi040.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-26T21:09:39.350 INFO:teuthology.orchestra.run.smithi040.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-26T21:09:39.353 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:09:39.353 INFO:teuthology.orchestra.run.smithi040.stdout:Total 19 MB/s | 222 MB 00:11 2024-04-26T21:09:40.455 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-26T21:09:40.618 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-26T21:09:40.618 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-26T21:09:42.485 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-26T21:09:42.485 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-26T21:09:43.366 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T21:09:45.484 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:09:45.484 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:09:45.495 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:09:45.495 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:09:45.496 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:09:45.497 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:09:45.499 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:09:45.501 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:09:45.502 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:09:45.562 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-04-26T21:09:45.571 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:09:45.571 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-04-26T21:09:45.571 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:09:45.571 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-04-26T21:09:45.571 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:09:45.572 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:09:45.573 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:09:45.574 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:09:45.575 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:09:45.576 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:09:45.577 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:09:45.578 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:09:45.579 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:09:45.580 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:09:45.581 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:09:45.582 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 4 Packages 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 222 M 2024-04-26T21:09:45.583 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-04-26T21:09:46.367 INFO:teuthology.orchestra.run.smithi184.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-26T21:09:47.284 INFO:teuthology.orchestra.run.smithi184.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 970 kB/s | 889 kB 00:00 2024-04-26T21:09:47.351 INFO:teuthology.orchestra.run.smithi184.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.1 MB/s | 145 kB 00:00 2024-04-26T21:09:47.910 INFO:teuthology.orchestra.run.smithi184.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-04-26T21:09:48.318 INFO:teuthology.orchestra.run.smithi184.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-26T21:09:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-26T21:09:48.614 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:09:48.864 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:09:48.926 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:09:48.966 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:09:48.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:09:49.044 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:09:49.387 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:09:49.498 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:09:49.622 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:09:49.649 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:09:49.710 INFO:teuthology.orchestra.run.smithi184.stdout:(6/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 21 MB 00:03 2024-04-26T21:09:49.741 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:09:49.777 INFO:teuthology.orchestra.run.smithi184.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 3.2 MB/s | 4.7 MB 00:01 2024-04-26T21:09:49.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:09:49.843 INFO:teuthology.orchestra.run.smithi184.stdout:(8/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-26T21:09:49.936 INFO:teuthology.orchestra.run.smithi184.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 275 kB/s | 25 kB 00:00 2024-04-26T21:09:49.940 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:09:49.994 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:09:50.020 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:09:50.125 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:09:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:09:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:09:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:09:50.261 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:09:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:09:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:09:50.568 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:09:50.672 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:09:50.705 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:09:50.740 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:09:50.816 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:09:50.864 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:09:50.908 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:09:50.940 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:09:50.968 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:09:51.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:09:51.100 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:09:51.129 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:09:51.173 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:09:51.251 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:09:51.283 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:09:51.327 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:09:51.355 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:09:51.411 INFO:teuthology.orchestra.run.smithi184.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 6.5 MB/s | 11 MB 00:01 2024-04-26T21:09:51.427 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:09:51.646 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:09:51.723 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:09:51.802 INFO:teuthology.orchestra.run.smithi184.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 574 kB/s | 225 kB 00:00 2024-04-26T21:09:51.868 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:09:51.894 INFO:teuthology.orchestra.run.smithi184.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 295 kB/s | 27 kB 00:00 2024-04-26T21:09:51.923 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:09:51.970 INFO:teuthology.orchestra.run.smithi184.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 423 kB/s | 32 kB 00:00 2024-04-26T21:09:52.212 INFO:teuthology.orchestra.run.smithi184.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 747 kB 00:00 2024-04-26T21:09:52.312 INFO:teuthology.orchestra.run.smithi184.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 163 kB 00:00 2024-04-26T21:09:52.395 INFO:teuthology.orchestra.run.smithi184.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.5 MB/s | 127 kB 00:00 2024-04-26T21:09:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:09:52.562 INFO:teuthology.orchestra.run.smithi184.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 2.9 MB/s | 501 kB 00:00 2024-04-26T21:09:52.571 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:09:52.605 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:09:52.648 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:09:52.686 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:09:52.729 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:09:52.884 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:09:52.947 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:09:53.004 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:09:53.087 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:09:53.123 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:09:53.160 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:09:53.207 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:09:53.242 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:09:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:09:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:09:53.523 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:09:53.524 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:09:53.546 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:09:53.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:09:53.688 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:09:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:09:53.761 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:09:53.762 INFO:teuthology.orchestra.run.smithi184.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 4.2 MB/s | 17 MB 00:04 2024-04-26T21:09:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:09:53.819 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:09:53.837 INFO:teuthology.orchestra.run.smithi184.stdout:(19/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.2 MB/s | 5.4 MB 00:01 2024-04-26T21:09:53.863 INFO:teuthology.orchestra.run.smithi184.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 451 kB/s | 45 kB 00:00 2024-04-26T21:09:53.882 INFO:teuthology.orchestra.run.smithi184.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 3.0 MB/s | 135 kB 00:00 2024-04-26T21:09:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:09:53.916 INFO:teuthology.orchestra.run.smithi184.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 3.0 MB/s | 161 kB 00:00 2024-04-26T21:09:53.966 INFO:teuthology.orchestra.run.smithi184.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T21:09:54.025 INFO:teuthology.orchestra.run.smithi184.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 298 kB 00:00 2024-04-26T21:09:54.028 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:09:54.050 INFO:teuthology.orchestra.run.smithi184.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T21:09:54.083 INFO:teuthology.orchestra.run.smithi184.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T21:09:54.119 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:09:54.146 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:09:54.158 INFO:teuthology.orchestra.run.smithi184.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-26T21:09:54.181 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:09:54.221 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:09:54.225 INFO:teuthology.orchestra.run.smithi184.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 513 kB/s | 34 kB 00:00 2024-04-26T21:09:54.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:09:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:09:54.293 INFO:teuthology.orchestra.run.smithi184.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T21:09:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:09:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:09:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:09:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:09:54.469 INFO:teuthology.orchestra.run.smithi184.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 7.4 MB/s | 3.1 MB 00:00 2024-04-26T21:09:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:09:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:09:54.603 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:09:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:09:54.764 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:09:54.815 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:09:54.985 INFO:teuthology.orchestra.run.smithi184.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.1 MB/s | 3.5 MB 00:00 2024-04-26T21:09:55.427 INFO:teuthology.orchestra.run.smithi184.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.9 MB/s | 49 MB 00:05 2024-04-26T21:09:55.453 INFO:teuthology.orchestra.run.smithi184.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 533 kB/s | 249 kB 00:00 2024-04-26T21:09:55.486 INFO:teuthology.orchestra.run.smithi184.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 495 kB/s | 16 kB 00:00 2024-04-26T21:09:55.503 INFO:teuthology.orchestra.run.smithi184.stdout:(35/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 657 kB/s | 49 kB 00:00 2024-04-26T21:09:55.587 INFO:teuthology.orchestra.run.smithi184.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-04-26T21:09:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:09:55.612 INFO:teuthology.orchestra.run.smithi184.stdout:(37/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T21:09:55.629 INFO:teuthology.orchestra.run.smithi184.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 435 kB/s | 18 kB 00:00 2024-04-26T21:09:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:09:55.662 INFO:teuthology.orchestra.run.smithi184.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-26T21:09:55.669 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:09:55.746 INFO:teuthology.orchestra.run.smithi184.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 600 kB/s | 80 kB 00:00 2024-04-26T21:09:55.772 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:09:55.813 INFO:teuthology.orchestra.run.smithi184.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-26T21:09:55.828 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:09:55.838 INFO:teuthology.orchestra.run.smithi184.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T21:09:55.872 INFO:teuthology.orchestra.run.smithi184.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-04-26T21:09:55.897 INFO:teuthology.orchestra.run.smithi184.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T21:09:55.918 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi184.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T21:09:55.948 INFO:teuthology.orchestra.run.smithi184.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:09:55.973 INFO:teuthology.orchestra.run.smithi184.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T21:09:56.023 INFO:teuthology.orchestra.run.smithi184.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-04-26T21:09:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:09:56.157 INFO:teuthology.orchestra.run.smithi184.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-26T21:09:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:09:56.174 INFO:teuthology.orchestra.run.smithi184.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-04-26T21:09:56.226 INFO:teuthology.orchestra.run.smithi184.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 315 kB/s | 21 kB 00:00 2024-04-26T21:09:56.249 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:09:56.266 INFO:teuthology.orchestra.run.smithi184.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-26T21:09:56.292 INFO:teuthology.orchestra.run.smithi184.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 500 kB/s | 33 kB 00:00 2024-04-26T21:09:56.317 INFO:teuthology.orchestra.run.smithi184.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-26T21:09:56.342 INFO:teuthology.orchestra.run.smithi184.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-26T21:09:56.368 INFO:teuthology.orchestra.run.smithi184.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-26T21:09:56.393 INFO:teuthology.orchestra.run.smithi184.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:09:56.418 INFO:teuthology.orchestra.run.smithi184.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-26T21:09:56.451 INFO:teuthology.orchestra.run.smithi184.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T21:09:56.502 INFO:teuthology.orchestra.run.smithi184.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-26T21:09:56.527 INFO:teuthology.orchestra.run.smithi184.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-26T21:09:56.552 INFO:teuthology.orchestra.run.smithi184.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-26T21:09:56.577 INFO:teuthology.orchestra.run.smithi184.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-26T21:09:56.603 INFO:teuthology.orchestra.run.smithi184.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-26T21:09:56.636 INFO:teuthology.orchestra.run.smithi184.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T21:09:56.653 INFO:teuthology.orchestra.run.smithi184.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-04-26T21:09:56.862 INFO:teuthology.orchestra.run.smithi184.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-26T21:09:56.904 INFO:teuthology.orchestra.run.smithi184.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-04-26T21:09:56.938 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:09:56.946 INFO:teuthology.orchestra.run.smithi184.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-26T21:09:56.971 INFO:teuthology.orchestra.run.smithi184.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T21:09:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:09:57.138 INFO:teuthology.orchestra.run.smithi184.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-26T21:09:57.230 INFO:teuthology.orchestra.run.smithi184.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-26T21:09:57.255 INFO:teuthology.orchestra.run.smithi184.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 749 kB/s | 87 kB 00:00 2024-04-26T21:09:57.322 INFO:teuthology.orchestra.run.smithi184.stdout:(74/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.6 MB/s | 4.7 MB 00:00 2024-04-26T21:09:57.348 INFO:teuthology.orchestra.run.smithi184.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T21:09:57.414 INFO:teuthology.orchestra.run.smithi184.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-26T21:09:57.440 INFO:teuthology.orchestra.run.smithi184.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-04-26T21:09:57.498 INFO:teuthology.orchestra.run.smithi184.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-04-26T21:09:57.524 INFO:teuthology.orchestra.run.smithi184.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-26T21:09:57.574 INFO:teuthology.orchestra.run.smithi184.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-04-26T21:09:57.675 INFO:teuthology.orchestra.run.smithi184.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-26T21:09:57.700 INFO:teuthology.orchestra.run.smithi184.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-26T21:09:57.733 INFO:teuthology.orchestra.run.smithi184.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-26T21:09:57.775 INFO:teuthology.orchestra.run.smithi184.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T21:09:57.809 INFO:teuthology.orchestra.run.smithi184.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi184.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-26T21:09:57.876 INFO:teuthology.orchestra.run.smithi184.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-26T21:09:57.901 INFO:teuthology.orchestra.run.smithi184.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-26T21:09:57.926 INFO:teuthology.orchestra.run.smithi184.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T21:09:57.952 INFO:teuthology.orchestra.run.smithi184.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T21:09:58.019 INFO:teuthology.orchestra.run.smithi184.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-04-26T21:09:58.044 INFO:teuthology.orchestra.run.smithi184.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-04-26T21:09:58.069 INFO:teuthology.orchestra.run.smithi184.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-04-26T21:09:58.095 INFO:teuthology.orchestra.run.smithi184.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T21:09:58.120 INFO:teuthology.orchestra.run.smithi184.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-04-26T21:09:58.145 INFO:teuthology.orchestra.run.smithi184.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T21:09:58.170 INFO:teuthology.orchestra.run.smithi184.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T21:09:58.196 INFO:teuthology.orchestra.run.smithi184.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:09:58.213 INFO:teuthology.orchestra.run.smithi184.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-26T21:09:58.246 INFO:teuthology.orchestra.run.smithi184.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:09:58.271 INFO:teuthology.orchestra.run.smithi184.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-04-26T21:09:58.322 INFO:teuthology.orchestra.run.smithi184.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-26T21:09:58.385 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:09:58.405 INFO:teuthology.orchestra.run.smithi184.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-26T21:09:58.474 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:09:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:09:58.535 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:09:58.664 INFO:teuthology.orchestra.run.smithi184.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-26T21:09:58.864 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:09:58.897 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:09:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:09:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:09:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:09:59.048 INFO:teuthology.orchestra.run.smithi184.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 55 kB/s | 19 kB 00:00 2024-04-26T21:09:59.156 INFO:teuthology.orchestra.run.smithi184.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 657 kB/s | 548 kB 00:00 2024-04-26T21:09:59.223 INFO:teuthology.orchestra.run.smithi184.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 390 kB/s | 309 kB 00:00 2024-04-26T21:09:59.248 INFO:teuthology.orchestra.run.smithi184.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-04-26T21:09:59.341 INFO:teuthology.orchestra.run.smithi184.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-04-26T21:09:59.366 INFO:teuthology.orchestra.run.smithi184.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-26T21:09:59.466 INFO:teuthology.orchestra.run.smithi184.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-26T21:09:59.545 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:09:59.558 INFO:teuthology.orchestra.run.smithi184.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-04-26T21:09:59.576 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:09:59.651 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:09:59.651 INFO:teuthology.orchestra.run.smithi184.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-04-26T21:09:59.677 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:09:59.709 INFO:teuthology.orchestra.run.smithi184.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-26T21:09:59.751 INFO:teuthology.orchestra.run.smithi184.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-26T21:09:59.809 INFO:teuthology.orchestra.run.smithi184.stdout:(116/152): libarrow-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-04-26T21:09:59.834 INFO:teuthology.orchestra.run.smithi184.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-26T21:09:59.868 INFO:teuthology.orchestra.run.smithi184.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-26T21:09:59.935 INFO:teuthology.orchestra.run.smithi184.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-26T21:09:59.960 INFO:teuthology.orchestra.run.smithi184.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-26T21:09:59.985 INFO:teuthology.orchestra.run.smithi184.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-26T21:10:00.044 INFO:teuthology.orchestra.run.smithi184.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-26T21:10:00.111 INFO:teuthology.orchestra.run.smithi184.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-26T21:10:00.136 INFO:teuthology.orchestra.run.smithi184.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 959 kB/s | 144 kB 00:00 2024-04-26T21:10:00.162 INFO:teuthology.orchestra.run.smithi184.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-26T21:10:00.212 INFO:teuthology.orchestra.run.smithi184.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-26T21:10:00.237 INFO:teuthology.orchestra.run.smithi184.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-26T21:10:00.263 INFO:teuthology.orchestra.run.smithi184.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-26T21:10:00.313 INFO:teuthology.orchestra.run.smithi184.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-26T21:10:00.372 INFO:teuthology.orchestra.run.smithi184.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-26T21:10:00.405 INFO:teuthology.orchestra.run.smithi184.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T21:10:00.430 INFO:teuthology.orchestra.run.smithi184.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-04-26T21:10:00.528 INFO:teuthology.orchestra.run.smithi184.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 370 kB/s | 58 kB 00:00 2024-04-26T21:10:00.556 INFO:teuthology.orchestra.run.smithi184.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-26T21:10:00.581 INFO:teuthology.orchestra.run.smithi184.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-26T21:10:00.623 INFO:teuthology.orchestra.run.smithi184.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 946 kB/s | 90 kB 00:00 2024-04-26T21:10:00.657 INFO:teuthology.orchestra.run.smithi184.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-26T21:10:00.691 INFO:teuthology.orchestra.run.smithi184.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-26T21:10:00.733 INFO:teuthology.orchestra.run.smithi184.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T21:10:00.758 INFO:teuthology.orchestra.run.smithi184.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-04-26T21:10:00.791 INFO:teuthology.orchestra.run.smithi184.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-26T21:10:00.833 INFO:teuthology.orchestra.run.smithi184.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-26T21:10:00.858 INFO:teuthology.orchestra.run.smithi184.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T21:10:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-04-26T21:10:00.959 INFO:teuthology.orchestra.run.smithi184.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-26T21:10:00.984 INFO:teuthology.orchestra.run.smithi184.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi184.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-26T21:10:01.093 INFO:teuthology.orchestra.run.smithi184.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T21:10:01.268 INFO:teuthology.orchestra.run.smithi184.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-04-26T21:10:01.385 INFO:teuthology.orchestra.run.smithi184.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-26T21:10:01.436 INFO:teuthology.orchestra.run.smithi184.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-04-26T21:10:01.502 INFO:teuthology.orchestra.run.smithi184.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.6 MB/s | 3.4 MB 00:00 2024-04-26T21:10:01.508 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:10:01.508 INFO:teuthology.orchestra.run.smithi184.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-26T21:10:02.097 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:02.140 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:02.644 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-04-26T21:10:02.713 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:10:02.794 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-04-26T21:10:02.794 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-04-26T21:10:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:10:02.843 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:10:02.843 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:10:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:02.870 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:10:04.581 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-04-26T21:10:04.581 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-04-26T21:10:10.623 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-04-26T21:10:10.698 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:10:10.778 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:10:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:10:10.924 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:10:10.949 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:10:10.992 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:10:11.378 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:10:11.434 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:10:11.557 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:10:11.585 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:10:11.667 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:10:11.697 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:10:11.847 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:10:11.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:10:12.018 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:10:12.040 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:10:12.102 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:10:12.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:10:12.145 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:10:12.299 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:10:12.334 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:10:12.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:10:12.543 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:10:12.581 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:10:12.616 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:10:12.681 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:10:12.715 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:10:12.767 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:10:12.799 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:10:12.827 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:10:12.896 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:10:12.958 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:10:12.988 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:10:13.023 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:10:13.102 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:10:13.135 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:10:13.178 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:10:13.206 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:10:13.278 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:10:13.329 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:10:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:10:13.538 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:10:13.590 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:10:14.122 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:10:14.219 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:10:14.249 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:10:14.291 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:10:14.329 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:10:14.371 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:10:14.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:10:14.591 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:10:14.645 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:10:14.719 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:10:14.750 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:10:14.786 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-26T21:10:14.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:14.834 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:10:14.869 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:10:14.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:10:14.954 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:10:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:10:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:10:15.033 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:15.033 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:10:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:15.089 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:10:15.143 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:10:15.173 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:10:15.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:10:15.310 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:10:15.348 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:10:15.388 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:10:15.413 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:10:15.438 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:10:15.514 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:10:15.647 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:10:15.687 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:10:15.715 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:10:15.750 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:10:15.789 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:10:15.820 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:10:15.850 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:10:16.062 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:10:16.094 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:10:16.136 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:10:16.170 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:10:16.251 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:10:16.309 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:10:16.340 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:10:16.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:10:16.494 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:10:16.548 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:10:16.570 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:10:16.597 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:10:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:10:16.852 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:16.853 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:10:16.853 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:10:16.853 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:10:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:17.275 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:10:17.321 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:10:17.321 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:10:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:10:17.345 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:17.345 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:10:17.345 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:10:17.345 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:10:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:17.355 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:10:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:10:17.454 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:10:17.505 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:10:17.510 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:10:17.574 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:10:17.592 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:10:17.640 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:10:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:10:17.808 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:10:17.814 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:10:17.829 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:10:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:10:17.907 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:10:17.993 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:10:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:10:18.206 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:10:18.238 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:10:18.265 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:10:18.553 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:10:18.678 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:10:18.738 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:10:18.764 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:10:18.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:10:18.862 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:10:18.887 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:10:19.898 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:10:19.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:10:19.986 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:10:20.012 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:10:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:10:20.045 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:10:20.357 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:10:20.419 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:10:20.515 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:10:20.546 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:10:20.582 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:10:21.040 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:10:21.060 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:10:21.083 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:10:21.086 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:10:21.157 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:10:21.187 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:10:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:10:21.358 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:10:21.382 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:10:21.406 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:10:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:10:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:10:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:10:21.407 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:10:21.470 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:10:21.571 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:10:21.616 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:10:22.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:10:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:10:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:10:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:10:22.710 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:22.710 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:10:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:22.762 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:10:22.786 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:23.104 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:10:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:23.472 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:23.509 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:10:24.096 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:10:24.160 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:10:24.160 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:10:24.161 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:10:24.161 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:24.189 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:10:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:10:28.143 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:10:28.173 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:10:28.201 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:10:28.397 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:10:28.439 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:10:28.468 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:10:29.093 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:29.133 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:29.153 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:30.096 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:10:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:10:30.230 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:10:30.315 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:10:30.350 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:10:30.350 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:10:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:10:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:10:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:10:30.554 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:10:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-04-26T21:10:35.702 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-04-26T21:10:35.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:35.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:10:35.910 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:10:35.910 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:35.910 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:10:35.910 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:10:36.469 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:10:36.470 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:10:36.470 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:10:36.470 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:10:36.470 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:10:36.470 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:10:36.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:10:36.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:10:36.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:10:36.484 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:10:36.485 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:10:36.486 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:10:36.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:10:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:10:36.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:10:36.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:10:36.491 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:10:36.492 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:10:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:10:36.494 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:10:36.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:10:37.397 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:10:37.422 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:10:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:10:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:10:37.423 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:10:37.423 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:37.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:10:37.679 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:38.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:10:38.146 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:10:38.146 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:38.146 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:10:38.147 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:10:38.147 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:10:38.147 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:38.262 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:10:38.319 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:10:38.376 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:10:38.562 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:10:38.608 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:10:38.686 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:10:38.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.749 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.750 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:10:38.752 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:10:38.753 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:10:38.754 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:10:38.755 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:10:38.756 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:10:38.757 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:10:38.758 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:10:38.759 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:10:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-26T21:10:38.800 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:10:38.957 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:10:39.000 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:10:39.032 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:10:39.059 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:10:39.108 DEBUG:teuthology.parallel:result is None 2024-04-26T21:10:39.471 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:10:39.500 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:10:39.524 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:10:39.585 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:10:39.615 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:10:40.659 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:10:40.700 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:10:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:10:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:10:41.804 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:10:41.962 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:10:41.994 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:10:42.017 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:10:42.018 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:42.018 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:10:42.018 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:10:42.018 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:10:42.018 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:42.069 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:10:42.081 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:10:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:10:42.227 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:10:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:10:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:10:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:10:43.204 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:43.204 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:10:43.204 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:10:43.204 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:10:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:10:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:10:43.278 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:43.279 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:10:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:43.323 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:10:43.346 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:10:43.346 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:43.346 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:10:43.347 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:10:43.347 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:10:43.347 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:43.649 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:10:43.672 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:48.303 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:10:48.344 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:10:48.385 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:10:48.417 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:10:48.445 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:10:48.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:10:48.674 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:10:48.696 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:10:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:49.269 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:49.288 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:10:50.119 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:10:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:10:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:10:50.330 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:10:50.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:10:50.361 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:10:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:10:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:10:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:10:50.570 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:10:50.635 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:10:56.718 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:10:56.719 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:10:56.720 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:10:56.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:10:56.722 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:10:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:10:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:10:56.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:10:56.726 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:10:56.727 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:10:56.727 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:10:56.727 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:10:56.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:10:56.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:10:56.731 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:10:58.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:10:58.795 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.796 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.797 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.798 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.799 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.800 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:10:58.801 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:10:58.802 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:10:58.803 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:10:58.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:10:58.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:10:58.806 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:10:58.807 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:10:58.808 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:10:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:10:58.809 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:10:58.809 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-04-26T21:10:59.206 DEBUG:teuthology.parallel:result is None 2024-04-26T21:10:59.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:10:59.353 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:10:59.380 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:10:59.380 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:10:59.380 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:10:59.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:10:59.525 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:10:59.547 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:10:59.547 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:10:59.547 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:10:59.549 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:10:59.549 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:10:59.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:10:59.581 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:10:59.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:10:59.613 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:10:59.614 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:10:59.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:10:59.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:10:59.716 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:10:59.716 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:10:59.740 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:10:59.803 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:10:59.803 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:10:59.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:10:59.831 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:10:59.899 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:10:59.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:10:59.922 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:10:59.986 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:10:59.986 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:10:59.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:11:00.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:11:00.076 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:00.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:11:00.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:11:00.163 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T21:11:00.275 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T21:11:00.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-04-26T21:11:00.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-04-26T21:11:00.301 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T21:11:00.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:11:00.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:11:00.367 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T21:11:00.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:11:00.412 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:11:00.441 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T21:11:00.441 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T21:11:00.442 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T21:11:00.442 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:00.442 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:11:00.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:00.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:00.625 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:00.625 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:11:00.640 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:00.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:00.794 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T21:11:00.794 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T21:11:00.794 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits}', 'email': None, 'first_in_suite': False, 'job_id': '7675205', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTlg7GVPYpp/X2ihN3cdKqWyO1mj7L8/bv0qnRLPYDHPt1O3G8H40aIN04dESu2r7RGHUU6VtdKY1jZzknFESw=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJa2DV+bFbMG+wuhCXZJ10Lv7sJANC29baJSEzsosys7EtPg9eg2zg3TUiXlpLW6lkT+bpd2O3e7DP2hE4zXkOU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T21:11:00.795 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:11:00.797 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:11:00.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:11:00.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:11:00.907 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:00.907 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:11:00.930 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:11:00.930 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-26T21:11:00.985 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:10:31.000939315 +0000 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:08:44.360938534 +0000 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:08:44.360938534 +0000 2024-04-26T21:11:00.986 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:08:44.360938534 +0000 2024-04-26T21:11:00.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:11:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:11:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:11:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000259207 s, 2.0 MB/s 2024-04-26T21:11:01.050 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:11:01.106 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:10:31.000939315 +0000 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:08:44.708934188 +0000 2024-04-26T21:11:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:08:44.708934188 +0000 2024-04-26T21:11:01.163 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:08:44.708934188 +0000 2024-04-26T21:11:01.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:11:01.224 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:11:01.224 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:11:01.225 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000201726 s, 2.5 MB/s 2024-04-26T21:11:01.226 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:11:01.281 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:01.337 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:10:31.001939303 +0000 2024-04-26T21:11:01.338 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:08:45.063929754 +0000 2024-04-26T21:11:01.338 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:08:45.063929754 +0000 2024-04-26T21:11:01.338 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:08:45.063929754 +0000 2024-04-26T21:11:01.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:11:01.400 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:11:01.400 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:11:01.400 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000193372 s, 2.6 MB/s 2024-04-26T21:11:01.402 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:11:01.457 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-26 21:10:31.001939303 +0000 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-26 21:08:45.409925432 +0000 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-26 21:08:45.409925432 +0000 2024-04-26T21:11:01.513 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-26 21:08:45.409925432 +0000 2024-04-26T21:11:01.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:11:01.577 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-26T21:11:01.577 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-26T21:11:01.577 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000231478 s, 2.2 MB/s 2024-04-26T21:11:01.577 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:11:01.633 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:01.633 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:01.634 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:11:01.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:11:01.649 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-04-26T21:11:01.703 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:10:50.991801537 +0000 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:08:41.066937756 +0000 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:08:41.066937756 +0000 2024-04-26T21:11:01.704 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:08:41.066937756 +0000 2024-04-26T21:11:01.704 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:11:01.766 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:11:01.766 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:11:01.766 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000193038 s, 2.7 MB/s 2024-04-26T21:11:01.767 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:11:01.823 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:10:50.992801517 +0000 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:08:41.412929877 +0000 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:08:41.412929877 +0000 2024-04-26T21:11:01.878 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:08:41.412929877 +0000 2024-04-26T21:11:01.879 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:11:01.939 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:11:01.940 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:11:01.940 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000183187 s, 2.8 MB/s 2024-04-26T21:11:01.941 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:11:01.996 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:10:50.992801517 +0000 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:08:41.757922021 +0000 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:08:41.757922021 +0000 2024-04-26T21:11:02.051 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:08:41.757922021 +0000 2024-04-26T21:11:02.052 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:11:02.112 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:11:02.112 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:11:02.112 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000169716 s, 3.0 MB/s 2024-04-26T21:11:02.113 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:11:02.169 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-04-26T21:11:02.223 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:11:02.223 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:11:02.223 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-04-26 21:10:50.992801517 +0000 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-04-26 21:08:42.089914461 +0000 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-04-26 21:08:42.089914461 +0000 2024-04-26T21:11:02.224 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-04-26 21:08:42.089914461 +0000 2024-04-26T21:11:02.224 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:11:02.287 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-04-26T21:11:02.287 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-04-26T21:11:02.287 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000245545 s, 2.1 MB/s 2024-04-26T21:11:02.288 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:11:02.343 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:02.343 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:11:02.343 INFO:tasks.ceph:Generating config... 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] pid file = 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T21:11:02.344 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T21:11:02.345 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T21:11:02.345 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T21:11:02.346 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T21:11:02.346 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T21:11:02.346 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T21:11:02.346 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T21:11:02.346 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T21:11:02.346 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T21:11:02.347 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T21:11:02.347 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T21:11:02.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T21:11:02.402 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T21:11:02.405 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T21:11:02.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:11:02.478 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2024-04-26T21:11:02.479 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T21:11:02.479 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:02.480 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:11:02.535 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:02.622 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:02.622 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid eee7569e-7e8d-4976-9855-170ac833730d 2024-04-26T21:11:02.622 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:fsid eee7569e-7e8d-4976-9855-170ac833730d 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-04-26T21:11:02.622502+0000 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-04-26T21:11:02.622502+0000 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2024-04-26T21:11:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T21:11:02.624 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:11:02.640 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eee7569e-7e8d-4976-9855-170ac833730d... 2024-04-26T21:11:02.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:11:02.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:11:02.722 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-04-26T21:11:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:11:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-04-26T21:11:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on skipped message = False 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = eee7569e-7e8d-4976-9855-170ac833730d 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-04-26T21:11:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:11:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore allocator = bitmap 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 1/20 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 1/20 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 4/10 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:11:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-26T21:11:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-04-26T21:11:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:11:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: mon down mkfs grace = 300 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:11:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-04-26T21:11:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-26T21:11:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.735 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.736 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:11:02.737 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-04-26T21:11:02.738 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.40,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = eee7569e-7e8d-4976-9855-170ac833730d 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.739 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:11:02.740 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bdev async discard = True 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bdev enable discard = True 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore allocator = bitmap 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:11:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-04-26T21:11:02.742 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.743 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon down mkfs grace = 300 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-04-26T21:11:02.744 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-04-26T21:11:02.745 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds balancer = 20 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = True 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2024-04-26T21:11:02.746 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2024-04-26T21:11:02.748 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T21:11:02.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T21:11:02.842 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T21:11:02.842 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:02.842 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T21:11:02.857 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:02.857 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T21:11:02.913 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:11:02.913 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:02.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:11:02.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:11:02.987 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:02.987 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:03.042 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:11:03.042 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:03.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:11:03.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:11:03.076 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:03.076 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:03.132 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T21:11:03.132 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T21:11:03.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:11:03.182 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:11:03.184 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:11:03.242 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:11:03.245 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T21:11:03.245 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:11:03.296 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:11:03.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T21:11:03.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:11:03.412 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:11:03.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T21:11:03.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:11:03.487 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:11:03.489 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T21:11:03.512 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:11:03.600 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:11:03.603 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T21:11:03.626 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T21:11:03.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:11:03.666 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T21:11:03.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T21:11:03.759 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T21:11:03.771 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T21:11:03.810 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T21:11:03.822 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T21:11:03.901 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T21:11:03.913 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T21:11:03.913 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:11:03.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T21:11:03.938 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:03.938 INFO:tasks.ceph:role: osd.0 2024-04-26T21:11:03.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:11:03.938 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:11:04.010 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:04.010 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:04.011 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:04.011 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:04.011 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:04.518 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:11:04.520 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:11:04.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T21:11:04.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T21:11:04.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T21:11:04.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:04.742 INFO:tasks.ceph:role: osd.1 2024-04-26T21:11:04.742 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:11:04.742 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:11:04.814 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:04.814 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:04.814 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:04.814 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:04.815 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:05.270 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:11:05.272 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:11:05.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T21:11:05.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T21:11:05.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T21:11:05.460 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:05.460 INFO:tasks.ceph:role: osd.2 2024-04-26T21:11:05.460 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:11:05.461 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:05.534 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:05.535 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:05.535 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:06.018 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:11:06.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:11:06.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T21:11:06.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T21:11:06.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T21:11:06.255 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:06.255 INFO:tasks.ceph:role: osd.3 2024-04-26T21:11:06.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:11:06.256 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:11:06.329 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:06.840 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-26T21:11:06.842 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-26T21:11:06.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T21:11:06.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T21:11:07.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.105+0000 7fbed3019640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T21:11:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.105+0000 7fbed3019640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T21:11:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.105+0000 7fbed3019640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:11:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.105+0000 7fbed3019640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:11:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.105+0000 7fbed3019640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:11:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:07.106+0000 7fbed3019640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T21:11:09.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T21:11:09.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.074+0000 7fb8c59c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T21:11:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.075+0000 7fb8c59c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T21:11:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.075+0000 7fb8c59c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:11:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.075+0000 7fb8c59c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:11:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.075+0000 7fb8c59c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:11:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:10.075+0000 7fb8c59c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T21:11:12.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T21:11:12.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:13.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.064+0000 7fd6709d8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T21:11:13.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.064+0000 7fd6709d8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T21:11:13.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.064+0000 7fd6709d8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:11:13.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.064+0000 7fd6709d8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:11:13.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.064+0000 7fd6709d8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:11:13.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:13.065+0000 7fd6709d8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T21:11:15.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T21:11:15.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:16.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.047+0000 7f129cc12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T21:11:16.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.047+0000 7f129cc12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T21:11:16.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.047+0000 7f129cc12640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:11:16.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.047+0000 7f129cc12640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:11:16.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.047+0000 7f129cc12640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:11:16.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:16.048+0000 7f129cc12640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T21:11:18.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T21:11:18.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T21:11:18.961 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:18.961 INFO:tasks.ceph:role: osd.4 2024-04-26T21:11:18.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:11:18.961 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:19.036 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:19.037 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:19.544 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:11:19.546 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:11:19.546 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T21:11:19.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T21:11:19.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T21:11:19.746 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:19.747 INFO:tasks.ceph:role: osd.5 2024-04-26T21:11:19.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:11:19.747 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:11:19.817 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:19.818 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:20.274 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:11:20.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:11:20.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T21:11:20.356 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T21:11:20.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T21:11:20.487 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:20.487 INFO:tasks.ceph:role: osd.6 2024-04-26T21:11:20.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:11:20.487 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:20.557 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:20.558 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:20.558 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:20.558 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:20.558 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:21.054 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:11:21.057 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:11:21.057 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T21:11:21.152 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T21:11:21.183 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T21:11:21.248 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:11:21.248 INFO:tasks.ceph:role: osd.7 2024-04-26T21:11:21.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:11:21.248 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:11:21.322 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:11:21.323 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:11:21.323 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:11:21.323 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:11:21.323 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:11:21.809 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-04-26T21:11:21.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-04-26T21:11:21.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T21:11:21.960 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T21:11:21.994 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:22.087 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.087+0000 7f5c19e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T21:11:22.090 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.089+0000 7f5c19e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T21:11:22.090 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.089+0000 7f5c19e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:11:22.090 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.089+0000 7f5c19e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:11:22.090 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.089+0000 7f5c19e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:11:22.097 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:22.096+0000 7f5c19e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T21:11:24.940 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T21:11:24.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:25.052 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.051+0000 7f4d451a1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T21:11:25.053 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.051+0000 7f4d451a1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T21:11:25.053 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.051+0000 7f4d451a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:11:25.053 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.051+0000 7f4d451a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:11:25.053 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.051+0000 7f4d451a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:11:25.053 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:25.052+0000 7f4d451a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T21:11:27.904 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T21:11:27.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:28.021 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.020+0000 7f994f425640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T21:11:28.021 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.020+0000 7f994f425640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T21:11:28.021 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.020+0000 7f994f425640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:11:28.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.020+0000 7f994f425640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:11:28.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.020+0000 7f994f425640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:11:28.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:28.021+0000 7f994f425640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T21:11:30.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:30.800 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:30.875 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T21:11:30.941 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:31.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.026+0000 7fbe4492c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T21:11:31.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.026+0000 7fbe4492c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T21:11:31.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.026+0000 7fbe4492c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:11:31.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.026+0000 7fbe4492c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:11:31.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.026+0000 7fbe4492c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:11:31.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:31.027+0000 7fbe4492c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T21:11:33.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T21:11:33.913 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T21:11:33.913 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:33.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T21:11:33.942 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:33.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T21:11:34.010 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T21:11:34.078 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T21:11:34.145 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T21:11:34.211 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T21:11:34.277 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T21:11:34.342 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T21:11:34.371 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.371 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T21:11:34.439 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T21:11:34.507 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T21:11:34.572 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.573 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T21:11:34.638 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.638 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T21:11:34.703 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T21:11:34.768 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.768 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T21:11:34.786 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:34.787 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T21:11:34.845 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.845 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T21:11:34.860 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:34.860 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T21:11:34.914 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T21:11:34.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:11:34.917 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ== 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ== 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ== 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw== 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ== 2024-04-26T21:11:34.958 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ== 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-04-26T21:11:34.959 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA== 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2024-04-26T21:11:34.960 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag== 2024-04-26T21:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2024-04-26T21:11:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw== 2024-04-26T21:11:34.977 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA== 2024-04-26T21:11:34.978 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA== 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg== 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw== 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA== 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2024-04-26T21:11:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag== 2024-04-26T21:11:34.981 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:11:35.004 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:11:35.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.072 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.233 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.284 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.335 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:11:35.386 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:11:35.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.490 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:11:35.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.593 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.645 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.697 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:11:35.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.802 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.856 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.910 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:11:35.962 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T21:11:35.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T21:11:35.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:11:36.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T21:11:36.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T21:11:36.129 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:11:36.225 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T21:11:36.248 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T21:11:36.311 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:11:36.405 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T21:11:36.428 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:36.430 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:11:36.483 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T21:11:36.484 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T21:11:36.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T21:11:36.486 INFO:tasks.ceph.mon.a:Started 2024-04-26T21:11:36.486 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T21:11:36.486 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T21:11:36.526 INFO:tasks.ceph.mon.b:Started 2024-04-26T21:11:36.526 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T21:11:36.526 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T21:11:36.528 INFO:tasks.ceph.mon.c:Started 2024-04-26T21:11:36.528 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T21:11:36.529 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T21:11:36.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T21:11:36.533 INFO:tasks.ceph.mgr.y:Started 2024-04-26T21:11:36.533 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T21:11:36.533 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T21:11:36.535 INFO:tasks.ceph.mgr.x:Started 2024-04-26T21:11:36.535 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T21:11:36.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T21:11:36.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.636+0000 7f3aae0c4640 1 Processor -- start 2024-04-26T21:11:36.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.636+0000 7f3aae0c4640 1 -- start start 2024-04-26T21:11:36.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aae0c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aae0c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa8148390 con 0x7f3aa8147fb0 2024-04-26T21:11:36.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aae0c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa806fe90 con 0x7f3aa81638b0 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aae0c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa8163e20 con 0x7f3aa806c9b0 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60162/0 (socket says 172.21.15.40:60162) 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3aa8147fb0 0x7f3aa806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59176/0 (socket says 172.21.15.40:59176) 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/4084184190 learned_addr learned my addr 172.21.15.40:0/4084184190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 msgr2=0x7f3aa80b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 msgr2=0x7f3aa80b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.637+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:11:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.638+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:11:36.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.800+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540920260 0 0) 0x7f3aa8148390 con 0x7f3aa8147fb0 2024-04-26T21:11:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.800+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a94003520 con 0x7f3aa8147fb0 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 226562559 0 0) 0x7f3a94003520 con 0x7f3aa8147fb0 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 msgr2=0x7f3aa80b5e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 --2- 172.21.15.40:0/4084184190 >> v2:172.21.15.40:3300/0 conn(0x7f3aa806c9b0 0x7f3aa80b5e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 >> v1:172.21.15.184:6789/0 conn(0x7f3aa81638b0 legacy=0x7f3aa80b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa8070170 con 0x7f3aa8147fb0 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3a90001670 con 0x7f3aa8147fb0 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 >> v1:172.21.15.40:6789/0 conn(0x7f3aa8147fb0 legacy=0x7f3aa806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 0x7f3a94003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94004800 0x7f3a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940052e0 0x7f3a9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8163e20 con 0x7f3a940039b0 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9400c1e0 con 0x7f3a940052e0 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9400c3e0 con 0x7f3a94004800 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.801+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 0x7f3a94003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940052e0 0x7f3a9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94004800 0x7f3a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa77fe640 1 -- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94004800 msgr2=0x7f3a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94004800 0x7f3a94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa77fe640 1 -- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940052e0 msgr2=0x7f3a9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940052e0 0x7f3a9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa77fe640 1 -- 172.21.15.40:0/4084184190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a94003520 con 0x7f3a940039b0 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940052e0 0x7f3a9400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a94004800 0x7f3a94004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.802+0000 7f3aa7fff640 1 -- 172.21.15.40:0/4084184190 reap_dead start 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 0x7f3a94003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c0e0 tx=0x7f3a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54e9542ff1b251f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90007c50 con 0x7f3a940039b0 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a90007270 con 0x7f3a940039b0 2024-04-26T21:11:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aa67fc640 1 -- 172.21.15.40:0/4084184190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90005810 con 0x7f3a940039b0 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 -- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 msgr2=0x7f3a94003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 0x7f3a94003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c0e0 tx=0x7f3a94005830 comp rx=0 tx=0).stop 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 -- 172.21.15.40:0/4084184190 shutdown_connections 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/4084184190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a940039b0 0x7f3a94003df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 -- 172.21.15.40:0/4084184190 >> 172.21.15.40:0/4084184190 conn(0x7f3aa80d1e90 msgr2=0x7f3aa80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.803+0000 7f3aae0c4640 1 -- 172.21.15.40:0/4084184190 shutdown_connections 2024-04-26T21:11:36.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.804+0000 7f3aae0c4640 1 -- 172.21.15.40:0/4084184190 wait complete. 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.805+0000 7f3aae0c4640 1 Processor -- start 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.805+0000 7f3aae0c4640 1 -- start start 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 0x7f3aa80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 0x7f3aa815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8148cd0 con 0x7f3aa813c380 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80b59f0 con 0x7f3aa81628d0 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aae0c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa8148e50 con 0x7f3a940039b0 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60202/0 (socket says 172.21.15.40:60202) 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/557030171 learned_addr learned my addr 172.21.15.40:0/557030171 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 0x7f3aa815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 0x7f3aa80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 msgr2=0x7f3aa80d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 0x7f3aa80d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 msgr2=0x7f3aa815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 0x7f3aa815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa6ffd640 1 -- 172.21.15.40:0/557030171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa813c840 con 0x7f3aa813c380 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa77fe640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 0x7f3aa80d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.806+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 0x7f3aa815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3aa6ffd640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a98010630 tx=0x7f3a98010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801a030 con 0x7f3aa813c380 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a98004020 con 0x7f3aa813c380 2024-04-26T21:11:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa813cb20 con 0x7f3aa813c380 2024-04-26T21:11:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800f040 con 0x7f3aa813c380 2024-04-26T21:11:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.807+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa813d0e0 con 0x7f3aa813c380 2024-04-26T21:11:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.808+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a9801d070 con 0x7f3aa813c380 2024-04-26T21:11:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.808+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a98003b70 con 0x7f3aa813c380 2024-04-26T21:11:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.809+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa80693d0 con 0x7f3aa813c380 2024-04-26T21:11:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.812+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a98003dc0 con 0x7f3aa813c380 2024-04-26T21:11:36.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.880+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3aa8145340 con 0x7f3aa813c380 2024-04-26T21:11:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.884+0000 7f3a9ffff640 1 -- 172.21.15.40:0/557030171 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3a98015680 con 0x7f3aa813c380 2024-04-26T21:11:36.885 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 msgr2=0x7f3aa8162390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a98010630 tx=0x7f3a98010af0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 shutdown_connections 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa81628d0 0x7f3aa815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa813c380 0x7f3aa8162390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 --2- 172.21.15.40:0/557030171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940039b0 0x7f3aa80d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 >> 172.21.15.40:0/557030171 conn(0x7f3aa80d1e90 msgr2=0x7f3aa81458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:36.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 shutdown_connections 2024-04-26T21:11:36.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:36.885+0000 7f3aae0c4640 1 -- 172.21.15.40:0/557030171 wait complete. 2024-04-26T21:11:36.897 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T21:11:36.897 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T21:11:36.897 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:36.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T21:11:36.913 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:36.911+0000 7f9731517040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:11:36.923 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:36.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T21:11:36.930 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:36.930+0000 7f92e2a2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:11:36.956 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:36.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T21:11:37.023 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:11:37.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T21:11:37.061 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:37.059+0000 7f9731517040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:11:37.087 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:37.086+0000 7f92e2a2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:11:37.088 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:37.088 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T21:11:37.112 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:37.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T21:11:37.114 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:37.113+0000 7f9731517040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:11:37.138 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:37.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T21:11:37.143 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:37.142+0000 7f92e2a2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:11:37.202 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:11:37.202 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T21:11:37.266 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new ed5d3890-562f-46b3-b113-08a83bd94972 0 2024-04-26T21:11:37.356 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:37.354+0000 7f9731517040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:11:37.399 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:37.398+0000 7f92e2a2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:11:37.403 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:37.401+0000 7f9731517040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:11:37.405 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.403+0000 7f96d41a4640 1 Processor -- start 2024-04-26T21:11:37.405 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.403+0000 7f96d41a4640 1 -- start start 2024-04-26T21:11:37.407 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d41a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 0x7f96cc160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.407 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d41a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d41a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96cc159dd0 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d41a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc15f8f0 con 0x7f96cc1599f0 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d41a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc1608b0 con 0x7f96cc0b0db0 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d2741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d2741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45642/0 (socket says 172.21.15.184:45642) 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.404+0000 7f96d2741640 1 -- 172.21.15.184:0/3922693909 learned_addr learned my addr 172.21.15.184:0/3922693909 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d1f40640 1 --2- 172.21.15.184:0/3922693909 >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 0x7f96cc160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d2741640 1 -- 172.21.15.184:0/3922693909 >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 msgr2=0x7f96cc160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d2741640 1 --2- 172.21.15.184:0/3922693909 >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 0x7f96cc160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d2741640 1 -- 172.21.15.184:0/3922693909 >> v1:172.21.15.40:6789/0 conn(0x7f96cc0b6ed0 legacy=0x7f96cc06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d2741640 1 -- 172.21.15.184:0/3922693909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0b1700 con 0x7f96cc0b0db0 2024-04-26T21:11:37.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d1f40640 1 --2- 172.21.15.184:0/3922693909 >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 0x7f96cc160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:37.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d2741640 1 --2- 172.21.15.184:0/3922693909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009e50 tx=0x7f96c402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80ead7ab707e723 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.405+0000 7f96d0f3e640 1 -- 172.21.15.184:0/3922693909 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4036070 con 0x7f96cc0b0db0 2024-04-26T21:11:37.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d0f3e640 1 -- 172.21.15.184:0/3922693909 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c4031600 con 0x7f96cc0b0db0 2024-04-26T21:11:37.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d0f3e640 1 -- 172.21.15.184:0/3922693909 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40319d0 con 0x7f96cc0b0db0 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 -- 172.21.15.184:0/3922693909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 msgr2=0x7f96cc0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3922693909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009e50 tx=0x7f96c402a710 comp rx=0 tx=0).stop 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 -- 172.21.15.184:0/3922693909 shutdown_connections 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3922693909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc0b0db0 0x7f96cc0b11c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3922693909 >> v2:172.21.15.40:3300/0 conn(0x7f96cc1599f0 0x7f96cc160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.406+0000 7f96d41a4640 1 -- 172.21.15.184:0/3922693909 >> 172.21.15.184:0/3922693909 conn(0x7f96cc0b3b60 msgr2=0x7f96cc0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.407+0000 7f96d41a4640 1 -- 172.21.15.184:0/3922693909 shutdown_connections 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.407+0000 7f96d41a4640 1 -- 172.21.15.184:0/3922693909 wait complete. 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 Processor -- start 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 -- start start 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96cc0b0db0 0x7f96cc148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 0x7f96cc1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.413 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc15fb70 con 0x7f96cc0b0db0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc070250 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d41a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc15fe50 con 0x7f96cc1599f0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d2741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.408+0000 7f96d173f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 0x7f96cc1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d2741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45650/0 (socket says 172.21.15.184:45650) 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d2741640 1 -- 172.21.15.184:0/3587974473 learned_addr learned my addr 172.21.15.184:0/3587974473 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d1f40640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96cc0b0db0 0x7f96cc148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 -- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 msgr2=0x7f96cc0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 -- 172.21.15.184:0/3587974473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96cc0b0db0 msgr2=0x7f96cc148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96cc0b0db0 0x7f96cc148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 -- 172.21.15.184:0/3587974473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1f6840 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d2741640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d173f640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 0x7f96cc1581a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c390 tx=0x7f96c800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c80052d0 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c8005470 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1576c0 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8006a60 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc157c80 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96c8003640 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.409+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f96c8011c50 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.410+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc159530 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.412+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f96cc159dd0 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.447 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:37.446+0000 7f92e2a2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:11:37.485 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.483+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ed5d3890-562f-46b3-b113-08a83bd94972", "id": 0} v 0) -- 0x7f96cc0693d0 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.486+0000 7f96c27fc640 1 -- 172.21.15.184:0/3587974473 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed5d3890-562f-46b3-b113-08a83bd94972", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f96c8011590 con 0x7f96cc0b6ed0 2024-04-26T21:11:37.489 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 msgr2=0x7f96cc1581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 0x7f96cc1581a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c390 tx=0x7f96c800c850 comp rx=0 tx=0).stop 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 shutdown_connections 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96cc1599f0 0x7f96cc0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96cc0b6ed0 0x7f96cc1581a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 --2- 172.21.15.184:0/3587974473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96cc0b0db0 0x7f96cc148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.487+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 >> 172.21.15.184:0/3587974473 conn(0x7f96cc0b3b60 msgr2=0x7f96cc0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.488+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 shutdown_connections 2024-04-26T21:11:37.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.488+0000 7f96d41a4640 1 -- 172.21.15.184:0/3587974473 wait complete. 2024-04-26T21:11:37.500 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c03230cc-d7f6-411b-b073-2cc11a7b77d1 1 2024-04-26T21:11:37.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 Processor -- start 2024-04-26T21:11:37.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 -- start start 2024-04-26T21:11:37.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f778015e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7780071ed0 con 0x7f7780142f70 2024-04-26T21:11:37.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77800721b0 con 0x7f77800d0dc0 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f778695f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7780140520 con 0x7f7780157870 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f777f7fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f77800d0dc0 0x7f7780142860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:43364/0 (socket says 172.21.15.184:43364) 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f777f7fe640 1 -- 172.21.15.184:0/3479682714 learned_addr learned my addr 172.21.15.184:0/3479682714 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.596+0000 7f777ffff640 1 --2- 172.21.15.184:0/3479682714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f778015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777effd640 1 -- 172.21.15.184:0/3479682714 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1543685223 0 0) 0x7f77800721b0 con 0x7f77800d0dc0 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777effd640 1 -- 172.21.15.184:0/3479682714 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7768003520 con 0x7f77800d0dc0 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777ffff640 1 -- 172.21.15.184:0/3479682714 >> v1:172.21.15.184:6789/0 conn(0x7f77800d0dc0 legacy=0x7f7780142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777ffff640 1 -- 172.21.15.184:0/3479682714 >> v1:172.21.15.40:6789/0 conn(0x7f7780142f70 legacy=0x7f77800ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777ffff640 1 -- 172.21.15.184:0/3479682714 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7780072490 con 0x7f7780157870 2024-04-26T21:11:37.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.597+0000 7f777ffff640 1 --2- 172.21.15.184:0/3479682714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f778015e280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77700097d0 tx=0x7f777002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b034eb7e073520 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f777effd640 1 -- 172.21.15.184:0/3479682714 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770037070 con 0x7f7780157870 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f777effd640 1 -- 172.21.15.184:0/3479682714 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777002ae40 con 0x7f7780157870 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f777effd640 1 -- 172.21.15.184:0/3479682714 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770032660 con 0x7f7780157870 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 -- 172.21.15.184:0/3479682714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 msgr2=0x7f778015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 --2- 172.21.15.184:0/3479682714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f778015e280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77700097d0 tx=0x7f777002a130 comp rx=0 tx=0).stop 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 -- 172.21.15.184:0/3479682714 shutdown_connections 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 --2- 172.21.15.184:0/3479682714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f778015e280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 -- 172.21.15.184:0/3479682714 >> 172.21.15.184:0/3479682714 conn(0x7f77800705b0 msgr2=0x7f77800709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 -- 172.21.15.184:0/3479682714 shutdown_connections 2024-04-26T21:11:37.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.598+0000 7f778695f640 1 -- 172.21.15.184:0/3479682714 wait complete. 2024-04-26T21:11:37.606 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.599+0000 7f778695f640 1 Processor -- start 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 -- start start 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77800d0dc0 0x7f778013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77800d0920 con 0x7f77800d0dc0 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7780072800 con 0x7f7780142f70 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f778695f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77800d1240 con 0x7f7780157870 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.607 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f7784efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f7784efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45670/0 (socket says 172.21.15.184:45670) 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58538/0 (socket says 172.21.15.184:58538) 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f7784efc640 1 -- 172.21.15.184:0/4111095079 learned_addr learned my addr 172.21.15.184:0/4111095079 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.600+0000 7f777ffff640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77800d0dc0 0x7f778013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 -- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 msgr2=0x7f7780135580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 -- 172.21.15.184:0/4111095079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77800d0dc0 msgr2=0x7f778013fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77800d0dc0 0x7f778013fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 -- 172.21.15.184:0/4111095079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77801449e0 con 0x7f7780157870 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777f7fe640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f7784efc640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f777800c780 tx=0x7f777800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778017070 con 0x7f7780157870 2024-04-26T21:11:37.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7778003da0 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77800b8500 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780052b0 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77800b8ac0 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f777801a070 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.601+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f777801c7f0 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.602+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77800693d0 con 0x7f7780157870 2024-04-26T21:11:37.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.604+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7778021070 con 0x7f7780157870 2024-04-26T21:11:37.676 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.674+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c03230cc-d7f6-411b-b073-2cc11a7b77d1", "id": 1} v 0) -- 0x7f7780071370 con 0x7f7780157870 2024-04-26T21:11:37.682 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.680+0000 7f777cff9640 1 -- 172.21.15.184:0/4111095079 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c03230cc-d7f6-411b-b073-2cc11a7b77d1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f777800a5e0 con 0x7f7780157870 2024-04-26T21:11:37.682 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:11:37.683 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 msgr2=0x7f7780160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.683 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f777800c780 tx=0x7f777800cc40 comp rx=0 tx=0).stop 2024-04-26T21:11:37.683 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 shutdown_connections 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7780157870 0x7f7780160f60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7780142f70 0x7f7780135580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 --2- 172.21.15.184:0/4111095079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77800d0dc0 0x7f778013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 >> 172.21.15.184:0/4111095079 conn(0x7f77800705b0 msgr2=0x7f778015f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 shutdown_connections 2024-04-26T21:11:37.684 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.681+0000 7f778695f640 1 -- 172.21.15.184:0/4111095079 wait complete. 2024-04-26T21:11:37.693 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 000d1a68-5653-4e07-85cc-c9dfbf4ebba7 2 2024-04-26T21:11:37.792 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.790+0000 7f504842a640 1 Processor -- start 2024-04-26T21:11:37.792 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.790+0000 7f504842a640 1 -- start start 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f504842a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f504006d2b0 0x7f50401602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f504842a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50400aea50 con 0x7f504006e960 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f504842a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50400aed30 con 0x7f5040070e80 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f504842a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f50400af010 con 0x7f504006d2b0 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50459c5640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f5040070e80 0x7f504006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:43378/0 (socket says 172.21.15.184:43378) 2024-04-26T21:11:37.800 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50459c5640 1 -- 172.21.15.184:0/2926439139 learned_addr learned my addr 172.21.15.184:0/2926439139 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50461c6640 1 --2- 172.21.15.184:0/2926439139 >> v2:172.21.15.40:3300/0 conn(0x7f504006d2b0 0x7f50401602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4185512320 0 0) 0x7f50400aed30 con 0x7f5040070e80 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f502c003520 con 0x7f5040070e80 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630779287 0 0) 0x7f50400aea50 con 0x7f504006e960 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.791+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50400aed30 con 0x7f504006e960 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2268446064 0 0) 0x7f502c003520 con 0x7f5040070e80 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 >> v2:172.21.15.40:3300/0 conn(0x7f504006d2b0 msgr2=0x7f50401602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 --2- 172.21.15.184:0/2926439139 >> v2:172.21.15.40:3300/0 conn(0x7f504006d2b0 0x7f50401602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 >> v1:172.21.15.40:6789/0 conn(0x7f504006e960 legacy=0x7f504006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50400af2f0 con 0x7f5040070e80 2024-04-26T21:11:37.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f503c001670 con 0x7f5040070e80 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 >> v1:172.21.15.184:6789/0 conn(0x7f5040070e80 legacy=0x7f504006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c003b40 0x7f502c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 0x7f502c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0051c0 0x7f502c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50400af010 con 0x7f502c003b40 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00c040 con 0x7f502c004800 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f502c00c240 con 0x7f502c0051c0 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50461c6640 1 --2- 172.21.15.184:0/2926439139 >> v2:172.21.15.40:3300/0 conn(0x7f504006d2b0 0x7f50401602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 0x7f502c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50469c7640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0051c0 0x7f502c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50461c6640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c003b40 0x7f502c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 -- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0051c0 msgr2=0x7f502c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0051c0 0x7f502c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 -- 172.21.15.184:0/2926439139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c003b40 msgr2=0x7f502c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c003b40 0x7f502c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50459c5640 1 -- 172.21.15.184:0/2926439139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c003520 con 0x7f502c004800 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50461c6640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c003b40 0x7f502c003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.792+0000 7f50469c7640 1 -- 172.21.15.184:0/2926439139 reap_dead start 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.793+0000 7f50459c5640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 0x7f502c004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c1c0 tx=0x7f502c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3275f023effa8d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.793+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c007cc0 con 0x7f502c004800 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.793+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503c005ad0 con 0x7f502c004800 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.793+0000 7f50451c4640 1 -- 172.21.15.184:0/2926439139 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c007070 con 0x7f502c004800 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 -- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 msgr2=0x7f502c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 0x7f502c004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c1c0 tx=0x7f502c005830 comp rx=0 tx=0).stop 2024-04-26T21:11:37.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 -- 172.21.15.184:0/2926439139 shutdown_connections 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 --2- 172.21.15.184:0/2926439139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c004800 0x7f502c004c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 -- 172.21.15.184:0/2926439139 >> 172.21.15.184:0/2926439139 conn(0x7f5040072900 msgr2=0x7f5040072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 -- 172.21.15.184:0/2926439139 shutdown_connections 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.794+0000 7f504842a640 1 -- 172.21.15.184:0/2926439139 wait complete. 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.795+0000 7f504842a640 1 Processor -- start 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.795+0000 7f504842a640 1 -- start start 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c004800 0x7f50400b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50400b71f0 0x7f50401f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50400af810 con 0x7f502c004800 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f504015d170 con 0x7f50401f8cc0 2024-04-26T21:11:37.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50400af990 con 0x7f50400b71f0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50459c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50400b71f0 0x7f50401f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58550/0 (socket says 172.21.15.184:58550) 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 -- 172.21.15.184:0/2462455577 learned_addr learned my addr 172.21.15.184:0/2462455577 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50461c6640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c004800 0x7f50400b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 -- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50400b71f0 msgr2=0x7f50401f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50400b71f0 0x7f50401f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 -- 172.21.15.184:0/2462455577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c004800 msgr2=0x7f50400b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c004800 0x7f50400b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 -- 172.21.15.184:0/2462455577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5040149d60 con 0x7f50401f8cc0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f50469c7640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50400cda80 tx=0x7f503800feb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003b80 con 0x7f50401f8cc0 2024-04-26T21:11:37.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50400b4ed0 con 0x7f50401f8cc0 2024-04-26T21:11:37.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.796+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50400b5490 con 0x7f50401f8cc0 2024-04-26T21:11:37.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.797+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50400693d0 con 0x7f50401f8cc0 2024-04-26T21:11:37.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.799+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038018070 con 0x7f50401f8cc0 2024-04-26T21:11:37.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.799+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380050d0 con 0x7f50401f8cc0 2024-04-26T21:11:37.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.799+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5038005350 con 0x7f50401f8cc0 2024-04-26T21:11:37.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.799+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f503801d880 con 0x7f50401f8cc0 2024-04-26T21:11:37.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.801+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5038005640 con 0x7f50401f8cc0 2024-04-26T21:11:37.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.873+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "000d1a68-5653-4e07-85cc-c9dfbf4ebba7", "id": 2} v 0) -- 0x7f50400cd6d0 con 0x7f50401f8cc0 2024-04-26T21:11:37.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.878+0000 7f5036ffd640 1 -- 172.21.15.184:0/2462455577 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "000d1a68-5653-4e07-85cc-c9dfbf4ebba7", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f503801d460 con 0x7f50401f8cc0 2024-04-26T21:11:37.880 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 msgr2=0x7f50401f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50400cda80 tx=0x7f503800feb0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 shutdown_connections 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401f8cc0 0x7f50401f9170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50400b71f0 0x7f50401f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 --2- 172.21.15.184:0/2462455577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c004800 0x7f50400b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 >> 172.21.15.184:0/2462455577 conn(0x7f5040072900 msgr2=0x7f50400b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 shutdown_connections 2024-04-26T21:11:37.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.879+0000 7f504842a640 1 -- 172.21.15.184:0/2462455577 wait complete. 2024-04-26T21:11:37.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new f4e0d16b-3813-4bbd-8b55-eed7726eb89c 3 2024-04-26T21:11:37.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.986+0000 7f46bbaea640 1 Processor -- start 2024-04-26T21:11:37.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.986+0000 7f46bbaea640 1 -- start start 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46bbaea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46b415f2b0 0x7f46b406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46bbaea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f46b4162ea0 0x7f46b406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46bbaea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b40716d0 con 0x7f46b40b1570 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46bbaea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f46b4073220 con 0x7f46b415f2b0 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46bbaea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f46b4073500 con 0x7f46b4162ea0 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46ba087640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f46b4162ea0 0x7f46b406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46b9085640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f46b40b1570 0x7f46b40b4f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:43386/0 (socket says 172.21.15.184:43386) 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46b9085640 1 -- 172.21.15.184:0/1900752213 learned_addr learned my addr 172.21.15.184:0/1900752213 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.987+0000 7f46b9886640 1 --2- 172.21.15.184:0/1900752213 >> v2:172.21.15.40:3300/0 conn(0x7f46b415f2b0 0x7f46b406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.988+0000 7f46ba087640 1 -- 172.21.15.184:0/1900752213 >> v2:172.21.15.40:3300/0 conn(0x7f46b415f2b0 msgr2=0x7f46b406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:37.991 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.988+0000 7f46ba087640 1 --2- 172.21.15.184:0/1900752213 >> v2:172.21.15.40:3300/0 conn(0x7f46b415f2b0 0x7f46b406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.991 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.988+0000 7f46ba087640 1 -- 172.21.15.184:0/1900752213 >> v1:172.21.15.184:6789/0 conn(0x7f46b40b1570 legacy=0x7f46b40b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.991 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.988+0000 7f46ba087640 1 -- 172.21.15.184:0/1900752213 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b40737e0 con 0x7f46b4162ea0 2024-04-26T21:11:37.991 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.988+0000 7f46b8884640 1 -- 172.21.15.184:0/1900752213 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407250580 0 0) 0x7f46a4001110 con 0x7f46b40b1570 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46ba087640 1 --2- 172.21.15.184:0/1900752213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b4162ea0 0x7f46b406c7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46b4070430 tx=0x7f46ac02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff94faed1d8fea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46b8884640 1 -- 172.21.15.184:0/1900752213 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac007740 con 0x7f46b4162ea0 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46b8884640 1 -- 172.21.15.184:0/1900752213 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac0078e0 con 0x7f46b4162ea0 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46b8884640 1 -- 172.21.15.184:0/1900752213 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac007cb0 con 0x7f46b4162ea0 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46bbaea640 1 -- 172.21.15.184:0/1900752213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b4162ea0 msgr2=0x7f46b406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.989+0000 7f46bbaea640 1 --2- 172.21.15.184:0/1900752213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b4162ea0 0x7f46b406c7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46b4070430 tx=0x7f46ac02a560 comp rx=0 tx=0).stop 2024-04-26T21:11:37.992 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 -- 172.21.15.184:0/1900752213 shutdown_connections 2024-04-26T21:11:37.993 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 --2- 172.21.15.184:0/1900752213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b4162ea0 0x7f46b406c7e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.993 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 --2- 172.21.15.184:0/1900752213 >> v2:172.21.15.40:3300/0 conn(0x7f46b415f2b0 0x7f46b406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.993 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 -- 172.21.15.184:0/1900752213 >> 172.21.15.184:0/1900752213 conn(0x7f46b40b3c40 msgr2=0x7f46b40b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:37.993 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 -- 172.21.15.184:0/1900752213 shutdown_connections 2024-04-26T21:11:37.993 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.990+0000 7f46bbaea640 1 -- 172.21.15.184:0/1900752213 wait complete. 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 Processor -- start 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 -- start start 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46b415f2b0 0x7f46b4148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46b4162ea0 0x7f46b40d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46b415f690 con 0x7f46b4162ea0 2024-04-26T21:11:37.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4073da0 con 0x7f46b40b1570 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.991+0000 7f46bbaea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46b40745a0 con 0x7f46b415f2b0 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58580/0 (socket says 172.21.15.184:58580) 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 -- 172.21.15.184:0/4257832144 learned_addr learned my addr 172.21.15.184:0/4257832144 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46b415f2b0 0x7f46b4148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46ba087640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46b4162ea0 0x7f46b40d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 -- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46b415f2b0 msgr2=0x7f46b4148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46b415f2b0 0x7f46b4148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 -- 172.21.15.184:0/4257832144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46b4162ea0 msgr2=0x7f46b40d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46b4162ea0 0x7f46b40d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 -- 172.21.15.184:0/4257832144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b4161ab0 con 0x7f46b40b1570 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46b9886640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e870 tx=0x7f46b000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:37.996 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0017070 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b000c940 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b4161d50 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000cd30 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41625f0 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46b001a070 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.992+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f46b0013470 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.993+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b40693d0 con 0x7f46b40b1570 2024-04-26T21:11:37.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:37.995+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f46b0006020 con 0x7f46b40b1570 2024-04-26T21:11:38.070 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.068+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4e0d16b-3813-4bbd-8b55-eed7726eb89c", "id": 3} v 0) -- 0x7f46b41617f0 con 0x7f46b40b1570 2024-04-26T21:11:38.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.073+0000 7f46aa7fc640 1 -- 172.21.15.184:0/4257832144 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4e0d16b-3813-4bbd-8b55-eed7726eb89c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f46b00064d0 con 0x7f46b40b1570 2024-04-26T21:11:38.075 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-04-26T21:11:38.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 msgr2=0x7f46b41612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e870 tx=0x7f46b000ed30 comp rx=0 tx=0).stop 2024-04-26T21:11:38.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 shutdown_connections 2024-04-26T21:11:38.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46b4162ea0 0x7f46b40d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46b415f2b0 0x7f46b4148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 --2- 172.21.15.184:0/4257832144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46b40b1570 0x7f46b41612b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 >> 172.21.15.184:0/4257832144 conn(0x7f46b40b3c40 msgr2=0x7f46b40afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 shutdown_connections 2024-04-26T21:11:38.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.074+0000 7f46bbaea640 1 -- 172.21.15.184:0/4257832144 wait complete. 2024-04-26T21:11:38.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new f1884ac6-1116-4ae6-8df5-baaa810c6939 4 2024-04-26T21:11:38.146 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.145+0000 7f9731517040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:11:38.184 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.183+0000 7f0699991640 1 Processor -- start 2024-04-26T21:11:38.184 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.183+0000 7f0699991640 1 -- start start 2024-04-26T21:11:38.185 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.183+0000 7f92e2a2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f0699991640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 0x7f06940b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f0699991640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06940b60b0 0x7f06940d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f0699991640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06940d3c90 con 0x7f069415f680 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f0699991640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f069415fa60 con 0x7f06940d3820 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f0699991640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06940d3f50 con 0x7f06940b60b0 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06940b60b0 0x7f06940d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06940b60b0 0x7f06940d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58592/0 (socket says 172.21.15.184:58592) 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 -- 172.21.15.184:0/1839380631 learned_addr learned my addr 172.21.15.184:0/1839380631 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06927fc640 1 --2- 172.21.15.184:0/1839380631 >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 0x7f06940b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 -- 172.21.15.184:0/1839380631 >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 msgr2=0x7f06940b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 --2- 172.21.15.184:0/1839380631 >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 0x7f06940b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.187 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 -- 172.21.15.184:0/1839380631 >> v1:172.21.15.40:6789/0 conn(0x7f069415f680 legacy=0x7f0694163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06937fe640 1 -- 172.21.15.184:0/1839380631 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06940b5090 con 0x7f06940b60b0 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.184+0000 7f06927fc640 1 --2- 172.21.15.184:0/1839380631 >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 0x7f06940b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.185+0000 7f06937fe640 1 --2- 172.21.15.184:0/1839380631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f06940d4f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06940cf840 tx=0x7f068c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95727904c706adb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.185+0000 7f0691ffb640 1 -- 172.21.15.184:0/1839380631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c036070 con 0x7f06940b60b0 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.185+0000 7f0691ffb640 1 -- 172.21.15.184:0/1839380631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c031600 con 0x7f06940b60b0 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.185+0000 7f0691ffb640 1 -- 172.21.15.184:0/1839380631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0319d0 con 0x7f06940b60b0 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 -- 172.21.15.184:0/1839380631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 msgr2=0x7f06940d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 --2- 172.21.15.184:0/1839380631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f06940d4f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06940cf840 tx=0x7f068c02a730 comp rx=0 tx=0).stop 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 -- 172.21.15.184:0/1839380631 shutdown_connections 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 --2- 172.21.15.184:0/1839380631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f06940d4f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 --2- 172.21.15.184:0/1839380631 >> v2:172.21.15.40:3300/0 conn(0x7f06940d3820 0x7f06940b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.189 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 -- 172.21.15.184:0/1839380631 >> 172.21.15.184:0/1839380631 conn(0x7f069408eca0 msgr2=0x7f069409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.189 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 -- 172.21.15.184:0/1839380631 shutdown_connections 2024-04-26T21:11:38.189 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.186+0000 7f0699991640 1 -- 172.21.15.184:0/1839380631 wait complete. 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.187+0000 7f0699991640 1 Processor -- start 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.187+0000 7f0699991640 1 -- start start 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06940d3820 0x7f0694153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f069415f680 0x7f06940cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06940d55e0 con 0x7f06940d3820 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06940d5460 con 0x7f06940b60b0 2024-04-26T21:11:38.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0699991640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06940d5760 con 0x7f069415f680 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f06937fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f069415f680 0x7f06940cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58608/0 (socket says 172.21.15.184:58608) 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 -- 172.21.15.184:0/1653573797 learned_addr learned my addr 172.21.15.184:0/1653573797 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f06927fc640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06940d3820 0x7f0694153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 -- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f069415f680 msgr2=0x7f06940cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f069415f680 0x7f06940cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 -- 172.21.15.184:0/1653573797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06940d3820 msgr2=0x7f0694153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06940d3820 0x7f0694153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 -- 172.21.15.184:0/1653573797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06940cedb0 con 0x7f06940b60b0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f0692ffd640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c3f0 tx=0x7f067c00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0052d0 con 0x7f06940b60b0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.188+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c005490 con 0x7f06940b60b0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c007260 con 0x7f06940b60b0 2024-04-26T21:11:38.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06940cf090 con 0x7f06940b60b0 2024-04-26T21:11:38.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0694160530 con 0x7f06940b60b0 2024-04-26T21:11:38.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f067c007400 con 0x7f06940b60b0 2024-04-26T21:11:38.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f067c011590 con 0x7f06940b60b0 2024-04-26T21:11:38.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.189+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069406a290 con 0x7f06940b60b0 2024-04-26T21:11:38.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.191+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f067c003a00 con 0x7f06940b60b0 2024-04-26T21:11:38.238 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.237+0000 7f9731517040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:11:38.265 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.264+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f1884ac6-1116-4ae6-8df5-baaa810c6939", "id": 4} v 0) -- 0x7f06940b4cd0 con 0x7f06940b60b0 2024-04-26T21:11:38.268 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.267+0000 7f068bfff640 1 -- 172.21.15.184:0/1653573797 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1884ac6-1116-4ae6-8df5-baaa810c6939", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f067c003be0 con 0x7f06940b60b0 2024-04-26T21:11:38.268 INFO:teuthology.orchestra.run.smithi184.stdout:4 2024-04-26T21:11:38.269 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 msgr2=0x7f0694162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.269 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c3f0 tx=0x7f067c00c8b0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 shutdown_connections 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f069415f680 0x7f06940cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06940d3820 0x7f0694153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 --2- 172.21.15.184:0/1653573797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06940b60b0 0x7f0694162640 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 >> 172.21.15.184:0/1653573797 conn(0x7f069408eca0 msgr2=0x7f06941636d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 shutdown_connections 2024-04-26T21:11:38.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.268+0000 7f0699991640 1 -- 172.21.15.184:0/1653573797 wait complete. 2024-04-26T21:11:38.277 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.276+0000 7f92e2a2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:11:38.280 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 0254928f-2523-4319-92f6-71fed0af94c8 5 2024-04-26T21:11:38.283 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.282+0000 7f9731517040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:11:38.322 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.321+0000 7f92e2a2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.376+0000 7f36dc420640 1 Processor -- start 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.376+0000 7f36dc420640 1 -- start start 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 0x7f36d4073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36d4073690 0x7f36d415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36d406f4d0 con 0x7f36d4073690 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36d406f7b0 con 0x7f36d40712f0 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36dc420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36d406fa90 con 0x7f36d4159cf0 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36d99bb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 0x7f36d4073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45706/0 (socket says 172.21.15.184:45706) 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 -- 172.21.15.184:0/486276874 learned_addr learned my addr 172.21.15.184:0/486276874 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da9bd640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.40:3300/0 conn(0x7f36d4073690 0x7f36d415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 -- 172.21.15.184:0/486276874 >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 msgr2=0x7f36d4073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 0x7f36d4073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 -- 172.21.15.184:0/486276874 >> v2:172.21.15.40:3300/0 conn(0x7f36d4073690 msgr2=0x7f36d415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.40:3300/0 conn(0x7f36d4073690 0x7f36d415fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36da1bc640 1 -- 172.21.15.184:0/486276874 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d406fd70 con 0x7f36d4159cf0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.377+0000 7f36d99bb640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 0x7f36d4073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36da1bc640 1 --2- 172.21.15.184:0/486276874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36d40d2b20 tx=0x7f36d002a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13e0787f9cb884f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36d91ba640 1 -- 172.21.15.184:0/486276874 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0037070 con 0x7f36d4159cf0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36d91ba640 1 -- 172.21.15.184:0/486276874 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d0032440 con 0x7f36d4159cf0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36d91ba640 1 -- 172.21.15.184:0/486276874 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0032810 con 0x7f36d4159cf0 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36dc420640 1 -- 172.21.15.184:0/486276874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 msgr2=0x7f36d4070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.378+0000 7f36dc420640 1 --2- 172.21.15.184:0/486276874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36d40d2b20 tx=0x7f36d002a140 comp rx=0 tx=0).stop 2024-04-26T21:11:38.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 -- 172.21.15.184:0/486276874 shutdown_connections 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.40:3300/0 conn(0x7f36d4073690 0x7f36d415fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 --2- 172.21.15.184:0/486276874 >> v2:172.21.15.184:3300/0 conn(0x7f36d40712f0 0x7f36d4073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 --2- 172.21.15.184:0/486276874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4070db0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 -- 172.21.15.184:0/486276874 >> 172.21.15.184:0/486276874 conn(0x7f36d40d1e90 msgr2=0x7f36d40d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 -- 172.21.15.184:0/486276874 shutdown_connections 2024-04-26T21:11:38.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.379+0000 7f36dc420640 1 -- 172.21.15.184:0/486276874 wait complete. 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 Processor -- start 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 -- start start 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36d40712f0 0x7f36d4153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40b6900 con 0x7f36d40712f0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40b6780 con 0x7f36d4073690 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.380+0000 7f36dc420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36d4160020 con 0x7f36d4159cf0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36d99bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36d99bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58636/0 (socket says 172.21.15.184:58636) 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36d99bb640 1 -- 172.21.15.184:0/2789921108 learned_addr learned my addr 172.21.15.184:0/2789921108 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da1bc640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36d40712f0 0x7f36d4153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 -- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 msgr2=0x7f36d4140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 -- 172.21.15.184:0/2789921108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36d40712f0 msgr2=0x7f36d4153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36d40712f0 0x7f36d4153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 -- 172.21.15.184:0/2789921108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d4140da0 con 0x7f36d4159cf0 2024-04-26T21:11:38.385 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36d99bb640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36da9bd640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4151740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007d90 tx=0x7f36cc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00edd0 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36cc003990 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d4153900 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc003d80 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d4153ec0 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36cc01b020 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.381+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f36cc014b80 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.382+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d40693d0 con 0x7f36d4159cf0 2024-04-26T21:11:38.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.384+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f36d4140da0 con 0x7f36d4159cf0 2024-04-26T21:11:38.457 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.456+0000 7f9731517040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:11:38.458 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.457+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0254928f-2523-4319-92f6-71fed0af94c8", "id": 5} v 0) -- 0x7f36d40b6420 con 0x7f36d4159cf0 2024-04-26T21:11:38.462 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.462+0000 7f36caffd640 1 -- 172.21.15.184:0/2789921108 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0254928f-2523-4319-92f6-71fed0af94c8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f36cc014dd0 con 0x7f36d4159cf0 2024-04-26T21:11:38.463 INFO:teuthology.orchestra.run.smithi184.stdout:5 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 msgr2=0x7f36d4151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4151740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007d90 tx=0x7f36cc00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 shutdown_connections 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36d4159cf0 0x7f36d4151740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36d4073690 0x7f36d4140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 --2- 172.21.15.184:0/2789921108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36d40712f0 0x7f36d4153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 >> 172.21.15.184:0/2789921108 conn(0x7f36d40d1e90 msgr2=0x7f36d4073d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 shutdown_connections 2024-04-26T21:11:38.464 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.463+0000 7f36dc420640 1 -- 172.21.15.184:0/2789921108 wait complete. 2024-04-26T21:11:38.474 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5808a23b-e704-4ab8-a08a-ca416254c458 6 2024-04-26T21:11:38.502 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.501+0000 7f92e2a2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:11:38.511 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.509+0000 7f9731517040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:11:38.565 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.564+0000 7f9731517040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:11:38.568 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.568+0000 7f92e2a2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:11:38.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.607+0000 7f3887922640 1 Processor -- start 2024-04-26T21:11:38.608 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.607+0000 7f3887922640 1 -- start start 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3887922640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3887922640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3887922640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f388015f970 con 0x7f388015cf70 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3887922640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f388015fc50 con 0x7f388006d9c0 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3887922640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f388015ff30 con 0x7f38800736b0 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3884ebd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3884ebd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58644/0 (socket says 172.21.15.184:58644) 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45722/0 (socket says 172.21.15.184:45722) 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3884ebd640 1 -- 172.21.15.184:0/2553564989 learned_addr learned my addr 172.21.15.184:0/2553564989 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 -- 172.21.15.184:0/2553564989 >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 msgr2=0x7f38800ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 --2- 172.21.15.184:0/2553564989 >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 -- 172.21.15.184:0/2553564989 >> v1:172.21.15.184:6789/0 conn(0x7f388015cf70 legacy=0x7f388006d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3885ebf640 1 -- 172.21.15.184:0/2553564989 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880160210 con 0x7f38800736b0 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3877fff640 1 -- 172.21.15.184:0/2553564989 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154521634 0 0) 0x7f388015f970 con 0x7f388015cf70 2024-04-26T21:11:38.610 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.608+0000 7f3884ebd640 1 --2- 172.21.15.184:0/2553564989 >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.611 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.609+0000 7f3885ebf640 1 --2- 172.21.15.184:0/2553564989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3878009d90 tx=0x7f387802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5799c515027fd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.611 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.609+0000 7f3877fff640 1 -- 172.21.15.184:0/2553564989 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878031600 con 0x7f38800736b0 2024-04-26T21:11:38.611 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.609+0000 7f3877fff640 1 -- 172.21.15.184:0/2553564989 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38780317a0 con 0x7f38800736b0 2024-04-26T21:11:38.611 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.609+0000 7f3877fff640 1 -- 172.21.15.184:0/2553564989 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878031b70 con 0x7f38800736b0 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 -- 172.21.15.184:0/2553564989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 msgr2=0x7f3880160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 --2- 172.21.15.184:0/2553564989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3878009d90 tx=0x7f387802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 -- 172.21.15.184:0/2553564989 shutdown_connections 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 --2- 172.21.15.184:0/2553564989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38800736b0 0x7f3880160a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 --2- 172.21.15.184:0/2553564989 >> v2:172.21.15.184:3300/0 conn(0x7f388006d9c0 0x7f38800ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 -- 172.21.15.184:0/2553564989 >> 172.21.15.184:0/2553564989 conn(0x7f3880072920 msgr2=0x7f3880072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 -- 172.21.15.184:0/2553564989 shutdown_connections 2024-04-26T21:11:38.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.610+0000 7f3887922640 1 -- 172.21.15.184:0/2553564989 wait complete. 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.611+0000 7f3887922640 1 Processor -- start 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.611+0000 7f3887922640 1 -- start start 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 0x7f38800cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3880160fd0 con 0x7f38800736b0 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3880073b40 con 0x7f388006d9c0 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3880161150 con 0x7f388015cf70 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f38856be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f38856be640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58648/0 (socket says 172.21.15.184:58648) 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f38856be640 1 -- 172.21.15.184:0/3318641485 learned_addr learned my addr 172.21.15.184:0/3318641485 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:45734/0 (socket says 172.21.15.184:45734) 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3884ebd640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 0x7f38800cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 -- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 msgr2=0x7f3880159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 -- 172.21.15.184:0/3318641485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 msgr2=0x7f38800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 0x7f38800cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 -- 172.21.15.184:0/3318641485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800cfbd0 con 0x7f388015cf70 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f38856be640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3884ebd640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 0x7f38800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3885ebf640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3878031000 tx=0x7f3878004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878040070 con 0x7f388015cf70 2024-04-26T21:11:38.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3878030ba0 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3880159b50 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.612+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878009450 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.613+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38800b5720 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.613+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3878039040 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.613+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f387803ca10 con 0x7f388015cf70 2024-04-26T21:11:38.617 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.614+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38800693d0 con 0x7f388015cf70 2024-04-26T21:11:38.618 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.616+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3878064ec0 con 0x7f388015cf70 2024-04-26T21:11:38.624 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.623+0000 7f92e2a2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:11:38.681 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:38.680+0000 7f9731517040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:11:38.687 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.686+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5808a23b-e704-4ab8-a08a-ca416254c458", "id": 6} v 0) -- 0x7f388015ac30 con 0x7f388015cf70 2024-04-26T21:11:38.692 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.691+0000 7f3875ffb640 1 -- 172.21.15.184:0/3318641485 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5808a23b-e704-4ab8-a08a-ca416254c458", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3878053030 con 0x7f388015cf70 2024-04-26T21:11:38.692 INFO:teuthology.orchestra.run.smithi184.stdout:6 2024-04-26T21:11:38.693 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 msgr2=0x7f38801f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3878031000 tx=0x7f3878004a40 comp rx=0 tx=0).stop 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 shutdown_connections 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f388015cf70 0x7f38801f5e90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38800736b0 0x7f38800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 --2- 172.21.15.184:0/3318641485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f388006d9c0 0x7f3880159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.692+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 >> 172.21.15.184:0/3318641485 conn(0x7f3880072920 msgr2=0x7f388015b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.693+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 shutdown_connections 2024-04-26T21:11:38.694 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.693+0000 7f3887922640 1 -- 172.21.15.184:0/3318641485 wait complete. 2024-04-26T21:11:38.704 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new a490edd1-3e4d-4620-a20b-200d529a06e3 7 2024-04-26T21:11:38.742 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:38.741+0000 7f92e2a2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:11:38.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.801+0000 7f3e8f1c1640 1 Processor -- start 2024-04-26T21:11:38.802 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.801+0000 7f3e8f1c1640 1 -- start start 2024-04-26T21:11:38.803 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e88073950 0x7f3e880af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 0x7f3e88162e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3e880afc30 con 0x7f3e88073950 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3e8815f730 con 0x7f3e8815f350 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8f1c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880b4890 con 0x7f3e880af7e0 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8cf5d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8d75e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 0x7f3e88162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8cf5d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58656/0 (socket says 172.21.15.184:58656) 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e8cf5d640 1 -- 172.21.15.184:0/372486318 learned_addr learned my addr 172.21.15.184:0/372486318 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.802+0000 7f3e87fff640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.40:3300/0 conn(0x7f3e88073950 0x7f3e880af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 -- 172.21.15.184:0/372486318 >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 msgr2=0x7f3e88073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.804 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 -- 172.21.15.184:0/372486318 >> v2:172.21.15.40:3300/0 conn(0x7f3e88073950 msgr2=0x7f3e880af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:38.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.40:3300/0 conn(0x7f3e88073950 0x7f3e880af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 -- 172.21.15.184:0/372486318 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880b4b70 con 0x7f3e880af7e0 2024-04-26T21:11:38.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8cf5d640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/372486318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 0x7f3e88162e60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3e88075e20 tx=0x7f3e8002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0851269bf23eef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e877fe640 1 -- 172.21.15.184:0/372486318 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e80031600 con 0x7f3e880af7e0 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e877fe640 1 -- 172.21.15.184:0/372486318 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e800317a0 con 0x7f3e880af7e0 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.803+0000 7f3e877fe640 1 -- 172.21.15.184:0/372486318 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e80031b70 con 0x7f3e880af7e0 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/372486318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 msgr2=0x7f3e88162e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/372486318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 0x7f3e88162e60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3e88075e20 tx=0x7f3e8002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/372486318 shutdown_connections 2024-04-26T21:11:38.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/372486318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880af7e0 0x7f3e88162e60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.40:3300/0 conn(0x7f3e88073950 0x7f3e880af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/372486318 >> v2:172.21.15.184:3300/0 conn(0x7f3e8815f350 0x7f3e88073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/372486318 >> 172.21.15.184:0/372486318 conn(0x7f3e88072920 msgr2=0x7f3e88072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/372486318 shutdown_connections 2024-04-26T21:11:38.807 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.804+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/372486318 wait complete. 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 Processor -- start 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 -- start start 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880af7e0 0x7f3e880ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 0x7f3e880ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.806+0000 7f3e8f1c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b5410 con 0x7f3e880af7e0 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8f1c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b5130 con 0x7f3e88073950 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8f1c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880aeb50 con 0x7f3e8815f350 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8cf5d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 0x7f3e880ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8cf5d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:58672/0 (socket says 172.21.15.184:58672) 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8cf5d640 1 -- 172.21.15.184:0/3227157716 learned_addr learned my addr 172.21.15.184:0/3227157716 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-04-26T21:11:38.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e87fff640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880af7e0 0x7f3e880ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 -- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 msgr2=0x7f3e880cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 -- 172.21.15.184:0/3227157716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880af7e0 msgr2=0x7f3e880ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880af7e0 0x7f3e880ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 -- 172.21.15.184:0/3227157716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8815fa10 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8cf5d640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8d75e640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 0x7f3e880ce5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e8815f8b0 tx=0x7f3e80004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e80040070 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8003c850 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8815fcf0 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8003cc40 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e881602b0 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e80039040 con 0x7f3e8815f350 2024-04-26T21:11:38.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.807+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3e80044a70 con 0x7f3e8815f350 2024-04-26T21:11:38.812 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.808+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e880693d0 con 0x7f3e8815f350 2024-04-26T21:11:38.812 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.810+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3e80044410 con 0x7f3e8815f350 2024-04-26T21:11:38.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.883+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a490edd1-3e4d-4620-a20b-200d529a06e3", "id": 7} v 0) -- 0x7f3e8806d0c0 con 0x7f3e8815f350 2024-04-26T21:11:38.888 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.888+0000 7f3e857fa640 1 -- 172.21.15.184:0/3227157716 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a490edd1-3e4d-4620-a20b-200d529a06e3", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3e800445f0 con 0x7f3e8815f350 2024-04-26T21:11:38.889 INFO:teuthology.orchestra.run.smithi184.stdout:7 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 msgr2=0x7f3e880ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 0x7f3e880ce5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e8815f8b0 tx=0x7f3e80004a40 comp rx=0 tx=0).stop 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 shutdown_connections 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e8815f350 0x7f3e880ce5c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880af7e0 0x7f3e880ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 --2- 172.21.15.184:0/3227157716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88073950 0x7f3e880cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 >> 172.21.15.184:0/3227157716 conn(0x7f3e88072920 msgr2=0x7f3e8806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 shutdown_connections 2024-04-26T21:11:38.890 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:11:38.889+0000 7f3e8f1c1640 1 -- 172.21.15.184:0/3227157716 wait complete. 2024-04-26T21:11:38.900 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T21:11:38.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T21:11:38.902 INFO:tasks.ceph.osd.0:Started 2024-04-26T21:11:38.903 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T21:11:38.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T21:11:38.905 INFO:tasks.ceph.osd.1:Started 2024-04-26T21:11:38.905 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T21:11:38.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T21:11:38.907 INFO:tasks.ceph.osd.2:Started 2024-04-26T21:11:38.907 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T21:11:38.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T21:11:38.909 INFO:tasks.ceph.osd.3:Started 2024-04-26T21:11:38.909 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T21:11:38.909 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T21:11:38.911 INFO:tasks.ceph.osd.4:Started 2024-04-26T21:11:38.912 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T21:11:38.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T21:11:38.914 INFO:tasks.ceph.osd.5:Started 2024-04-26T21:11:38.914 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T21:11:38.914 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T21:11:38.916 INFO:tasks.ceph.osd.6:Started 2024-04-26T21:11:38.916 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T21:11:38.916 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T21:11:38.918 INFO:tasks.ceph.osd.7:Started 2024-04-26T21:11:38.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.037+0000 7fa620b51640 1 Processor -- start 2024-04-26T21:11:39.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.037+0000 7fa620b51640 1 -- start start 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa620b51640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa61c15f340 0x7fa61c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa620b51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa620b51640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa61c0b2510 con 0x7fa61c0d4f00 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa620b51640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c0b27f0 con 0x7fa61c15f340 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa620b51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c0b2ad0 con 0x7fa61c15f720 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa61ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa61c0d4f00 0x7fa61c0d5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59188/0 (socket says 172.21.15.40:59188) 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa61ad9d640 1 -- 172.21.15.40:0/533665292 learned_addr learned my addr 172.21.15.40:0/533665292 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa61a59c640 1 --2- 172.21.15.40:0/533665292 >> v2:172.21.15.184:3300/0 conn(0x7fa61c15f340 0x7fa61c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa613fff640 1 --2- 172.21.15.40:0/533665292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa613fff640 1 -- 172.21.15.40:0/533665292 >> v2:172.21.15.184:3300/0 conn(0x7fa61c15f340 msgr2=0x7fa61c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:39.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa613fff640 1 --2- 172.21.15.40:0/533665292 >> v2:172.21.15.184:3300/0 conn(0x7fa61c15f340 0x7fa61c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa619d9b640 1 -- 172.21.15.40:0/533665292 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276733207 0 0) 0x7fa61c0b2510 con 0x7fa61c0d4f00 2024-04-26T21:11:39.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.038+0000 7fa613fff640 1 -- 172.21.15.40:0/533665292 >> v1:172.21.15.40:6789/0 conn(0x7fa61c0d4f00 legacy=0x7fa61c0d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.039+0000 7fa613fff640 1 -- 172.21.15.40:0/533665292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c0b2db0 con 0x7fa61c15f720 2024-04-26T21:11:39.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.040+0000 7fa613fff640 1 --2- 172.21.15.40:0/533665292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c0d3d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8ebac9c8d91711 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:39.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.040+0000 7fa619d9b640 1 -- 172.21.15.40:0/533665292 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604037070 con 0x7fa61c15f720 2024-04-26T21:11:39.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.040+0000 7fa619d9b640 1 -- 172.21.15.40:0/533665292 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa60402ae50 con 0x7fa61c15f720 2024-04-26T21:11:39.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.040+0000 7fa619d9b640 1 -- 172.21.15.40:0/533665292 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604032660 con 0x7fa61c15f720 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 -- 172.21.15.40:0/533665292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 msgr2=0x7fa61c0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 --2- 172.21.15.40:0/533665292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c0d3d40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402a100 comp rx=0 tx=0).stop 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 -- 172.21.15.40:0/533665292 shutdown_connections 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 --2- 172.21.15.40:0/533665292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c0d3d40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 --2- 172.21.15.40:0/533665292 >> v2:172.21.15.184:3300/0 conn(0x7fa61c15f340 0x7fa61c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 -- 172.21.15.40:0/533665292 >> 172.21.15.40:0/533665292 conn(0x7fa61c08ec70 msgr2=0x7fa61c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 -- 172.21.15.40:0/533665292 shutdown_connections 2024-04-26T21:11:39.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.041+0000 7fa620b51640 1 -- 172.21.15.40:0/533665292 wait complete. 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.042+0000 7fa620b51640 1 Processor -- start 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.042+0000 7fa620b51640 1 -- start start 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 0x7fa61c0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0b3980 con 0x7fa61c15f340 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0b3800 con 0x7fa61c0d4f00 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa620b51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c0b3b00 con 0x7fa61c15f720 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa613fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 0x7fa61c0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa613fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60232/0 (socket says 172.21.15.40:60232) 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa613fff640 1 -- 172.21.15.40:0/2898916678 learned_addr learned my addr 172.21.15.40:0/2898916678 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 -- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 msgr2=0x7fa61c0b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 0x7fa61c0b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 -- 172.21.15.40:0/2898916678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 msgr2=0x7fa61c150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61ad9d640 1 -- 172.21.15.40:0/2898916678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c151400 con 0x7fa61c15f720 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.043+0000 7fa61a59c640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 0x7fa61c0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa613fff640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa61ad9d640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c06da30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c6d0 tx=0x7fa61400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614018070 con 0x7fa61c15f720 2024-04-26T21:11:39.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa614004d60 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400a880 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c0ca220 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c0ca7e0 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa61400ac30 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.044+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa6140077a0 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.045+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c06a290 con 0x7fa61c15f720 2024-04-26T21:11:39.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.047+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa6140070d0 con 0x7fa61c15f720 2024-04-26T21:11:39.100 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.099+0000 7f9731517040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:11:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.110+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa61c0d4640 con 0x7fa61c15f720 2024-04-26T21:11:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.110+0000 7fa612ffd640 1 -- 172.21.15.40:0/2898916678 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa6140072b0 con 0x7fa61c15f720 2024-04-26T21:11:39.111 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:39.112 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:38.885359+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 msgr2=0x7fa61c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c06da30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c6d0 tx=0x7fa61400cb90 comp rx=0 tx=0).stop 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 shutdown_connections 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c15f720 0x7fa61c06da30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c15f340 0x7fa61c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 --2- 172.21.15.40:0/2898916678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0d4f00 0x7fa61c0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 >> 172.21.15.40:0/2898916678 conn(0x7fa61c08ec70 msgr2=0x7fa61c0d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 shutdown_connections 2024-04-26T21:11:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:39.111+0000 7fa620b51640 1 -- 172.21.15.40:0/2898916678 wait complete. 2024-04-26T21:11:39.123 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T21:11:39.123 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T21:11:39.176 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.175+0000 7f92e2a2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:11:39.271 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:11:39.270+0000 7f65a423f640 -1 Falling back to public interface 2024-04-26T21:11:39.356 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.355+0000 7f9731517040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:11:39.408 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.407+0000 7f9731517040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:11:39.448 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.448+0000 7f92e2a2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:11:39.503 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.502+0000 7f92e2a2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:11:39.517 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:11:39.516+0000 7f1b4ec11640 -1 Falling back to public interface 2024-04-26T21:11:39.517 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:11:39.517+0000 7f7c39133640 -1 Falling back to public interface 2024-04-26T21:11:39.527 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.526+0000 7f9731517040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:11:39.530 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:11:39.529+0000 7f3da000d640 -1 Falling back to public interface 2024-04-26T21:11:39.530 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:11:39.529+0000 7f2dd9c2f640 -1 Falling back to public interface 2024-04-26T21:11:39.534 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:11:39.533+0000 7fd49221c640 -1 Falling back to public interface 2024-04-26T21:11:39.535 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:11:39.534+0000 7f3cea2ea640 -1 Falling back to public interface 2024-04-26T21:11:39.540 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:11:39.540+0000 7fcf92b42640 -1 Falling back to public interface 2024-04-26T21:11:39.630 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.629+0000 7f92e2a2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:11:39.809 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.808+0000 7f9731517040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:11:39.857 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:39.856+0000 7f9731517040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:11:39.926 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.925+0000 7f92e2a2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:11:39.928 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:39.977 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:39.976+0000 7f92e2a2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:11:40.010 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.009+0000 7f9731517040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:11:40.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.020+0000 7fc63ad33640 1 Processor -- start 2024-04-26T21:11:40.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.020+0000 7fc63ad33640 1 -- start start 2024-04-26T21:11:40.022 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc63ad33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc63415f5e0 0x7fc6340d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc63ad33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc63409b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc63ad33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc634070670 con 0x7fc6340b55d0 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc63ad33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc6340d1bd0 con 0x7fc63415f5e0 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc63ad33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6340d1580 con 0x7fc63409c710 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc638acf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc63415f5e0 0x7fc6340d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc633fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc63409b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc6392d0640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc6340b55d0 0x7fc63406d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57112/0 (socket says 172.21.15.40:57112) 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc6392d0640 1 -- 172.21.15.40:0/2621590050 learned_addr learned my addr 172.21.15.40:0/2621590050 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc638acf640 1 -- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 msgr2=0x7fc63409b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc638acf640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc63409b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc638acf640 1 -- 172.21.15.40:0/2621590050 >> v1:172.21.15.184:6789/0 conn(0x7fc6340b55d0 legacy=0x7fc63406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc638acf640 1 -- 172.21.15.40:0/2621590050 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6340d1860 con 0x7fc63415f5e0 2024-04-26T21:11:40.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.021+0000 7fc633fff640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc63409b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.022+0000 7fc638acf640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340d2060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097a0 tx=0x7fc62802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1e8b8b303cbc06c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.022+0000 7fc6337fe640 1 -- 172.21.15.40:0/2621590050 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628037070 con 0x7fc63415f5e0 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.022+0000 7fc6337fe640 1 -- 172.21.15.40:0/2621590050 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62802ae50 con 0x7fc63415f5e0 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.022+0000 7fc6337fe640 1 -- 172.21.15.40:0/2621590050 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628032660 con 0x7fc63415f5e0 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 -- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 msgr2=0x7fc6340d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340d2060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097a0 tx=0x7fc62802a100 comp rx=0 tx=0).stop 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 -- 172.21.15.40:0/2621590050 shutdown_connections 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc63409b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 --2- 172.21.15.40:0/2621590050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340d2060 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 -- 172.21.15.40:0/2621590050 >> 172.21.15.40:0/2621590050 conn(0x7fc6340b4ab0 msgr2=0x7fc6340b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 -- 172.21.15.40:0/2621590050 shutdown_connections 2024-04-26T21:11:40.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.023+0000 7fc63ad33640 1 -- 172.21.15.40:0/2621590050 wait complete. 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.024+0000 7fc63ad33640 1 Processor -- start 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.024+0000 7fc63ad33640 1 -- start start 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc6340b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340d1230 con 0x7fc6340b55d0 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340d25a0 con 0x7fc63415f5e0 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc63ad33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc634070290 con 0x7fc63409c710 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc633fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc633fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60244/0 (socket says 172.21.15.40:60244) 2024-04-26T21:11:40.028 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc633fff640 1 -- 172.21.15.40:0/223995458 learned_addr learned my addr 172.21.15.40:0/223995458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc6392d0640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.025+0000 7fc638acf640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc6340b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 -- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 msgr2=0x7fc6340b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc6340b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 -- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 msgr2=0x7fc6340b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 -- 172.21.15.40:0/223995458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341f6140 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc633fff640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc62000e990 tx=0x7fc62000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620017070 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62000c9e0 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6340b8950 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62000cdb0 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6340b5bc0 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc62001a070 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.026+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc620013b40 con 0x7fc6340b55d0 2024-04-26T21:11:40.029 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.027+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc634076870 con 0x7fc6340b55d0 2024-04-26T21:11:40.030 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.029+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc620013de0 con 0x7fc6340b55d0 2024-04-26T21:11:40.093 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.092+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6340b8ee0 con 0x7fc6340b55d0 2024-04-26T21:11:40.093 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.092+0000 7fc6317fa640 1 -- 172.21.15.40:0/223995458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc6200064d0 con 0x7fc6340b55d0 2024-04-26T21:11:40.093 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:11:40.093 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:38.885359+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:40.094 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 msgr2=0x7fc6340b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc62000e990 tx=0x7fc62000ee50 comp rx=0 tx=0).stop 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 shutdown_connections 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63415f5e0 0x7fc6340b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6340b55d0 0x7fc6340b8410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 --2- 172.21.15.40:0/223995458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc63409c710 0x7fc6340b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 >> 172.21.15.40:0/223995458 conn(0x7fc6340b4ab0 msgr2=0x7fc6340b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 shutdown_connections 2024-04-26T21:11:40.095 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:40.093+0000 7fc63ad33640 1 -- 172.21.15.40:0/223995458 wait complete. 2024-04-26T21:11:40.103 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:11:40.146 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.145+0000 7f92e2a2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:11:40.147 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.146+0000 7f9731517040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:11:40.196 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.195+0000 7f9731517040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:11:40.264 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.263+0000 7f9731517040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:11:40.290 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.289+0000 7f92e2a2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:11:40.309 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.307+0000 7f9731517040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:11:40.342 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.341+0000 7f92e2a2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:11:40.416 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.416+0000 7f92e2a2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:11:40.464 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.463+0000 7f92e2a2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:11:40.785 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.784+0000 7f9731517040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:11:40.903 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:40.902+0000 7f9731517040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:11:40.971 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:40.970+0000 7f92e2a2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:11:41.092 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-26T21:11:41.091+0000 7f92e2a2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:11:41.922 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:41.921+0000 7f96d20fb640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:11:41.922 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-04-26T21:11:41.922+0000 7f96d20fb640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:11:42.766 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:11:42.765+0000 7f65a423f640 -1 osd.1 0 log_to_monitors true 2024-04-26T21:11:42.985 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:11:42.985+0000 7f2dd9c2f640 -1 osd.3 0 log_to_monitors true 2024-04-26T21:11:43.004 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:11:43.003+0000 7f1b4ec11640 -1 osd.5 0 log_to_monitors true 2024-04-26T21:11:43.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:11:43.051+0000 7fd49221c640 -1 osd.0 0 log_to_monitors true 2024-04-26T21:11:43.058 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:11:43.057+0000 7fcf92b42640 -1 osd.2 0 log_to_monitors true 2024-04-26T21:11:43.069 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:11:43.068+0000 7f7c39133640 -1 osd.7 0 log_to_monitors true 2024-04-26T21:11:43.069 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:11:43.068+0000 7f3da000d640 -1 osd.4 0 log_to_monitors true 2024-04-26T21:11:43.070 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:11:43.069+0000 7f3cea2ea640 -1 osd.6 0 log_to_monitors true 2024-04-26T21:11:44.809 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:11:44.809+0000 7f65a01d5640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:11:44.819 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-26T21:11:44.819+0000 7f6597ff5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:46.602 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:11:46.601+0000 7f2dd5180640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:11:46.603 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:11:46.601+0000 7fcf8ead8640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:11:46.613 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-26T21:11:46.613+0000 7fcf868f8640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:46.614 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-26T21:11:46.613+0000 7f2dccfb1640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:46.629 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:11:46.628+0000 7fd48dba0640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:11:46.638 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-26T21:11:46.638+0000 7fd4859c0640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:46.909 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:46.954 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:11:46.953+0000 7f1b4a524640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:11:46.967 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-04-26T21:11:46.967+0000 7f1b42344640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:46.995 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:11:46.994+0000 7f3d9c213640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:11:47.005 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.003+0000 7ff448493640 1 Processor -- start 2024-04-26T21:11:47.005 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.003+0000 7ff448493640 1 -- start start 2024-04-26T21:11:47.007 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff448493640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff44015f480 0x7ff4400cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.007 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff448493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 0x7ff4400d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.007 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff448493640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4400d0800 con 0x7ff4400d3c90 2024-04-26T21:11:47.007 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff448493640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff4400b4b40 con 0x7ff44015f480 2024-04-26T21:11:47.007 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff448493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4400d0ae0 con 0x7ff4400d01c0 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff445a2e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 0x7ff4400d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44622f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff44015f480 0x7ff4400cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff446a30640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff4400d3c90 0x7ff4400d1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44798/0 (socket says 172.21.15.40:44798) 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff446a30640 1 -- 172.21.15.40:0/263500881 learned_addr learned my addr 172.21.15.40:0/263500881 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44522d640 1 -- 172.21.15.40:0/263500881 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042148036 0 0) 0x7ff4400d0800 con 0x7ff4400d3c90 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44522d640 1 -- 172.21.15.40:0/263500881 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff42c003520 con 0x7ff4400d3c90 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44622f640 1 -- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 msgr2=0x7ff4400d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44622f640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 0x7ff4400d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44622f640 1 -- 172.21.15.40:0/263500881 >> v1:172.21.15.184:6789/0 conn(0x7ff4400d3c90 legacy=0x7ff4400d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff44622f640 1 -- 172.21.15.40:0/263500881 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4400d0dc0 con 0x7ff44015f480 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.004+0000 7ff445a2e640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 0x7ff4400d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.005+0000 7ff44622f640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44015f480 0x7ff4400cfc80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e295d577ab8f9599 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.005+0000 7ff44522d640 1 -- 172.21.15.40:0/263500881 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c037070 con 0x7ff44015f480 2024-04-26T21:11:47.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.005+0000 7ff44522d640 1 -- 172.21.15.40:0/263500881 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43c032440 con 0x7ff44015f480 2024-04-26T21:11:47.009 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.005+0000 7ff44522d640 1 -- 172.21.15.40:0/263500881 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0327f0 con 0x7ff44015f480 2024-04-26T21:11:47.009 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-04-26T21:11:47.007+0000 7f3d9381f640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 -- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44015f480 msgr2=0x7ff4400cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44015f480 0x7ff4400cfc80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02a180 comp rx=0 tx=0).stop 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 -- 172.21.15.40:0/263500881 shutdown_connections 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d01c0 0x7ff4400d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 --2- 172.21.15.40:0/263500881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44015f480 0x7ff4400cfc80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 -- 172.21.15.40:0/263500881 >> 172.21.15.40:0/263500881 conn(0x7ff44009e4b0 msgr2=0x7ff4400d5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 -- 172.21.15.40:0/263500881 shutdown_connections 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.006+0000 7ff448493640 1 -- 172.21.15.40:0/263500881 wait complete. 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.007+0000 7ff448493640 1 Processor -- start 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.007+0000 7ff448493640 1 -- start start 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4400d01c0 0x7ff4400b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d3c90 0x7ff4400c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.012 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4400b4690 con 0x7ff44015f480 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff44015f860 con 0x7ff4400d01c0 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff448493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4400b4810 con 0x7ff4400d3c90 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff44622f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4400d01c0 0x7ff4400b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50576/0 (socket says 172.21.15.40:50576) 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 -- 172.21.15.40:0/3383294924 learned_addr learned my addr 172.21.15.40:0/3383294924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff445a2e640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d3c90 0x7ff4400c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 -- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d3c90 msgr2=0x7ff4400c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d3c90 0x7ff4400c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 -- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4400d01c0 msgr2=0x7ff4400b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4400d01c0 0x7ff4400b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.013 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 -- 172.21.15.40:0/3383294924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4400b9bb0 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff446a30640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43800cb40 tx=0x7ff43800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.008+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.009+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43800aca0 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.009+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438012620 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.009+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4400b9e90 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.009+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4400ba450 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.010+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4401613b0 con 0x7ff44015f480 2024-04-26T21:11:47.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.010+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff43801a070 con 0x7ff44015f480 2024-04-26T21:11:47.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.010+0000 7ff436ffd640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 0x7ff42406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:47.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.010+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7ff43808abb0 con 0x7ff44015f480 2024-04-26T21:11:47.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.010+0000 7ff44622f640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 0x7ff42406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:47.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.011+0000 7ff44622f640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 0x7ff42406ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:47.017 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.015+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4380596f0 con 0x7ff44015f480 2024-04-26T21:11:47.046 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:11:47.045+0000 7f7c350c9640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:11:47.062 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-04-26T21:11:47.059+0000 7f7c2cee9640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:47.072 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:11:47.072+0000 7f3ce6280640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:11:47.092 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-04-26T21:11:47.092+0000 7f3cde0a0640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:11:47.204 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.202+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff440076870 con 0x7ff44015f480 2024-04-26T21:11:47.204 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.202+0000 7ff436ffd640 1 -- 172.21.15.40:0/3383294924 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7578 (secure 0 0 0) 0x7ff43805d9f0 con 0x7ff44015f480 2024-04-26T21:11:47.205 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:11:47.205 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":14,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:46.172859+0000","last_up_change":"2024-04-26T21:11:44.921763+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.204+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 msgr2=0x7ff42406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.204+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 0x7ff42406ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c02e000 comp rx=0 tx=0).stop 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.204+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 msgr2=0x7ff4400ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.204+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43800cb40 tx=0x7ff43800a060 comp rx=0 tx=0).stop 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 shutdown_connections 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff42406c890 0x7ff42406ed50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44015f480 0x7ff4400ca450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.207 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4400d3c90 0x7ff4400c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.208 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 --2- 172.21.15.40:0/3383294924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4400d01c0 0x7ff4400b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:47.208 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 >> 172.21.15.40:0/3383294924 conn(0x7ff44009e4b0 msgr2=0x7ff4400b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:47.208 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 shutdown_connections 2024-04-26T21:11:47.208 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:47.205+0000 7ff448493640 1 -- 172.21.15.40:0/3383294924 wait complete. 2024-04-26T21:11:47.216 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-26T21:11:54.023 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:54.127 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.124+0000 7fbb2c4c5640 1 Processor -- start 2024-04-26T21:11:54.127 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.125+0000 7fbb2c4c5640 1 -- start start 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2c4c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb240d1b30 0x7fbb240d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2c4c5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb240ba4a0 0x7fbb240ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2c4c5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb240d31a0 con 0x7fbb2415f470 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2c4c5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbb240d24d0 con 0x7fbb240d1b30 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2c4c5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbb240d3db0 con 0x7fbb240ba4a0 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb29a60640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb240d1b30 0x7fbb240d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.128 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2a261640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb2415f470 0x7fbb240d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45628/0 (socket says 172.21.15.40:45628) 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2a261640 1 -- 172.21.15.40:0/1503827739 learned_addr learned my addr 172.21.15.40:0/1503827739 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb2aa62640 1 --2- 172.21.15.40:0/1503827739 >> v2:172.21.15.184:3300/0 conn(0x7fbb240ba4a0 0x7fbb240ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb29a60640 1 -- 172.21.15.40:0/1503827739 >> v2:172.21.15.184:3300/0 conn(0x7fbb240ba4a0 msgr2=0x7fbb240ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb29a60640 1 --2- 172.21.15.40:0/1503827739 >> v2:172.21.15.184:3300/0 conn(0x7fbb240ba4a0 0x7fbb240ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb29a60640 1 -- 172.21.15.40:0/1503827739 >> v1:172.21.15.40:6789/0 conn(0x7fbb2415f470 legacy=0x7fbb240d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.129 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.126+0000 7fbb29a60640 1 -- 172.21.15.40:0/1503827739 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240d3480 con 0x7fbb240d1b30 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb29a60640 1 --2- 172.21.15.40:0/1503827739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb240d1b30 0x7fbb240d1f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb2409e9d0 tx=0x7fbb1402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23871a5d678bdd9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb2925f640 1 -- 172.21.15.40:0/1503827739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14037070 con 0x7fbb240d1b30 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb2925f640 1 -- 172.21.15.40:0/1503827739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb14032440 con 0x7fbb240d1b30 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb2925f640 1 -- 172.21.15.40:0/1503827739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14032810 con 0x7fbb240d1b30 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/1503827739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb240d1b30 msgr2=0x7fbb240d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.130 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.127+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/1503827739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb240d1b30 0x7fbb240d1f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb2409e9d0 tx=0x7fbb1402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/1503827739 shutdown_connections 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/1503827739 >> v2:172.21.15.184:3300/0 conn(0x7fbb240ba4a0 0x7fbb240ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/1503827739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb240d1b30 0x7fbb240d1f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/1503827739 >> 172.21.15.40:0/1503827739 conn(0x7fbb2409e530 msgr2=0x7fbb240d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/1503827739 shutdown_connections 2024-04-26T21:11:54.131 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.128+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/1503827739 wait complete. 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.129+0000 7fbb2c4c5640 1 Processor -- start 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.129+0000 7fbb2c4c5640 1 -- start start 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 0x7fbb241f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb240d1b30 0x7fbb240b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240d4240 con 0x7fbb2415f470 2024-04-26T21:11:54.133 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240d3a40 con 0x7fbb240d1b30 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2415f850 con 0x7fbb240ba4a0 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43010/0 (socket says 172.21.15.40:43010) 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 -- 172.21.15.40:0/4168834396 learned_addr learned my addr 172.21.15.40:0/4168834396 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb29a60640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb240d1b30 0x7fbb240b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2a261640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 0x7fbb241f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 -- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 msgr2=0x7fbb241f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 0x7fbb241f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 -- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb240d1b30 msgr2=0x7fbb240b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb240d1b30 0x7fbb240b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 -- 172.21.15.40:0/4168834396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241f8f80 con 0x7fbb2415f470 2024-04-26T21:11:54.134 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2a261640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 0x7fbb241f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2aa62640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007e60 tx=0x7fbb1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c014420 con 0x7fbb2415f470 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c019070 con 0x7fbb2415f470 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241f9260 con 0x7fbb2415f470 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c014820 con 0x7fbb2415f470 2024-04-26T21:11:54.135 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.130+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241f9820 con 0x7fbb2415f470 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.132+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb24076870 con 0x7fbb2415f470 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.132+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb1c0149c0 con 0x7fbb2415f470 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.132+0000 7fbb1affd640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.132+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fbb1c08c060 con 0x7fbb2415f470 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.133+0000 7fbb2a261640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.136 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.133+0000 7fbb2a261640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 0x7fbb0806edb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f60 tx=0x7fbb20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.139 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.137+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb1c05a7e0 con 0x7fbb2415f470 2024-04-26T21:11:54.325 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.323+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb2406ee70 con 0x7fbb2415f470 2024-04-26T21:11:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.324+0000 7fbb1affd640 1 -- 172.21.15.40:0/4168834396 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7fbb1c05a0e0 con 0x7fbb2415f470 2024-04-26T21:11:54.326 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-26T21:11:54.326 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":19,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:51.648424+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 msgr2=0x7fbb0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 0x7fbb0806edb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f60 tx=0x7fbb20009040 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 msgr2=0x7fbb241617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007e60 tx=0x7fbb1c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 shutdown_connections 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2415f470 0x7fbb241617b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb240d1b30 0x7fbb240b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 --2- 172.21.15.40:0/4168834396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb240ba4a0 0x7fbb241f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 >> 172.21.15.40:0/4168834396 conn(0x7fbb2409e530 msgr2=0x7fbb240b8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 shutdown_connections 2024-04-26T21:11:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-04-26T21:11:54.326+0000 7fbb2c4c5640 1 -- 172.21.15.40:0/4168834396 wait complete. 2024-04-26T21:11:54.338 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T21:11:54.338 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T21:11:54.338 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:11:54.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:11:54.341 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:11:54.341 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:11:54.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:11:54.343 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:11:54.343 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:11:54.343 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:11:54.346 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:11:54.346 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:11:54.346 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:11:54.348 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:11:54.349 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T21:11:54.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:54.416 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:11:54.417 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:11:54.430 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:11:54.430 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:11:54.430 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:11:54.429+0000 7fddedc06640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:11:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.452+0000 7f2138f2b640 1 Processor -- start 2024-04-26T21:11:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.452+0000 7f2138f2b640 1 -- start start 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21341489a0 0x7f21340ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2134071900 0x7f213406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21340aedf0 0x7f21340d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21340cfcb0 con 0x7f21341489a0 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21340cff90 con 0x7f21340aedf0 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2138f2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21340d0270 con 0x7f2134071900 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21341489a0 0x7f21340ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21341489a0 0x7f21340ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43034/0 (socket says 172.21.15.40:43034) 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 -- 172.21.15.40:0/3533290770 learned_addr learned my addr 172.21.15.40:0/3533290770 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f212bfff640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2134071900 0x7f213406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f2132d9d640 1 --2- 172.21.15.40:0/3533290770 >> v2:172.21.15.184:3300/0 conn(0x7f21340aedf0 0x7f21340d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 -- 172.21.15.40:0/3533290770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2134071900 msgr2=0x7f213406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2134071900 0x7f213406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 -- 172.21.15.40:0/3533290770 >> v2:172.21.15.184:3300/0 conn(0x7f21340aedf0 msgr2=0x7f21340d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 --2- 172.21.15.40:0/3533290770 >> v2:172.21.15.184:3300/0 conn(0x7f21340aedf0 0x7f21340d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.453+0000 7f213259c640 1 -- 172.21.15.40:0/3533290770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21340d0550 con 0x7f21341489a0 2024-04-26T21:11:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f213259c640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21341489a0 0x7f21340ae8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21340d1a00 tx=0x7f212402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6cd7f815fc1b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f2131d9b640 1 -- 172.21.15.40:0/3533290770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124037070 con 0x7f21341489a0 2024-04-26T21:11:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f2131d9b640 1 -- 172.21.15.40:0/3533290770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2124032440 con 0x7f21341489a0 2024-04-26T21:11:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f2131d9b640 1 -- 172.21.15.40:0/3533290770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124032810 con 0x7f21341489a0 2024-04-26T21:11:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f2138f2b640 1 -- 172.21.15.40:0/3533290770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21341489a0 msgr2=0x7f21340ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.454+0000 7f2138f2b640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21341489a0 0x7f21340ae8b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21340d1a00 tx=0x7f212402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 -- 172.21.15.40:0/3533290770 shutdown_connections 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 --2- 172.21.15.40:0/3533290770 >> v2:172.21.15.184:3300/0 conn(0x7f21340aedf0 0x7f21340d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2134071900 0x7f213406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 --2- 172.21.15.40:0/3533290770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21341489a0 0x7f21340ae8b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 -- 172.21.15.40:0/3533290770 >> 172.21.15.40:0/3533290770 conn(0x7f21340d1e30 msgr2=0x7f21340d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 -- 172.21.15.40:0/3533290770 shutdown_connections 2024-04-26T21:11:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.455+0000 7f2138f2b640 1 -- 172.21.15.40:0/3533290770 wait complete. 2024-04-26T21:11:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.456+0000 7f2138f2b640 1 Processor -- start 2024-04-26T21:11:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.456+0000 7f2138f2b640 1 -- start start 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21341489a0 0x7f21340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 0x7f21340b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f213406ff70 con 0x7f21340aedf0 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2134148d80 con 0x7f213413e510 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21340700f0 con 0x7f21341489a0 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43040/0 (socket says 172.21.15.40:43040) 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 -- 172.21.15.40:0/1352365944 learned_addr learned my addr 172.21.15.40:0/1352365944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2132d9d640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 0x7f21340b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f212bfff640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21341489a0 0x7f21340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 -- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21341489a0 msgr2=0x7f21340b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21341489a0 0x7f21340b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 -- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 msgr2=0x7f21340b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 0x7f21340b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 -- 172.21.15.40:0/1352365944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21341f6580 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2132d9d640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 0x7f21340b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f213259c640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f21240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124041020 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212403d620 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21341f69a0 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124037040 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.457+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341f6f20 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.459+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21340693d0 con 0x7f21340aedf0 2024-04-26T21:11:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.459+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2124044020 con 0x7f21340aedf0 2024-04-26T21:11:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.459+0000 7f212affd640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.459+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f21240abe10 con 0x7f21340aedf0 2024-04-26T21:11:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.460+0000 7f212bfff640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.460+0000 7f212bfff640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 0x7f211406ede0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21340b6c10 tx=0x7f211c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.464+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f212404b030 con 0x7f21340aedf0 2024-04-26T21:11:54.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.647+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21340d0f70 con 0x7f21340aedf0 2024-04-26T21:11:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.648+0000 7f212affd640 1 -- 172.21.15.40:0/1352365944 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7f212407a620 con 0x7f21340aedf0 2024-04-26T21:11:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:54.650 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:51.648424+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 msgr2=0x7f211406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 0x7f211406ede0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21340b6c10 tx=0x7f211c008040 comp rx=0 tx=0).stop 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 msgr2=0x7f213413ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f21240046c0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 shutdown_connections 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f211406c920 0x7f211406ede0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213413e510 0x7f21340b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21341489a0 0x7f21340b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 --2- 172.21.15.40:0/1352365944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21340aedf0 0x7f213413ddf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 >> 172.21.15.40:0/1352365944 conn(0x7f21340d1e30 msgr2=0x7f213406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 shutdown_connections 2024-04-26T21:11:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.650+0000 7f2138f2b640 1 -- 172.21.15.40:0/1352365944 wait complete. 2024-04-26T21:11:54.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:54.663 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.764+0000 7f0a04e59640 1 Processor -- start 2024-04-26T21:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.764+0000 7f0a04e59640 1 -- start start 2024-04-26T21:11:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f0a04e59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0a000d1be0 0x7f0a000d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f0a04e59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a000ba3a0 0x7f0a000ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f0a04e59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a000d4240 con 0x7f0a0015f490 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f0a04e59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0a000d31a0 con 0x7f0a000ba3a0 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f0a04e59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0a000d3480 con 0x7f0a000d1be0 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a000ba3a0 0x7f0a000ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fe59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0a0015f490 0x7f0a000d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45638/0 (socket says 172.21.15.40:45638) 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a000ba3a0 0x7f0a000ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43048/0 (socket says 172.21.15.40:43048) 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fe59c640 1 -- 172.21.15.40:0/225679883 learned_addr learned my addr 172.21.15.40:0/225679883 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fdd9b640 1 --2- 172.21.15.40:0/225679883 >> v2:172.21.15.184:3300/0 conn(0x7f0a000d1be0 0x7f0a000d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 -- 172.21.15.40:0/225679883 >> v2:172.21.15.184:3300/0 conn(0x7f0a000d1be0 msgr2=0x7f0a000d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 --2- 172.21.15.40:0/225679883 >> v2:172.21.15.184:3300/0 conn(0x7f0a000d1be0 0x7f0a000d2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 -- 172.21.15.40:0/225679883 >> v1:172.21.15.40:6789/0 conn(0x7f0a0015f490 legacy=0x7f0a000d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fed9d640 1 -- 172.21.15.40:0/225679883 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a000d3760 con 0x7f0a000ba3a0 2024-04-26T21:11:54.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.765+0000 7f09fd59a640 1 -- 172.21.15.40:0/225679883 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069831144 0 0) 0x7f0a000d4240 con 0x7f0a0015f490 2024-04-26T21:11:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.766+0000 7f09fed9d640 1 --2- 172.21.15.40:0/225679883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a000ba7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a0008eb10 tx=0x7f09f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d1500d547875df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.766+0000 7f09fd59a640 1 -- 172.21.15.40:0/225679883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8037070 con 0x7f0a000ba3a0 2024-04-26T21:11:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.766+0000 7f09fd59a640 1 -- 172.21.15.40:0/225679883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f8032440 con 0x7f0a000ba3a0 2024-04-26T21:11:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.766+0000 7f09fd59a640 1 -- 172.21.15.40:0/225679883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8032810 con 0x7f0a000ba3a0 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 -- 172.21.15.40:0/225679883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 msgr2=0x7f0a000ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 --2- 172.21.15.40:0/225679883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a000ba7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a0008eb10 tx=0x7f09f802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 -- 172.21.15.40:0/225679883 shutdown_connections 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 --2- 172.21.15.40:0/225679883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a000ba7b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 --2- 172.21.15.40:0/225679883 >> v2:172.21.15.184:3300/0 conn(0x7f0a000d1be0 0x7f0a000d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 -- 172.21.15.40:0/225679883 >> 172.21.15.40:0/225679883 conn(0x7f0a0009e530 msgr2=0x7f0a000d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 -- 172.21.15.40:0/225679883 shutdown_connections 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.767+0000 7f0a04e59640 1 -- 172.21.15.40:0/225679883 wait complete. 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.768+0000 7f0a04e59640 1 Processor -- start 2024-04-26T21:11:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.768+0000 7f0a04e59640 1 -- start start 2024-04-26T21:11:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a000d1be0 0x7f0a00161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a0015f490 0x7f0a000c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0a000d11c0 con 0x7f0a000ba3a0 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a000d3d20 con 0x7f0a000d1be0 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f0a04e59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0015f870 con 0x7f0a0015f490 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43060/0 (socket says 172.21.15.40:43060) 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 -- 172.21.15.40:0/2568265377 learned_addr learned my addr 172.21.15.40:0/2568265377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fdd9b640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a000d1be0 0x7f0a00161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 -- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a0015f490 msgr2=0x7f0a000c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a0015f490 0x7f0a000c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 -- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a000d1be0 msgr2=0x7f0a00161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a000d1be0 0x7f0a00161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 -- 172.21.15.40:0/2568265377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a00161f90 con 0x7f0a000ba3a0 2024-04-26T21:11:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.769+0000 7f09fe59c640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c890 tx=0x7f09f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.770+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0018070 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.770+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f000a640 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.770+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a00162270 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.770+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f000aa30 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.770+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0015bc70 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.771+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0006a290 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.771+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09f001b070 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.771+0000 7f09f6ffd640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 0x7f09e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.771+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f09f008af80 con 0x7f0a000ba3a0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.772+0000 7f09fdd9b640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 0x7f09e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.772+0000 7f09fdd9b640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 0x7f09e006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a0006d370 tx=0x7f09e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.776+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09f0059670 con 0x7f0a000ba3a0 2024-04-26T21:11:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.960+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a0015ec90 con 0x7f0a000ba3a0 2024-04-26T21:11:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.960+0000 7f09f6ffd640 1 -- 172.21.15.40:0/2568265377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f09f0058f70 con 0x7f0a000ba3a0 2024-04-26T21:11:54.962 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 msgr2=0x7f09e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 0x7f09e006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a0006d370 tx=0x7f09e8008040 comp rx=0 tx=0).stop 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 msgr2=0x7f0a0014e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c890 tx=0x7f09f000cd50 comp rx=0 tx=0).stop 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 shutdown_connections 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f09e006c8f0 0x7f09e006edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a0015f490 0x7f0a000c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a000d1be0 0x7f0a00161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 --2- 172.21.15.40:0/2568265377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a000ba3a0 0x7f0a0014e740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 >> 172.21.15.40:0/2568265377 conn(0x7f0a0009e530 msgr2=0x7f0a0006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 shutdown_connections 2024-04-26T21:11:54.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:54.962+0000 7f0a04e59640 1 -- 172.21.15.40:0/2568265377 wait complete. 2024-04-26T21:11:54.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.073+0000 7f49d3a99640 1 Processor -- start 2024-04-26T21:11:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.073+0000 7f49d3a99640 1 -- start start 2024-04-26T21:11:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d3a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f370 0x7f49cc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d3a99640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d3a99640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc0d03a0 con 0x7f49cc15f750 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d3a99640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc0b4d10 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d3a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0d0680 con 0x7f49cc15f370 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43064/0 (socket says 172.21.15.40:43064) 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 -- 172.21.15.40:0/10151865 learned_addr learned my addr 172.21.15.40:0/10151865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d1034640 1 --1- 172.21.15.40:0/10151865 >> v1:172.21.15.40:6789/0 conn(0x7f49cc15f750 0x7f49cc0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45640/0 (socket says 172.21.15.40:45640) 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 -- 172.21.15.40:0/10151865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f370 msgr2=0x7f49cc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 --2- 172.21.15.40:0/10151865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f370 0x7f49cc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 -- 172.21.15.40:0/10151865 >> v1:172.21.15.40:6789/0 conn(0x7f49cc15f750 legacy=0x7f49cc0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.074+0000 7f49d2036640 1 -- 172.21.15.40:0/10151865 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc0d0960 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.075+0000 7f49d2036640 1 --2- 172.21.15.40:0/10151865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009870 tx=0x7f49c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49330a08471fce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.075+0000 7f49d0833640 1 -- 172.21.15.40:0/10151865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4037070 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.075+0000 7f49d0833640 1 -- 172.21.15.40:0/10151865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4032440 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.075+0000 7f49d0833640 1 -- 172.21.15.40:0/10151865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4032810 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 -- 172.21.15.40:0/10151865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 msgr2=0x7f49cc0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 --2- 172.21.15.40:0/10151865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009870 tx=0x7f49c402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 -- 172.21.15.40:0/10151865 shutdown_connections 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 --2- 172.21.15.40:0/10151865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc0b47d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 --2- 172.21.15.40:0/10151865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f370 0x7f49cc0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 -- 172.21.15.40:0/10151865 >> 172.21.15.40:0/10151865 conn(0x7f49cc08ec10 msgr2=0x7f49cc09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 -- 172.21.15.40:0/10151865 shutdown_connections 2024-04-26T21:11:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.076+0000 7f49d3a99640 1 -- 172.21.15.40:0/10151865 wait complete. 2024-04-26T21:11:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.077+0000 7f49d3a99640 1 Processor -- start 2024-04-26T21:11:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.077+0000 7f49d3a99640 1 -- start start 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49cc15f370 0x7f49cc0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f750 0x7f49cc0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc0b51c0 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc0d0f20 con 0x7f49cc15f370 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0b5340 con 0x7f49cc15f750 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43066/0 (socket says 172.21.15.40:43066) 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 -- 172.21.15.40:0/861863134 learned_addr learned my addr 172.21.15.40:0/861863134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d2036640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f750 0x7f49cc0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1034640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49cc15f370 0x7f49cc0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 -- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f750 msgr2=0x7f49cc0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f750 0x7f49cc0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 -- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49cc15f370 msgr2=0x7f49cc0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49cc15f370 0x7f49cc0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 -- 172.21.15.40:0/861863134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc0b8c40 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d1835640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00ba50 tx=0x7f49bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc019070 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49bc00cc00 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc0b8f20 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc014440 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.078+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc0b94e0 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.080+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc06a290 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.080+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49bc01c070 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.080+0000 7f49ca7fc640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.080+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f49bc08b500 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.081+0000 7f49d1034640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.081+0000 7f49d1034640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f49cc15e0b0 tx=0x7f49c0007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.085+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49bc059630 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.307+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49cc0cff80 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.307+0000 7f49ca7fc640 1 -- 172.21.15.40:0/861863134 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f49bc05d930 con 0x7f49cc0d3ee0 2024-04-26T21:11:55.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:55.309 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:11:55.310 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 msgr2=0x7f49b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f49cc15e0b0 tx=0x7f49c0007e80 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 msgr2=0x7f49cc154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00ba50 tx=0x7f49bc00bf10 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 shutdown_connections 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49cc15f750 0x7f49cc0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49cc15f370 0x7f49cc0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.309+0000 7f49d3a99640 1 --2- 172.21.15.40:0/861863134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49cc0d3ee0 0x7f49cc154580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.310+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 >> 172.21.15.40:0/861863134 conn(0x7f49cc08ec10 msgr2=0x7f49cc06cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.310+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 shutdown_connections 2024-04-26T21:11:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:55.310+0000 7f49d3a99640 1 -- 172.21.15.40:0/861863134 wait complete. 2024-04-26T21:11:57.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:11:57.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.434+0000 7f9b94f22640 1 Processor -- start 2024-04-26T21:11:57.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.434+0000 7f9b94f22640 1 -- start start 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b94f22640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9b900d3ee0 0x7f9b900cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b94f22640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b900d03a0 con 0x7f9b9015f750 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b94f22640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b900d0680 con 0x7f9b9015f370 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b94f22640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9b900d0960 con 0x7f9b900d3ee0 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b8ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9b9015f750 0x7f9b900b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45650/0 (socket says 172.21.15.40:45650) 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b8ed9d640 1 -- 172.21.15.40:0/3342243516 learned_addr learned my addr 172.21.15.40:0/3342243516 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.435+0000 7f9b8dd9b640 1 --2- 172.21.15.40:0/3342243516 >> v2:172.21.15.184:3300/0 conn(0x7f9b900d3ee0 0x7f9b900cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326058734 0 0) 0x7f9b900d03a0 con 0x7f9b9015f750 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b7c003520 con 0x7f9b9015f750 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691630655 0 0) 0x7f9b900d0680 con 0x7f9b9015f370 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b900d03a0 con 0x7f9b9015f370 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8dd9b640 1 -- 172.21.15.40:0/3342243516 >> v1:172.21.15.184:6789/0 conn(0x7f9b9015f370 legacy=0x7f9b900d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8dd9b640 1 -- 172.21.15.40:0/3342243516 >> v1:172.21.15.40:6789/0 conn(0x7f9b9015f750 legacy=0x7f9b900b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4022699416 0 0) 0x7f9b7c003520 con 0x7f9b9015f750 2024-04-26T21:11:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.436+0000 7f9b8dd9b640 1 -- 172.21.15.40:0/3342243516 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900d0c40 con 0x7f9b900d3ee0 2024-04-26T21:11:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b8dd9b640 1 --2- 172.21.15.40:0/3342243516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b900d3ee0 0x7f9b900cfd70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097d0 tx=0x7f9b7802a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38cdfc8fd3f5077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78037070 con 0x7f9b900d3ee0 2024-04-26T21:11:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7802ac90 con 0x7f9b900d3ee0 2024-04-26T21:11:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b8d59a640 1 -- 172.21.15.40:0/3342243516 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800019f0 con 0x7f9b900d3ee0 2024-04-26T21:11:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b94f22640 1 -- 172.21.15.40:0/3342243516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b900d3ee0 msgr2=0x7f9b900cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.437+0000 7f9b94f22640 1 --2- 172.21.15.40:0/3342243516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b900d3ee0 0x7f9b900cfd70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097d0 tx=0x7f9b7802a200 comp rx=0 tx=0).stop 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.438+0000 7f9b94f22640 1 -- 172.21.15.40:0/3342243516 shutdown_connections 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.438+0000 7f9b94f22640 1 --2- 172.21.15.40:0/3342243516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b900d3ee0 0x7f9b900cfd70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.438+0000 7f9b94f22640 1 -- 172.21.15.40:0/3342243516 >> 172.21.15.40:0/3342243516 conn(0x7f9b9008ec10 msgr2=0x7f9b9009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.438+0000 7f9b94f22640 1 -- 172.21.15.40:0/3342243516 shutdown_connections 2024-04-26T21:11:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.438+0000 7f9b94f22640 1 -- 172.21.15.40:0/3342243516 wait complete. 2024-04-26T21:11:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.439+0000 7f9b94f22640 1 Processor -- start 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.439+0000 7f9b94f22640 1 -- start start 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9015f370 0x7f9b900b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9015f750 0x7f9b900cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900b4950 con 0x7f9b900d3ee0 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900b47d0 con 0x7f9b9015f750 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b94f22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900b4ad0 con 0x7f9b9015f370 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43072/0 (socket says 172.21.15.40:43072) 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 -- 172.21.15.40:0/1970564475 learned_addr learned my addr 172.21.15.40:0/1970564475 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8ed9d640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9015f750 0x7f9b900cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8dd9b640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9015f370 0x7f9b900b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 -- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9015f370 msgr2=0x7f9b900b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9015f370 0x7f9b900b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 -- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9015f750 msgr2=0x7f9b900cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9015f750 0x7f9b900cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 -- 172.21.15.40:0/1970564475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b90158e20 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.440+0000 7f9b8e59c640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c4c0 tx=0x7f9b8800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.441+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880052d0 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.441+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b88005470 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.441+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b901609d0 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.441+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88012870 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.441+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b90160f90 con 0x7f9b900d3ee0 2024-04-26T21:11:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.442+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b90068460 con 0x7f9b900d3ee0 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.442+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b88012a10 con 0x7f9b900d3ee0 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.442+0000 7f9b86ffd640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.443+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f9b8808b410 con 0x7f9b900d3ee0 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.443+0000 7f9b8dd9b640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.443+0000 7f9b8dd9b640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 0x7f9b7006ede0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b78034000 tx=0x7f9b7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.448+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b90068460 con 0x7f9b900d3ee0 2024-04-26T21:11:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.671+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b900b43b0 con 0x7f9b900d3ee0 2024-04-26T21:11:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.672+0000 7f9b86ffd640 1 -- 172.21.15.40:0/1970564475 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f9b900b43b0 con 0x7f9b900d3ee0 2024-04-26T21:11:57.674 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:57.675 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:11:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 msgr2=0x7f9b7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 0x7f9b7006ede0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b78034000 tx=0x7f9b7802e000 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 msgr2=0x7f9b901588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c4c0 tx=0x7f9b8800c980 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 shutdown_connections 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9015f750 0x7f9b900cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9015f370 0x7f9b900b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 --2- 172.21.15.40:0/1970564475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b900d3ee0 0x7f9b901588e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 >> 172.21.15.40:0/1970564475 conn(0x7f9b9008ec10 msgr2=0x7f9b9015e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 shutdown_connections 2024-04-26T21:11:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.674+0000 7f9b94f22640 1 -- 172.21.15.40:0/1970564475 wait complete. 2024-04-26T21:11:57.687 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:11:57.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 Processor -- start 2024-04-26T21:11:57.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 -- start start 2024-04-26T21:11:57.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f017415f2e0 0x7f0174163900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01740705f0 0x7f017406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0174163e40 con 0x7f017406c2a0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f017406cc40 con 0x7f017415f2e0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017ad9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f017406f150 con 0x7f01740705f0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f0179d9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f017415f2e0 0x7f0174163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017959b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f017406c2a0 0x7f0174073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45656/0 (socket says 172.21.15.40:45656) 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f0179d9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f017415f2e0 0x7f0174163900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43088/0 (socket says 172.21.15.40:43088) 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.786+0000 7f017959b640 1 -- 172.21.15.40:0/897749543 learned_addr learned my addr 172.21.15.40:0/897749543 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f017a59d640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01740705f0 0x7f017406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0179d9c640 1 -- 172.21.15.40:0/897749543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01740705f0 msgr2=0x7f017406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0179d9c640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01740705f0 0x7f017406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0179d9c640 1 -- 172.21.15.40:0/897749543 >> v1:172.21.15.40:6789/0 conn(0x7f017406c2a0 legacy=0x7f0174073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0179d9c640 1 -- 172.21.15.40:0/897749543 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01740cc430 con 0x7f017415f2e0 2024-04-26T21:11:57.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0178d9a640 1 -- 172.21.15.40:0/897749543 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3173261970 0 0) 0x7f0174163e40 con 0x7f017406c2a0 2024-04-26T21:11:57.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.787+0000 7f0179d9c640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017415f2e0 0x7f0174163900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01700097a0 tx=0x7f017002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37702f266e8f528 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f0178d9a640 1 -- 172.21.15.40:0/897749543 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170037070 con 0x7f017415f2e0 2024-04-26T21:11:57.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f0178d9a640 1 -- 172.21.15.40:0/897749543 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0170032440 con 0x7f017415f2e0 2024-04-26T21:11:57.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f0178d9a640 1 -- 172.21.15.40:0/897749543 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170032810 con 0x7f017415f2e0 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 -- 172.21.15.40:0/897749543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017415f2e0 msgr2=0x7f0174163900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017415f2e0 0x7f0174163900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01700097a0 tx=0x7f017002a180 comp rx=0 tx=0).stop 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 -- 172.21.15.40:0/897749543 shutdown_connections 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01740705f0 0x7f017406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 --2- 172.21.15.40:0/897749543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017415f2e0 0x7f0174163900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 -- 172.21.15.40:0/897749543 >> 172.21.15.40:0/897749543 conn(0x7f01740d1de0 msgr2=0x7f01740d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 -- 172.21.15.40:0/897749543 shutdown_connections 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.788+0000 7f017ad9e640 1 -- 172.21.15.40:0/897749543 wait complete. 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 Processor -- start 2024-04-26T21:11:57.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 -- start start 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017406c2a0 0x7f017415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f017415f2e0 0x7f017406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f01740cccb0 con 0x7f01740705f0 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01740cc9d0 con 0x7f017406c2a0 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017ad9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f017415f6c0 con 0x7f017415f2e0 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017959b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017959b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43092/0 (socket says 172.21.15.40:43092) 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.790+0000 7f017959b640 1 -- 172.21.15.40:0/906544921 learned_addr learned my addr 172.21.15.40:0/906544921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 -- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f017415f2e0 msgr2=0x7f017406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017a59d640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f017415f2e0 0x7f017406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f0179d9c640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017406c2a0 0x7f017415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f017415f2e0 0x7f017406d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 -- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017406c2a0 msgr2=0x7f017415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017406c2a0 0x7f017415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 -- 172.21.15.40:0/906544921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01740c9f50 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017959b640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f016400c500 tx=0x7f016400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164018070 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164007580 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01740ca230 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164004a20 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.791+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01740ca7f0 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.792+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01740a0100 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.793+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f016401b070 con 0x7f01740705f0 2024-04-26T21:11:57.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.793+0000 7f016a7fc640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:57.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.793+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f016408aab0 con 0x7f01740705f0 2024-04-26T21:11:57.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.793+0000 7f0179d9c640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:57.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.794+0000 7f0179d9c640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 0x7f015806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0170034000 tx=0x7f017002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:57.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.797+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01640592c0 con 0x7f01740705f0 2024-04-26T21:11:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.982+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01740b7d60 con 0x7f01740705f0 2024-04-26T21:11:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.982+0000 7f016a7fc640 1 -- 172.21.15.40:0/906544921 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7f0164058bc0 con 0x7f01740705f0 2024-04-26T21:11:57.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:57.985 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:11:51.648424+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 msgr2=0x7f015806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 0x7f015806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0170034000 tx=0x7f017002e000 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 msgr2=0x7f01740c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f016400c500 tx=0x7f016400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 shutdown_connections 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f017415f2e0 0x7f017406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01740705f0 0x7f01740c95e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 --2- 172.21.15.40:0/906544921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017406c2a0 0x7f017415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 >> 172.21.15.40:0/906544921 conn(0x7f01740d1de0 msgr2=0x7f01740d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 shutdown_connections 2024-04-26T21:11:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:57.984+0000 7f017ad9e640 1 -- 172.21.15.40:0/906544921 wait complete. 2024-04-26T21:11:57.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:11:57.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:11:58.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 Processor -- start 2024-04-26T21:11:58.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 -- start start 2024-04-26T21:11:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa07415f5b0 0x7fa0740b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0740b6aa0 con 0x7fa0740b55b0 2024-04-26T21:11:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0740d3eb0 con 0x7fa07415f990 2024-04-26T21:11:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa07a6dc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740d4190 con 0x7fa07415f5b0 2024-04-26T21:11:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa073fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa07415f5b0 0x7fa0740b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa078c79640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa0740b55b0 0x7fa0740d3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45666/0 (socket says 172.21.15.40:45666) 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa078c79640 1 -- 172.21.15.40:0/2433153109 learned_addr learned my addr 172.21.15.40:0/2433153109 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa073fff640 1 -- 172.21.15.40:0/2433153109 >> v1:172.21.15.184:6789/0 conn(0x7fa07415f990 legacy=0x7fa0740d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa073fff640 1 -- 172.21.15.40:0/2433153109 >> v1:172.21.15.40:6789/0 conn(0x7fa0740b55b0 legacy=0x7fa0740d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa073fff640 1 -- 172.21.15.40:0/2433153109 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740d0200 con 0x7fa07415f5b0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.099+0000 7fa072ffd640 1 -- 172.21.15.40:0/2433153109 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239847067 0 0) 0x7fa0740b6aa0 con 0x7fa0740b55b0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa073fff640 1 --2- 172.21.15.40:0/2433153109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa07415f5b0 0x7fa0740b5070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097a0 tx=0x7fa06402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da93a41cbbabb814 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa072ffd640 1 -- 172.21.15.40:0/2433153109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064037070 con 0x7fa07415f5b0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa072ffd640 1 -- 172.21.15.40:0/2433153109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa064032440 con 0x7fa07415f5b0 2024-04-26T21:11:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa072ffd640 1 -- 172.21.15.40:0/2433153109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064032810 con 0x7fa07415f5b0 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 -- 172.21.15.40:0/2433153109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa07415f5b0 msgr2=0x7fa0740b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/2433153109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa07415f5b0 0x7fa0740b5070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097a0 tx=0x7fa06402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 -- 172.21.15.40:0/2433153109 shutdown_connections 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/2433153109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa07415f5b0 0x7fa0740b5070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 -- 172.21.15.40:0/2433153109 >> 172.21.15.40:0/2433153109 conn(0x7fa07408ec40 msgr2=0x7fa07409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 -- 172.21.15.40:0/2433153109 shutdown_connections 2024-04-26T21:11:58.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.100+0000 7fa07a6dc640 1 -- 172.21.15.40:0/2433153109 wait complete. 2024-04-26T21:11:58.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 Processor -- start 2024-04-26T21:11:58.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 -- start start 2024-04-26T21:11:58.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa07415f5b0 0x7fa0740cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa07415f990 0x7fa074154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740d1290 con 0x7fa0740b55b0 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740d1110 con 0x7fa07415f990 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.102+0000 7fa07a6dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0740d3a50 con 0x7fa07415f5b0 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43116/0 (socket says 172.21.15.40:43116) 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 -- 172.21.15.40:0/3358950598 learned_addr learned my addr 172.21.15.40:0/3358950598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa0737fe640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa07415f5b0 0x7fa0740cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa078c79640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa07415f990 0x7fa074154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 -- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa07415f5b0 msgr2=0x7fa0740cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa07415f5b0 0x7fa0740cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 -- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa07415f990 msgr2=0x7fa074154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa07415f990 0x7fa074154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 -- 172.21.15.40:0/3358950598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740cc840 con 0x7fa0740b55b0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa073fff640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098c0 tx=0x7fa0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064037040 con 0x7fa0740b55b0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa064042070 con 0x7fa0740b55b0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa074154c90 con 0x7fa0740b55b0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064030a70 con 0x7fa0740b55b0 2024-04-26T21:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.103+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0740ccda0 con 0x7fa0740b55b0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.104+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07406a290 con 0x7fa0740b55b0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.105+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa064030c10 con 0x7fa0740b55b0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.105+0000 7fa070ff9640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 0x7fa06006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.105+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa0640ac170 con 0x7fa0740b55b0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.105+0000 7fa0737fe640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 0x7fa06006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.106+0000 7fa0737fe640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 0x7fa06006ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0740cf390 tx=0x7fa068009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.109+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0740cc840 con 0x7fa0740b55b0 2024-04-26T21:11:58.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.293+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0740d60c0 con 0x7fa0740b55b0 2024-04-26T21:11:58.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.294+0000 7fa070ff9640 1 -- 172.21.15.40:0/3358950598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7fa06407a2a0 con 0x7fa0740b55b0 2024-04-26T21:11:58.296 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 msgr2=0x7fa06006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 0x7fa06006ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0740cf390 tx=0x7fa068009040 comp rx=0 tx=0).stop 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 msgr2=0x7fa07415fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098c0 tx=0x7fa0640046c0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 shutdown_connections 2024-04-26T21:11:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa06006c920 0x7fa06006ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa07415f990 0x7fa074154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa07415f5b0 0x7fa0740cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 --2- 172.21.15.40:0/3358950598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0740b55b0 0x7fa07415fdd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 >> 172.21.15.40:0/3358950598 conn(0x7fa07408ec40 msgr2=0x7fa0740b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 shutdown_connections 2024-04-26T21:11:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.297+0000 7fa07a6dc640 1 -- 172.21.15.40:0/3358950598 wait complete. 2024-04-26T21:11:58.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:11:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.409+0000 7f83db114640 1 Processor -- start 2024-04-26T21:11:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.409+0000 7f83db114640 1 -- start start 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83db114640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83d415f370 0x7f83d40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83db114640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40d03a0 con 0x7f83d415f750 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83db114640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40d0680 con 0x7f83d40d3ee0 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83db114640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40d0960 con 0x7f83d415f370 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83d415f370 0x7f83d40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83d415f370 0x7f83d40cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43126/0 (socket says 172.21.15.40:43126) 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 -- 172.21.15.40:0/3900524062 learned_addr learned my addr 172.21.15.40:0/3900524062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f83d415f750 0x7f83d40d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45680/0 (socket says 172.21.15.40:45680) 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 -- 172.21.15.40:0/3900524062 >> v1:172.21.15.184:6789/0 conn(0x7f83d40d3ee0 legacy=0x7f83d40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 -- 172.21.15.40:0/3900524062 >> v1:172.21.15.40:6789/0 conn(0x7f83d415f750 legacy=0x7f83d40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 -- 172.21.15.40:0/3900524062 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40d0c40 con 0x7f83d415f370 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d37fe640 1 -- 172.21.15.40:0/3900524062 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156333783 0 0) 0x7f83d40d03a0 con 0x7f83d415f750 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d8eb0640 1 --2- 172.21.15.40:0/3900524062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d40cfd70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37cca6dbdab9319 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d37fe640 1 -- 172.21.15.40:0/3900524062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8037070 con 0x7f83d415f370 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d37fe640 1 -- 172.21.15.40:0/3900524062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c802ae50 con 0x7f83d415f370 2024-04-26T21:11:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.410+0000 7f83d37fe640 1 -- 172.21.15.40:0/3900524062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8032860 con 0x7f83d415f370 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 -- 172.21.15.40:0/3900524062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 msgr2=0x7f83d40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 --2- 172.21.15.40:0/3900524062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d40cfd70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a300 comp rx=0 tx=0).stop 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 -- 172.21.15.40:0/3900524062 shutdown_connections 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 --2- 172.21.15.40:0/3900524062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d40cfd70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 -- 172.21.15.40:0/3900524062 >> 172.21.15.40:0/3900524062 conn(0x7f83d408ec10 msgr2=0x7f83d409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 -- 172.21.15.40:0/3900524062 shutdown_connections 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.411+0000 7f83db114640 1 -- 172.21.15.40:0/3900524062 wait complete. 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.413+0000 7f83db114640 1 Processor -- start 2024-04-26T21:11:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.413+0000 7f83db114640 1 -- start start 2024-04-26T21:11:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3ee0 0x7f83d40cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d415f750 0x7f83d40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b4950 con 0x7f83d415f370 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b47d0 con 0x7f83d415f750 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40b4ad0 con 0x7f83d40d3ee0 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43136/0 (socket says 172.21.15.40:43136) 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 -- 172.21.15.40:0/519824190 learned_addr learned my addr 172.21.15.40:0/519824190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d8eb0640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3ee0 0x7f83d40cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d96b1640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d415f750 0x7f83d40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 -- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3ee0 msgr2=0x7f83d40cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3ee0 0x7f83d40cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 -- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d415f750 msgr2=0x7f83d40c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d415f750 0x7f83d40c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 -- 172.21.15.40:0/519824190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40c95f0 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d3fff640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c4a0 tx=0x7f83c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0018070 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c00074e0 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004a20 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d40c98d0 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.414+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d40cb140 con 0x7f83d415f370 2024-04-26T21:11:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.415+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d40684b0 con 0x7f83d415f370 2024-04-26T21:11:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.415+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83c001b070 con 0x7f83d415f370 2024-04-26T21:11:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.415+0000 7f83d17fa640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.416+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f83c008aa80 con 0x7f83d415f370 2024-04-26T21:11:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.416+0000 7f83d8eb0640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.416+0000 7f83d8eb0640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 0x7f83b806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83c8034000 tx=0x7f83c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.421+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83d40684b0 con 0x7f83d415f370 2024-04-26T21:11:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.605+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f83d406a290 con 0x7f83d415f370 2024-04-26T21:11:58.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.605+0000 7f83d17fa640 1 -- 172.21.15.40:0/519824190 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f83c0058bb0 con 0x7f83d415f370 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:11:58.608 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 msgr2=0x7f83b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 0x7f83b806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83c8034000 tx=0x7f83c802e000 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 msgr2=0x7f83d415bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c4a0 tx=0x7f83c000c960 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 shutdown_connections 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83d415f750 0x7f83d40c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d415f370 0x7f83d415bbd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 --2- 172.21.15.40:0/519824190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83d40d3ee0 0x7f83d40cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 >> 172.21.15.40:0/519824190 conn(0x7f83d408ec10 msgr2=0x7f83d40d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 shutdown_connections 2024-04-26T21:11:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.608+0000 7f83db114640 1 -- 172.21.15.40:0/519824190 wait complete. 2024-04-26T21:11:58.620 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:11:58.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:11:58.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.723+0000 7f90cf415640 1 Processor -- start 2024-04-26T21:11:58.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.723+0000 7f90cf415640 1 -- start start 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cf415640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90c815d2b0 0x7f90c80d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cf415640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90c809ba40 0x7f90c809be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cf415640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90c80d12c0 con 0x7f90c80d0ae0 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cf415640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f90c80cd9b0 con 0x7f90c815d2b0 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cf415640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f90c809c3d0 con 0x7f90c809ba40 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cd1b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90c815d2b0 0x7f90c80d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cd1b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f90c815d2b0 0x7f90c80d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43140/0 (socket says 172.21.15.40:43140) 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cd1b1640 1 -- 172.21.15.40:0/3301150304 learned_addr learned my addr 172.21.15.40:0/3301150304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.724+0000 7f90cd9b2640 1 --2- 172.21.15.40:0/3301150304 >> v2:172.21.15.184:3300/0 conn(0x7f90c809ba40 0x7f90c809be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90cd1b1640 1 -- 172.21.15.40:0/3301150304 >> v2:172.21.15.184:3300/0 conn(0x7f90c809ba40 msgr2=0x7f90c809be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90cd1b1640 1 --2- 172.21.15.40:0/3301150304 >> v2:172.21.15.184:3300/0 conn(0x7f90c809ba40 0x7f90c809be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90cd1b1640 1 -- 172.21.15.40:0/3301150304 >> v1:172.21.15.184:6789/0 conn(0x7f90c80d0ae0 legacy=0x7f90c80ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:58.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90cd1b1640 1 -- 172.21.15.40:0/3301150304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c80d0fe0 con 0x7f90c815d2b0 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90cd1b1640 1 --2- 172.21.15.40:0/3301150304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c815d2b0 0x7f90c80d3a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90c8069050 tx=0x7f90c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db9833085c42ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90c7fff640 1 -- 172.21.15.40:0/3301150304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0037070 con 0x7f90c815d2b0 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90c7fff640 1 -- 172.21.15.40:0/3301150304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c0032440 con 0x7f90c815d2b0 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.725+0000 7f90c7fff640 1 -- 172.21.15.40:0/3301150304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0032810 con 0x7f90c815d2b0 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 -- 172.21.15.40:0/3301150304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c815d2b0 msgr2=0x7f90c80d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/3301150304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c815d2b0 0x7f90c80d3a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90c8069050 tx=0x7f90c002a270 comp rx=0 tx=0).stop 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 -- 172.21.15.40:0/3301150304 shutdown_connections 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/3301150304 >> v2:172.21.15.184:3300/0 conn(0x7f90c809ba40 0x7f90c809be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/3301150304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c815d2b0 0x7f90c80d3a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 -- 172.21.15.40:0/3301150304 >> 172.21.15.40:0/3301150304 conn(0x7f90c80d2f10 msgr2=0x7f90c80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 -- 172.21.15.40:0/3301150304 shutdown_connections 2024-04-26T21:11:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.726+0000 7f90cf415640 1 -- 172.21.15.40:0/3301150304 wait complete. 2024-04-26T21:11:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.727+0000 7f90cf415640 1 Processor -- start 2024-04-26T21:11:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.727+0000 7f90cf415640 1 -- start start 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 0x7f90c81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 0x7f90c80cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90c80cdec0 con 0x7f90c809ba40 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90c80cedd0 con 0x7f90c80d0ae0 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cf415640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90c80ce1a0 con 0x7f90c815d2b0 2024-04-26T21:11:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43142/0 (socket says 172.21.15.40:43142) 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 -- 172.21.15.40:0/1196022218 learned_addr learned my addr 172.21.15.40:0/1196022218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd9b2640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 0x7f90c80cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 -- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 msgr2=0x7f90c80cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cc9b0640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 0x7f90c81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 0x7f90c80cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 -- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 msgr2=0x7f90c81505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 0x7f90c81505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd1b1640 1 -- 172.21.15.40:0/1196022218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c8150ae0 con 0x7f90c809ba40 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cd9b2640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 0x7f90c80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.728+0000 7f90cc9b0640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 0x7f90c81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90cd1b1640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90c00097f0 tx=0x7f90c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0037040 con 0x7f90c809ba40 2024-04-26T21:11:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c0004250 con 0x7f90c809ba40 2024-04-26T21:11:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c81e36a0 con 0x7f90c809ba40 2024-04-26T21:11:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0003610 con 0x7f90c809ba40 2024-04-26T21:11:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.729+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c81e6060 con 0x7f90c809ba40 2024-04-26T21:11:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.730+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c806a290 con 0x7f90c809ba40 2024-04-26T21:11:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.730+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90c00037b0 con 0x7f90c809ba40 2024-04-26T21:11:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.731+0000 7f90c5ffb640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.731+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f90c00ac5a0 con 0x7f90c809ba40 2024-04-26T21:11:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.731+0000 7f90cc9b0640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.731+0000 7f90cc9b0640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 0x7f90b406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90c80b7290 tx=0x7f90bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.735+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90c8150ae0 con 0x7f90c809ba40 2024-04-26T21:11:58.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:58.919+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f90c8069580 con 0x7f90c809ba40 2024-04-26T21:11:59.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.721+0000 7f90c5ffb640 1 -- 172.21.15.40:0/1196022218 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f90c007ad10 con 0x7f90c809ba40 2024-04-26T21:11:59.723 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:11:59.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.725+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 msgr2=0x7f90b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.725+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 0x7f90b406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90c80b7290 tx=0x7f90bc008040 comp rx=0 tx=0).stop 2024-04-26T21:11:59.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.725+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 msgr2=0x7f90c81f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.725+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90c00097f0 tx=0x7f90c00046c0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 shutdown_connections 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c815d2b0 0x7f90c80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c80d0ae0 0x7f90c81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 --2- 172.21.15.40:0/1196022218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c809ba40 0x7f90c81f37a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 >> 172.21.15.40:0/1196022218 conn(0x7f90c80d2f10 msgr2=0x7f90c80b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 shutdown_connections 2024-04-26T21:11:59.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.726+0000 7f90cf415640 1 -- 172.21.15.40:0/1196022218 wait complete. 2024-04-26T21:11:59.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:11:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.857+0000 7f2c66859640 1 Processor -- start 2024-04-26T21:11:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.857+0000 7f2c66859640 1 -- start start 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.858+0000 7f2c66859640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2c6015e780 0x7f2c60160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.858+0000 7f2c66859640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 0x7f2c601614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.858+0000 7f2c66859640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c600ce7d0 con 0x7f2c60161a10 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.858+0000 7f2c66859640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2c60161ea0 con 0x7f2c6015e780 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.858+0000 7f2c66859640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c600d0b50 con 0x7f2c600b43d0 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 0x7f2c601614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c64df6640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2c60161a10 0x7f2c6009ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50668/0 (socket says 172.21.15.40:50668) 2024-04-26T21:11:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c64df6640 1 -- 172.21.15.40:0/2246821380 learned_addr learned my addr 172.21.15.40:0/2246821380 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/2246821380 >> v2:172.21.15.184:3300/0 conn(0x7f2c6015e780 0x7f2c60160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5effd640 1 -- 172.21.15.40:0/2246821380 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814354226 0 0) 0x7f2c600ce7d0 con 0x7f2c60161a10 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5effd640 1 -- 172.21.15.40:0/2246821380 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c48003520 con 0x7f2c60161a10 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5ffff640 1 -- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 msgr2=0x7f2c601614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 0x7f2c601614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5ffff640 1 -- 172.21.15.40:0/2246821380 >> v1:172.21.15.184:6789/0 conn(0x7f2c60161a10 legacy=0x7f2c6009ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5ffff640 1 -- 172.21.15.40:0/2246821380 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c600ceab0 con 0x7f2c6015e780 2024-04-26T21:11:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.859+0000 7f2c5f7fe640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 0x7f2c601614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:11:59.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c6015e780 0x7f2c60160f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c500097a0 tx=0x7f2c5002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d2ea1786c310f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c5effd640 1 -- 172.21.15.40:0/2246821380 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50037070 con 0x7f2c6015e780 2024-04-26T21:11:59.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c5effd640 1 -- 172.21.15.40:0/2246821380 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5002adb0 con 0x7f2c6015e780 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c5effd640 1 -- 172.21.15.40:0/2246821380 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50032640 con 0x7f2c6015e780 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c66859640 1 -- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c6015e780 msgr2=0x7f2c60160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.860+0000 7f2c66859640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c6015e780 0x7f2c60160f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c500097a0 tx=0x7f2c5002a060 comp rx=0 tx=0).stop 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 -- 172.21.15.40:0/2246821380 shutdown_connections 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c600b43d0 0x7f2c601614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 --2- 172.21.15.40:0/2246821380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c6015e780 0x7f2c60160f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 -- 172.21.15.40:0/2246821380 >> 172.21.15.40:0/2246821380 conn(0x7f2c600b4ab0 msgr2=0x7f2c600b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 -- 172.21.15.40:0/2246821380 shutdown_connections 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.861+0000 7f2c66859640 1 -- 172.21.15.40:0/2246821380 wait complete. 2024-04-26T21:11:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.862+0000 7f2c66859640 1 Processor -- start 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.862+0000 7f2c66859640 1 -- start start 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6015e780 0x7f2c601602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c601607f0 0x7f2c600c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600b27e0 con 0x7f2c600b43d0 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600d1ff0 con 0x7f2c601607f0 2024-04-26T21:11:59.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c66859640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c60157dc0 con 0x7f2c6015e780 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43144/0 (socket says 172.21.15.40:43144) 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 -- 172.21.15.40:0/3039783201 learned_addr learned my addr 172.21.15.40:0/3039783201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5f7fe640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6015e780 0x7f2c601602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c64df6640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c601607f0 0x7f2c600c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 -- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6015e780 msgr2=0x7f2c601602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6015e780 0x7f2c601602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 -- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c601607f0 msgr2=0x7f2c600c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c601607f0 0x7f2c600c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 -- 172.21.15.40:0/3039783201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6015f210 con 0x7f2c600b43d0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5ffff640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c5002a0a0 tx=0x7f2c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.863+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50041070 con 0x7f2c600b43d0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.864+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5003c460 con 0x7f2c600b43d0 2024-04-26T21:11:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.864+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6015f4f0 con 0x7f2c600b43d0 2024-04-26T21:11:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.864+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50037040 con 0x7f2c600b43d0 2024-04-26T21:11:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.864+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6015fab0 con 0x7f2c600b43d0 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.865+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6006a290 con 0x7f2c600b43d0 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.865+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c50044070 con 0x7f2c600b43d0 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.866+0000 7f2c5cff9640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 0x7f2c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.866+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f2c500abe50 con 0x7f2c600b43d0 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.866+0000 7f2c5f7fe640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 0x7f2c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:11:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.866+0000 7f2c5f7fe640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 0x7f2c4c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c6015de50 tx=0x7f2c54009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:11:59.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:11:59.870+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c50079ea0 con 0x7f2c600b43d0 2024-04-26T21:12:00.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.054+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2c600b1840 con 0x7f2c600b43d0 2024-04-26T21:12:00.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.724+0000 7f2c5cff9640 1 -- 172.21.15.40:0/3039783201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f2c5007e1a0 con 0x7f2c600b43d0 2024-04-26T21:12:00.725 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:12:00.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 msgr2=0x7f2c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 0x7f2c4c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c6015de50 tx=0x7f2c54009190 comp rx=0 tx=0).stop 2024-04-26T21:12:00.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 msgr2=0x7f2c60157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c5002a0a0 tx=0x7f2c500046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 shutdown_connections 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c4c06c8f0 0x7f2c4c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c601607f0 0x7f2c600c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c6015e780 0x7f2c601602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 --2- 172.21.15.40:0/3039783201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c600b43d0 0x7f2c60157880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 >> 172.21.15.40:0/3039783201 conn(0x7f2c600b4ab0 msgr2=0x7f2c600b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 shutdown_connections 2024-04-26T21:12:00.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.727+0000 7f2c66859640 1 -- 172.21.15.40:0/3039783201 wait complete. 2024-04-26T21:12:00.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:12:00.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:12:00.861 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:12:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.861+0000 7f3e90522640 1 Processor -- start 2024-04-26T21:12:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.861+0000 7f3e90522640 1 -- start start 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e90522640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e88162ca0 0x7f3e88070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e90522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880d1c20 0x7f3e88070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e90522640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e880d2470 con 0x7f3e88071160 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e90522640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3e880715d0 con 0x7f3e88162ca0 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e90522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880cdc40 con 0x7f3e880d1c20 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e88162ca0 0x7f3e88070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e88162ca0 0x7f3e88070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51708/0 (socket says 172.21.15.40:51708) 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 -- 172.21.15.40:0/2519898398 learned_addr learned my addr 172.21.15.40:0/2519898398 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8dabd640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880d1c20 0x7f3e88070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 -- 172.21.15.40:0/2519898398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880d1c20 msgr2=0x7f3e88070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880d1c20 0x7f3e88070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 -- 172.21.15.40:0/2519898398 >> v1:172.21.15.184:6789/0 conn(0x7f3e88071160 legacy=0x7f3e880cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.862+0000 7f3e8e2be640 1 -- 172.21.15.40:0/2519898398 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880d2750 con 0x7f3e88162ca0 2024-04-26T21:12:00.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.863+0000 7f3e8e2be640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e88162ca0 0x7f3e88070640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009870 tx=0x7f3e8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca84fee2991fbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.863+0000 7f3e8d2bc640 1 -- 172.21.15.40:0/2519898398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84037070 con 0x7f3e88162ca0 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.863+0000 7f3e8d2bc640 1 -- 172.21.15.40:0/2519898398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e84032440 con 0x7f3e88162ca0 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.863+0000 7f3e8d2bc640 1 -- 172.21.15.40:0/2519898398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84032810 con 0x7f3e88162ca0 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 -- 172.21.15.40:0/2519898398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e88162ca0 msgr2=0x7f3e88070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e88162ca0 0x7f3e88070640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009870 tx=0x7f3e8402a270 comp rx=0 tx=0).stop 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 -- 172.21.15.40:0/2519898398 shutdown_connections 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e880d1c20 0x7f3e88070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 --2- 172.21.15.40:0/2519898398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e88162ca0 0x7f3e88070640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 -- 172.21.15.40:0/2519898398 >> 172.21.15.40:0/2519898398 conn(0x7f3e8809ba40 msgr2=0x7f3e8809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 -- 172.21.15.40:0/2519898398 shutdown_connections 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.864+0000 7f3e90522640 1 -- 172.21.15.40:0/2519898398 wait complete. 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.865+0000 7f3e90522640 1 Processor -- start 2024-04-26T21:12:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 -- start start 2024-04-26T21:12:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88071160 0x7f3e880cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e88162ca0 0x7f3e880b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880d2080 con 0x7f3e880d1c20 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880cd8d0 con 0x7f3e88071160 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e90522640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880d2200 con 0x7f3e88162ca0 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e8dabd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e8dabd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51716/0 (socket says 172.21.15.40:51716) 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e8dabd640 1 -- 172.21.15.40:0/3404745647 learned_addr learned my addr 172.21.15.40:0/3404745647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e8e2be640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88071160 0x7f3e880cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.866+0000 7f3e8eabf640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e88162ca0 0x7f3e880b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 -- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e88162ca0 msgr2=0x7f3e880b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e88162ca0 0x7f3e880b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 -- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88071160 msgr2=0x7f3e880cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88071160 0x7f3e880cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 -- 172.21.15.40:0/3404745647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880ccc40 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e8dabd640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800d800 tx=0x7f3e7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800b6c0 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78019070 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800ba80 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e880cb1b0 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.867+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e880cc910 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.868+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e880693d0 con 0x7f3e880d1c20 2024-04-26T21:12:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.869+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e7800bc20 con 0x7f3e880d1c20 2024-04-26T21:12:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.869+0000 7f3e7effd640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 0x7f3e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.869+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f3e7808bda0 con 0x7f3e880d1c20 2024-04-26T21:12:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.870+0000 7f3e8e2be640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 0x7f3e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.870+0000 7f3e8e2be640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 0x7f3e6c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e84034000 tx=0x7f3e8402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:00.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:00.874+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e78059c70 con 0x7f3e880d1c20 2024-04-26T21:12:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.102+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3e8806d9e0 con 0x7f3e880d1c20 2024-04-26T21:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.739+0000 7f3e7effd640 1 -- 172.21.15.40:0/3404745647 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f3e7805df70 con 0x7f3e880d1c20 2024-04-26T21:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.743+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 msgr2=0x7f3e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.743+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 0x7f3e6c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e84034000 tx=0x7f3e8402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 msgr2=0x7f3e880b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800d800 tx=0x7f3e7800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 shutdown_connections 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3e6c06c8f0 0x7f3e6c06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e88162ca0 0x7f3e880b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e880d1c20 0x7f3e880b1df0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 --2- 172.21.15.40:0/3404745647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e88071160 0x7f3e880cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.744+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 >> 172.21.15.40:0/3404745647 conn(0x7f3e8809ba40 msgr2=0x7f3e8815fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.745+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 shutdown_connections 2024-04-26T21:12:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.745+0000 7f3e90522640 1 -- 172.21.15.40:0/3404745647 wait complete. 2024-04-26T21:12:01.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.881+0000 7fc6be883640 1 Processor -- start 2024-04-26T21:12:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.881+0000 7fc6be883640 1 -- start start 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6be883640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6b815d3b0 0x7fc6b80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6be883640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b8073b10 0x7fc6b806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6be883640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b80b6b80 con 0x7fc6b806c7e0 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6be883640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b80aedf0 con 0x7fc6b815d3b0 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6be883640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b806cc40 con 0x7fc6b8073b10 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6b815d3b0 0x7fc6b80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6b815d3b0 0x7fc6b80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51722/0 (socket says 172.21.15.40:51722) 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6bce20640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc6b806c7e0 0x7fc6b8160f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46152/0 (socket says 172.21.15.40:46152) 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 -- 172.21.15.40:0/61183082 learned_addr learned my addr 172.21.15.40:0/61183082 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b77fe640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b8073b10 0x7fc6b806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 -- 172.21.15.40:0/61183082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b8073b10 msgr2=0x7fc6b806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b8073b10 0x7fc6b806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 -- 172.21.15.40:0/61183082 >> v1:172.21.15.40:6789/0 conn(0x7fc6b806c7e0 legacy=0x7fc6b8160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b7fff640 1 -- 172.21.15.40:0/61183082 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b80b6e60 con 0x7fc6b815d3b0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.882+0000 7fc6b6ffd640 1 -- 172.21.15.40:0/61183082 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738898803 0 0) 0x7fc6b80b6b80 con 0x7fc6b806c7e0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.883+0000 7fc6b7fff640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b80735d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1127136bf43e001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.883+0000 7fc6b6ffd640 1 -- 172.21.15.40:0/61183082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0037070 con 0x7fc6b815d3b0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.883+0000 7fc6b6ffd640 1 -- 172.21.15.40:0/61183082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a0032440 con 0x7fc6b815d3b0 2024-04-26T21:12:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.883+0000 7fc6b6ffd640 1 -- 172.21.15.40:0/61183082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0032810 con 0x7fc6b815d3b0 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 -- 172.21.15.40:0/61183082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 msgr2=0x7fc6b80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b80735d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 -- 172.21.15.40:0/61183082 shutdown_connections 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b8073b10 0x7fc6b806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 --2- 172.21.15.40:0/61183082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b80735d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 -- 172.21.15.40:0/61183082 >> 172.21.15.40:0/61183082 conn(0x7fc6b80b3b10 msgr2=0x7fc6b80b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 -- 172.21.15.40:0/61183082 shutdown_connections 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.884+0000 7fc6be883640 1 -- 172.21.15.40:0/61183082 wait complete. 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.885+0000 7fc6be883640 1 Processor -- start 2024-04-26T21:12:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.885+0000 7fc6be883640 1 -- start start 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b806c7e0 0x7fc6b814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6b8073b10 0x7fc6b8149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b80d0170 con 0x7fc6b815d3b0 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b80b6930 con 0x7fc6b8073b10 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6be883640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b815d790 con 0x7fc6b806c7e0 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51728/0 (socket says 172.21.15.40:51728) 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 -- 172.21.15.40:0/3360008644 learned_addr learned my addr 172.21.15.40:0/3360008644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6b7fff640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b806c7e0 0x7fc6b814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6b77fe640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6b8073b10 0x7fc6b8149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 -- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b806c7e0 msgr2=0x7fc6b814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b806c7e0 0x7fc6b814c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 -- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6b8073b10 msgr2=0x7fc6b8149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6b8073b10 0x7fc6b8149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 -- 172.21.15.40:0/3360008644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b814a260 con 0x7fc6b815d3b0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.886+0000 7fc6bce20640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c7a0 tx=0x7fc6b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.887+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0018070 con 0x7fc6b815d3b0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.887+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b000a640 con 0x7fc6b815d3b0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.887+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b814a540 con 0x7fc6b815d3b0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.887+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000aab0 con 0x7fc6b815d3b0 2024-04-26T21:12:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.887+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b814ab00 con 0x7fc6b815d3b0 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.888+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b80693d0 con 0x7fc6b815d3b0 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.888+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6b001b070 con 0x7fc6b815d3b0 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.888+0000 7fc6b4ff9640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 0x7fc69806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.889+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc6b008afc0 con 0x7fc6b815d3b0 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.889+0000 7fc6b7fff640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 0x7fc69806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.889+0000 7fc6b7fff640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 0x7fc69806ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:01.893+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6b0058e30 con 0x7fc6b815d3b0 2024-04-26T21:12:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.081+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6b80b1700 con 0x7fc6b815d3b0 2024-04-26T21:12:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.083+0000 7fc6b4ff9640 1 -- 172.21.15.40:0/3360008644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16618 (secure 0 0 0) 0x7fc6b005d130 con 0x7fc6b815d3b0 2024-04-26T21:12:02.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:01.103960+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 msgr2=0x7fc69806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 0x7fc69806ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 msgr2=0x7fc6b814b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c7a0 tx=0x7fc6b000cc60 comp rx=0 tx=0).stop 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 shutdown_connections 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc69806c920 0x7fc69806ede0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6b815d3b0 0x7fc6b814b300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6b8073b10 0x7fc6b8149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 --2- 172.21.15.40:0/3360008644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6b806c7e0 0x7fc6b814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 >> 172.21.15.40:0/3360008644 conn(0x7fc6b80b3b10 msgr2=0x7fc6b815bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 shutdown_connections 2024-04-26T21:12:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.085+0000 7fc6be883640 1 -- 172.21.15.40:0/3360008644 wait complete. 2024-04-26T21:12:02.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.196+0000 7feefe19d640 1 Processor -- start 2024-04-26T21:12:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.196+0000 7feefe19d640 1 -- start start 2024-04-26T21:12:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feefe19d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feef815f340 0x7feef80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feefe19d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80b2510 con 0x7feef815f720 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feefe19d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80b27f0 con 0x7feef80d5610 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feefe19d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7feef80b2ad0 con 0x7feef815f340 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feef815f720 0x7feef80d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46158/0 (socket says 172.21.15.40:46158) 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef6ffd640 1 -- 172.21.15.40:0/3461643788 learned_addr learned my addr 172.21.15.40:0/3461643788 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef77fe640 1 --2- 172.21.15.40:0/3461643788 >> v2:172.21.15.184:3300/0 conn(0x7feef815f340 0x7feef80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef67fc640 1 -- 172.21.15.40:0/3461643788 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520957147 0 0) 0x7feef80b2510 con 0x7feef815f720 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef67fc640 1 -- 172.21.15.40:0/3461643788 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee0003520 con 0x7feef815f720 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef77fe640 1 -- 172.21.15.40:0/3461643788 >> v1:172.21.15.184:6789/0 conn(0x7feef80d5610 legacy=0x7feef80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef77fe640 1 -- 172.21.15.40:0/3461643788 >> v1:172.21.15.40:6789/0 conn(0x7feef815f720 legacy=0x7feef80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.197+0000 7feef77fe640 1 -- 172.21.15.40:0/3461643788 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80b2db0 con 0x7feef815f340 2024-04-26T21:12:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.198+0000 7feef77fe640 1 --2- 172.21.15.40:0/3461643788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef80b1e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5423519ce4fa4b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.198+0000 7feef67fc640 1 -- 172.21.15.40:0/3461643788 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8037070 con 0x7feef815f340 2024-04-26T21:12:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.198+0000 7feef67fc640 1 -- 172.21.15.40:0/3461643788 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8032440 con 0x7feef815f340 2024-04-26T21:12:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.198+0000 7feef67fc640 1 -- 172.21.15.40:0/3461643788 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8032810 con 0x7feef815f340 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 -- 172.21.15.40:0/3461643788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 msgr2=0x7feef80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 --2- 172.21.15.40:0/3461643788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef80b1e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 -- 172.21.15.40:0/3461643788 shutdown_connections 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 --2- 172.21.15.40:0/3461643788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef80b1e80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 -- 172.21.15.40:0/3461643788 >> 172.21.15.40:0/3461643788 conn(0x7feef808ec70 msgr2=0x7feef809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 -- 172.21.15.40:0/3461643788 shutdown_connections 2024-04-26T21:12:02.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.199+0000 7feefe19d640 1 -- 172.21.15.40:0/3461643788 wait complete. 2024-04-26T21:12:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.200+0000 7feefe19d640 1 Processor -- start 2024-04-26T21:12:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.200+0000 7feefe19d640 1 -- start start 2024-04-26T21:12:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 0x7feef80ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef81585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feef80d40d0 con 0x7feef815f720 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feef80d3f50 con 0x7feef815f340 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feefe19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feef80b3cc0 con 0x7feef80d5610 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51744/0 (socket says 172.21.15.40:51744) 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 -- 172.21.15.40:0/3053926765 learned_addr learned my addr 172.21.15.40:0/3053926765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef77fe640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 0x7feef80ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef6ffd640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef81585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 -- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 msgr2=0x7feef80ba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 0x7feef80ba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 -- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 msgr2=0x7feef81585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef81585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 -- 172.21.15.40:0/3053926765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80b8fc0 con 0x7feef815f720 2024-04-26T21:12:02.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef77fe640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 0x7feef80ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.201+0000 7feef7fff640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feef000c730 tx=0x7feef000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.202+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0003b70 con 0x7feef815f720 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.202+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0003d10 con 0x7feef815f720 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.202+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef80b92a0 con 0x7feef815f720 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.202+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef000aa90 con 0x7feef815f720 2024-04-26T21:12:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.202+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef80b9b40 con 0x7feef815f720 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.203+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef80b9690 con 0x7feef815f720 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.203+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feef000ac50 con 0x7feef815f720 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.204+0000 7feedffff640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.204+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feef008baf0 con 0x7feef815f720 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.204+0000 7feef77fe640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.205+0000 7feef77fe640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 0x7feee406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.208+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feef0059e80 con 0x7feef815f720 2024-04-26T21:12:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.395+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feef806a290 con 0x7feef815f720 2024-04-26T21:12:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.396+0000 7feedffff640 1 -- 172.21.15.40:0/3053926765 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16618 (secure 0 0 0) 0x7feef0059780 con 0x7feef815f720 2024-04-26T21:12:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:02.398 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:01.103960+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.398+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 msgr2=0x7feee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.398+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 0x7feee406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.398+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 msgr2=0x7feef80a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.398+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feef000c730 tx=0x7feef000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 shutdown_connections 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef815f720 0x7feef80a0a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feef815f340 0x7feef81585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 --2- 172.21.15.40:0/3053926765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feef80d5610 0x7feef80ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 >> 172.21.15.40:0/3053926765 conn(0x7feef808ec70 msgr2=0x7feef80b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 shutdown_connections 2024-04-26T21:12:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.399+0000 7feefe19d640 1 -- 172.21.15.40:0/3053926765 wait complete. 2024-04-26T21:12:02.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.513+0000 7ff84099d640 1 Processor -- start 2024-04-26T21:12:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.513+0000 7ff84099d640 1 -- start start 2024-04-26T21:12:02.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff84099d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff83c0d3e50 0x7ff83c0cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff84099d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 0x7ff83c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff84099d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0b63e0 con 0x7ff83c15f950 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff84099d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0d5ed0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff84099d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0d3120 con 0x7ff83c0d2230 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff83c0d3e50 0x7ff83c0cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff83c0d3e50 0x7ff83c0cfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51752/0 (socket says 172.21.15.40:51752) 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff83a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff83c15f950 0x7ff83c0d3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46168/0 (socket says 172.21.15.40:46168) 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 -- 172.21.15.40:0/2245040906 learned_addr learned my addr 172.21.15.40:0/2245040906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 -- 172.21.15.40:0/2245040906 >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 msgr2=0x7ff83c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff83ad9d640 1 --2- 172.21.15.40:0/2245040906 >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 0x7ff83c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 --2- 172.21.15.40:0/2245040906 >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 0x7ff83c0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 -- 172.21.15.40:0/2245040906 >> v1:172.21.15.40:6789/0 conn(0x7ff83c15f950 legacy=0x7ff83c0d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff83959a640 1 -- 172.21.15.40:0/2245040906 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934229621 0 0) 0x7ff83c0b63e0 con 0x7ff83c15f950 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff839d9b640 1 -- 172.21.15.40:0/2245040906 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0b69c0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.514+0000 7ff83ad9d640 1 --2- 172.21.15.40:0/2245040906 >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 0x7ff83c0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.515+0000 7ff839d9b640 1 --2- 172.21.15.40:0/2245040906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0cfd00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0d2c40 tx=0x7ff83002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d9b38c75d22aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.515+0000 7ff83959a640 1 -- 172.21.15.40:0/2245040906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830037070 con 0x7ff83c0d3e50 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.515+0000 7ff83959a640 1 -- 172.21.15.40:0/2245040906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830032440 con 0x7ff83c0d3e50 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.515+0000 7ff83959a640 1 -- 172.21.15.40:0/2245040906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830032810 con 0x7ff83c0d3e50 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 -- 172.21.15.40:0/2245040906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 msgr2=0x7ff83c0cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 --2- 172.21.15.40:0/2245040906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0cfd00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0d2c40 tx=0x7ff83002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 -- 172.21.15.40:0/2245040906 shutdown_connections 2024-04-26T21:12:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 --2- 172.21.15.40:0/2245040906 >> v2:172.21.15.184:3300/0 conn(0x7ff83c0d2230 0x7ff83c0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 --2- 172.21.15.40:0/2245040906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0cfd00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 -- 172.21.15.40:0/2245040906 >> 172.21.15.40:0/2245040906 conn(0x7ff83c0d5160 msgr2=0x7ff83c0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 -- 172.21.15.40:0/2245040906 shutdown_connections 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.516+0000 7ff84099d640 1 -- 172.21.15.40:0/2245040906 wait complete. 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.517+0000 7ff84099d640 1 Processor -- start 2024-04-26T21:12:02.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.517+0000 7ff84099d640 1 -- start start 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff83c0d2230 0x7ff83c1f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 0x7ff83c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d0780 con 0x7ff83c0d3e50 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0c9fd0 con 0x7ff83c0d2230 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff84099d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c0d0900 con 0x7ff83c15f950 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51766/0 (socket says 172.21.15.40:51766) 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 -- 172.21.15.40:0/1161111768 learned_addr learned my addr 172.21.15.40:0/1161111768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff83ad9d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 0x7ff83c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff83a59c640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff83c0d2230 0x7ff83c1f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 -- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 msgr2=0x7ff83c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 0x7ff83c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 -- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff83c0d2230 msgr2=0x7ff83c1f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff83c0d2230 0x7ff83c1f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 -- 172.21.15.40:0/1161111768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0b7f00 con 0x7ff83c0d3e50 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff83ad9d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 0x7ff83c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.518+0000 7ff839d9b640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff83002a220 tx=0x7ff8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.519+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830037040 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.519+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830042070 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.519+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c1f8ee0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.519+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830030bb0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.519+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c0b81c0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.520+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c0693d0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.520+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff830047460 con 0x7ff83c0d3e50 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.521+0000 7ff82affd640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.521+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff8300ac7d0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.521+0000 7ff83a59c640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.521+0000 7ff83a59c640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000f60 tx=0x7ff82c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.525+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff830045360 con 0x7ff83c0d3e50 2024-04-26T21:12:02.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.750+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff83c0b8390 con 0x7ff83c0d3e50 2024-04-26T21:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.760+0000 7ff82affd640 1 -- 172.21.15.40:0/1161111768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7ff83007abf0 con 0x7ff83c0d3e50 2024-04-26T21:12:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 msgr2=0x7ff81c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000f60 tx=0x7ff82c007c00 comp rx=0 tx=0).stop 2024-04-26T21:12:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 msgr2=0x7ff83c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff83002a220 tx=0x7ff8300046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 shutdown_connections 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff83c15f950 0x7ff83c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff83c0d3e50 0x7ff83c0ce240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 --2- 172.21.15.40:0/1161111768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff83c0d2230 0x7ff83c1f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 >> 172.21.15.40:0/1161111768 conn(0x7ff83c0d5160 msgr2=0x7ff83c15e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 shutdown_connections 2024-04-26T21:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.762+0000 7ff84099d640 1 -- 172.21.15.40:0/1161111768 wait complete. 2024-04-26T21:12:02.774 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:12:02.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:12:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.877+0000 7f932b704640 1 Processor -- start 2024-04-26T21:12:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.877+0000 7f932b704640 1 -- start start 2024-04-26T21:12:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f932b704640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f932415f470 0x7f93240af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f932b704640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93240b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f932b704640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93240aeb50 con 0x7f93240b03e0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f932b704640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f93240aee30 con 0x7f932415f470 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f932b704640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93240af110 con 0x7f9324069ae0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f9329ca1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f93240b03e0 0x7f932406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46178/0 (socket says 172.21.15.40:46178) 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f9329ca1640 1 -- 172.21.15.40:0/2840632955 learned_addr learned my addr 172.21.15.40:0/2840632955 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f9328c9f640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93240b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f93294a0640 1 --2- 172.21.15.40:0/2840632955 >> v2:172.21.15.184:3300/0 conn(0x7f932415f470 0x7f93240af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254178981 0 0) 0x7f93240aeb50 con 0x7f93240b03e0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9310003520 con 0x7f93240b03e0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617538240 0 0) 0x7f9310003520 con 0x7f93240b03e0 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 msgr2=0x7f93240b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93240b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 >> v2:172.21.15.184:3300/0 conn(0x7f932415f470 msgr2=0x7f93240af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 --2- 172.21.15.40:0/2840632955 >> v2:172.21.15.184:3300/0 conn(0x7f932415f470 0x7f93240af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93240af3f0 con 0x7f93240b03e0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.878+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f931c001670 con 0x7f93240b03e0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 >> v1:172.21.15.40:6789/0 conn(0x7f93240b03e0 legacy=0x7f932406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9310004800 0x7f9310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93240aee30 con 0x7f93100039b0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93240af110 con 0x7f9310004800 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f931000b9c0 con 0x7f9324069ae0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f9328c9f640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9310004800 0x7f9310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f9329ca1640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 -- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 msgr2=0x7f93100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 -- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9310004800 msgr2=0x7f9310004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9310004800 0x7f9310004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f93294a0640 1 -- 172.21.15.40:0/2840632955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9310003520 con 0x7f93100039b0 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.879+0000 7f9329ca1640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9324069ae0 0x7f93100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f93294a0640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9320009fc0 tx=0x7f9310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a05370150a817b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320009c10 con 0x7f93100039b0 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320036070 con 0x7f93100039b0 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f931bfff640 1 -- 172.21.15.40:0/2840632955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932002d070 con 0x7f93100039b0 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 -- 172.21.15.40:0/2840632955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100039b0 msgr2=0x7f9310003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 --2- 172.21.15.40:0/2840632955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9320009fc0 tx=0x7f9310005830 comp rx=0 tx=0).stop 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f9329ca1640 1 -- 172.21.15.40:0/2840632955 reap_dead start 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 -- 172.21.15.40:0/2840632955 shutdown_connections 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 -- 172.21.15.40:0/2840632955 >> 172.21.15.40:0/2840632955 conn(0x7f9324073910 msgr2=0x7f9324073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:02.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 -- 172.21.15.40:0/2840632955 shutdown_connections 2024-04-26T21:12:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.880+0000 7f932b704640 1 -- 172.21.15.40:0/2840632955 wait complete. 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 Processor -- start 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 -- start start 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93100039b0 0x7f932415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 0x7f9324156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93240b63a0 con 0x7f93241f2410 2024-04-26T21:12:02.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93240b6220 con 0x7f9324156840 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f932b704640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f93240afe50 con 0x7f93100039b0 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f9328c9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f9328c9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51776/0 (socket says 172.21.15.40:51776) 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.882+0000 7f9328c9f640 1 -- 172.21.15.40:0/1168467241 learned_addr learned my addr 172.21.15.40:0/1168467241 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f93294a0640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93100039b0 0x7f932415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9329ca1640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 0x7f9324156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 -- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93100039b0 msgr2=0x7f932415cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93100039b0 0x7f932415cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 -- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 msgr2=0x7f9324156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 0x7f9324156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 -- 172.21.15.40:0/1168467241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9324157200 con 0x7f93241f2410 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9329ca1640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 0x7f9324156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9328c9f640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9314011940 tx=0x7f9314011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314019070 con 0x7f93241f2410 2024-04-26T21:12:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9314004240 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93241f4eb0 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931400f8e0 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.883+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241f5470 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.884+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932406a290 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.885+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931401c070 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.885+0000 7f9319ffb640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 0x7f930806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.885+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f931408bf30 con 0x7f93241f2410 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.885+0000 7f93294a0640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 0x7f930806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.886+0000 7f93294a0640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 0x7f930806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9320035000 tx=0x7f932002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:02.890+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9314059d10 con 0x7f93241f2410 2024-04-26T21:12:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.116+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f932406d520 con 0x7f93241f2410 2024-04-26T21:12:03.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.768+0000 7f9319ffb640 1 -- 172.21.15.40:0/1168467241 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f931405e010 con 0x7f93241f2410 2024-04-26T21:12:03.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 msgr2=0x7f930806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 0x7f930806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9320035000 tx=0x7f932002f000 comp rx=0 tx=0).stop 2024-04-26T21:12:03.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 msgr2=0x7f932415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9314011940 tx=0x7f9314011e00 comp rx=0 tx=0).stop 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 shutdown_connections 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f930806c8f0 0x7f930806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9324156840 0x7f9324156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93241f2410 0x7f932415d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 --2- 172.21.15.40:0/1168467241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93100039b0 0x7f932415cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 >> 172.21.15.40:0/1168467241 conn(0x7f9324073910 msgr2=0x7f932406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 shutdown_connections 2024-04-26T21:12:03.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.771+0000 7f932b704640 1 -- 172.21.15.40:0/1168467241 wait complete. 2024-04-26T21:12:03.785 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:12:03.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:03.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.891+0000 7f7d3376a640 1 Processor -- start 2024-04-26T21:12:03.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.891+0000 7f7d3376a640 1 -- start start 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d3376a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d2c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d3376a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d2c0d03a0 con 0x7f7d2c15f750 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d3376a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d2c0d0680 con 0x7f7d2c15f370 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d3376a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2c0d0960 con 0x7f7d2c0d3ee0 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d31d07640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7d2c15f750 0x7f7d2c0b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46188/0 (socket says 172.21.15.40:46188) 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d31d07640 1 -- 172.21.15.40:0/3238961409 learned_addr learned my addr 172.21.15.40:0/3238961409 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.892+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d2c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186397321 0 0) 0x7f7d2c0d03a0 con 0x7f7d2c15f750 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d18003520 con 0x7f7d2c15f750 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 425087904 0 0) 0x7f7d18003520 con 0x7f7d2c15f750 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 msgr2=0x7f7d2c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:03.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d2c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 >> v1:172.21.15.184:6789/0 conn(0x7f7d2c15f370 legacy=0x7f7d2c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c0d0c40 con 0x7f7d2c15f750 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7d240019f0 con 0x7f7d2c15f750 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 >> v1:172.21.15.40:6789/0 conn(0x7f7d2c15f750 legacy=0x7f7d2c0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d180039e0 0x7f7d18003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d18004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d180050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c0d0960 con 0x7f7d18004390 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1800b9c0 con 0x7f7d180039e0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1800bbc0 con 0x7f7d2c0d3ee0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d18004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d31506640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d180039e0 0x7f7d18003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d31d07640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d180050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 -- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 msgr2=0x7f7d180050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:03.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0d3ee0 0x7f7d180050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 -- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d180039e0 msgr2=0x7f7d18003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:03.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d180039e0 0x7f7d18003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.893+0000 7f7d30d05640 1 -- 172.21.15.40:0/3238961409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d18003520 con 0x7f7d18004390 2024-04-26T21:12:03.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.894+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d18004860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ac0 tx=0x7f7d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742fca3d0a4f6c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.894+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c036070 con 0x7f7d18004390 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.894+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1c031400 con 0x7f7d18004390 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.894+0000 7f7d23fff640 1 -- 172.21.15.40:0/3238961409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009690 con 0x7f7d18004390 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 -- 172.21.15.40:0/3238961409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 msgr2=0x7f7d18004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3238961409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d18004860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ac0 tx=0x7f7d18005830 comp rx=0 tx=0).stop 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d31d07640 1 -- 172.21.15.40:0/3238961409 reap_dead start 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 -- 172.21.15.40:0/3238961409 shutdown_connections 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 -- 172.21.15.40:0/3238961409 >> 172.21.15.40:0/3238961409 conn(0x7f7d2c08ec10 msgr2=0x7f7d2c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 -- 172.21.15.40:0/3238961409 shutdown_connections 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.895+0000 7f7d3376a640 1 -- 172.21.15.40:0/3238961409 wait complete. 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.896+0000 7f7d3376a640 1 Processor -- start 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.896+0000 7f7d3376a640 1 -- start start 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0cac90 0x7f7d2c161e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d2c0cb110 0x7f7d2c162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c0cfef0 con 0x7f7d18004390 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c0cfd70 con 0x7f7d2c0cb110 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d3376a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2c0d0070 con 0x7f7d2c0cac90 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51794/0 (socket says 172.21.15.40:51794) 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 -- 172.21.15.40:0/3630630276 learned_addr learned my addr 172.21.15.40:0/3630630276 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0cac90 0x7f7d2c161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31d07640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d2c0cb110 0x7f7d2c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 -- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0cac90 msgr2=0x7f7d2c161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0cac90 0x7f7d2c161e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 -- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d2c0cb110 msgr2=0x7f7d2c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d2c0cb110 0x7f7d2c162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 -- 172.21.15.40:0/3630630276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c0cc3f0 con 0x7f7d18004390 2024-04-26T21:12:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.897+0000 7f7d31506640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800f820 tx=0x7f7d2800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.898+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800fe70 con 0x7f7d18004390 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.898+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d28018070 con 0x7f7d18004390 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.898+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c0cc690 con 0x7f7d18004390 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.898+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800d6d0 con 0x7f7d18004390 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.898+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c0ccc50 con 0x7f7d18004390 2024-04-26T21:12:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.899+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2c06a290 con 0x7f7d18004390 2024-04-26T21:12:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.899+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d2800d870 con 0x7f7d18004390 2024-04-26T21:12:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.900+0000 7f7d21ffb640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 0x7f7d1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.900+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7d2808b960 con 0x7f7d18004390 2024-04-26T21:12:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.900+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 0x7f7d1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.901+0000 7f7d30d05640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 0x7f7d1006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ac0 tx=0x7f7d1c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:03.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:03.904+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d28059d80 con 0x7f7d18004390 2024-04-26T21:12:04.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.128+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d2c0d1330 con 0x7f7d18004390 2024-04-26T21:12:04.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.129+0000 7f7d21ffb640 1 -- 172.21.15.40:0/3630630276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f7d28059680 con 0x7f7d18004390 2024-04-26T21:12:04.130 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:04.130 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:04.131 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 msgr2=0x7f7d1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 0x7f7d1006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ac0 tx=0x7f7d1c008030 comp rx=0 tx=0).stop 2024-04-26T21:12:04.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 msgr2=0x7f7d2c15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800f820 tx=0x7f7d2800fc20 comp rx=0 tx=0).stop 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 shutdown_connections 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d1006c920 0x7f7d1006ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d2c0cb110 0x7f7d2c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d2c0cac90 0x7f7d2c161e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 --2- 172.21.15.40:0/3630630276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d18004390 0x7f7d2c15d8f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 >> 172.21.15.40:0/3630630276 conn(0x7f7d2c08ec10 msgr2=0x7f7d2c06de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 shutdown_connections 2024-04-26T21:12:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.131+0000 7f7d3376a640 1 -- 172.21.15.40:0/3630630276 wait complete. 2024-04-26T21:12:04.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.243+0000 7f29a24d1640 1 Processor -- start 2024-04-26T21:12:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.243+0000 7f29a24d1640 1 -- start start 2024-04-26T21:12:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f29a24d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c15f340 0x7f299c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f29a24d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f299c15f720 0x7f299c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f29a24d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0b2510 con 0x7f299c0d4f00 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f29a24d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f299c0b27f0 con 0x7f299c15f720 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f29a24d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0b2ad0 con 0x7f299c15f340 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f299c15f720 0x7f299c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f299c15f720 0x7f299c0d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51798/0 (socket says 172.21.15.40:51798) 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 -- 172.21.15.40:0/2539295796 learned_addr learned my addr 172.21.15.40:0/2539295796 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299bfff640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c15f340 0x7f299c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 -- 172.21.15.40:0/2539295796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c15f340 msgr2=0x7f299c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c15f340 0x7f299c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 -- 172.21.15.40:0/2539295796 >> v1:172.21.15.40:6789/0 conn(0x7f299c0d4f00 legacy=0x7f299c0d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299affd640 1 -- 172.21.15.40:0/2539295796 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1552004127 0 0) 0x7f299c0b2510 con 0x7f299c0d4f00 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.244+0000 7f299b7fe640 1 -- 172.21.15.40:0/2539295796 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0b2db0 con 0x7f299c15f720 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.245+0000 7f299b7fe640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0d3d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f298c009870 tx=0x7f298c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7118e0916c1d1ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.245+0000 7f299affd640 1 -- 172.21.15.40:0/2539295796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c037070 con 0x7f299c15f720 2024-04-26T21:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.245+0000 7f299affd640 1 -- 172.21.15.40:0/2539295796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298c032440 con 0x7f299c15f720 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.245+0000 7f299affd640 1 -- 172.21.15.40:0/2539295796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c032810 con 0x7f299c15f720 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 -- 172.21.15.40:0/2539295796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 msgr2=0x7f299c0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0d3d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f298c009870 tx=0x7f298c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 -- 172.21.15.40:0/2539295796 shutdown_connections 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0d3d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 --2- 172.21.15.40:0/2539295796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c15f340 0x7f299c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 -- 172.21.15.40:0/2539295796 >> 172.21.15.40:0/2539295796 conn(0x7f299c08ec70 msgr2=0x7f299c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 -- 172.21.15.40:0/2539295796 shutdown_connections 2024-04-26T21:12:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.246+0000 7f29a24d1640 1 -- 172.21.15.40:0/2539295796 wait complete. 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.247+0000 7f29a24d1640 1 Processor -- start 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.247+0000 7f29a24d1640 1 -- start start 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 0x7f299c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 0x7f299c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b3980 con 0x7f299c15f720 2024-04-26T21:12:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b3800 con 0x7f299c15f340 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a24d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0b3b00 con 0x7f299c0d4f00 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51814/0 (socket says 172.21.15.40:51814) 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 -- 172.21.15.40:0/1607421364 learned_addr learned my addr 172.21.15.40:0/1607421364 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f299bfff640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 0x7f299c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f299b7fe640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 0x7f299c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 -- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 msgr2=0x7f299c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 0x7f299c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 -- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 msgr2=0x7f299c0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 0x7f299c0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 -- 172.21.15.40:0/1607421364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0bad90 con 0x7f299c15f720 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f299b7fe640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 0x7f299c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f299bfff640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 0x7f299c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.248+0000 7f29a0a6e640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f299400c760 tx=0x7f299400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.249+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994003da0 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.249+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2994005070 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.249+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994005460 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.249+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c0b9cf0 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.249+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c0ba2b0 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.250+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c06a290 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.250+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2994005600 con 0x7f299c15f720 2024-04-26T21:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.251+0000 7f2998ff9640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 0x7f298806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.251+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f299408b9d0 con 0x7f299c15f720 2024-04-26T21:12:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.251+0000 7f299bfff640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 0x7f298806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.251+0000 7f299bfff640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 0x7f298806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.255+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2994059e90 con 0x7f299c15f720 2024-04-26T21:12:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.438+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f299c0d3020 con 0x7f299c15f720 2024-04-26T21:12:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.439+0000 7f2998ff9640 1 -- 172.21.15.40:0/1607421364 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f2994059790 con 0x7f299c15f720 2024-04-26T21:12:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:04.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 msgr2=0x7f298806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 0x7f298806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 msgr2=0x7f299c0cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f299400c760 tx=0x7f299400cc20 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 shutdown_connections 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f298806c8f0 0x7f298806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f299c15f720 0x7f299c0cfd20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f299c15f340 0x7f299c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 --2- 172.21.15.40:0/1607421364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f299c0d4f00 0x7f299c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 >> 172.21.15.40:0/1607421364 conn(0x7f299c08ec70 msgr2=0x7f299c0b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 shutdown_connections 2024-04-26T21:12:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.441+0000 7f29a24d1640 1 -- 172.21.15.40:0/1607421364 wait complete. 2024-04-26T21:12:04.453 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:12:04.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:04.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 Processor -- start 2024-04-26T21:12:04.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 -- start start 2024-04-26T21:12:04.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f114c15f3e0 0x7f114c0d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f114c0cfce0 0x7f114c0d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f114c0d05f0 con 0x7f114c0cfce0 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f114c0d08d0 con 0x7f114c15f3e0 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.554+0000 7f1153208640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f114c0d0bb0 con 0x7f114c0d0170 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f114c0cfce0 0x7f114c0d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f114c0cfce0 0x7f114c0d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51826/0 (socket says 172.21.15.40:51826) 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 -- 172.21.15.40:0/1106778704 learned_addr learned my addr 172.21.15.40:0/1106778704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f11517a5640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f1150fa4640 1 --2- 172.21.15.40:0/1106778704 >> v2:172.21.15.184:3300/0 conn(0x7f114c15f3e0 0x7f114c0d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 -- 172.21.15.40:0/1106778704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 msgr2=0x7f114c0d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0d5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 -- 172.21.15.40:0/1106778704 >> v2:172.21.15.184:3300/0 conn(0x7f114c15f3e0 msgr2=0x7f114c0d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 --2- 172.21.15.40:0/1106778704 >> v2:172.21.15.184:3300/0 conn(0x7f114c15f3e0 0x7f114c0d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.555+0000 7f114bfff640 1 -- 172.21.15.40:0/1106778704 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c0d0e90 con 0x7f114c0cfce0 2024-04-26T21:12:04.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f114bfff640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c0cfce0 0x7f114c0d4df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f114c06cea0 tx=0x7f113402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7f4139c52affae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f114b7fe640 1 -- 172.21.15.40:0/1106778704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134037070 con 0x7f114c0cfce0 2024-04-26T21:12:04.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f114b7fe640 1 -- 172.21.15.40:0/1106778704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1134032440 con 0x7f114c0cfce0 2024-04-26T21:12:04.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f114b7fe640 1 -- 172.21.15.40:0/1106778704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134032810 con 0x7f114c0cfce0 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 -- 172.21.15.40:0/1106778704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c0cfce0 msgr2=0x7f114c0d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c0cfce0 0x7f114c0d4df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f114c06cea0 tx=0x7f113402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 -- 172.21.15.40:0/1106778704 shutdown_connections 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0d5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 --2- 172.21.15.40:0/1106778704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c0cfce0 0x7f114c0d4df0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 --2- 172.21.15.40:0/1106778704 >> v2:172.21.15.184:3300/0 conn(0x7f114c15f3e0 0x7f114c0d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 -- 172.21.15.40:0/1106778704 >> 172.21.15.40:0/1106778704 conn(0x7f114c08ebf0 msgr2=0x7f114c09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 -- 172.21.15.40:0/1106778704 shutdown_connections 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.556+0000 7f1153208640 1 -- 172.21.15.40:0/1106778704 wait complete. 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.557+0000 7f1153208640 1 Processor -- start 2024-04-26T21:12:04.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 -- start start 2024-04-26T21:12:04.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f114c0cfce0 0x7f114c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0ba890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0d1d40 con 0x7f114c15f3e0 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0d1bc0 con 0x7f114c0cfce0 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f1153208640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f114c0d1ec0 con 0x7f114c0d0170 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.558+0000 7f11517a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51842/0 (socket says 172.21.15.40:51842) 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 -- 172.21.15.40:0/4225615064 learned_addr learned my addr 172.21.15.40:0/4225615064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f114bfff640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f1150fa4640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f114c0cfce0 0x7f114c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 -- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 msgr2=0x7f114c0ba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0ba890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 -- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f114c0cfce0 msgr2=0x7f114c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f114c0cfce0 0x7f114c0c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 -- 172.21.15.40:0/4225615064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c0ba530 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11517a5640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c5c0 tx=0x7f113c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c015070 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c004220 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c0c92e0 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c003610 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.559+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c0c98a0 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.560+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c06a290 con 0x7f114c15f3e0 2024-04-26T21:12:04.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.561+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f113c0037b0 con 0x7f114c15f3e0 2024-04-26T21:12:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.561+0000 7f11497fa640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 0x7f112c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.561+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f113c08b8a0 con 0x7f114c15f3e0 2024-04-26T21:12:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.561+0000 7f1150fa4640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 0x7f112c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.562+0000 7f1150fa4640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 0x7f112c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1144007f60 tx=0x7f1144007710 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.565+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f113c059680 con 0x7f114c15f3e0 2024-04-26T21:12:04.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.747+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f114c0d2570 con 0x7f112c06c8f0 2024-04-26T21:12:04.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.753+0000 7f11497fa640 1 -- 172.21.15.40:0/4225615064 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237536 (secure 0 0 0) 0x7f114c0d2570 con 0x7f112c06c8f0 2024-04-26T21:12:04.757 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:04.758 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 msgr2=0x7f112c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 0x7f112c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1144007f60 tx=0x7f1144007710 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 msgr2=0x7f114c1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c5c0 tx=0x7f113c00ca80 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 shutdown_connections 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f112c06c8f0 0x7f112c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f114c15f3e0 0x7f114c1587a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f114c0d0170 0x7f114c0ba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 --2- 172.21.15.40:0/4225615064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f114c0cfce0 0x7f114c0c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 >> 172.21.15.40:0/4225615064 conn(0x7f114c08ebf0 msgr2=0x7f114c0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 shutdown_connections 2024-04-26T21:12:04.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.757+0000 7f1153208640 1 -- 172.21.15.40:0/4225615064 wait complete. 2024-04-26T21:12:04.768 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-26T21:12:02.918173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":219028,"kb_used_data":2612,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755692,"statfs":{"total":773094113280,"available":772869828608,"internally_reserved":0,"allocated":2674688,"data_stored":1635097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000673"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729661+0000","last_change":"2024-04-26T21:11:59.729661+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729661+0000","last_undegraded":"2024-04-26T21:11:59.729661+0000","last_fullsized":"2024-04-26T21:11:59.729661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729179+0000","last_change":"2024-04-26T21:11:59.729179+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729179+0000","last_undegraded":"2024-04-26T21:11:59.729179+0000","last_fullsized":"2024-04-26T21:11:59.729179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:11:52.170694+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:11:52.170694+0000","last_peered":"2024-04-26T21:11:52.170694+0000","last_clean":"2024-04-26T21:11:52.170694+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:11:52.170694+0000","last_undegraded":"2024-04-26T21:11:52.170694+0000","last_fullsized":"2024-04-26T21:11:52.170694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728962+0000","last_change":"2024-04-26T21:11:59.728962+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728962+0000","last_undegraded":"2024-04-26T21:11:59.728962+0000","last_fullsized":"2024-04-26T21:11:59.728962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728646+0000","last_change":"2024-04-26T21:11:59.728646+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728646+0000","last_undegraded":"2024-04-26T21:11:59.728646+0000","last_fullsized":"2024-04-26T21:11:59.728646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728806+0000","last_change":"2024-04-26T21:11:59.728806+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728806+0000","last_undegraded":"2024-04-26T21:11:59.728806+0000","last_fullsized":"2024-04-26T21:11:59.728806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728592+0000","last_change":"2024-04-26T21:11:59.728592+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728592+0000","last_undegraded":"2024-04-26T21:11:59.728592+0000","last_fullsized":"2024-04-26T21:11:59.728592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:11:59.713707+0000","last_change":"2024-04-26T21:11:59.713707+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.713707+0000","last_undegraded":"2024-04-26T21:11:59.713707+0000","last_fullsized":"2024-04-26T21:11:59.713707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":778240,"data_stored":646436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":56068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":56068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":56068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":56068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":646436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:04.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.908+0000 7f7bca607640 1 Processor -- start 2024-04-26T21:12:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.908+0000 7f7bca607640 1 -- start start 2024-04-26T21:12:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bca607640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bca607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bca607640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc40b2510 con 0x7f7bc415f340 2024-04-26T21:12:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bca607640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc40b27f0 con 0x7f7bc415f720 2024-04-26T21:12:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bca607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc40b2ad0 con 0x7f7bc40d5610 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc3fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f7bc415f340 0x7f7bc40d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50008/0 (socket says 172.21.15.40:50008) 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc3fff640 1 -- 172.21.15.40:0/2820397381 learned_addr learned my addr 172.21.15.40:0/2820397381 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49526/0 (socket says 172.21.15.40:49526) 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 --2- 172.21.15.40:0/2820397381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56424/0 (socket says 172.21.15.40:56424) 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 -- 172.21.15.40:0/2820397381 >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 msgr2=0x7f7bc40b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 --2- 172.21.15.40:0/2820397381 >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 -- 172.21.15.40:0/2820397381 >> v1:172.21.15.184:6789/0 conn(0x7f7bc415f340 legacy=0x7f7bc40d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc8ba4640 1 -- 172.21.15.40:0/2820397381 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc40b2db0 con 0x7f7bc40d5610 2024-04-26T21:12:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.909+0000 7f7bc37fe640 1 --2- 172.21.15.40:0/2820397381 >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.910+0000 7f7bc8ba4640 1 --2- 172.21.15.40:0/2820397381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009d90 tx=0x7f7bbc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9808a25c5d96abbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.910+0000 7f7bc2ffd640 1 -- 172.21.15.40:0/2820397381 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc031600 con 0x7f7bc40d5610 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.910+0000 7f7bc2ffd640 1 -- 172.21.15.40:0/2820397381 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0317a0 con 0x7f7bc40d5610 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.910+0000 7f7bc2ffd640 1 -- 172.21.15.40:0/2820397381 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc031b70 con 0x7f7bc40d5610 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 -- 172.21.15.40:0/2820397381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 msgr2=0x7f7bc40d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 --2- 172.21.15.40:0/2820397381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009d90 tx=0x7f7bbc02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 -- 172.21.15.40:0/2820397381 shutdown_connections 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 --2- 172.21.15.40:0/2820397381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40d3d40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 --2- 172.21.15.40:0/2820397381 >> v2:172.21.15.184:3300/0 conn(0x7f7bc415f720 0x7f7bc40b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 -- 172.21.15.40:0/2820397381 >> 172.21.15.40:0/2820397381 conn(0x7f7bc408ec70 msgr2=0x7f7bc409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 -- 172.21.15.40:0/2820397381 shutdown_connections 2024-04-26T21:12:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.911+0000 7f7bca607640 1 -- 172.21.15.40:0/2820397381 wait complete. 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.912+0000 7f7bca607640 1 Processor -- start 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.912+0000 7f7bca607640 1 -- start start 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc415f720 0x7f7bc40b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40b3940 con 0x7f7bc415f340 2024-04-26T21:12:04.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40b37c0 con 0x7f7bc415f720 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bca607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc40b3ac0 con 0x7f7bc40d5610 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51850/0 (socket says 172.21.15.40:51850) 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bc37fe640 1 -- 172.21.15.40:0/3267862212 learned_addr learned my addr 172.21.15.40:0/3267862212 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bc3fff640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.913+0000 7f7bc8ba4640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc415f720 0x7f7bc40b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 -- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 msgr2=0x7f7bc40cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 -- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc415f720 msgr2=0x7f7bc40b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc415f720 0x7f7bc40b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 -- 172.21.15.40:0/3267862212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc40ca7f0 con 0x7f7bc415f340 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc37fe640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e930 tx=0x7f7bb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8017070 con 0x7f7bc415f340 2024-04-26T21:12:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb800ca00 con 0x7f7bc415f340 2024-04-26T21:12:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cdf0 con 0x7f7bc415f340 2024-04-26T21:12:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc40caad0 con 0x7f7bc415f340 2024-04-26T21:12:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.914+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc40cb090 con 0x7f7bc415f340 2024-04-26T21:12:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.915+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc40b5bf0 con 0x7f7bc415f340 2024-04-26T21:12:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.915+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bb801a070 con 0x7f7bc415f340 2024-04-26T21:12:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.916+0000 7f7bc0ff9640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.916+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7bb808b9d0 con 0x7f7bc415f340 2024-04-26T21:12:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.916+0000 7f7bc3fff640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.917+0000 7f7bc3fff640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 0x7f7bb006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007fd0 tx=0x7f7bb4007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:04.920+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bc40b5bf0 con 0x7f7bc415f340 2024-04-26T21:12:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.104+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bc406a290 con 0x7f7bb006c920 2024-04-26T21:12:05.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.111+0000 7f7bc0ff9640 1 -- 172.21.15.40:0/3267862212 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250484 (secure 0 0 0) 0x7f7bc406a290 con 0x7f7bb006c920 2024-04-26T21:12:05.114 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 msgr2=0x7f7bb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 0x7f7bb006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007fd0 tx=0x7f7bb4007e80 comp rx=0 tx=0).stop 2024-04-26T21:12:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 msgr2=0x7f7bc40ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e930 tx=0x7f7bb800edf0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 shutdown_connections 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bc415f720 0x7f7bc40b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc415f340 0x7f7bc40ca180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 --2- 172.21.15.40:0/3267862212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bc40d5610 0x7f7bc40cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 >> 172.21.15.40:0/3267862212 conn(0x7f7bc408ec70 msgr2=0x7f7bc40ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 shutdown_connections 2024-04-26T21:12:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.114+0000 7f7bca607640 1 -- 172.21.15.40:0/3267862212 wait complete. 2024-04-26T21:12:05.126 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-26T21:12:04.918506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1771826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.195155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775824+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:01.775824+0000","last_peered":"2024-04-26T21:12:01.775824+0000","last_clean":"2024-04-26T21:12:01.775824+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:01.775824+0000","last_undegraded":"2024-04-26T21:12:01.775824+0000","last_fullsized":"2024-04-26T21:12:01.775824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768415+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:01.768415+0000","last_peered":"2024-04-26T21:12:01.768415+0000","last_clean":"2024-04-26T21:12:01.768415+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:01.768415+0000","last_undegraded":"2024-04-26T21:12:01.768415+0000","last_fullsized":"2024-04-26T21:12:01.768415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769579+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:01.769579+0000","last_peered":"2024-04-26T21:12:01.769579+0000","last_clean":"2024-04-26T21:12:01.769579+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:01.769579+0000","last_undegraded":"2024-04-26T21:12:01.769579+0000","last_fullsized":"2024-04-26T21:12:01.769579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775535+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:01.775535+0000","last_peered":"2024-04-26T21:12:01.775535+0000","last_clean":"2024-04-26T21:12:01.775535+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:01.775535+0000","last_undegraded":"2024-04-26T21:12:01.775535+0000","last_fullsized":"2024-04-26T21:12:01.775535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766071+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:01.766071+0000","last_peered":"2024-04-26T21:12:01.766071+0000","last_clean":"2024-04-26T21:12:01.766071+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:01.766071+0000","last_undegraded":"2024-04-26T21:12:01.766071+0000","last_fullsized":"2024-04-26T21:12:01.766071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775714+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:01.775714+0000","last_peered":"2024-04-26T21:12:01.775714+0000","last_clean":"2024-04-26T21:12:01.775714+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:01.775714+0000","last_undegraded":"2024-04-26T21:12:01.775714+0000","last_fullsized":"2024-04-26T21:12:01.775714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780541+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:01.780541+0000","last_peered":"2024-04-26T21:12:01.780541+0000","last_clean":"2024-04-26T21:12:01.780541+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:01.780541+0000","last_undegraded":"2024-04-26T21:12:01.780541+0000","last_fullsized":"2024-04-26T21:12:01.780541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.763195+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:01.763195+0000","last_peered":"2024-04-26T21:12:01.763195+0000","last_clean":"2024-04-26T21:12:01.763195+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:01.763195+0000","last_undegraded":"2024-04-26T21:12:01.763195+0000","last_fullsized":"2024-04-26T21:12:01.763195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773447+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:01.773447+0000","last_peered":"2024-04-26T21:12:01.773447+0000","last_clean":"2024-04-26T21:12:01.773447+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:01.773447+0000","last_undegraded":"2024-04-26T21:12:01.773447+0000","last_fullsized":"2024-04-26T21:12:01.773447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768037+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:01.768037+0000","last_peered":"2024-04-26T21:12:01.768037+0000","last_clean":"2024-04-26T21:12:01.768037+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:01.768037+0000","last_undegraded":"2024-04-26T21:12:01.768037+0000","last_fullsized":"2024-04-26T21:12:01.768037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:06.177139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767765+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:01.767765+0000","last_peered":"2024-04-26T21:12:01.767765+0000","last_clean":"2024-04-26T21:12:01.767765+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:01.767765+0000","last_undegraded":"2024-04-26T21:12:01.767765+0000","last_fullsized":"2024-04-26T21:12:01.767765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766038+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:01.766038+0000","last_peered":"2024-04-26T21:12:01.766038+0000","last_clean":"2024-04-26T21:12:01.766038+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:01.766038+0000","last_undegraded":"2024-04-26T21:12:01.766038+0000","last_fullsized":"2024-04-26T21:12:01.766038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769527+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:01.769527+0000","last_peered":"2024-04-26T21:12:01.769527+0000","last_clean":"2024-04-26T21:12:01.769527+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:01.769527+0000","last_undegraded":"2024-04-26T21:12:01.769527+0000","last_fullsized":"2024-04-26T21:12:01.769527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775703+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:01.775703+0000","last_peered":"2024-04-26T21:12:01.775703+0000","last_clean":"2024-04-26T21:12:01.775703+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:01.775703+0000","last_undegraded":"2024-04-26T21:12:01.775703+0000","last_fullsized":"2024-04-26T21:12:01.775703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770158+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:01.770158+0000","last_peered":"2024-04-26T21:12:01.770158+0000","last_clean":"2024-04-26T21:12:01.770158+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:01.770158+0000","last_undegraded":"2024-04-26T21:12:01.770158+0000","last_fullsized":"2024-04-26T21:12:01.770158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770460+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:01.770460+0000","last_peered":"2024-04-26T21:12:01.770460+0000","last_clean":"2024-04-26T21:12:01.770460+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:01.770460+0000","last_undegraded":"2024-04-26T21:12:01.770460+0000","last_fullsized":"2024-04-26T21:12:01.770460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769708+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:01.769708+0000","last_peered":"2024-04-26T21:12:01.769708+0000","last_clean":"2024-04-26T21:12:01.769708+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:01.769708+0000","last_undegraded":"2024-04-26T21:12:01.769708+0000","last_fullsized":"2024-04-26T21:12:01.769708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769651+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:01.769651+0000","last_peered":"2024-04-26T21:12:01.769651+0000","last_clean":"2024-04-26T21:12:01.769651+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:01.769651+0000","last_undegraded":"2024-04-26T21:12:01.769651+0000","last_fullsized":"2024-04-26T21:12:01.769651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762738+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:01.762738+0000","last_peered":"2024-04-26T21:12:01.762738+0000","last_clean":"2024-04-26T21:12:01.762738+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:01.762738+0000","last_undegraded":"2024-04-26T21:12:01.762738+0000","last_fullsized":"2024-04-26T21:12:01.762738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770442+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:01.770442+0000","last_peered":"2024-04-26T21:12:01.770442+0000","last_clean":"2024-04-26T21:12:01.770442+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:01.770442+0000","last_undegraded":"2024-04-26T21:12:01.770442+0000","last_fullsized":"2024-04-26T21:12:01.770442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769688+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:01.769688+0000","last_peered":"2024-04-26T21:12:01.769688+0000","last_clean":"2024-04-26T21:12:01.769688+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:01.769688+0000","last_undegraded":"2024-04-26T21:12:01.769688+0000","last_fullsized":"2024-04-26T21:12:01.769688+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770730+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:01.770730+0000","last_peered":"2024-04-26T21:12:01.770730+0000","last_clean":"2024-04-26T21:12:01.770730+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:01.770730+0000","last_undegraded":"2024-04-26T21:12:01.770730+0000","last_fullsized":"2024-04-26T21:12:01.770730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775620+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:01.775620+0000","last_peered":"2024-04-26T21:12:01.775620+0000","last_clean":"2024-04-26T21:12:01.775620+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:01.775620+0000","last_undegraded":"2024-04-26T21:12:01.775620+0000","last_fullsized":"2024-04-26T21:12:01.775620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770423+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:01.770423+0000","last_peered":"2024-04-26T21:12:01.770423+0000","last_clean":"2024-04-26T21:12:01.770423+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:01.770423+0000","last_undegraded":"2024-04-26T21:12:01.770423+0000","last_fullsized":"2024-04-26T21:12:01.770423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767658+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:01.767658+0000","last_peered":"2024-04-26T21:12:01.767658+0000","last_clean":"2024-04-26T21:12:01.767658+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:01.767658+0000","last_undegraded":"2024-04-26T21:12:01.767658+0000","last_fullsized":"2024-04-26T21:12:01.767658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745082+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:01.745082+0000","last_peered":"2024-04-26T21:12:01.745082+0000","last_clean":"2024-04-26T21:12:01.745082+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:01.745082+0000","last_undegraded":"2024-04-26T21:12:01.745082+0000","last_fullsized":"2024-04-26T21:12:01.745082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754834+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:01.754834+0000","last_peered":"2024-04-26T21:12:01.754834+0000","last_clean":"2024-04-26T21:12:01.754834+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:01.754834+0000","last_undegraded":"2024-04-26T21:12:01.754834+0000","last_fullsized":"2024-04-26T21:12:01.754834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.755564+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:01.755564+0000","last_peered":"2024-04-26T21:12:01.755564+0000","last_clean":"2024-04-26T21:12:01.755564+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:01.755564+0000","last_undegraded":"2024-04-26T21:12:01.755564+0000","last_fullsized":"2024-04-26T21:12:01.755564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744773+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:01.744773+0000","last_peered":"2024-04-26T21:12:01.744773+0000","last_clean":"2024-04-26T21:12:01.744773+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:01.744773+0000","last_undegraded":"2024-04-26T21:12:01.744773+0000","last_fullsized":"2024-04-26T21:12:01.744773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768019+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:01.768019+0000","last_peered":"2024-04-26T21:12:01.768019+0000","last_clean":"2024-04-26T21:12:01.768019+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:01.768019+0000","last_undegraded":"2024-04-26T21:12:01.768019+0000","last_fullsized":"2024-04-26T21:12:01.768019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729661+0000","last_change":"2024-04-26T21:11:59.729661+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729661+0000","last_undegraded":"2024-04-26T21:11:59.729661+0000","last_fullsized":"2024-04-26T21:11:59.729661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770745+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:01.770745+0000","last_peered":"2024-04-26T21:12:01.770745+0000","last_clean":"2024-04-26T21:12:01.770745+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:01.770745+0000","last_undegraded":"2024-04-26T21:12:01.770745+0000","last_fullsized":"2024-04-26T21:12:01.770745+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.778176+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:01.778176+0000","last_peered":"2024-04-26T21:12:01.778176+0000","last_clean":"2024-04-26T21:12:01.778176+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:01.778176+0000","last_undegraded":"2024-04-26T21:12:01.778176+0000","last_fullsized":"2024-04-26T21:12:01.778176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776418+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:01.776418+0000","last_peered":"2024-04-26T21:12:01.776418+0000","last_clean":"2024-04-26T21:12:01.776418+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:01.776418+0000","last_undegraded":"2024-04-26T21:12:01.776418+0000","last_fullsized":"2024-04-26T21:12:01.776418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770363+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:01.770363+0000","last_peered":"2024-04-26T21:12:01.770363+0000","last_clean":"2024-04-26T21:12:01.770363+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:01.770363+0000","last_undegraded":"2024-04-26T21:12:01.770363+0000","last_fullsized":"2024-04-26T21:12:01.770363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768054+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:01.768054+0000","last_peered":"2024-04-26T21:12:01.768054+0000","last_clean":"2024-04-26T21:12:01.768054+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:01.768054+0000","last_undegraded":"2024-04-26T21:12:01.768054+0000","last_fullsized":"2024-04-26T21:12:01.768054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745579+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:01.745579+0000","last_peered":"2024-04-26T21:12:01.745579+0000","last_clean":"2024-04-26T21:12:01.745579+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:01.745579+0000","last_undegraded":"2024-04-26T21:12:01.745579+0000","last_fullsized":"2024-04-26T21:12:01.745579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775512+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:01.775512+0000","last_peered":"2024-04-26T21:12:01.775512+0000","last_clean":"2024-04-26T21:12:01.775512+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:01.775512+0000","last_undegraded":"2024-04-26T21:12:01.775512+0000","last_fullsized":"2024-04-26T21:12:01.775512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754709+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:01.754709+0000","last_peered":"2024-04-26T21:12:01.754709+0000","last_clean":"2024-04-26T21:12:01.754709+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:01.754709+0000","last_undegraded":"2024-04-26T21:12:01.754709+0000","last_fullsized":"2024-04-26T21:12:01.754709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770711+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:01.770711+0000","last_peered":"2024-04-26T21:12:01.770711+0000","last_clean":"2024-04-26T21:12:01.770711+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:01.770711+0000","last_undegraded":"2024-04-26T21:12:01.770711+0000","last_fullsized":"2024-04-26T21:12:01.770711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775574+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:01.775574+0000","last_peered":"2024-04-26T21:12:01.775574+0000","last_clean":"2024-04-26T21:12:01.775574+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:01.775574+0000","last_undegraded":"2024-04-26T21:12:01.775574+0000","last_fullsized":"2024-04-26T21:12:01.775574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771315+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:01.771315+0000","last_peered":"2024-04-26T21:12:01.771315+0000","last_clean":"2024-04-26T21:12:01.771315+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:01.771315+0000","last_undegraded":"2024-04-26T21:12:01.771315+0000","last_fullsized":"2024-04-26T21:12:01.771315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775488+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:01.775488+0000","last_peered":"2024-04-26T21:12:01.775488+0000","last_clean":"2024-04-26T21:12:01.775488+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:01.775488+0000","last_undegraded":"2024-04-26T21:12:01.775488+0000","last_fullsized":"2024-04-26T21:12:01.775488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775800+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:01.775800+0000","last_peered":"2024-04-26T21:12:01.775800+0000","last_clean":"2024-04-26T21:12:01.775800+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:01.775800+0000","last_undegraded":"2024-04-26T21:12:01.775800+0000","last_fullsized":"2024-04-26T21:12:01.775800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729179+0000","last_change":"2024-04-26T21:11:59.729179+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729179+0000","last_undegraded":"2024-04-26T21:11:59.729179+0000","last_fullsized":"2024-04-26T21:11:59.729179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769605+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:01.769605+0000","last_peered":"2024-04-26T21:12:01.769605+0000","last_clean":"2024-04-26T21:12:01.769605+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:01.769605+0000","last_undegraded":"2024-04-26T21:12:01.769605+0000","last_fullsized":"2024-04-26T21:12:01.769605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780530+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:01.780530+0000","last_peered":"2024-04-26T21:12:01.780530+0000","last_clean":"2024-04-26T21:12:01.780530+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:01.780530+0000","last_undegraded":"2024-04-26T21:12:01.780530+0000","last_fullsized":"2024-04-26T21:12:01.780530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.761140+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:01.761140+0000","last_peered":"2024-04-26T21:12:01.761140+0000","last_clean":"2024-04-26T21:12:01.761140+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:01.761140+0000","last_undegraded":"2024-04-26T21:12:01.761140+0000","last_fullsized":"2024-04-26T21:12:01.761140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744168+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:01.744168+0000","last_peered":"2024-04-26T21:12:01.744168+0000","last_clean":"2024-04-26T21:12:01.744168+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:01.744168+0000","last_undegraded":"2024-04-26T21:12:01.744168+0000","last_fullsized":"2024-04-26T21:12:01.744168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756819+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:01.756819+0000","last_peered":"2024-04-26T21:12:01.756819+0000","last_clean":"2024-04-26T21:12:01.756819+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:01.756819+0000","last_undegraded":"2024-04-26T21:12:01.756819+0000","last_fullsized":"2024-04-26T21:12:01.756819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768018+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:01.768018+0000","last_peered":"2024-04-26T21:12:01.768018+0000","last_clean":"2024-04-26T21:12:01.768018+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:01.768018+0000","last_undegraded":"2024-04-26T21:12:01.768018+0000","last_fullsized":"2024-04-26T21:12:01.768018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745738+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:01.745738+0000","last_peered":"2024-04-26T21:12:01.745738+0000","last_clean":"2024-04-26T21:12:01.745738+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:01.745738+0000","last_undegraded":"2024-04-26T21:12:01.745738+0000","last_fullsized":"2024-04-26T21:12:01.745738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770852+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:01.770852+0000","last_peered":"2024-04-26T21:12:01.770852+0000","last_clean":"2024-04-26T21:12:01.770852+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:01.770852+0000","last_undegraded":"2024-04-26T21:12:01.770852+0000","last_fullsized":"2024-04-26T21:12:01.770852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728962+0000","last_change":"2024-04-26T21:11:59.728962+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728962+0000","last_undegraded":"2024-04-26T21:11:59.728962+0000","last_fullsized":"2024-04-26T21:11:59.728962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769375+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:01.769375+0000","last_peered":"2024-04-26T21:12:01.769375+0000","last_clean":"2024-04-26T21:12:01.769375+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:01.769375+0000","last_undegraded":"2024-04-26T21:12:01.769375+0000","last_fullsized":"2024-04-26T21:12:01.769375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.760256+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:01.760256+0000","last_peered":"2024-04-26T21:12:01.760256+0000","last_clean":"2024-04-26T21:12:01.760256+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:01.760256+0000","last_undegraded":"2024-04-26T21:12:01.760256+0000","last_fullsized":"2024-04-26T21:12:01.760256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728646+0000","last_change":"2024-04-26T21:11:59.728646+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728646+0000","last_undegraded":"2024-04-26T21:11:59.728646+0000","last_fullsized":"2024-04-26T21:11:59.728646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.774307+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:01.774307+0000","last_peered":"2024-04-26T21:12:01.774307+0000","last_clean":"2024-04-26T21:12:01.774307+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:01.774307+0000","last_undegraded":"2024-04-26T21:12:01.774307+0000","last_fullsized":"2024-04-26T21:12:01.774307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767747+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:01.767747+0000","last_peered":"2024-04-26T21:12:01.767747+0000","last_clean":"2024-04-26T21:12:01.767747+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:01.767747+0000","last_undegraded":"2024-04-26T21:12:01.767747+0000","last_fullsized":"2024-04-26T21:12:01.767747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.272254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756630+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:01.756630+0000","last_peered":"2024-04-26T21:12:01.756630+0000","last_clean":"2024-04-26T21:12:01.756630+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:01.756630+0000","last_undegraded":"2024-04-26T21:12:01.756630+0000","last_fullsized":"2024-04-26T21:12:01.756630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767783+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:01.767783+0000","last_peered":"2024-04-26T21:12:01.767783+0000","last_clean":"2024-04-26T21:12:01.767783+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:01.767783+0000","last_undegraded":"2024-04-26T21:12:01.767783+0000","last_fullsized":"2024-04-26T21:12:01.767783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:55.309625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728806+0000","last_change":"2024-04-26T21:11:59.728806+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728806+0000","last_undegraded":"2024-04-26T21:11:59.728806+0000","last_fullsized":"2024-04-26T21:11:59.728806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728592+0000","last_change":"2024-04-26T21:11:59.728592+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728592+0000","last_undegraded":"2024-04-26T21:11:59.728592+0000","last_fullsized":"2024-04-26T21:11:59.728592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.772470+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:01.772470+0000","last_peered":"2024-04-26T21:12:01.772470+0000","last_clean":"2024-04-26T21:12:01.772470+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:01.772470+0000","last_undegraded":"2024-04-26T21:12:01.772470+0000","last_fullsized":"2024-04-26T21:12:01.772470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762707+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:01.762707+0000","last_peered":"2024-04-26T21:12:01.762707+0000","last_clean":"2024-04-26T21:12:01.762707+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:01.762707+0000","last_undegraded":"2024-04-26T21:12:01.762707+0000","last_fullsized":"2024-04-26T21:12:01.762707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775354+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:01.775354+0000","last_peered":"2024-04-26T21:12:01.775354+0000","last_clean":"2024-04-26T21:12:01.775354+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:01.775354+0000","last_undegraded":"2024-04-26T21:12:01.775354+0000","last_fullsized":"2024-04-26T21:12:01.775354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776943+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:01.776943+0000","last_peered":"2024-04-26T21:12:01.776943+0000","last_clean":"2024-04-26T21:12:01.776943+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:01.776943+0000","last_undegraded":"2024-04-26T21:12:01.776943+0000","last_fullsized":"2024-04-26T21:12:01.776943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775441+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:01.775441+0000","last_peered":"2024-04-26T21:12:01.775441+0000","last_clean":"2024-04-26T21:12:01.775441+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:01.775441+0000","last_undegraded":"2024-04-26T21:12:01.775441+0000","last_fullsized":"2024-04-26T21:12:01.775441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773877+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:01.773877+0000","last_peered":"2024-04-26T21:12:01.773877+0000","last_clean":"2024-04-26T21:12:01.773877+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:01.773877+0000","last_undegraded":"2024-04-26T21:12:01.773877+0000","last_fullsized":"2024-04-26T21:12:01.773877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767996+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:01.767996+0000","last_peered":"2024-04-26T21:12:01.767996+0000","last_clean":"2024-04-26T21:12:01.767996+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:01.767996+0000","last_undegraded":"2024-04-26T21:12:01.767996+0000","last_fullsized":"2024-04-26T21:12:01.767996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771405+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:01.771405+0000","last_peered":"2024-04-26T21:12:01.771405+0000","last_clean":"2024-04-26T21:12:01.771405+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:01.771405+0000","last_undegraded":"2024-04-26T21:12:01.771405+0000","last_fullsized":"2024-04-26T21:12:01.771405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":75061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":665941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:05.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.266+0000 7f60b336a640 1 Processor -- start 2024-04-26T21:12:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.266+0000 7f60b336a640 1 -- start start 2024-04-26T21:12:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b336a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 0x7f60ac0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b336a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 0x7f60ac0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b336a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac0d03a0 con 0x7f60ac0d37d0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b336a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f60ac0b4d10 con 0x7f60ac15f750 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b336a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac0d0680 con 0x7f60ac15f370 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1907640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f60ac0d37d0 0x7f60ac0d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46202/0 (socket says 172.21.15.40:46202) 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1907640 1 -- 172.21.15.40:0/3705087335 learned_addr learned my addr 172.21.15.40:0/3705087335 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1106640 1 --2- 172.21.15.40:0/3705087335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 0x7f60ac0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b0905640 1 --2- 172.21.15.40:0/3705087335 >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 0x7f60ac0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955940573 0 0) 0x7f60ac0d03a0 con 0x7f60ac0d37d0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6098003520 con 0x7f60ac0d37d0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 495438393 0 0) 0x7f6098003520 con 0x7f60ac0d37d0 2024-04-26T21:12:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1106640 1 -- 172.21.15.40:0/3705087335 >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 msgr2=0x7f60ac0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1106640 1 --2- 172.21.15.40:0/3705087335 >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 0x7f60ac0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1106640 1 -- 172.21.15.40:0/3705087335 >> v1:172.21.15.40:6789/0 conn(0x7f60ac0d37d0 legacy=0x7f60ac0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b1106640 1 -- 172.21.15.40:0/3705087335 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac0d0960 con 0x7f60ac15f370 2024-04-26T21:12:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.267+0000 7f60b0905640 1 --2- 172.21.15.40:0/3705087335 >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 0x7f60ac0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:05.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.268+0000 7f60b1106640 1 --2- 172.21.15.40:0/3705087335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 0x7f60ac0cfd70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60a80097a0 tx=0x7f60a802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f125d776154c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.268+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8037070 con 0x7f60ac15f370 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.268+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a802ab10 con 0x7f60ac15f370 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.268+0000 7f60a3fff640 1 -- 172.21.15.40:0/3705087335 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40019f0 con 0x7f60ac15f370 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 -- 172.21.15.40:0/3705087335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 msgr2=0x7f60ac0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 --2- 172.21.15.40:0/3705087335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 0x7f60ac0cfd70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60a80097a0 tx=0x7f60a802a080 comp rx=0 tx=0).stop 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 -- 172.21.15.40:0/3705087335 shutdown_connections 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 --2- 172.21.15.40:0/3705087335 >> v2:172.21.15.184:3300/0 conn(0x7f60ac15f750 0x7f60ac0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 --2- 172.21.15.40:0/3705087335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f370 0x7f60ac0cfd70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 -- 172.21.15.40:0/3705087335 >> 172.21.15.40:0/3705087335 conn(0x7f60ac08ec10 msgr2=0x7f60ac09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 -- 172.21.15.40:0/3705087335 shutdown_connections 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.269+0000 7f60b336a640 1 -- 172.21.15.40:0/3705087335 wait complete. 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.270+0000 7f60b336a640 1 Processor -- start 2024-04-26T21:12:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.270+0000 7f60b336a640 1 -- start start 2024-04-26T21:12:05.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac15f370 0x7f60ac15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f750 0x7f60ac1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac0b51c0 con 0x7f60ac0d37d0 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac0d0f20 con 0x7f60ac15f370 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b336a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac0b5340 con 0x7f60ac15f750 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51854/0 (socket says 172.21.15.40:51854) 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 -- 172.21.15.40:0/502886469 learned_addr learned my addr 172.21.15.40:0/502886469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1907640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f750 0x7f60ac1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b0905640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac15f370 0x7f60ac15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 -- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f750 msgr2=0x7f60ac1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f750 0x7f60ac1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 -- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac15f370 msgr2=0x7f60ac15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac15f370 0x7f60ac15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.271+0000 7f60b1106640 1 -- 172.21.15.40:0/502886469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac0c8ce0 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60b1106640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098c0 tx=0x7f60a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8037050 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a8043070 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac0b9690 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8006e10 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.272+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac0b9f30 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.273+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac06a290 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.273+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60a803ca10 con 0x7f60ac0d37d0 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.274+0000 7f60a1ffb640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.274+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f60a80ac6d0 con 0x7f60ac0d37d0 2024-04-26T21:12:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.274+0000 7f60b0905640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.275+0000 7f60b0905640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 0x7f609006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60ac15e5e0 tx=0x7f609c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.278+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60a807aa60 con 0x7f60ac0d37d0 2024-04-26T21:12:05.430 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:12:05.429+0000 7fdde7bfa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:12:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.459+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60ac0d5260 con 0x7f609006c8f0 2024-04-26T21:12:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.465+0000 7f60a1ffb640 1 -- 172.21.15.40:0/502886469 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250484 (secure 0 0 0) 0x7f60ac0d5260 con 0x7f609006c8f0 2024-04-26T21:12:05.469 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 msgr2=0x7f609006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 0x7f609006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60ac15e5e0 tx=0x7f609c008040 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 msgr2=0x7f60ac0b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098c0 tx=0x7f60a80046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 shutdown_connections 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60ac15f750 0x7f60ac1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac15f370 0x7f60ac15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 --2- 172.21.15.40:0/502886469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60ac0d37d0 0x7f60ac0b8e40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 >> 172.21.15.40:0/502886469 conn(0x7f60ac08ec10 msgr2=0x7f60ac0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 shutdown_connections 2024-04-26T21:12:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.468+0000 7f60b336a640 1 -- 172.21.15.40:0/502886469 wait complete. 2024-04-26T21:12:05.479 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-26T21:12:04.918506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1771826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.195155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775824+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:01.775824+0000","last_peered":"2024-04-26T21:12:01.775824+0000","last_clean":"2024-04-26T21:12:01.775824+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:01.775824+0000","last_undegraded":"2024-04-26T21:12:01.775824+0000","last_fullsized":"2024-04-26T21:12:01.775824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768415+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:01.768415+0000","last_peered":"2024-04-26T21:12:01.768415+0000","last_clean":"2024-04-26T21:12:01.768415+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:01.768415+0000","last_undegraded":"2024-04-26T21:12:01.768415+0000","last_fullsized":"2024-04-26T21:12:01.768415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769579+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:01.769579+0000","last_peered":"2024-04-26T21:12:01.769579+0000","last_clean":"2024-04-26T21:12:01.769579+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:01.769579+0000","last_undegraded":"2024-04-26T21:12:01.769579+0000","last_fullsized":"2024-04-26T21:12:01.769579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775535+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:01.775535+0000","last_peered":"2024-04-26T21:12:01.775535+0000","last_clean":"2024-04-26T21:12:01.775535+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:01.775535+0000","last_undegraded":"2024-04-26T21:12:01.775535+0000","last_fullsized":"2024-04-26T21:12:01.775535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766071+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:01.766071+0000","last_peered":"2024-04-26T21:12:01.766071+0000","last_clean":"2024-04-26T21:12:01.766071+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:01.766071+0000","last_undegraded":"2024-04-26T21:12:01.766071+0000","last_fullsized":"2024-04-26T21:12:01.766071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775714+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:01.775714+0000","last_peered":"2024-04-26T21:12:01.775714+0000","last_clean":"2024-04-26T21:12:01.775714+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:01.775714+0000","last_undegraded":"2024-04-26T21:12:01.775714+0000","last_fullsized":"2024-04-26T21:12:01.775714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780541+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:01.780541+0000","last_peered":"2024-04-26T21:12:01.780541+0000","last_clean":"2024-04-26T21:12:01.780541+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:01.780541+0000","last_undegraded":"2024-04-26T21:12:01.780541+0000","last_fullsized":"2024-04-26T21:12:01.780541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.763195+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:01.763195+0000","last_peered":"2024-04-26T21:12:01.763195+0000","last_clean":"2024-04-26T21:12:01.763195+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:01.763195+0000","last_undegraded":"2024-04-26T21:12:01.763195+0000","last_fullsized":"2024-04-26T21:12:01.763195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773447+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:01.773447+0000","last_peered":"2024-04-26T21:12:01.773447+0000","last_clean":"2024-04-26T21:12:01.773447+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:01.773447+0000","last_undegraded":"2024-04-26T21:12:01.773447+0000","last_fullsized":"2024-04-26T21:12:01.773447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768037+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:01.768037+0000","last_peered":"2024-04-26T21:12:01.768037+0000","last_clean":"2024-04-26T21:12:01.768037+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:01.768037+0000","last_undegraded":"2024-04-26T21:12:01.768037+0000","last_fullsized":"2024-04-26T21:12:01.768037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:06.177139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767765+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:01.767765+0000","last_peered":"2024-04-26T21:12:01.767765+0000","last_clean":"2024-04-26T21:12:01.767765+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:01.767765+0000","last_undegraded":"2024-04-26T21:12:01.767765+0000","last_fullsized":"2024-04-26T21:12:01.767765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766038+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:01.766038+0000","last_peered":"2024-04-26T21:12:01.766038+0000","last_clean":"2024-04-26T21:12:01.766038+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:01.766038+0000","last_undegraded":"2024-04-26T21:12:01.766038+0000","last_fullsized":"2024-04-26T21:12:01.766038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769527+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:01.769527+0000","last_peered":"2024-04-26T21:12:01.769527+0000","last_clean":"2024-04-26T21:12:01.769527+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:01.769527+0000","last_undegraded":"2024-04-26T21:12:01.769527+0000","last_fullsized":"2024-04-26T21:12:01.769527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775703+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:01.775703+0000","last_peered":"2024-04-26T21:12:01.775703+0000","last_clean":"2024-04-26T21:12:01.775703+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:01.775703+0000","last_undegraded":"2024-04-26T21:12:01.775703+0000","last_fullsized":"2024-04-26T21:12:01.775703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770158+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:01.770158+0000","last_peered":"2024-04-26T21:12:01.770158+0000","last_clean":"2024-04-26T21:12:01.770158+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:01.770158+0000","last_undegraded":"2024-04-26T21:12:01.770158+0000","last_fullsized":"2024-04-26T21:12:01.770158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770460+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:01.770460+0000","last_peered":"2024-04-26T21:12:01.770460+0000","last_clean":"2024-04-26T21:12:01.770460+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:01.770460+0000","last_undegraded":"2024-04-26T21:12:01.770460+0000","last_fullsized":"2024-04-26T21:12:01.770460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769708+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:01.769708+0000","last_peered":"2024-04-26T21:12:01.769708+0000","last_clean":"2024-04-26T21:12:01.769708+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:01.769708+0000","last_undegraded":"2024-04-26T21:12:01.769708+0000","last_fullsized":"2024-04-26T21:12:01.769708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769651+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:01.769651+0000","last_peered":"2024-04-26T21:12:01.769651+0000","last_clean":"2024-04-26T21:12:01.769651+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:01.769651+0000","last_undegraded":"2024-04-26T21:12:01.769651+0000","last_fullsized":"2024-04-26T21:12:01.769651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762738+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:01.762738+0000","last_peered":"2024-04-26T21:12:01.762738+0000","last_clean":"2024-04-26T21:12:01.762738+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:01.762738+0000","last_undegraded":"2024-04-26T21:12:01.762738+0000","last_fullsized":"2024-04-26T21:12:01.762738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770442+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:01.770442+0000","last_peered":"2024-04-26T21:12:01.770442+0000","last_clean":"2024-04-26T21:12:01.770442+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:01.770442+0000","last_undegraded":"2024-04-26T21:12:01.770442+0000","last_fullsized":"2024-04-26T21:12:01.770442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769688+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:01.769688+0000","last_peered":"2024-04-26T21:12:01.769688+0000","last_clean":"2024-04-26T21:12:01.769688+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:01.769688+0000","last_undegraded":"2024-04-26T21:12:01.769688+0000","last_fullsized":"2024-04-26T21:12:01.769688+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770730+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:01.770730+0000","last_peered":"2024-04-26T21:12:01.770730+0000","last_clean":"2024-04-26T21:12:01.770730+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:01.770730+0000","last_undegraded":"2024-04-26T21:12:01.770730+0000","last_fullsized":"2024-04-26T21:12:01.770730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775620+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:01.775620+0000","last_peered":"2024-04-26T21:12:01.775620+0000","last_clean":"2024-04-26T21:12:01.775620+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:01.775620+0000","last_undegraded":"2024-04-26T21:12:01.775620+0000","last_fullsized":"2024-04-26T21:12:01.775620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770423+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:01.770423+0000","last_peered":"2024-04-26T21:12:01.770423+0000","last_clean":"2024-04-26T21:12:01.770423+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:01.770423+0000","last_undegraded":"2024-04-26T21:12:01.770423+0000","last_fullsized":"2024-04-26T21:12:01.770423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767658+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:01.767658+0000","last_peered":"2024-04-26T21:12:01.767658+0000","last_clean":"2024-04-26T21:12:01.767658+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:01.767658+0000","last_undegraded":"2024-04-26T21:12:01.767658+0000","last_fullsized":"2024-04-26T21:12:01.767658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745082+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:01.745082+0000","last_peered":"2024-04-26T21:12:01.745082+0000","last_clean":"2024-04-26T21:12:01.745082+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:01.745082+0000","last_undegraded":"2024-04-26T21:12:01.745082+0000","last_fullsized":"2024-04-26T21:12:01.745082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754834+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:01.754834+0000","last_peered":"2024-04-26T21:12:01.754834+0000","last_clean":"2024-04-26T21:12:01.754834+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:01.754834+0000","last_undegraded":"2024-04-26T21:12:01.754834+0000","last_fullsized":"2024-04-26T21:12:01.754834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.755564+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:01.755564+0000","last_peered":"2024-04-26T21:12:01.755564+0000","last_clean":"2024-04-26T21:12:01.755564+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:01.755564+0000","last_undegraded":"2024-04-26T21:12:01.755564+0000","last_fullsized":"2024-04-26T21:12:01.755564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744773+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:01.744773+0000","last_peered":"2024-04-26T21:12:01.744773+0000","last_clean":"2024-04-26T21:12:01.744773+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:01.744773+0000","last_undegraded":"2024-04-26T21:12:01.744773+0000","last_fullsized":"2024-04-26T21:12:01.744773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768019+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:01.768019+0000","last_peered":"2024-04-26T21:12:01.768019+0000","last_clean":"2024-04-26T21:12:01.768019+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:01.768019+0000","last_undegraded":"2024-04-26T21:12:01.768019+0000","last_fullsized":"2024-04-26T21:12:01.768019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729661+0000","last_change":"2024-04-26T21:11:59.729661+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729661+0000","last_undegraded":"2024-04-26T21:11:59.729661+0000","last_fullsized":"2024-04-26T21:11:59.729661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770745+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:01.770745+0000","last_peered":"2024-04-26T21:12:01.770745+0000","last_clean":"2024-04-26T21:12:01.770745+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:01.770745+0000","last_undegraded":"2024-04-26T21:12:01.770745+0000","last_fullsized":"2024-04-26T21:12:01.770745+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.778176+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:01.778176+0000","last_peered":"2024-04-26T21:12:01.778176+0000","last_clean":"2024-04-26T21:12:01.778176+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:01.778176+0000","last_undegraded":"2024-04-26T21:12:01.778176+0000","last_fullsized":"2024-04-26T21:12:01.778176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776418+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:01.776418+0000","last_peered":"2024-04-26T21:12:01.776418+0000","last_clean":"2024-04-26T21:12:01.776418+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:01.776418+0000","last_undegraded":"2024-04-26T21:12:01.776418+0000","last_fullsized":"2024-04-26T21:12:01.776418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770363+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:01.770363+0000","last_peered":"2024-04-26T21:12:01.770363+0000","last_clean":"2024-04-26T21:12:01.770363+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:01.770363+0000","last_undegraded":"2024-04-26T21:12:01.770363+0000","last_fullsized":"2024-04-26T21:12:01.770363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768054+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:01.768054+0000","last_peered":"2024-04-26T21:12:01.768054+0000","last_clean":"2024-04-26T21:12:01.768054+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:01.768054+0000","last_undegraded":"2024-04-26T21:12:01.768054+0000","last_fullsized":"2024-04-26T21:12:01.768054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745579+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:01.745579+0000","last_peered":"2024-04-26T21:12:01.745579+0000","last_clean":"2024-04-26T21:12:01.745579+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:01.745579+0000","last_undegraded":"2024-04-26T21:12:01.745579+0000","last_fullsized":"2024-04-26T21:12:01.745579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775512+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:01.775512+0000","last_peered":"2024-04-26T21:12:01.775512+0000","last_clean":"2024-04-26T21:12:01.775512+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:01.775512+0000","last_undegraded":"2024-04-26T21:12:01.775512+0000","last_fullsized":"2024-04-26T21:12:01.775512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754709+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:01.754709+0000","last_peered":"2024-04-26T21:12:01.754709+0000","last_clean":"2024-04-26T21:12:01.754709+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:01.754709+0000","last_undegraded":"2024-04-26T21:12:01.754709+0000","last_fullsized":"2024-04-26T21:12:01.754709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770711+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:01.770711+0000","last_peered":"2024-04-26T21:12:01.770711+0000","last_clean":"2024-04-26T21:12:01.770711+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:01.770711+0000","last_undegraded":"2024-04-26T21:12:01.770711+0000","last_fullsized":"2024-04-26T21:12:01.770711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775574+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:01.775574+0000","last_peered":"2024-04-26T21:12:01.775574+0000","last_clean":"2024-04-26T21:12:01.775574+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:01.775574+0000","last_undegraded":"2024-04-26T21:12:01.775574+0000","last_fullsized":"2024-04-26T21:12:01.775574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771315+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:01.771315+0000","last_peered":"2024-04-26T21:12:01.771315+0000","last_clean":"2024-04-26T21:12:01.771315+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:01.771315+0000","last_undegraded":"2024-04-26T21:12:01.771315+0000","last_fullsized":"2024-04-26T21:12:01.771315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775488+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:01.775488+0000","last_peered":"2024-04-26T21:12:01.775488+0000","last_clean":"2024-04-26T21:12:01.775488+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:01.775488+0000","last_undegraded":"2024-04-26T21:12:01.775488+0000","last_fullsized":"2024-04-26T21:12:01.775488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775800+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:01.775800+0000","last_peered":"2024-04-26T21:12:01.775800+0000","last_clean":"2024-04-26T21:12:01.775800+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:01.775800+0000","last_undegraded":"2024-04-26T21:12:01.775800+0000","last_fullsized":"2024-04-26T21:12:01.775800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729179+0000","last_change":"2024-04-26T21:11:59.729179+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729179+0000","last_undegraded":"2024-04-26T21:11:59.729179+0000","last_fullsized":"2024-04-26T21:11:59.729179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769605+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:01.769605+0000","last_peered":"2024-04-26T21:12:01.769605+0000","last_clean":"2024-04-26T21:12:01.769605+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:01.769605+0000","last_undegraded":"2024-04-26T21:12:01.769605+0000","last_fullsized":"2024-04-26T21:12:01.769605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780530+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:01.780530+0000","last_peered":"2024-04-26T21:12:01.780530+0000","last_clean":"2024-04-26T21:12:01.780530+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:01.780530+0000","last_undegraded":"2024-04-26T21:12:01.780530+0000","last_fullsized":"2024-04-26T21:12:01.780530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.761140+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:01.761140+0000","last_peered":"2024-04-26T21:12:01.761140+0000","last_clean":"2024-04-26T21:12:01.761140+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:01.761140+0000","last_undegraded":"2024-04-26T21:12:01.761140+0000","last_fullsized":"2024-04-26T21:12:01.761140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744168+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:01.744168+0000","last_peered":"2024-04-26T21:12:01.744168+0000","last_clean":"2024-04-26T21:12:01.744168+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:01.744168+0000","last_undegraded":"2024-04-26T21:12:01.744168+0000","last_fullsized":"2024-04-26T21:12:01.744168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756819+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:01.756819+0000","last_peered":"2024-04-26T21:12:01.756819+0000","last_clean":"2024-04-26T21:12:01.756819+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:01.756819+0000","last_undegraded":"2024-04-26T21:12:01.756819+0000","last_fullsized":"2024-04-26T21:12:01.756819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768018+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:01.768018+0000","last_peered":"2024-04-26T21:12:01.768018+0000","last_clean":"2024-04-26T21:12:01.768018+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:01.768018+0000","last_undegraded":"2024-04-26T21:12:01.768018+0000","last_fullsized":"2024-04-26T21:12:01.768018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745738+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:01.745738+0000","last_peered":"2024-04-26T21:12:01.745738+0000","last_clean":"2024-04-26T21:12:01.745738+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:01.745738+0000","last_undegraded":"2024-04-26T21:12:01.745738+0000","last_fullsized":"2024-04-26T21:12:01.745738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770852+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:01.770852+0000","last_peered":"2024-04-26T21:12:01.770852+0000","last_clean":"2024-04-26T21:12:01.770852+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:01.770852+0000","last_undegraded":"2024-04-26T21:12:01.770852+0000","last_fullsized":"2024-04-26T21:12:01.770852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728962+0000","last_change":"2024-04-26T21:11:59.728962+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728962+0000","last_undegraded":"2024-04-26T21:11:59.728962+0000","last_fullsized":"2024-04-26T21:11:59.728962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769375+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:01.769375+0000","last_peered":"2024-04-26T21:12:01.769375+0000","last_clean":"2024-04-26T21:12:01.769375+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:01.769375+0000","last_undegraded":"2024-04-26T21:12:01.769375+0000","last_fullsized":"2024-04-26T21:12:01.769375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.760256+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:01.760256+0000","last_peered":"2024-04-26T21:12:01.760256+0000","last_clean":"2024-04-26T21:12:01.760256+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:01.760256+0000","last_undegraded":"2024-04-26T21:12:01.760256+0000","last_fullsized":"2024-04-26T21:12:01.760256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728646+0000","last_change":"2024-04-26T21:11:59.728646+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728646+0000","last_undegraded":"2024-04-26T21:11:59.728646+0000","last_fullsized":"2024-04-26T21:11:59.728646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.774307+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:01.774307+0000","last_peered":"2024-04-26T21:12:01.774307+0000","last_clean":"2024-04-26T21:12:01.774307+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:01.774307+0000","last_undegraded":"2024-04-26T21:12:01.774307+0000","last_fullsized":"2024-04-26T21:12:01.774307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767747+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:01.767747+0000","last_peered":"2024-04-26T21:12:01.767747+0000","last_clean":"2024-04-26T21:12:01.767747+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:01.767747+0000","last_undegraded":"2024-04-26T21:12:01.767747+0000","last_fullsized":"2024-04-26T21:12:01.767747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.272254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756630+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:01.756630+0000","last_peered":"2024-04-26T21:12:01.756630+0000","last_clean":"2024-04-26T21:12:01.756630+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:01.756630+0000","last_undegraded":"2024-04-26T21:12:01.756630+0000","last_fullsized":"2024-04-26T21:12:01.756630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767783+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:01.767783+0000","last_peered":"2024-04-26T21:12:01.767783+0000","last_clean":"2024-04-26T21:12:01.767783+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:01.767783+0000","last_undegraded":"2024-04-26T21:12:01.767783+0000","last_fullsized":"2024-04-26T21:12:01.767783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:55.309625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728806+0000","last_change":"2024-04-26T21:11:59.728806+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728806+0000","last_undegraded":"2024-04-26T21:11:59.728806+0000","last_fullsized":"2024-04-26T21:11:59.728806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728592+0000","last_change":"2024-04-26T21:11:59.728592+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728592+0000","last_undegraded":"2024-04-26T21:11:59.728592+0000","last_fullsized":"2024-04-26T21:11:59.728592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.772470+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:01.772470+0000","last_peered":"2024-04-26T21:12:01.772470+0000","last_clean":"2024-04-26T21:12:01.772470+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:01.772470+0000","last_undegraded":"2024-04-26T21:12:01.772470+0000","last_fullsized":"2024-04-26T21:12:01.772470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762707+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:01.762707+0000","last_peered":"2024-04-26T21:12:01.762707+0000","last_clean":"2024-04-26T21:12:01.762707+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:01.762707+0000","last_undegraded":"2024-04-26T21:12:01.762707+0000","last_fullsized":"2024-04-26T21:12:01.762707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775354+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:01.775354+0000","last_peered":"2024-04-26T21:12:01.775354+0000","last_clean":"2024-04-26T21:12:01.775354+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:01.775354+0000","last_undegraded":"2024-04-26T21:12:01.775354+0000","last_fullsized":"2024-04-26T21:12:01.775354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776943+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:01.776943+0000","last_peered":"2024-04-26T21:12:01.776943+0000","last_clean":"2024-04-26T21:12:01.776943+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:01.776943+0000","last_undegraded":"2024-04-26T21:12:01.776943+0000","last_fullsized":"2024-04-26T21:12:01.776943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775441+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:01.775441+0000","last_peered":"2024-04-26T21:12:01.775441+0000","last_clean":"2024-04-26T21:12:01.775441+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:01.775441+0000","last_undegraded":"2024-04-26T21:12:01.775441+0000","last_fullsized":"2024-04-26T21:12:01.775441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773877+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:01.773877+0000","last_peered":"2024-04-26T21:12:01.773877+0000","last_clean":"2024-04-26T21:12:01.773877+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:01.773877+0000","last_undegraded":"2024-04-26T21:12:01.773877+0000","last_fullsized":"2024-04-26T21:12:01.773877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767996+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:01.767996+0000","last_peered":"2024-04-26T21:12:01.767996+0000","last_clean":"2024-04-26T21:12:01.767996+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:01.767996+0000","last_undegraded":"2024-04-26T21:12:01.767996+0000","last_fullsized":"2024-04-26T21:12:01.767996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771405+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:01.771405+0000","last_peered":"2024-04-26T21:12:01.771405+0000","last_clean":"2024-04-26T21:12:01.771405+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:01.771405+0000","last_undegraded":"2024-04-26T21:12:01.771405+0000","last_fullsized":"2024-04-26T21:12:01.771405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":75061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":665941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-26T21:12:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.729661+0000', 'last_change': '2024-04-26T21:11:59.729661+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.729661+0000', 'last_undegraded': '2024-04-26T21:11:59.729661+0000', 'last_fullsized': '2024-04-26T21:11:59.729661+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-26T21:12:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.729179+0000', 'last_change': '2024-04-26T21:11:59.729179+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.729179+0000', 'last_undegraded': '2024-04-26T21:11:59.729179+0000', 'last_fullsized': '2024-04-26T21:11:59.729179+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.728962+0000', 'last_change': '2024-04-26T21:11:59.728962+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.728962+0000', 'last_undegraded': '2024-04-26T21:11:59.728962+0000', 'last_fullsized': '2024-04-26T21:11:59.728962+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:12:00.722397+0000', 'last_change': '2024-04-26T21:12:00.722397+0000', 'last_active': '2024-04-26T21:12:00.722397+0000', 'last_peered': '2024-04-26T21:12:00.722397+0000', 'last_clean': '2024-04-26T21:12:00.722397+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:00.722397+0000', 'last_undegraded': '2024-04-26T21:12:00.722397+0000', 'last_fullsized': '2024-04-26T21:12:00.722397+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:00.722397+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.728646+0000', 'last_change': '2024-04-26T21:11:59.728646+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.728646+0000', 'last_undegraded': '2024-04-26T21:11:59.728646+0000', 'last_fullsized': '2024-04-26T21:11:59.728646+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.728806+0000', 'last_change': '2024-04-26T21:11:59.728806+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.728806+0000', 'last_undegraded': '2024-04-26T21:11:59.728806+0000', 'last_fullsized': '2024-04-26T21:11:59.728806+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-26T21:12:05.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:11:59.728592+0000', 'last_change': '2024-04-26T21:11:59.728592+0000', 'last_active': '2024-04-26T21:11:59.713707+0000', 'last_peered': '2024-04-26T21:11:59.713707+0000', 'last_clean': '2024-04-26T21:11:59.713707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:11:59.728592+0000', 'last_undegraded': '2024-04-26T21:11:59.728592+0000', 'last_fullsized': '2024-04-26T21:11:59.728592+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'last_clean_scrub_stamp': '2024-04-26T21:11:59.713707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:12:05.486 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:12:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.618+0000 7fe0c30cc640 1 Processor -- start 2024-04-26T21:12:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.618+0000 7fe0c30cc640 1 -- start start 2024-04-26T21:12:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c30cc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0bc0cfda0 0x7fe0bc0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c30cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c30cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0bc0d0220 con 0x7fe0bc0d1b80 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c30cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0bc0d0500 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c30cc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0bc0d07e0 con 0x7fe0bc0cfda0 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51864/0 (socket says 172.21.15.40:51864) 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 -- 172.21.15.40:0/3590623183 learned_addr learned my addr 172.21.15.40:0/3590623183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c0e68640 1 --1- 172.21.15.40:0/3590623183 >> v1:172.21.15.40:6789/0 conn(0x7fe0bc0d1b80 0x7fe0bc0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46206/0 (socket says 172.21.15.40:46206) 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0bbfff640 1 --2- 172.21.15.40:0/3590623183 >> v2:172.21.15.184:3300/0 conn(0x7fe0bc0cfda0 0x7fe0bc0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 -- 172.21.15.40:0/3590623183 >> v2:172.21.15.184:3300/0 conn(0x7fe0bc0cfda0 msgr2=0x7fe0bc0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 --2- 172.21.15.40:0/3590623183 >> v2:172.21.15.184:3300/0 conn(0x7fe0bc0cfda0 0x7fe0bc0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 -- 172.21.15.40:0/3590623183 >> v1:172.21.15.40:6789/0 conn(0x7fe0bc0d1b80 legacy=0x7fe0bc0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.619+0000 7fe0c1669640 1 -- 172.21.15.40:0/3590623183 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc0d0ac0 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.620+0000 7fe0c1669640 1 --2- 172.21.15.40:0/3590623183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc08ddb0 tx=0x7fe0b402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875fad3d17505f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.620+0000 7fe0bb7fe640 1 -- 172.21.15.40:0/3590623183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4036070 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.620+0000 7fe0bb7fe640 1 -- 172.21.15.40:0/3590623183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b4031600 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.620+0000 7fe0bb7fe640 1 -- 172.21.15.40:0/3590623183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40319d0 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 -- 172.21.15.40:0/3590623183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 msgr2=0x7fe0bc0b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/3590623183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc08ddb0 tx=0x7fe0b402a730 comp rx=0 tx=0).stop 2024-04-26T21:12:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 -- 172.21.15.40:0/3590623183 shutdown_connections 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/3590623183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0b6530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/3590623183 >> v2:172.21.15.184:3300/0 conn(0x7fe0bc0cfda0 0x7fe0bc0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 -- 172.21.15.40:0/3590623183 >> 172.21.15.40:0/3590623183 conn(0x7fe0bc09e540 msgr2=0x7fe0bc0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 -- 172.21.15.40:0/3590623183 shutdown_connections 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.621+0000 7fe0c30cc640 1 -- 172.21.15.40:0/3590623183 wait complete. 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.622+0000 7fe0c30cc640 1 Processor -- start 2024-04-26T21:12:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.622+0000 7fe0c30cc640 1 -- start start 2024-04-26T21:12:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 0x7fe0bc1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0bc0d1b80 0x7fe0bc0b9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0d4170 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0b6a70 con 0x7fe0bc0d1b80 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc15ee50 con 0x7fe0bc0cfda0 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51876/0 (socket says 172.21.15.40:51876) 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 -- 172.21.15.40:0/2253246081 learned_addr learned my addr 172.21.15.40:0/2253246081 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0bbfff640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 0x7fe0bc1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c1669640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0bc0d1b80 0x7fe0bc0b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 -- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 msgr2=0x7fe0bc1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 0x7fe0bc1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 -- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0bc0d1b80 msgr2=0x7fe0bc0b9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0bc0d1b80 0x7fe0bc0b9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 -- 172.21.15.40:0/2253246081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc0c8e30 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0bbfff640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 0x7fe0bc1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c0e68640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c3f0 tx=0x7fe0b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00074e0 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0007680 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.623+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc0c90d0 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.624+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004a00 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.624+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc0c9690 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.625+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc06a290 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.625+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0b001a020 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.626+0000 7fe0b97fa640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 0x7fe0a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.626+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe0b008b250 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.626+0000 7fe0bbfff640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 0x7fe0a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.626+0000 7fe0bbfff640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 0x7fe0a006ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0cc1e0 tx=0x7fe0a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:05.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.630+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0b0059710 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.885+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe0bc0d27b0 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.886+0000 7fe0b97fa640 1 -- 172.21.15.40:0/2253246081 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1173 (secure 0 0 0) 0x7fe0b0059010 con 0x7fe0bc0b60a0 2024-04-26T21:12:05.887 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:05.887 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"unknown","count":6},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592618,"bytes_used":224337920,"bytes_avail":772869775360,"bytes_total":773094113280,"unknown_pgs_ratio":0.061855670064687729,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":2365,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4291}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-26T21:11:36.785645+0000","services":{}},"progress_events":{}} 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 msgr2=0x7fe0a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 0x7fe0a006ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0cc1e0 tx=0x7fe0a8008040 comp rx=0 tx=0).stop 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 msgr2=0x7fe0bc0c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c3f0 tx=0x7fe0b000c8b0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 shutdown_connections 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0a006c920 0x7fe0a006ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0bc0d1b80 0x7fe0bc0b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0bc0cfda0 0x7fe0bc1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 --2- 172.21.15.40:0/2253246081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0bc0b60a0 0x7fe0bc0c9e90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 >> 172.21.15.40:0/2253246081 conn(0x7fe0bc09e540 msgr2=0x7fe0bc0ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 shutdown_connections 2024-04-26T21:12:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.888+0000 7fe0c30cc640 1 -- 172.21.15.40:0/2253246081 wait complete. 2024-04-26T21:12:05.900 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:12:05.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.999+0000 7f173c924640 1 Processor -- start 2024-04-26T21:12:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:05.999+0000 7f173c924640 1 -- start start 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173c924640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f173809c400 0x7f17380cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173c924640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17380d14c0 con 0x7f1738070d20 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173c924640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17380d1760 con 0x7f1738070520 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173c924640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17380d1290 con 0x7f173809c400 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f172ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f173809c400 0x7f17380cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1738070d20 0x7f1738071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46218/0 (socket says 172.21.15.40:46218) 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f173659c640 1 -- 172.21.15.40:0/250066311 learned_addr learned my addr 172.21.15.40:0/250066311 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f172ffff640 1 --2- 172.21.15.40:0/250066311 >> v2:172.21.15.40:3300/0 conn(0x7f173809c400 0x7f17380cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51886/0 (socket says 172.21.15.40:51886) 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f172ffff640 1 -- 172.21.15.40:0/250066311 >> v1:172.21.15.184:6789/0 conn(0x7f1738070520 legacy=0x7f17380cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f172ffff640 1 -- 172.21.15.40:0/250066311 >> v1:172.21.15.40:6789/0 conn(0x7f1738070d20 legacy=0x7f1738071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f172ffff640 1 -- 172.21.15.40:0/250066311 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17380d1a40 con 0x7f173809c400 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.000+0000 7f1735d9b640 1 -- 172.21.15.40:0/250066311 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3085724962 0 0) 0x7f17380d14c0 con 0x7f1738070d20 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f172ffff640 1 --2- 172.21.15.40:0/250066311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173809c400 0x7f17380cd5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17200097a0 tx=0x7f172002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213c21e2e0b9585c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f1735d9b640 1 -- 172.21.15.40:0/250066311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720037070 con 0x7f173809c400 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f1735d9b640 1 -- 172.21.15.40:0/250066311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1720032440 con 0x7f173809c400 2024-04-26T21:12:06.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f1735d9b640 1 -- 172.21.15.40:0/250066311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720032810 con 0x7f173809c400 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f173c924640 1 -- 172.21.15.40:0/250066311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173809c400 msgr2=0x7f17380cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.001+0000 7f173c924640 1 --2- 172.21.15.40:0/250066311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173809c400 0x7f17380cd5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17200097a0 tx=0x7f172002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.002+0000 7f173c924640 1 -- 172.21.15.40:0/250066311 shutdown_connections 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.002+0000 7f173c924640 1 --2- 172.21.15.40:0/250066311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173809c400 0x7f17380cd5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.002+0000 7f173c924640 1 -- 172.21.15.40:0/250066311 >> 172.21.15.40:0/250066311 conn(0x7f17380b4a40 msgr2=0x7f17380b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.002+0000 7f173c924640 1 -- 172.21.15.40:0/250066311 shutdown_connections 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.002+0000 7f173c924640 1 -- 172.21.15.40:0/250066311 wait complete. 2024-04-26T21:12:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.003+0000 7f173c924640 1 Processor -- start 2024-04-26T21:12:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.003+0000 7f173c924640 1 -- start start 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 0x7f17380c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f173809c400 0x7f173815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17380cf350 con 0x7f1738070d20 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17380d2000 con 0x7f1738070520 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17380d2520 con 0x7f173809c400 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51900/0 (socket says 172.21.15.40:51900) 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 -- 172.21.15.40:0/2108974408 learned_addr learned my addr 172.21.15.40:0/2108974408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f1736d9d640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f173809c400 0x7f173815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173659c640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 0x7f17380c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 -- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f173809c400 msgr2=0x7f173815fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f173809c400 0x7f173815fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 -- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 msgr2=0x7f17380c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 0x7f17380c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 -- 172.21.15.40:0/2108974408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17380c8c00 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173659c640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 0x7f17380c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172ffff640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1720035040 tx=0x7f17200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720042020 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17200309b0 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17380c7620 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720037040 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.004+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17380c7be0 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.005+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1738072250 con 0x7f1738070d20 2024-04-26T21:12:06.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.006+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1720045020 con 0x7f1738070d20 2024-04-26T21:12:06.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.006+0000 7f172effd640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 0x7f171806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:06.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.006+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f17200ac870 con 0x7f1738070d20 2024-04-26T21:12:06.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.006+0000 7f173659c640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 0x7f171806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:06.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.007+0000 7f173659c640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 0x7f171806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1728000f60 tx=0x7f1728009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.010+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1738072250 con 0x7f1738070d20 2024-04-26T21:12:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.191+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17380693d0 con 0x7f171806c8f0 2024-04-26T21:12:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.198+0000 7f172effd640 1 -- 172.21.15.40:0/2108974408 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250484 (secure 0 0 0) 0x7f17380693d0 con 0x7f171806c8f0 2024-04-26T21:12:06.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:06.203 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:06.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 msgr2=0x7f171806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 0x7f171806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1728000f60 tx=0x7f1728009040 comp rx=0 tx=0).stop 2024-04-26T21:12:06.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 msgr2=0x7f1738158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1720035040 tx=0x7f17200046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 shutdown_connections 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f171806c8f0 0x7f171806edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f173809c400 0x7f173815fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1738070d20 0x7f1738158440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 --2- 172.21.15.40:0/2108974408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1738070520 0x7f17380c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 >> 172.21.15.40:0/2108974408 conn(0x7f17380b4a40 msgr2=0x7f1738071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 shutdown_connections 2024-04-26T21:12:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:06.201+0000 7f173c924640 1 -- 172.21.15.40:0/2108974408 wait complete. 2024-04-26T21:12:06.212 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-26T21:12:04.918506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1771826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.195155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775824+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:01.775824+0000","last_peered":"2024-04-26T21:12:01.775824+0000","last_clean":"2024-04-26T21:12:01.775824+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:01.775824+0000","last_undegraded":"2024-04-26T21:12:01.775824+0000","last_fullsized":"2024-04-26T21:12:01.775824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768415+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:01.768415+0000","last_peered":"2024-04-26T21:12:01.768415+0000","last_clean":"2024-04-26T21:12:01.768415+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:01.768415+0000","last_undegraded":"2024-04-26T21:12:01.768415+0000","last_fullsized":"2024-04-26T21:12:01.768415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769579+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:01.769579+0000","last_peered":"2024-04-26T21:12:01.769579+0000","last_clean":"2024-04-26T21:12:01.769579+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:01.769579+0000","last_undegraded":"2024-04-26T21:12:01.769579+0000","last_fullsized":"2024-04-26T21:12:01.769579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775535+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:01.775535+0000","last_peered":"2024-04-26T21:12:01.775535+0000","last_clean":"2024-04-26T21:12:01.775535+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:01.775535+0000","last_undegraded":"2024-04-26T21:12:01.775535+0000","last_fullsized":"2024-04-26T21:12:01.775535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766071+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:01.766071+0000","last_peered":"2024-04-26T21:12:01.766071+0000","last_clean":"2024-04-26T21:12:01.766071+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:01.766071+0000","last_undegraded":"2024-04-26T21:12:01.766071+0000","last_fullsized":"2024-04-26T21:12:01.766071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775714+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:01.775714+0000","last_peered":"2024-04-26T21:12:01.775714+0000","last_clean":"2024-04-26T21:12:01.775714+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:01.775714+0000","last_undegraded":"2024-04-26T21:12:01.775714+0000","last_fullsized":"2024-04-26T21:12:01.775714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780541+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:01.780541+0000","last_peered":"2024-04-26T21:12:01.780541+0000","last_clean":"2024-04-26T21:12:01.780541+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:01.780541+0000","last_undegraded":"2024-04-26T21:12:01.780541+0000","last_fullsized":"2024-04-26T21:12:01.780541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.763195+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:01.763195+0000","last_peered":"2024-04-26T21:12:01.763195+0000","last_clean":"2024-04-26T21:12:01.763195+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:01.763195+0000","last_undegraded":"2024-04-26T21:12:01.763195+0000","last_fullsized":"2024-04-26T21:12:01.763195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773447+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:01.773447+0000","last_peered":"2024-04-26T21:12:01.773447+0000","last_clean":"2024-04-26T21:12:01.773447+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:01.773447+0000","last_undegraded":"2024-04-26T21:12:01.773447+0000","last_fullsized":"2024-04-26T21:12:01.773447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768037+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:01.768037+0000","last_peered":"2024-04-26T21:12:01.768037+0000","last_clean":"2024-04-26T21:12:01.768037+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:01.768037+0000","last_undegraded":"2024-04-26T21:12:01.768037+0000","last_fullsized":"2024-04-26T21:12:01.768037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:06.177139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767765+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:01.767765+0000","last_peered":"2024-04-26T21:12:01.767765+0000","last_clean":"2024-04-26T21:12:01.767765+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:01.767765+0000","last_undegraded":"2024-04-26T21:12:01.767765+0000","last_fullsized":"2024-04-26T21:12:01.767765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.766038+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:01.766038+0000","last_peered":"2024-04-26T21:12:01.766038+0000","last_clean":"2024-04-26T21:12:01.766038+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:01.766038+0000","last_undegraded":"2024-04-26T21:12:01.766038+0000","last_fullsized":"2024-04-26T21:12:01.766038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769527+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:01.769527+0000","last_peered":"2024-04-26T21:12:01.769527+0000","last_clean":"2024-04-26T21:12:01.769527+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:01.769527+0000","last_undegraded":"2024-04-26T21:12:01.769527+0000","last_fullsized":"2024-04-26T21:12:01.769527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775703+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:01.775703+0000","last_peered":"2024-04-26T21:12:01.775703+0000","last_clean":"2024-04-26T21:12:01.775703+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:01.775703+0000","last_undegraded":"2024-04-26T21:12:01.775703+0000","last_fullsized":"2024-04-26T21:12:01.775703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770158+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:01.770158+0000","last_peered":"2024-04-26T21:12:01.770158+0000","last_clean":"2024-04-26T21:12:01.770158+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:01.770158+0000","last_undegraded":"2024-04-26T21:12:01.770158+0000","last_fullsized":"2024-04-26T21:12:01.770158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770460+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:01.770460+0000","last_peered":"2024-04-26T21:12:01.770460+0000","last_clean":"2024-04-26T21:12:01.770460+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:01.770460+0000","last_undegraded":"2024-04-26T21:12:01.770460+0000","last_fullsized":"2024-04-26T21:12:01.770460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769708+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:01.769708+0000","last_peered":"2024-04-26T21:12:01.769708+0000","last_clean":"2024-04-26T21:12:01.769708+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:01.769708+0000","last_undegraded":"2024-04-26T21:12:01.769708+0000","last_fullsized":"2024-04-26T21:12:01.769708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769651+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:01.769651+0000","last_peered":"2024-04-26T21:12:01.769651+0000","last_clean":"2024-04-26T21:12:01.769651+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:01.769651+0000","last_undegraded":"2024-04-26T21:12:01.769651+0000","last_fullsized":"2024-04-26T21:12:01.769651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762738+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:01.762738+0000","last_peered":"2024-04-26T21:12:01.762738+0000","last_clean":"2024-04-26T21:12:01.762738+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:01.762738+0000","last_undegraded":"2024-04-26T21:12:01.762738+0000","last_fullsized":"2024-04-26T21:12:01.762738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770442+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:01.770442+0000","last_peered":"2024-04-26T21:12:01.770442+0000","last_clean":"2024-04-26T21:12:01.770442+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:01.770442+0000","last_undegraded":"2024-04-26T21:12:01.770442+0000","last_fullsized":"2024-04-26T21:12:01.770442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769688+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:01.769688+0000","last_peered":"2024-04-26T21:12:01.769688+0000","last_clean":"2024-04-26T21:12:01.769688+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:01.769688+0000","last_undegraded":"2024-04-26T21:12:01.769688+0000","last_fullsized":"2024-04-26T21:12:01.769688+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770730+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:01.770730+0000","last_peered":"2024-04-26T21:12:01.770730+0000","last_clean":"2024-04-26T21:12:01.770730+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:01.770730+0000","last_undegraded":"2024-04-26T21:12:01.770730+0000","last_fullsized":"2024-04-26T21:12:01.770730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775620+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:01.775620+0000","last_peered":"2024-04-26T21:12:01.775620+0000","last_clean":"2024-04-26T21:12:01.775620+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:01.775620+0000","last_undegraded":"2024-04-26T21:12:01.775620+0000","last_fullsized":"2024-04-26T21:12:01.775620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:10.212486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770423+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:01.770423+0000","last_peered":"2024-04-26T21:12:01.770423+0000","last_clean":"2024-04-26T21:12:01.770423+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:01.770423+0000","last_undegraded":"2024-04-26T21:12:01.770423+0000","last_fullsized":"2024-04-26T21:12:01.770423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767658+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:01.767658+0000","last_peered":"2024-04-26T21:12:01.767658+0000","last_clean":"2024-04-26T21:12:01.767658+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:01.767658+0000","last_undegraded":"2024-04-26T21:12:01.767658+0000","last_fullsized":"2024-04-26T21:12:01.767658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745082+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:01.745082+0000","last_peered":"2024-04-26T21:12:01.745082+0000","last_clean":"2024-04-26T21:12:01.745082+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:01.745082+0000","last_undegraded":"2024-04-26T21:12:01.745082+0000","last_fullsized":"2024-04-26T21:12:01.745082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754834+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:01.754834+0000","last_peered":"2024-04-26T21:12:01.754834+0000","last_clean":"2024-04-26T21:12:01.754834+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:01.754834+0000","last_undegraded":"2024-04-26T21:12:01.754834+0000","last_fullsized":"2024-04-26T21:12:01.754834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.755564+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:01.755564+0000","last_peered":"2024-04-26T21:12:01.755564+0000","last_clean":"2024-04-26T21:12:01.755564+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:01.755564+0000","last_undegraded":"2024-04-26T21:12:01.755564+0000","last_fullsized":"2024-04-26T21:12:01.755564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744773+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:01.744773+0000","last_peered":"2024-04-26T21:12:01.744773+0000","last_clean":"2024-04-26T21:12:01.744773+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:01.744773+0000","last_undegraded":"2024-04-26T21:12:01.744773+0000","last_fullsized":"2024-04-26T21:12:01.744773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768019+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:01.768019+0000","last_peered":"2024-04-26T21:12:01.768019+0000","last_clean":"2024-04-26T21:12:01.768019+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:01.768019+0000","last_undegraded":"2024-04-26T21:12:01.768019+0000","last_fullsized":"2024-04-26T21:12:01.768019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729661+0000","last_change":"2024-04-26T21:11:59.729661+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729661+0000","last_undegraded":"2024-04-26T21:11:59.729661+0000","last_fullsized":"2024-04-26T21:11:59.729661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770745+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:01.770745+0000","last_peered":"2024-04-26T21:12:01.770745+0000","last_clean":"2024-04-26T21:12:01.770745+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:01.770745+0000","last_undegraded":"2024-04-26T21:12:01.770745+0000","last_fullsized":"2024-04-26T21:12:01.770745+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:39.200651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.778176+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:01.778176+0000","last_peered":"2024-04-26T21:12:01.778176+0000","last_clean":"2024-04-26T21:12:01.778176+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:01.778176+0000","last_undegraded":"2024-04-26T21:12:01.778176+0000","last_fullsized":"2024-04-26T21:12:01.778176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776418+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:01.776418+0000","last_peered":"2024-04-26T21:12:01.776418+0000","last_clean":"2024-04-26T21:12:01.776418+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:01.776418+0000","last_undegraded":"2024-04-26T21:12:01.776418+0000","last_fullsized":"2024-04-26T21:12:01.776418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770363+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:01.770363+0000","last_peered":"2024-04-26T21:12:01.770363+0000","last_clean":"2024-04-26T21:12:01.770363+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:01.770363+0000","last_undegraded":"2024-04-26T21:12:01.770363+0000","last_fullsized":"2024-04-26T21:12:01.770363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768054+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:01.768054+0000","last_peered":"2024-04-26T21:12:01.768054+0000","last_clean":"2024-04-26T21:12:01.768054+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:01.768054+0000","last_undegraded":"2024-04-26T21:12:01.768054+0000","last_fullsized":"2024-04-26T21:12:01.768054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745579+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:01.745579+0000","last_peered":"2024-04-26T21:12:01.745579+0000","last_clean":"2024-04-26T21:12:01.745579+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:01.745579+0000","last_undegraded":"2024-04-26T21:12:01.745579+0000","last_fullsized":"2024-04-26T21:12:01.745579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775512+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:01.775512+0000","last_peered":"2024-04-26T21:12:01.775512+0000","last_clean":"2024-04-26T21:12:01.775512+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:01.775512+0000","last_undegraded":"2024-04-26T21:12:01.775512+0000","last_fullsized":"2024-04-26T21:12:01.775512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.754709+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:01.754709+0000","last_peered":"2024-04-26T21:12:01.754709+0000","last_clean":"2024-04-26T21:12:01.754709+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:01.754709+0000","last_undegraded":"2024-04-26T21:12:01.754709+0000","last_fullsized":"2024-04-26T21:12:01.754709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770711+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:01.770711+0000","last_peered":"2024-04-26T21:12:01.770711+0000","last_clean":"2024-04-26T21:12:01.770711+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:01.770711+0000","last_undegraded":"2024-04-26T21:12:01.770711+0000","last_fullsized":"2024-04-26T21:12:01.770711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775574+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:01.775574+0000","last_peered":"2024-04-26T21:12:01.775574+0000","last_clean":"2024-04-26T21:12:01.775574+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:01.775574+0000","last_undegraded":"2024-04-26T21:12:01.775574+0000","last_fullsized":"2024-04-26T21:12:01.775574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771315+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:01.771315+0000","last_peered":"2024-04-26T21:12:01.771315+0000","last_clean":"2024-04-26T21:12:01.771315+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:01.771315+0000","last_undegraded":"2024-04-26T21:12:01.771315+0000","last_fullsized":"2024-04-26T21:12:01.771315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775488+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:01.775488+0000","last_peered":"2024-04-26T21:12:01.775488+0000","last_clean":"2024-04-26T21:12:01.775488+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:01.775488+0000","last_undegraded":"2024-04-26T21:12:01.775488+0000","last_fullsized":"2024-04-26T21:12:01.775488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775800+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:01.775800+0000","last_peered":"2024-04-26T21:12:01.775800+0000","last_clean":"2024-04-26T21:12:01.775800+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:01.775800+0000","last_undegraded":"2024-04-26T21:12:01.775800+0000","last_fullsized":"2024-04-26T21:12:01.775800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.729179+0000","last_change":"2024-04-26T21:11:59.729179+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.729179+0000","last_undegraded":"2024-04-26T21:11:59.729179+0000","last_fullsized":"2024-04-26T21:11:59.729179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769605+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:01.769605+0000","last_peered":"2024-04-26T21:12:01.769605+0000","last_clean":"2024-04-26T21:12:01.769605+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:01.769605+0000","last_undegraded":"2024-04-26T21:12:01.769605+0000","last_fullsized":"2024-04-26T21:12:01.769605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.780530+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:01.780530+0000","last_peered":"2024-04-26T21:12:01.780530+0000","last_clean":"2024-04-26T21:12:01.780530+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:01.780530+0000","last_undegraded":"2024-04-26T21:12:01.780530+0000","last_fullsized":"2024-04-26T21:12:01.780530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.761140+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:01.761140+0000","last_peered":"2024-04-26T21:12:01.761140+0000","last_clean":"2024-04-26T21:12:01.761140+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:01.761140+0000","last_undegraded":"2024-04-26T21:12:01.761140+0000","last_fullsized":"2024-04-26T21:12:01.761140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.744168+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:01.744168+0000","last_peered":"2024-04-26T21:12:01.744168+0000","last_clean":"2024-04-26T21:12:01.744168+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:01.744168+0000","last_undegraded":"2024-04-26T21:12:01.744168+0000","last_fullsized":"2024-04-26T21:12:01.744168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756819+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:01.756819+0000","last_peered":"2024-04-26T21:12:01.756819+0000","last_clean":"2024-04-26T21:12:01.756819+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:01.756819+0000","last_undegraded":"2024-04-26T21:12:01.756819+0000","last_fullsized":"2024-04-26T21:12:01.756819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.768018+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:01.768018+0000","last_peered":"2024-04-26T21:12:01.768018+0000","last_clean":"2024-04-26T21:12:01.768018+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:01.768018+0000","last_undegraded":"2024-04-26T21:12:01.768018+0000","last_fullsized":"2024-04-26T21:12:01.768018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.745738+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:01.745738+0000","last_peered":"2024-04-26T21:12:01.745738+0000","last_clean":"2024-04-26T21:12:01.745738+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:01.745738+0000","last_undegraded":"2024-04-26T21:12:01.745738+0000","last_fullsized":"2024-04-26T21:12:01.745738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.770852+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:01.770852+0000","last_peered":"2024-04-26T21:12:01.770852+0000","last_clean":"2024-04-26T21:12:01.770852+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:01.770852+0000","last_undegraded":"2024-04-26T21:12:01.770852+0000","last_fullsized":"2024-04-26T21:12:01.770852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728962+0000","last_change":"2024-04-26T21:11:59.728962+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728962+0000","last_undegraded":"2024-04-26T21:11:59.728962+0000","last_fullsized":"2024-04-26T21:11:59.728962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.769375+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:01.769375+0000","last_peered":"2024-04-26T21:12:01.769375+0000","last_clean":"2024-04-26T21:12:01.769375+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:01.769375+0000","last_undegraded":"2024-04-26T21:12:01.769375+0000","last_fullsized":"2024-04-26T21:12:01.769375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:15.814641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.760256+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:01.760256+0000","last_peered":"2024-04-26T21:12:01.760256+0000","last_clean":"2024-04-26T21:12:01.760256+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:01.760256+0000","last_undegraded":"2024-04-26T21:12:01.760256+0000","last_fullsized":"2024-04-26T21:12:01.760256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:12:00.722397+0000","last_change":"2024-04-26T21:12:00.722397+0000","last_active":"2024-04-26T21:12:00.722397+0000","last_peered":"2024-04-26T21:12:00.722397+0000","last_clean":"2024-04-26T21:12:00.722397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:00.722397+0000","last_undegraded":"2024-04-26T21:12:00.722397+0000","last_fullsized":"2024-04-26T21:12:00.722397+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728646+0000","last_change":"2024-04-26T21:11:59.728646+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728646+0000","last_undegraded":"2024-04-26T21:11:59.728646+0000","last_fullsized":"2024-04-26T21:11:59.728646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.774307+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:01.774307+0000","last_peered":"2024-04-26T21:12:01.774307+0000","last_clean":"2024-04-26T21:12:01.774307+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:01.774307+0000","last_undegraded":"2024-04-26T21:12:01.774307+0000","last_fullsized":"2024-04-26T21:12:01.774307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767747+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:01.767747+0000","last_peered":"2024-04-26T21:12:01.767747+0000","last_clean":"2024-04-26T21:12:01.767747+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:01.767747+0000","last_undegraded":"2024-04-26T21:12:01.767747+0000","last_fullsized":"2024-04-26T21:12:01.767747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.272254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.756630+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:01.756630+0000","last_peered":"2024-04-26T21:12:01.756630+0000","last_clean":"2024-04-26T21:12:01.756630+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:01.756630+0000","last_undegraded":"2024-04-26T21:12:01.756630+0000","last_fullsized":"2024-04-26T21:12:01.756630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767783+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:01.767783+0000","last_peered":"2024-04-26T21:12:01.767783+0000","last_clean":"2024-04-26T21:12:01.767783+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:01.767783+0000","last_undegraded":"2024-04-26T21:12:01.767783+0000","last_fullsized":"2024-04-26T21:12:01.767783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:55.309625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728806+0000","last_change":"2024-04-26T21:11:59.728806+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728806+0000","last_undegraded":"2024-04-26T21:11:59.728806+0000","last_fullsized":"2024-04-26T21:11:59.728806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-04-26T21:11:59.728592+0000","last_change":"2024-04-26T21:11:59.728592+0000","last_active":"2024-04-26T21:11:59.713707+0000","last_peered":"2024-04-26T21:11:59.713707+0000","last_clean":"2024-04-26T21:11:59.713707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:11:59.728592+0000","last_undegraded":"2024-04-26T21:11:59.728592+0000","last_fullsized":"2024-04-26T21:11:59.728592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.772470+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:01.772470+0000","last_peered":"2024-04-26T21:12:01.772470+0000","last_clean":"2024-04-26T21:12:01.772470+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:01.772470+0000","last_undegraded":"2024-04-26T21:12:01.772470+0000","last_fullsized":"2024-04-26T21:12:01.772470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.762707+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:01.762707+0000","last_peered":"2024-04-26T21:12:01.762707+0000","last_clean":"2024-04-26T21:12:01.762707+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:01.762707+0000","last_undegraded":"2024-04-26T21:12:01.762707+0000","last_fullsized":"2024-04-26T21:12:01.762707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775354+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:01.775354+0000","last_peered":"2024-04-26T21:12:01.775354+0000","last_clean":"2024-04-26T21:12:01.775354+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:01.775354+0000","last_undegraded":"2024-04-26T21:12:01.775354+0000","last_fullsized":"2024-04-26T21:12:01.775354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.776943+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:01.776943+0000","last_peered":"2024-04-26T21:12:01.776943+0000","last_clean":"2024-04-26T21:12:01.776943+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:01.776943+0000","last_undegraded":"2024-04-26T21:12:01.776943+0000","last_fullsized":"2024-04-26T21:12:01.776943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.775441+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:01.775441+0000","last_peered":"2024-04-26T21:12:01.775441+0000","last_clean":"2024-04-26T21:12:01.775441+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:01.775441+0000","last_undegraded":"2024-04-26T21:12:01.775441+0000","last_fullsized":"2024-04-26T21:12:01.775441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.773877+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:01.773877+0000","last_peered":"2024-04-26T21:12:01.773877+0000","last_clean":"2024-04-26T21:12:01.773877+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:01.773877+0000","last_undegraded":"2024-04-26T21:12:01.773877+0000","last_fullsized":"2024-04-26T21:12:01.773877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.767996+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:01.767996+0000","last_peered":"2024-04-26T21:12:01.767996+0000","last_clean":"2024-04-26T21:12:01.767996+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:01.767996+0000","last_undegraded":"2024-04-26T21:12:01.767996+0000","last_fullsized":"2024-04-26T21:12:01.767996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:16.216043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-26T21:12:01.771405+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:01.771405+0000","last_peered":"2024-04-26T21:12:01.771405+0000","last_clean":"2024-04-26T21:12:01.771405+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:01.771405+0000","last_undegraded":"2024-04-26T21:12:01.771405+0000","last_fullsized":"2024-04-26T21:12:01.771405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":75061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":665941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:09.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.330+0000 7f74f9d91640 1 Processor -- start 2024-04-26T21:12:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.330+0000 7f74f9d91640 1 -- start start 2024-04-26T21:12:09.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f9d91640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74f415f680 0x7f74f40d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f9d91640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f415fa60 0x7f74f40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f9d91640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f40d3b80 con 0x7f74f40b2440 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f9d91640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f74f40d3e60 con 0x7f74f415f680 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f9d91640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74f40d4140 con 0x7f74f415fa60 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74f415f680 0x7f74f40d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f74f40b2440 0x7f74f40d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46224/0 (socket says 172.21.15.40:46224) 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f3fff640 1 -- 172.21.15.40:0/1346602405 learned_addr learned my addr 172.21.15.40:0/1346602405 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f2ffd640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f415fa60 0x7f74f40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f37fe640 1 -- 172.21.15.40:0/1346602405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f415fa60 msgr2=0x7f74f40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f37fe640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f415fa60 0x7f74f40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f37fe640 1 -- 172.21.15.40:0/1346602405 >> v1:172.21.15.40:6789/0 conn(0x7f74f40b2440 legacy=0x7f74f40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f37fe640 1 -- 172.21.15.40:0/1346602405 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f40d0270 con 0x7f74f415f680 2024-04-26T21:12:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.331+0000 7f74f27fc640 1 -- 172.21.15.40:0/1346602405 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744465332 0 0) 0x7f74f40d3b80 con 0x7f74f40b2440 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.332+0000 7f74f37fe640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415f680 0x7f74f40d5e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f74e40097a0 tx=0x7f74e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e979362c83d9ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.332+0000 7f74f27fc640 1 -- 172.21.15.40:0/1346602405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4037070 con 0x7f74f415f680 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.332+0000 7f74f27fc640 1 -- 172.21.15.40:0/1346602405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e4032440 con 0x7f74f415f680 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.332+0000 7f74f27fc640 1 -- 172.21.15.40:0/1346602405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4032810 con 0x7f74f415f680 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 -- 172.21.15.40:0/1346602405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415f680 msgr2=0x7f74f40d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415f680 0x7f74f40d5e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f74e40097a0 tx=0x7f74e402a180 comp rx=0 tx=0).stop 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 -- 172.21.15.40:0/1346602405 shutdown_connections 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f415fa60 0x7f74f40b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 --2- 172.21.15.40:0/1346602405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415f680 0x7f74f40d5e50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 -- 172.21.15.40:0/1346602405 >> 172.21.15.40:0/1346602405 conn(0x7f74f409e5a0 msgr2=0x7f74f40d5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 -- 172.21.15.40:0/1346602405 shutdown_connections 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.333+0000 7f74f9d91640 1 -- 172.21.15.40:0/1346602405 wait complete. 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.334+0000 7f74f9d91640 1 Processor -- start 2024-04-26T21:12:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.334+0000 7f74f9d91640 1 -- start start 2024-04-26T21:12:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f40b2440 0x7f74f4161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74f415f680 0x7f74f40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74f40d2300 con 0x7f74f415fa60 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74f40d4420 con 0x7f74f415f680 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f9d91640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74f40d2480 con 0x7f74f40b2440 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51914/0 (socket says 172.21.15.40:51914) 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f3fff640 1 -- 172.21.15.40:0/3837495472 learned_addr learned my addr 172.21.15.40:0/3837495472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f37fe640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f40b2440 0x7f74f4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.335+0000 7f74f2ffd640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74f415f680 0x7f74f40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 -- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f40b2440 msgr2=0x7f74f4161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f40b2440 0x7f74f4161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 -- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74f415f680 msgr2=0x7f74f40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74f415f680 0x7f74f40ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 -- 172.21.15.40:0/3837495472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f41f5150 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f3fff640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c760 tx=0x7f74ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec018070 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec00a640 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00aa30 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f41f5430 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.336+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f41f59f0 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.337+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f406a290 con 0x7f74f415fa60 2024-04-26T21:12:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.337+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74ec01b070 con 0x7f74f415fa60 2024-04-26T21:12:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.338+0000 7f74dbfff640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 0x7f74e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.338+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f74ec08afa0 con 0x7f74f415fa60 2024-04-26T21:12:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.338+0000 7f74f37fe640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 0x7f74e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.339+0000 7f74f37fe640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 0x7f74e006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74e4034000 tx=0x7f74e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.343+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74ec058e10 con 0x7f74f415fa60 2024-04-26T21:12:09.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.524+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74f408f350 con 0x7f74e006c920 2024-04-26T21:12:09.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.530+0000 7f74dbfff640 1 -- 172.21.15.40:0/3837495472 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251859 (secure 0 0 0) 0x7f74f408f350 con 0x7f74e006c920 2024-04-26T21:12:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:09.536 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.533+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 msgr2=0x7f74e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.533+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 0x7f74e006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74e4034000 tx=0x7f74e402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.533+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 msgr2=0x7f74f40cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.533+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c760 tx=0x7f74ec00cc20 comp rx=0 tx=0).stop 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 shutdown_connections 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f74e006c920 0x7f74e006ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74f415fa60 0x7f74f40cea50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74f415f680 0x7f74f40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 --2- 172.21.15.40:0/3837495472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f40b2440 0x7f74f4161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 >> 172.21.15.40:0/3837495472 conn(0x7f74f409e5a0 msgr2=0x7f74f40b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 shutdown_connections 2024-04-26T21:12:09.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.534+0000 7f74f9d91640 1 -- 172.21.15.40:0/3837495472 wait complete. 2024-04-26T21:12:09.545 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-26T21:12:08.919562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219236,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755484,"statfs":{"total":773094113280,"available":772869615616,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001691"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075715+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:04.075715+0000","last_peered":"2024-04-26T21:12:04.075715+0000","last_clean":"2024-04-26T21:12:04.075715+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:04.075715+0000","last_undegraded":"2024-04-26T21:12:04.075715+0000","last_fullsized":"2024-04-26T21:12:04.075715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:17.992415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768372+0000","last_change":"2024-04-26T21:12:01.767311+0000","last_active":"2024-04-26T21:12:02.768372+0000","last_peered":"2024-04-26T21:12:02.768372+0000","last_clean":"2024-04-26T21:12:02.768372+0000","last_became_active":"2024-04-26T21:12:01.766955+0000","last_became_peered":"2024-04-26T21:12:01.766955+0000","last_unstale":"2024-04-26T21:12:02.768372+0000","last_undegraded":"2024-04-26T21:12:02.768372+0000","last_fullsized":"2024-04-26T21:12:02.768372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:29.829212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175853+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:04.175853+0000","last_peered":"2024-04-26T21:12:04.175853+0000","last_clean":"2024-04-26T21:12:04.175853+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:04.175853+0000","last_undegraded":"2024-04-26T21:12:04.175853+0000","last_fullsized":"2024-04-26T21:12:04.175853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175990+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:04.175990+0000","last_peered":"2024-04-26T21:12:04.175990+0000","last_clean":"2024-04-26T21:12:04.175990+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:04.175990+0000","last_undegraded":"2024-04-26T21:12:04.175990+0000","last_fullsized":"2024-04-26T21:12:04.175990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075284+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:04.075284+0000","last_peered":"2024-04-26T21:12:04.075284+0000","last_clean":"2024-04-26T21:12:04.075284+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:04.075284+0000","last_undegraded":"2024-04-26T21:12:04.075284+0000","last_fullsized":"2024-04-26T21:12:04.075284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175693+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:04.175693+0000","last_peered":"2024-04-26T21:12:04.175693+0000","last_clean":"2024-04-26T21:12:04.175693+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:04.175693+0000","last_undegraded":"2024-04-26T21:12:04.175693+0000","last_fullsized":"2024-04-26T21:12:04.175693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075067+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:04.075067+0000","last_peered":"2024-04-26T21:12:04.075067+0000","last_clean":"2024-04-26T21:12:04.075067+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:04.075067+0000","last_undegraded":"2024-04-26T21:12:04.075067+0000","last_fullsized":"2024-04-26T21:12:04.075067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075793+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:04.075793+0000","last_peered":"2024-04-26T21:12:04.075793+0000","last_clean":"2024-04-26T21:12:04.075793+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:04.075793+0000","last_undegraded":"2024-04-26T21:12:04.075793+0000","last_fullsized":"2024-04-26T21:12:04.075793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.197913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075042+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:03.075042+0000","last_peered":"2024-04-26T21:12:03.075042+0000","last_clean":"2024-04-26T21:12:03.075042+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:03.075042+0000","last_undegraded":"2024-04-26T21:12:03.075042+0000","last_fullsized":"2024-04-26T21:12:03.075042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074816+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:03.074816+0000","last_peered":"2024-04-26T21:12:03.074816+0000","last_clean":"2024-04-26T21:12:03.074816+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:03.074816+0000","last_undegraded":"2024-04-26T21:12:03.074816+0000","last_fullsized":"2024-04-26T21:12:03.074816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974567+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:02.974567+0000","last_peered":"2024-04-26T21:12:02.974567+0000","last_clean":"2024-04-26T21:12:02.974567+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:02.974567+0000","last_undegraded":"2024-04-26T21:12:02.974567+0000","last_fullsized":"2024-04-26T21:12:02.974567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175587+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:04.175587+0000","last_peered":"2024-04-26T21:12:04.175587+0000","last_clean":"2024-04-26T21:12:04.175587+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:04.175587+0000","last_undegraded":"2024-04-26T21:12:04.175587+0000","last_fullsized":"2024-04-26T21:12:04.175587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768447+0000","last_change":"2024-04-26T21:12:01.769541+0000","last_active":"2024-04-26T21:12:02.768447+0000","last_peered":"2024-04-26T21:12:02.768447+0000","last_clean":"2024-04-26T21:12:02.768447+0000","last_became_active":"2024-04-26T21:12:01.769171+0000","last_became_peered":"2024-04-26T21:12:01.769171+0000","last_unstale":"2024-04-26T21:12:02.768447+0000","last_undegraded":"2024-04-26T21:12:02.768447+0000","last_fullsized":"2024-04-26T21:12:02.768447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:22.619826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175595+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:04.175595+0000","last_peered":"2024-04-26T21:12:04.175595+0000","last_clean":"2024-04-26T21:12:04.175595+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:04.175595+0000","last_undegraded":"2024-04-26T21:12:04.175595+0000","last_fullsized":"2024-04-26T21:12:04.175595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176236+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:04.176236+0000","last_peered":"2024-04-26T21:12:04.176236+0000","last_clean":"2024-04-26T21:12:04.176236+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:04.176236+0000","last_undegraded":"2024-04-26T21:12:04.176236+0000","last_fullsized":"2024-04-26T21:12:04.176236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075698+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:04.075698+0000","last_peered":"2024-04-26T21:12:04.075698+0000","last_clean":"2024-04-26T21:12:04.075698+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:04.075698+0000","last_undegraded":"2024-04-26T21:12:04.075698+0000","last_fullsized":"2024-04-26T21:12:04.075698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:29.707431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074796+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:03.074796+0000","last_peered":"2024-04-26T21:12:03.074796+0000","last_clean":"2024-04-26T21:12:03.074796+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:03.074796+0000","last_undegraded":"2024-04-26T21:12:03.074796+0000","last_fullsized":"2024-04-26T21:12:03.074796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175801+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:04.175801+0000","last_peered":"2024-04-26T21:12:04.175801+0000","last_clean":"2024-04-26T21:12:04.175801+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:04.175801+0000","last_undegraded":"2024-04-26T21:12:04.175801+0000","last_fullsized":"2024-04-26T21:12:04.175801+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175896+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:04.175896+0000","last_peered":"2024-04-26T21:12:04.175896+0000","last_clean":"2024-04-26T21:12:04.175896+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:04.175896+0000","last_undegraded":"2024-04-26T21:12:04.175896+0000","last_fullsized":"2024-04-26T21:12:04.175896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175677+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:04.175677+0000","last_peered":"2024-04-26T21:12:04.175677+0000","last_clean":"2024-04-26T21:12:04.175677+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:04.175677+0000","last_undegraded":"2024-04-26T21:12:04.175677+0000","last_fullsized":"2024-04-26T21:12:04.175677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075061+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:03.075061+0000","last_peered":"2024-04-26T21:12:03.075061+0000","last_clean":"2024-04-26T21:12:03.075061+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:03.075061+0000","last_undegraded":"2024-04-26T21:12:03.075061+0000","last_fullsized":"2024-04-26T21:12:03.075061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768216+0000","last_change":"2024-04-26T21:12:01.757848+0000","last_active":"2024-04-26T21:12:02.768216+0000","last_peered":"2024-04-26T21:12:02.768216+0000","last_clean":"2024-04-26T21:12:02.768216+0000","last_became_active":"2024-04-26T21:12:01.757563+0000","last_became_peered":"2024-04-26T21:12:01.757563+0000","last_unstale":"2024-04-26T21:12:02.768216+0000","last_undegraded":"2024-04-26T21:12:02.768216+0000","last_fullsized":"2024-04-26T21:12:02.768216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:51.758916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175879+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:04.175879+0000","last_peered":"2024-04-26T21:12:04.175879+0000","last_clean":"2024-04-26T21:12:04.175879+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:04.175879+0000","last_undegraded":"2024-04-26T21:12:04.175879+0000","last_fullsized":"2024-04-26T21:12:04.175879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175961+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:04.175961+0000","last_peered":"2024-04-26T21:12:04.175961+0000","last_clean":"2024-04-26T21:12:04.175961+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:04.175961+0000","last_undegraded":"2024-04-26T21:12:04.175961+0000","last_fullsized":"2024-04-26T21:12:04.175961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074711+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:03.074711+0000","last_peered":"2024-04-26T21:12:03.074711+0000","last_clean":"2024-04-26T21:12:03.074711+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:03.074711+0000","last_undegraded":"2024-04-26T21:12:03.074711+0000","last_fullsized":"2024-04-26T21:12:03.074711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974321+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:02.974321+0000","last_peered":"2024-04-26T21:12:02.974321+0000","last_clean":"2024-04-26T21:12:02.974321+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:02.974321+0000","last_undegraded":"2024-04-26T21:12:02.974321+0000","last_fullsized":"2024-04-26T21:12:02.974321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175972+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:04.175972+0000","last_peered":"2024-04-26T21:12:04.175972+0000","last_clean":"2024-04-26T21:12:04.175972+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:04.175972+0000","last_undegraded":"2024-04-26T21:12:04.175972+0000","last_fullsized":"2024-04-26T21:12:04.175972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074650+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:03.074650+0000","last_peered":"2024-04-26T21:12:03.074650+0000","last_clean":"2024-04-26T21:12:03.074650+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:03.074650+0000","last_undegraded":"2024-04-26T21:12:03.074650+0000","last_fullsized":"2024-04-26T21:12:03.074650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175834+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:04.175834+0000","last_peered":"2024-04-26T21:12:04.175834+0000","last_clean":"2024-04-26T21:12:04.175834+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:04.175834+0000","last_undegraded":"2024-04-26T21:12:04.175834+0000","last_fullsized":"2024-04-26T21:12:04.175834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768245+0000","last_change":"2024-04-26T21:12:01.765248+0000","last_active":"2024-04-26T21:12:02.768245+0000","last_peered":"2024-04-26T21:12:02.768245+0000","last_clean":"2024-04-26T21:12:02.768245+0000","last_became_active":"2024-04-26T21:12:01.764951+0000","last_became_peered":"2024-04-26T21:12:01.764951+0000","last_unstale":"2024-04-26T21:12:02.768245+0000","last_undegraded":"2024-04-26T21:12:02.768245+0000","last_fullsized":"2024-04-26T21:12:02.768245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:24.962313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074890+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:03.074890+0000","last_peered":"2024-04-26T21:12:03.074890+0000","last_clean":"2024-04-26T21:12:03.074890+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:03.074890+0000","last_undegraded":"2024-04-26T21:12:03.074890+0000","last_fullsized":"2024-04-26T21:12:03.074890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176145+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:04.176145+0000","last_peered":"2024-04-26T21:12:04.176145+0000","last_clean":"2024-04-26T21:12:04.176145+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:04.176145+0000","last_undegraded":"2024-04-26T21:12:04.176145+0000","last_fullsized":"2024-04-26T21:12:04.176145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175969+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:04.175969+0000","last_peered":"2024-04-26T21:12:04.175969+0000","last_clean":"2024-04-26T21:12:04.175969+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:04.175969+0000","last_undegraded":"2024-04-26T21:12:04.175969+0000","last_fullsized":"2024-04-26T21:12:04.175969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176007+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:04.176007+0000","last_peered":"2024-04-26T21:12:04.176007+0000","last_clean":"2024-04-26T21:12:04.176007+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:04.176007+0000","last_undegraded":"2024-04-26T21:12:04.176007+0000","last_fullsized":"2024-04-26T21:12:04.176007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768176+0000","last_change":"2024-04-26T21:12:00.744708+0000","last_active":"2024-04-26T21:12:02.768176+0000","last_peered":"2024-04-26T21:12:02.768176+0000","last_clean":"2024-04-26T21:12:02.768176+0000","last_became_active":"2024-04-26T21:12:00.744396+0000","last_became_peered":"2024-04-26T21:12:00.744396+0000","last_unstale":"2024-04-26T21:12:02.768176+0000","last_undegraded":"2024-04-26T21:12:02.768176+0000","last_fullsized":"2024-04-26T21:12:02.768176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:40.330925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974569+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:02.974569+0000","last_peered":"2024-04-26T21:12:02.974569+0000","last_clean":"2024-04-26T21:12:02.974569+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:02.974569+0000","last_undegraded":"2024-04-26T21:12:02.974569+0000","last_fullsized":"2024-04-26T21:12:02.974569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074901+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:03.074901+0000","last_peered":"2024-04-26T21:12:03.074901+0000","last_clean":"2024-04-26T21:12:03.074901+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:03.074901+0000","last_undegraded":"2024-04-26T21:12:03.074901+0000","last_fullsized":"2024-04-26T21:12:03.074901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075445+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:04.075445+0000","last_peered":"2024-04-26T21:12:04.075445+0000","last_clean":"2024-04-26T21:12:04.075445+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:04.075445+0000","last_undegraded":"2024-04-26T21:12:04.075445+0000","last_fullsized":"2024-04-26T21:12:04.075445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175978+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:04.175978+0000","last_peered":"2024-04-26T21:12:04.175978+0000","last_clean":"2024-04-26T21:12:04.175978+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:04.175978+0000","last_undegraded":"2024-04-26T21:12:04.175978+0000","last_fullsized":"2024-04-26T21:12:04.175978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175604+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:04.175604+0000","last_peered":"2024-04-26T21:12:04.175604+0000","last_clean":"2024-04-26T21:12:04.175604+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:04.175604+0000","last_undegraded":"2024-04-26T21:12:04.175604+0000","last_fullsized":"2024-04-26T21:12:04.175604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176048+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:04.176048+0000","last_peered":"2024-04-26T21:12:04.176048+0000","last_clean":"2024-04-26T21:12:04.176048+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:04.176048+0000","last_undegraded":"2024-04-26T21:12:04.176048+0000","last_fullsized":"2024-04-26T21:12:04.176048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075449+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:04.075449+0000","last_peered":"2024-04-26T21:12:04.075449+0000","last_clean":"2024-04-26T21:12:04.075449+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:04.075449+0000","last_undegraded":"2024-04-26T21:12:04.075449+0000","last_fullsized":"2024-04-26T21:12:04.075449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074625+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:03.074625+0000","last_peered":"2024-04-26T21:12:03.074625+0000","last_clean":"2024-04-26T21:12:03.074625+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:03.074625+0000","last_undegraded":"2024-04-26T21:12:03.074625+0000","last_fullsized":"2024-04-26T21:12:03.074625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074691+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:03.074691+0000","last_peered":"2024-04-26T21:12:03.074691+0000","last_clean":"2024-04-26T21:12:03.074691+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:03.074691+0000","last_undegraded":"2024-04-26T21:12:03.074691+0000","last_fullsized":"2024-04-26T21:12:03.074691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075075+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:04.075075+0000","last_peered":"2024-04-26T21:12:04.075075+0000","last_clean":"2024-04-26T21:12:04.075075+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:04.075075+0000","last_undegraded":"2024-04-26T21:12:04.075075+0000","last_fullsized":"2024-04-26T21:12:04.075075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974587+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:02.974587+0000","last_peered":"2024-04-26T21:12:02.974587+0000","last_clean":"2024-04-26T21:12:02.974587+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:02.974587+0000","last_undegraded":"2024-04-26T21:12:02.974587+0000","last_fullsized":"2024-04-26T21:12:02.974587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075598+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:04.075598+0000","last_peered":"2024-04-26T21:12:04.075598+0000","last_clean":"2024-04-26T21:12:04.075598+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:04.075598+0000","last_undegraded":"2024-04-26T21:12:04.075598+0000","last_fullsized":"2024-04-26T21:12:04.075598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075831+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:04.075831+0000","last_peered":"2024-04-26T21:12:04.075831+0000","last_clean":"2024-04-26T21:12:04.075831+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:04.075831+0000","last_undegraded":"2024-04-26T21:12:04.075831+0000","last_fullsized":"2024-04-26T21:12:04.075831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:39.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768273+0000","last_change":"2024-04-26T21:12:00.742717+0000","last_active":"2024-04-26T21:12:02.768273+0000","last_peered":"2024-04-26T21:12:02.768273+0000","last_clean":"2024-04-26T21:12:02.768273+0000","last_became_active":"2024-04-26T21:12:00.741757+0000","last_became_peered":"2024-04-26T21:12:00.741757+0000","last_unstale":"2024-04-26T21:12:02.768273+0000","last_undegraded":"2024-04-26T21:12:02.768273+0000","last_fullsized":"2024-04-26T21:12:02.768273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:33.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176261+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:04.176261+0000","last_peered":"2024-04-26T21:12:04.176261+0000","last_clean":"2024-04-26T21:12:04.176261+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:04.176261+0000","last_undegraded":"2024-04-26T21:12:04.176261+0000","last_fullsized":"2024-04-26T21:12:04.176261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075479+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:04.075479+0000","last_peered":"2024-04-26T21:12:04.075479+0000","last_clean":"2024-04-26T21:12:04.075479+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:04.075479+0000","last_undegraded":"2024-04-26T21:12:04.075479+0000","last_fullsized":"2024-04-26T21:12:04.075479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075131+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:04.075131+0000","last_peered":"2024-04-26T21:12:04.075131+0000","last_clean":"2024-04-26T21:12:04.075131+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:04.075131+0000","last_undegraded":"2024-04-26T21:12:04.075131+0000","last_fullsized":"2024-04-26T21:12:04.075131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175661+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:04.175661+0000","last_peered":"2024-04-26T21:12:04.175661+0000","last_clean":"2024-04-26T21:12:04.175661+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:04.175661+0000","last_undegraded":"2024-04-26T21:12:04.175661+0000","last_fullsized":"2024-04-26T21:12:04.175661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074973+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:04.074973+0000","last_peered":"2024-04-26T21:12:04.074973+0000","last_clean":"2024-04-26T21:12:04.074973+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:04.074973+0000","last_undegraded":"2024-04-26T21:12:04.074973+0000","last_fullsized":"2024-04-26T21:12:04.074973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175510+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:04.175510+0000","last_peered":"2024-04-26T21:12:04.175510+0000","last_clean":"2024-04-26T21:12:04.175510+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:04.175510+0000","last_undegraded":"2024-04-26T21:12:04.175510+0000","last_fullsized":"2024-04-26T21:12:04.175510+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767742+0000","last_change":"2024-04-26T21:12:01.758217+0000","last_active":"2024-04-26T21:12:02.767742+0000","last_peered":"2024-04-26T21:12:02.767742+0000","last_clean":"2024-04-26T21:12:02.767742+0000","last_became_active":"2024-04-26T21:12:01.756525+0000","last_became_peered":"2024-04-26T21:12:01.756525+0000","last_unstale":"2024-04-26T21:12:02.767742+0000","last_undegraded":"2024-04-26T21:12:02.767742+0000","last_fullsized":"2024-04-26T21:12:02.767742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:17.830865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176252+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:04.176252+0000","last_peered":"2024-04-26T21:12:04.176252+0000","last_clean":"2024-04-26T21:12:04.176252+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:04.176252+0000","last_undegraded":"2024-04-26T21:12:04.176252+0000","last_fullsized":"2024-04-26T21:12:04.176252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974416+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:02.974416+0000","last_peered":"2024-04-26T21:12:02.974416+0000","last_clean":"2024-04-26T21:12:02.974416+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:02.974416+0000","last_undegraded":"2024-04-26T21:12:02.974416+0000","last_fullsized":"2024-04-26T21:12:02.974416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768484+0000","last_change":"2024-04-26T21:12:00.745395+0000","last_active":"2024-04-26T21:12:02.768484+0000","last_peered":"2024-04-26T21:12:02.768484+0000","last_clean":"2024-04-26T21:12:02.768484+0000","last_became_active":"2024-04-26T21:12:00.745088+0000","last_became_peered":"2024-04-26T21:12:00.745088+0000","last_unstale":"2024-04-26T21:12:02.768484+0000","last_undegraded":"2024-04-26T21:12:02.768484+0000","last_fullsized":"2024-04-26T21:12:02.768484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:29.139093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176129+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:04.176129+0000","last_peered":"2024-04-26T21:12:04.176129+0000","last_clean":"2024-04-26T21:12:04.176129+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:04.176129+0000","last_undegraded":"2024-04-26T21:12:04.176129+0000","last_fullsized":"2024-04-26T21:12:04.176129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075714+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:04.075714+0000","last_peered":"2024-04-26T21:12:04.075714+0000","last_clean":"2024-04-26T21:12:04.075714+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:04.075714+0000","last_undegraded":"2024-04-26T21:12:04.075714+0000","last_fullsized":"2024-04-26T21:12:04.075714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768539+0000","last_change":"2024-04-26T21:12:01.755969+0000","last_active":"2024-04-26T21:12:02.768539+0000","last_peered":"2024-04-26T21:12:02.768539+0000","last_clean":"2024-04-26T21:12:02.768539+0000","last_became_active":"2024-04-26T21:12:01.755628+0000","last_became_peered":"2024-04-26T21:12:01.755628+0000","last_unstale":"2024-04-26T21:12:02.768539+0000","last_undegraded":"2024-04-26T21:12:02.768539+0000","last_fullsized":"2024-04-26T21:12:02.768539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:42:31.761499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768360+0000","last_change":"2024-04-26T21:12:00.740944+0000","last_active":"2024-04-26T21:12:02.768360+0000","last_peered":"2024-04-26T21:12:02.768360+0000","last_clean":"2024-04-26T21:12:02.768360+0000","last_became_active":"2024-04-26T21:12:00.740532+0000","last_became_peered":"2024-04-26T21:12:00.740532+0000","last_unstale":"2024-04-26T21:12:02.768360+0000","last_undegraded":"2024-04-26T21:12:02.768360+0000","last_fullsized":"2024-04-26T21:12:02.768360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:16.536321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074939+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:03.074939+0000","last_peered":"2024-04-26T21:12:03.074939+0000","last_clean":"2024-04-26T21:12:03.074939+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:03.074939+0000","last_undegraded":"2024-04-26T21:12:03.074939+0000","last_fullsized":"2024-04-26T21:12:03.074939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176063+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:04.176063+0000","last_peered":"2024-04-26T21:12:04.176063+0000","last_clean":"2024-04-26T21:12:04.176063+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:04.176063+0000","last_undegraded":"2024-04-26T21:12:04.176063+0000","last_fullsized":"2024-04-26T21:12:04.176063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074967+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:04.074967+0000","last_peered":"2024-04-26T21:12:04.074967+0000","last_clean":"2024-04-26T21:12:04.074967+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:04.074967+0000","last_undegraded":"2024-04-26T21:12:04.074967+0000","last_fullsized":"2024-04-26T21:12:04.074967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974333+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:02.974333+0000","last_peered":"2024-04-26T21:12:02.974333+0000","last_clean":"2024-04-26T21:12:02.974333+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:02.974333+0000","last_undegraded":"2024-04-26T21:12:02.974333+0000","last_fullsized":"2024-04-26T21:12:02.974333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768356+0000","last_change":"2024-04-26T21:12:00.749992+0000","last_active":"2024-04-26T21:12:02.768356+0000","last_peered":"2024-04-26T21:12:02.768356+0000","last_clean":"2024-04-26T21:12:02.768356+0000","last_became_active":"2024-04-26T21:12:00.749601+0000","last_became_peered":"2024-04-26T21:12:00.749601+0000","last_unstale":"2024-04-26T21:12:02.768356+0000","last_undegraded":"2024-04-26T21:12:02.768356+0000","last_fullsized":"2024-04-26T21:12:02.768356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:39.587954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768336+0000","last_change":"2024-04-26T21:12:00.747054+0000","last_active":"2024-04-26T21:12:02.768336+0000","last_peered":"2024-04-26T21:12:02.768336+0000","last_clean":"2024-04-26T21:12:02.768336+0000","last_became_active":"2024-04-26T21:12:00.746691+0000","last_became_peered":"2024-04-26T21:12:00.746691+0000","last_unstale":"2024-04-26T21:12:02.768336+0000","last_undegraded":"2024-04-26T21:12:02.768336+0000","last_fullsized":"2024-04-26T21:12:02.768336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:28.714030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075403+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:04.075403+0000","last_peered":"2024-04-26T21:12:04.075403+0000","last_clean":"2024-04-26T21:12:04.075403+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:04.075403+0000","last_undegraded":"2024-04-26T21:12:04.075403+0000","last_fullsized":"2024-04-26T21:12:04.075403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074723+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:03.074723+0000","last_peered":"2024-04-26T21:12:03.074723+0000","last_clean":"2024-04-26T21:12:03.074723+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:03.074723+0000","last_undegraded":"2024-04-26T21:12:03.074723+0000","last_fullsized":"2024-04-26T21:12:03.074723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175995+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:04.175995+0000","last_peered":"2024-04-26T21:12:04.175995+0000","last_clean":"2024-04-26T21:12:04.175995+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:04.175995+0000","last_undegraded":"2024-04-26T21:12:04.175995+0000","last_fullsized":"2024-04-26T21:12:04.175995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075331+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:04.075331+0000","last_peered":"2024-04-26T21:12:04.075331+0000","last_clean":"2024-04-26T21:12:04.075331+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:04.075331+0000","last_undegraded":"2024-04-26T21:12:04.075331+0000","last_fullsized":"2024-04-26T21:12:04.075331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176188+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:04.176188+0000","last_peered":"2024-04-26T21:12:04.176188+0000","last_clean":"2024-04-26T21:12:04.176188+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:04.176188+0000","last_undegraded":"2024-04-26T21:12:04.176188+0000","last_fullsized":"2024-04-26T21:12:04.176188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075571+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:04.075571+0000","last_peered":"2024-04-26T21:12:04.075571+0000","last_clean":"2024-04-26T21:12:04.075571+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:04.075571+0000","last_undegraded":"2024-04-26T21:12:04.075571+0000","last_fullsized":"2024-04-26T21:12:04.075571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175980+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:04.175980+0000","last_peered":"2024-04-26T21:12:04.175980+0000","last_clean":"2024-04-26T21:12:04.175980+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:04.175980+0000","last_undegraded":"2024-04-26T21:12:04.175980+0000","last_fullsized":"2024-04-26T21:12:04.175980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176091+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:04.176091+0000","last_peered":"2024-04-26T21:12:04.176091+0000","last_clean":"2024-04-26T21:12:04.176091+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:04.176091+0000","last_undegraded":"2024-04-26T21:12:04.176091+0000","last_fullsized":"2024-04-26T21:12:04.176091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":81325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":672241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:09.549 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:12:09.549 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:12:09.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:12:09.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T21:12:09.550 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T21:12:09.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 Processor -- start 2024-04-26T21:12:09.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 -- start start 2024-04-26T21:12:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fad6c0d5610 0x7fad6c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0b2510 con 0x7fad6c15f720 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0b27f0 con 0x7fad6c15f340 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.686+0000 7fad7106d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c0b2ad0 con 0x7fad6c0d5610 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad63fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad6c15f720 0x7fad6c0b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46226/0 (socket says 172.21.15.40:46226) 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad63fff640 1 -- 172.21.15.40:0/1919147358 learned_addr learned my addr 172.21.15.40:0/1919147358 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6b59e640 1 --2- 172.21.15.40:0/1919147358 >> v2:172.21.15.184:3300/0 conn(0x7fad6c0d5610 0x7fad6c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667278981 0 0) 0x7fad6c0b2510 con 0x7fad6c15f720 2024-04-26T21:12:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad58003520 con 0x7fad6c15f720 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6b59e640 1 -- 172.21.15.40:0/1919147358 >> v1:172.21.15.184:6789/0 conn(0x7fad6c15f340 legacy=0x7fad6c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025193628 0 0) 0x7fad6c0b27f0 con 0x7fad6c15f340 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6b59e640 1 -- 172.21.15.40:0/1919147358 >> v1:172.21.15.40:6789/0 conn(0x7fad6c15f720 legacy=0x7fad6c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.687+0000 7fad6b59e640 1 -- 172.21.15.40:0/1919147358 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0b2db0 con 0x7fad6c0d5610 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad6b59e640 1 --2- 172.21.15.40:0/1919147358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c0d5610 0x7fad6c0b1e80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad64009d90 tx=0x7fad6402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e2e12422cdeb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64007740 con 0x7fad6c0d5610 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad640078e0 con 0x7fad6c0d5610 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad6a59c640 1 -- 172.21.15.40:0/1919147358 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64007c90 con 0x7fad6c0d5610 2024-04-26T21:12:09.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad7106d640 1 -- 172.21.15.40:0/1919147358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c0d5610 msgr2=0x7fad6c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad7106d640 1 --2- 172.21.15.40:0/1919147358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c0d5610 0x7fad6c0b1e80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad64009d90 tx=0x7fad6402a660 comp rx=0 tx=0).stop 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad7106d640 1 -- 172.21.15.40:0/1919147358 shutdown_connections 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad7106d640 1 --2- 172.21.15.40:0/1919147358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c0d5610 0x7fad6c0b1e80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.688+0000 7fad7106d640 1 -- 172.21.15.40:0/1919147358 >> 172.21.15.40:0/1919147358 conn(0x7fad6c08ec70 msgr2=0x7fad6c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.689+0000 7fad7106d640 1 -- 172.21.15.40:0/1919147358 shutdown_connections 2024-04-26T21:12:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.689+0000 7fad7106d640 1 -- 172.21.15.40:0/1919147358 wait complete. 2024-04-26T21:12:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.690+0000 7fad7106d640 1 Processor -- start 2024-04-26T21:12:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.690+0000 7fad7106d640 1 -- start start 2024-04-26T21:12:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.690+0000 7fad7106d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c15f340 0x7fad6c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad6c15f720 0x7fad6c0b9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0d40d0 con 0x7fad6c0d5610 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0d3f50 con 0x7fad6c15f340 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0b3cc0 con 0x7fad6c15f720 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51926/0 (socket says 172.21.15.40:51926) 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 -- 172.21.15.40:0/2583127057 learned_addr learned my addr 172.21.15.40:0/2583127057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad63fff640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c15f340 0x7fad6c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6b59e640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad6c15f720 0x7fad6c0b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 -- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad6c15f720 msgr2=0x7fad6c0b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad6c15f720 0x7fad6c0b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 -- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c15f340 msgr2=0x7fad6c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c15f340 0x7fad6c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 -- 172.21.15.40:0/2583127057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0d1800 con 0x7fad6c0d5610 2024-04-26T21:12:09.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad6ad9d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c4b0 tx=0x7fad5c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c017070 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5c005480 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c0d1520 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c003830 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.691+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c0c9100 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.692+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c0c9530 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.693+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad5c01a070 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.693+0000 7fad637fe640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 0x7fad4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.693+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fad5c08ae40 con 0x7fad6c0d5610 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.693+0000 7fad63fff640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 0x7fad4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.694+0000 7fad63fff640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 0x7fad4c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0cb070 tx=0x7fad54007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.697+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad5c058c20 con 0x7fad6c0d5610 2024-04-26T21:12:09.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.938+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fad6c06a290 con 0x7fad6c0d5610 2024-04-26T21:12:09.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.941+0000 7fad637fe640 1 -- 172.21.15.40:0/2583127057 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fad5c05cf20 con 0x7fad6c0d5610 2024-04-26T21:12:09.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 msgr2=0x7fad4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 0x7fad4c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0cb070 tx=0x7fad54007d30 comp rx=0 tx=0).stop 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 msgr2=0x7fad6c0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c4b0 tx=0x7fad5c00c970 comp rx=0 tx=0).stop 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 shutdown_connections 2024-04-26T21:12:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fad4c06c920 0x7fad4c06ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad6c15f720 0x7fad6c0b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad6c15f340 0x7fad6c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 --2- 172.21.15.40:0/2583127057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad6c0d5610 0x7fad6c0cb840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 >> 172.21.15.40:0/2583127057 conn(0x7fad6c08ec70 msgr2=0x7fad6c06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 shutdown_connections 2024-04-26T21:12:09.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:09.944+0000 7fad7106d640 1 -- 172.21.15.40:0/2583127057 wait complete. 2024-04-26T21:12:09.956 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2784003994},{"type":"v1","addr":"172.21.15.184:6833","nonce":2784003994}]},"active_addr":"172.21.15.184:6833/2784003994","active_change":"2024-04-26T21:11:40.905438+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2909963690}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3913736030}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":745912994}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2454560523}]}]} 2024-04-26T21:12:09.959 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T21:12:09.959 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T21:12:09.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.059+0000 7f4470baf640 1 Processor -- start 2024-04-26T21:12:10.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.059+0000 7f4470baf640 1 -- start start 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4470baf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f446c0d11b0 0x7f446c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4470baf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 0x7f446c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4470baf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446c0d1740 con 0x7f446c0cf2e0 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4470baf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f446c06e710 con 0x7f446c0d11b0 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4470baf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f446c0d1a20 con 0x7f446c06e2c0 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f446c0d11b0 0x7f446c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f446c0d11b0 0x7f446c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51934/0 (socket says 172.21.15.40:51934) 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 -- 172.21.15.40:0/1952820737 learned_addr learned my addr 172.21.15.40:0/1952820737 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f446ad9d640 1 --2- 172.21.15.40:0/1952820737 >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 0x7f446c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 -- 172.21.15.40:0/1952820737 >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 msgr2=0x7f446c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 --2- 172.21.15.40:0/1952820737 >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 0x7f446c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 -- 172.21.15.40:0/1952820737 >> v1:172.21.15.184:6789/0 conn(0x7f446c0cf2e0 legacy=0x7f446c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f4461d9b640 1 -- 172.21.15.40:0/1952820737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c0d1d00 con 0x7f446c0d11b0 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.060+0000 7f446ad9d640 1 --2- 172.21.15.40:0/1952820737 >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 0x7f446c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4461d9b640 1 --2- 172.21.15.40:0/1952820737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0d11b0 0x7f446c06dd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4454009870 tx=0x7f445402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7925ba97570ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4469d9b640 1 -- 172.21.15.40:0/1952820737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454037070 con 0x7f446c0d11b0 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4469d9b640 1 -- 172.21.15.40:0/1952820737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4454032440 con 0x7f446c0d11b0 2024-04-26T21:12:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4469d9b640 1 -- 172.21.15.40:0/1952820737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454032810 con 0x7f446c0d11b0 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 -- 172.21.15.40:0/1952820737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0d11b0 msgr2=0x7f446c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 --2- 172.21.15.40:0/1952820737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0d11b0 0x7f446c06dd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4454009870 tx=0x7f445402a250 comp rx=0 tx=0).stop 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 -- 172.21.15.40:0/1952820737 shutdown_connections 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 --2- 172.21.15.40:0/1952820737 >> v2:172.21.15.184:3300/0 conn(0x7f446c06e2c0 0x7f446c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 --2- 172.21.15.40:0/1952820737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0d11b0 0x7f446c06dd80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.061+0000 7f4470baf640 1 -- 172.21.15.40:0/1952820737 >> 172.21.15.40:0/1952820737 conn(0x7f446c0b4a60 msgr2=0x7f446c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.062+0000 7f4470baf640 1 -- 172.21.15.40:0/1952820737 shutdown_connections 2024-04-26T21:12:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.062+0000 7f4470baf640 1 -- 172.21.15.40:0/1952820737 wait complete. 2024-04-26T21:12:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.063+0000 7f4470baf640 1 Processor -- start 2024-04-26T21:12:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.063+0000 7f4470baf640 1 -- start start 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 0x7f446c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c0d11b0 0x7f446c0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0cf6c0 con 0x7f446c0cf2e0 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d22c0 con 0x7f446c0d11b0 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f446c0cf840 con 0x7f446c06e2c0 2024-04-26T21:12:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51938/0 (socket says 172.21.15.40:51938) 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 -- 172.21.15.40:0/889964310 learned_addr learned my addr 172.21.15.40:0/889964310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f446ad9d640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c0d11b0 0x7f446c0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f446a59c640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 0x7f446c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 -- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 msgr2=0x7f446c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 0x7f446c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 -- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c0d11b0 msgr2=0x7f446c0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c0d11b0 0x7f446c0b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 -- 172.21.15.40:0/889964310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c0b6210 con 0x7f446c0cf2e0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f446a59c640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 0x7f446c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4461d9b640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44540090b0 tx=0x7f44540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454037040 con 0x7f446c0cf2e0 2024-04-26T21:12:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4454004250 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c0b64f0 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.064+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454030770 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.065+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c0b6d90 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.066+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446c0b66c0 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.066+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4454030910 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.066+0000 7f44637fe640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 0x7f444c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.066+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f44540ac9d0 con 0x7f446c0cf2e0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.066+0000 7f446a59c640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 0x7f444c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.067+0000 7f446a59c640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 0x7f444c06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f445c006fd0 tx=0x7f445c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.071+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f445407ae90 con 0x7f446c0cf2e0 2024-04-26T21:12:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.255+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f446c0b66c0 con 0x7f446c0cf2e0 2024-04-26T21:12:10.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.256+0000 7f44637fe640 1 -- 172.21.15.40:0/889964310 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f445407a790 con 0x7f446c0cf2e0 2024-04-26T21:12:10.257 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:10.258 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 msgr2=0x7f444c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 0x7f444c06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f445c006fd0 tx=0x7f445c008040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 msgr2=0x7f446c0b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44540090b0 tx=0x7f44540046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 shutdown_connections 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f444c06c920 0x7f444c06ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c0d11b0 0x7f446c0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f446c0cf2e0 0x7f446c0b5790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 --2- 172.21.15.40:0/889964310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f446c06e2c0 0x7f446c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 >> 172.21.15.40:0/889964310 conn(0x7f446c0b4a60 msgr2=0x7f446c0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 shutdown_connections 2024-04-26T21:12:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.258+0000 7f4470baf640 1 -- 172.21.15.40:0/889964310 wait complete. 2024-04-26T21:12:10.271 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T21:12:10.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:10.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.370+0000 7f7af4609640 1 Processor -- start 2024-04-26T21:12:10.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.370+0000 7f7af4609640 1 -- start start 2024-04-26T21:12:10.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af4609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af4609640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7aec15fa20 0x7f7aec0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af4609640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aec06cbc0 con 0x7f7aec162f30 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af4609640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7aec163df0 con 0x7f7aec15fa20 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af4609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec163830 con 0x7f7aec06c2a0 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af1ba4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7aec15fa20 0x7f7aec0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af1ba4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7aec15fa20 0x7f7aec0cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51952/0 (socket says 172.21.15.40:51952) 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af2ba6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7aec162f30 0x7f7aec06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46230/0 (socket says 172.21.15.40:46230) 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.371+0000 7f7af1ba4640 1 -- 172.21.15.40:0/2810301808 learned_addr learned my addr 172.21.15.40:0/2810301808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af23a5640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af1ba4640 1 -- 172.21.15.40:0/2810301808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 msgr2=0x7f7aec06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af1ba4640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af1ba4640 1 -- 172.21.15.40:0/2810301808 >> v1:172.21.15.40:6789/0 conn(0x7f7aec162f30 legacy=0x7f7aec06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af13a3640 1 -- 172.21.15.40:0/2810301808 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316932467 0 0) 0x7f7aec06cbc0 con 0x7f7aec162f30 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af1ba4640 1 -- 172.21.15.40:0/2810301808 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec163b10 con 0x7f7aec15fa20 2024-04-26T21:12:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af23a5640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.372+0000 7f7af1ba4640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec0cf150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009870 tx=0x7f7ae002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61fed403ba74a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af13a3640 1 -- 172.21.15.40:0/2810301808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0037070 con 0x7f7aec15fa20 2024-04-26T21:12:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af13a3640 1 -- 172.21.15.40:0/2810301808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae0032440 con 0x7f7aec15fa20 2024-04-26T21:12:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af13a3640 1 -- 172.21.15.40:0/2810301808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0032810 con 0x7f7aec15fa20 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 -- 172.21.15.40:0/2810301808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 msgr2=0x7f7aec0cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec0cf150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009870 tx=0x7f7ae002a130 comp rx=0 tx=0).stop 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 -- 172.21.15.40:0/2810301808 shutdown_connections 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec0cf150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 --2- 172.21.15.40:0/2810301808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 -- 172.21.15.40:0/2810301808 >> 172.21.15.40:0/2810301808 conn(0x7f7aec0b3b10 msgr2=0x7f7aec0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 -- 172.21.15.40:0/2810301808 shutdown_connections 2024-04-26T21:12:10.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.373+0000 7f7af4609640 1 -- 172.21.15.40:0/2810301808 wait complete. 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.374+0000 7f7af4609640 1 Processor -- start 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.374+0000 7f7af4609640 1 -- start start 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7aec162f30 0x7f7aec1f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec0cf690 con 0x7f7aec15fa20 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec0cec30 con 0x7f7aec162f30 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af4609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7aec1606a0 con 0x7f7aec06c2a0 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51954/0 (socket says 172.21.15.40:51954) 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 -- 172.21.15.40:0/1642920726 learned_addr learned my addr 172.21.15.40:0/1642920726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af2ba6640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7aec162f30 0x7f7aec1f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af23a5640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 -- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 msgr2=0x7f7aec15a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec15a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 -- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7aec162f30 msgr2=0x7f7aec1f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7aec162f30 0x7f7aec1f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.375+0000 7f7af1ba4640 1 -- 172.21.15.40:0/1642920726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec162590 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7af1ba4640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009870 tx=0x7f7ae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0041070 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae003c890 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0037040 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec162870 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.376+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec0b1a80 con 0x7f7aec15fa20 2024-04-26T21:12:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.377+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec0af730 con 0x7f7aec15fa20 2024-04-26T21:12:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.377+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae0044070 con 0x7f7aec15fa20 2024-04-26T21:12:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.378+0000 7f7aeaffd640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 0x7f7ad806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.378+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7ae00ac860 con 0x7f7aec15fa20 2024-04-26T21:12:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.378+0000 7f7af23a5640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 0x7f7ad806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.379+0000 7f7af23a5640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 0x7f7ad806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400b840 tx=0x7f7ae4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.382+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ae007ad20 con 0x7f7aec15fa20 2024-04-26T21:12:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.569+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7aec0693d0 con 0x7f7aec15fa20 2024-04-26T21:12:10.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.570+0000 7f7aeaffd640 1 -- 172.21.15.40:0/1642920726 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f7ae007a620 con 0x7f7aec15fa20 2024-04-26T21:12:10.571 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 msgr2=0x7f7ad806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 0x7f7ad806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400b840 tx=0x7f7ae4008040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 msgr2=0x7f7aec1f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009870 tx=0x7f7ae00046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 shutdown_connections 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7ad806c920 0x7f7ad806ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7aec162f30 0x7f7aec1f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7aec15fa20 0x7f7aec1f8520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 --2- 172.21.15.40:0/1642920726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7aec06c2a0 0x7f7aec15a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 >> 172.21.15.40:0/1642920726 conn(0x7f7aec0b3b10 msgr2=0x7f7aec0d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 shutdown_connections 2024-04-26T21:12:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.572+0000 7f7af4609640 1 -- 172.21.15.40:0/1642920726 wait complete. 2024-04-26T21:12:10.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T21:12:10.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T21:12:10.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T21:12:10.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T21:12:10.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T21:12:10.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T21:12:10.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T21:12:10.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T21:12:10.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.762+0000 7f73ecb5f640 1 Processor -- start 2024-04-26T21:12:10.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.762+0000 7f73ecb5f640 1 -- start start 2024-04-26T21:12:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.763+0000 7fe7316c0640 1 Processor -- start 2024-04-26T21:12:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f2e95b38640 1 Processor -- start 2024-04-26T21:12:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f2e95b38640 1 -- start start 2024-04-26T21:12:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.763+0000 7fe7316c0640 1 -- start start 2024-04-26T21:12:10.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f290a7e7640 1 Processor -- start 2024-04-26T21:12:10.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f73ecb5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73e8073ae0 0x7f73e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f73ecb5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 0x7f73e80b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73e8073ae0 0x7f73e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e5d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 0x7f73e80b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73e8073ae0 0x7f73e806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48310/0 (socket says 172.21.15.40:48310) 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e5d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 0x7f73e80b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40252/0 (socket says 172.21.15.40:40252) 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 -- 172.21.15.40:0/2755433777 learned_addr learned my addr 172.21.15.40:0/2755433777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e80b5cc0 con 0x7f73e80b58b0 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f73e806f3c0 con 0x7f73e8073ec0 2024-04-26T21:12:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f73e80b5f30 con 0x7f73e8073ae0 2024-04-26T21:12:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7ffa9f59e640 1 Processor -- start 2024-04-26T21:12:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7ffa9f59e640 1 -- start start 2024-04-26T21:12:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7f290a7e7640 1 -- start start 2024-04-26T21:12:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.764+0000 7fe7316c0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe724009190 0x7fe724052f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe7316c0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe724009570 con 0x7fe72403aa20 2024-04-26T21:12:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe7316c0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe724035970 con 0x7fe724048c40 2024-04-26T21:12:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe7316c0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe724035c30 con 0x7fe724009190 2024-04-26T21:12:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe730ebf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe72403aa20 0x7fe72403ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58426/0 (socket says 172.21.15.40:58426) 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe730ebf640 1 -- 172.21.15.40:0/1871109561 learned_addr learned my addr 172.21.15.40:0/1871109561 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe72bfff640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe724009190 0x7fe724052f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86466590 0 0) 0x7fe724035970 con 0x7fe724048c40 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe71c003520 con 0x7fe724048c40 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f2e95b38640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e88050500 0x7f2e88007bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e95b38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e95b38640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e880512f0 con 0x7f2e880dfd40 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e95b38640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e880549d0 con 0x7f2e88050500 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e95b38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e88054cb0 con 0x7f2e880e0120 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 -- 172.21.15.40:0/2755433777 >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 msgr2=0x7f73e80b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 --2- 172.21.15.40:0/2755433777 >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 0x7f73e80b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 -- 172.21.15.40:0/2755433777 >> v1:172.21.15.40:6789/0 conn(0x7f73e80b58b0 legacy=0x7f73e8072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f73e659c640 1 -- 172.21.15.40:0/2755433777 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80b6210 con 0x7f73e8073ae0 2024-04-26T21:12:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f73e659c640 1 --2- 172.21.15.40:0/2755433777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ae0 0x7f73e806ee80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b7170 tx=0x7f73d802c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286a68e4226faaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 0x7ffa980502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9f59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffa98053e70 0x7ffa980542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9f59e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa98050d50 con 0x7ffa98054980 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9f59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffa98009500 con 0x7ffa98053e70 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa98050ff0 con 0x7ffa98053a90 2024-04-26T21:12:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9dd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffa98054980 0x7ffa98008df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58444/0 (socket says 172.21.15.40:58444) 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7ffa9dd9b640 1 -- 172.21.15.40:0/2115700430 learned_addr learned my addr 172.21.15.40:0/2115700430 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/2115700430 >> v2:172.21.15.184:3300/0 conn(0x7ffa98053e70 0x7ffa980542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f290a7e7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29040d2080 0x7f29040b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.765+0000 7f290a7e7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29040d0980 con 0x7f29040b60c0 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f290a7e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29040d0c60 con 0x7f29040d5b50 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f290a7e7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f29040d0720 con 0x7f29040d2080 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f2903fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f29040b60c0 0x7f29040d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58438/0 (socket says 172.21.15.40:58438) 2024-04-26T21:12:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f2903fff640 1 -- 172.21.15.40:0/2522570651 learned_addr learned my addr 172.21.15.40:0/2522570651 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f29037fe640 1 --2- 172.21.15.40:0/2522570651 >> v2:172.21.15.40:3300/0 conn(0x7f29040d2080 0x7f29040b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f29037fe640 1 -- 172.21.15.40:0/2522570651 >> v1:172.21.15.184:6789/0 conn(0x7f29040d5b50 legacy=0x7f29040b80f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:12:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f29037fe640 1 -- 172.21.15.40:0/2522570651 >> v1:172.21.15.40:6789/0 conn(0x7f29040b60c0 legacy=0x7f29040d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7f29037fe640 1 -- 172.21.15.40:0/2522570651 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29040d0f40 con 0x7f29040d2080 2024-04-26T21:12:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266847418 0 0) 0x7fe71c003520 con 0x7fe724048c40 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe724009190 msgr2=0x7fe724052f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe724009190 0x7fe724052f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 >> v1:172.21.15.40:6789/0 conn(0x7fe72403aa20 legacy=0x7fe72403ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe724035f10 con 0x7fe724048c40 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe718001670 con 0x7fe724048c40 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 >> v1:172.21.15.184:6789/0 conn(0x7fe724048c40 legacy=0x7fe724049060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0039b0 0x7fe71c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.766+0000 7fe72affd640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c004800 0x7fe71c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe72affd640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0051c0 0x7fe71c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe724035c30 con 0x7fe71c004800 2024-04-26T21:12:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c00c090 con 0x7fe71c0039b0 2024-04-26T21:12:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c00c290 con 0x7fe71c0051c0 2024-04-26T21:12:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9e59c640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 0x7ffa980502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f51cffff640 1 Processor -- start 2024-04-26T21:12:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f73e559a640 1 -- 172.21.15.40:0/2755433777 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d800b040 con 0x7f73e8073ae0 2024-04-26T21:12:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0051c0 0x7fe71c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f51cffff640 1 -- start start 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe72bfff640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0039b0 0x7fe71c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9ed9d640 1 -- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 msgr2=0x7ffa980502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 0x7ffa980502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9ed9d640 1 -- 172.21.15.40:0/2115700430 >> v1:172.21.15.40:6789/0 conn(0x7ffa98054980 legacy=0x7ffa98008df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9ed9d640 1 -- 172.21.15.40:0/2115700430 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa980085f0 con 0x7ffa98053e70 2024-04-26T21:12:10.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7ffa9e59c640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 0x7ffa980502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f73e559a640 1 -- 172.21.15.40:0/2755433777 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d8031710 con 0x7f73e8073ae0 2024-04-26T21:12:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7feae17a4640 1 Processor -- start 2024-04-26T21:12:10.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f29037fe640 1 --2- 172.21.15.40:0/2522570651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d2080 0x7f29040b64a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097a0 tx=0x7f28f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c429b99c0e22dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f73e559a640 1 -- 172.21.15.40:0/2755433777 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8031ae0 con 0x7f73e8073ae0 2024-04-26T21:12:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e88050500 0x7f2e88007bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56910/0 (socket says 172.21.15.40:56910) 2024-04-26T21:12:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8ffff640 1 -- 172.21.15.40:0/1164415883 learned_addr learned my addr 172.21.15.40:0/1164415883 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8effd640 1 -- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 msgr2=0x7f2e88008110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8effd640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8effd640 1 -- 172.21.15.40:0/1164415883 >> v1:172.21.15.40:6789/0 conn(0x7f2e880dfd40 legacy=0x7f2e88050be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8effd640 1 -- 172.21.15.40:0/1164415883 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e88054f90 con 0x7f2e88050500 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7f2e8ffff640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053e70 0x7ffa980542e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0051470 tx=0x7ffaa00a8960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57f1e07032010fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7ffa9d59a640 1 -- 172.21.15.40:0/2115700430 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b2020 con 0x7ffa98053e70 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 -- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0039b0 msgr2=0x7fe71c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0039b0 0x7fe71c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 -- 172.21.15.40:0/1871109561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c004800 msgr2=0x7fe71c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe71c004800 0x7fe71c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7fe730ebf640 1 -- 172.21.15.40:0/1871109561 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c003520 con 0x7fe71c0051c0 2024-04-26T21:12:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7ffa9d59a640 1 -- 172.21.15.40:0/2115700430 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa00723a0 con 0x7ffa98053e70 2024-04-26T21:12:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7ffa9d59a640 1 -- 172.21.15.40:0/2115700430 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0070a70 con 0x7ffa98053e70 2024-04-26T21:12:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f51cffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c80061f0 0x7f51c8056fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7fe730ebf640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0051c0 0x7fe71c00b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe72c06b0c0 tx=0x7fe71c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02e30b5f049a675 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f51cffff640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c8057a60 con 0x7f51c80075e0 2024-04-26T21:12:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f51cffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c8057d40 con 0x7f51c80574e0 2024-04-26T21:12:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f51cffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51c8058020 con 0x7f51c80061f0 2024-04-26T21:12:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f2e8effd640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e88050500 0x7f2e88007bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400b850 tx=0x7f2e84032e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7822551bcb7bb191 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f2902ffd640 1 -- 172.21.15.40:0/2522570651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4037070 con 0x7f29040d2080 2024-04-26T21:12:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f2902ffd640 1 -- 172.21.15.40:0/2522570651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f402ae50 con 0x7f29040d2080 2024-04-26T21:12:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f2902ffd640 1 -- 172.21.15.40:0/2522570651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4032660 con 0x7f29040d2080 2024-04-26T21:12:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.768+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ae0 msgr2=0x7f73e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/2755433777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ae0 0x7f73e806ee80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b7170 tx=0x7f73d802c6f0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0b4070 con 0x7fe71c0051c0 2024-04-26T21:12:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c0a7b30 con 0x7fe71c0051c0 2024-04-26T21:12:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f51cf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51c80075e0 0x7f51c803c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58458/0 (socket says 172.21.15.40:58458) 2024-04-26T21:12:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f51cf7fe640 1 -- 172.21.15.40:0/3778284614 learned_addr learned my addr 172.21.15.40:0/3778284614 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:10.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.767+0000 7feae17a4640 1 -- start start 2024-04-26T21:12:10.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7feae17a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fead404b3f0 0x7fead4034050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 shutdown_connections 2024-04-26T21:12:10.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/2755433777 >> v2:172.21.15.40:3300/0 conn(0x7f73e8073ec0 0x7f73e80b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/2755433777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ae0 0x7f73e806ee80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 >> 172.21.15.40:0/2755433777 conn(0x7f73e80b2c50 msgr2=0x7f73e80b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f2970f46640 1 Processor -- start 2024-04-26T21:12:10.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 -- 172.21.15.40:0/2522570651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d2080 msgr2=0x7f29040b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 --2- 172.21.15.40:0/2522570651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d2080 0x7f29040b64a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097a0 tx=0x7f28f402a140 comp rx=0 tx=0).stop 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 -- 172.21.15.40:0/2522570651 shutdown_connections 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 --2- 172.21.15.40:0/2522570651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d2080 0x7f29040b64a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 -- 172.21.15.40:0/2522570651 >> 172.21.15.40:0/2522570651 conn(0x7f29040d5030 msgr2=0x7f29040d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7feadaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fead404b3f0 0x7fead4034050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f2970f46640 1 -- start start 2024-04-26T21:12:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7feadaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fead404b3f0 0x7fead4034050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40284/0 (socket says 172.21.15.40:40284) 2024-04-26T21:12:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 -- 172.21.15.40:0/2522570651 shutdown_connections 2024-04-26T21:12:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f290a7e7640 1 -- 172.21.15.40:0/2522570651 wait complete. 2024-04-26T21:12:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7feadaffd640 1 -- 172.21.15.40:0/1914259539 learned_addr learned my addr 172.21.15.40:0/1914259539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 shutdown_connections 2024-04-26T21:12:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742405943 0 0) 0x7f51c8057a60 con 0x7f51c80075e0 2024-04-26T21:12:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51b8003520 con 0x7f51c80075e0 2024-04-26T21:12:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7fe72affd640 1 -- 172.21.15.40:0/1871109561 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c050c30 con 0x7fe71c0051c0 2024-04-26T21:12:10.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7f73ecb5f640 1 -- 172.21.15.40:0/2755433777 wait complete. 2024-04-26T21:12:10.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.769+0000 7feae17a4640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7fead4006ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 460928706 0 0) 0x7f51b8003520 con 0x7f51c80075e0 2024-04-26T21:12:10.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c80061f0 msgr2=0x7f51c8056fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c80061f0 0x7f51c8056fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 -- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053e70 msgr2=0x7ffa980542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7fe7316c0640 1 -- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0051c0 msgr2=0x7fe71c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7fe7316c0640 1 --2- 172.21.15.40:0/1871109561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0051c0 0x7fe71c00b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe72c06b0c0 tx=0x7fe71c005830 comp rx=0 tx=0).stop 2024-04-26T21:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7fe730ebf640 1 -- 172.21.15.40:0/1871109561 reap_dead start 2024-04-26T21:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f2e8e7fc640 1 -- 172.21.15.40:0/1164415883 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84040070 con 0x7f2e88050500 2024-04-26T21:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c80061f0 0x7f51c8056fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7fe7316c0640 1 -- 172.21.15.40:0/1871109561 shutdown_connections 2024-04-26T21:12:10.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7fe7316c0640 1 -- 172.21.15.40:0/1871109561 >> 172.21.15.40:0/1871109561 conn(0x7fe72401ef40 msgr2=0x7fe724055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f2e8e7fc640 1 -- 172.21.15.40:0/1164415883 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84033b20 con 0x7f2e88050500 2024-04-26T21:12:10.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7feadb7fe640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7fead4006ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f2e8e7fc640 1 -- 172.21.15.40:0/1164415883 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8403b440 con 0x7f2e88050500 2024-04-26T21:12:10.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead4054c40 con 0x7fead404f500 2024-04-26T21:12:10.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fead4006310 con 0x7fead404b3f0 2024-04-26T21:12:10.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 >> v1:172.21.15.184:6789/0 conn(0x7f51c80574e0 legacy=0x7f51c803c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8058300 con 0x7f51c80075e0 2024-04-26T21:12:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f51d006b630 con 0x7f51c80075e0 2024-04-26T21:12:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 >> v1:172.21.15.40:6789/0 conn(0x7f51c80075e0 legacy=0x7f51c803c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053e70 0x7ffa980542e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0051470 tx=0x7ffaa00a8960 comp rx=0 tx=0).stop 2024-04-26T21:12:10.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8004800 0x7f51b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51b80051c0 0x7f51b80141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8058020 con 0x7f51b8004800 2024-04-26T21:12:10.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 -- 172.21.15.40:0/2115700430 shutdown_connections 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053e70 0x7ffa980542e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 --2- 172.21.15.40:0/2115700430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98053a90 0x7ffa980502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 -- 172.21.15.40:0/2115700430 >> 172.21.15.40:0/2115700430 conn(0x7ffa98055c50 msgr2=0x7ffa98056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8014880 con 0x7f51b80039e0 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51b8014a80 con 0x7f51b80051c0 2024-04-26T21:12:10.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f2970f46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f296c163880 0x7f296c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c80061f0 0x7f51c8056fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51ce7fc640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8004800 0x7f51b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7ffa9f59e640 1 -- 172.21.15.40:0/2115700430 shutdown_connections 2024-04-26T21:12:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7f2970f46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f296c163c60 0x7f296c167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7ffa9f59e640 1 -- 172.21.15.40:0/2115700430 wait complete. 2024-04-26T21:12:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.770+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fead4053a80 con 0x7fead4006a80 2024-04-26T21:12:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2970f46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c0b4590 0x7f296c167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2970f46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f296c1684f0 con 0x7f296c163c60 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155886893 0 0) 0x7fead4054c40 con 0x7fead404f500 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac4003520 con 0x7fead404f500 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2970f46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f296c168000 con 0x7f296c163880 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2970f46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0d4e60 con 0x7f296c0b4590 2024-04-26T21:12:10.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f296c163880 0x7f296c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2e95b38640 1 -- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e88050500 msgr2=0x7f2e88007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f290a7e7640 1 Processor -- start 2024-04-26T21:12:10.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f296c163880 0x7f296c0b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48344/0 (socket says 172.21.15.40:48344) 2024-04-26T21:12:10.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 -- 172.21.15.40:0/2036828299 learned_addr learned my addr 172.21.15.40:0/2036828299 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2969d9b640 1 --2- 172.21.15.40:0/2036828299 >> v2:172.21.15.40:3300/0 conn(0x7f296c163c60 0x7f296c167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296ad9d640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c0b4590 0x7f296c167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51b80051c0 0x7f51b80141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2092216071 0 0) 0x7feac4003520 con 0x7fead404f500 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 msgr2=0x7fead4006ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7fead4006ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 >> v2:172.21.15.40:3300/0 conn(0x7fead404b3f0 msgr2=0x7fead4034050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 --2- 172.21.15.40:0/1914259539 >> v2:172.21.15.40:3300/0 conn(0x7fead404b3f0 0x7fead4034050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead4053d60 con 0x7fead404f500 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fead0001670 con 0x7fead404f500 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 >> v1:172.21.15.184:6789/0 conn(0x7fead404f500 legacy=0x7fead404f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2e95b38640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e88050500 0x7f2e88007bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400b850 tx=0x7f2e84032e10 comp rx=0 tx=0).stop 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7feac4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac4004d40 0x7feac40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fead4006310 con 0x7feac40039b0 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fead4053a80 con 0x7feac4004d40 2024-04-26T21:12:10.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feac400b9f0 con 0x7fead4006a80 2024-04-26T21:12:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51ceffd640 1 -- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51b80051c0 msgr2=0x7f51b80141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2e95b38640 1 -- 172.21.15.40:0/1164415883 shutdown_connections 2024-04-26T21:12:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2e95b38640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e880e0120 0x7f2e88008110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7feadaffd640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7feada7fc640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7feac4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f2e95b38640 1 --2- 172.21.15.40:0/1164415883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e88050500 0x7f2e88007bd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f2e95b38640 1 -- 172.21.15.40:0/1164415883 >> 172.21.15.40:0/1164415883 conn(0x7f2e88055bc0 msgr2=0x7f2e88055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7feadb7fe640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac4004d40 0x7feac40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f73ecb5f640 1 Processor -- start 2024-04-26T21:12:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7fe7316c0640 1 -- 172.21.15.40:0/1871109561 shutdown_connections 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 -- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c0b4590 msgr2=0x7f296c167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c0b4590 0x7f296c167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 -- 172.21.15.40:0/2036828299 >> v2:172.21.15.40:3300/0 conn(0x7f296c163c60 msgr2=0x7f296c167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 --2- 172.21.15.40:0/2036828299 >> v2:172.21.15.40:3300/0 conn(0x7f296c163c60 0x7f296c167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51b80051c0 0x7f51b80141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51ceffd640 1 -- 172.21.15.40:0/3778284614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8004800 msgr2=0x7f51b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f296a59c640 1 -- 172.21.15.40:0/2036828299 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c168250 con 0x7f296c163880 2024-04-26T21:12:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51b8004800 0x7f51b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.771+0000 7f51ceffd640 1 -- 172.21.15.40:0/3778284614 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8058300 con 0x7f51b80039e0 2024-04-26T21:12:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f73ecb5f640 1 -- start start 2024-04-26T21:12:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f290a7e7640 1 -- start start 2024-04-26T21:12:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51b80051c0 0x7f51b80141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f2e95b38640 1 -- 172.21.15.40:0/1164415883 shutdown_connections 2024-04-26T21:12:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7f2e95b38640 1 -- 172.21.15.40:0/1164415883 wait complete. 2024-04-26T21:12:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7ffa9f59e640 1 Processor -- start 2024-04-26T21:12:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7ffa9f59e640 1 -- start start 2024-04-26T21:12:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7fe7316c0640 1 -- 172.21.15.40:0/1871109561 wait complete. 2024-04-26T21:12:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7feadb7fe640 1 -- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 msgr2=0x7feac4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7feadb7fe640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fead4006a80 0x7feac4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7feadb7fe640 1 -- 172.21.15.40:0/1914259539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40039b0 msgr2=0x7feac4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7feadb7fe640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40039b0 0x7feac4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.772+0000 7feadb7fe640 1 -- 172.21.15.40:0/1914259539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead4053d60 con 0x7feac4004d40 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7feadb7fe640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac4004d40 0x7feac40051c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feadc06b9c0 tx=0x7feac4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30a2e34f69a45c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f51ceffd640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51b8003e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009930 tx=0x7f51b8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d361e5303627a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0b5070 con 0x7feac4004d40 2024-04-26T21:12:10.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc0b0410 con 0x7feac4004d40 2024-04-26T21:12:10.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7fead9ffb640 1 -- 172.21.15.40:0/1914259539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc050c30 con 0x7feac4004d40 2024-04-26T21:12:10.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f290a7e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 0x7f290415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4037070 con 0x7f51b80039e0 2024-04-26T21:12:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f290a7e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29040d2080 0x7f29041f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f296a59c640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b3fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f296c08d3c0 tx=0x7f296402a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def1168c503ddf49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f290a7e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d5b50 0x7f29040caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f290a7e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29040c8060 con 0x7f29040d5b50 2024-04-26T21:12:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f290a7e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29040c87b0 con 0x7f29040d2080 2024-04-26T21:12:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f290a7e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f29040c81e0 con 0x7f29040b60c0 2024-04-26T21:12:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c402aa50 con 0x7f51b80039e0 2024-04-26T21:12:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e8073ae0 0x7f73e8148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ec0 0x7f73e81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73e80b58b0 0x7f73e80b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73e809eef0 con 0x7f73e80b58b0 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73e806e680 con 0x7f73e8073ec0 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.773+0000 7f73ecb5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73e8074350 con 0x7f73e8073ae0 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73e80b58b0 0x7f73e80b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73e80b58b0 0x7f73e80b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40320/0 (socket says 172.21.15.40:40320) 2024-04-26T21:12:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e6d9d640 1 -- 172.21.15.40:0/543228715 learned_addr learned my addr 172.21.15.40:0/543228715 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 0x7f290415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 0x7f290415ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56976/0 (socket says 172.21.15.40:56976) 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 -- 172.21.15.40:0/2535999111 learned_addr learned my addr 172.21.15.40:0/2535999111 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f29037fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29040d2080 0x7f29041f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7ffa9f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 0x7ffa980499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2908d84640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d5b50 0x7f29040caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7ffa9f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 0x7ffa98178b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e8073ae0 0x7f73e8148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e5d9b640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ec0 0x7f73e81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 -- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29040d2080 msgr2=0x7f29041f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29040d2080 0x7f29041f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 -- 172.21.15.40:0/2535999111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d5b50 msgr2=0x7f29040caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29040d5b50 0x7f29040caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f2903fff640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29041fa4d0 con 0x7f29040b60c0 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 -- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ec0 msgr2=0x7f73e81537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e8073ec0 0x7f73e81537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 -- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73e80b58b0 msgr2=0x7f73e80b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73e80b58b0 0x7f73e80b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f73e659c640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e8155fd0 con 0x7f73e8073ae0 2024-04-26T21:12:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7f51cdffb640 1 -- 172.21.15.40:0/3778284614 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400a4d0 con 0x7f51b80039e0 2024-04-26T21:12:10.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7ffa9dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 0x7ffa98178b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 0x7ffa98178b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40324/0 (socket says 172.21.15.40:40324) 2024-04-26T21:12:10.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7fe7316c0640 1 Processor -- start 2024-04-26T21:12:10.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.774+0000 7fe7316c0640 1 -- start start 2024-04-26T21:12:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 0x7ffa980499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2e95b38640 1 Processor -- start 2024-04-26T21:12:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 0x7ffa980499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48372/0 (socket says 172.21.15.40:48372) 2024-04-26T21:12:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 -- 172.21.15.40:0/1742692387 learned_addr learned my addr 172.21.15.40:0/1742692387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2903fff640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 0x7f290415ff00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28f800c3c0 tx=0x7f28f800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f51cffff640 1 -- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 msgr2=0x7f51b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f51cffff640 1 --2- 172.21.15.40:0/3778284614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51b8003e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009930 tx=0x7f51b8005830 comp rx=0 tx=0).stop 2024-04-26T21:12:10.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f51cf7fe640 1 -- 172.21.15.40:0/3778284614 reap_dead start 2024-04-26T21:12:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9f59e640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98054980 0x7ffa98179050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f296959a640 1 -- 172.21.15.40:0/2036828299 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964036070 con 0x7f296c163880 2024-04-26T21:12:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f296959a640 1 -- 172.21.15.40:0/2036828299 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2964031600 con 0x7f296c163880 2024-04-26T21:12:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f296959a640 1 -- 172.21.15.40:0/2036828299 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640319d0 con 0x7f296c163880 2024-04-26T21:12:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73e659c640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e8073ae0 0x7f73e8148760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b60b0 tx=0x7f73d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa980507e0 con 0x7ffa98053e70 2024-04-26T21:12:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa980512d0 con 0x7ffa98053a90 2024-04-26T21:12:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8017070 con 0x7f29040b60c0 2024-04-26T21:12:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2e95b38640 1 -- start start 2024-04-26T21:12:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa98038d80 con 0x7ffa98054980 2024-04-26T21:12:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f8005ce0 con 0x7f29040b60c0 2024-04-26T21:12:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80072d0 con 0x7f29040b60c0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98054980 0x7ffa98179050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29040c9100 con 0x7f29040b60c0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8040020 con 0x7f73e8073ae0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d8033d20 con 0x7f73e8073ae0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d803c670 con 0x7f73e8073ae0 2024-04-26T21:12:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e81562b0 con 0x7f73e8073ae0 2024-04-26T21:12:10.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29040c96c0 con 0x7f29040b60c0 2024-04-26T21:12:10.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e8155c20 con 0x7f73e8073ae0 2024-04-26T21:12:10.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 -- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98054980 msgr2=0x7ffa98179050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 0x7fe7240545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98054980 0x7ffa98179050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac4004d40 msgr2=0x7feac40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7feae17a4640 1 --2- 172.21.15.40:0/1914259539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac4004d40 0x7feac40051c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feadc06b9c0 tx=0x7feac4005830 comp rx=0 tx=0).stop 2024-04-26T21:12:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7feadb7fe640 1 -- 172.21.15.40:0/1914259539 reap_dead start 2024-04-26T21:12:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 -- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 msgr2=0x7ffa980499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7241809f0 0x7fe724180e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7f2970f46640 1 -- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 msgr2=0x7f296c0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7f2970f46640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b3fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f296c08d3c0 tx=0x7f296402a690 comp rx=0 tx=0).stop 2024-04-26T21:12:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 0x7ffa980499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.775+0000 7ffa9dd9b640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa980e0850 con 0x7ffa98053e70 2024-04-26T21:12:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 0x7fe7240545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 0x7fe7240545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48386/0 (socket says 172.21.15.40:48386) 2024-04-26T21:12:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe72bfff640 1 -- 172.21.15.40:0/885568409 learned_addr learned my addr 172.21.15.40:0/885568409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7240dcce0 0x7fe7240dd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7ffa9e59c640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa98053a90 0x7ffa980499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa98054980 0x7ffa98179050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7ffa9dd9b640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 0x7ffa98178b10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b780 tx=0x7ffa9000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7241809f0 0x7fe724180e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240367b0 con 0x7fe7240dcce0 2024-04-26T21:12:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe730ebf640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7240dcce0 0x7fe7240dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28e4001030 con 0x7f29040b60c0 2024-04-26T21:12:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7f2e95b38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88050500 0x7f2e880cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240364d0 con 0x7fe71c0051c0 2024-04-26T21:12:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe7240540a0 con 0x7fe7241809f0 2024-04-26T21:12:10.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e95b38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e880dfd40 0x7f2e880c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.776+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90019070 con 0x7ffa98053e70 2024-04-26T21:12:10.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9000c740 con 0x7ffa98053e70 2024-04-26T21:12:10.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e95b38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 0x7f2e880e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000cb30 con 0x7ffa98053e70 2024-04-26T21:12:10.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e95b38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88039e00 con 0x7f2e880e0120 2024-04-26T21:12:10.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e95b38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88055550 con 0x7f2e880dfd40 2024-04-26T21:12:10.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e95b38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e88039f80 con 0x7f2e88050500 2024-04-26T21:12:10.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa980e0290 con 0x7ffa98053e70 2024-04-26T21:12:10.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 shutdown_connections 2024-04-26T21:12:10.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 >> 172.21.15.40:0/1914259539 conn(0x7fead400a430 msgr2=0x7fead400a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f51cffff640 1 -- 172.21.15.40:0/3778284614 shutdown_connections 2024-04-26T21:12:10.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f51cffff640 1 -- 172.21.15.40:0/3778284614 >> 172.21.15.40:0/3778284614 conn(0x7f51c8058ed0 msgr2=0x7f51c80592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 -- 172.21.15.40:0/2036828299 shutdown_connections 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa980e2960 con 0x7ffa98053e70 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c0b4590 0x7f296c167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 --2- 172.21.15.40:0/2036828299 >> v2:172.21.15.40:3300/0 conn(0x7f296c163c60 0x7f296c167580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 --2- 172.21.15.40:0/2036828299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b3fe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 -- 172.21.15.40:0/2036828299 >> 172.21.15.40:0/2036828299 conn(0x7f296c0d5030 msgr2=0x7f296c0d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 0x7f2e880e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 -- 172.21.15.40:0/2036828299 shutdown_connections 2024-04-26T21:12:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 0x7f2e880e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40332/0 (socket says 172.21.15.40:40332) 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 -- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7241809f0 msgr2=0x7fe724180e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7241809f0 0x7fe724180e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 -- 172.21.15.40:0/885568409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7240dcce0 msgr2=0x7fe7240dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7240dcce0 0x7fe7240dd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7240dd6a0 con 0x7fe71c0051c0 2024-04-26T21:12:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 -- 172.21.15.40:0/4100828732 learned_addr learned my addr 172.21.15.40:0/4100828732 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73e809f070 con 0x7f73e8073ae0 2024-04-26T21:12:10.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2970f46640 1 -- 172.21.15.40:0/2036828299 wait complete. 2024-04-26T21:12:10.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe72bfff640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 0x7fe7240545a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e5f0 tx=0x7fe72000eab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 -- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88050500 msgr2=0x7f2e880cc980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e88050500 0x7f2e880cc980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 -- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e880dfd40 msgr2=0x7f2e880c8c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e880dfd40 0x7f2e880c8c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880c91a0 con 0x7f2e880e0120 2024-04-26T21:12:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7f2e8ffff640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 0x7f2e880e1b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e90063ef0 tx=0x7f2e90069eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e900749a0 con 0x7f2e880e0120 2024-04-26T21:12:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.779+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa980e2d40 con 0x7ffa98053e70 2024-04-26T21:12:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28f801a070 con 0x7f29040b60c0 2024-04-26T21:12:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2900ff9640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28f006c9c0 0x7f28f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.777+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720019020 con 0x7fe71c0051c0 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720015690 con 0x7fe71c0051c0 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241814b0 con 0x7fe71c0051c0 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe730ebf640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7240dcce0 0x7fe7240dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7241809f0 0x7fe724180e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7240e90d0 con 0x7fe71c0051c0 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720015a80 con 0x7fe71c0051c0 2024-04-26T21:12:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.779+0000 7f2970f46640 1 Processor -- start 2024-04-26T21:12:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.779+0000 7f2970f46640 1 -- start start 2024-04-26T21:12:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 shutdown_connections 2024-04-26T21:12:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e90070430 con 0x7f2e880e0120 2024-04-26T21:12:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2e95b38640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e880c9630 con 0x7f2e880e0120 2024-04-26T21:12:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7f2e95b38640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e880c9bf0 con 0x7f2e880e0120 2024-04-26T21:12:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.778+0000 7feae17a4640 1 -- 172.21.15.40:0/1914259539 wait complete. 2024-04-26T21:12:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e880ccec0 con 0x7f2e880e0120 2024-04-26T21:12:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7feae17a4640 1 Processor -- start 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe72001c020 con 0x7fe71c0051c0 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e900708c0 con 0x7f2e880e0120 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa9001c070 con 0x7ffa98053e70 2024-04-26T21:12:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2970f46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f296c0b4590 0x7f296c0b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7ffa8effd640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa8006c9c0 0x7ffa8006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffa9008b020 con 0x7ffa98053e70 2024-04-26T21:12:10.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe70c001030 con 0x7fe71c0051c0 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f28f808ad00 con 0x7f29040b60c0 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2900ff9640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f28f0074c40 0x7f28f0077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28f0077710 con 0x7f28f0074c40 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f28f808d360 con 0x7f29040b60c0 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f29037fe640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28f006c9c0 0x7f28f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f2908d84640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f28f0074c40 0x7f28f0077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73d8043020 con 0x7f73e8073ae0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f73d6ffd640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73c806c9c0 0x7f73c806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f73d80ae530 con 0x7f73e8073ae0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73d6ffd640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f73c8074b60 0x7f73c8076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73d8030e50 con 0x7f73c8074b60 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73d80b5050 con 0x7f73e8073ae0 2024-04-26T21:12:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7fe728ff9640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71006c9f0 0x7fe71006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f2970f46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f296a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f296c0b4590 0x7f296c0b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e90070b20 con 0x7f2e880e0120 2024-04-26T21:12:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73e6d9d640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f73c8074b60 0x7f73c8076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe72008c310 con 0x7fe71c0051c0 2024-04-26T21:12:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7feae17a4640 1 -- start start 2024-04-26T21:12:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feae17a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 0x7fead40db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73e5d9b640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73c806c9c0 0x7f73c806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7ffa8effd640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7ffa80074c40 0x7ffa80077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa80077710 con 0x7ffa80074c40 2024-04-26T21:12:10.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feae17a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fead4043b00 0x7fead4047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa90092050 con 0x7ffa98053e70 2024-04-26T21:12:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7ffa80074c40 0x7ffa80077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7fe728ff9640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7fe710074b90 0x7fe710076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feadaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 0x7fead40db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feadaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 0x7fead40db5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57018/0 (socket says 172.21.15.40:57018) 2024-04-26T21:12:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71006c9f0 0x7fe71006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feadaffd640 1 -- 172.21.15.40:0/3336227400 learned_addr learned my addr 172.21.15.40:0/3336227400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f73e6d9d640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f73c8074b60 0x7f73c8076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7feae17a4640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead4043f80 0x7fead40480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7feada7fc640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fead4043b00 0x7fead4047bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2908d84640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f28f0074c40 0x7f28f0077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe720017e50 con 0x7fe710074b90 2024-04-26T21:12:10.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7ffa9ed9d640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7ffa80074c40 0x7ffa80077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe720093050 con 0x7fe71c0051c0 2024-04-26T21:12:10.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f296a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f296c0b4590 0x7f296c0b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40336/0 (socket says 172.21.15.40:40336) 2024-04-26T21:12:10.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f2969d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2969d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48394/0 (socket says 172.21.15.40:48394) 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f296a59c640 1 -- 172.21.15.40:0/3458835589 learned_addr learned my addr 172.21.15.40:0/3458835589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.781+0000 7f2e7bfff640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e7406c9f0 0x7f2e7406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2e90108240 con 0x7f2e880e0120 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e7bfff640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7f2e74074c10 0x7f2e74077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e90098eb0 con 0x7f2e74074c10 2024-04-26T21:12:10.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2e9010f050 con 0x7f2e880e0120 2024-04-26T21:12:10.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e8f7fe640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e7406c9f0 0x7f2e7406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2970f46640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c163c60 0x7f296c0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f2e8effd640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7f2e74074c10 0x7f2e74077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7fe730ebf640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7fe710074b90 0x7fe710076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7ffa9e59c640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa8006c9c0 0x7ffa8006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f29037fe640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28f006c9c0 0x7f28f006ee80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28f4034000 tx=0x7f28f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.780+0000 7f51cffff640 1 -- 172.21.15.40:0/3778284614 shutdown_connections 2024-04-26T21:12:10.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7f51cffff640 1 -- 172.21.15.40:0/3778284614 wait complete. 2024-04-26T21:12:10.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7ffa9e59c640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa8006c9c0 0x7ffa8006ee80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa94008a10 tx=0x7ffa94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71006c9f0 0x7fe71006eeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe718001eb0 tx=0x7fe718009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2970f46640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0b8230 con 0x7f296c0b4590 2024-04-26T21:12:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2970f46640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0b8b30 con 0x7f296c163880 2024-04-26T21:12:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2970f46640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0b8dd0 con 0x7f296c163c60 2024-04-26T21:12:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== osd.2 v2:172.21.15.40:6824/3577609501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73d8030e50 con 0x7f73c8074b60 2024-04-26T21:12:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== osd.4 v2:172.21.15.184:6816/1784643509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28f0077710 con 0x7f28f0074c40 2024-04-26T21:12:10.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7fe730ebf640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7fe710074b90 0x7fe710076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f296ad9d640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c163c60 0x7f296c0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== osd.3 v2:172.21.15.40:6808/592958762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa80077710 con 0x7ffa80074c40 2024-04-26T21:12:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.782+0000 7feae17a4640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fead4050090 con 0x7fead4043f80 2024-04-26T21:12:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feae17a4640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fead404b7d0 con 0x7fead4043b00 2024-04-26T21:12:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2e8f7fe640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e7406c9f0 0x7f2e7406eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0014f0 tx=0x7f2e7c007990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2e8effd640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7f2e74074c10 0x7f2e74077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feae17a4640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fead4054860 con 0x7feac4004d40 2024-04-26T21:12:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2969d9b640 1 -- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c163c60 msgr2=0x7f296c0c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2969d9b640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f296c163c60 0x7f296c0c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2969d9b640 1 -- 172.21.15.40:0/3458835589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f296c0b4590 msgr2=0x7f296c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2969d9b640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f296c0b4590 0x7f296c0b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f2969d9b640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0c8eb0 con 0x7f296c163880 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadb7fe640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead4043f80 0x7fead40480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadaffd640 1 -- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fead4043b00 msgr2=0x7fead4047bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadaffd640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fead4043b00 0x7fead4047bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadaffd640 1 -- 172.21.15.40:0/3336227400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead4043f80 msgr2=0x7fead40480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadaffd640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead4043f80 0x7fead40480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7feadaffd640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead40dbaf0 con 0x7feac4004d40 2024-04-26T21:12:10.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== osd.5 v2:172.21.15.184:6800/2962645716 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe720017e50 con 0x7fe710074b90 2024-04-26T21:12:10.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7f2969d9b640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b4fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d520 tx=0x7f295c00d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7feadaffd640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 0x7fead40db5b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feacc011960 tx=0x7feacc011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.783+0000 7f73e5d9b640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73c806c9c0 0x7f73c806ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e80ce110 tx=0x7f73dc010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== osd.0 v2:172.21.15.40:6816/2954344348 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e90098eb0 con 0x7f2e74074c10 2024-04-26T21:12:10.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c0125f0 con 0x7f296c163880 2024-04-26T21:12:10.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f295c019070 con 0x7f296c163880 2024-04-26T21:12:10.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.784+0000 7f51cffff640 1 Processor -- start 2024-04-26T21:12:10.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00faf0 con 0x7feac4004d40 2024-04-26T21:12:10.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc00fc90 con 0x7feac4004d40 2024-04-26T21:12:10.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc014440 con 0x7feac4004d40 2024-04-26T21:12:10.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7feae17a4640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead404ee10 con 0x7feac4004d40 2024-04-26T21:12:10.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7feae17a4640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fead40473b0 con 0x7feac4004d40 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c0129d0 con 0x7f296c163880 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7f2970f46640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c0c9190 con 0x7f296c163880 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.785+0000 7f2970f46640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c0c9750 con 0x7f296c163880 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.786+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f296c09e5f0 con 0x7f296c163880 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.787+0000 7f51cffff640 1 -- start start 2024-04-26T21:12:10.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.787+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fead4005810 con 0x7feac4004d40 2024-04-26T21:12:10.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51cffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51c80db300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 0x7f51c80e0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51cffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c804c9c0 0x7f51c80e0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51cffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51c803edc0 con 0x7f51c80e3770 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51ce7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 0x7f51c80e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51ce7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 0x7f51c80e0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40354/0 (socket says 172.21.15.40:40354) 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51ce7fc640 1 -- 172.21.15.40:0/705503136 learned_addr learned my addr 172.21.15.40:0/705503136 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51ceffd640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51c80db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.788+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c804c9c0 0x7f51c80e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51cffff640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80e3510 con 0x7f51b80039e0 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51cffff640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51c8007a70 con 0x7f51c804c9c0 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51ce7fc640 1 -- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c804c9c0 msgr2=0x7f51c80e0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51ce7fc640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c804c9c0 0x7f51c80e0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51ce7fc640 1 -- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 msgr2=0x7f51c80db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51ce7fc640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b80039e0 0x7f51c80db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51ce7fc640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c804cf30 con 0x7f51c80e3770 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.789+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51c804c9c0 0x7f51c80e0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.790+0000 7f51ce7fc640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 0x7f51c80e0230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51c000f820 tx=0x7f51c000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.790+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0001ce0 con 0x7f51c80e3770 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.790+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c0018070 con 0x7f51c80e3770 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.790+0000 7f51cffff640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c804d210 con 0x7f51c80e3770 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.790+0000 7f51cffff640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c804d7d0 con 0x7f51c80e3770 2024-04-26T21:12:10.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.791+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000d730 con 0x7f51c80e3770 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.791+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feacc012040 con 0x7feac4004d40 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feac37fe640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac806c9f0 0x7feac806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feacc08c5f0 con 0x7feac4004d40 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feac37fe640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7feac8074c10 0x7feac8077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feacc023e80 con 0x7feac8074c10 2024-04-26T21:12:10.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feacc08c9e0 con 0x7feac4004d40 2024-04-26T21:12:10.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51c8038720 con 0x7f51c80e3770 2024-04-26T21:12:10.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feada7fc640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac806c9f0 0x7feac806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.792+0000 7feadb7fe640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7feac8074c10 0x7feac8077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.793+0000 7feadb7fe640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7feac8074c10 0x7feac8077030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.793+0000 7feada7fc640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac806c9f0 0x7feac806eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fead00017e0 tx=0x7fead0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.793+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f295c012b70 con 0x7f296c163880 2024-04-26T21:12:10.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.794+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== osd.7 v2:172.21.15.184:6804/2214404971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feacc023e80 con 0x7feac8074c10 2024-04-26T21:12:10.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.794+0000 7f295affd640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f294c06c9c0 0x7f294c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.794+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c000d8d0 con 0x7f51c80e3770 2024-04-26T21:12:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.794+0000 7f51b7fff640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51bc06cb20 0x7f51bc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.794+0000 7f296a59c640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f294c06c9c0 0x7f294c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f295c08c710 con 0x7f296c163880 2024-04-26T21:12:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f51c00052e0 con 0x7f51c80e3770 2024-04-26T21:12:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f296a59c640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f294c06c9c0 0x7f294c06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2964009850 tx=0x7f2964009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f295affd640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7f294c074c40 0x7f294c077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f295c019070 con 0x7f294c074c40 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f295c08cb00 con 0x7f296c163880 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f296ad9d640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7f294c074c40 0x7f294c077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f51b7fff640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f51bc075850 0x7f51bc077cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51bc0783c0 con 0x7f51bc075850 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.795+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f51bc075850 0x7f51bc077cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.796+0000 7f51ceffd640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51bc06cb20 0x7f51bc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:10.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.796+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51c0056770 con 0x7f51c80e3770 2024-04-26T21:12:10.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.796+0000 7f296ad9d640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7f294c074c40 0x7f294c077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.796+0000 7f51cf7fe640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f51bc075850 0x7f51bc077cf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.797+0000 7f51ceffd640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51bc06cb20 0x7f51bc06efe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51c8053720 tx=0x7f51c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:10.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.797+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== osd.6 v2:172.21.15.184:6824/3739826890 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f295c019070 con 0x7f294c074c40 2024-04-26T21:12:10.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.797+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== osd.1 v2:172.21.15.40:6800/2244954825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51bc0783c0 con 0x7f51bc075850 2024-04-26T21:12:10.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.817+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 --> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28e4002dd0 con 0x7f28f0074c40 2024-04-26T21:12:10.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.817+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 --> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e880057d0 con 0x7f2e74074c10 2024-04-26T21:12:10.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.817+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 --> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73e806fd50 con 0x7f73c8074b60 2024-04-26T21:12:10.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f2900ff9640 1 -- 172.21.15.40:0/2535999111 <== osd.4 v2:172.21.15.184:6816/1784643509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f28e4002dd0 con 0x7f28f0074c40 2024-04-26T21:12:10.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7ffa9f59e640 1 -- 172.21.15.40:0/1742692387 --> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa980057e0 con 0x7ffa80074c40 2024-04-26T21:12:10.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f2e7bfff640 1 -- 172.21.15.40:0/4100828732 <== osd.0 v2:172.21.15.40:6816/2954344348 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2e880057d0 con 0x7f2e74074c10 2024-04-26T21:12:10.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f73d6ffd640 1 -- 172.21.15.40:0/543228715 <== osd.2 v2:172.21.15.40:6824/3577609501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73e806fd50 con 0x7f73c8074b60 2024-04-26T21:12:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f28f0074c40 msgr2=0x7f28f0077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f290a7e7640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f28f0074c40 0x7f28f0077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 --> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe70c002dd0 con 0x7fe710074b90 2024-04-26T21:12:10.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7ffa8effd640 1 -- 172.21.15.40:0/1742692387 <== osd.3 v2:172.21.15.40:6808/592958762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa980057e0 con 0x7ffa80074c40 2024-04-26T21:12:10.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28f006c9c0 msgr2=0x7f28f006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f290a7e7640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28f006c9c0 0x7f28f006ee80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28f4034000 tx=0x7f28f402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:10.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.818+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f73c8074b60 msgr2=0x7f73c8076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f73c8074b60 0x7f73c8076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73c806c9c0 msgr2=0x7f73c806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73c806c9c0 0x7f73c806ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e80ce110 tx=0x7f73dc010040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e8073ae0 msgr2=0x7f73e8148760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 --2- 172.21.15.40:0/543228715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e8073ae0 0x7f73e8148760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b60b0 tx=0x7f73d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73e6d9d640 1 -- 172.21.15.40:0/543228715 reap_dead start 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 shutdown_connections 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 >> 172.21.15.40:0/543228715 conn(0x7f73e80b2c50 msgr2=0x7f73e80cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 shutdown_connections 2024-04-26T21:12:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f73ecb5f640 1 -- 172.21.15.40:0/543228715 wait complete. 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7ffa80074c40 msgr2=0x7ffa80077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7ffa80074c40 0x7ffa80077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 msgr2=0x7f290415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 --2- 172.21.15.40:0/2535999111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29040b60c0 0x7f290415ff00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28f800c3c0 tx=0x7f28f800c880 comp rx=0 tx=0).stop 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2908d84640 1 -- 172.21.15.40:0/2535999111 reap_dead start 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 shutdown_connections 2024-04-26T21:12:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 >> 172.21.15.40:0/2535999111 conn(0x7f29040d5030 msgr2=0x7f29040b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7fe728ff9640 1 -- 172.21.15.40:0/885568409 <== osd.5 v2:172.21.15.184:6800/2962645716 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe70c002dd0 con 0x7fe710074b90 2024-04-26T21:12:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7f2e74074c10 msgr2=0x7f2e74077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7f2e74074c10 0x7f2e74077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 shutdown_connections 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e7406c9f0 msgr2=0x7f2e7406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e7406c9f0 0x7f2e7406eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0014f0 tx=0x7f2e7c007990 comp rx=0 tx=0).stop 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 msgr2=0x7f2e880e1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 --2- 172.21.15.40:0/4100828732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e880e0120 0x7f2e880e1b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e90063ef0 tx=0x7f2e90069eb0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e8ffff640 1 -- 172.21.15.40:0/4100828732 reap_dead start 2024-04-26T21:12:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa8006c9c0 msgr2=0x7ffa8006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa8006c9c0 0x7ffa8006ee80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa94008a10 tx=0x7ffa94008040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 msgr2=0x7ffa98178b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa8cff9640 1 --2- 172.21.15.40:0/1742692387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa98053e70 0x7ffa98178b10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b780 tx=0x7ffa9000bc40 comp rx=0 tx=0).stop 2024-04-26T21:12:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7ffa9ed9d640 1 -- 172.21.15.40:0/1742692387 reap_dead start 2024-04-26T21:12:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f290a7e7640 1 -- 172.21.15.40:0/2535999111 wait complete. 2024-04-26T21:12:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 shutdown_connections 2024-04-26T21:12:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 >> 172.21.15.40:0/4100828732 conn(0x7f2e88055bc0 msgr2=0x7f2e8803a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 shutdown_connections 2024-04-26T21:12:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7f2e79ffb640 1 -- 172.21.15.40:0/4100828732 wait complete. 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7fe710074b90 msgr2=0x7fe710076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.819+0000 7fe7316c0640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7fe710074b90 0x7fe710076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71006c9f0 msgr2=0x7fe71006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71006c9f0 0x7fe71006eeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe718001eb0 tx=0x7fe718009040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 msgr2=0x7fe7240545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 --2- 172.21.15.40:0/885568409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c0051c0 0x7fe7240545a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e5f0 tx=0x7fe72000eab0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe730ebf640 1 -- 172.21.15.40:0/885568409 reap_dead start 2024-04-26T21:12:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 shutdown_connections 2024-04-26T21:12:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 >> 172.21.15.40:0/885568409 conn(0x7fe72401ef40 msgr2=0x7fe7240087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 shutdown_connections 2024-04-26T21:12:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 >> 172.21.15.40:0/1742692387 conn(0x7ffa98055c50 msgr2=0x7ffa980da6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 shutdown_connections 2024-04-26T21:12:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7fe7316c0640 1 -- 172.21.15.40:0/885568409 wait complete. 2024-04-26T21:12:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 shutdown_connections 2024-04-26T21:12:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.820+0000 7ffa8cff9640 1 -- 172.21.15.40:0/1742692387 wait complete. 2024-04-26T21:12:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.825+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 --> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fead4005810 con 0x7feac8074c10 2024-04-26T21:12:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.826+0000 7feac37fe640 1 -- 172.21.15.40:0/3336227400 <== osd.7 v2:172.21.15.184:6804/2214404971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fead4005810 con 0x7feac8074c10 2024-04-26T21:12:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.827+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7feac8074c10 msgr2=0x7feac8077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.827+0000 7feac17fa640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7feac8074c10 0x7feac8077030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.827+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac806c9f0 msgr2=0x7feac806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.827+0000 7feac17fa640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac806c9f0 0x7feac806eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fead00017e0 tx=0x7fead0009040 comp rx=0 tx=0).stop 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 msgr2=0x7fead40db5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 --2- 172.21.15.40:0/3336227400 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feac4004d40 0x7fead40db5b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feacc011960 tx=0x7feacc011e20 comp rx=0 tx=0).stop 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feadb7fe640 1 -- 172.21.15.40:0/3336227400 reap_dead start 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 shutdown_connections 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 >> 172.21.15.40:0/3336227400 conn(0x7fead400a430 msgr2=0x7fead40e8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 shutdown_connections 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.828+0000 7feac17fa640 1 -- 172.21.15.40:0/3336227400 wait complete. 2024-04-26T21:12:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.832+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 --> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f296c09e5f0 con 0x7f294c074c40 2024-04-26T21:12:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.832+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 --> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51c80057d0 con 0x7f51bc075850 2024-04-26T21:12:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.832+0000 7f295affd640 1 -- 172.21.15.40:0/3458835589 <== osd.6 v2:172.21.15.184:6824/3739826890 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f296c09e5f0 con 0x7f294c074c40 2024-04-26T21:12:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.833+0000 7f51b7fff640 1 -- 172.21.15.40:0/705503136 <== osd.1 v2:172.21.15.40:6800/2244954825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51c80057d0 con 0x7f51bc075850 2024-04-26T21:12:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.833+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7f294c074c40 msgr2=0x7f294c077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.833+0000 7f2958ff9640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7f294c074c40 0x7f294c077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f51bc075850 msgr2=0x7f51bc077cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f51bc075850 0x7f51bc077cf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f294c06c9c0 msgr2=0x7f294c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f294c06c9c0 0x7f294c06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2964009850 tx=0x7f2964009000 comp rx=0 tx=0).stop 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 msgr2=0x7f296c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 --2- 172.21.15.40:0/3458835589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f296c163880 0x7f296c0b4fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f295c00d520 tx=0x7f295c00d9e0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f296ad9d640 1 -- 172.21.15.40:0/3458835589 reap_dead start 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 shutdown_connections 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 >> 172.21.15.40:0/3458835589 conn(0x7f296c0d5030 msgr2=0x7f296c1672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51bc06cb20 msgr2=0x7f51bc06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51bc06cb20 0x7f51bc06efe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51c8053720 tx=0x7f51c402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 msgr2=0x7f51c80e0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 --2- 172.21.15.40:0/705503136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80e3770 0x7f51c80e0230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51c000f820 tx=0x7f51c000fce0 comp rx=0 tx=0).stop 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51cf7fe640 1 -- 172.21.15.40:0/705503136 reap_dead start 2024-04-26T21:12:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 shutdown_connections 2024-04-26T21:12:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 >> 172.21.15.40:0/705503136 conn(0x7f51c8058ed0 msgr2=0x7f51c80527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 shutdown_connections 2024-04-26T21:12:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.834+0000 7f2958ff9640 1 -- 172.21.15.40:0/3458835589 wait complete. 2024-04-26T21:12:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.835+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 shutdown_connections 2024-04-26T21:12:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:10.836+0000 7f51b5ffb640 1 -- 172.21.15.40:0/705503136 wait complete. 2024-04-26T21:12:10.907 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:10.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:12:10.909 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:10.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:12:10.910 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:10.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:12:10.911 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:10.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:12:10.912 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:10.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:12:10.913 INFO:teuthology.orchestra.run.smithi040.stdout:64424509448 2024-04-26T21:12:10.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:12:10.914 INFO:teuthology.orchestra.run.smithi040.stdout:64424509448 2024-04-26T21:12:10.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:12:10.915 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2024-04-26T21:12:10.916 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:12:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.119+0000 7f86b8845640 1 Processor -- start 2024-04-26T21:12:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.119+0000 7f86b8845640 1 -- start start 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.120+0000 7f86b8845640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86b40d0010 0x7f86b415c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b8845640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b40704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b8845640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b415c740 con 0x7f86b4070040 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b8845640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40709f0 con 0x7f86b40d0010 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b8845640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40732e0 con 0x7f86b40b4ef0 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b1d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86b4070040 0x7f86b415edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58474/0 (socket says 172.21.15.40:58474) 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b1d9b640 1 -- 172.21.15.40:0/1213482590 learned_addr learned my addr 172.21.15.40:0/1213482590 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b259c640 1 --2- 172.21.15.40:0/1213482590 >> v2:172.21.15.184:3300/0 conn(0x7f86b40d0010 0x7f86b415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.121+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b40704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f3db772b640 1 Processor -- start 2024-04-26T21:12:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f3db772b640 1 -- start start 2024-04-26T21:12:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f733c831640 1 Processor -- start 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f86b259c640 1 -- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 msgr2=0x7f86b40704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f86b259c640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b40704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f86b259c640 1 -- 172.21.15.40:0/1213482590 >> v1:172.21.15.40:6789/0 conn(0x7f86b4070040 legacy=0x7f86b415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.122+0000 7f86b259c640 1 -- 172.21.15.40:0/1213482590 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40d03f0 con 0x7f86b40d0010 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f3db772b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3da8037c20 0x7f3da80380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f3db772b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 0x7f3da804f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f3db772b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da8052630 con 0x7f3da8033f80 2024-04-26T21:12:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f3db772b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3da8053520 con 0x7f3da804f0b0 2024-04-26T21:12:11.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f3db772b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3da8052910 con 0x7f3da8037c20 2024-04-26T21:12:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7efd5a647640 1 Processor -- start 2024-04-26T21:12:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7efd5a647640 1 -- start start 2024-04-26T21:12:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f3db4cc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3da8037c20 0x7f3da80380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f733c831640 1 -- start start 2024-04-26T21:12:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f3db4cc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3da8037c20 0x7f3da80380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48438/0 (socket says 172.21.15.40:48438) 2024-04-26T21:12:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f3db4cc6640 1 -- 172.21.15.40:0/2371291813 learned_addr learned my addr 172.21.15.40:0/2371291813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7ff56c2f9640 1 Processor -- start 2024-04-26T21:12:11.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f86b259c640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b415c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00aea0 tx=0x7f86ac02b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61fd51d8f450ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f733c831640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7338148880 0x7f7338163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f3db4cc6640 1 -- 172.21.15.40:0/2371291813 >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 msgr2=0x7f3da804f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:12:11.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f86b159a640 1 -- 172.21.15.40:0/1213482590 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac037070 con 0x7f86b40d0010 2024-04-26T21:12:11.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7f86b159a640 1 -- 172.21.15.40:0/1213482590 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac008640 con 0x7f86b40d0010 2024-04-26T21:12:11.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f733c831640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7338148d70 con 0x7f733806c2a0 2024-04-26T21:12:11.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f733c831640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7338149050 con 0x7f73381641d0 2024-04-26T21:12:11.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f733c831640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7338149330 con 0x7f7338148880 2024-04-26T21:12:11.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.124+0000 7ff56c2f9640 1 -- start start 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd5a647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd540ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd5a647640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd540b4d70 0x7efd540aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd5a647640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd540b61e0 con 0x7efd5415b0e0 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd5a647640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efd5406e5d0 con 0x7efd540b4d70 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd5a647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd54072430 con 0x7efd540b5c90 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd53fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd5415b0e0 0x7efd5406dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58502/0 (socket says 172.21.15.40:58502) 2024-04-26T21:12:11.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd53fff640 1 -- 172.21.15.40:0/3668259135 learned_addr learned my addr 172.21.15.40:0/3668259135 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:11.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f3db5cc8640 1 --2- 172.21.15.40:0/2371291813 >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 0x7f3da804f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd58be4640 1 --2- 172.21.15.40:0/3668259135 >> v2:172.21.15.40:3300/0 conn(0x7efd540b4d70 0x7efd540aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f86b8845640 1 -- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 msgr2=0x7f86b415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7efd537fe640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd540ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f86b8845640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b415c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00aea0 tx=0x7f86ac02b640 comp rx=0 tx=0).stop 2024-04-26T21:12:11.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f7336ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7338148880 0x7f7338163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f7336ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7338148880 0x7f7338163c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48446/0 (socket says 172.21.15.40:48446) 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f7336ffd640 1 -- 172.21.15.40:0/4142504176 learned_addr learned my addr 172.21.15.40:0/4142504176 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7ff56c2f9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5640d1210 0x7ff5640b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7ff56c2f9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 0x7ff56406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7ff56c2f9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5640b6b90 con 0x7ff56406d2b0 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7ff56c2f9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff56406dc30 con 0x7ff5640b6530 2024-04-26T21:12:11.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7ff56c2f9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5640b4790 con 0x7ff5640d1210 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f86b8845640 1 -- 172.21.15.40:0/1213482590 shutdown_connections 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f86b8845640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b40704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f86b8845640 1 --2- 172.21.15.40:0/1213482590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b415c200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f86b8845640 1 -- 172.21.15.40:0/1213482590 >> 172.21.15.40:0/1213482590 conn(0x7f86b40b2ae0 msgr2=0x7f86b40b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7fe05ffff640 1 Processor -- start 2024-04-26T21:12:11.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7fe05ffff640 1 -- start start 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f7336ffd640 1 -- 172.21.15.40:0/4142504176 >> v1:172.21.15.184:6789/0 conn(0x7f73381641d0 legacy=0x7f7338146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f7336ffd640 1 -- 172.21.15.40:0/4142504176 >> v1:172.21.15.40:6789/0 conn(0x7f733806c2a0 legacy=0x7f733806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f7336ffd640 1 -- 172.21.15.40:0/4142504176 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338149610 con 0x7f7338148880 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd52ffd640 1 -- 172.21.15.40:0/3668259135 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619960430 0 0) 0x7efd540b61e0 con 0x7efd5415b0e0 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd52ffd640 1 -- 172.21.15.40:0/3668259135 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd3c003520 con 0x7efd5415b0e0 2024-04-26T21:12:11.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f3db4cc6640 1 --2- 172.21.15.40:0/2371291813 >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 0x7f3da804f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd58be4640 1 -- 172.21.15.40:0/3668259135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 msgr2=0x7efd540ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd58be4640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd540ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd58be4640 1 -- 172.21.15.40:0/3668259135 >> v1:172.21.15.40:6789/0 conn(0x7efd5415b0e0 legacy=0x7efd5406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7efd58be4640 1 -- 172.21.15.40:0/3668259135 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd54072710 con 0x7efd540b4d70 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f3db4cc6640 1 -- 172.21.15.40:0/2371291813 >> v1:172.21.15.40:6789/0 conn(0x7f3da8033f80 legacy=0x7f3da80539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f3db4cc6640 1 -- 172.21.15.40:0/2371291813 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da8052bf0 con 0x7f3da8037c20 2024-04-26T21:12:11.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff56a095640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5640d1210 0x7ff5640b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f86b8845640 1 -- 172.21.15.40:0/1213482590 shutdown_connections 2024-04-26T21:12:11.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f9a5a85b640 1 Processor -- start 2024-04-26T21:12:11.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.123+0000 7f9a5a85b640 1 -- start start 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff56a095640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5640d1210 0x7ff5640b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48460/0 (socket says 172.21.15.40:48460) 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a5a85b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a5a85b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c050220 con 0x7f9a4c0e7640 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a5a85b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c0485c0 con 0x7f9a4c04fbe0 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a5a85b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c053970 con 0x7f9a4c054b30 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a59058640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a59058640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57058/0 (socket says 172.21.15.40:57058) 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff56a095640 1 -- 172.21.15.40:0/2589147851 learned_addr learned my addr 172.21.15.40:0/2589147851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.125+0000 7f9a59058640 1 -- 172.21.15.40:0/1873323282 learned_addr learned my addr 172.21.15.40:0/1873323282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f9a59058640 1 -- 172.21.15.40:0/1873323282 >> v1:172.21.15.184:6789/0 conn(0x7f9a4c04fbe0 legacy=0x7f9a4c0063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f9a59058640 1 -- 172.21.15.40:0/1873323282 >> v1:172.21.15.40:6789/0 conn(0x7f9a4c0e7640 legacy=0x7f9a4c04f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.126+0000 7f9a59058640 1 -- 172.21.15.40:0/1873323282 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c053bc0 con 0x7f9a4c054b30 2024-04-26T21:12:11.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3ae30f8640 1 Processor -- start 2024-04-26T21:12:11.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f9a59058640 1 --2- 172.21.15.40:0/1873323282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a54063ef0 tx=0x7f9a540a6fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15512840b144d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3ae30f8640 1 -- start start 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3db5cc8640 1 --2- 172.21.15.40:0/2371291813 >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 0x7f3da804f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3db4cc6640 1 --2- 172.21.15.40:0/2371291813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da80380a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3da8008590 tx=0x7f3dac02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8e3a6194fd73a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3da7fff640 1 -- 172.21.15.40:0/2371291813 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac037070 con 0x7f3da8037c20 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3da7fff640 1 -- 172.21.15.40:0/2371291813 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac032640 con 0x7f3da8037c20 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f3da7fff640 1 -- 172.21.15.40:0/2371291813 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0329f0 con 0x7f3da8037c20 2024-04-26T21:12:11.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7fe05ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe06015f5e0 0x7fe0600d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7f86b8845640 1 -- 172.21.15.40:0/1213482590 wait complete. 2024-04-26T21:12:11.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff569093640 1 -- 172.21.15.40:0/2589147851 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135675095 0 0) 0x7ff5640b6b90 con 0x7ff56406d2b0 2024-04-26T21:12:11.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff569093640 1 -- 172.21.15.40:0/2589147851 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff550003520 con 0x7ff56406d2b0 2024-04-26T21:12:11.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.127+0000 7ff56a896640 1 --2- 172.21.15.40:0/2589147851 >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 0x7ff56406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05ffff640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0600d5c60 con 0x7fe0600d2150 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05ffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0600b1d30 con 0x7fe0600b3540 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05ffff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0600d31b0 con 0x7fe06015f5e0 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f9a58857640 1 -- 172.21.15.40:0/1873323282 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540b4070 con 0x7f9a4c054b30 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f9a58857640 1 -- 172.21.15.40:0/1873323282 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a540a7df0 con 0x7f9a4c054b30 2024-04-26T21:12:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f9a58857640 1 -- 172.21.15.40:0/1873323282 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540af640 con 0x7f9a4c054b30 2024-04-26T21:12:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05e7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe0600d2150 0x7fe0600cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58512/0 (socket says 172.21.15.40:58512) 2024-04-26T21:12:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05e7fc640 1 -- 172.21.15.40:0/3742025103 learned_addr learned my addr 172.21.15.40:0/3742025103 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f7336ffd640 1 --2- 172.21.15.40:0/4142504176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7338148880 0x7f7338163c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7330009e50 tx=0x7f733002a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba700742b81c2ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7ff56a095640 1 -- 172.21.15.40:0/2589147851 >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 msgr2=0x7ff56406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7ff56a095640 1 --2- 172.21.15.40:0/2589147851 >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 0x7ff56406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7ff56a095640 1 -- 172.21.15.40:0/2589147851 >> v1:172.21.15.40:6789/0 conn(0x7ff56406d2b0 legacy=0x7ff5640b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7fe05effd640 1 --2- 172.21.15.40:0/3742025103 >> v2:172.21.15.184:3300/0 conn(0x7fe06015f5e0 0x7fe0600d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7ff56a095640 1 -- 172.21.15.40:0/2589147851 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5640b4a70 con 0x7ff5640d1210 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 -- 172.21.15.40:0/2371291813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 msgr2=0x7f3da80380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 --2- 172.21.15.40:0/2371291813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da80380a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3da8008590 tx=0x7f3dac02a2c0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 -- 172.21.15.40:0/2371291813 shutdown_connections 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 --2- 172.21.15.40:0/2371291813 >> v2:172.21.15.40:3300/0 conn(0x7f3da804f0b0 0x7f3da804f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 --2- 172.21.15.40:0/2371291813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da80380a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3db772b640 1 -- 172.21.15.40:0/2371291813 >> 172.21.15.40:0/2371291813 conn(0x7f3da8009fe0 msgr2=0x7f3da800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975472880 0 0) 0x7fe0600d5c60 con 0x7fe0600d2150 2024-04-26T21:12:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe048003520 con 0x7fe0600d2150 2024-04-26T21:12:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7f3ae30f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3adc15b0d0 0x7f3adc0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127045156 0 0) 0x7fe0600b1d30 con 0x7fe0600b3540 2024-04-26T21:12:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 -- 172.21.15.40:0/1873323282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 msgr2=0x7f9a4c054560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/1873323282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a54063ef0 tx=0x7f9a540a6fa0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 -- 172.21.15.40:0/1873323282 shutdown_connections 2024-04-26T21:12:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0600d5c60 con 0x7fe0600b3540 2024-04-26T21:12:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/1873323282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c054b30 0x7f9a4c054560 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 -- 172.21.15.40:0/1873323282 >> 172.21.15.40:0/1873323282 conn(0x7f9a4c00a400 msgr2=0x7f9a4c00a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.128+0000 7efd58be4640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd540b4d70 0x7efd540aeba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009e10 tx=0x7efd4c02e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6937ac5de74d5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7ff56a896640 1 --2- 172.21.15.40:0/2589147851 >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 0x7ff56406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7efd52ffd640 1 -- 172.21.15.40:0/3668259135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c037070 con 0x7efd540b4d70 2024-04-26T21:12:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7efd52ffd640 1 -- 172.21.15.40:0/3668259135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c033850 con 0x7efd540b4d70 2024-04-26T21:12:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae0e94640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3adc15b0d0 0x7f3adc0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 -- 172.21.15.40:0/1873323282 shutdown_connections 2024-04-26T21:12:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f9a5a85b640 1 -- 172.21.15.40:0/1873323282 wait complete. 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae0e94640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3adc15b0d0 0x7f3adc0b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48478/0 (socket says 172.21.15.40:48478) 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae0e94640 1 -- 172.21.15.40:0/1139242734 learned_addr learned my addr 172.21.15.40:0/1139242734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc072730 con 0x7f3adc0add70 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc0b4be0 con 0x7f3adc0d1210 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4188906045 0 0) 0x7fe0600d5c60 con 0x7fe0600b3540 2024-04-26T21:12:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3adc073320 con 0x7f3adc15b0d0 2024-04-26T21:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 >> v2:172.21.15.184:3300/0 conn(0x7fe06015f5e0 msgr2=0x7fe0600d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 --2- 172.21.15.40:0/3742025103 >> v2:172.21.15.184:3300/0 conn(0x7fe06015f5e0 0x7fe0600d35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 >> v1:172.21.15.40:6789/0 conn(0x7fe0600d2150 legacy=0x7fe0600cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7ff56a095640 1 --2- 172.21.15.40:0/2589147851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640d1210 0x7ff5640b5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5640724c0 tx=0x7ff55402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f949cba7d28fa868 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7ff569093640 1 -- 172.21.15.40:0/2589147851 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554037070 con 0x7ff5640d1210 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7ff569093640 1 -- 172.21.15.40:0/2589147851 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff554032440 con 0x7ff5640d1210 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7ff569093640 1 -- 172.21.15.40:0/2589147851 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554037070 con 0x7ff5640d1210 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f86b8845640 1 Processor -- start 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f86b8845640 1 -- start start 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3db772b640 1 -- 172.21.15.40:0/2371291813 shutdown_connections 2024-04-26T21:12:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0600b2010 con 0x7fe0600b3540 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe054001520 con 0x7fe0600b3540 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 >> v1:172.21.15.184:6789/0 conn(0x7fe0600b3540 legacy=0x7fe0600d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7fe05dffb640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe048004c40 0x7fe048005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05dffb640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0480037a0 0x7fe048003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05dffb640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 0x7fe04800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0600d31b0 con 0x7fe048004c40 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800c340 con 0x7fe048004320 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe04800c540 con 0x7fe0480037a0 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05effd640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe048004c40 0x7fe048005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 -- 172.21.15.40:0/3668259135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd540b4d70 msgr2=0x7efd540aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd540b4d70 0x7efd540aeba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009e10 tx=0x7efd4c02e9f0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 -- 172.21.15.40:0/3668259135 shutdown_connections 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd540b4d70 0x7efd540aeba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 --2- 172.21.15.40:0/3668259135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd540ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 -- 172.21.15.40:0/3668259135 >> 172.21.15.40:0/3668259135 conn(0x7efd540b2990 msgr2=0x7efd540b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 -- 172.21.15.40:0/3668259135 shutdown_connections 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f73367fc640 1 -- 172.21.15.40:0/4142504176 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330036070 con 0x7f7338148880 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f73367fc640 1 -- 172.21.15.40:0/4142504176 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7330031600 con 0x7f7338148880 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f73367fc640 1 -- 172.21.15.40:0/4142504176 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300319d0 con 0x7f7338148880 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f733c831640 1 -- 172.21.15.40:0/4142504176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7338148880 msgr2=0x7f7338163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f733c831640 1 --2- 172.21.15.40:0/4142504176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7338148880 0x7f7338163c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7330009e50 tx=0x7f733002a640 comp rx=0 tx=0).stop 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214200041 0 0) 0x7f3adc0b4be0 con 0x7f3adc0d1210 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3acc003520 con 0x7f3adc0d1210 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625509910 0 0) 0x7f3adc072730 con 0x7f3adc0add70 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3adc0b4be0 con 0x7f3adc0add70 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793598923 0 0) 0x7f3acc003520 con 0x7f3adc0d1210 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 >> v2:172.21.15.184:3300/0 conn(0x7f3adc15b0d0 msgr2=0x7f3adc0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 --2- 172.21.15.40:0/1139242734 >> v2:172.21.15.184:3300/0 conn(0x7f3adc15b0d0 0x7f3adc0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 >> v1:172.21.15.40:6789/0 conn(0x7f3adc0add70 legacy=0x7f3adc0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc0729f0 con 0x7f3adc0d1210 2024-04-26T21:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3ac8001670 con 0x7f3adc0d1210 2024-04-26T21:12:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.129+0000 7f3db772b640 1 -- 172.21.15.40:0/2371291813 wait complete. 2024-04-26T21:12:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f9a5a85b640 1 Processor -- start 2024-04-26T21:12:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 >> v1:172.21.15.184:6789/0 conn(0x7f3adc0d1210 legacy=0x7f3adc0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3db772b640 1 Processor -- start 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3db772b640 1 -- start start 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 0x7fe04800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7fe05f7fe640 1 -- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0480037a0 msgr2=0x7fe048003c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0480037a0 0x7fe048003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7fe05f7fe640 1 -- 172.21.15.40:0/3742025103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe048004c40 msgr2=0x7fe048005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe048004c40 0x7fe048005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7fe05f7fe640 1 -- 172.21.15.40:0/3742025103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0600d5c60 con 0x7fe048004320 2024-04-26T21:12:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc003b40 0x7f3acc003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 0x7f3acc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0051c0 0x7f3acc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc073320 con 0x7f3acc003b40 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adbfff640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 0x7f3acc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc00c060 con 0x7f3acc0051c0 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc00c260 con 0x7f3acc004800 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f86b8845640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 0x7f86b40adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f86b8845640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b4156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7ff56c2f9640 1 -- 172.21.15.40:0/2589147851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640d1210 msgr2=0x7ff5640b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/2589147851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640d1210 0x7ff5640b5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5640724c0 tx=0x7ff55402a270 comp rx=0 tx=0).stop 2024-04-26T21:12:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f86b8845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b8845640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86b406ed30 con 0x7f86b40d0010 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b8845640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86b406ea50 con 0x7f86b4070040 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b8845640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b406f010 con 0x7f86b40b4ef0 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 0x7f86b40adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40434/0 (socket says 172.21.15.40:40434) 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1801653286 learned_addr learned my addr 172.21.15.40:0/1801653286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7fe05f7fe640 1 -- 172.21.15.40:0/3742025103 reap_dead start 2024-04-26T21:12:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f733c831640 1 -- 172.21.15.40:0/4142504176 shutdown_connections 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f733c831640 1 --2- 172.21.15.40:0/4142504176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7338148880 0x7f7338163c90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f733c831640 1 -- 172.21.15.40:0/4142504176 >> 172.21.15.40:0/4142504176 conn(0x7f733806d3b0 msgr2=0x7f733806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3ae1695640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0051c0 0x7f3acc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3adbfff640 1 -- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0051c0 msgr2=0x7f3acc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3adbfff640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0051c0 0x7f3acc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3adbfff640 1 -- 172.21.15.40:0/1139242734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc003b40 msgr2=0x7f3acc003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3ae0e94640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc003b40 0x7f3acc003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.130+0000 7efd5a647640 1 -- 172.21.15.40:0/3668259135 wait complete. 2024-04-26T21:12:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.131+0000 7f9a5a85b640 1 -- start start 2024-04-26T21:12:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3adbfff640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc003b40 0x7f3acc003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 0x7fe04800ba60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe054007960 tx=0x7fe048005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8223e673ff1f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3adbfff640 1 -- 172.21.15.40:0/1139242734 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc003520 con 0x7f3acc004800 2024-04-26T21:12:11.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7efd5a647640 1 Processor -- start 2024-04-26T21:12:11.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3ae1695640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3acc0051c0 0x7f3acc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3ae0e94640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc003b40 0x7f3acc003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3ae1695640 1 -- 172.21.15.40:0/1139242734 reap_dead start 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7ff56c2f9640 1 -- 172.21.15.40:0/2589147851 shutdown_connections 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/2589147851 >> v2:172.21.15.40:3300/0 conn(0x7ff5640b6530 0x7ff56406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/2589147851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640d1210 0x7ff5640b5550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7ff56c2f9640 1 -- 172.21.15.40:0/2589147851 >> 172.21.15.40:0/2589147851 conn(0x7ff564072900 msgr2=0x7ff564072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b1d9b640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b4156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 msgr2=0x7f86b4156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b4156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 msgr2=0x7f86b40adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 0x7f86b40adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1801653286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b4156ab0 con 0x7f86b40d0010 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b259c640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 0x7f86b40adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054040070 con 0x7fe048004320 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe054033830 con 0x7fe048004320 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7fe05dffb640 1 -- 172.21.15.40:0/3742025103 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540079a0 con 0x7fe048004320 2024-04-26T21:12:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f733c831640 1 -- 172.21.15.40:0/4142504176 shutdown_connections 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3db772b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 0x7f3da817da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f3db772b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 0x7f9a4c0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 0x7f9a4c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0e7a20 con 0x7f9a4c054b30 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c054210 con 0x7f9a4c0e7640 2024-04-26T21:12:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f9a5a85b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c048270 con 0x7f9a4c04fbe0 2024-04-26T21:12:11.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86a800d5d0 tx=0x7f86a800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7ff56c2f9640 1 -- 172.21.15.40:0/2589147851 shutdown_connections 2024-04-26T21:12:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db772b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da804f0b0 0x7f3da804a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db4cc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48508/0 (socket says 172.21.15.40:48508) 2024-04-26T21:12:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 -- 172.21.15.40:0/3576641667 learned_addr learned my addr 172.21.15.40:0/3576641667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db4cc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48512/0 (socket says 172.21.15.40:48512) 2024-04-26T21:12:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db4cc6640 1 -- 172.21.15.40:0/147426765 learned_addr learned my addr 172.21.15.40:0/147426765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7efd5a647640 1 -- start start 2024-04-26T21:12:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a59058640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 0x7f9a4c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a80125f0 con 0x7f86b40d0010 2024-04-26T21:12:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db5cc8640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da804f0b0 0x7f3da804a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a59859640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 0x7f9a4c0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 -- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 msgr2=0x7f9a4c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f86b8845640 1 -- 172.21.15.40:0/1801653286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b4156d90 con 0x7f86b40d0010 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80385e0 con 0x7f3da8033f80 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 0x7f9a4c0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 -- 172.21.15.40:0/3576641667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 msgr2=0x7f9a4c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 0x7f9a4c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a5a05a640 1 -- 172.21.15.40:0/3576641667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0e1aa0 con 0x7f9a4c0e7640 2024-04-26T21:12:11.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a59859640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 0x7f9a4c0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f86b8845640 1 -- 172.21.15.40:0/1801653286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b4157350 con 0x7f86b40d0010 2024-04-26T21:12:11.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f9a59058640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 0x7f9a4c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80531b0 con 0x7f3da8037c20 2024-04-26T21:12:11.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3adbfff640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 0x7f3acc004c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8003c70 tx=0x7f3acc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e9ad1755206bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80540c0 con 0x7f3da804f0b0 2024-04-26T21:12:11.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f9a5a05a640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400c3c0 tx=0x7f9a4400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7fe05ffff640 1 -- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 msgr2=0x7fe04800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7fe05ffff640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 0x7fe04800ba60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe054007960 tx=0x7fe048005830 comp rx=0 tx=0).stop 2024-04-26T21:12:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.133+0000 7ff56c2f9640 1 -- 172.21.15.40:0/2589147851 wait complete. 2024-04-26T21:12:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8003870 con 0x7f3acc004800 2024-04-26T21:12:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8019070 con 0x7f86b40d0010 2024-04-26T21:12:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 0x7efd54157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd54156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac802f070 con 0x7f3acc004800 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3adb7fe640 1 -- 172.21.15.40:0/1139242734 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80064b0 con 0x7f3acc004800 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 0x7efd5409e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd5415b4c0 con 0x7efd5415b0e0 2024-04-26T21:12:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd54072cd0 con 0x7efd540b4d70 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd5a647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd540af2e0 con 0x7efd540b5c90 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a80129f0 con 0x7f86b40d0010 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.132+0000 7f733c831640 1 -- 172.21.15.40:0/4142504176 wait complete. 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3db4cc6640 1 -- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da804f0b0 msgr2=0x7f3da804a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3db4cc6640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da804f0b0 0x7f3da804a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f3db4cc6640 1 -- 172.21.15.40:0/147426765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 msgr2=0x7f3da817da30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f733c831640 1 Processor -- start 2024-04-26T21:12:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd58be4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 0x7efd5409e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd53fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 0x7efd54157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd58be4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 0x7efd5409e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40460/0 (socket says 172.21.15.40:40460) 2024-04-26T21:12:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7efd58be4640 1 -- 172.21.15.40:0/1863160670 learned_addr learned my addr 172.21.15.40:0/1863160670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7ff56c2f9640 1 Processor -- start 2024-04-26T21:12:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7ff56c2f9640 1 -- start start 2024-04-26T21:12:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f733c831640 1 -- start start 2024-04-26T21:12:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3db54c7640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 0x7f3da817da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 -- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 msgr2=0x7efd54156440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd54156440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 -- 172.21.15.40:0/1863160670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 msgr2=0x7efd5409e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 0x7efd5409e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 -- 172.21.15.40:0/1863160670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd54156a60 con 0x7efd540b4d70 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3db4cc6640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 0x7f3da817da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3db4cc6640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da817df70 con 0x7f3da8037c20 2024-04-26T21:12:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3db54c7640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 0x7f3da817da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 msgr2=0x7f3acc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 0x7f3acc004c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8003c70 tx=0x7f3acc005830 comp rx=0 tx=0).stop 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 shutdown_connections 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/1139242734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3acc004800 0x7f3acc004c80 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8003c70 tx=0x7f3acc005830 comp rx=0 tx=0).stop 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 >> 172.21.15.40:0/1139242734 conn(0x7f3adc06c6f0 msgr2=0x7f3adc06cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd53fff640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 0x7efd54157b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c3c0 tx=0x7efd44001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f86b8845640 1 -- 172.21.15.40:0/1801653286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b406a290 con 0x7f86b40d0010 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440052d0 con 0x7f9a4c0e7640 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a44005470 con 0x7f9a4c0e7640 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.134+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44007220 con 0x7f9a4c0e7640 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c0e1d40 con 0x7f9a4c0e7640 2024-04-26T21:12:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c0e2300 con 0x7f9a4c0e7640 2024-04-26T21:12:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44017070 con 0x7efd540b4d70 2024-04-26T21:12:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd54156d40 con 0x7efd540b4d70 2024-04-26T21:12:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3db4cc6640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0090b0 tx=0x7f3dac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7fe05ffff640 1 -- 172.21.15.40:0/3742025103 shutdown_connections 2024-04-26T21:12:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7fe05ffff640 1 --2- 172.21.15.40:0/3742025103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe048004320 0x7fe04800ba60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.135+0000 7fe05ffff640 1 -- 172.21.15.40:0/3742025103 >> 172.21.15.40:0/3742025103 conn(0x7fe0600d5010 msgr2=0x7fe0600d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd54157300 con 0x7efd540b4d70 2024-04-26T21:12:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 shutdown_connections 2024-04-26T21:12:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd44005ce0 con 0x7efd540b4d70 2024-04-26T21:12:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7fe05ffff640 1 -- 172.21.15.40:0/3742025103 shutdown_connections 2024-04-26T21:12:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3ae30f8640 1 -- 172.21.15.40:0/1139242734 wait complete. 2024-04-26T21:12:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44017070 con 0x7efd540b4d70 2024-04-26T21:12:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac037040 con 0x7f3da8037c20 2024-04-26T21:12:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da817a9b0 con 0x7f3da8037c20 2024-04-26T21:12:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86a8012b90 con 0x7f86b40d0010 2024-04-26T21:12:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f733c831640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f733806c2a0 0x7f7338167220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f733c831640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 0x7f7338145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f733c831640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.136+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da8048ae0 con 0x7f3da8037c20 2024-04-26T21:12:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7f86a2ffd640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 0x7f869406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f86a808bea0 con 0x7f86b40d0010 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f86b259c640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 0x7f869406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640b6530 0x7ff56415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 0x7ff564146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff564068090 con 0x7ff5640d1210 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5640b5030 con 0x7ff5640b6530 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56c2f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff564068210 con 0x7ff56406d2b0 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56a095640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56a095640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57124/0 (socket says 172.21.15.40:57124) 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7ff56a095640 1 -- 172.21.15.40:0/1785317090 learned_addr learned my addr 172.21.15.40:0/1785317090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7ff569894640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640b6530 0x7ff56415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7ff56a896640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 0x7ff564146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f7337fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f7337fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40476/0 (socket says 172.21.15.40:40476) 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f7337fff640 1 -- 172.21.15.40:0/2720154835 learned_addr learned my addr 172.21.15.40:0/2720154835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73381473d0 con 0x7f73381641d0 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7338147250 con 0x7f733806c2a0 2024-04-26T21:12:11.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7338147550 con 0x7f7338148880 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac042070 con 0x7f3da8037c20 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0044e0 con 0x7f3da8037c20 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7ff56a095640 1 -- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640b6530 msgr2=0x7ff56415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7ff56a095640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640b6530 0x7ff56415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7ff56a095640 1 -- 172.21.15.40:0/1785317090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 msgr2=0x7ff564146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff56a095640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 0x7ff564146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff56a095640 1 -- 172.21.15.40:0/1785317090 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56415b650 con 0x7ff56406d2b0 2024-04-26T21:12:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f3ae30f8640 1 Processor -- start 2024-04-26T21:12:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f3ae30f8640 1 -- start start 2024-04-26T21:12:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff56a896640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 0x7ff564146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.137+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c0057e0 con 0x7f9a4c0e7640 2024-04-26T21:12:11.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7336ffd640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 0x7f7338145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a440073c0 con 0x7f9a4c0e7640 2024-04-26T21:12:11.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.138+0000 7f9a4a7fc640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 0x7f9a3806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9a4408af70 con 0x7f9a4c0e7640 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff56a095640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff554009960 tx=0x7ff5540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554037070 con 0x7ff56406d2b0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff554004030 con 0x7ff56406d2b0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56414ba30 con 0x7ff56406d2b0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554004420 con 0x7ff56406d2b0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56414bff0 con 0x7ff56406d2b0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f86b259c640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 0x7f869406ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86ac034000 tx=0x7f86ac00a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90005290 con 0x7f3da8037c20 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dac003810 con 0x7f3da8037c20 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7fe05ffff640 1 -- 172.21.15.40:0/3742025103 wait complete. 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd4401a070 con 0x7efd540b4d70 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7337fff640 1 -- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 msgr2=0x7f7338145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7337fff640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 0x7f7338145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7337fff640 1 -- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f733806c2a0 msgr2=0x7f7338167220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7337fff640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f733806c2a0 0x7f7338167220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f7337fff640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338145f10 con 0x7f73381641d0 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f7337fff640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c3f0 tx=0x7f732c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f7336ffd640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 0x7f7338145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f9a59859640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 0x7f9a3806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f9a59859640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 0x7f9a3806eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a50000f60 tx=0x7f9a50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3ae30f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3ae30f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3adc156a30 0x7f3adc154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3ae0e94640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3ae0e94640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57138/0 (socket says 172.21.15.40:57138) 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.139+0000 7f3ae30f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae1695640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae1695640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40496/0 (socket says 172.21.15.40:40496) 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae0e94640 1 -- 172.21.15.40:0/2849313987 learned_addr learned my addr 172.21.15.40:0/2849313987 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc073bf0 con 0x7f3adc1e5640 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc072fb0 con 0x7f3adc156a30 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc15b4b0 con 0x7f3adc1e5260 2024-04-26T21:12:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.140+0000 7f3da5ffb640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 0x7f3d9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 -- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3adc156a30 msgr2=0x7f3adc154d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3adc156a30 0x7f3adc154d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 -- 172.21.15.40:0/2849313987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 msgr2=0x7f3adc155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc09bbc0 con 0x7f3adc1e5260 2024-04-26T21:12:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3dac0ac4d0 con 0x7f3da8037c20 2024-04-26T21:12:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae1695640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd34005290 con 0x7efd540b4d70 2024-04-26T21:12:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae0e94640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000dfd0 tx=0x7f3ad000f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3db54c7640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 0x7f3d9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0018070 con 0x7f3adc1e5260 2024-04-26T21:12:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff564070600 con 0x7ff56406d2b0 2024-04-26T21:12:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad0014890 con 0x7f3adc1e5260 2024-04-26T21:12:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7efd50ff9640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 0x7efd4006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc09bea0 con 0x7f3adc1e5260 2024-04-26T21:12:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc15e4f0 con 0x7f3adc1e5260 2024-04-26T21:12:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0014c80 con 0x7f3adc1e5260 2024-04-26T21:12:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.141+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efd44005ae0 con 0x7efd540b4d70 2024-04-26T21:12:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.142+0000 7fe05ffff640 1 Processor -- start 2024-04-26T21:12:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.142+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff55403b6d0 con 0x7ff56406d2b0 2024-04-26T21:12:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.142+0000 7fe05ffff640 1 -- start start 2024-04-26T21:12:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.142+0000 7f3db54c7640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 0x7f3d9406ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db006aae0 tx=0x7f3db00690a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.142+0000 7ff55affd640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 0x7ff54806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff5540abe40 con 0x7ff56406d2b0 2024-04-26T21:12:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7efd537fe640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 0x7efd4006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7ff569894640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 0x7ff54806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc09cb20 con 0x7f3adc1e5260 2024-04-26T21:12:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7efd537fe640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 0x7efd4006f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd5415b660 tx=0x7efd48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7ff569894640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 0x7ff54806ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff56414b960 tx=0x7ff560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.143+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 0x7fe0600ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 0x7fe0600cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 0x7fe0600b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0600b3f40 con 0x7fe06015a870 2024-04-26T21:12:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ad001b070 con 0x7f3adc1e5260 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 0x7fe0600cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 0x7fe0600ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 0x7fe0600cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40510/0 (socket says 172.21.15.40:40510) 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05e7fc640 1 -- 172.21.15.40:0/1222723130 learned_addr learned my addr 172.21.15.40:0/1222723130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05ffff640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0600d5f40 con 0x7fe0600b59c0 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 0x7fe0600b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05ffff640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0600b40c0 con 0x7fe048004320 2024-04-26T21:12:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7f3ad97fa640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 0x7f3ac006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 -- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 msgr2=0x7fe0600b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 0x7fe0600b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 -- 172.21.15.40:0/1222723130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 msgr2=0x7fe0600cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 0x7fe0600cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05effd640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0600ce490 con 0x7fe048004320 2024-04-26T21:12:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7fe05f7fe640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 0x7fe0600b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a44059380 con 0x7f9a4c0e7640 2024-04-26T21:12:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.144+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ad008ba10 con 0x7f3adc1e5260 2024-04-26T21:12:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.145+0000 7f3adbfff640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 0x7f3ac006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.145+0000 7fe05effd640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 0x7fe0600ccaa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe058004690 tx=0x7fe058007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.145+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86a805a2c0 con 0x7f86b40d0010 2024-04-26T21:12:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.145+0000 7f3adbfff640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 0x7f3ac006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8033000 tx=0x7f3ac800a030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.145+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c017070 con 0x7f73381641d0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.146+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c0052d0 con 0x7f73381641d0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.146+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733815e680 con 0x7f73381641d0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.146+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733815ec40 con 0x7f73381641d0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.146+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c003810 con 0x7f73381641d0 2024-04-26T21:12:11.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.148+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff564070600 con 0x7ff56406d2b0 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.148+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73380693d0 con 0x7f73381641d0 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.148+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f732c01a070 con 0x7f73381641d0 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.148+0000 7f731ffff640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 0x7f732406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.149+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f732c08abe0 con 0x7f73381641d0 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.149+0000 7f73377fe640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 0x7f732406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.149+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3da817df70 con 0x7f3da8037c20 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.149+0000 7f73377fe640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 0x7f732406ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f73280016a0 tx=0x7f73280079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.152+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ad0059960 con 0x7f3adc1e5260 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.152+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058007e50 con 0x7fe048004320 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.152+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05801a070 con 0x7fe048004320 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.152+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058015620 con 0x7fe048004320 2024-04-26T21:12:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.155+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd44058540 con 0x7efd540b4d70 2024-04-26T21:12:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.157+0000 7fe05ffff640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0600cd0b0 con 0x7fe048004320 2024-04-26T21:12:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.158+0000 7fe05ffff640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0600cd670 con 0x7fe048004320 2024-04-26T21:12:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.158+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f732c058aa0 con 0x7f73381641d0 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.160+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0580157c0 con 0x7fe048004320 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.160+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0600693d0 con 0x7fe048004320 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.161+0000 7fe0477fe640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 0x7fe04c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.161+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe058058010 con 0x7fe048004320 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.162+0000 7fe05e7fc640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 0x7fe04c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.162+0000 7fe05e7fc640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 0x7fe04c06efd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe050005880 tx=0x7fe050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.166+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe058059f70 con 0x7fe048004320 2024-04-26T21:12:11.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.577+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3d90005580 con 0x7f3da8037c20 2024-04-26T21:12:11.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.578+0000 7f3da5ffb640 1 -- 172.21.15.40:0/147426765 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dac07a860 con 0x7f3da8037c20 2024-04-26T21:12:11.579 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2024-04-26T21:12:11.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.579+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9a4c0e81e0 con 0x7f9a4c0e7640 2024-04-26T21:12:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.580+0000 7f9a4a7fc640 1 -- 172.21.15.40:0/3576641667 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a44058c80 con 0x7f9a4c0e7640 2024-04-26T21:12:11.581 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:11.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.581+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff5640693d0 con 0x7ff56406d2b0 2024-04-26T21:12:11.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.581+0000 7ff55affd640 1 -- 172.21.15.40:0/1785317090 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff554042020 con 0x7ff56406d2b0 2024-04-26T21:12:11.582 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2024-04-26T21:12:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 msgr2=0x7f3d9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 0x7f3d9406ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db006aae0 tx=0x7f3db00690a0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 msgr2=0x7f3da817a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0090b0 tx=0x7f3dac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 shutdown_connections 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d9406c920 0x7f3d9406ede0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3da804f0b0 0x7f3da804a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3da8037c20 0x7f3da817a470 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0090b0 tx=0x7f3dac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 --2- 172.21.15.40:0/147426765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3da8033f80 0x7f3da817da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.582+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 >> 172.21.15.40:0/147426765 conn(0x7f3da8009fe0 msgr2=0x7f3da8047a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.583+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 shutdown_connections 2024-04-26T21:12:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.583+0000 7f3db772b640 1 -- 172.21.15.40:0/147426765 wait complete. 2024-04-26T21:12:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.583+0000 7f86b8845640 1 -- 172.21.15.40:0/1801653286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f86b40b0dd0 con 0x7f86b40d0010 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.583+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 msgr2=0x7f9a3806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 0x7f9a3806eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a50000f60 tx=0x7f9a50009040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 msgr2=0x7f9a4c0e60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400c3c0 tx=0x7f9a4400c880 comp rx=0 tx=0).stop 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 shutdown_connections 2024-04-26T21:12:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9a3806c9f0 0x7f9a3806eeb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a4c0e7640 0x7f9a4c0e60b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a4c054b30 0x7f9a4c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 --2- 172.21.15.40:0/3576641667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a4c04fbe0 0x7f9a4c0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f86a2ffd640 1 -- 172.21.15.40:0/1801653286 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86a8059bc0 con 0x7f86b40d0010 2024-04-26T21:12:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 >> 172.21.15.40:0/3576641667 conn(0x7f9a4c00a400 msgr2=0x7f9a4c04f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.589 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2024-04-26T21:12:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.584+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 shutdown_connections 2024-04-26T21:12:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.585+0000 7f9a5a85b640 1 -- 172.21.15.40:0/3576641667 wait complete. 2024-04-26T21:12:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 msgr2=0x7ff54806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 0x7ff54806ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff56414b960 tx=0x7ff560009040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 msgr2=0x7ff56414b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff554009960 tx=0x7ff5540046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 shutdown_connections 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff54806c9c0 0x7ff54806ee80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.586+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5640d1210 0x7ff564146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5640b6530 0x7ff56415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7ff56c2f9640 1 --2- 172.21.15.40:0/1785317090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff56406d2b0 0x7ff56414b420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 >> 172.21.15.40:0/1785317090 conn(0x7ff564072900 msgr2=0x7ff5640b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7efd34005850 con 0x7efd540b4d70 2024-04-26T21:12:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 shutdown_connections 2024-04-26T21:12:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7ff56c2f9640 1 -- 172.21.15.40:0/1785317090 wait complete. 2024-04-26T21:12:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.587+0000 7efd50ff9640 1 -- 172.21.15.40:0/1863160670 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd4405c840 con 0x7efd540b4d70 2024-04-26T21:12:11.593 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 msgr2=0x7f869406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 0x7f869406ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86ac034000 tx=0x7f86ac00a000 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 msgr2=0x7f86b40afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86a800d5d0 tx=0x7f86a800da90 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 shutdown_connections 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c9c0 0x7f869406ee80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b40afd70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b4ef0 0x7f86b4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 --2- 172.21.15.40:0/1801653286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b4070040 0x7f86b40adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.590+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 >> 172.21.15.40:0/1801653286 conn(0x7f86b40b2ae0 msgr2=0x7f86b40b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 shutdown_connections 2024-04-26T21:12:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7f86a0ff9640 1 -- 172.21.15.40:0/1801653286 wait complete. 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 msgr2=0x7efd4006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 0x7efd4006f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd5415b660 tx=0x7efd48008040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 msgr2=0x7efd54157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.591+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 0x7efd54157b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c3c0 tx=0x7efd44001e70 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 shutdown_connections 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd4006ccf0 0x7efd4006f1b0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd5415b660 tx=0x7efd48008040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd5415b0e0 0x7efd5409e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd540b5c90 0x7efd54156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 --2- 172.21.15.40:0/1863160670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd540b4d70 0x7efd54157b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 >> 172.21.15.40:0/1863160670 conn(0x7efd540b2990 msgr2=0x7efd540b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3adc06a290 con 0x7f3adc1e5260 2024-04-26T21:12:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.592+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7338073a70 con 0x7f73381641d0 2024-04-26T21:12:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.593+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 shutdown_connections 2024-04-26T21:12:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.593+0000 7f3ad97fa640 1 -- 172.21.15.40:0/2849313987 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ad0014e20 con 0x7f3adc1e5260 2024-04-26T21:12:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.593+0000 7efd5a647640 1 -- 172.21.15.40:0/1863160670 wait complete. 2024-04-26T21:12:11.599 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-26T21:12:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 msgr2=0x7f3ac006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 0x7f3ac006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8033000 tx=0x7f3ac800a030 comp rx=0 tx=0).stop 2024-04-26T21:12:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 msgr2=0x7f3adc156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000dfd0 tx=0x7f3ad000f920 comp rx=0 tx=0).stop 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 shutdown_connections 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ac006c940 0x7f3ac006ee00 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8033000 tx=0x7f3ac800a030 comp rx=0 tx=0).stop 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3adc1e5640 0x7f3adc155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3adc156a30 0x7f3adc154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.596+0000 7f3ae30f8640 1 --2- 172.21.15.40:0/2849313987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3adc1e5260 0x7f3adc156e50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.597+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 >> 172.21.15.40:0/2849313987 conn(0x7f3adc06c6f0 msgr2=0x7f3adc1589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.597+0000 7f731ffff640 1 -- 172.21.15.40:0/2720154835 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f732c05cda0 con 0x7f73381641d0 2024-04-26T21:12:11.601 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.599+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe0600b8050 con 0x7fe048004320 2024-04-26T21:12:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.600+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 shutdown_connections 2024-04-26T21:12:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.600+0000 7f3ae30f8640 1 -- 172.21.15.40:0/2849313987 wait complete. 2024-04-26T21:12:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.600+0000 7fe0477fe640 1 -- 172.21.15.40:0/1222723130 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe05805e270 con 0x7fe048004320 2024-04-26T21:12:11.603 INFO:teuthology.orchestra.run.smithi040.stdout:64424509446 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.601+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 msgr2=0x7f732406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.601+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 0x7f732406ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f73280016a0 tx=0x7f73280079b0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.601+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 msgr2=0x7f7338160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.601+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c3f0 tx=0x7f732c00c8b0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 shutdown_connections 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f732406c970 0x7f732406ee30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73381641d0 0x7f7338160a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7338148880 0x7f7338145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 --2- 172.21.15.40:0/2720154835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f733806c2a0 0x7f7338167220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 >> 172.21.15.40:0/2720154835 conn(0x7f733806d3b0 msgr2=0x7f733806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 shutdown_connections 2024-04-26T21:12:11.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.602+0000 7f733c831640 1 -- 172.21.15.40:0/2720154835 wait complete. 2024-04-26T21:12:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 msgr2=0x7fe04c06efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 0x7fe04c06efd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe050005880 tx=0x7fe050009040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 msgr2=0x7fe0600ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 0x7fe0600ccaa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe058004690 tx=0x7fe058007700 comp rx=0 tx=0).stop 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 shutdown_connections 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe04c06cb10 0x7fe04c06efd0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe050005880 tx=0x7fe050009040 comp rx=0 tx=0).stop 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0600b59c0 0x7fe0600b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06015a870 0x7fe0600cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 --2- 172.21.15.40:0/1222723130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe048004320 0x7fe0600ccaa0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe058004690 tx=0x7fe058007700 comp rx=0 tx=0).stop 2024-04-26T21:12:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 >> 172.21.15.40:0/1222723130 conn(0x7fe0600d5010 msgr2=0x7fe06015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 shutdown_connections 2024-04-26T21:12:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:11.604+0000 7fe0457fa640 1 -- 172.21.15.40:0/1222723130 wait complete. 2024-04-26T21:12:11.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-04-26T21:12:11.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-04-26T21:12:11.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-04-26T21:12:11.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-04-26T21:12:11.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-04-26T21:12:11.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.1 2024-04-26T21:12:11.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2024-04-26T21:12:11.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-04-26T21:12:12.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:12:12.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:12:12.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:12:12.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:12:12.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:12:12.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:12:12.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:12:12.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:12:12.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.779+0000 7fb998324640 1 Processor -- start 2024-04-26T21:12:12.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.780+0000 7fb998324640 1 -- start start 2024-04-26T21:12:12.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.780+0000 7f76cd8d1640 1 Processor -- start 2024-04-26T21:12:12.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.780+0000 7f76cd8d1640 1 -- start start 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fb998324640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9900b16e0 0x7fb99015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fb998324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb9900ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fb998324640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9900b2c40 con 0x7fb99015e800 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.779+0000 7fcf3c81c640 1 Processor -- start 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.779+0000 7fcf3c81c640 1 -- start start 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3c81c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf38166300 0x7fcf3806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3c81c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf381666e0 con 0x7fcf380b3c90 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3c81c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3816cfa0 con 0x7fcf3816c950 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3c81c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3816cd60 con 0x7fcf38166300 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf36d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf38166300 0x7fcf3806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcf380b3c90 0x7fcf3816c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58524/0 (socket says 172.21.15.40:58524) 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fcf3659c640 1 -- 172.21.15.40:0/1731410775 learned_addr learned my addr 172.21.15.40:0/1731410775 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fb998324640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9900b27b0 con 0x7fb9900b16e0 2024-04-26T21:12:12.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.781+0000 7fb998324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900b1ba0 con 0x7fb99015ebe0 2024-04-26T21:12:12.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76cd8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76c806f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7fb9958bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9900b16e0 0x7fb99015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7fb9960c0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb99015e800 0x7fb99015f0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58540/0 (socket says 172.21.15.40:58540) 2024-04-26T21:12:12.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7fb9960c0640 1 -- 172.21.15.40:0/820256364 learned_addr learned my addr 172.21.15.40:0/820256364 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76cd8d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c806ff90 con 0x7f76c81672f0 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76c806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76c77fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f76c8072fe0 0x7f76c806e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49428/0 (socket says 172.21.15.40:49428) 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76c77fe640 1 -- 172.21.15.40:0/1515384840 learned_addr learned my addr 172.21.15.40:0/1515384840 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c8070270 con 0x7f76c8072fe0 2024-04-26T21:12:12.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80b4130 con 0x7f76c814d180 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915280822 0 0) 0x7fb9900b2c40 con 0x7fb99015e800 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb97c003520 con 0x7fb99015e800 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375522347 0 0) 0x7f76c8070270 con 0x7f76c8072fe0 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76b0003520 con 0x7f76c8072fe0 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139470610 0 0) 0x7f76c806ff90 con 0x7f76c81672f0 2024-04-26T21:12:12.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76c8070270 con 0x7f76c81672f0 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb9900ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4264805623 0 0) 0x7f76b0003520 con 0x7f76c8072fe0 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 msgr2=0x7f76c806f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76c806f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 >> v1:172.21.15.40:6789/0 conn(0x7f76c81672f0 legacy=0x7f76c80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8070550 con 0x7f76c8072fe0 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76c806f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f76b8001670 con 0x7f76c8072fe0 2024-04-26T21:12:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 >> v1:172.21.15.184:6789/0 conn(0x7f76c8072fe0 legacy=0x7f76c806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b0003b40 0x7f76b0003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b0004d40 0x7f76b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80b4130 con 0x7f76b0003b40 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76b000bb50 con 0x7f76b0004d40 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76b000bd50 con 0x7f76c814d180 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b0003b40 0x7f76b0003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c67fc640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f76c77fe640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b0004d40 0x7f76b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4238550303 0 0) 0x7fb97c003520 con 0x7fb99015e800 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 msgr2=0x7fb9900ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb9900ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 >> v2:172.21.15.40:3300/0 conn(0x7fb9900b16e0 msgr2=0x7fb99015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 --2- 172.21.15.40:0/820256364 >> v2:172.21.15.40:3300/0 conn(0x7fb9900b16e0 0x7fb99015f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900b1e80 con 0x7fb99015e800 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fb980001670 con 0x7fb99015e800 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9958bf640 1 --2- 172.21.15.40:0/820256364 >> v2:172.21.15.40:3300/0 conn(0x7fb9900b16e0 0x7fb99015f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb9900ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 >> v1:172.21.15.40:6789/0 conn(0x7fb99015e800 legacy=0x7fb99015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c0039b0 0x7fb97c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fb9950be640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb97c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9950be640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb97c004d40 0x7fb97c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 msgr2=0x7f76b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c814d180 0x7f76b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b0004d40 msgr2=0x7f76b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b0004d40 0x7f76b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1515384840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b0003520 con 0x7f76b0003b40 2024-04-26T21:12:12.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7f8987989640 1 Processor -- start 2024-04-26T21:12:12.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7f8987989640 1 -- start start 2024-04-26T21:12:12.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337706116 0 0) 0x7fcf381666e0 con 0x7fcf380b3c90 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.782+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf20003520 con 0x7fcf380b3c90 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2630391735 0 0) 0x7fcf20003520 con 0x7fcf380b3c90 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 >> v2:172.21.15.184:3300/0 conn(0x7fcf38166300 msgr2=0x7fcf3806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 --2- 172.21.15.40:0/1731410775 >> v2:172.21.15.184:3300/0 conn(0x7fcf38166300 0x7fcf3806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 >> v1:172.21.15.184:6789/0 conn(0x7fcf3816c950 legacy=0x7fcf380add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3806de60 con 0x7fcf380b3c90 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fcf300019f0 con 0x7fcf380b3c90 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 >> v1:172.21.15.40:6789/0 conn(0x7fcf380b3c90 legacy=0x7fcf3816c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf200039b0 0x7fcf20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf200052e0 0x7fcf2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3816cd60 con 0x7fcf20004800 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2000c1b0 con 0x7fcf200039b0 2024-04-26T21:12:12.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.783+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2000c3b0 con 0x7fcf200052e0 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fcf3659c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.784+0000 7fcf3759e640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf200052e0 0x7fcf2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fcf36d9d640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf200039b0 0x7fcf20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3659c640 1 -- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf200052e0 msgr2=0x7fcf2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3659c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf200052e0 0x7fcf2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3659c640 1 -- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf200039b0 msgr2=0x7fcf20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3659c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf200039b0 0x7fcf20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3659c640 1 -- 172.21.15.40:0/1731410775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf20003520 con 0x7fcf20004800 2024-04-26T21:12:12.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fcf3759e640 1 -- 172.21.15.40:0/1731410775 reap_dead start 2024-04-26T21:12:12.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b27b0 con 0x7fb97c0039b0 2024-04-26T21:12:12.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b1ba0 con 0x7fb97c004d40 2024-04-26T21:12:12.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c00ba70 con 0x7fb99015ebe0 2024-04-26T21:12:12.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9958bf640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb97c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb97c004d40 0x7fb97c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9968c1640 1 -- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 msgr2=0x7fb97c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99015ebe0 0x7fb97c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9968c1640 1 -- 172.21.15.40:0/820256364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c0039b0 msgr2=0x7fb97c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.785+0000 7fb9960c0640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c0039b0 0x7fb97c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c0039b0 0x7fb97c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fb9968c1640 1 -- 172.21.15.40:0/820256364 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c003520 con 0x7fb97c004d40 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7fb9960c0640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c0039b0 0x7fb97c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:12.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b0003b40 0x7f76b0003f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c490 tx=0x7f76b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5957e0adcb98c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0038070 con 0x7f76b0003b40 2024-04-26T21:12:12.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7fcf3659c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000a460 tx=0x7fcf20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd84205ea4de0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f8987989640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 0x7f8980072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7fb9968c1640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb97c004d40 0x7fb97c0051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb99006dbc0 tx=0x7fb97c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b327a4b112eda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f8987989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c009c80 con 0x7fb97c004d40 2024-04-26T21:12:12.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c002bb10 con 0x7f76b0003b40 2024-04-26T21:12:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c036600 con 0x7fb97c004d40 2024-04-26T21:12:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f76c5ffb640 1 -- 172.21.15.40:0/1515384840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0007c10 con 0x7f76b0003b40 2024-04-26T21:12:12.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b0003b40 msgr2=0x7f76b0003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7f76cd8d1640 1 --2- 172.21.15.40:0/1515384840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76b0003b40 0x7f76b0003f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c490 tx=0x7f76b0005830 comp rx=0 tx=0).stop 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f8987989640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89801649c0 con 0x7f89800b5bb0 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f8987989640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f89800ae170 con 0x7f89801635d0 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.786+0000 7f8987989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89800b48b0 con 0x7f89800b3f60 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57206/0 (socket says 172.21.15.40:57206) 2024-04-26T21:12:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000a4a0 con 0x7fcf20004800 2024-04-26T21:12:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30033070 con 0x7fcf20004800 2024-04-26T21:12:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7fcf35d9b640 1 -- 172.21.15.40:0/1731410775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf300386b0 con 0x7fcf20004800 2024-04-26T21:12:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 -- 172.21.15.40:0/3203908079 learned_addr learned my addr 172.21.15.40:0/3203908079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f76c77fe640 1 -- 172.21.15.40:0/1515384840 reap_dead start 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 shutdown_connections 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 >> 172.21.15.40:0/1515384840 conn(0x7f76c80b2ae0 msgr2=0x7f76c80b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7fb9950be640 1 -- 172.21.15.40:0/820256364 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0369b0 con 0x7fb97c004d40 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fcf3c81c640 1 -- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 msgr2=0x7fcf20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000a460 tx=0x7fcf20005830 comp rx=0 tx=0).stop 2024-04-26T21:12:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8985725640 1 --2- 172.21.15.40:0/3203908079 >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 0x7f8980072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 -- 172.21.15.40:0/3203908079 >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 msgr2=0x7f8980072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 --2- 172.21.15.40:0/3203908079 >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 0x7f8980072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7fcf3c81c640 1 -- 172.21.15.40:0/1731410775 shutdown_connections 2024-04-26T21:12:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/1731410775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf20004800 0x7fcf20004c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7fcf3c81c640 1 -- 172.21.15.40:0/1731410775 >> 172.21.15.40:0/1731410775 conn(0x7fcf3806c2a0 msgr2=0x7fcf3806c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7fcf3c81c640 1 -- 172.21.15.40:0/1731410775 shutdown_connections 2024-04-26T21:12:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7fcf3c81c640 1 -- 172.21.15.40:0/1731410775 wait complete. 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 -- 172.21.15.40:0/3203908079 >> v1:172.21.15.40:6789/0 conn(0x7f89800b5bb0 legacy=0x7f8980167110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.787+0000 7f8984f24640 1 -- 172.21.15.40:0/3203908079 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89800b4b90 con 0x7f89800b3f60 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.788+0000 7f8985725640 1 --2- 172.21.15.40:0/3203908079 >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 0x7f8980072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f8984f24640 1 --2- 172.21.15.40:0/3203908079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8978009e50 tx=0x7f897802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0d394460220052 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f8977fff640 1 -- 172.21.15.40:0/3203908079 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978036070 con 0x7f89800b3f60 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f8977fff640 1 -- 172.21.15.40:0/3203908079 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8978031600 con 0x7f89800b3f60 2024-04-26T21:12:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 -- 172.21.15.40:0/3203908079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 msgr2=0x7f89800b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 --2- 172.21.15.40:0/3203908079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8978009e50 tx=0x7f897802a6d0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 shutdown_connections 2024-04-26T21:12:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1515384840 wait complete. 2024-04-26T21:12:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.791+0000 7f76cd8d1640 1 Processor -- start 2024-04-26T21:12:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 -- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb97c004d40 msgr2=0x7fb97c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 --2- 172.21.15.40:0/820256364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb97c004d40 0x7fb97c0051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb99006dbc0 tx=0x7fb97c005830 comp rx=0 tx=0).stop 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb9968c1640 1 -- 172.21.15.40:0/820256364 reap_dead start 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 -- 172.21.15.40:0/820256364 shutdown_connections 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 -- 172.21.15.40:0/820256364 >> 172.21.15.40:0/820256364 conn(0x7fb9900b68a0 msgr2=0x7fb990073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 -- 172.21.15.40:0/820256364 shutdown_connections 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.789+0000 7fb998324640 1 -- 172.21.15.40:0/820256364 wait complete. 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.791+0000 7fb998324640 1 Processor -- start 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.791+0000 7fb998324640 1 -- start start 2024-04-26T21:12:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.791+0000 7f76cd8d1640 1 -- start start 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.792+0000 7fcf3c81c640 1 Processor -- start 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.792+0000 7fcf3c81c640 1 -- start start 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 -- 172.21.15.40:0/3203908079 shutdown_connections 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 --2- 172.21.15.40:0/3203908079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b3f60 0x7f89800b5670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 --2- 172.21.15.40:0/3203908079 >> v2:172.21.15.40:3300/0 conn(0x7f89801635d0 0x7f8980072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.790+0000 7f8987989640 1 -- 172.21.15.40:0/3203908079 >> 172.21.15.40:0/3203908079 conn(0x7f89800b2990 msgr2=0x7f89800b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.792+0000 7fb998324640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb998324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb990152cd0 0x7fb9901542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb998324640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 0x7fb99006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb998324640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900cd6c0 con 0x7fb97c004d40 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb998324640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b2440 con 0x7fb99014e8a0 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb998324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900cd840 con 0x7fb990152cd0 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb9960c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb9960c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40540/0 (socket says 172.21.15.40:40540) 2024-04-26T21:12:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fb9960c0640 1 -- 172.21.15.40:0/4061080377 learned_addr learned my addr 172.21.15.40:0/4061080377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.791+0000 7f8987989640 1 -- 172.21.15.40:0/3203908079 shutdown_connections 2024-04-26T21:12:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf20004800 0x7fcf38168480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.792+0000 7f8987989640 1 -- 172.21.15.40:0/3203908079 wait complete. 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf38204f60 0x7fcf38165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380b4290 con 0x7fcf38202810 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380b4110 con 0x7fcf20004800 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.793+0000 7fcf3c81c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf380b4410 con 0x7fcf38204f60 2024-04-26T21:12:12.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fb9958bf640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb990152cd0 0x7fb9901542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40542/0 (socket says 172.21.15.40:40542) 2024-04-26T21:12:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 -- 172.21.15.40:0/2018149219 learned_addr learned my addr 172.21.15.40:0/2018149219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf36d9d640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf20004800 0x7fcf38168480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fb9968c1640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 0x7fb99006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7f8987989640 1 Processor -- start 2024-04-26T21:12:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7f8987989640 1 -- start start 2024-04-26T21:12:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7fb9968c1640 1 -- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb990152cd0 msgr2=0x7fb9901542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7fb9968c1640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb990152cd0 0x7fb9901542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7fb9968c1640 1 -- 172.21.15.40:0/4061080377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 msgr2=0x7fb99009aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 -- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf38204f60 msgr2=0x7fcf38165260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:12:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf38204f60 0x7fcf38165260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 -- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf20004800 msgr2=0x7fcf38168480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf20004800 0x7fcf38168480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.794+0000 7fcf3659c640 1 -- 172.21.15.40:0/2018149219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3816c750 con 0x7fcf38202810 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7fcf3659c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000a940 tx=0x7fcf3816d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b5bb0 0x7f898009e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89801635d0 0x7f89801446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89800726d0 con 0x7f89800b3f60 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89800b5150 con 0x7f89801635d0 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.795+0000 7f8987989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89800b6060 con 0x7f89800b5bb0 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c8164d40 0x7f76c81651c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76c8201f40 0x7f76c82023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f76c806f2e0 con 0x7f76c8201f40 2024-04-26T21:12:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80cded0 con 0x7f76c815f5c0 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76cd8d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f76c806f460 con 0x7f76c8164d40 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985f26640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89801635d0 0x7f89801446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48596/0 (socket says 172.21.15.40:48596) 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1597807174 learned_addr learned my addr 172.21.15.40:0/1597807174 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c67fc640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c8164d40 0x7f76c81651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40546/0 (socket says 172.21.15.40:40546) 2024-04-26T21:12:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 -- 172.21.15.40:0/2156946386 learned_addr learned my addr 172.21.15.40:0/2156946386 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c8164d40 msgr2=0x7f76c81651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c8164d40 0x7f76c81651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1597807174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76c8201f40 msgr2=0x7f76c82023f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:12:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76c8201f40 0x7f76c82023f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f76c6ffd640 1 -- 172.21.15.40:0/1597807174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8202c70 con 0x7f76c815f5c0 2024-04-26T21:12:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 -- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b5bb0 msgr2=0x7f898009e390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:12:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b5bb0 0x7f898009e390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76c6ffd640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76c002b600 tx=0x7f76c0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 -- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89801635d0 msgr2=0x7f89801446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89801635d0 0x7f89801446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.796+0000 7f8985725640 1 -- 172.21.15.40:0/2156946386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8980158dd0 con 0x7f89800b3f60 2024-04-26T21:12:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7fb9968c1640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7fb9968c1640 1 -- 172.21.15.40:0/4061080377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990153240 con 0x7fb99014e8a0 2024-04-26T21:12:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0043020 con 0x7f76c815f5c0 2024-04-26T21:12:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c003e470 con 0x7f76c815f5c0 2024-04-26T21:12:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0038040 con 0x7f76c815f5c0 2024-04-26T21:12:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7fb9960c0640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1597807174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c815fee0 con 0x7f76c815f5c0 2024-04-26T21:12:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1597807174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c8141fe0 con 0x7f76c815f5c0 2024-04-26T21:12:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.797+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30048070 con 0x7fcf38202810 2024-04-26T21:12:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30004d60 con 0x7fcf38202810 2024-04-26T21:12:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30043670 con 0x7fcf38202810 2024-04-26T21:12:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7f8985725640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f897c01f8b0 tx=0x7f897c01fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf38168bc0 con 0x7fcf38202810 2024-04-26T21:12:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3816a2e0 con 0x7fcf38202810 2024-04-26T21:12:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c028070 con 0x7f89800b3f60 2024-04-26T21:12:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.798+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c01db50 con 0x7f89800b3f60 2024-04-26T21:12:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.799+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1597807174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c806a290 con 0x7f76c815f5c0 2024-04-26T21:12:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.800+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76c0046020 con 0x7f76c815f5c0 2024-04-26T21:12:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.799+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c009e60 con 0x7f89800b3f60 2024-04-26T21:12:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.799+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801590b0 con 0x7f89800b3f60 2024-04-26T21:12:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.800+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8980144be0 con 0x7f89800b3f60 2024-04-26T21:12:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.800+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3809f0f0 con 0x7fcf38202810 2024-04-26T21:12:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.800+0000 7f76affff640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 0x7f76b406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.800+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f76c00ad370 con 0x7f76c815f5c0 2024-04-26T21:12:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.801+0000 7f76c67fc640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 0x7f76b406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.801+0000 7fb9968c1640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 0x7fb99006f900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00ba40 tx=0x7fb98c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.801+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898006a290 con 0x7f89800b3f60 2024-04-26T21:12:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.801+0000 7f76c67fc640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 0x7f76b406ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f76c8148630 tx=0x7f76bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.802+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf30038470 con 0x7fcf38202810 2024-04-26T21:12:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.803+0000 7fcf1f7fe640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 0x7fcf2406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.803+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f897c02b070 con 0x7f89800b3f60 2024-04-26T21:12:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.803+0000 7fcf36d9d640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 0x7fcf2406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.803+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf300b09b0 con 0x7fcf38202810 2024-04-26T21:12:12.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.804+0000 7f8975ffb640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 0x7f896406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.804+0000 7fcf36d9d640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 0x7fcf2406ee80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf3806b290 tx=0x7fcf2800a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.804+0000 7f8984f24640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 0x7f896406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7f8984f24640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 0x7f896406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8978033000 tx=0x7f8978009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76c004d030 con 0x7f76c815f5c0 2024-04-26T21:12:12.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c009c80 con 0x7fb99014e8a0 2024-04-26T21:12:12.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c036400 con 0x7fb99014e8a0 2024-04-26T21:12:12.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c038500 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7fb998324640 1 -- 172.21.15.40:0/4061080377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb990153520 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.805+0000 7fb998324640 1 -- 172.21.15.40:0/4061080377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb990153ae0 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.807+0000 7fb998324640 1 -- 172.21.15.40:0/4061080377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9900693d0 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.807+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb98c0365b0 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.808+0000 7fb986ffd640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 0x7fb9740731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.808+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb98c0ac1b0 con 0x7fb99014e8a0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.809+0000 7fb9960c0640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 0x7fb9740731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.809+0000 7fb9960c0640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 0x7fb9740731b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb980005c90 tx=0x7fb980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.810+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf30051020 con 0x7fcf38202810 2024-04-26T21:12:12.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.810+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f897c068ed0 con 0x7f89800b3f60 2024-04-26T21:12:12.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.813+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb98c032030 con 0x7fb99014e8a0 2024-04-26T21:12:12.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.820+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f897c06ae30 con 0x7f89800b3f60 2024-04-26T21:12:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.835+0000 7effb0f82640 1 Processor -- start 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.835+0000 7effb0f82640 1 -- start start 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effa40e1210 0x7effa4036130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa4051da0 0x7effa4038750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effa4052200 0x7effa4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7effa40545f0 con 0x7effa40e1210 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7effa4039320 con 0x7effa4052200 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.836+0000 7effb0f82640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7effa4039600 con 0x7effa4051da0 2024-04-26T21:12:12.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effa40e1210 0x7effa4036130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effa40e1210 0x7effa4036130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40558/0 (socket says 172.21.15.40:40558) 2024-04-26T21:12:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 -- 172.21.15.40:0/4007033928 learned_addr learned my addr 172.21.15.40:0/4007033928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effaaffd640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa4051da0 0x7effa4038750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effabfff640 1 --2- 172.21.15.40:0/4007033928 >> v2:172.21.15.184:3300/0 conn(0x7effa4052200 0x7effa4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7fa0c0b04640 1 Processor -- start 2024-04-26T21:12:12.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7fa0c0b04640 1 -- start start 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 -- 172.21.15.40:0/4007033928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa4051da0 msgr2=0x7effa4038750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa4051da0 0x7effa4038750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 -- 172.21.15.40:0/4007033928 >> v2:172.21.15.184:3300/0 conn(0x7effa4052200 msgr2=0x7effa4038c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 --2- 172.21.15.40:0/4007033928 >> v2:172.21.15.184:3300/0 conn(0x7effa4052200 0x7effa4038c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.837+0000 7effab7fe640 1 -- 172.21.15.40:0/4007033928 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa40515c0 con 0x7effa40e1210 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.838+0000 7effab7fe640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa40e1210 0x7effa4036130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7effac050be0 tx=0x7effac0a6f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9554a458fa937172 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7effaa7fc640 1 -- 172.21.15.40:0/4007033928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b4070 con 0x7effa40e1210 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7effaa7fc640 1 -- 172.21.15.40:0/4007033928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0a7d90 con 0x7effa40e1210 2024-04-26T21:12:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7effaa7fc640 1 -- 172.21.15.40:0/4007033928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0af640 con 0x7effa40e1210 2024-04-26T21:12:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0c0b04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0b4053c70 0x7fa0b4038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0c0b04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40544c0 con 0x7fa0b40501a0 2024-04-26T21:12:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0c0b04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40541e0 con 0x7fa0b40df220 2024-04-26T21:12:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0c0b04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b4005f30 con 0x7fa0b4053c70 2024-04-26T21:12:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0bad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0b4053c70 0x7fa0b4038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 -- 172.21.15.40:0/4007033928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa40e1210 msgr2=0x7effa4036130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa40e1210 0x7effa4036130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7effac050be0 tx=0x7effac0a6f80 comp rx=0 tx=0).stop 2024-04-26T21:12:12.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0bad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0b4053c70 0x7fa0b4038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40568/0 (socket says 172.21.15.40:40568) 2024-04-26T21:12:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0bad9d640 1 -- 172.21.15.40:0/356531790 learned_addr learned my addr 172.21.15.40:0/356531790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.839+0000 7fa0b9d9b640 1 --1- 172.21.15.40:0/356531790 >> v1:172.21.15.40:6789/0 conn(0x7fa0b40501a0 0x7fa0b4006a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58556/0 (socket says 172.21.15.40:58556) 2024-04-26T21:12:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7fa0bad9d640 1 -- 172.21.15.40:0/356531790 >> v1:172.21.15.184:6789/0 conn(0x7fa0b40df220 legacy=0x7fa0b404fa90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:12:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7fa0bad9d640 1 -- 172.21.15.40:0/356531790 >> v1:172.21.15.40:6789/0 conn(0x7fa0b40501a0 legacy=0x7fa0b4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7fa0bad9d640 1 -- 172.21.15.40:0/356531790 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b4006210 con 0x7fa0b4053c70 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 -- 172.21.15.40:0/4007033928 shutdown_connections 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 --2- 172.21.15.40:0/4007033928 >> v2:172.21.15.184:3300/0 conn(0x7effa4052200 0x7effa4038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa4051da0 0x7effa4038750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 --2- 172.21.15.40:0/4007033928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa40e1210 0x7effa4036130 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.840+0000 7effb0f82640 1 -- 172.21.15.40:0/4007033928 >> 172.21.15.40:0/4007033928 conn(0x7effa4053750 msgr2=0x7effa4053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7f970359e640 1 Processor -- start 2024-04-26T21:12:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7effb0f82640 1 -- 172.21.15.40:0/4007033928 shutdown_connections 2024-04-26T21:12:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7f970359e640 1 -- start start 2024-04-26T21:12:12.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7fa0bad9d640 1 --2- 172.21.15.40:0/356531790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b4038080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc050be0 tx=0x7fa0bc0ab090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1623be8cb0d7941f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7effb0f82640 1 -- 172.21.15.40:0/4007033928 wait complete. 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7fa0b959a640 1 -- 172.21.15.40:0/356531790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0b1070 con 0x7fa0b4053c70 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7f970359e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f970406e960 0x7f97040ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7f970359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97040d1040 0x7f97040d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7f970359e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970406edc0 con 0x7f970415bda0 2024-04-26T21:12:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7f970359e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f97040d19b0 con 0x7f970406e960 2024-04-26T21:12:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7f970359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f970406f060 con 0x7f97040d1040 2024-04-26T21:12:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7effb0f82640 1 Processor -- start 2024-04-26T21:12:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7fa0b959a640 1 -- 172.21.15.40:0/356531790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0b5400 con 0x7fa0b4053c70 2024-04-26T21:12:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.841+0000 7fa0b959a640 1 -- 172.21.15.40:0/356531790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0b5870 con 0x7fa0b4053c70 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7fa0c0b04640 1 -- 172.21.15.40:0/356531790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 msgr2=0x7fa0b4038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.842+0000 7fa0c0b04640 1 --2- 172.21.15.40:0/356531790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b4038080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc050be0 tx=0x7fa0bc0ab090 comp rx=0 tx=0).stop 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7fa0c0b04640 1 -- 172.21.15.40:0/356531790 shutdown_connections 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7fa0c0b04640 1 --2- 172.21.15.40:0/356531790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b4038080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7fa0c0b04640 1 -- 172.21.15.40:0/356531790 >> 172.21.15.40:0/356531790 conn(0x7fa0b4009fb0 msgr2=0x7fa0b400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7f970259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f970415bda0 0x7f9704160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58566/0 (socket says 172.21.15.40:58566) 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7f970259c640 1 -- 172.21.15.40:0/920429584 learned_addr learned my addr 172.21.15.40:0/920429584 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7f9702d9d640 1 --2- 172.21.15.40:0/920429584 >> v2:172.21.15.184:3300/0 conn(0x7f97040d1040 0x7f97040d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7f9701d9b640 1 --2- 172.21.15.40:0/920429584 >> v2:172.21.15.40:3300/0 conn(0x7f970406e960 0x7f97040ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7effb0f82640 1 -- start start 2024-04-26T21:12:12.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7fa0c0b04640 1 -- 172.21.15.40:0/356531790 shutdown_connections 2024-04-26T21:12:12.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.844+0000 7f9701d9b640 1 -- 172.21.15.40:0/920429584 >> v2:172.21.15.184:3300/0 conn(0x7f97040d1040 msgr2=0x7f97040d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:12.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.844+0000 7f9701d9b640 1 --2- 172.21.15.40:0/920429584 >> v2:172.21.15.184:3300/0 conn(0x7f97040d1040 0x7f97040d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.844+0000 7f9701d9b640 1 -- 172.21.15.40:0/920429584 >> v1:172.21.15.40:6789/0 conn(0x7f970415bda0 legacy=0x7f9704160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.844+0000 7f9701d9b640 1 -- 172.21.15.40:0/920429584 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970406f340 con 0x7f970406e960 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7effb0f82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7effab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7effab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40586/0 (socket says 172.21.15.40:40586) 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7effab7fe640 1 -- 172.21.15.40:0/1935335953 learned_addr learned my addr 172.21.15.40:0/1935335953 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4052200 0x7effa40cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7f9701d9b640 1 --2- 172.21.15.40:0/920429584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406e960 0x7f97040ae6c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97040b4240 tx=0x7f96fc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8924369f2ca2947 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40e1210 0x7effa40d87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effaaffd640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4052200 0x7effa40cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7f970159a640 1 -- 172.21.15.40:0/920429584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc007740 con 0x7f970406e960 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7effa40564f0 con 0x7effa4051da0 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effa4054270 con 0x7effa4052200 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7effa4056670 con 0x7effa40e1210 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7effabfff640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40e1210 0x7effa40d87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7f970159a640 1 -- 172.21.15.40:0/920429584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc0078e0 con 0x7f970406e960 2024-04-26T21:12:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970159a640 1 -- 172.21.15.40:0/920429584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc007740 con 0x7f970406e960 2024-04-26T21:12:12.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 -- 172.21.15.40:0/920429584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406e960 msgr2=0x7f97040ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 --2- 172.21.15.40:0/920429584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406e960 0x7f97040ae6c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97040b4240 tx=0x7f96fc02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.843+0000 7fa0c0b04640 1 -- 172.21.15.40:0/356531790 wait complete. 2024-04-26T21:12:12.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.845+0000 7fa0c0b04640 1 Processor -- start 2024-04-26T21:12:12.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 -- 172.21.15.40:0/920429584 shutdown_connections 2024-04-26T21:12:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 --2- 172.21.15.40:0/920429584 >> v2:172.21.15.184:3300/0 conn(0x7f97040d1040 0x7f97040d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 --2- 172.21.15.40:0/920429584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406e960 0x7f97040ae6c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f970359e640 1 -- 172.21.15.40:0/920429584 >> 172.21.15.40:0/920429584 conn(0x7f970406c6c0 msgr2=0x7f970406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.846+0000 7fa0c0b04640 1 -- start start 2024-04-26T21:12:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 0x7fa0b404c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7effab7fe640 1 -- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40e1210 msgr2=0x7effa40d87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7effab7fe640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40e1210 0x7effa40d87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b40d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4037a90 con 0x7fa0b4053c70 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4037910 con 0x7fa0b40501a0 2024-04-26T21:12:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0c0b04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b40df600 con 0x7fa0b40df220 2024-04-26T21:12:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7effab7fe640 1 -- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4052200 msgr2=0x7effa40cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7f4037faf640 1 Processor -- start 2024-04-26T21:12:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7f4037faf640 1 -- start start 2024-04-26T21:12:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7f970359e640 1 -- 172.21.15.40:0/920429584 shutdown_connections 2024-04-26T21:12:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0bad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0bad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57266/0 (socket says 172.21.15.40:57266) 2024-04-26T21:12:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7effab7fe640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4052200 0x7effa40cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7effab7fe640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa403b320 con 0x7effa4051da0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7effab7fe640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effac063ef0 tx=0x7effac06a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0bf020 con 0x7effa4051da0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0ae650 con 0x7effa4051da0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b4040 con 0x7effa4051da0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa40d1880 con 0x7effa4051da0 2024-04-26T21:12:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7f970359e640 1 -- 172.21.15.40:0/920429584 wait complete. 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7f4037faf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f403014d180 0x7f403006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7f4037faf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403006ff90 con 0x7f40301672f0 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7f4037faf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4030070270 con 0x7f4030072fe0 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7f4037faf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f40300b4130 con 0x7f403014d180 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.849+0000 7f4035d4b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f403014d180 0x7f403006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f4035d4b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f403014d180 0x7f403006f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40610/0 (socket says 172.21.15.40:40610) 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f4035d4b640 1 -- 172.21.15.40:0/1762977421 learned_addr learned my addr 172.21.15.40:0/1762977421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f970359e640 1 Processor -- start 2024-04-26T21:12:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f970359e640 1 -- start start 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f4034d49640 1 -- 172.21.15.40:0/1762977421 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369677451 0 0) 0x7f4030070270 con 0x7f4030072fe0 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.850+0000 7f4034d49640 1 -- 172.21.15.40:0/1762977421 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f401c003520 con 0x7f4030072fe0 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f4035d4b640 1 -- 172.21.15.40:0/1762977421 >> v1:172.21.15.184:6789/0 conn(0x7f4030072fe0 legacy=0x7f403006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f4035d4b640 1 -- 172.21.15.40:0/1762977421 >> v1:172.21.15.40:6789/0 conn(0x7f40301672f0 legacy=0x7f40300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f4035d4b640 1 -- 172.21.15.40:0/1762977421 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030070550 con 0x7f403014d180 2024-04-26T21:12:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 0x7f97042028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97040d1040 0x7f970414d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0bad9d640 1 -- 172.21.15.40:0/1325499401 learned_addr learned my addr 172.21.15.40:0/1325499401 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0b9d9b640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b40d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0bad9d640 1 -- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 msgr2=0x7fa0b404c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.847+0000 7fa0ba59c640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 0x7fa0b404c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0bad9d640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 0x7fa0b404c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0bad9d640 1 -- 172.21.15.40:0/1325499401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 msgr2=0x7fa0b40d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0bad9d640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b40d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0bad9d640 1 -- 172.21.15.40:0/1325499401 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b404cdf0 con 0x7fa0b40df220 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0b9d9b640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b40d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0ba59c640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 0x7fa0b404c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0bad9d640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc06aae0 tx=0x7fa0bc06a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0c3070 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0b4b50 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0be480 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0c0b04640 1 -- 172.21.15.40:0/1325499401 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b404d0d0 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.848+0000 7fa0c0b04640 1 -- 172.21.15.40:0/1325499401 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b404d690 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0bc0c6070 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7fa0aaffd640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 0x7fa09c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa0bc1287b0 con 0x7fa0b40df220 2024-04-26T21:12:12.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b40335a0 con 0x7fa0b40df220 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f970406f900 con 0x7f970415bda0 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f970415c180 con 0x7f970406e960 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.851+0000 7f970359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97040aec00 con 0x7f97040d1040 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40616/0 (socket says 172.21.15.40:40616) 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 -- 172.21.15.40:0/3956799759 learned_addr learned my addr 172.21.15.40:0/3956799759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f970259c640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 0x7f97042028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 -- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97040d1040 msgr2=0x7f970414d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97040d1040 0x7f970414d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 -- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 msgr2=0x7f97042028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 0x7f97042028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7f9702d9d640 1 -- 172.21.15.40:0/3956799759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704202df0 con 0x7f970415bda0 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f970259c640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 0x7f97042028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f4035d4b640 1 --2- 172.21.15.40:0/1762977421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403014d180 0x7f403006f820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f402002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bddfe3c67626fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f9702d9d640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96f800d580 tx=0x7f96f800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.852+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa40d1e40 con 0x7effa4051da0 2024-04-26T21:12:12.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80125f0 con 0x7f970415bda0 2024-04-26T21:12:12.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f4034d49640 1 -- 172.21.15.40:0/1762977421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020037070 con 0x7f403014d180 2024-04-26T21:12:12.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7f4034d49640 1 -- 172.21.15.40:0/1762977421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4020032440 con 0x7f403014d180 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f8019070 con 0x7f970415bda0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970415c8b0 con 0x7f970415bda0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80129f0 con 0x7f970415bda0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.853+0000 7fa0ba59c640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 0x7fa09c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effac0c2020 con 0x7effa4051da0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa40057e0 con 0x7effa4051da0 2024-04-26T21:12:12.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7eff97fff640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 0x7eff9006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7f4037faf640 1 -- 172.21.15.40:0/1762977421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403014d180 msgr2=0x7f403006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7f4037faf640 1 --2- 172.21.15.40:0/1762977421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403014d180 0x7f403006f820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f402002a230 comp rx=0 tx=0).stop 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.854+0000 7f4034d49640 1 -- 172.21.15.40:0/1762977421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020032810 con 0x7f403014d180 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7f4037faf640 1 -- 172.21.15.40:0/1762977421 shutdown_connections 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7f4037faf640 1 --2- 172.21.15.40:0/1762977421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f403014d180 0x7f403006f820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7f4037faf640 1 -- 172.21.15.40:0/1762977421 >> 172.21.15.40:0/1762977421 conn(0x7f40300b2ae0 msgr2=0x7f40300b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:12.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970415ce70 con 0x7f970415bda0 2024-04-26T21:12:12.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7effac0f4d00 con 0x7effa4051da0 2024-04-26T21:12:12.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7fa0ba59c640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 0x7fa09c06ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a810 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.855+0000 7effaaffd640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 0x7eff9006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.856+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96f8012b90 con 0x7f970415bda0 2024-04-26T21:12:12.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.856+0000 7f4037faf640 1 -- 172.21.15.40:0/1762977421 shutdown_connections 2024-04-26T21:12:12.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.857+0000 7f96f2ffd640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 0x7f96e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.857+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f96f808bfa0 con 0x7f970415bda0 2024-04-26T21:12:12.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.857+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970406a290 con 0x7f970415bda0 2024-04-26T21:12:12.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.857+0000 7effaaffd640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 0x7eff9006ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effa40d9650 tx=0x7effa0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.857+0000 7f970259c640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 0x7f96e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.856+0000 7f4037faf640 1 -- 172.21.15.40:0/1762977421 wait complete. 2024-04-26T21:12:12.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.858+0000 7f4037faf640 1 Processor -- start 2024-04-26T21:12:12.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.858+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0b40335a0 con 0x7fa0b40df220 2024-04-26T21:12:12.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.858+0000 7f4037faf640 1 -- start start 2024-04-26T21:12:12.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.858+0000 7f970259c640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 0x7f96e406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96f4005880 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.859+0000 7f4037faf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 0x7f4030162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.859+0000 7f4037faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.859+0000 7f4037faf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40301672f0 0x7f403015f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.859+0000 7f403554a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.859+0000 7f403554a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57292/0 (socket says 172.21.15.40:57292) 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 -- 172.21.15.40:0/1944793826 learned_addr learned my addr 172.21.15.40:0/1944793826 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f4035d4b640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 0x7f4030162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f4037faf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f403006f2e0 con 0x7f4030072fe0 2024-04-26T21:12:12.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40300cded0 con 0x7f40301672f0 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4030163c00 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403654c640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40301672f0 0x7f403015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 -- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40301672f0 msgr2=0x7f403015f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40301672f0 0x7f403015f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 -- 172.21.15.40:0/1944793826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 msgr2=0x7f4030162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 0x7f4030162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.860+0000 7f403554a640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030147cd0 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f403554a640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f402c00cb20 tx=0x7f402c002070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f4035d4b640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 0x7f4030162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c017070 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c003da0 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4030162de0 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00a900 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.861+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4030164130 con 0x7f403014d180 2024-04-26T21:12:12.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.863+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403006a290 con 0x7f403014d180 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.863+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f402c01a070 con 0x7f403014d180 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.864+0000 7f40267fc640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.864+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f402c08b310 con 0x7f403014d180 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.864+0000 7f4035d4b640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.865+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7effac0f6c60 con 0x7effa4051da0 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.869+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f402c059050 con 0x7f403014d180 2024-04-26T21:12:12.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.873+0000 7f4035d4b640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 0x7f401406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:12.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:12.875+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96f805a340 con 0x7f970415bda0 2024-04-26T21:12:13.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.224+0000 7f76cd8d1640 1 -- 172.21.15.40:0/1597807174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f76c8070990 con 0x7f76c815f5c0 2024-04-26T21:12:13.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.225+0000 7f76affff640 1 -- 172.21.15.40:0/1597807174 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f76c007b930 con 0x7f76c815f5c0 2024-04-26T21:12:13.226 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.228+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 msgr2=0x7f76b406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.228+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 0x7f76b406ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f76c8148630 tx=0x7f76bc009040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.228+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 msgr2=0x7f76c815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.228+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76c002b600 tx=0x7f76c0004ab0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 shutdown_connections 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76b406c940 0x7f76b406ee00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76c8201f40 0x7f76c82023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76c8164d40 0x7f76c81651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 --2- 172.21.15.40:0/1597807174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76c815f5c0 0x7f76c815f9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 >> 172.21.15.40:0/1597807174 conn(0x7f76c80b2ae0 msgr2=0x7f76c80b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.229+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 shutdown_connections 2024-04-26T21:12:13.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.230+0000 7f76adffb640 1 -- 172.21.15.40:0/1597807174 wait complete. 2024-04-26T21:12:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.245+0000 7fb998324640 1 -- 172.21.15.40:0/4061080377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb99006d560 con 0x7fb99014e8a0 2024-04-26T21:12:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.246+0000 7fb986ffd640 1 -- 172.21.15.40:0/4061080377 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb98c07a5c0 con 0x7fb99014e8a0 2024-04-26T21:12:13.247 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:13.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-04-26T21:12:13.248 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f89800ae800 con 0x7f89800b3f60 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 msgr2=0x7fb9740731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 0x7fb9740731b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb980005c90 tx=0x7fb980009040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 msgr2=0x7fb99006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 0x7fb99006f900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00ba40 tx=0x7fb98c004ab0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 shutdown_connections 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb974070cf0 0x7fb9740731b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb99014e8a0 0x7fb99006f900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb990152cd0 0x7fb9901542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 --2- 172.21.15.40:0/4061080377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb97c004d40 0x7fb99009aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.249+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 >> 172.21.15.40:0/4061080377 conn(0x7fb9900b68a0 msgr2=0x7fb990158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.250+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 shutdown_connections 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.250+0000 7fb984ff9640 1 -- 172.21.15.40:0/4061080377 wait complete. 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.250+0000 7f8975ffb640 1 -- 172.21.15.40:0/2156946386 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f897c06f130 con 0x7f89800b3f60 2024-04-26T21:12:13.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.250+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcf380507a0 con 0x7fcf38202810 2024-04-26T21:12:13.254 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:13.255 INFO:teuthology.orchestra.run.smithi040.stdout:64424509448 2024-04-26T21:12:13.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.251+0000 7fcf1f7fe640 1 -- 172.21.15.40:0/2018149219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf3007e6e0 con 0x7fcf38202810 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.253+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 msgr2=0x7fcf2406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.253+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 0x7fcf2406ee80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf3806b290 tx=0x7fcf2800a040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.253+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 msgr2=0x7fcf3816c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.253+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000a940 tx=0x7fcf3816d170 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 shutdown_connections 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c9c0 0x7fcf2406ee80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf38204f60 0x7fcf38165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf38202810 0x7fcf3816c210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 --2- 172.21.15.40:0/2018149219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf20004800 0x7fcf38168480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 >> 172.21.15.40:0/2018149219 conn(0x7fcf3806c2a0 msgr2=0x7fcf380cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 shutdown_connections 2024-04-26T21:12:13.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7fcf3c81c640 1 -- 172.21.15.40:0/2018149219 wait complete. 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 msgr2=0x7f896406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 0x7f896406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8978033000 tx=0x7f8978009000 comp rx=0 tx=0).stop 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 msgr2=0x7f8980156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.254+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f897c01f8b0 tx=0x7f897c01fd70 comp rx=0 tx=0).stop 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 shutdown_connections 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c940 0x7f896406ee00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89801635d0 0x7f89801446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89800b5bb0 0x7f898009e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 --2- 172.21.15.40:0/2156946386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89800b3f60 0x7f8980156590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 >> 172.21.15.40:0/2156946386 conn(0x7f89800b2990 msgr2=0x7f89800b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 shutdown_connections 2024-04-26T21:12:13.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.255+0000 7f8987989640 1 -- 172.21.15.40:0/2156946386 wait complete. 2024-04-26T21:12:13.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-04-26T21:12:13.269 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-04-26T21:12:13.270 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-04-26T21:12:13.274 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.281+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7effa4035ec0 con 0x7effa4051da0 2024-04-26T21:12:13.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.281+0000 7eff97fff640 1 -- 172.21.15.40:0/1935335953 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effac0faf60 con 0x7effa4051da0 2024-04-26T21:12:13.282 INFO:teuthology.orchestra.run.smithi040.stdout:64424509448 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 msgr2=0x7eff9006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 0x7eff9006ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effa40d9650 tx=0x7effa0008040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 msgr2=0x7effa40d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effac063ef0 tx=0x7effac06a150 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 shutdown_connections 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff9006c940 0x7eff9006ee00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effa40e1210 0x7effa40d87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effa4052200 0x7effa40cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 --2- 172.21.15.40:0/1935335953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effa4051da0 0x7effa40d1230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.283+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 >> 172.21.15.40:0/1935335953 conn(0x7effa4053750 msgr2=0x7effa4007be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.284+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 shutdown_connections 2024-04-26T21:12:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.284+0000 7effb0f82640 1 -- 172.21.15.40:0/1935335953 wait complete. 2024-04-26T21:12:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.288+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f97040b4ba0 con 0x7f970415bda0 2024-04-26T21:12:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.288+0000 7f96f2ffd640 1 -- 172.21.15.40:0/3956799759 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96f8059c40 con 0x7f970415bda0 2024-04-26T21:12:13.289 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 msgr2=0x7f96e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 0x7f96e406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96f4005880 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 msgr2=0x7f970416ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96f800d580 tx=0x7f96f800da40 comp rx=0 tx=0).stop 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 shutdown_connections 2024-04-26T21:12:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f96e406c8f0 0x7f96e406edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970415bda0 0x7f970416ee80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97040d1040 0x7f970414d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 --2- 172.21.15.40:0/3956799759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f970406e960 0x7f97042028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 >> 172.21.15.40:0/3956799759 conn(0x7f970406c6c0 msgr2=0x7f97040b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 shutdown_connections 2024-04-26T21:12:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.292+0000 7f970359e640 1 -- 172.21.15.40:0/3956799759 wait complete. 2024-04-26T21:12:13.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-04-26T21:12:13.297 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.297+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa0b4005390 con 0x7fa0b40df220 2024-04-26T21:12:13.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.297+0000 7fa0aaffd640 1 -- 172.21.15.40:0/1325499401 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0bc091070 con 0x7fa0b40df220 2024-04-26T21:12:13.298 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2024-04-26T21:12:13.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4030070990 con 0x7f403014d180 2024-04-26T21:12:13.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7f40267fc640 1 -- 172.21.15.40:0/1944793826 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f402c05d350 con 0x7f403014d180 2024-04-26T21:12:13.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 msgr2=0x7fa09c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.301 INFO:teuthology.orchestra.run.smithi040.stdout:64424509447 2024-04-26T21:12:13.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 0x7fa09c06ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a810 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 msgr2=0x7fa0b40da3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc06aae0 tx=0x7fa0bc06a150 comp rx=0 tx=0).stop 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 shutdown_connections 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa09c06c9c0 0x7fa09c06ee80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b40df220 0x7fa0b40da3b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0b4053c70 0x7fa0b40d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 --2- 172.21.15.40:0/1325499401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b40501a0 0x7fa0b404c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 >> 172.21.15.40:0/1325499401 conn(0x7fa0b4009fb0 msgr2=0x7fa0b4034350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 shutdown_connections 2024-04-26T21:12:13.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.300+0000 7fa0a8ff9640 1 -- 172.21.15.40:0/1325499401 wait complete. 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 msgr2=0x7f401406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 0x7f401406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 msgr2=0x7f4030162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f402c00cb20 tx=0x7f402c002070 comp rx=0 tx=0).stop 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 shutdown_connections 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40301672f0 0x7f403015f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f403014d180 0x7f4030162560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 --2- 172.21.15.40:0/1944793826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4030072fe0 0x7f4030162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 >> 172.21.15.40:0/1944793826 conn(0x7f40300b2ae0 msgr2=0x7f403014c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 shutdown_connections 2024-04-26T21:12:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.303+0000 7f4037faf640 1 -- 172.21.15.40:0/1944793826 wait complete. 2024-04-26T21:12:13.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-04-26T21:12:13.306 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-04-26T21:12:13.314 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-04-26T21:12:13.315 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:13.316 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:12:13.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:13.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.455+0000 7ff3f0093640 1 Processor -- start 2024-04-26T21:12:13.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.455+0000 7ff3f0093640 1 -- start start 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3f0093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 0x7ff3e80cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3f0093640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3e8162f30 0x7ff3e806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3f0093640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8163830 con 0x7ff3e806c2a0 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3f0093640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e8163df0 con 0x7ff3e8162f30 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3f0093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e8163b10 con 0x7ff3e815fa20 2024-04-26T21:12:13.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3ee630640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3e8162f30 0x7ff3e806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3ee630640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3e8162f30 0x7ff3e806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40638/0 (socket says 172.21.15.40:40638) 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.456+0000 7ff3ee630640 1 -- 172.21.15.40:0/3567419659 learned_addr learned my addr 172.21.15.40:0/3567419659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ed62e640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 0x7ff3e80cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ee630640 1 -- 172.21.15.40:0/3567419659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 msgr2=0x7ff3e80cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ee630640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 0x7ff3e80cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ee630640 1 -- 172.21.15.40:0/3567419659 >> v1:172.21.15.40:6789/0 conn(0x7ff3e806c2a0 legacy=0x7ff3e806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ece2d640 1 -- 172.21.15.40:0/3567419659 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960814360 0 0) 0x7ff3e8163830 con 0x7ff3e806c2a0 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ee630640 1 -- 172.21.15.40:0/3567419659 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e80ce950 con 0x7ff3e8162f30 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ed62e640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 0x7ff3e80cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.457+0000 7ff3ee630640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e8162f30 0x7ff3e806fb40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57f36478f8f0e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3ece2d640 1 -- 172.21.15.40:0/3567419659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0037070 con 0x7ff3e8162f30 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3ece2d640 1 -- 172.21.15.40:0/3567419659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e0032440 con 0x7ff3e8162f30 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3ece2d640 1 -- 172.21.15.40:0/3567419659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0032810 con 0x7ff3e8162f30 2024-04-26T21:12:13.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 -- 172.21.15.40:0/3567419659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e8162f30 msgr2=0x7ff3e806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e8162f30 0x7ff3e806fb40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e002a130 comp rx=0 tx=0).stop 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 -- 172.21.15.40:0/3567419659 shutdown_connections 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e8162f30 0x7ff3e806fb40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 --2- 172.21.15.40:0/3567419659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e815fa20 0x7ff3e80cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 -- 172.21.15.40:0/3567419659 >> 172.21.15.40:0/3567419659 conn(0x7ff3e80b3b10 msgr2=0x7ff3e80b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 -- 172.21.15.40:0/3567419659 shutdown_connections 2024-04-26T21:12:13.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.458+0000 7ff3f0093640 1 -- 172.21.15.40:0/3567419659 wait complete. 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.459+0000 7ff3f0093640 1 Processor -- start 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.459+0000 7ff3f0093640 1 -- start start 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e806c2a0 0x7ff3e814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 0x7ff3e80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81606a0 con 0x7ff3e815fa20 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e80cf690 con 0x7ff3e806c2a0 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3f0093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e8160820 con 0x7ff3e8162f30 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40650/0 (socket says 172.21.15.40:40650) 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 -- 172.21.15.40:0/1473006498 learned_addr learned my addr 172.21.15.40:0/1473006498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ee630640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 0x7ff3e80b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ede2f640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e806c2a0 0x7ff3e814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 -- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 msgr2=0x7ff3e80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 0x7ff3e80b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 -- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e806c2a0 msgr2=0x7ff3e814d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.460+0000 7ff3ed62e640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e806c2a0 0x7ff3e814d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3ed62e640 1 -- 172.21.15.40:0/1473006498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e80703f0 con 0x7ff3e815fa20 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3ee630640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 0x7ff3e80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3ed62e640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c7f0 tx=0x7ff3dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc015070 con 0x7ff3e815fa20 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc007820 con 0x7ff3e815fa20 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e8159360 con 0x7ff3e815fa20 2024-04-26T21:12:13.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007bf0 con 0x7ff3e815fa20 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.461+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e8159920 con 0x7ff3e815fa20 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.462+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e80693d0 con 0x7ff3e815fa20 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.462+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3dc018070 con 0x7ff3e815fa20 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.463+0000 7ff3e67fc640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 0x7ff3d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.463+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff3dc08bc70 con 0x7ff3e815fa20 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.463+0000 7ff3ede2f640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 0x7ff3d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.464+0000 7ff3ede2f640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 0x7ff3d406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80012e0 tx=0x7ff3d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.467+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3dc05a000 con 0x7ff3e815fa20 2024-04-26T21:12:13.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.650+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3e80b11e0 con 0x7ff3d406c8f0 2024-04-26T21:12:13.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.656+0000 7ff3e67fc640 1 -- 172.21.15.40:0/1473006498 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7ff3e80b11e0 con 0x7ff3d406c8f0 2024-04-26T21:12:13.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 msgr2=0x7ff3d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 0x7ff3d406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80012e0 tx=0x7ff3d8009040 comp rx=0 tx=0).stop 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 msgr2=0x7ff3e8158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c7f0 tx=0x7ff3dc00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 shutdown_connections 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3d406c8f0 0x7ff3d406edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e8162f30 0x7ff3e80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e815fa20 0x7ff3e8158560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 --2- 172.21.15.40:0/1473006498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e806c2a0 0x7ff3e814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 >> 172.21.15.40:0/1473006498 conn(0x7ff3e80b3b10 msgr2=0x7ff3e815ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 shutdown_connections 2024-04-26T21:12:13.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.659+0000 7ff3f0093640 1 -- 172.21.15.40:0/1473006498 wait complete. 2024-04-26T21:12:13.671 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-26T21:12:12.920600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219236,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755484,"statfs":{"total":773094113280,"available":772869615616,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.154550"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075715+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:04.075715+0000","last_peered":"2024-04-26T21:12:04.075715+0000","last_clean":"2024-04-26T21:12:04.075715+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:04.075715+0000","last_undegraded":"2024-04-26T21:12:04.075715+0000","last_fullsized":"2024-04-26T21:12:04.075715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:17.992415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768372+0000","last_change":"2024-04-26T21:12:01.767311+0000","last_active":"2024-04-26T21:12:02.768372+0000","last_peered":"2024-04-26T21:12:02.768372+0000","last_clean":"2024-04-26T21:12:02.768372+0000","last_became_active":"2024-04-26T21:12:01.766955+0000","last_became_peered":"2024-04-26T21:12:01.766955+0000","last_unstale":"2024-04-26T21:12:02.768372+0000","last_undegraded":"2024-04-26T21:12:02.768372+0000","last_fullsized":"2024-04-26T21:12:02.768372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:29.829212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175853+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:04.175853+0000","last_peered":"2024-04-26T21:12:04.175853+0000","last_clean":"2024-04-26T21:12:04.175853+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:04.175853+0000","last_undegraded":"2024-04-26T21:12:04.175853+0000","last_fullsized":"2024-04-26T21:12:04.175853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175990+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:04.175990+0000","last_peered":"2024-04-26T21:12:04.175990+0000","last_clean":"2024-04-26T21:12:04.175990+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:04.175990+0000","last_undegraded":"2024-04-26T21:12:04.175990+0000","last_fullsized":"2024-04-26T21:12:04.175990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075284+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:04.075284+0000","last_peered":"2024-04-26T21:12:04.075284+0000","last_clean":"2024-04-26T21:12:04.075284+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:04.075284+0000","last_undegraded":"2024-04-26T21:12:04.075284+0000","last_fullsized":"2024-04-26T21:12:04.075284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175693+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:04.175693+0000","last_peered":"2024-04-26T21:12:04.175693+0000","last_clean":"2024-04-26T21:12:04.175693+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:04.175693+0000","last_undegraded":"2024-04-26T21:12:04.175693+0000","last_fullsized":"2024-04-26T21:12:04.175693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075067+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:04.075067+0000","last_peered":"2024-04-26T21:12:04.075067+0000","last_clean":"2024-04-26T21:12:04.075067+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:04.075067+0000","last_undegraded":"2024-04-26T21:12:04.075067+0000","last_fullsized":"2024-04-26T21:12:04.075067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075793+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:04.075793+0000","last_peered":"2024-04-26T21:12:04.075793+0000","last_clean":"2024-04-26T21:12:04.075793+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:04.075793+0000","last_undegraded":"2024-04-26T21:12:04.075793+0000","last_fullsized":"2024-04-26T21:12:04.075793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.197913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075042+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:03.075042+0000","last_peered":"2024-04-26T21:12:03.075042+0000","last_clean":"2024-04-26T21:12:03.075042+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:03.075042+0000","last_undegraded":"2024-04-26T21:12:03.075042+0000","last_fullsized":"2024-04-26T21:12:03.075042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074816+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:03.074816+0000","last_peered":"2024-04-26T21:12:03.074816+0000","last_clean":"2024-04-26T21:12:03.074816+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:03.074816+0000","last_undegraded":"2024-04-26T21:12:03.074816+0000","last_fullsized":"2024-04-26T21:12:03.074816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974567+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:02.974567+0000","last_peered":"2024-04-26T21:12:02.974567+0000","last_clean":"2024-04-26T21:12:02.974567+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:02.974567+0000","last_undegraded":"2024-04-26T21:12:02.974567+0000","last_fullsized":"2024-04-26T21:12:02.974567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175587+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:04.175587+0000","last_peered":"2024-04-26T21:12:04.175587+0000","last_clean":"2024-04-26T21:12:04.175587+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:04.175587+0000","last_undegraded":"2024-04-26T21:12:04.175587+0000","last_fullsized":"2024-04-26T21:12:04.175587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768447+0000","last_change":"2024-04-26T21:12:01.769541+0000","last_active":"2024-04-26T21:12:02.768447+0000","last_peered":"2024-04-26T21:12:02.768447+0000","last_clean":"2024-04-26T21:12:02.768447+0000","last_became_active":"2024-04-26T21:12:01.769171+0000","last_became_peered":"2024-04-26T21:12:01.769171+0000","last_unstale":"2024-04-26T21:12:02.768447+0000","last_undegraded":"2024-04-26T21:12:02.768447+0000","last_fullsized":"2024-04-26T21:12:02.768447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:22.619826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175595+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:04.175595+0000","last_peered":"2024-04-26T21:12:04.175595+0000","last_clean":"2024-04-26T21:12:04.175595+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:04.175595+0000","last_undegraded":"2024-04-26T21:12:04.175595+0000","last_fullsized":"2024-04-26T21:12:04.175595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176236+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:04.176236+0000","last_peered":"2024-04-26T21:12:04.176236+0000","last_clean":"2024-04-26T21:12:04.176236+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:04.176236+0000","last_undegraded":"2024-04-26T21:12:04.176236+0000","last_fullsized":"2024-04-26T21:12:04.176236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075698+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:04.075698+0000","last_peered":"2024-04-26T21:12:04.075698+0000","last_clean":"2024-04-26T21:12:04.075698+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:04.075698+0000","last_undegraded":"2024-04-26T21:12:04.075698+0000","last_fullsized":"2024-04-26T21:12:04.075698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:29.707431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074796+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:03.074796+0000","last_peered":"2024-04-26T21:12:03.074796+0000","last_clean":"2024-04-26T21:12:03.074796+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:03.074796+0000","last_undegraded":"2024-04-26T21:12:03.074796+0000","last_fullsized":"2024-04-26T21:12:03.074796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175801+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:04.175801+0000","last_peered":"2024-04-26T21:12:04.175801+0000","last_clean":"2024-04-26T21:12:04.175801+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:04.175801+0000","last_undegraded":"2024-04-26T21:12:04.175801+0000","last_fullsized":"2024-04-26T21:12:04.175801+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175896+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:04.175896+0000","last_peered":"2024-04-26T21:12:04.175896+0000","last_clean":"2024-04-26T21:12:04.175896+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:04.175896+0000","last_undegraded":"2024-04-26T21:12:04.175896+0000","last_fullsized":"2024-04-26T21:12:04.175896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175677+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:04.175677+0000","last_peered":"2024-04-26T21:12:04.175677+0000","last_clean":"2024-04-26T21:12:04.175677+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:04.175677+0000","last_undegraded":"2024-04-26T21:12:04.175677+0000","last_fullsized":"2024-04-26T21:12:04.175677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075061+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:03.075061+0000","last_peered":"2024-04-26T21:12:03.075061+0000","last_clean":"2024-04-26T21:12:03.075061+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:03.075061+0000","last_undegraded":"2024-04-26T21:12:03.075061+0000","last_fullsized":"2024-04-26T21:12:03.075061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768216+0000","last_change":"2024-04-26T21:12:01.757848+0000","last_active":"2024-04-26T21:12:02.768216+0000","last_peered":"2024-04-26T21:12:02.768216+0000","last_clean":"2024-04-26T21:12:02.768216+0000","last_became_active":"2024-04-26T21:12:01.757563+0000","last_became_peered":"2024-04-26T21:12:01.757563+0000","last_unstale":"2024-04-26T21:12:02.768216+0000","last_undegraded":"2024-04-26T21:12:02.768216+0000","last_fullsized":"2024-04-26T21:12:02.768216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:51.758916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175879+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:04.175879+0000","last_peered":"2024-04-26T21:12:04.175879+0000","last_clean":"2024-04-26T21:12:04.175879+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:04.175879+0000","last_undegraded":"2024-04-26T21:12:04.175879+0000","last_fullsized":"2024-04-26T21:12:04.175879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175961+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:04.175961+0000","last_peered":"2024-04-26T21:12:04.175961+0000","last_clean":"2024-04-26T21:12:04.175961+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:04.175961+0000","last_undegraded":"2024-04-26T21:12:04.175961+0000","last_fullsized":"2024-04-26T21:12:04.175961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074711+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:03.074711+0000","last_peered":"2024-04-26T21:12:03.074711+0000","last_clean":"2024-04-26T21:12:03.074711+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:03.074711+0000","last_undegraded":"2024-04-26T21:12:03.074711+0000","last_fullsized":"2024-04-26T21:12:03.074711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974321+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:02.974321+0000","last_peered":"2024-04-26T21:12:02.974321+0000","last_clean":"2024-04-26T21:12:02.974321+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:02.974321+0000","last_undegraded":"2024-04-26T21:12:02.974321+0000","last_fullsized":"2024-04-26T21:12:02.974321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175972+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:04.175972+0000","last_peered":"2024-04-26T21:12:04.175972+0000","last_clean":"2024-04-26T21:12:04.175972+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:04.175972+0000","last_undegraded":"2024-04-26T21:12:04.175972+0000","last_fullsized":"2024-04-26T21:12:04.175972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074650+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:03.074650+0000","last_peered":"2024-04-26T21:12:03.074650+0000","last_clean":"2024-04-26T21:12:03.074650+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:03.074650+0000","last_undegraded":"2024-04-26T21:12:03.074650+0000","last_fullsized":"2024-04-26T21:12:03.074650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175834+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:04.175834+0000","last_peered":"2024-04-26T21:12:04.175834+0000","last_clean":"2024-04-26T21:12:04.175834+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:04.175834+0000","last_undegraded":"2024-04-26T21:12:04.175834+0000","last_fullsized":"2024-04-26T21:12:04.175834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768245+0000","last_change":"2024-04-26T21:12:01.765248+0000","last_active":"2024-04-26T21:12:02.768245+0000","last_peered":"2024-04-26T21:12:02.768245+0000","last_clean":"2024-04-26T21:12:02.768245+0000","last_became_active":"2024-04-26T21:12:01.764951+0000","last_became_peered":"2024-04-26T21:12:01.764951+0000","last_unstale":"2024-04-26T21:12:02.768245+0000","last_undegraded":"2024-04-26T21:12:02.768245+0000","last_fullsized":"2024-04-26T21:12:02.768245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:24.962313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074890+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:03.074890+0000","last_peered":"2024-04-26T21:12:03.074890+0000","last_clean":"2024-04-26T21:12:03.074890+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:03.074890+0000","last_undegraded":"2024-04-26T21:12:03.074890+0000","last_fullsized":"2024-04-26T21:12:03.074890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176145+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:04.176145+0000","last_peered":"2024-04-26T21:12:04.176145+0000","last_clean":"2024-04-26T21:12:04.176145+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:04.176145+0000","last_undegraded":"2024-04-26T21:12:04.176145+0000","last_fullsized":"2024-04-26T21:12:04.176145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175969+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:04.175969+0000","last_peered":"2024-04-26T21:12:04.175969+0000","last_clean":"2024-04-26T21:12:04.175969+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:04.175969+0000","last_undegraded":"2024-04-26T21:12:04.175969+0000","last_fullsized":"2024-04-26T21:12:04.175969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176007+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:04.176007+0000","last_peered":"2024-04-26T21:12:04.176007+0000","last_clean":"2024-04-26T21:12:04.176007+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:04.176007+0000","last_undegraded":"2024-04-26T21:12:04.176007+0000","last_fullsized":"2024-04-26T21:12:04.176007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768176+0000","last_change":"2024-04-26T21:12:00.744708+0000","last_active":"2024-04-26T21:12:02.768176+0000","last_peered":"2024-04-26T21:12:02.768176+0000","last_clean":"2024-04-26T21:12:02.768176+0000","last_became_active":"2024-04-26T21:12:00.744396+0000","last_became_peered":"2024-04-26T21:12:00.744396+0000","last_unstale":"2024-04-26T21:12:02.768176+0000","last_undegraded":"2024-04-26T21:12:02.768176+0000","last_fullsized":"2024-04-26T21:12:02.768176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:40.330925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974569+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:02.974569+0000","last_peered":"2024-04-26T21:12:02.974569+0000","last_clean":"2024-04-26T21:12:02.974569+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:02.974569+0000","last_undegraded":"2024-04-26T21:12:02.974569+0000","last_fullsized":"2024-04-26T21:12:02.974569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074901+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:03.074901+0000","last_peered":"2024-04-26T21:12:03.074901+0000","last_clean":"2024-04-26T21:12:03.074901+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:03.074901+0000","last_undegraded":"2024-04-26T21:12:03.074901+0000","last_fullsized":"2024-04-26T21:12:03.074901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075445+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:04.075445+0000","last_peered":"2024-04-26T21:12:04.075445+0000","last_clean":"2024-04-26T21:12:04.075445+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:04.075445+0000","last_undegraded":"2024-04-26T21:12:04.075445+0000","last_fullsized":"2024-04-26T21:12:04.075445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175978+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:04.175978+0000","last_peered":"2024-04-26T21:12:04.175978+0000","last_clean":"2024-04-26T21:12:04.175978+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:04.175978+0000","last_undegraded":"2024-04-26T21:12:04.175978+0000","last_fullsized":"2024-04-26T21:12:04.175978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175604+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:04.175604+0000","last_peered":"2024-04-26T21:12:04.175604+0000","last_clean":"2024-04-26T21:12:04.175604+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:04.175604+0000","last_undegraded":"2024-04-26T21:12:04.175604+0000","last_fullsized":"2024-04-26T21:12:04.175604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176048+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:04.176048+0000","last_peered":"2024-04-26T21:12:04.176048+0000","last_clean":"2024-04-26T21:12:04.176048+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:04.176048+0000","last_undegraded":"2024-04-26T21:12:04.176048+0000","last_fullsized":"2024-04-26T21:12:04.176048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075449+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:04.075449+0000","last_peered":"2024-04-26T21:12:04.075449+0000","last_clean":"2024-04-26T21:12:04.075449+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:04.075449+0000","last_undegraded":"2024-04-26T21:12:04.075449+0000","last_fullsized":"2024-04-26T21:12:04.075449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074625+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:03.074625+0000","last_peered":"2024-04-26T21:12:03.074625+0000","last_clean":"2024-04-26T21:12:03.074625+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:03.074625+0000","last_undegraded":"2024-04-26T21:12:03.074625+0000","last_fullsized":"2024-04-26T21:12:03.074625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074691+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:03.074691+0000","last_peered":"2024-04-26T21:12:03.074691+0000","last_clean":"2024-04-26T21:12:03.074691+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:03.074691+0000","last_undegraded":"2024-04-26T21:12:03.074691+0000","last_fullsized":"2024-04-26T21:12:03.074691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075075+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:04.075075+0000","last_peered":"2024-04-26T21:12:04.075075+0000","last_clean":"2024-04-26T21:12:04.075075+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:04.075075+0000","last_undegraded":"2024-04-26T21:12:04.075075+0000","last_fullsized":"2024-04-26T21:12:04.075075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974587+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:02.974587+0000","last_peered":"2024-04-26T21:12:02.974587+0000","last_clean":"2024-04-26T21:12:02.974587+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:02.974587+0000","last_undegraded":"2024-04-26T21:12:02.974587+0000","last_fullsized":"2024-04-26T21:12:02.974587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075598+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:04.075598+0000","last_peered":"2024-04-26T21:12:04.075598+0000","last_clean":"2024-04-26T21:12:04.075598+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:04.075598+0000","last_undegraded":"2024-04-26T21:12:04.075598+0000","last_fullsized":"2024-04-26T21:12:04.075598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075831+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:04.075831+0000","last_peered":"2024-04-26T21:12:04.075831+0000","last_clean":"2024-04-26T21:12:04.075831+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:04.075831+0000","last_undegraded":"2024-04-26T21:12:04.075831+0000","last_fullsized":"2024-04-26T21:12:04.075831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:39.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768273+0000","last_change":"2024-04-26T21:12:00.742717+0000","last_active":"2024-04-26T21:12:02.768273+0000","last_peered":"2024-04-26T21:12:02.768273+0000","last_clean":"2024-04-26T21:12:02.768273+0000","last_became_active":"2024-04-26T21:12:00.741757+0000","last_became_peered":"2024-04-26T21:12:00.741757+0000","last_unstale":"2024-04-26T21:12:02.768273+0000","last_undegraded":"2024-04-26T21:12:02.768273+0000","last_fullsized":"2024-04-26T21:12:02.768273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:33.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176261+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:04.176261+0000","last_peered":"2024-04-26T21:12:04.176261+0000","last_clean":"2024-04-26T21:12:04.176261+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:04.176261+0000","last_undegraded":"2024-04-26T21:12:04.176261+0000","last_fullsized":"2024-04-26T21:12:04.176261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075479+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:04.075479+0000","last_peered":"2024-04-26T21:12:04.075479+0000","last_clean":"2024-04-26T21:12:04.075479+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:04.075479+0000","last_undegraded":"2024-04-26T21:12:04.075479+0000","last_fullsized":"2024-04-26T21:12:04.075479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075131+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:04.075131+0000","last_peered":"2024-04-26T21:12:04.075131+0000","last_clean":"2024-04-26T21:12:04.075131+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:04.075131+0000","last_undegraded":"2024-04-26T21:12:04.075131+0000","last_fullsized":"2024-04-26T21:12:04.075131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175661+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:04.175661+0000","last_peered":"2024-04-26T21:12:04.175661+0000","last_clean":"2024-04-26T21:12:04.175661+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:04.175661+0000","last_undegraded":"2024-04-26T21:12:04.175661+0000","last_fullsized":"2024-04-26T21:12:04.175661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074973+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:04.074973+0000","last_peered":"2024-04-26T21:12:04.074973+0000","last_clean":"2024-04-26T21:12:04.074973+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:04.074973+0000","last_undegraded":"2024-04-26T21:12:04.074973+0000","last_fullsized":"2024-04-26T21:12:04.074973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175510+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:04.175510+0000","last_peered":"2024-04-26T21:12:04.175510+0000","last_clean":"2024-04-26T21:12:04.175510+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:04.175510+0000","last_undegraded":"2024-04-26T21:12:04.175510+0000","last_fullsized":"2024-04-26T21:12:04.175510+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767742+0000","last_change":"2024-04-26T21:12:01.758217+0000","last_active":"2024-04-26T21:12:02.767742+0000","last_peered":"2024-04-26T21:12:02.767742+0000","last_clean":"2024-04-26T21:12:02.767742+0000","last_became_active":"2024-04-26T21:12:01.756525+0000","last_became_peered":"2024-04-26T21:12:01.756525+0000","last_unstale":"2024-04-26T21:12:02.767742+0000","last_undegraded":"2024-04-26T21:12:02.767742+0000","last_fullsized":"2024-04-26T21:12:02.767742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:17.830865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176252+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:04.176252+0000","last_peered":"2024-04-26T21:12:04.176252+0000","last_clean":"2024-04-26T21:12:04.176252+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:04.176252+0000","last_undegraded":"2024-04-26T21:12:04.176252+0000","last_fullsized":"2024-04-26T21:12:04.176252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974416+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:02.974416+0000","last_peered":"2024-04-26T21:12:02.974416+0000","last_clean":"2024-04-26T21:12:02.974416+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:02.974416+0000","last_undegraded":"2024-04-26T21:12:02.974416+0000","last_fullsized":"2024-04-26T21:12:02.974416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768484+0000","last_change":"2024-04-26T21:12:00.745395+0000","last_active":"2024-04-26T21:12:02.768484+0000","last_peered":"2024-04-26T21:12:02.768484+0000","last_clean":"2024-04-26T21:12:02.768484+0000","last_became_active":"2024-04-26T21:12:00.745088+0000","last_became_peered":"2024-04-26T21:12:00.745088+0000","last_unstale":"2024-04-26T21:12:02.768484+0000","last_undegraded":"2024-04-26T21:12:02.768484+0000","last_fullsized":"2024-04-26T21:12:02.768484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:29.139093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176129+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:04.176129+0000","last_peered":"2024-04-26T21:12:04.176129+0000","last_clean":"2024-04-26T21:12:04.176129+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:04.176129+0000","last_undegraded":"2024-04-26T21:12:04.176129+0000","last_fullsized":"2024-04-26T21:12:04.176129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075714+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:04.075714+0000","last_peered":"2024-04-26T21:12:04.075714+0000","last_clean":"2024-04-26T21:12:04.075714+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:04.075714+0000","last_undegraded":"2024-04-26T21:12:04.075714+0000","last_fullsized":"2024-04-26T21:12:04.075714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768539+0000","last_change":"2024-04-26T21:12:01.755969+0000","last_active":"2024-04-26T21:12:02.768539+0000","last_peered":"2024-04-26T21:12:02.768539+0000","last_clean":"2024-04-26T21:12:02.768539+0000","last_became_active":"2024-04-26T21:12:01.755628+0000","last_became_peered":"2024-04-26T21:12:01.755628+0000","last_unstale":"2024-04-26T21:12:02.768539+0000","last_undegraded":"2024-04-26T21:12:02.768539+0000","last_fullsized":"2024-04-26T21:12:02.768539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:42:31.761499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768360+0000","last_change":"2024-04-26T21:12:00.740944+0000","last_active":"2024-04-26T21:12:02.768360+0000","last_peered":"2024-04-26T21:12:02.768360+0000","last_clean":"2024-04-26T21:12:02.768360+0000","last_became_active":"2024-04-26T21:12:00.740532+0000","last_became_peered":"2024-04-26T21:12:00.740532+0000","last_unstale":"2024-04-26T21:12:02.768360+0000","last_undegraded":"2024-04-26T21:12:02.768360+0000","last_fullsized":"2024-04-26T21:12:02.768360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:16.536321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074939+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:03.074939+0000","last_peered":"2024-04-26T21:12:03.074939+0000","last_clean":"2024-04-26T21:12:03.074939+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:03.074939+0000","last_undegraded":"2024-04-26T21:12:03.074939+0000","last_fullsized":"2024-04-26T21:12:03.074939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176063+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:04.176063+0000","last_peered":"2024-04-26T21:12:04.176063+0000","last_clean":"2024-04-26T21:12:04.176063+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:04.176063+0000","last_undegraded":"2024-04-26T21:12:04.176063+0000","last_fullsized":"2024-04-26T21:12:04.176063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074967+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:04.074967+0000","last_peered":"2024-04-26T21:12:04.074967+0000","last_clean":"2024-04-26T21:12:04.074967+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:04.074967+0000","last_undegraded":"2024-04-26T21:12:04.074967+0000","last_fullsized":"2024-04-26T21:12:04.074967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974333+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:02.974333+0000","last_peered":"2024-04-26T21:12:02.974333+0000","last_clean":"2024-04-26T21:12:02.974333+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:02.974333+0000","last_undegraded":"2024-04-26T21:12:02.974333+0000","last_fullsized":"2024-04-26T21:12:02.974333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768356+0000","last_change":"2024-04-26T21:12:00.749992+0000","last_active":"2024-04-26T21:12:02.768356+0000","last_peered":"2024-04-26T21:12:02.768356+0000","last_clean":"2024-04-26T21:12:02.768356+0000","last_became_active":"2024-04-26T21:12:00.749601+0000","last_became_peered":"2024-04-26T21:12:00.749601+0000","last_unstale":"2024-04-26T21:12:02.768356+0000","last_undegraded":"2024-04-26T21:12:02.768356+0000","last_fullsized":"2024-04-26T21:12:02.768356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:39.587954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768336+0000","last_change":"2024-04-26T21:12:00.747054+0000","last_active":"2024-04-26T21:12:02.768336+0000","last_peered":"2024-04-26T21:12:02.768336+0000","last_clean":"2024-04-26T21:12:02.768336+0000","last_became_active":"2024-04-26T21:12:00.746691+0000","last_became_peered":"2024-04-26T21:12:00.746691+0000","last_unstale":"2024-04-26T21:12:02.768336+0000","last_undegraded":"2024-04-26T21:12:02.768336+0000","last_fullsized":"2024-04-26T21:12:02.768336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:28.714030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075403+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:04.075403+0000","last_peered":"2024-04-26T21:12:04.075403+0000","last_clean":"2024-04-26T21:12:04.075403+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:04.075403+0000","last_undegraded":"2024-04-26T21:12:04.075403+0000","last_fullsized":"2024-04-26T21:12:04.075403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074723+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:03.074723+0000","last_peered":"2024-04-26T21:12:03.074723+0000","last_clean":"2024-04-26T21:12:03.074723+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:03.074723+0000","last_undegraded":"2024-04-26T21:12:03.074723+0000","last_fullsized":"2024-04-26T21:12:03.074723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175995+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:04.175995+0000","last_peered":"2024-04-26T21:12:04.175995+0000","last_clean":"2024-04-26T21:12:04.175995+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:04.175995+0000","last_undegraded":"2024-04-26T21:12:04.175995+0000","last_fullsized":"2024-04-26T21:12:04.175995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075331+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:04.075331+0000","last_peered":"2024-04-26T21:12:04.075331+0000","last_clean":"2024-04-26T21:12:04.075331+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:04.075331+0000","last_undegraded":"2024-04-26T21:12:04.075331+0000","last_fullsized":"2024-04-26T21:12:04.075331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176188+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:04.176188+0000","last_peered":"2024-04-26T21:12:04.176188+0000","last_clean":"2024-04-26T21:12:04.176188+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:04.176188+0000","last_undegraded":"2024-04-26T21:12:04.176188+0000","last_fullsized":"2024-04-26T21:12:04.176188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075571+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:04.075571+0000","last_peered":"2024-04-26T21:12:04.075571+0000","last_clean":"2024-04-26T21:12:04.075571+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:04.075571+0000","last_undegraded":"2024-04-26T21:12:04.075571+0000","last_fullsized":"2024-04-26T21:12:04.075571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175980+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:04.175980+0000","last_peered":"2024-04-26T21:12:04.175980+0000","last_clean":"2024-04-26T21:12:04.175980+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:04.175980+0000","last_undegraded":"2024-04-26T21:12:04.175980+0000","last_fullsized":"2024-04-26T21:12:04.175980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176091+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:04.176091+0000","last_peered":"2024-04-26T21:12:04.176091+0000","last_clean":"2024-04-26T21:12:04.176091+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:04.176091+0000","last_undegraded":"2024-04-26T21:12:04.176091+0000","last_fullsized":"2024-04-26T21:12:04.176091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":81325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":672241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:13.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.811+0000 7fb08931b640 1 Processor -- start 2024-04-26T21:12:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 -- start start 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb08415f340 0x7fb0840b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0840d5610 0x7fb0840d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0840b2510 con 0x7fb08415f720 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb0840b27f0 con 0x7fb08415f340 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb08931b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840b2ad0 con 0x7fb0840d5610 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb082ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb08415f340 0x7fb0840b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb0827fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb08415f720 0x7fb0840d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58602/0 (socket says 172.21.15.40:58602) 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb082ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb08415f340 0x7fb0840b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40652/0 (socket says 172.21.15.40:40652) 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.812+0000 7fb0827fc640 1 -- 172.21.15.40:0/1710052866 learned_addr learned my addr 172.21.15.40:0/1710052866 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb0837fe640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0840d5610 0x7fb0840d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb082ffd640 1 -- 172.21.15.40:0/1710052866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0840d5610 msgr2=0x7fb0840d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb082ffd640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0840d5610 0x7fb0840d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb082ffd640 1 -- 172.21.15.40:0/1710052866 >> v1:172.21.15.40:6789/0 conn(0x7fb08415f720 legacy=0x7fb0840d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb082ffd640 1 -- 172.21.15.40:0/1710052866 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0840b2db0 con 0x7fb08415f340 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb081ffb640 1 -- 172.21.15.40:0/1710052866 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573928657 0 0) 0x7fb0840b2510 con 0x7fb08415f720 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.813+0000 7fb082ffd640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f340 0x7fb0840b1e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097a0 tx=0x7fb07c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2844c5e8d78f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb081ffb640 1 -- 172.21.15.40:0/1710052866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c037070 con 0x7fb08415f340 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb081ffb640 1 -- 172.21.15.40:0/1710052866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c02ae50 con 0x7fb08415f340 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb081ffb640 1 -- 172.21.15.40:0/1710052866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c032660 con 0x7fb08415f340 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 -- 172.21.15.40:0/1710052866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f340 msgr2=0x7fb0840b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f340 0x7fb0840b1e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097a0 tx=0x7fb07c02a100 comp rx=0 tx=0).stop 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 -- 172.21.15.40:0/1710052866 shutdown_connections 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0840d5610 0x7fb0840d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 --2- 172.21.15.40:0/1710052866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f340 0x7fb0840b1e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 -- 172.21.15.40:0/1710052866 >> 172.21.15.40:0/1710052866 conn(0x7fb08408ec70 msgr2=0x7fb08409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 -- 172.21.15.40:0/1710052866 shutdown_connections 2024-04-26T21:12:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.814+0000 7fb08931b640 1 -- 172.21.15.40:0/1710052866 wait complete. 2024-04-26T21:12:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.815+0000 7fb08931b640 1 Processor -- start 2024-04-26T21:12:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.815+0000 7fb08931b640 1 -- start start 2024-04-26T21:12:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 0x7fb08415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 0x7fb0840c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b3940 con 0x7fb08415f720 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b37c0 con 0x7fb0840d5610 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb08931b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840b3ac0 con 0x7fb08415f340 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb0837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb0837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40664/0 (socket says 172.21.15.40:40664) 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.816+0000 7fb0837fe640 1 -- 172.21.15.40:0/538670607 learned_addr learned my addr 172.21.15.40:0/538670607 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0827fc640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 0x7fb0840c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb082ffd640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 0x7fb08415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 -- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 msgr2=0x7fb0840c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 0x7fb0840c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 -- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 msgr2=0x7fb08415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 0x7fb08415da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 -- 172.21.15.40:0/538670607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0840c95d0 con 0x7fb08415f720 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0827fc640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 0x7fb0840c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb082ffd640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 0x7fb08415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb0837fe640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb07400efc0 tx=0x7fb07400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074018020 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb074014480 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074014870 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0840c98b0 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.817+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0840c9e70 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.818+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08406a290 con 0x7fb08415f720 2024-04-26T21:12:13.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.818+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb074005d10 con 0x7fb08415f720 2024-04-26T21:12:13.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.819+0000 7fb07b7fe640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:13.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.819+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb07408c980 con 0x7fb08415f720 2024-04-26T21:12:13.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.819+0000 7fb082ffd640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:13.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.820+0000 7fb082ffd640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb07c034000 tx=0x7fb07c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:13.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:13.823+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb074022030 con 0x7fb08415f720 2024-04-26T21:12:14.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.004+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0840d3020 con 0x7fb06406c8f0 2024-04-26T21:12:14.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.010+0000 7fb07b7fe640 1 -- 172.21.15.40:0/538670607 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fb0840d3020 con 0x7fb06406c8f0 2024-04-26T21:12:14.014 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:14.015 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 msgr2=0x7fb06406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb07c034000 tx=0x7fb07c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 msgr2=0x7fb0840ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb07400efc0 tx=0x7fb07400c490 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 shutdown_connections 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08415f720 0x7fb0840ca930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb08415f340 0x7fb0840c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 --2- 172.21.15.40:0/538670607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0840d5610 0x7fb08415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 >> 172.21.15.40:0/538670607 conn(0x7fb08408ec70 msgr2=0x7fb0840b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 shutdown_connections 2024-04-26T21:12:14.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.013+0000 7fb08931b640 1 -- 172.21.15.40:0/538670607 wait complete. 2024-04-26T21:12:14.025 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-26T21:12:12.920600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219236,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755484,"statfs":{"total":773094113280,"available":772869615616,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.154550"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075715+0000","last_change":"2024-04-26T21:12:01.772706+0000","last_active":"2024-04-26T21:12:04.075715+0000","last_peered":"2024-04-26T21:12:04.075715+0000","last_clean":"2024-04-26T21:12:04.075715+0000","last_became_active":"2024-04-26T21:12:01.772073+0000","last_became_peered":"2024-04-26T21:12:01.772073+0000","last_unstale":"2024-04-26T21:12:04.075715+0000","last_undegraded":"2024-04-26T21:12:04.075715+0000","last_fullsized":"2024-04-26T21:12:04.075715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:17.992415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768372+0000","last_change":"2024-04-26T21:12:01.767311+0000","last_active":"2024-04-26T21:12:02.768372+0000","last_peered":"2024-04-26T21:12:02.768372+0000","last_clean":"2024-04-26T21:12:02.768372+0000","last_became_active":"2024-04-26T21:12:01.766955+0000","last_became_peered":"2024-04-26T21:12:01.766955+0000","last_unstale":"2024-04-26T21:12:02.768372+0000","last_undegraded":"2024-04-26T21:12:02.768372+0000","last_fullsized":"2024-04-26T21:12:02.768372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:29.829212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175853+0000","last_change":"2024-04-26T21:12:01.754644+0000","last_active":"2024-04-26T21:12:04.175853+0000","last_peered":"2024-04-26T21:12:04.175853+0000","last_clean":"2024-04-26T21:12:04.175853+0000","last_became_active":"2024-04-26T21:12:01.754354+0000","last_became_peered":"2024-04-26T21:12:01.754354+0000","last_unstale":"2024-04-26T21:12:04.175853+0000","last_undegraded":"2024-04-26T21:12:04.175853+0000","last_fullsized":"2024-04-26T21:12:04.175853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175990+0000","last_change":"2024-04-26T21:12:01.761292+0000","last_active":"2024-04-26T21:12:04.175990+0000","last_peered":"2024-04-26T21:12:04.175990+0000","last_clean":"2024-04-26T21:12:04.175990+0000","last_became_active":"2024-04-26T21:12:01.760948+0000","last_became_peered":"2024-04-26T21:12:01.760948+0000","last_unstale":"2024-04-26T21:12:04.175990+0000","last_undegraded":"2024-04-26T21:12:04.175990+0000","last_fullsized":"2024-04-26T21:12:04.175990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075284+0000","last_change":"2024-04-26T21:12:01.769224+0000","last_active":"2024-04-26T21:12:04.075284+0000","last_peered":"2024-04-26T21:12:04.075284+0000","last_clean":"2024-04-26T21:12:04.075284+0000","last_became_active":"2024-04-26T21:12:01.767947+0000","last_became_peered":"2024-04-26T21:12:01.767947+0000","last_unstale":"2024-04-26T21:12:04.075284+0000","last_undegraded":"2024-04-26T21:12:04.075284+0000","last_fullsized":"2024-04-26T21:12:04.075284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175693+0000","last_change":"2024-04-26T21:12:01.754645+0000","last_active":"2024-04-26T21:12:04.175693+0000","last_peered":"2024-04-26T21:12:04.175693+0000","last_clean":"2024-04-26T21:12:04.175693+0000","last_became_active":"2024-04-26T21:12:01.754365+0000","last_became_peered":"2024-04-26T21:12:01.754365+0000","last_unstale":"2024-04-26T21:12:04.175693+0000","last_undegraded":"2024-04-26T21:12:04.175693+0000","last_fullsized":"2024-04-26T21:12:04.175693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075067+0000","last_change":"2024-04-26T21:12:01.772022+0000","last_active":"2024-04-26T21:12:04.075067+0000","last_peered":"2024-04-26T21:12:04.075067+0000","last_clean":"2024-04-26T21:12:04.075067+0000","last_became_active":"2024-04-26T21:12:01.771728+0000","last_became_peered":"2024-04-26T21:12:01.771728+0000","last_unstale":"2024-04-26T21:12:04.075067+0000","last_undegraded":"2024-04-26T21:12:04.075067+0000","last_fullsized":"2024-04-26T21:12:04.075067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767370+0000","last_change":"2024-04-26T21:12:01.758119+0000","last_active":"2024-04-26T21:12:02.767370+0000","last_peered":"2024-04-26T21:12:02.767370+0000","last_clean":"2024-04-26T21:12:02.767370+0000","last_became_active":"2024-04-26T21:12:01.757685+0000","last_became_peered":"2024-04-26T21:12:01.757685+0000","last_unstale":"2024-04-26T21:12:02.767370+0000","last_undegraded":"2024-04-26T21:12:02.767370+0000","last_fullsized":"2024-04-26T21:12:02.767370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075793+0000","last_change":"2024-04-26T21:12:01.776650+0000","last_active":"2024-04-26T21:12:04.075793+0000","last_peered":"2024-04-26T21:12:04.075793+0000","last_clean":"2024-04-26T21:12:04.075793+0000","last_became_active":"2024-04-26T21:12:01.776363+0000","last_became_peered":"2024-04-26T21:12:01.776363+0000","last_unstale":"2024-04-26T21:12:04.075793+0000","last_undegraded":"2024-04-26T21:12:04.075793+0000","last_fullsized":"2024-04-26T21:12:04.075793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.197913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075042+0000","last_change":"2024-04-26T21:12:01.762208+0000","last_active":"2024-04-26T21:12:03.075042+0000","last_peered":"2024-04-26T21:12:03.075042+0000","last_clean":"2024-04-26T21:12:03.075042+0000","last_became_active":"2024-04-26T21:12:01.761941+0000","last_became_peered":"2024-04-26T21:12:01.761941+0000","last_unstale":"2024-04-26T21:12:03.075042+0000","last_undegraded":"2024-04-26T21:12:03.075042+0000","last_fullsized":"2024-04-26T21:12:03.075042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074816+0000","last_change":"2024-04-26T21:12:01.770557+0000","last_active":"2024-04-26T21:12:03.074816+0000","last_peered":"2024-04-26T21:12:03.074816+0000","last_clean":"2024-04-26T21:12:03.074816+0000","last_became_active":"2024-04-26T21:12:01.770304+0000","last_became_peered":"2024-04-26T21:12:01.770304+0000","last_unstale":"2024-04-26T21:12:03.074816+0000","last_undegraded":"2024-04-26T21:12:03.074816+0000","last_fullsized":"2024-04-26T21:12:03.074816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974567+0000","last_change":"2024-04-26T21:12:01.764648+0000","last_active":"2024-04-26T21:12:02.974567+0000","last_peered":"2024-04-26T21:12:02.974567+0000","last_clean":"2024-04-26T21:12:02.974567+0000","last_became_active":"2024-04-26T21:12:01.763818+0000","last_became_peered":"2024-04-26T21:12:01.763818+0000","last_unstale":"2024-04-26T21:12:02.974567+0000","last_undegraded":"2024-04-26T21:12:02.974567+0000","last_fullsized":"2024-04-26T21:12:02.974567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767738+0000","last_change":"2024-04-26T21:12:01.762279+0000","last_active":"2024-04-26T21:12:02.767738+0000","last_peered":"2024-04-26T21:12:02.767738+0000","last_clean":"2024-04-26T21:12:02.767738+0000","last_became_active":"2024-04-26T21:12:01.761958+0000","last_became_peered":"2024-04-26T21:12:01.761958+0000","last_unstale":"2024-04-26T21:12:02.767738+0000","last_undegraded":"2024-04-26T21:12:02.767738+0000","last_fullsized":"2024-04-26T21:12:02.767738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175587+0000","last_change":"2024-04-26T21:12:01.757658+0000","last_active":"2024-04-26T21:12:04.175587+0000","last_peered":"2024-04-26T21:12:04.175587+0000","last_clean":"2024-04-26T21:12:04.175587+0000","last_became_active":"2024-04-26T21:12:01.756761+0000","last_became_peered":"2024-04-26T21:12:01.756761+0000","last_unstale":"2024-04-26T21:12:04.175587+0000","last_undegraded":"2024-04-26T21:12:04.175587+0000","last_fullsized":"2024-04-26T21:12:04.175587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768447+0000","last_change":"2024-04-26T21:12:01.769541+0000","last_active":"2024-04-26T21:12:02.768447+0000","last_peered":"2024-04-26T21:12:02.768447+0000","last_clean":"2024-04-26T21:12:02.768447+0000","last_became_active":"2024-04-26T21:12:01.769171+0000","last_became_peered":"2024-04-26T21:12:01.769171+0000","last_unstale":"2024-04-26T21:12:02.768447+0000","last_undegraded":"2024-04-26T21:12:02.768447+0000","last_fullsized":"2024-04-26T21:12:02.768447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:22.619826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175595+0000","last_change":"2024-04-26T21:12:01.755206+0000","last_active":"2024-04-26T21:12:04.175595+0000","last_peered":"2024-04-26T21:12:04.175595+0000","last_clean":"2024-04-26T21:12:04.175595+0000","last_became_active":"2024-04-26T21:12:01.754834+0000","last_became_peered":"2024-04-26T21:12:01.754834+0000","last_unstale":"2024-04-26T21:12:04.175595+0000","last_undegraded":"2024-04-26T21:12:04.175595+0000","last_fullsized":"2024-04-26T21:12:04.175595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176236+0000","last_change":"2024-04-26T21:12:01.756659+0000","last_active":"2024-04-26T21:12:04.176236+0000","last_peered":"2024-04-26T21:12:04.176236+0000","last_clean":"2024-04-26T21:12:04.176236+0000","last_became_active":"2024-04-26T21:12:01.756409+0000","last_became_peered":"2024-04-26T21:12:01.756409+0000","last_unstale":"2024-04-26T21:12:04.176236+0000","last_undegraded":"2024-04-26T21:12:04.176236+0000","last_fullsized":"2024-04-26T21:12:04.176236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.763004+0000","last_change":"2024-04-26T21:12:01.764565+0000","last_active":"2024-04-26T21:12:02.763004+0000","last_peered":"2024-04-26T21:12:02.763004+0000","last_clean":"2024-04-26T21:12:02.763004+0000","last_became_active":"2024-04-26T21:12:01.764276+0000","last_became_peered":"2024-04-26T21:12:01.764276+0000","last_unstale":"2024-04-26T21:12:02.763004+0000","last_undegraded":"2024-04-26T21:12:02.763004+0000","last_fullsized":"2024-04-26T21:12:02.763004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.477864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075698+0000","last_change":"2024-04-26T21:12:01.771740+0000","last_active":"2024-04-26T21:12:04.075698+0000","last_peered":"2024-04-26T21:12:04.075698+0000","last_clean":"2024-04-26T21:12:04.075698+0000","last_became_active":"2024-04-26T21:12:01.771227+0000","last_became_peered":"2024-04-26T21:12:01.771227+0000","last_unstale":"2024-04-26T21:12:04.075698+0000","last_undegraded":"2024-04-26T21:12:04.075698+0000","last_fullsized":"2024-04-26T21:12:04.075698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:29.707431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074796+0000","last_change":"2024-04-26T21:12:01.766293+0000","last_active":"2024-04-26T21:12:03.074796+0000","last_peered":"2024-04-26T21:12:03.074796+0000","last_clean":"2024-04-26T21:12:03.074796+0000","last_became_active":"2024-04-26T21:12:01.765950+0000","last_became_peered":"2024-04-26T21:12:01.765950+0000","last_unstale":"2024-04-26T21:12:03.074796+0000","last_undegraded":"2024-04-26T21:12:03.074796+0000","last_fullsized":"2024-04-26T21:12:03.074796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175801+0000","last_change":"2024-04-26T21:12:01.766421+0000","last_active":"2024-04-26T21:12:04.175801+0000","last_peered":"2024-04-26T21:12:04.175801+0000","last_clean":"2024-04-26T21:12:04.175801+0000","last_became_active":"2024-04-26T21:12:01.764981+0000","last_became_peered":"2024-04-26T21:12:01.764981+0000","last_unstale":"2024-04-26T21:12:04.175801+0000","last_undegraded":"2024-04-26T21:12:04.175801+0000","last_fullsized":"2024-04-26T21:12:04.175801+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:15.250787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175896+0000","last_change":"2024-04-26T21:12:01.764167+0000","last_active":"2024-04-26T21:12:04.175896+0000","last_peered":"2024-04-26T21:12:04.175896+0000","last_clean":"2024-04-26T21:12:04.175896+0000","last_became_active":"2024-04-26T21:12:01.763913+0000","last_became_peered":"2024-04-26T21:12:01.763913+0000","last_unstale":"2024-04-26T21:12:04.175896+0000","last_undegraded":"2024-04-26T21:12:04.175896+0000","last_fullsized":"2024-04-26T21:12:04.175896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767466+0000","last_change":"2024-04-26T21:12:01.761106+0000","last_active":"2024-04-26T21:12:02.767466+0000","last_peered":"2024-04-26T21:12:02.767466+0000","last_clean":"2024-04-26T21:12:02.767466+0000","last_became_active":"2024-04-26T21:12:01.760823+0000","last_became_peered":"2024-04-26T21:12:01.760823+0000","last_unstale":"2024-04-26T21:12:02.767466+0000","last_undegraded":"2024-04-26T21:12:02.767466+0000","last_fullsized":"2024-04-26T21:12:02.767466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175677+0000","last_change":"2024-04-26T21:12:01.763599+0000","last_active":"2024-04-26T21:12:04.175677+0000","last_peered":"2024-04-26T21:12:04.175677+0000","last_clean":"2024-04-26T21:12:04.175677+0000","last_became_active":"2024-04-26T21:12:01.763332+0000","last_became_peered":"2024-04-26T21:12:01.763332+0000","last_unstale":"2024-04-26T21:12:04.175677+0000","last_undegraded":"2024-04-26T21:12:04.175677+0000","last_fullsized":"2024-04-26T21:12:04.175677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767971+0000","last_change":"2024-04-26T21:12:01.760541+0000","last_active":"2024-04-26T21:12:02.767971+0000","last_peered":"2024-04-26T21:12:02.767971+0000","last_clean":"2024-04-26T21:12:02.767971+0000","last_became_active":"2024-04-26T21:12:01.760212+0000","last_became_peered":"2024-04-26T21:12:01.760212+0000","last_unstale":"2024-04-26T21:12:02.767971+0000","last_undegraded":"2024-04-26T21:12:02.767971+0000","last_fullsized":"2024-04-26T21:12:02.767971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.075061+0000","last_change":"2024-04-26T21:12:01.761260+0000","last_active":"2024-04-26T21:12:03.075061+0000","last_peered":"2024-04-26T21:12:03.075061+0000","last_clean":"2024-04-26T21:12:03.075061+0000","last_became_active":"2024-04-26T21:12:01.760993+0000","last_became_peered":"2024-04-26T21:12:01.760993+0000","last_unstale":"2024-04-26T21:12:03.075061+0000","last_undegraded":"2024-04-26T21:12:03.075061+0000","last_fullsized":"2024-04-26T21:12:03.075061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768216+0000","last_change":"2024-04-26T21:12:01.757848+0000","last_active":"2024-04-26T21:12:02.768216+0000","last_peered":"2024-04-26T21:12:02.768216+0000","last_clean":"2024-04-26T21:12:02.768216+0000","last_became_active":"2024-04-26T21:12:01.757563+0000","last_became_peered":"2024-04-26T21:12:01.757563+0000","last_unstale":"2024-04-26T21:12:02.768216+0000","last_undegraded":"2024-04-26T21:12:02.768216+0000","last_fullsized":"2024-04-26T21:12:02.768216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:51.758916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767942+0000","last_change":"2024-04-26T21:12:01.763415+0000","last_active":"2024-04-26T21:12:02.767942+0000","last_peered":"2024-04-26T21:12:02.767942+0000","last_clean":"2024-04-26T21:12:02.767942+0000","last_became_active":"2024-04-26T21:12:01.761972+0000","last_became_peered":"2024-04-26T21:12:01.761972+0000","last_unstale":"2024-04-26T21:12:02.767942+0000","last_undegraded":"2024-04-26T21:12:02.767942+0000","last_fullsized":"2024-04-26T21:12:02.767942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:51.149483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175879+0000","last_change":"2024-04-26T21:12:01.765869+0000","last_active":"2024-04-26T21:12:04.175879+0000","last_peered":"2024-04-26T21:12:04.175879+0000","last_clean":"2024-04-26T21:12:04.175879+0000","last_became_active":"2024-04-26T21:12:01.765043+0000","last_became_peered":"2024-04-26T21:12:01.765043+0000","last_unstale":"2024-04-26T21:12:04.175879+0000","last_undegraded":"2024-04-26T21:12:04.175879+0000","last_fullsized":"2024-04-26T21:12:04.175879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175961+0000","last_change":"2024-04-26T21:12:01.762870+0000","last_active":"2024-04-26T21:12:04.175961+0000","last_peered":"2024-04-26T21:12:04.175961+0000","last_clean":"2024-04-26T21:12:04.175961+0000","last_became_active":"2024-04-26T21:12:01.762603+0000","last_became_peered":"2024-04-26T21:12:01.762603+0000","last_unstale":"2024-04-26T21:12:04.175961+0000","last_undegraded":"2024-04-26T21:12:04.175961+0000","last_fullsized":"2024-04-26T21:12:04.175961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:26.321739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762730+0000","last_change":"2024-04-26T21:12:01.752848+0000","last_active":"2024-04-26T21:12:02.762730+0000","last_peered":"2024-04-26T21:12:02.762730+0000","last_clean":"2024-04-26T21:12:02.762730+0000","last_became_active":"2024-04-26T21:12:01.752505+0000","last_became_peered":"2024-04-26T21:12:01.752505+0000","last_unstale":"2024-04-26T21:12:02.762730+0000","last_undegraded":"2024-04-26T21:12:02.762730+0000","last_fullsized":"2024-04-26T21:12:02.762730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074711+0000","last_change":"2024-04-26T21:12:01.767970+0000","last_active":"2024-04-26T21:12:03.074711+0000","last_peered":"2024-04-26T21:12:03.074711+0000","last_clean":"2024-04-26T21:12:03.074711+0000","last_became_active":"2024-04-26T21:12:01.767238+0000","last_became_peered":"2024-04-26T21:12:01.767238+0000","last_unstale":"2024-04-26T21:12:03.074711+0000","last_undegraded":"2024-04-26T21:12:03.074711+0000","last_fullsized":"2024-04-26T21:12:03.074711+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974321+0000","last_change":"2024-04-26T21:12:01.770798+0000","last_active":"2024-04-26T21:12:02.974321+0000","last_peered":"2024-04-26T21:12:02.974321+0000","last_clean":"2024-04-26T21:12:02.974321+0000","last_became_active":"2024-04-26T21:12:01.770527+0000","last_became_peered":"2024-04-26T21:12:01.770527+0000","last_unstale":"2024-04-26T21:12:02.974321+0000","last_undegraded":"2024-04-26T21:12:02.974321+0000","last_fullsized":"2024-04-26T21:12:02.974321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:38.640212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767948+0000","last_change":"2024-04-26T21:12:00.753795+0000","last_active":"2024-04-26T21:12:02.767948+0000","last_peered":"2024-04-26T21:12:02.767948+0000","last_clean":"2024-04-26T21:12:02.767948+0000","last_became_active":"2024-04-26T21:12:00.753495+0000","last_became_peered":"2024-04-26T21:12:00.753495+0000","last_unstale":"2024-04-26T21:12:02.767948+0000","last_undegraded":"2024-04-26T21:12:02.767948+0000","last_fullsized":"2024-04-26T21:12:02.767948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175972+0000","last_change":"2024-04-26T21:12:01.765368+0000","last_active":"2024-04-26T21:12:04.175972+0000","last_peered":"2024-04-26T21:12:04.175972+0000","last_clean":"2024-04-26T21:12:04.175972+0000","last_became_active":"2024-04-26T21:12:01.764520+0000","last_became_peered":"2024-04-26T21:12:01.764520+0000","last_unstale":"2024-04-26T21:12:04.175972+0000","last_undegraded":"2024-04-26T21:12:04.175972+0000","last_fullsized":"2024-04-26T21:12:04.175972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074650+0000","last_change":"2024-04-26T21:12:00.749030+0000","last_active":"2024-04-26T21:12:03.074650+0000","last_peered":"2024-04-26T21:12:03.074650+0000","last_clean":"2024-04-26T21:12:03.074650+0000","last_became_active":"2024-04-26T21:12:00.747629+0000","last_became_peered":"2024-04-26T21:12:00.747629+0000","last_unstale":"2024-04-26T21:12:03.074650+0000","last_undegraded":"2024-04-26T21:12:03.074650+0000","last_fullsized":"2024-04-26T21:12:03.074650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767576+0000","last_change":"2024-04-26T21:12:00.751363+0000","last_active":"2024-04-26T21:12:02.767576+0000","last_peered":"2024-04-26T21:12:02.767576+0000","last_clean":"2024-04-26T21:12:02.767576+0000","last_became_active":"2024-04-26T21:12:00.750924+0000","last_became_peered":"2024-04-26T21:12:00.750924+0000","last_unstale":"2024-04-26T21:12:02.767576+0000","last_undegraded":"2024-04-26T21:12:02.767576+0000","last_fullsized":"2024-04-26T21:12:02.767576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:38.191961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767371+0000","last_change":"2024-04-26T21:12:01.758492+0000","last_active":"2024-04-26T21:12:02.767371+0000","last_peered":"2024-04-26T21:12:02.767371+0000","last_clean":"2024-04-26T21:12:02.767371+0000","last_became_active":"2024-04-26T21:12:01.754846+0000","last_became_peered":"2024-04-26T21:12:01.754846+0000","last_unstale":"2024-04-26T21:12:02.767371+0000","last_undegraded":"2024-04-26T21:12:02.767371+0000","last_fullsized":"2024-04-26T21:12:02.767371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:58.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175834+0000","last_change":"2024-04-26T21:12:00.748028+0000","last_active":"2024-04-26T21:12:04.175834+0000","last_peered":"2024-04-26T21:12:04.175834+0000","last_clean":"2024-04-26T21:12:04.175834+0000","last_became_active":"2024-04-26T21:12:00.747726+0000","last_became_peered":"2024-04-26T21:12:00.747726+0000","last_unstale":"2024-04-26T21:12:04.175834+0000","last_undegraded":"2024-04-26T21:12:04.175834+0000","last_fullsized":"2024-04-26T21:12:04.175834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768245+0000","last_change":"2024-04-26T21:12:01.765248+0000","last_active":"2024-04-26T21:12:02.768245+0000","last_peered":"2024-04-26T21:12:02.768245+0000","last_clean":"2024-04-26T21:12:02.768245+0000","last_became_active":"2024-04-26T21:12:01.764951+0000","last_became_peered":"2024-04-26T21:12:01.764951+0000","last_unstale":"2024-04-26T21:12:02.768245+0000","last_undegraded":"2024-04-26T21:12:02.768245+0000","last_fullsized":"2024-04-26T21:12:02.768245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:24.962313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074890+0000","last_change":"2024-04-26T21:12:00.745499+0000","last_active":"2024-04-26T21:12:03.074890+0000","last_peered":"2024-04-26T21:12:03.074890+0000","last_clean":"2024-04-26T21:12:03.074890+0000","last_became_active":"2024-04-26T21:12:00.745191+0000","last_became_peered":"2024-04-26T21:12:00.745191+0000","last_unstale":"2024-04-26T21:12:03.074890+0000","last_undegraded":"2024-04-26T21:12:03.074890+0000","last_fullsized":"2024-04-26T21:12:03.074890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176145+0000","last_change":"2024-04-26T21:12:01.751713+0000","last_active":"2024-04-26T21:12:04.176145+0000","last_peered":"2024-04-26T21:12:04.176145+0000","last_clean":"2024-04-26T21:12:04.176145+0000","last_became_active":"2024-04-26T21:12:01.751381+0000","last_became_peered":"2024-04-26T21:12:01.751381+0000","last_unstale":"2024-04-26T21:12:04.176145+0000","last_undegraded":"2024-04-26T21:12:04.176145+0000","last_fullsized":"2024-04-26T21:12:04.176145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175969+0000","last_change":"2024-04-26T21:12:00.751375+0000","last_active":"2024-04-26T21:12:04.175969+0000","last_peered":"2024-04-26T21:12:04.175969+0000","last_clean":"2024-04-26T21:12:04.175969+0000","last_became_active":"2024-04-26T21:12:00.751141+0000","last_became_peered":"2024-04-26T21:12:00.751141+0000","last_unstale":"2024-04-26T21:12:04.175969+0000","last_undegraded":"2024-04-26T21:12:04.175969+0000","last_fullsized":"2024-04-26T21:12:04.175969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176007+0000","last_change":"2024-04-26T21:12:01.759494+0000","last_active":"2024-04-26T21:12:04.176007+0000","last_peered":"2024-04-26T21:12:04.176007+0000","last_clean":"2024-04-26T21:12:04.176007+0000","last_became_active":"2024-04-26T21:12:01.758693+0000","last_became_peered":"2024-04-26T21:12:01.758693+0000","last_unstale":"2024-04-26T21:12:04.176007+0000","last_undegraded":"2024-04-26T21:12:04.176007+0000","last_fullsized":"2024-04-26T21:12:04.176007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:09.628638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768176+0000","last_change":"2024-04-26T21:12:00.744708+0000","last_active":"2024-04-26T21:12:02.768176+0000","last_peered":"2024-04-26T21:12:02.768176+0000","last_clean":"2024-04-26T21:12:02.768176+0000","last_became_active":"2024-04-26T21:12:00.744396+0000","last_became_peered":"2024-04-26T21:12:00.744396+0000","last_unstale":"2024-04-26T21:12:02.768176+0000","last_undegraded":"2024-04-26T21:12:02.768176+0000","last_fullsized":"2024-04-26T21:12:02.768176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:40.330925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974569+0000","last_change":"2024-04-26T21:12:01.768869+0000","last_active":"2024-04-26T21:12:02.974569+0000","last_peered":"2024-04-26T21:12:02.974569+0000","last_clean":"2024-04-26T21:12:02.974569+0000","last_became_active":"2024-04-26T21:12:01.768597+0000","last_became_peered":"2024-04-26T21:12:01.768597+0000","last_unstale":"2024-04-26T21:12:02.974569+0000","last_undegraded":"2024-04-26T21:12:02.974569+0000","last_fullsized":"2024-04-26T21:12:02.974569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.806563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074901+0000","last_change":"2024-04-26T21:12:00.748560+0000","last_active":"2024-04-26T21:12:03.074901+0000","last_peered":"2024-04-26T21:12:03.074901+0000","last_clean":"2024-04-26T21:12:03.074901+0000","last_became_active":"2024-04-26T21:12:00.747597+0000","last_became_peered":"2024-04-26T21:12:00.747597+0000","last_unstale":"2024-04-26T21:12:03.074901+0000","last_undegraded":"2024-04-26T21:12:03.074901+0000","last_fullsized":"2024-04-26T21:12:03.074901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075445+0000","last_change":"2024-04-26T21:12:01.774240+0000","last_active":"2024-04-26T21:12:04.075445+0000","last_peered":"2024-04-26T21:12:04.075445+0000","last_clean":"2024-04-26T21:12:04.075445+0000","last_became_active":"2024-04-26T21:12:01.773973+0000","last_became_peered":"2024-04-26T21:12:01.773973+0000","last_unstale":"2024-04-26T21:12:04.075445+0000","last_undegraded":"2024-04-26T21:12:04.075445+0000","last_fullsized":"2024-04-26T21:12:04.075445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.117039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175978+0000","last_change":"2024-04-26T21:12:00.743432+0000","last_active":"2024-04-26T21:12:04.175978+0000","last_peered":"2024-04-26T21:12:04.175978+0000","last_clean":"2024-04-26T21:12:04.175978+0000","last_became_active":"2024-04-26T21:12:00.742378+0000","last_became_peered":"2024-04-26T21:12:00.742378+0000","last_unstale":"2024-04-26T21:12:04.175978+0000","last_undegraded":"2024-04-26T21:12:04.175978+0000","last_fullsized":"2024-04-26T21:12:04.175978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:05.168449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175604+0000","last_change":"2024-04-26T21:12:01.761415+0000","last_active":"2024-04-26T21:12:04.175604+0000","last_peered":"2024-04-26T21:12:04.175604+0000","last_clean":"2024-04-26T21:12:04.175604+0000","last_became_active":"2024-04-26T21:12:01.761120+0000","last_became_peered":"2024-04-26T21:12:01.761120+0000","last_unstale":"2024-04-26T21:12:04.175604+0000","last_undegraded":"2024-04-26T21:12:04.175604+0000","last_fullsized":"2024-04-26T21:12:04.175604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.569186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176048+0000","last_change":"2024-04-26T21:12:00.750917+0000","last_active":"2024-04-26T21:12:04.176048+0000","last_peered":"2024-04-26T21:12:04.176048+0000","last_clean":"2024-04-26T21:12:04.176048+0000","last_became_active":"2024-04-26T21:12:00.750447+0000","last_became_peered":"2024-04-26T21:12:00.750447+0000","last_unstale":"2024-04-26T21:12:04.176048+0000","last_undegraded":"2024-04-26T21:12:04.176048+0000","last_fullsized":"2024-04-26T21:12:04.176048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075449+0000","last_change":"2024-04-26T21:12:01.770271+0000","last_active":"2024-04-26T21:12:04.075449+0000","last_peered":"2024-04-26T21:12:04.075449+0000","last_clean":"2024-04-26T21:12:04.075449+0000","last_became_active":"2024-04-26T21:12:01.770011+0000","last_became_peered":"2024-04-26T21:12:01.770011+0000","last_unstale":"2024-04-26T21:12:04.075449+0000","last_undegraded":"2024-04-26T21:12:04.075449+0000","last_fullsized":"2024-04-26T21:12:04.075449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074625+0000","last_change":"2024-04-26T21:12:00.748122+0000","last_active":"2024-04-26T21:12:03.074625+0000","last_peered":"2024-04-26T21:12:03.074625+0000","last_clean":"2024-04-26T21:12:03.074625+0000","last_became_active":"2024-04-26T21:12:00.747693+0000","last_became_peered":"2024-04-26T21:12:00.747693+0000","last_unstale":"2024-04-26T21:12:03.074625+0000","last_undegraded":"2024-04-26T21:12:03.074625+0000","last_fullsized":"2024-04-26T21:12:03.074625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074691+0000","last_change":"2024-04-26T21:12:01.767464+0000","last_active":"2024-04-26T21:12:03.074691+0000","last_peered":"2024-04-26T21:12:03.074691+0000","last_clean":"2024-04-26T21:12:03.074691+0000","last_became_active":"2024-04-26T21:12:01.766805+0000","last_became_peered":"2024-04-26T21:12:01.766805+0000","last_unstale":"2024-04-26T21:12:03.074691+0000","last_undegraded":"2024-04-26T21:12:03.074691+0000","last_fullsized":"2024-04-26T21:12:03.074691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:13.669257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762692+0000","last_change":"2024-04-26T21:12:00.743133+0000","last_active":"2024-04-26T21:12:02.762692+0000","last_peered":"2024-04-26T21:12:02.762692+0000","last_clean":"2024-04-26T21:12:02.762692+0000","last_became_active":"2024-04-26T21:12:00.742716+0000","last_became_peered":"2024-04-26T21:12:00.742716+0000","last_unstale":"2024-04-26T21:12:02.762692+0000","last_undegraded":"2024-04-26T21:12:02.762692+0000","last_fullsized":"2024-04-26T21:12:02.762692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075075+0000","last_change":"2024-04-26T21:12:01.770975+0000","last_active":"2024-04-26T21:12:04.075075+0000","last_peered":"2024-04-26T21:12:04.075075+0000","last_clean":"2024-04-26T21:12:04.075075+0000","last_became_active":"2024-04-26T21:12:01.770699+0000","last_became_peered":"2024-04-26T21:12:01.770699+0000","last_unstale":"2024-04-26T21:12:04.075075+0000","last_undegraded":"2024-04-26T21:12:04.075075+0000","last_fullsized":"2024-04-26T21:12:04.075075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974587+0000","last_change":"2024-04-26T21:12:00.745024+0000","last_active":"2024-04-26T21:12:02.974587+0000","last_peered":"2024-04-26T21:12:02.974587+0000","last_clean":"2024-04-26T21:12:02.974587+0000","last_became_active":"2024-04-26T21:12:00.744690+0000","last_became_peered":"2024-04-26T21:12:00.744690+0000","last_unstale":"2024-04-26T21:12:02.974587+0000","last_undegraded":"2024-04-26T21:12:02.974587+0000","last_fullsized":"2024-04-26T21:12:02.974587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:12.906129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075598+0000","last_change":"2024-04-26T21:12:01.769551+0000","last_active":"2024-04-26T21:12:04.075598+0000","last_peered":"2024-04-26T21:12:04.075598+0000","last_clean":"2024-04-26T21:12:04.075598+0000","last_became_active":"2024-04-26T21:12:01.769269+0000","last_became_peered":"2024-04-26T21:12:01.769269+0000","last_unstale":"2024-04-26T21:12:04.075598+0000","last_undegraded":"2024-04-26T21:12:04.075598+0000","last_fullsized":"2024-04-26T21:12:04.075598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762961+0000","last_change":"2024-04-26T21:12:00.746817+0000","last_active":"2024-04-26T21:12:02.762961+0000","last_peered":"2024-04-26T21:12:02.762961+0000","last_clean":"2024-04-26T21:12:02.762961+0000","last_became_active":"2024-04-26T21:12:00.746513+0000","last_became_peered":"2024-04-26T21:12:00.746513+0000","last_unstale":"2024-04-26T21:12:02.762961+0000","last_undegraded":"2024-04-26T21:12:02.762961+0000","last_fullsized":"2024-04-26T21:12:02.762961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075831+0000","last_change":"2024-04-26T21:12:01.772333+0000","last_active":"2024-04-26T21:12:04.075831+0000","last_peered":"2024-04-26T21:12:04.075831+0000","last_clean":"2024-04-26T21:12:04.075831+0000","last_became_active":"2024-04-26T21:12:01.771703+0000","last_became_peered":"2024-04-26T21:12:01.771703+0000","last_unstale":"2024-04-26T21:12:04.075831+0000","last_undegraded":"2024-04-26T21:12:04.075831+0000","last_fullsized":"2024-04-26T21:12:04.075831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:39.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768273+0000","last_change":"2024-04-26T21:12:00.742717+0000","last_active":"2024-04-26T21:12:02.768273+0000","last_peered":"2024-04-26T21:12:02.768273+0000","last_clean":"2024-04-26T21:12:02.768273+0000","last_became_active":"2024-04-26T21:12:00.741757+0000","last_became_peered":"2024-04-26T21:12:00.741757+0000","last_unstale":"2024-04-26T21:12:02.768273+0000","last_undegraded":"2024-04-26T21:12:02.768273+0000","last_fullsized":"2024-04-26T21:12:02.768273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:33.222015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176261+0000","last_change":"2024-04-26T21:12:01.761365+0000","last_active":"2024-04-26T21:12:04.176261+0000","last_peered":"2024-04-26T21:12:04.176261+0000","last_clean":"2024-04-26T21:12:04.176261+0000","last_became_active":"2024-04-26T21:12:01.760340+0000","last_became_peered":"2024-04-26T21:12:01.760340+0000","last_unstale":"2024-04-26T21:12:04.176261+0000","last_undegraded":"2024-04-26T21:12:04.176261+0000","last_fullsized":"2024-04-26T21:12:04.176261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:48.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762895+0000","last_change":"2024-04-26T21:12:00.744678+0000","last_active":"2024-04-26T21:12:02.762895+0000","last_peered":"2024-04-26T21:12:02.762895+0000","last_clean":"2024-04-26T21:12:02.762895+0000","last_became_active":"2024-04-26T21:12:00.744326+0000","last_became_peered":"2024-04-26T21:12:00.744326+0000","last_unstale":"2024-04-26T21:12:02.762895+0000","last_undegraded":"2024-04-26T21:12:02.762895+0000","last_fullsized":"2024-04-26T21:12:02.762895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075479+0000","last_change":"2024-04-26T21:12:01.776212+0000","last_active":"2024-04-26T21:12:04.075479+0000","last_peered":"2024-04-26T21:12:04.075479+0000","last_clean":"2024-04-26T21:12:04.075479+0000","last_became_active":"2024-04-26T21:12:01.775808+0000","last_became_peered":"2024-04-26T21:12:01.775808+0000","last_unstale":"2024-04-26T21:12:04.075479+0000","last_undegraded":"2024-04-26T21:12:04.075479+0000","last_fullsized":"2024-04-26T21:12:04.075479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:39.514161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075131+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:04.075131+0000","last_peered":"2024-04-26T21:12:04.075131+0000","last_clean":"2024-04-26T21:12:04.075131+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:04.075131+0000","last_undegraded":"2024-04-26T21:12:04.075131+0000","last_fullsized":"2024-04-26T21:12:04.075131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175661+0000","last_change":"2024-04-26T21:12:00.748029+0000","last_active":"2024-04-26T21:12:04.175661+0000","last_peered":"2024-04-26T21:12:04.175661+0000","last_clean":"2024-04-26T21:12:04.175661+0000","last_became_active":"2024-04-26T21:12:00.745973+0000","last_became_peered":"2024-04-26T21:12:00.745973+0000","last_unstale":"2024-04-26T21:12:04.175661+0000","last_undegraded":"2024-04-26T21:12:04.175661+0000","last_fullsized":"2024-04-26T21:12:04.175661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:15.207354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767741+0000","last_change":"2024-04-26T21:12:01.757426+0000","last_active":"2024-04-26T21:12:02.767741+0000","last_peered":"2024-04-26T21:12:02.767741+0000","last_clean":"2024-04-26T21:12:02.767741+0000","last_became_active":"2024-04-26T21:12:01.757023+0000","last_became_peered":"2024-04-26T21:12:01.757023+0000","last_unstale":"2024-04-26T21:12:02.767741+0000","last_undegraded":"2024-04-26T21:12:02.767741+0000","last_fullsized":"2024-04-26T21:12:02.767741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074973+0000","last_change":"2024-04-26T21:12:00.753065+0000","last_active":"2024-04-26T21:12:04.074973+0000","last_peered":"2024-04-26T21:12:04.074973+0000","last_clean":"2024-04-26T21:12:04.074973+0000","last_became_active":"2024-04-26T21:12:00.752748+0000","last_became_peered":"2024-04-26T21:12:00.752748+0000","last_unstale":"2024-04-26T21:12:04.074973+0000","last_undegraded":"2024-04-26T21:12:04.074973+0000","last_fullsized":"2024-04-26T21:12:04.074973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175510+0000","last_change":"2024-04-26T21:12:01.763521+0000","last_active":"2024-04-26T21:12:04.175510+0000","last_peered":"2024-04-26T21:12:04.175510+0000","last_clean":"2024-04-26T21:12:04.175510+0000","last_became_active":"2024-04-26T21:12:01.763203+0000","last_became_peered":"2024-04-26T21:12:01.763203+0000","last_unstale":"2024-04-26T21:12:04.175510+0000","last_undegraded":"2024-04-26T21:12:04.175510+0000","last_fullsized":"2024-04-26T21:12:04.175510+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.762822+0000","last_change":"2024-04-26T21:12:00.748050+0000","last_active":"2024-04-26T21:12:02.762822+0000","last_peered":"2024-04-26T21:12:02.762822+0000","last_clean":"2024-04-26T21:12:02.762822+0000","last_became_active":"2024-04-26T21:12:00.747742+0000","last_became_peered":"2024-04-26T21:12:00.747742+0000","last_unstale":"2024-04-26T21:12:02.762822+0000","last_undegraded":"2024-04-26T21:12:02.762822+0000","last_fullsized":"2024-04-26T21:12:02.762822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:11.263565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767742+0000","last_change":"2024-04-26T21:12:01.758217+0000","last_active":"2024-04-26T21:12:02.767742+0000","last_peered":"2024-04-26T21:12:02.767742+0000","last_clean":"2024-04-26T21:12:02.767742+0000","last_became_active":"2024-04-26T21:12:01.756525+0000","last_became_peered":"2024-04-26T21:12:01.756525+0000","last_unstale":"2024-04-26T21:12:02.767742+0000","last_undegraded":"2024-04-26T21:12:02.767742+0000","last_fullsized":"2024-04-26T21:12:02.767742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:17.830865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176252+0000","last_change":"2024-04-26T21:12:00.748407+0000","last_active":"2024-04-26T21:12:04.176252+0000","last_peered":"2024-04-26T21:12:04.176252+0000","last_clean":"2024-04-26T21:12:04.176252+0000","last_became_active":"2024-04-26T21:12:00.748119+0000","last_became_peered":"2024-04-26T21:12:00.748119+0000","last_unstale":"2024-04-26T21:12:04.176252+0000","last_undegraded":"2024-04-26T21:12:04.176252+0000","last_fullsized":"2024-04-26T21:12:04.176252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974416+0000","last_change":"2024-04-26T21:12:01.769283+0000","last_active":"2024-04-26T21:12:02.974416+0000","last_peered":"2024-04-26T21:12:02.974416+0000","last_clean":"2024-04-26T21:12:02.974416+0000","last_became_active":"2024-04-26T21:12:01.769021+0000","last_became_peered":"2024-04-26T21:12:01.769021+0000","last_unstale":"2024-04-26T21:12:02.974416+0000","last_undegraded":"2024-04-26T21:12:02.974416+0000","last_fullsized":"2024-04-26T21:12:02.974416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:57.003120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768484+0000","last_change":"2024-04-26T21:12:00.745395+0000","last_active":"2024-04-26T21:12:02.768484+0000","last_peered":"2024-04-26T21:12:02.768484+0000","last_clean":"2024-04-26T21:12:02.768484+0000","last_became_active":"2024-04-26T21:12:00.745088+0000","last_became_peered":"2024-04-26T21:12:00.745088+0000","last_unstale":"2024-04-26T21:12:02.768484+0000","last_undegraded":"2024-04-26T21:12:02.768484+0000","last_fullsized":"2024-04-26T21:12:02.768484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:29.139093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176129+0000","last_change":"2024-04-26T21:12:01.755545+0000","last_active":"2024-04-26T21:12:04.176129+0000","last_peered":"2024-04-26T21:12:04.176129+0000","last_clean":"2024-04-26T21:12:04.176129+0000","last_became_active":"2024-04-26T21:12:01.755225+0000","last_became_peered":"2024-04-26T21:12:01.755225+0000","last_unstale":"2024-04-26T21:12:04.176129+0000","last_undegraded":"2024-04-26T21:12:04.176129+0000","last_fullsized":"2024-04-26T21:12:04.176129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:53:34.514332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075714+0000","last_change":"2024-04-26T21:12:00.753051+0000","last_active":"2024-04-26T21:12:04.075714+0000","last_peered":"2024-04-26T21:12:04.075714+0000","last_clean":"2024-04-26T21:12:04.075714+0000","last_became_active":"2024-04-26T21:12:00.752700+0000","last_became_peered":"2024-04-26T21:12:00.752700+0000","last_unstale":"2024-04-26T21:12:04.075714+0000","last_undegraded":"2024-04-26T21:12:04.075714+0000","last_fullsized":"2024-04-26T21:12:04.075714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:15.469175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768539+0000","last_change":"2024-04-26T21:12:01.755969+0000","last_active":"2024-04-26T21:12:02.768539+0000","last_peered":"2024-04-26T21:12:02.768539+0000","last_clean":"2024-04-26T21:12:02.768539+0000","last_became_active":"2024-04-26T21:12:01.755628+0000","last_became_peered":"2024-04-26T21:12:01.755628+0000","last_unstale":"2024-04-26T21:12:02.768539+0000","last_undegraded":"2024-04-26T21:12:02.768539+0000","last_fullsized":"2024-04-26T21:12:02.768539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:42:31.761499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768360+0000","last_change":"2024-04-26T21:12:00.740944+0000","last_active":"2024-04-26T21:12:02.768360+0000","last_peered":"2024-04-26T21:12:02.768360+0000","last_clean":"2024-04-26T21:12:02.768360+0000","last_became_active":"2024-04-26T21:12:00.740532+0000","last_became_peered":"2024-04-26T21:12:00.740532+0000","last_unstale":"2024-04-26T21:12:02.768360+0000","last_undegraded":"2024-04-26T21:12:02.768360+0000","last_fullsized":"2024-04-26T21:12:02.768360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:16.536321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767968+0000","last_change":"2024-04-26T21:12:01.759020+0000","last_active":"2024-04-26T21:12:02.767968+0000","last_peered":"2024-04-26T21:12:02.767968+0000","last_clean":"2024-04-26T21:12:02.767968+0000","last_became_active":"2024-04-26T21:12:01.758730+0000","last_became_peered":"2024-04-26T21:12:01.758730+0000","last_unstale":"2024-04-26T21:12:02.767968+0000","last_undegraded":"2024-04-26T21:12:02.767968+0000","last_fullsized":"2024-04-26T21:12:02.767968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074939+0000","last_change":"2024-04-26T21:12:00.750411+0000","last_active":"2024-04-26T21:12:03.074939+0000","last_peered":"2024-04-26T21:12:03.074939+0000","last_clean":"2024-04-26T21:12:03.074939+0000","last_became_active":"2024-04-26T21:12:00.749721+0000","last_became_peered":"2024-04-26T21:12:00.749721+0000","last_unstale":"2024-04-26T21:12:03.074939+0000","last_undegraded":"2024-04-26T21:12:03.074939+0000","last_fullsized":"2024-04-26T21:12:03.074939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:37.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176063+0000","last_change":"2024-04-26T21:12:01.758288+0000","last_active":"2024-04-26T21:12:04.176063+0000","last_peered":"2024-04-26T21:12:04.176063+0000","last_clean":"2024-04-26T21:12:04.176063+0000","last_became_active":"2024-04-26T21:12:01.757976+0000","last_became_peered":"2024-04-26T21:12:01.757976+0000","last_unstale":"2024-04-26T21:12:04.176063+0000","last_undegraded":"2024-04-26T21:12:04.176063+0000","last_fullsized":"2024-04-26T21:12:04.176063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:33.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.074967+0000","last_change":"2024-04-26T21:12:00.753711+0000","last_active":"2024-04-26T21:12:04.074967+0000","last_peered":"2024-04-26T21:12:04.074967+0000","last_clean":"2024-04-26T21:12:04.074967+0000","last_became_active":"2024-04-26T21:12:00.753386+0000","last_became_peered":"2024-04-26T21:12:00.753386+0000","last_unstale":"2024-04-26T21:12:04.074967+0000","last_undegraded":"2024-04-26T21:12:04.074967+0000","last_fullsized":"2024-04-26T21:12:04.074967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:37.631523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.974333+0000","last_change":"2024-04-26T21:12:01.764584+0000","last_active":"2024-04-26T21:12:02.974333+0000","last_peered":"2024-04-26T21:12:02.974333+0000","last_clean":"2024-04-26T21:12:02.974333+0000","last_became_active":"2024-04-26T21:12:01.764328+0000","last_became_peered":"2024-04-26T21:12:01.764328+0000","last_unstale":"2024-04-26T21:12:02.974333+0000","last_undegraded":"2024-04-26T21:12:02.974333+0000","last_fullsized":"2024-04-26T21:12:02.974333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:38.743981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768356+0000","last_change":"2024-04-26T21:12:00.749992+0000","last_active":"2024-04-26T21:12:02.768356+0000","last_peered":"2024-04-26T21:12:02.768356+0000","last_clean":"2024-04-26T21:12:02.768356+0000","last_became_active":"2024-04-26T21:12:00.749601+0000","last_became_peered":"2024-04-26T21:12:00.749601+0000","last_unstale":"2024-04-26T21:12:02.768356+0000","last_undegraded":"2024-04-26T21:12:02.768356+0000","last_fullsized":"2024-04-26T21:12:02.768356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:08:39.587954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767538+0000","last_change":"2024-04-26T21:12:01.760108+0000","last_active":"2024-04-26T21:12:02.767538+0000","last_peered":"2024-04-26T21:12:02.767538+0000","last_clean":"2024-04-26T21:12:02.767538+0000","last_became_active":"2024-04-26T21:12:01.759760+0000","last_became_peered":"2024-04-26T21:12:01.759760+0000","last_unstale":"2024-04-26T21:12:02.767538+0000","last_undegraded":"2024-04-26T21:12:02.767538+0000","last_fullsized":"2024-04-26T21:12:02.767538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768336+0000","last_change":"2024-04-26T21:12:00.747054+0000","last_active":"2024-04-26T21:12:02.768336+0000","last_peered":"2024-04-26T21:12:02.768336+0000","last_clean":"2024-04-26T21:12:02.768336+0000","last_became_active":"2024-04-26T21:12:00.746691+0000","last_became_peered":"2024-04-26T21:12:00.746691+0000","last_unstale":"2024-04-26T21:12:02.768336+0000","last_undegraded":"2024-04-26T21:12:02.768336+0000","last_fullsized":"2024-04-26T21:12:02.768336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:28.714030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767872+0000","last_change":"2024-04-26T21:12:01.768901+0000","last_active":"2024-04-26T21:12:02.767872+0000","last_peered":"2024-04-26T21:12:02.767872+0000","last_clean":"2024-04-26T21:12:02.767872+0000","last_became_active":"2024-04-26T21:12:01.768435+0000","last_became_peered":"2024-04-26T21:12:01.768435+0000","last_unstale":"2024-04-26T21:12:02.767872+0000","last_undegraded":"2024-04-26T21:12:02.767872+0000","last_fullsized":"2024-04-26T21:12:02.767872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:46.260074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075403+0000","last_change":"2024-04-26T21:12:00.752713+0000","last_active":"2024-04-26T21:12:04.075403+0000","last_peered":"2024-04-26T21:12:04.075403+0000","last_clean":"2024-04-26T21:12:04.075403+0000","last_became_active":"2024-04-26T21:12:00.752436+0000","last_became_peered":"2024-04-26T21:12:00.752436+0000","last_unstale":"2024-04-26T21:12:04.075403+0000","last_undegraded":"2024-04-26T21:12:04.075403+0000","last_fullsized":"2024-04-26T21:12:04.075403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:55.994430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:03.074723+0000","last_change":"2024-04-26T21:12:01.760741+0000","last_active":"2024-04-26T21:12:03.074723+0000","last_peered":"2024-04-26T21:12:03.074723+0000","last_clean":"2024-04-26T21:12:03.074723+0000","last_became_active":"2024-04-26T21:12:01.760445+0000","last_became_peered":"2024-04-26T21:12:01.760445+0000","last_unstale":"2024-04-26T21:12:03.074723+0000","last_undegraded":"2024-04-26T21:12:03.074723+0000","last_fullsized":"2024-04-26T21:12:03.074723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:31.183983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175995+0000","last_change":"2024-04-26T21:12:01.768719+0000","last_active":"2024-04-26T21:12:04.175995+0000","last_peered":"2024-04-26T21:12:04.175995+0000","last_clean":"2024-04-26T21:12:04.175995+0000","last_became_active":"2024-04-26T21:12:01.767988+0000","last_became_peered":"2024-04-26T21:12:01.767988+0000","last_unstale":"2024-04-26T21:12:04.175995+0000","last_undegraded":"2024-04-26T21:12:04.175995+0000","last_fullsized":"2024-04-26T21:12:04.175995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075331+0000","last_change":"2024-04-26T21:12:00.751897+0000","last_active":"2024-04-26T21:12:04.075331+0000","last_peered":"2024-04-26T21:12:04.075331+0000","last_clean":"2024-04-26T21:12:04.075331+0000","last_became_active":"2024-04-26T21:12:00.751349+0000","last_became_peered":"2024-04-26T21:12:00.751349+0000","last_unstale":"2024-04-26T21:12:04.075331+0000","last_undegraded":"2024-04-26T21:12:04.075331+0000","last_fullsized":"2024-04-26T21:12:04.075331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:14.805951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176188+0000","last_change":"2024-04-26T21:12:01.765872+0000","last_active":"2024-04-26T21:12:04.176188+0000","last_peered":"2024-04-26T21:12:04.176188+0000","last_clean":"2024-04-26T21:12:04.176188+0000","last_became_active":"2024-04-26T21:12:01.765595+0000","last_became_peered":"2024-04-26T21:12:01.765595+0000","last_unstale":"2024-04-26T21:12:04.176188+0000","last_undegraded":"2024-04-26T21:12:04.176188+0000","last_fullsized":"2024-04-26T21:12:04.176188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:24.252689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.075571+0000","last_change":"2024-04-26T21:12:00.756041+0000","last_active":"2024-04-26T21:12:04.075571+0000","last_peered":"2024-04-26T21:12:04.075571+0000","last_clean":"2024-04-26T21:12:04.075571+0000","last_became_active":"2024-04-26T21:12:00.755745+0000","last_became_peered":"2024-04-26T21:12:00.755745+0000","last_unstale":"2024-04-26T21:12:04.075571+0000","last_undegraded":"2024-04-26T21:12:04.075571+0000","last_fullsized":"2024-04-26T21:12:04.075571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:44.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.175980+0000","last_change":"2024-04-26T21:12:01.757210+0000","last_active":"2024-04-26T21:12:04.175980+0000","last_peered":"2024-04-26T21:12:04.175980+0000","last_clean":"2024-04-26T21:12:04.175980+0000","last_became_active":"2024-04-26T21:12:01.756939+0000","last_became_peered":"2024-04-26T21:12:01.756939+0000","last_unstale":"2024-04-26T21:12:04.175980+0000","last_undegraded":"2024-04-26T21:12:04.175980+0000","last_fullsized":"2024-04-26T21:12:04.175980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:34.117304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.768099+0000","last_change":"2024-04-26T21:12:00.745219+0000","last_active":"2024-04-26T21:12:02.768099+0000","last_peered":"2024-04-26T21:12:02.768099+0000","last_clean":"2024-04-26T21:12:02.768099+0000","last_became_active":"2024-04-26T21:12:00.744925+0000","last_became_peered":"2024-04-26T21:12:00.744925+0000","last_unstale":"2024-04-26T21:12:02.768099+0000","last_undegraded":"2024-04-26T21:12:02.768099+0000","last_fullsized":"2024-04-26T21:12:02.768099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:02.767946+0000","last_change":"2024-04-26T21:12:01.763995+0000","last_active":"2024-04-26T21:12:02.767946+0000","last_peered":"2024-04-26T21:12:02.767946+0000","last_clean":"2024-04-26T21:12:02.767946+0000","last_became_active":"2024-04-26T21:12:01.762167+0000","last_became_peered":"2024-04-26T21:12:01.762167+0000","last_unstale":"2024-04-26T21:12:02.767946+0000","last_undegraded":"2024-04-26T21:12:02.767946+0000","last_fullsized":"2024-04-26T21:12:02.767946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:00.722397+0000","last_clean_scrub_stamp":"2024-04-26T21:12:00.722397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:37.928240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T21:12:04.176091+0000","last_change":"2024-04-26T21:12:00.753974+0000","last_active":"2024-04-26T21:12:04.176091+0000","last_peered":"2024-04-26T21:12:04.176091+0000","last_clean":"2024-04-26T21:12:04.176091+0000","last_became_active":"2024-04-26T21:12:00.753429+0000","last_became_peered":"2024-04-26T21:12:00.753429+0000","last_unstale":"2024-04-26T21:12:04.176091+0000","last_undegraded":"2024-04-26T21:12:04.176091+0000","last_fullsized":"2024-04-26T21:12:04.176091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:59.713707+0000","last_clean_scrub_stamp":"2024-04-26T21:11:59.713707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:54.300935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":81325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":672241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:14.029 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:12:14.029 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:12:14.029 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:12:14.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:12:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.164+0000 7f5c0e9b7640 1 Processor -- start 2024-04-26T21:12:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.164+0000 7f5c0e9b7640 1 -- start start 2024-04-26T21:12:14.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.164+0000 7f5c0e9b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5c080b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.164+0000 7f5c0e9b7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c080b4b10 con 0x7f5c080d3790 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.164+0000 7f5c0e9b7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c080d40c0 con 0x7f5c08162ad0 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c0e9b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c080b4db0 con 0x7f5c080d3c10 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5c080d3790 0x7f5c080b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58608/0 (socket says 172.21.15.40:58608) 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c077fe640 1 -- 172.21.15.40:0/170813073 learned_addr learned my addr 172.21.15.40:0/170813073 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c0cf54640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5c080b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924074076 0 0) 0x7f5c080b4b10 con 0x7f5c080d3790 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bf0003520 con 0x7f5c080d3790 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3837933331 0 0) 0x7f5bf0003520 con 0x7f5c080d3790 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 msgr2=0x7f5c080b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5c080b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 >> v1:172.21.15.184:6789/0 conn(0x7f5c08162ad0 legacy=0x7f5c080d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c080b5090 con 0x7f5c080d3790 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5bfc001670 con 0x7f5c080d3790 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 >> v1:172.21.15.40:6789/0 conn(0x7f5c080d3790 legacy=0x7f5c080b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bf00039e0 0x7f5bf0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf0004390 0x7f5bf0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5bf00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080b4db0 con 0x7f5bf00039e0 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf000b9f0 con 0x7f5bf0004390 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf000bbf0 con 0x7f5c080d3c10 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c07fff640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bf00039e0 0x7f5bf0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c077fe640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf0004390 0x7f5bf0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.165+0000 7f5c0cf54640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5bf00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 -- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 msgr2=0x7f5bf00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c080d3c10 0x7f5bf00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 -- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf0004390 msgr2=0x7f5bf0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf0004390 0x7f5bf0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 -- 172.21.15.40:0/170813073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf0003520 con 0x7f5bf00039e0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c077fe640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf0004390 0x7f5bf0004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c07fff640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bf00039e0 0x7f5bf0003dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8001830 tx=0x7f5bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49289e0feea13244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8001430 con 0x7f5bf00039e0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf80052f0 con 0x7f5bf00039e0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.166+0000 7f5c06ffd640 1 -- 172.21.15.40:0/170813073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf802ac10 con 0x7f5bf00039e0 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/170813073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bf00039e0 msgr2=0x7f5bf0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/170813073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bf00039e0 0x7f5bf0003dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8001830 tx=0x7f5bf0005830 comp rx=0 tx=0).stop 2024-04-26T21:12:14.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0cf54640 1 -- 172.21.15.40:0/170813073 reap_dead start 2024-04-26T21:12:14.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/170813073 shutdown_connections 2024-04-26T21:12:14.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/170813073 >> 172.21.15.40:0/170813073 conn(0x7f5c0809e510 msgr2=0x7f5c080d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/170813073 shutdown_connections 2024-04-26T21:12:14.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.167+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/170813073 wait complete. 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.168+0000 7f5c0e9b7640 1 Processor -- start 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.168+0000 7f5c0e9b7640 1 -- start start 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 0x7f5c080c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c081559c0 0x7f5c080cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080d34b0 con 0x7f5c080cce20 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080b5650 con 0x7f5c081555e0 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0e9b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c080d1040 con 0x7f5c081559c0 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0cf54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0cf54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40686/0 (socket says 172.21.15.40:40686) 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c0cf54640 1 -- 172.21.15.40:0/1032540686 learned_addr learned my addr 172.21.15.40:0/1032540686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c077fe640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c081559c0 0x7f5c080cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.169+0000 7f5c07fff640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 0x7f5c080c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 -- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c081559c0 msgr2=0x7f5c080cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c081559c0 0x7f5c080cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 -- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 msgr2=0x7f5c080c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 0x7f5c080c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 -- 172.21.15.40:0/1032540686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c080caf30 con 0x7f5c080cce20 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c07fff640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 0x7f5c080c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0cf54640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009e90 tx=0x7f5c000115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011e80 con 0x7f5c080cce20 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0001b070 con 0x7f5c080cce20 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c080cb210 con 0x7f5c080cce20 2024-04-26T21:12:14.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00003990 con 0x7f5c080cce20 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.170+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c080cb7d0 con 0x7f5c080cce20 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.171+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0806a290 con 0x7f5c080cce20 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.171+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c00005d10 con 0x7f5c080cce20 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.172+0000 7f5c04ff9640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 0x7f5bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.172+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5c0008bf80 con 0x7f5c080cce20 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.172+0000 7f5c07fff640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 0x7f5bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.173+0000 7f5c07fff640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 0x7f5bf406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8032000 tx=0x7f5bf8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.176+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c00025020 con 0x7f5c080cce20 2024-04-26T21:12:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.400+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5c0815fb10 con 0x7f5c080cce20 2024-04-26T21:12:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.400+0000 7f5c04ff9640 1 -- 172.21.15.40:0/1032540686 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5c0005a440 con 0x7f5c080cce20 2024-04-26T21:12:14.401 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:14.401 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 msgr2=0x7f5bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 0x7f5bf406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8032000 tx=0x7f5bf8009030 comp rx=0 tx=0).stop 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 msgr2=0x7f5c080cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009e90 tx=0x7f5c000115a0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 shutdown_connections 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5bf406c8f0 0x7f5bf406edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c080cce20 0x7f5c080cd2a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c081559c0 0x7f5c080cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 --2- 172.21.15.40:0/1032540686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081555e0 0x7f5c080c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 >> 172.21.15.40:0/1032540686 conn(0x7f5c0809e510 msgr2=0x7f5c080d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 shutdown_connections 2024-04-26T21:12:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.402+0000 7f5c0e9b7640 1 -- 172.21.15.40:0/1032540686 wait complete. 2024-04-26T21:12:14.414 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:12:14.414 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.513+0000 7fc196223640 1 Processor -- start 2024-04-26T21:12:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.513+0000 7fc196223640 1 -- start start 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19015f370 0x7fc1900cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc19015f750 0x7fc1900b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1900d37d0 0x7fc1900d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1900b4d10 con 0x7fc19015f370 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1900d4160 con 0x7fc19015f750 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc196223640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900d03a0 con 0x7fc1900d37d0 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc18f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19015f370 0x7fc1900cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc18f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19015f370 0x7fc1900cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40690/0 (socket says 172.21.15.40:40690) 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.514+0000 7fc18f7fe640 1 -- 172.21.15.40:0/809980546 learned_addr learned my addr 172.21.15.40:0/809980546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18ffff640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1900d37d0 0x7fc1900d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18effd640 1 --2- 172.21.15.40:0/809980546 >> v2:172.21.15.184:3300/0 conn(0x7fc19015f750 0x7fc1900b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 -- 172.21.15.40:0/809980546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1900d37d0 msgr2=0x7fc1900d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1900d37d0 0x7fc1900d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 -- 172.21.15.40:0/809980546 >> v2:172.21.15.184:3300/0 conn(0x7fc19015f750 msgr2=0x7fc1900b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/809980546 >> v2:172.21.15.184:3300/0 conn(0x7fc19015f750 0x7fc1900b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 -- 172.21.15.40:0/809980546 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1900d0680 con 0x7fc19015f370 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f370 0x7fc1900cfd70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1900d02b0 tx=0x7fc18002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c69747bed669c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.515+0000 7fc18e7fc640 1 -- 172.21.15.40:0/809980546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180037070 con 0x7fc19015f370 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc18e7fc640 1 -- 172.21.15.40:0/809980546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc180032440 con 0x7fc19015f370 2024-04-26T21:12:14.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc18e7fc640 1 -- 172.21.15.40:0/809980546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180032810 con 0x7fc19015f370 2024-04-26T21:12:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 -- 172.21.15.40:0/809980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f370 msgr2=0x7fc1900cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f370 0x7fc1900cfd70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1900d02b0 tx=0x7fc18002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 -- 172.21.15.40:0/809980546 shutdown_connections 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1900d37d0 0x7fc1900d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 --2- 172.21.15.40:0/809980546 >> v2:172.21.15.184:3300/0 conn(0x7fc19015f750 0x7fc1900b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 --2- 172.21.15.40:0/809980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f370 0x7fc1900cfd70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 -- 172.21.15.40:0/809980546 >> 172.21.15.40:0/809980546 conn(0x7fc19008ec10 msgr2=0x7fc19009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 -- 172.21.15.40:0/809980546 shutdown_connections 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.516+0000 7fc196223640 1 -- 172.21.15.40:0/809980546 wait complete. 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.517+0000 7fc196223640 1 Processor -- start 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.517+0000 7fc196223640 1 -- start start 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1900d37d0 0x7fc19015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 0x7fc19015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900d0f20 con 0x7fc19015f750 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900d0c40 con 0x7fc1900d37d0 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc196223640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900d1bd0 con 0x7fc19015f370 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc18ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc18ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40692/0 (socket says 172.21.15.40:40692) 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc18ffff640 1 -- 172.21.15.40:0/4076797999 learned_addr learned my addr 172.21.15.40:0/4076797999 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc18effd640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 0x7fc19015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.518+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1900d37d0 0x7fc19015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 -- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 msgr2=0x7fc19015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 0x7fc19015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 -- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1900d37d0 msgr2=0x7fc19015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1900d37d0 0x7fc19015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 -- 172.21.15.40:0/4076797999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1900b7df0 con 0x7fc19015f750 2024-04-26T21:12:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18effd640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 0x7fc19015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc18ffff640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc188007e60 tx=0x7fc18800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188014620 con 0x7fc19015f750 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc188019070 con 0x7fc19015f750 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1900b80d0 con 0x7fc19015f750 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188014a20 con 0x7fc19015f750 2024-04-26T21:12:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.519+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1900b8690 con 0x7fc19015f750 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.520+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19006a290 con 0x7fc19015f750 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.521+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc188014bc0 con 0x7fc19015f750 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.521+0000 7fc177fff640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.521+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc18808c350 con 0x7fc19015f750 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.521+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.522+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc180034000 tx=0x7fc18002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.525+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc18805a780 con 0x7fc19015f750 2024-04-26T21:12:14.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.709+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1900d1550 con 0x7fc19015f750 2024-04-26T21:12:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.710+0000 7fc177fff640 1 -- 172.21.15.40:0/4076797999 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7fc18805a080 con 0x7fc19015f750 2024-04-26T21:12:14.712 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:14.712 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 msgr2=0x7fc17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc180034000 tx=0x7fc18002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 msgr2=0x7fc1900cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc188007e60 tx=0x7fc18800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 shutdown_connections 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19015f750 0x7fc1900cccd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19015f370 0x7fc19015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 --2- 172.21.15.40:0/4076797999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1900d37d0 0x7fc19015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 >> 172.21.15.40:0/4076797999 conn(0x7fc19008ec10 msgr2=0x7fc19015e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 shutdown_connections 2024-04-26T21:12:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.712+0000 7fc196223640 1 -- 172.21.15.40:0/4076797999 wait complete. 2024-04-26T21:12:14.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:14.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.829+0000 7faa1277f640 1 Processor -- start 2024-04-26T21:12:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.829+0000 7faa1277f640 1 -- start start 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa1277f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 0x7faa0c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa1277f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa0c0d0f40 0x7faa0c0d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa1277f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c0b6aa0 con 0x7faa0c15f5b0 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa1277f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0d3eb0 con 0x7faa0c0d0f40 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa1277f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c0d4190 con 0x7faa0c15f990 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa10d1c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa0c0d0f40 0x7faa0c0d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 0x7faa0c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa0bfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7faa0c15f5b0 0x7faa0c0b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49464/0 (socket says 172.21.15.40:49464) 2024-04-26T21:12:14.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa0bfff640 1 -- 172.21.15.40:0/1902841199 learned_addr learned my addr 172.21.15.40:0/1902841199 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.830+0000 7faa0affd640 1 -- 172.21.15.40:0/1902841199 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591240646 0 0) 0x7faa0c0b6aa0 con 0x7faa0c15f5b0 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa0affd640 1 -- 172.21.15.40:0/1902841199 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9f4003520 con 0x7faa0c15f5b0 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa10d1c640 1 -- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 msgr2=0x7faa0c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa10d1c640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 0x7faa0c0d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa10d1c640 1 -- 172.21.15.40:0/1902841199 >> v1:172.21.15.184:6789/0 conn(0x7faa0c15f5b0 legacy=0x7faa0c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa10d1c640 1 -- 172.21.15.40:0/1902841199 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0d0200 con 0x7faa0c0d0f40 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa0b7fe640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 0x7faa0c0d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.831+0000 7faa10d1c640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c0d0f40 0x7faa0c0d3340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa04009d90 tx=0x7faa0402a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b96179927af18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa0affd640 1 -- 172.21.15.40:0/1902841199 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04007740 con 0x7faa0c0d0f40 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa0affd640 1 -- 172.21.15.40:0/1902841199 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa040078e0 con 0x7faa0c0d0f40 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa0affd640 1 -- 172.21.15.40:0/1902841199 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04007c90 con 0x7faa0c0d0f40 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 -- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c0d0f40 msgr2=0x7faa0c0d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c0d0f40 0x7faa0c0d3340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa04009d90 tx=0x7faa0402a530 comp rx=0 tx=0).stop 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 -- 172.21.15.40:0/1902841199 shutdown_connections 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c0d0f40 0x7faa0c0d3340 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 --2- 172.21.15.40:0/1902841199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c15f990 0x7faa0c0d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 -- 172.21.15.40:0/1902841199 >> 172.21.15.40:0/1902841199 conn(0x7faa0c08ec40 msgr2=0x7faa0c09e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 -- 172.21.15.40:0/1902841199 shutdown_connections 2024-04-26T21:12:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.832+0000 7faa1277f640 1 -- 172.21.15.40:0/1902841199 wait complete. 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.833+0000 7faa1277f640 1 Processor -- start 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.833+0000 7faa1277f640 1 -- start start 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 0x7faa0c0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c15f990 0x7faa0c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0b5900 con 0x7faa0c15f5b0 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0b5780 con 0x7faa0c15f990 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa1277f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c0d3880 con 0x7faa0c0d0f40 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40698/0 (socket says 172.21.15.40:40698) 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa0b7fe640 1 -- 172.21.15.40:0/1455787386 learned_addr learned my addr 172.21.15.40:0/1455787386 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa10d1c640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c15f990 0x7faa0c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.834+0000 7faa0bfff640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 0x7faa0c0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 -- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 msgr2=0x7faa0c0cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 0x7faa0c0cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 -- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c15f990 msgr2=0x7faa0c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c15f990 0x7faa0c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 -- 172.21.15.40:0/1455787386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1f5100 con 0x7faa0c15f5b0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0bfff640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 0x7faa0c0cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa0b7fe640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e960 tx=0x7fa9fc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc017070 con 0x7faa0c15f5b0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc00ca00 con 0x7faa0c15f5b0 2024-04-26T21:12:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c1f53a0 con 0x7faa0c15f5b0 2024-04-26T21:12:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00cdd0 con 0x7faa0c15f5b0 2024-04-26T21:12:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.835+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c0cf090 con 0x7faa0c15f5b0 2024-04-26T21:12:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.836+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c06a290 con 0x7faa0c15f5b0 2024-04-26T21:12:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.836+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9fc01a070 con 0x7faa0c15f5b0 2024-04-26T21:12:14.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.837+0000 7faa08ff9640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:14.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.837+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa9fc08b8d0 con 0x7faa0c15f5b0 2024-04-26T21:12:14.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.837+0000 7faa0bfff640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:14.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.837+0000 7faa0bfff640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 0x7fa9f806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa00007fd0 tx=0x7faa00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:14.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:14.841+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9fc059740 con 0x7faa0c15f5b0 2024-04-26T21:12:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.026+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa0c0b44c0 con 0x7faa0c15f5b0 2024-04-26T21:12:15.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.027+0000 7faa08ff9640 1 -- 172.21.15.40:0/1455787386 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fa9fc05da40 con 0x7faa0c15f5b0 2024-04-26T21:12:15.028 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 msgr2=0x7fa9f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 0x7fa9f806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa00007fd0 tx=0x7faa00009040 comp rx=0 tx=0).stop 2024-04-26T21:12:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 msgr2=0x7faa0c15e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e960 tx=0x7fa9fc00ee20 comp rx=0 tx=0).stop 2024-04-26T21:12:15.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 shutdown_connections 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa0c15f990 0x7faa0c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa0c15f5b0 0x7faa0c15e450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 --2- 172.21.15.40:0/1455787386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa0c0d0f40 0x7faa0c0cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 >> 172.21.15.40:0/1455787386 conn(0x7faa0c08ec40 msgr2=0x7faa0c163ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 shutdown_connections 2024-04-26T21:12:15.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.029+0000 7faa1277f640 1 -- 172.21.15.40:0/1455787386 wait complete. 2024-04-26T21:12:15.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.143+0000 7fa539462640 1 Processor -- start 2024-04-26T21:12:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.143+0000 7fa539462640 1 -- start start 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa539462640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 0x7fa53409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa539462640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5340d1940 0x7fa5340702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa539462640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5340cee60 con 0x7fa53415d340 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa539462640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa5340ce9d0 con 0x7fa5340d1940 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa539462640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340d0ff0 con 0x7fa5340b2420 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa532ffd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa53415d340 0x7fa5340d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49468/0 (socket says 172.21.15.40:49468) 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa5337fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5340d1940 0x7fa5340702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa532ffd640 1 -- 172.21.15.40:0/650368877 learned_addr learned my addr 172.21.15.40:0/650368877 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.144+0000 7fa5327fc640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 0x7fa53409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa531ffb640 1 -- 172.21.15.40:0/650368877 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109147066 0 0) 0x7fa5340cee60 con 0x7fa53415d340 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa531ffb640 1 -- 172.21.15.40:0/650368877 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa520003520 con 0x7fa53415d340 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5337fe640 1 -- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 msgr2=0x7fa53409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5337fe640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 0x7fa53409c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5337fe640 1 -- 172.21.15.40:0/650368877 >> v1:172.21.15.184:6789/0 conn(0x7fa53415d340 legacy=0x7fa5340d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5337fe640 1 -- 172.21.15.40:0/650368877 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340d0d10 con 0x7fa5340d1940 2024-04-26T21:12:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5327fc640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 0x7fa53409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.145+0000 7fa5337fe640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa5340702a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa524009d90 tx=0x7fa52402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7e1f5e9287da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa531ffb640 1 -- 172.21.15.40:0/650368877 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524007740 con 0x7fa5340d1940 2024-04-26T21:12:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa531ffb640 1 -- 172.21.15.40:0/650368877 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240078e0 con 0x7fa5340d1940 2024-04-26T21:12:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa531ffb640 1 -- 172.21.15.40:0/650368877 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524007c90 con 0x7fa5340d1940 2024-04-26T21:12:15.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 -- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 msgr2=0x7fa5340702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa5340702a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa524009d90 tx=0x7fa52402a660 comp rx=0 tx=0).stop 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 -- 172.21.15.40:0/650368877 shutdown_connections 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa5340702a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 --2- 172.21.15.40:0/650368877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340b2420 0x7fa53409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 -- 172.21.15.40:0/650368877 >> 172.21.15.40:0/650368877 conn(0x7fa53409ba40 msgr2=0x7fa53409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 -- 172.21.15.40:0/650368877 shutdown_connections 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.146+0000 7fa539462640 1 -- 172.21.15.40:0/650368877 wait complete. 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.147+0000 7fa539462640 1 Processor -- start 2024-04-26T21:12:15.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 -- start start 2024-04-26T21:12:15.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa534156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340a0040 0x7fa5341f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340b2090 con 0x7fa53415d340 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa53415d720 con 0x7fa5340d1940 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa539462640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340b2210 con 0x7fa5340a0040 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa5327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa5327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40702/0 (socket says 172.21.15.40:40702) 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.148+0000 7fa5327fc640 1 -- 172.21.15.40:0/3935500896 learned_addr learned my addr 172.21.15.40:0/3935500896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5337fe640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340a0040 0x7fa5341f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa532ffd640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa534156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 -- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340a0040 msgr2=0x7fa5341f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340a0040 0x7fa5341f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 -- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 msgr2=0x7fa534156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa534156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 -- 172.21.15.40:0/3935500896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340b7870 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa532ffd640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa534156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa5327fc640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e960 tx=0x7fa51c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c017070 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c00ca00 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5341f33d0 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cdd0 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.149+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5341f7770 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.150+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5340693d0 con 0x7fa53415d340 2024-04-26T21:12:15.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.151+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa51c01a070 con 0x7fa53415d340 2024-04-26T21:12:15.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.151+0000 7fa52b7fe640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 0x7fa51406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.151+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa51c08ba30 con 0x7fa53415d340 2024-04-26T21:12:15.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.151+0000 7fa532ffd640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 0x7fa51406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.152+0000 7fa532ffd640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 0x7fa51406ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005880 tx=0x7fa52c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.156+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa51c059e70 con 0x7fa53415d340 2024-04-26T21:12:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.339+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa53415c240 con 0x7fa53415d340 2024-04-26T21:12:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.339+0000 7fa52b7fe640 1 -- 172.21.15.40:0/3935500896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa51c059770 con 0x7fa53415d340 2024-04-26T21:12:15.340 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 msgr2=0x7fa51406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 0x7fa51406ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005880 tx=0x7fa52c009040 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 msgr2=0x7fa53409fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e960 tx=0x7fa51c00ee20 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 shutdown_connections 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa51406c920 0x7fa51406ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5340a0040 0x7fa5341f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53415d340 0x7fa53409fb00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 --2- 172.21.15.40:0/3935500896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5340d1940 0x7fa534156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 >> 172.21.15.40:0/3935500896 conn(0x7fa53409ba40 msgr2=0x7fa5340ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 shutdown_connections 2024-04-26T21:12:15.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.341+0000 7fa539462640 1 -- 172.21.15.40:0/3935500896 wait complete. 2024-04-26T21:12:15.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.455+0000 7f774ab04640 1 Processor -- start 2024-04-26T21:12:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.455+0000 7f774ab04640 1 -- start start 2024-04-26T21:12:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f774ab04640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77440d3820 0x7f77440b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f774ab04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77440d3c90 con 0x7f77440b60b0 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f774ab04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77440d3f50 con 0x7f774415f680 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f774ab04640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f774415fa60 con 0x7f77440d3820 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77490a1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f77440b60b0 0x7f77440d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58622/0 (socket says 172.21.15.40:58622) 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77490a1640 1 -- 172.21.15.40:0/1254003792 learned_addr learned my addr 172.21.15.40:0/1254003792 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f7743fff640 1 --2- 172.21.15.40:0/1254003792 >> v2:172.21.15.184:3300/0 conn(0x7f77440d3820 0x7f77440b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284411199 0 0) 0x7f77440d3c90 con 0x7f77440b60b0 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7734003520 con 0x7f77440b60b0 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423613969 0 0) 0x7f77440d3f50 con 0x7f774415f680 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77440d3c90 con 0x7f774415f680 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f7743fff640 1 -- 172.21.15.40:0/1254003792 >> v1:172.21.15.184:6789/0 conn(0x7f774415f680 legacy=0x7f7744163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f7743fff640 1 -- 172.21.15.40:0/1254003792 >> v1:172.21.15.40:6789/0 conn(0x7f77440b60b0 legacy=0x7f77440d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638996865 0 0) 0x7f7734003520 con 0x7f77440b60b0 2024-04-26T21:12:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.456+0000 7f7743fff640 1 -- 172.21.15.40:0/1254003792 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77440b5090 con 0x7f77440d3820 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.457+0000 7f7743fff640 1 --2- 172.21.15.40:0/1254003792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77440d3820 0x7f77440b5b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77300097a0 tx=0x7f773002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb628b5237f21ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.457+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730037070 con 0x7f77440d3820 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.457+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773002ac50 con 0x7f77440d3820 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.457+0000 7f77437fe640 1 -- 172.21.15.40:0/1254003792 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c0019f0 con 0x7f77440d3820 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 -- 172.21.15.40:0/1254003792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77440d3820 msgr2=0x7f77440b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 --2- 172.21.15.40:0/1254003792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77440d3820 0x7f77440b5b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77300097a0 tx=0x7f773002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 -- 172.21.15.40:0/1254003792 shutdown_connections 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 --2- 172.21.15.40:0/1254003792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77440d3820 0x7f77440b5b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 -- 172.21.15.40:0/1254003792 >> 172.21.15.40:0/1254003792 conn(0x7f774408eca0 msgr2=0x7f774409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 -- 172.21.15.40:0/1254003792 shutdown_connections 2024-04-26T21:12:15.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.458+0000 7f774ab04640 1 -- 172.21.15.40:0/1254003792 wait complete. 2024-04-26T21:12:15.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.459+0000 7f774ab04640 1 Processor -- start 2024-04-26T21:12:15.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.459+0000 7f774ab04640 1 -- start start 2024-04-26T21:12:15.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77440d3820 0x7f77441539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774415f680 0x7f77440b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77440d57b0 con 0x7f77440b60b0 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77440d5630 con 0x7f774415f680 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f774ab04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77440d5930 con 0x7f77440d3820 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40704/0 (socket says 172.21.15.40:40704) 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 -- 172.21.15.40:0/2144726664 learned_addr learned my addr 172.21.15.40:0/2144726664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f7743fff640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77440d3820 0x7f77441539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77490a1640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774415f680 0x7f77440b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 -- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77440d3820 msgr2=0x7f77441539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77440d3820 0x7f77441539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 -- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774415f680 msgr2=0x7f77440b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.460+0000 7f77488a0640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774415f680 0x7f77440b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f77488a0640 1 -- 172.21.15.40:0/2144726664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77440ba020 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f77488a0640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f773800c460 tx=0x7f773800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77380052d0 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7738005490 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77440cd700 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738012870 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.461+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7744160120 con 0x7f77440b60b0 2024-04-26T21:12:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.462+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774406a290 con 0x7f77440b60b0 2024-04-26T21:12:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.462+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7738012a10 con 0x7f77440b60b0 2024-04-26T21:12:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.463+0000 7f77417fa640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.463+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f773808b780 con 0x7f77440b60b0 2024-04-26T21:12:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.463+0000 7f7743fff640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.463+0000 7f7743fff640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 0x7f772806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7730034000 tx=0x7f773002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.467+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7738059bb0 con 0x7f77440b60b0 2024-04-26T21:12:15.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.655+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77440b4cd0 con 0x7f77440b60b0 2024-04-26T21:12:15.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.655+0000 7f77417fa640 1 -- 172.21.15.40:0/2144726664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f77380594b0 con 0x7f77440b60b0 2024-04-26T21:12:15.656 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.657+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 msgr2=0x7f772806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.657+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 0x7f772806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7730034000 tx=0x7f773002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.657+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 msgr2=0x7f774415fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.657+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f773800c460 tx=0x7f773800c920 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 shutdown_connections 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774415f680 0x7f77440b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77440d3820 0x7f77441539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 --2- 172.21.15.40:0/2144726664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77440b60b0 0x7f774415fbe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 >> 172.21.15.40:0/2144726664 conn(0x7f774408eca0 msgr2=0x7f774406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 shutdown_connections 2024-04-26T21:12:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.658+0000 7f774ab04640 1 -- 172.21.15.40:0/2144726664 wait complete. 2024-04-26T21:12:15.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:15.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.768+0000 7fc30430a640 1 Processor -- start 2024-04-26T21:12:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.768+0000 7fc30430a640 1 -- start start 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc15f370 0x7fc2fc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2fc15f750 0x7fc2fc0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc2fc0d37d0 0x7fc2fc0d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc2fc0b4d10 con 0x7fc2fc15f750 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc2fc0d4120 con 0x7fc2fc0d37d0 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc30430a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc0d03a0 con 0x7fc2fc15f370 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.769+0000 7fc3018a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2fc15f750 0x7fc2fc0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2fc15f750 0x7fc2fc0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40706/0 (socket says 172.21.15.40:40706) 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 -- 172.21.15.40:0/3171685814 learned_addr learned my addr 172.21.15.40:0/3171685814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3020a6640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc15f370 0x7fc2fc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3028a7640 1 --2- 172.21.15.40:0/3171685814 >> v2:172.21.15.184:3300/0 conn(0x7fc2fc0d37d0 0x7fc2fc0d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 -- 172.21.15.40:0/3171685814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc15f370 msgr2=0x7fc2fc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc15f370 0x7fc2fc0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 -- 172.21.15.40:0/3171685814 >> v2:172.21.15.184:3300/0 conn(0x7fc2fc0d37d0 msgr2=0x7fc2fc0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 --2- 172.21.15.40:0/3171685814 >> v2:172.21.15.184:3300/0 conn(0x7fc2fc0d37d0 0x7fc2fc0d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 -- 172.21.15.40:0/3171685814 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc0d0680 con 0x7fc2fc15f750 2024-04-26T21:12:15.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.770+0000 7fc3018a5640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc0b47d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0d02b0 tx=0x7fc2ec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74d6610f79cecb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc3010a4640 1 -- 172.21.15.40:0/3171685814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec037070 con 0x7fc2fc15f750 2024-04-26T21:12:15.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc3010a4640 1 -- 172.21.15.40:0/3171685814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ec032440 con 0x7fc2fc15f750 2024-04-26T21:12:15.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc3010a4640 1 -- 172.21.15.40:0/3171685814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec032810 con 0x7fc2fc15f750 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 -- 172.21.15.40:0/3171685814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 msgr2=0x7fc2fc0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc0b47d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0d02b0 tx=0x7fc2ec02a140 comp rx=0 tx=0).stop 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 -- 172.21.15.40:0/3171685814 shutdown_connections 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 --2- 172.21.15.40:0/3171685814 >> v2:172.21.15.184:3300/0 conn(0x7fc2fc0d37d0 0x7fc2fc0d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc0b47d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 --2- 172.21.15.40:0/3171685814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc15f370 0x7fc2fc0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 -- 172.21.15.40:0/3171685814 >> 172.21.15.40:0/3171685814 conn(0x7fc2fc08ec10 msgr2=0x7fc2fc09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 -- 172.21.15.40:0/3171685814 shutdown_connections 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.771+0000 7fc30430a640 1 -- 172.21.15.40:0/3171685814 wait complete. 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.772+0000 7fc30430a640 1 Processor -- start 2024-04-26T21:12:15.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.772+0000 7fc30430a640 1 -- start start 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc0d37d0 0x7fc2fc0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 0x7fc2fc161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc0d0f20 con 0x7fc2fc15f750 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc0d0c40 con 0x7fc2fc15f370 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc30430a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc0d1bd0 con 0x7fc2fc0d37d0 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc3028a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc3028a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40716/0 (socket says 172.21.15.40:40716) 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.773+0000 7fc3028a7640 1 -- 172.21.15.40:0/2361478200 learned_addr learned my addr 172.21.15.40:0/2361478200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3020a6640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc0d37d0 0x7fc2fc0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3018a5640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 0x7fc2fc161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 -- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc0d37d0 msgr2=0x7fc2fc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc0d37d0 0x7fc2fc0cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 -- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 msgr2=0x7fc2fc161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 0x7fc2fc161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 -- 172.21.15.40:0/2361478200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc161f00 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3018a5640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 0x7fc2fc161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc3028a7640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400ccf0 tx=0x7fc2f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003b90 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f4003d30 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc1621e0 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4004240 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.774+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc0c8fc0 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.775+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc06a290 con 0x7fc2fc15f750 2024-04-26T21:12:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.775+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2f40043e0 con 0x7fc2fc15f750 2024-04-26T21:12:15.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.776+0000 7fc2f2ffd640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 0x7fc2e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:15.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.776+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc2f408c510 con 0x7fc2fc15f750 2024-04-26T21:12:15.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.776+0000 7fc3020a6640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 0x7fc2e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:15.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.776+0000 7fc3020a6640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 0x7fc2e006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:15.780+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2f405a9d0 con 0x7fc2fc15f750 2024-04-26T21:12:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.003+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2fc0cc080 con 0x7fc2fc15f750 2024-04-26T21:12:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.004+0000 7fc2f2ffd640 1 -- 172.21.15.40:0/2361478200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7fc2f405a2d0 con 0x7fc2fc15f750 2024-04-26T21:12:16.005 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:16.005 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 msgr2=0x7fc2e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 0x7fc2e006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).stop 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 msgr2=0x7fc2fc1f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400ccf0 tx=0x7fc2f400a660 comp rx=0 tx=0).stop 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 shutdown_connections 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc2e006c8f0 0x7fc2e006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2fc15f750 0x7fc2fc1f7380 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2fc15f370 0x7fc2fc161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 --2- 172.21.15.40:0/2361478200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2fc0d37d0 0x7fc2fc0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 >> 172.21.15.40:0/2361478200 conn(0x7fc2fc08ec10 msgr2=0x7fc2fc0d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 shutdown_connections 2024-04-26T21:12:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.006+0000 7fc30430a640 1 -- 172.21.15.40:0/2361478200 wait complete. 2024-04-26T21:12:16.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:16.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.117+0000 7ffa28140640 1 Processor -- start 2024-04-26T21:12:16.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.117+0000 7ffa28140640 1 -- start start 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa28140640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa20160640 0x7ffa201636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa28140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 0x7ffa200b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa28140640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa200cfa80 con 0x7ffa20148a30 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa28140640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffa200745e0 con 0x7ffa20160640 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa28140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa200cfd60 con 0x7ffa20163be0 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa20160640 0x7ffa201636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa20160640 0x7ffa201636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40728/0 (socket says 172.21.15.40:40728) 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa25edc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffa20148a30 0x7ffa20073ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58624/0 (socket says 172.21.15.40:58624) 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 -- 172.21.15.40:0/3268207117 learned_addr learned my addr 172.21.15.40:0/3268207117 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa266dd640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 0x7ffa200b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 -- 172.21.15.40:0/3268207117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 msgr2=0x7ffa200b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 0x7ffa200b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa256db640 1 -- 172.21.15.40:0/3268207117 >> v1:172.21.15.40:6789/0 conn(0x7ffa20148a30 legacy=0x7ffa20073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.118+0000 7ffa24eda640 1 -- 172.21.15.40:0/3268207117 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494613070 0 0) 0x7ffa200cfa80 con 0x7ffa20148a30 2024-04-26T21:12:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa256db640 1 -- 172.21.15.40:0/3268207117 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa200d0040 con 0x7ffa20160640 2024-04-26T21:12:16.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa266dd640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 0x7ffa200b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa256db640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20160640 0x7ffa201636a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009870 tx=0x7ffa1402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e765f780112b3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa24eda640 1 -- 172.21.15.40:0/3268207117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14037070 con 0x7ffa20160640 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa24eda640 1 -- 172.21.15.40:0/3268207117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14032440 con 0x7ffa20160640 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.119+0000 7ffa24eda640 1 -- 172.21.15.40:0/3268207117 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14032810 con 0x7ffa20160640 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 -- 172.21.15.40:0/3268207117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20160640 msgr2=0x7ffa201636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20160640 0x7ffa201636a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009870 tx=0x7ffa1402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 -- 172.21.15.40:0/3268207117 shutdown_connections 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20163be0 0x7ffa200b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 --2- 172.21.15.40:0/3268207117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20160640 0x7ffa201636a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 -- 172.21.15.40:0/3268207117 >> 172.21.15.40:0/3268207117 conn(0x7ffa200ce8b0 msgr2=0x7ffa200cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 -- 172.21.15.40:0/3268207117 shutdown_connections 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.120+0000 7ffa28140640 1 -- 172.21.15.40:0/3268207117 wait complete. 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.121+0000 7ffa28140640 1 Processor -- start 2024-04-26T21:12:16.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.121+0000 7ffa28140640 1 -- start start 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 0x7ffa2015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa20160640 0x7ffa200b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20148e10 con 0x7ffa20163be0 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20163180 con 0x7ffa20160640 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa28140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa20148f90 con 0x7ffa20148a30 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40736/0 (socket says 172.21.15.40:40736) 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 -- 172.21.15.40:0/803308553 learned_addr learned my addr 172.21.15.40:0/803308553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa25edc640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 0x7ffa2015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa256db640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa20160640 0x7ffa200b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 -- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 msgr2=0x7ffa2015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 0x7ffa2015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 -- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa20160640 msgr2=0x7ffa200b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa20160640 0x7ffa200b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 -- 172.21.15.40:0/803308553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa200b5df0 con 0x7ffa20163be0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa25edc640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 0x7ffa2015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.122+0000 7ffa266dd640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c590 tx=0x7ffa1000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.123+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10015070 con 0x7ffa20163be0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.123+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa10011820 con 0x7ffa20163be0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.123+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa200b60d0 con 0x7ffa20163be0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.123+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10011c10 con 0x7ffa20163be0 2024-04-26T21:12:16.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.123+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa200ae950 con 0x7ffa20163be0 2024-04-26T21:12:16.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.124+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2006a290 con 0x7ffa20163be0 2024-04-26T21:12:16.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.124+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa10018070 con 0x7ffa20163be0 2024-04-26T21:12:16.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.125+0000 7ffa1e7fc640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 0x7ffa0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.125+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffa1008b9d0 con 0x7ffa20163be0 2024-04-26T21:12:16.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.125+0000 7ffa25edc640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 0x7ffa0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.125+0000 7ffa25edc640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 0x7ffa0c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa180012e0 tx=0x7ffa18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.129+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa10059e00 con 0x7ffa20163be0 2024-04-26T21:12:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.315+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa200739f0 con 0x7ffa20163be0 2024-04-26T21:12:16.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.316+0000 7ffa1e7fc640 1 -- 172.21.15.40:0/803308553 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7ffa10059700 con 0x7ffa20163be0 2024-04-26T21:12:16.317 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:16.317 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 msgr2=0x7ffa0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 0x7ffa0c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa180012e0 tx=0x7ffa18009040 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 msgr2=0x7ffa20135f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c590 tx=0x7ffa1000ca50 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 shutdown_connections 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa0c06c8f0 0x7ffa0c06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa20163be0 0x7ffa20135f90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa20160640 0x7ffa200b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 --2- 172.21.15.40:0/803308553 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa20148a30 0x7ffa2015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 >> 172.21.15.40:0/803308553 conn(0x7ffa200ce8b0 msgr2=0x7ffa20147c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 shutdown_connections 2024-04-26T21:12:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.318+0000 7ffa28140640 1 -- 172.21.15.40:0/803308553 wait complete. 2024-04-26T21:12:16.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:16.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:16.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.434+0000 7f1e6a88c640 1 Processor -- start 2024-04-26T21:12:16.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.434+0000 7f1e6a88c640 1 -- start start 2024-04-26T21:12:16.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e6a88c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e6406db10 0x7f1e6416a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e6a88c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6409c700 con 0x7f1e640b3c20 2024-04-26T21:12:16.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e6a88c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e640d1c50 con 0x7f1e640d0010 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e6a88c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1e640b4480 con 0x7f1e6406db10 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e6406db10 0x7f1e6416a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e6406db10 0x7f1e6416a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40748/0 (socket says 172.21.15.40:40748) 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 -- 172.21.15.40:0/3743027698 learned_addr learned my addr 172.21.15.40:0/3743027698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 -- 172.21.15.40:0/3743027698 >> v1:172.21.15.184:6789/0 conn(0x7f1e640d0010 legacy=0x7f1e640b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e62ffd640 1 -- 172.21.15.40:0/3743027698 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659966060 0 0) 0x7f1e6409c700 con 0x7f1e640b3c20 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 -- 172.21.15.40:0/3743027698 >> v1:172.21.15.40:6789/0 conn(0x7f1e640b3c20 legacy=0x7f1e640b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.435+0000 7f1e68e29640 1 -- 172.21.15.40:0/3743027698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640d1f30 con 0x7f1e6406db10 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.436+0000 7f1e68e29640 1 --2- 172.21.15.40:0/3743027698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e6416a5b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35c49495231350a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.436+0000 7f1e62ffd640 1 -- 172.21.15.40:0/3743027698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c036070 con 0x7f1e6406db10 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.436+0000 7f1e62ffd640 1 -- 172.21.15.40:0/3743027698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c031600 con 0x7f1e6406db10 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.436+0000 7f1e62ffd640 1 -- 172.21.15.40:0/3743027698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0319f0 con 0x7f1e6406db10 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3743027698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 msgr2=0x7f1e6416a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3743027698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e6416a5b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a7c0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3743027698 shutdown_connections 2024-04-26T21:12:16.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3743027698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e6416a5b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3743027698 >> 172.21.15.40:0/3743027698 conn(0x7f1e6409ba40 msgr2=0x7f1e6409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3743027698 shutdown_connections 2024-04-26T21:12:16.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.437+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3743027698 wait complete. 2024-04-26T21:12:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.438+0000 7f1e6a88c640 1 Processor -- start 2024-04-26T21:12:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.438+0000 7f1e6a88c640 1 -- start start 2024-04-26T21:12:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e640b3c20 0x7f1e641591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 0x7f1e641628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6416ac70 con 0x7f1e6406db10 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6416aaf0 con 0x7f1e640b3c20 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6416adf0 con 0x7f1e640d0010 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40758/0 (socket says 172.21.15.40:40758) 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 -- 172.21.15.40:0/3098789437 learned_addr learned my addr 172.21.15.40:0/3098789437 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e637fe640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e640b3c20 0x7f1e641591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e68e29640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 0x7f1e641628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 -- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 msgr2=0x7f1e641628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 0x7f1e641628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 -- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e640b3c20 msgr2=0x7f1e641591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e640b3c20 0x7f1e641591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 -- 172.21.15.40:0/3098789437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e64161010 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e68e29640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 0x7f1e641628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e63fff640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c400 tx=0x7f1e5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580052d0 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e58005490 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e641612f0 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.439+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58007260 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.440+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641618b0 con 0x7f1e6406db10 2024-04-26T21:12:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.440+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6406a290 con 0x7f1e6406db10 2024-04-26T21:12:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.441+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e58007400 con 0x7f1e6406db10 2024-04-26T21:12:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.441+0000 7f1e60ff9640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.441+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5808afd0 con 0x7f1e6406db10 2024-04-26T21:12:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.442+0000 7f1e637fe640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.442+0000 7f1e637fe640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 0x7f1e5006ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e6416aa60 tx=0x7f1e54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.445+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e58059490 con 0x7f1e6406db10 2024-04-26T21:12:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.630+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e64068e00 con 0x7f1e6406db10 2024-04-26T21:12:16.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.630+0000 7f1e60ff9640 1 -- 172.21.15.40:0/3098789437 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f1e58058d90 con 0x7f1e6406db10 2024-04-26T21:12:16.631 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 msgr2=0x7f1e5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 0x7f1e5006ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e6416aa60 tx=0x7f1e54009040 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 msgr2=0x7f1e641620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c400 tx=0x7f1e5800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 shutdown_connections 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e5006c920 0x7f1e5006ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e640d0010 0x7f1e641628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e640b3c20 0x7f1e641591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 --2- 172.21.15.40:0/3098789437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e6406db10 0x7f1e641620b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.632+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 >> 172.21.15.40:0/3098789437 conn(0x7f1e6409ba40 msgr2=0x7f1e640b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.633+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 shutdown_connections 2024-04-26T21:12:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.633+0000 7f1e6a88c640 1 -- 172.21.15.40:0/3098789437 wait complete. 2024-04-26T21:12:16.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:16.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.747+0000 7f22e3659640 1 Processor -- start 2024-04-26T21:12:16.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.747+0000 7f22e3659640 1 -- start start 2024-04-26T21:12:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e3659640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc15f950 0x7f22dc0d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e3659640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22dc0d2230 0x7f22dc0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e3659640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc0b63e0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e3659640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0d5ed0 con 0x7f22dc15f950 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e3659640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0d3120 con 0x7f22dc0d2230 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc15f950 0x7f22dc0d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc15f950 0x7f22dc0d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40766/0 (socket says 172.21.15.40:40766) 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e1bf6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f22dc0c8db0 0x7f22dc0c9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58644/0 (socket says 172.21.15.40:58644) 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 -- 172.21.15.40:0/1008734605 learned_addr learned my addr 172.21.15.40:0/1008734605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e0bf4640 1 --2- 172.21.15.40:0/1008734605 >> v2:172.21.15.184:3300/0 conn(0x7f22dc0d2230 0x7f22dc0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 -- 172.21.15.40:0/1008734605 >> v2:172.21.15.184:3300/0 conn(0x7f22dc0d2230 msgr2=0x7f22dc0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 --2- 172.21.15.40:0/1008734605 >> v2:172.21.15.184:3300/0 conn(0x7f22dc0d2230 0x7f22dc0d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 -- 172.21.15.40:0/1008734605 >> v1:172.21.15.40:6789/0 conn(0x7f22dc0c8db0 legacy=0x7f22dc0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22e13f5640 1 -- 172.21.15.40:0/1008734605 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0b69c0 con 0x7f22dc15f950 2024-04-26T21:12:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.748+0000 7f22d3fff640 1 -- 172.21.15.40:0/1008734605 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776557611 0 0) 0x7f22dc0b63e0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.749+0000 7f22e13f5640 1 --2- 172.21.15.40:0/1008734605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f950 0x7f22dc0d3740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0d4ac0 tx=0x7f22d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b717e43fa6b78380 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.749+0000 7f22d3fff640 1 -- 172.21.15.40:0/1008734605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8037070 con 0x7f22dc15f950 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.749+0000 7f22d3fff640 1 -- 172.21.15.40:0/1008734605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d8032440 con 0x7f22dc15f950 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.749+0000 7f22d3fff640 1 -- 172.21.15.40:0/1008734605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8032810 con 0x7f22dc15f950 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 -- 172.21.15.40:0/1008734605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f950 msgr2=0x7f22dc0d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 --2- 172.21.15.40:0/1008734605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f950 0x7f22dc0d3740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0d4ac0 tx=0x7f22d802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 -- 172.21.15.40:0/1008734605 shutdown_connections 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 --2- 172.21.15.40:0/1008734605 >> v2:172.21.15.184:3300/0 conn(0x7f22dc0d2230 0x7f22dc0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 --2- 172.21.15.40:0/1008734605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f950 0x7f22dc0d3740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 -- 172.21.15.40:0/1008734605 >> 172.21.15.40:0/1008734605 conn(0x7f22dc0d5160 msgr2=0x7f22dc0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 -- 172.21.15.40:0/1008734605 shutdown_connections 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.750+0000 7f22e3659640 1 -- 172.21.15.40:0/1008734605 wait complete. 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.751+0000 7f22e3659640 1 Processor -- start 2024-04-26T21:12:16.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.751+0000 7f22e3659640 1 -- start start 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d2230 0x7f22dc1f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f950 0x7f22dc1f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0d10a0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0d0f20 con 0x7f22dc15f950 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e3659640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0c9950 con 0x7f22dc0d2230 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40782/0 (socket says 172.21.15.40:40782) 2024-04-26T21:12:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 -- 172.21.15.40:0/2273186787 learned_addr learned my addr 172.21.15.40:0/2273186787 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e1bf6640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f950 0x7f22dc1f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e0bf4640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d2230 0x7f22dc1f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 -- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d2230 msgr2=0x7f22dc1f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d2230 0x7f22dc1f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 -- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f950 msgr2=0x7f22dc1f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f950 0x7f22dc1f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.752+0000 7f22e13f5640 1 -- 172.21.15.40:0/2273186787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0b8cf0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22e13f5640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8037040 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d8042070 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc14af20 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8030880 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.753+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc14b4e0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.754+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc1f94a0 con 0x7f22dc0c8db0 2024-04-26T21:12:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.755+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22d8030a20 con 0x7f22dc0c8db0 2024-04-26T21:12:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.755+0000 7f22d1ffb640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 0x7f22c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.755+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f22d80ac190 con 0x7f22dc0c8db0 2024-04-26T21:12:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.755+0000 7f22e0bf4640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 0x7f22c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.756+0000 7f22e0bf4640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 0x7f22c006ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0cf460 tx=0x7f22cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.759+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22d807a000 con 0x7f22dc0c8db0 2024-04-26T21:12:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.946+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22dc0d3520 con 0x7f22dc0c8db0 2024-04-26T21:12:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.946+0000 7f22d1ffb640 1 -- 172.21.15.40:0/2273186787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f22d807e300 con 0x7f22dc0c8db0 2024-04-26T21:12:16.947 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 msgr2=0x7f22c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 0x7f22c006ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0cf460 tx=0x7f22cc009190 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 msgr2=0x7f22dc0b9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 shutdown_connections 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c006c920 0x7f22c006ede0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f950 0x7f22dc1f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d2230 0x7f22dc1f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 --2- 172.21.15.40:0/2273186787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0c8db0 0x7f22dc0b9790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.948+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 >> 172.21.15.40:0/2273186787 conn(0x7f22dc0d5160 msgr2=0x7f22dc0d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:16.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.949+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 shutdown_connections 2024-04-26T21:12:16.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:16.949+0000 7f22e3659640 1 -- 172.21.15.40:0/2273186787 wait complete. 2024-04-26T21:12:16.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.062+0000 7f7a8c883640 1 Processor -- start 2024-04-26T21:12:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.062+0000 7f7a8c883640 1 -- start start 2024-04-26T21:12:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8c883640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a8806f340 0x7f7a880b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8c883640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a880b46a0 con 0x7f7a880b3e50 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8c883640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a880b5db0 con 0x7f7a880d0300 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8c883640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7a880b6090 con 0x7f7a8806f340 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a8806f340 0x7f7a880b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a86d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a880b3e50 0x7f7a880720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58654/0 (socket says 172.21.15.40:58654) 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a86d9d640 1 -- 172.21.15.40:0/1578324872 learned_addr learned my addr 172.21.15.40:0/1578324872 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a8806f340 0x7f7a880b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40786/0 (socket says 172.21.15.40:40786) 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8659c640 1 -- 172.21.15.40:0/1578324872 >> v1:172.21.15.184:6789/0 conn(0x7f7a880d0300 legacy=0x7f7a8806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a85d9b640 1 -- 172.21.15.40:0/1578324872 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3387839164 0 0) 0x7f7a880b46a0 con 0x7f7a880b3e50 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8659c640 1 -- 172.21.15.40:0/1578324872 >> v1:172.21.15.40:6789/0 conn(0x7f7a880b3e50 legacy=0x7f7a880720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.063+0000 7f7a8659c640 1 -- 172.21.15.40:0/1578324872 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a880b6370 con 0x7f7a8806f340 2024-04-26T21:12:17.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.064+0000 7f7a8659c640 1 --2- 172.21.15.40:0/1578324872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a880b6b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a7802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a22fa13741cd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.064+0000 7f7a85d9b640 1 -- 172.21.15.40:0/1578324872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78037070 con 0x7f7a8806f340 2024-04-26T21:12:17.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.064+0000 7f7a85d9b640 1 -- 172.21.15.40:0/1578324872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78032440 con 0x7f7a8806f340 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.064+0000 7f7a85d9b640 1 -- 172.21.15.40:0/1578324872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7802ae60 con 0x7f7a8806f340 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 -- 172.21.15.40:0/1578324872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 msgr2=0x7f7a880b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1578324872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a880b6b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a7802a110 comp rx=0 tx=0).stop 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 -- 172.21.15.40:0/1578324872 shutdown_connections 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1578324872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a880b6b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 -- 172.21.15.40:0/1578324872 >> 172.21.15.40:0/1578324872 conn(0x7f7a880cd8a0 msgr2=0x7f7a880cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 -- 172.21.15.40:0/1578324872 shutdown_connections 2024-04-26T21:12:17.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.065+0000 7f7a8c883640 1 -- 172.21.15.40:0/1578324872 wait complete. 2024-04-26T21:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.066+0000 7f7a8c883640 1 Processor -- start 2024-04-26T21:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.066+0000 7f7a8c883640 1 -- start start 2024-04-26T21:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b3e50 0x7f7a8815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a880d0300 0x7f7a8815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880b4460 con 0x7f7a8806f340 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880b42e0 con 0x7f7a880b3e50 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8c883640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a88142a30 con 0x7f7a880d0300 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40798/0 (socket says 172.21.15.40:40798) 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 -- 172.21.15.40:0/1937809025 learned_addr learned my addr 172.21.15.40:0/1937809025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a7ffff640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b3e50 0x7f7a8815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a86d9d640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a880d0300 0x7f7a8815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 -- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a880d0300 msgr2=0x7f7a8815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a880d0300 0x7f7a8815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 -- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b3e50 msgr2=0x7f7a8815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b3e50 0x7f7a8815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 -- 172.21.15.40:0/1937809025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8814edc0 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a8659c640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.067+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78037050 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.068+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78038040 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.068+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a881f7010 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.068+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004470 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.068+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881e4e50 con 0x7f7a8806f340 2024-04-26T21:12:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.069+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a881e4960 con 0x7f7a8806f340 2024-04-26T21:12:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.069+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a78030880 con 0x7f7a8806f340 2024-04-26T21:12:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.069+0000 7f7a7effd640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.070+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7a780ac770 con 0x7f7a8806f340 2024-04-26T21:12:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.070+0000 7f7a7ffff640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.070+0000 7f7a7ffff640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 0x7f7a6806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7a881f61d0 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.074+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a7807aba0 con 0x7f7a8806f340 2024-04-26T21:12:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.257+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a8806a290 con 0x7f7a8806f340 2024-04-26T21:12:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.257+0000 7f7a7effd640 1 -- 172.21.15.40:0/1937809025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f7a7807a4a0 con 0x7f7a8806f340 2024-04-26T21:12:17.258 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 msgr2=0x7f7a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 0x7f7a6806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7a881f61d0 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2024-04-26T21:12:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 msgr2=0x7f7a8814e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097a0 tx=0x7f7a780046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 shutdown_connections 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a880d0300 0x7f7a8815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b3e50 0x7f7a8815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 --2- 172.21.15.40:0/1937809025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a8806f340 0x7f7a8814e880 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 >> 172.21.15.40:0/1937809025 conn(0x7f7a880cd8a0 msgr2=0x7f7a880cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 shutdown_connections 2024-04-26T21:12:17.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.259+0000 7f7a8c883640 1 -- 172.21.15.40:0/1937809025 wait complete. 2024-04-26T21:12:17.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.370+0000 7f74a9207640 1 Processor -- start 2024-04-26T21:12:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.370+0000 7f74a9207640 1 -- start start 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a9207640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 0x7f74a40d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a9207640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 0x7f74a40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a9207640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a40d3b80 con 0x7f74a415f680 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a9207640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f74a40d3e60 con 0x7f74a40b4290 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a9207640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74a40d4140 con 0x7f74a415fa60 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f74a415f680 0x7f74a40b1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58660/0 (socket says 172.21.15.40:58660) 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a2d9d640 1 -- 172.21.15.40:0/3849923342 learned_addr learned my addr 172.21.15.40:0/3849923342 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a259c640 1 --2- 172.21.15.40:0/3849923342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 0x7f74a40d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a359e640 1 --2- 172.21.15.40:0/3849923342 >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 0x7f74a40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024262299 0 0) 0x7f74a40d3b80 con 0x7f74a415f680 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f748c003520 con 0x7f74a415f680 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462396803 0 0) 0x7f748c003520 con 0x7f74a415f680 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a259c640 1 -- 172.21.15.40:0/3849923342 >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 msgr2=0x7f74a40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a259c640 1 --2- 172.21.15.40:0/3849923342 >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 0x7f74a40d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.371+0000 7f74a259c640 1 -- 172.21.15.40:0/3849923342 >> v1:172.21.15.40:6789/0 conn(0x7f74a415f680 legacy=0x7f74a40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.372+0000 7f74a259c640 1 -- 172.21.15.40:0/3849923342 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a40d0270 con 0x7f74a415fa60 2024-04-26T21:12:17.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.372+0000 7f74a359e640 1 --2- 172.21.15.40:0/3849923342 >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 0x7f74a40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.372+0000 7f74a259c640 1 --2- 172.21.15.40:0/3849923342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 0x7f74a40d5e50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74980097a0 tx=0x7f749802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec1e972de851e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498037070 con 0x7f74a415fa60 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749802aba0 con 0x7f74a415fa60 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a1d9b640 1 -- 172.21.15.40:0/3849923342 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940019f0 con 0x7f74a415fa60 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 -- 172.21.15.40:0/3849923342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 msgr2=0x7f74a40d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 --2- 172.21.15.40:0/3849923342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 0x7f74a40d5e50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74980097a0 tx=0x7f749802a110 comp rx=0 tx=0).stop 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 -- 172.21.15.40:0/3849923342 shutdown_connections 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 --2- 172.21.15.40:0/3849923342 >> v2:172.21.15.184:3300/0 conn(0x7f74a40b4290 0x7f74a40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 --2- 172.21.15.40:0/3849923342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a415fa60 0x7f74a40d5e50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 -- 172.21.15.40:0/3849923342 >> 172.21.15.40:0/3849923342 conn(0x7f74a409e5a0 msgr2=0x7f74a40d5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 -- 172.21.15.40:0/3849923342 shutdown_connections 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.373+0000 7f74a9207640 1 -- 172.21.15.40:0/3849923342 wait complete. 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 Processor -- start 2024-04-26T21:12:17.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 -- start start 2024-04-26T21:12:17.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a40b4290 0x7f74a40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a415fa60 0x7f74a40b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74a40d4420 con 0x7f74a415f680 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74a40b2610 con 0x7f74a415fa60 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.375+0000 7f74a9207640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f74a40d2130 con 0x7f74a40b4290 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40802/0 (socket says 172.21.15.40:40802) 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 -- 172.21.15.40:0/1611191499 learned_addr learned my addr 172.21.15.40:0/1611191499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a2d9d640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a40b4290 0x7f74a40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a359e640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a415fa60 0x7f74a40b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 -- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a40b4290 msgr2=0x7f74a40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a40b4290 0x7f74a40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 -- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a415fa60 msgr2=0x7f74a40b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a415fa60 0x7f74a40b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 -- 172.21.15.40:0/1611191499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a40b9d80 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a259c640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74980097a0 tx=0x7f74980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498037050 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7498043070 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a40ba060 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498006e10 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.376+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a40ba900 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.377+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a406a290 con 0x7f74a415f680 2024-04-26T21:12:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.378+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f749803ca50 con 0x7f74a415f680 2024-04-26T21:12:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.378+0000 7f748b7fe640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 0x7f749006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.378+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f74980ac5a0 con 0x7f74a415f680 2024-04-26T21:12:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.378+0000 7f74a2d9d640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 0x7f749006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.379+0000 7f74a2d9d640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 0x7f749006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7494005c00 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.382+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f749807aaf0 con 0x7f74a415f680 2024-04-26T21:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.604+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74a408f350 con 0x7f74a415f680 2024-04-26T21:12:17.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.604+0000 7f748b7fe640 1 -- 172.21.15.40:0/1611191499 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f749807a3f0 con 0x7f74a415f680 2024-04-26T21:12:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:17.605 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:17.606 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 msgr2=0x7f749006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 0x7f749006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7494005c00 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 msgr2=0x7f74a415d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74980097a0 tx=0x7f74980046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 shutdown_connections 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f749006c8f0 0x7f749006edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a415fa60 0x7f74a40b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a415f680 0x7f74a415d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 --2- 172.21.15.40:0/1611191499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74a40b4290 0x7f74a40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 >> 172.21.15.40:0/1611191499 conn(0x7f74a409e5a0 msgr2=0x7f74a40d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 shutdown_connections 2024-04-26T21:12:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.606+0000 7f74a9207640 1 -- 172.21.15.40:0/1611191499 wait complete. 2024-04-26T21:12:17.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:17.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.718+0000 7f57997e2640 1 Processor -- start 2024-04-26T21:12:17.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.718+0000 7f57997e2640 1 -- start start 2024-04-26T21:12:17.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57997e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57940d11b0 0x7f579406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57997e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f579406e2c0 0x7f579415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57997e2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57940d1740 con 0x7f57940cf2e0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57997e2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f579406e710 con 0x7f57940d11b0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57997e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f57940d1a20 con 0x7f579406e2c0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57940d11b0 0x7f579406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57940d11b0 0x7f579406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40814/0 (socket says 172.21.15.40:40814) 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 -- 172.21.15.40:0/3639984926 learned_addr learned my addr 172.21.15.40:0/3639984926 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f57937fe640 1 --2- 172.21.15.40:0/3639984926 >> v2:172.21.15.184:3300/0 conn(0x7f579406e2c0 0x7f579415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 -- 172.21.15.40:0/3639984926 >> v2:172.21.15.184:3300/0 conn(0x7f579406e2c0 msgr2=0x7f579415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 --2- 172.21.15.40:0/3639984926 >> v2:172.21.15.184:3300/0 conn(0x7f579406e2c0 0x7f579415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 -- 172.21.15.40:0/3639984926 >> v1:172.21.15.184:6789/0 conn(0x7f57940cf2e0 legacy=0x7f579406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.719+0000 7f578bfff640 1 -- 172.21.15.40:0/3639984926 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57940d1d00 con 0x7f57940d11b0 2024-04-26T21:12:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f578bfff640 1 --2- 172.21.15.40:0/3639984926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d11b0 0x7f579406dd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f577c009870 tx=0x7f577c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e941644fe72602 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f57927fc640 1 -- 172.21.15.40:0/3639984926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c037070 con 0x7f57940d11b0 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f57927fc640 1 -- 172.21.15.40:0/3639984926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c032440 con 0x7f57940d11b0 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f57927fc640 1 -- 172.21.15.40:0/3639984926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c032810 con 0x7f57940d11b0 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f57997e2640 1 -- 172.21.15.40:0/3639984926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d11b0 msgr2=0x7f579406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.720+0000 7f57997e2640 1 --2- 172.21.15.40:0/3639984926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d11b0 0x7f579406dd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f577c009870 tx=0x7f577c02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 -- 172.21.15.40:0/3639984926 shutdown_connections 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 --2- 172.21.15.40:0/3639984926 >> v2:172.21.15.184:3300/0 conn(0x7f579406e2c0 0x7f579415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 --2- 172.21.15.40:0/3639984926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d11b0 0x7f579406dd80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 -- 172.21.15.40:0/3639984926 >> 172.21.15.40:0/3639984926 conn(0x7f57940b4a60 msgr2=0x7f57940b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 -- 172.21.15.40:0/3639984926 shutdown_connections 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.721+0000 7f57997e2640 1 -- 172.21.15.40:0/3639984926 wait complete. 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 Processor -- start 2024-04-26T21:12:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 -- start start 2024-04-26T21:12:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 0x7f57940b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 0x7f57940b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57940cf6c0 con 0x7f579406e2c0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57940d22c0 con 0x7f57940cf2e0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.722+0000 7f57997e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57940cf840 con 0x7f57940d11b0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40826/0 (socket says 172.21.15.40:40826) 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 -- 172.21.15.40:0/1038599623 learned_addr learned my addr 172.21.15.40:0/1038599623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f57937fe640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 0x7f57940b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f578bfff640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 0x7f57940b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 -- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 msgr2=0x7f57940b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 0x7f57940b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 -- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 msgr2=0x7f57940b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 0x7f57940b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 -- 172.21.15.40:0/1038599623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57941ea5b0 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f57937fe640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 0x7f57940b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f578bfff640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 0x7f57940b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f5792ffd640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f578400c3c0 tx=0x7f578400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57840052d0 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5784005490 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57941f5510 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784007260 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.723+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941f5ad0 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.724+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57941f5330 con 0x7f579406e2c0 2024-04-26T21:12:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.725+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5784007400 con 0x7f579406e2c0 2024-04-26T21:12:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.725+0000 7f578b7fe640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.725+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f578408afc0 con 0x7f579406e2c0 2024-04-26T21:12:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.725+0000 7f578bfff640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.726+0000 7f578bfff640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 0x7f577406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.729+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57941f5330 con 0x7f579406e2c0 2024-04-26T21:12:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.917+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f579406a290 con 0x7f579406e2c0 2024-04-26T21:12:17.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.918+0000 7f578b7fe640 1 -- 172.21.15.40:0/1038599623 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f5784059480 con 0x7f579406e2c0 2024-04-26T21:12:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:17.920 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 msgr2=0x7f577406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 0x7f577406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 msgr2=0x7f579415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f578400c3c0 tx=0x7f578400c880 comp rx=0 tx=0).stop 2024-04-26T21:12:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 shutdown_connections 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d11b0 0x7f57940b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940cf2e0 0x7f57940b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 --2- 172.21.15.40:0/1038599623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f579406e2c0 0x7f579415e5a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 >> 172.21.15.40:0/1038599623 conn(0x7f57940b4a60 msgr2=0x7f57940c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 shutdown_connections 2024-04-26T21:12:17.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:17.920+0000 7f57997e2640 1 -- 172.21.15.40:0/1038599623 wait complete. 2024-04-26T21:12:17.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:18.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.031+0000 7f8024789640 1 Processor -- start 2024-04-26T21:12:18.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.031+0000 7f8024789640 1 -- start start 2024-04-26T21:12:18.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8024789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f340 0x7f801c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8024789640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f801c15f720 0x7f801c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8024789640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c0b2510 con 0x7f801c0d4f00 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8024789640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f801c0b27f0 con 0x7f801c15f720 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8024789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0b2ad0 con 0x7f801c15f340 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f801c15f720 0x7f801c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f801c15f720 0x7f801c0d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40828/0 (socket says 172.21.15.40:40828) 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 -- 172.21.15.40:0/1081495356 learned_addr learned my addr 172.21.15.40:0/1081495356 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8022525640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f340 0x7f801c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 -- 172.21.15.40:0/1081495356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f340 msgr2=0x7f801c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f340 0x7f801c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 -- 172.21.15.40:0/1081495356 >> v1:172.21.15.184:6789/0 conn(0x7f801c0d4f00 legacy=0x7f801c0d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.032+0000 7f8021d24640 1 -- 172.21.15.40:0/1081495356 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c0b2db0 con 0x7f801c15f720 2024-04-26T21:12:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.033+0000 7f8021d24640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c15f720 0x7f801c0d3d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123013e3dc158bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.033+0000 7f8021523640 1 -- 172.21.15.40:0/1081495356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c037070 con 0x7f801c15f720 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.033+0000 7f8021523640 1 -- 172.21.15.40:0/1081495356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c032440 con 0x7f801c15f720 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.033+0000 7f8021523640 1 -- 172.21.15.40:0/1081495356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c032810 con 0x7f801c15f720 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 -- 172.21.15.40:0/1081495356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c15f720 msgr2=0x7f801c0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c15f720 0x7f801c0d3d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02a180 comp rx=0 tx=0).stop 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 -- 172.21.15.40:0/1081495356 shutdown_connections 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c15f720 0x7f801c0d3d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 --2- 172.21.15.40:0/1081495356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f340 0x7f801c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 -- 172.21.15.40:0/1081495356 >> 172.21.15.40:0/1081495356 conn(0x7f801c08ec70 msgr2=0x7f801c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 -- 172.21.15.40:0/1081495356 shutdown_connections 2024-04-26T21:12:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.034+0000 7f8024789640 1 -- 172.21.15.40:0/1081495356 wait complete. 2024-04-26T21:12:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.035+0000 7f8024789640 1 Processor -- start 2024-04-26T21:12:18.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.035+0000 7f8024789640 1 -- start start 2024-04-26T21:12:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.035+0000 7f8024789640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.035+0000 7f8024789640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f801c15f340 0x7f801c0ba880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.035+0000 7f8024789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f720 0x7f801c1f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8024789640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0b3980 con 0x7f801c0d4f00 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8024789640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0b3800 con 0x7f801c15f340 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8024789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0b3b00 con 0x7f801c15f720 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40832/0 (socket says 172.21.15.40:40832) 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 -- 172.21.15.40:0/3203849668 learned_addr learned my addr 172.21.15.40:0/3203849668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022d26640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f720 0x7f801c1f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8021d24640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f801c15f340 0x7f801c0ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 -- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f720 msgr2=0x7f801c1f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f720 0x7f801c1f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 -- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f801c15f340 msgr2=0x7f801c0ba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f801c15f340 0x7f801c0ba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 -- 172.21.15.40:0/3203849668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c06d6f0 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8022525640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f801800d800 tx=0x7f801800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801800b660 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8018019070 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c06d9d0 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801800ba20 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.036+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c06df90 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.037+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c06a290 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.038+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f801800bbc0 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.038+0000 7f8012ffd640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 0x7f800006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.038+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f801808be30 con 0x7f801c0d4f00 2024-04-26T21:12:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.039+0000 7f8021d24640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 0x7f800006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.039+0000 7f8021d24640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 0x7f800006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f800c034000 tx=0x7f800c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.043+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8018059c10 con 0x7f801c0d4f00 2024-04-26T21:12:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.264+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f801c0d3020 con 0x7f801c0d4f00 2024-04-26T21:12:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.264+0000 7f8012ffd640 1 -- 172.21.15.40:0/3203849668 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f801805df10 con 0x7f801c0d4f00 2024-04-26T21:12:18.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:18.266 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 msgr2=0x7f800006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 0x7f800006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f800c034000 tx=0x7f800c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 msgr2=0x7f801c06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f801800d800 tx=0x7f801800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 shutdown_connections 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f800006c8f0 0x7f800006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f801c15f720 0x7f801c1f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f801c15f340 0x7f801c0ba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 --2- 172.21.15.40:0/3203849668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f801c0d4f00 0x7f801c06d010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 >> 172.21.15.40:0/3203849668 conn(0x7f801c08ec70 msgr2=0x7f801c0b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 shutdown_connections 2024-04-26T21:12:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.266+0000 7f8024789640 1 -- 172.21.15.40:0/3203849668 wait complete. 2024-04-26T21:12:18.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:12:01.103924+0000', 'modified': '2024-04-26T21:12:03.117727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6837/2363826920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 2363826920}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2363826920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4291, 'qdb_cluster': [4291]} 2024-04-26T21:12:18.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:12:18.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:12:18.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.794+0000 7fe1587e9640 1 Processor -- start 2024-04-26T21:12:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.794+0000 7fe1587e9640 1 -- start start 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.795+0000 7fe1587e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe1500d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.795+0000 7fe1587e9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1500ceef0 con 0x7fe1500d0b20 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.795+0000 7fe1587e9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1500cda60 con 0x7fe15015d360 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.795+0000 7fe1587e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe15015d740 con 0x7fe1500d1510 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155d84640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe1500d0b20 0x7fe1500ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58672/0 (socket says 172.21.15.40:58672) 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155d84640 1 -- 172.21.15.40:0/3812667245 learned_addr learned my addr 172.21.15.40:0/3812667245 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe156d86640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe1500d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693711185 0 0) 0x7fe1500ceef0 con 0x7fe1500d0b20 2024-04-26T21:12:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe138003520 con 0x7fe1500d0b20 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2170355368 0 0) 0x7fe138003520 con 0x7fe1500d0b20 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 msgr2=0x7fe1500d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe1500d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 >> v1:172.21.15.184:6789/0 conn(0x7fe15015d360 legacy=0x7fe15009c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500d1100 con 0x7fe1500d0b20 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe144001670 con 0x7fe1500d0b20 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 >> v1:172.21.15.40:6789/0 conn(0x7fe1500d0b20 legacy=0x7fe1500ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe138004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe138004d40 0x7fe1380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe15015d740 con 0x7fe1380039e0 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe13800baa0 con 0x7fe138004d40 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe13800bca0 con 0x7fe1500d1510 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe156585640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.796+0000 7fe155d84640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe138004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156d86640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe138004d40 0x7fe1380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 -- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 msgr2=0x7fe138004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1500d1510 0x7fe138004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 -- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe138004d40 msgr2=0x7fe1380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe138004d40 0x7fe1380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 -- 172.21.15.40:0/3812667245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe138003520 con 0x7fe1380039e0 2024-04-26T21:12:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.797+0000 7fe156585640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe138003e00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe148001720 tx=0x7fe138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3300eb8f36cf3d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148001370 con 0x7fe1380039e0 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1480052d0 con 0x7fe1380039e0 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe155583640 1 -- 172.21.15.40:0/3812667245 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14802abc0 con 0x7fe1380039e0 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 -- 172.21.15.40:0/3812667245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 msgr2=0x7fe138003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 --2- 172.21.15.40:0/3812667245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe138003e00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe148001720 tx=0x7fe138005830 comp rx=0 tx=0).stop 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe156d86640 1 -- 172.21.15.40:0/3812667245 reap_dead start 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 -- 172.21.15.40:0/3812667245 shutdown_connections 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 -- 172.21.15.40:0/3812667245 >> 172.21.15.40:0/3812667245 conn(0x7fe1500b4a40 msgr2=0x7fe1500b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 -- 172.21.15.40:0/3812667245 shutdown_connections 2024-04-26T21:12:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.798+0000 7fe1587e9640 1 -- 172.21.15.40:0/3812667245 wait complete. 2024-04-26T21:12:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.799+0000 7fe1587e9640 1 Processor -- start 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.799+0000 7fe1587e9640 1 -- start start 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 0x7fe1500c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 0x7fe15015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500d19a0 con 0x7fe1380039e0 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500cdfe0 con 0x7fe1500c6fc0 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe1587e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1500d1b20 con 0x7fe15015f480 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe156585640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe156585640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40842/0 (socket says 172.21.15.40:40842) 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.800+0000 7fe156585640 1 -- 172.21.15.40:0/2128133824 learned_addr learned my addr 172.21.15.40:0/2128133824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 -- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 msgr2=0x7fe1500c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe155d84640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 0x7fe1500c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156d86640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 0x7fe15015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 0x7fe1500c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 -- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 msgr2=0x7fe15015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 0x7fe15015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 -- 172.21.15.40:0/2128133824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1501606c0 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156d86640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 0x7fe15015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe155d84640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 0x7fe1500c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe156585640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe148001840 tx=0x7fe148001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe14effd640 1 -- 172.21.15.40:0/2128133824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148037040 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe14effd640 1 -- 172.21.15.40:0/2128133824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148005480 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1500c82a0 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe14effd640 1 -- 172.21.15.40:0/2128133824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148007a30 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.801+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1500c8860 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.803+0000 7fe14effd640 1 -- 172.21.15.40:0/2128133824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe148044020 con 0x7fe1380039e0 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.803+0000 7fe14effd640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.803+0000 7fe14effd640 1 -- 172.21.15.40:0/2128133824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe1480ab820 con 0x7fe1380039e0 2024-04-26T21:12:18.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.803+0000 7fe155d84640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.804+0000 7fe155d84640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 0x7fe13c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1500c7d20 tx=0x7fe144009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.804+0000 7fe15b5de740 1 Processor -- start 2024-04-26T21:12:18.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.804+0000 7fe15b5de740 1 -- start start 2024-04-26T21:12:18.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d49f0 0x55ca871ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ca87111740 con 0x55ca871d7790 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ca870ad3a0 con 0x55ca871d4dd0 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ca8710d700 con 0x55ca871d49f0 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40844/0 (socket says 172.21.15.40:40844) 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 -- 172.21.15.40:0/3435855919 learned_addr learned my addr 172.21.15.40:0/3435855919 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe155d84640 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156585640 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d49f0 0x55ca871ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 -- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d49f0 msgr2=0x55ca871ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d49f0 0x55ca871ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 -- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 msgr2=0x55ca871d7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 -- 172.21.15.40:0/3435855919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca870c9750 con 0x55ca871d7790 2024-04-26T21:12:18.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.805+0000 7fe156d86640 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe150073790 tx=0x7fe140009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b9e10487e0ada9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe137fff640 1 -- 172.21.15.40:0/3435855919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140003b90 con 0x55ca871d7790 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe137fff640 1 -- 172.21.15.40:0/3435855919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14001c070 con 0x55ca871d7790 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe137fff640 1 -- 172.21.15.40:0/3435855919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140004030 con 0x55ca871d7790 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 -- 172.21.15.40:0/3435855919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 msgr2=0x55ca871aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe150073790 tx=0x7fe140009750 comp rx=0 tx=0).stop 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 -- 172.21.15.40:0/3435855919 shutdown_connections 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871aaff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3435855919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d49f0 0x55ca871ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 -- 172.21.15.40:0/3435855919 >> 172.21.15.40:0/3435855919 conn(0x55ca871a9a50 msgr2=0x55ca871f2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 -- 172.21.15.40:0/3435855919 shutdown_connections 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.806+0000 7fe15b5de740 1 -- 172.21.15.40:0/3435855919 wait complete. 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.807+0000 7fe15b5de740 1 -- start start 2024-04-26T21:12:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.807+0000 7fe15b5de740 1 Processor -- start 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d7790 0x55ca871f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca87206f60 0x55ca871e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ca87098fe0 con 0x55ca871d4dd0 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ca871eff50 con 0x55ca871d7790 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ca87207630 con 0x55ca87206f60 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe15b5de740 10 client.0 _ll_register_callbacks cb 0x55ca870af150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40850/0 (socket says 172.21.15.40:40850) 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 -- 172.21.15.40:0/2963631472 learned_addr learned my addr 172.21.15.40:0/2963631472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156d86640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca87206f60 0x55ca871e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe155d84640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d7790 0x55ca871f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 -- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca87206f60 msgr2=0x55ca871e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca87206f60 0x55ca871e9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 -- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d7790 msgr2=0x55ca871f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d7790 0x55ca871f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.808+0000 7fe156585640 1 -- 172.21.15.40:0/2963631472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca86e232d0 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe156585640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1480759b0 tx=0x7fe148079610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c000f5c9fe26cda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ab040 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe148001370 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14001c070 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fe14802be10 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 10 client.4701 fetch_fsmap learned FSMap version 6 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55ca86eed6b0 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14001c070 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7fe14803a030 con 0x55ca871d4dd0 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T21:12:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 20 client.4701 populate_metadata read hostname 'smithi040' 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871d8880 0x55ca871d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 4 client.4701 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 --> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8711f350 con 0x55ca871d8880 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.809+0000 7fe155d84640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871d8880 0x55ca871d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.810+0000 7fe1357fa640 10 client.4701 ms_handle_connect on v2:172.21.15.184:6835/2363826920 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.810+0000 7fe155d84640 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871d8880 0x55ca871d8c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895ba7df9b683d8b server_cookie=bdeaef801e7acca5 in_seq=0 out_seq=0 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe1357fa640 1 -- 172.21.15.40:0/2963631472 <== mds.0 v2:172.21.15.184:6835/2363826920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca8711f350 con 0x55ca871d8880 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe1357fa640 10 client.4701 handle_command_reply: tid=0 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 client.4701 shutdown 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 msgr2=0x55ca871f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1480759b0 tx=0x7fe148079610 comp rx=0 tx=0).stop 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 shutdown_connections 2024-04-26T21:12:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871d8880 0x55ca871d8c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca87206f60 0x55ca871e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d7790 0x55ca871f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2963631472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d4dd0 0x55ca871f1430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe1357fa640 0 client.4701 ms_handle_reset on v2:172.21.15.184:6835/2363826920 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 >> 172.21.15.40:0/2963631472 conn(0x55ca871d49f0 msgr2=0x55ca871f57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 shutdown_connections 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 1 -- 172.21.15.40:0/2963631472 wait complete. 2024-04-26T21:12:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.811+0000 7fe15b5de740 20 client.4701 trim_cache size 0 max 16384 2024-04-26T21:12:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 Processor -- start 2024-04-26T21:12:18.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 -- start start 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d7790 0x55ca871f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe15006e7e0 con 0x55ca871d9e50 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1501f76f0 con 0x55ca871d4dd0 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1500b1430 con 0x55ca871d7790 2024-04-26T21:12:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe156d86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe156d86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40852/0 (socket says 172.21.15.40:40852) 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.830+0000 7fe156d86640 1 -- 172.21.15.40:0/2053514180 learned_addr learned my addr 172.21.15.40:0/2053514180 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156585640 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe155d84640 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d7790 0x55ca871f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 -- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d7790 msgr2=0x55ca871f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d7790 0x55ca871f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 -- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 msgr2=0x55ca871d9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 -- 172.21.15.40:0/2053514180 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca871a8e30 con 0x55ca871d9e50 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe156d86640 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b2980 tx=0x7fe140009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410db7d7530b7279 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe1377fe640 1 -- 172.21.15.40:0/2053514180 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140026070 con 0x55ca871d9e50 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe1377fe640 1 -- 172.21.15.40:0/2053514180 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14001c040 con 0x55ca871d9e50 2024-04-26T21:12:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.831+0000 7fe1377fe640 1 -- 172.21.15.40:0/2053514180 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140004040 con 0x55ca871d9e50 2024-04-26T21:12:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- 172.21.15.40:0/2053514180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 msgr2=0x55ca871f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b2980 tx=0x7fe140009f90 comp rx=0 tx=0).stop 2024-04-26T21:12:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- 172.21.15.40:0/2053514180 shutdown_connections 2024-04-26T21:12:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d9e50 0x55ca871f3890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d7790 0x55ca871f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 --2- 172.21.15.40:0/2053514180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca871d9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- 172.21.15.40:0/2053514180 >> 172.21.15.40:0/2053514180 conn(0x55ca871d49f0 msgr2=0x55ca871e9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- 172.21.15.40:0/2053514180 shutdown_connections 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- 172.21.15.40:0/2053514180 wait complete. 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 -- start start 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.832+0000 7fe15b5de740 1 Processor -- start 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca87271800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d9e50 0x55ca871eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ca871d3920 con 0x55ca871d7790 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ca87207370 con 0x55ca871d4dd0 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55ca871d3ed0 con 0x55ca871d9e50 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe15b5de740 10 client.0 _ll_register_callbacks cb 0x7fe148079100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:12:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe155d84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe155d84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40858/0 (socket says 172.21.15.40:40858) 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe155d84640 1 -- 172.21.15.40:0/3594474108 learned_addr learned my addr 172.21.15.40:0/3594474108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe156585640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca87271800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.833+0000 7fe156d86640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d9e50 0x55ca871eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 -- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d9e50 msgr2=0x55ca871eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d9e50 0x55ca871eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 -- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 msgr2=0x55ca87271800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca87271800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 -- 172.21.15.40:0/3594474108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca871d16a0 con 0x55ca871d7790 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe156585640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca87271800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe155d84640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe144007e80 tx=0x7fe144001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f25ebb5c896a2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144015e00 con 0x55ca871d7790 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe144027ac0 con 0x55ca871d7790 2024-04-26T21:12:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14001c040 con 0x55ca871d7790 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fe144027cc0 con 0x55ca871d7790 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 10 client.4707 fetch_fsmap learned FSMap version 6 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55ca871f4360 con 0x55ca871d7790 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14001c040 con 0x55ca871d7790 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7fe1440290b0 con 0x55ca871d7790 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 10 client.4707 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 20 client.4707 populate_metadata read hostname 'smithi040' 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871eb280 0x55ca871ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 4 client.4707 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.834+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 --> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] -- command(tid 0: {"prefix": "status"}) -- 0x55ca871191c0 con 0x55ca871eb280 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe156585640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871eb280 0x55ca871ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe1357fa640 10 client.4707 ms_handle_connect on v2:172.21.15.184:6835/2363826920 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe156585640 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871eb280 0x55ca871ac3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441c0d068ac763d7 server_cookie=8c4a47488ff50fe7 in_seq=0 out_seq=0 2024-04-26T21:12:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe1357fa640 1 -- 172.21.15.40:0/3594474108 <== mds.0 v2:172.21.15.184:6835/2363826920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55ca871191c0 con 0x55ca871eb280 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe1357fa640 10 client.4707 handle_command_reply: tid=0 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 client.4707 shutdown 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 msgr2=0x55ca871d1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe144007e80 tx=0x7fe144001a40 comp rx=0 tx=0).stop 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 shutdown_connections 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:6835/2363826920,v1:172.21.15.184:6837/2363826920] conn(0x55ca871eb280 0x55ca871ac3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55ca871d9e50 0x55ca871eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ca871d7790 0x55ca871d1e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe1357fa640 0 client.4707 ms_handle_reset on v2:172.21.15.184:6835/2363826920 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 --2- 172.21.15.40:0/3594474108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca871d4dd0 0x55ca87271800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.835+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 >> 172.21.15.40:0/3594474108 conn(0x55ca871d49f0 msgr2=0x55ca87210cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 shutdown_connections 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe15b5de740 1 -- 172.21.15.40:0/3594474108 wait complete. 2024-04-26T21:12:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe15b5de740 20 client.4707 trim_cache size 0 max 16384 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4291, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 17.095046626999999, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 6, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 23, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 23, 2024-04-26T21:12:18.843 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 24.405091958 2024-04-26T21:12:18.844 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:12:18.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 msgr2=0x7fe13c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 0x7fe13c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1500c7d20 tx=0x7fe144009040 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 msgr2=0x7fe150160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe148001840 tx=0x7fe148001120 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 shutdown_connections 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1500c6fc0 0x7fe15015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe15015f480 0x7fe1500c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 --2- 172.21.15.40:0/2128133824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1380039e0 0x7fe150160180 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 >> 172.21.15.40:0/2128133824 conn(0x7fe1500b4a40 msgr2=0x7fe150070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 shutdown_connections 2024-04-26T21:12:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:18.836+0000 7fe1587e9640 1 -- 172.21.15.40:0/2128133824 wait complete. 2024-04-26T21:12:18.848 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:12:19.022 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:12:19.022 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:12:19.022 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:12:19.022 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T21:12:19.046 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T21:12:19.046 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T21:12:19.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.047 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:12:19.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:12:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-26T21:12:19.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:12:19.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.133 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:12:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:12:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-26T21:12:19.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:12:19.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.260 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:12:19.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:12:19.347 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:12:19.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:12:19.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.349 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:12:19.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:12:19.474 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:12:19.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:12:19.476 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T21:12:19.476 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:12:19.476 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:19.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:12:19.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.577+0000 7f6378fc1640 1 Processor -- start 2024-04-26T21:12:19.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.577+0000 7f6378fc1640 1 -- start start 2024-04-26T21:12:19.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6378fc1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63740cf2e0 0x7f637406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6378fc1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740d11b0 0x7f637406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6378fc1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63740d1740 con 0x7f637406e2c0 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6378fc1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f637406e710 con 0x7f63740d11b0 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6378fc1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f63740d1a20 con 0x7f63740cf2e0 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6371d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740d11b0 0x7f637406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6371d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740d11b0 0x7f637406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40872/0 (socket says 172.21.15.40:40872) 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f6371d9b640 1 -- 172.21.15.40:0/3314066263 learned_addr learned my addr 172.21.15.40:0/3314066263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.578+0000 7f637259c640 1 --2- 172.21.15.40:0/3314066263 >> v2:172.21.15.184:3300/0 conn(0x7f63740cf2e0 0x7f637406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f6371d9b640 1 -- 172.21.15.40:0/3314066263 >> v2:172.21.15.184:3300/0 conn(0x7f63740cf2e0 msgr2=0x7f637406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f6371d9b640 1 --2- 172.21.15.40:0/3314066263 >> v2:172.21.15.184:3300/0 conn(0x7f63740cf2e0 0x7f637406d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f6371d9b640 1 -- 172.21.15.40:0/3314066263 >> v1:172.21.15.184:6789/0 conn(0x7f637406e2c0 legacy=0x7f637415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f6371d9b640 1 -- 172.21.15.40:0/3314066263 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63740d1d00 con 0x7f63740d11b0 2024-04-26T21:12:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f6371d9b640 1 --2- 172.21.15.40:0/3314066263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740d11b0 0x7f637406dd80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f635c009870 tx=0x7f635c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ca3d297fc24179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f637159a640 1 -- 172.21.15.40:0/3314066263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c037070 con 0x7f63740d11b0 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f637159a640 1 -- 172.21.15.40:0/3314066263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c032440 con 0x7f63740d11b0 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.579+0000 7f637159a640 1 -- 172.21.15.40:0/3314066263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c032810 con 0x7f63740d11b0 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 -- 172.21.15.40:0/3314066263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740d11b0 msgr2=0x7f637406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 --2- 172.21.15.40:0/3314066263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740d11b0 0x7f637406dd80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f635c009870 tx=0x7f635c02a270 comp rx=0 tx=0).stop 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 -- 172.21.15.40:0/3314066263 shutdown_connections 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 --2- 172.21.15.40:0/3314066263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740d11b0 0x7f637406dd80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 --2- 172.21.15.40:0/3314066263 >> v2:172.21.15.184:3300/0 conn(0x7f63740cf2e0 0x7f637406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 -- 172.21.15.40:0/3314066263 >> 172.21.15.40:0/3314066263 conn(0x7f63740b4a60 msgr2=0x7f63740b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 -- 172.21.15.40:0/3314066263 shutdown_connections 2024-04-26T21:12:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.580+0000 7f6378fc1640 1 -- 172.21.15.40:0/3314066263 wait complete. 2024-04-26T21:12:19.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.581+0000 7f6378fc1640 1 Processor -- start 2024-04-26T21:12:19.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.581+0000 7f6378fc1640 1 -- start start 2024-04-26T21:12:19.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f637406e2c0 0x7f63741f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63740d11b0 0x7f637409d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f637406da10 con 0x7f63740cf2e0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63740d22c0 con 0x7f63740d11b0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6378fc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63740cf6c0 con 0x7f637406e2c0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6371d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6371d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40878/0 (socket says 172.21.15.40:40878) 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6371d9b640 1 -- 172.21.15.40:0/1972665776 learned_addr learned my addr 172.21.15.40:0/1972665776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6372d9d640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63740d11b0 0x7f637409d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f637259c640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f637406e2c0 0x7f63741f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.582+0000 7f6371d9b640 1 -- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f637406e2c0 msgr2=0x7f63741f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6371d9b640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f637406e2c0 0x7f63741f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6371d9b640 1 -- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63740d11b0 msgr2=0x7f637409d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6371d9b640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63740d11b0 0x7f637409d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6371d9b640 1 -- 172.21.15.40:0/1972665776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637409e290 con 0x7f63740cf2e0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6371d9b640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f635c0090b0 tx=0x7f635c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c037040 con 0x7f63740cf2e0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c004250 con 0x7f63740cf2e0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63740ca1e0 con 0x7f63740cf2e0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c030770 con 0x7f63740cf2e0 2024-04-26T21:12:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.583+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63740ca7a0 con 0x7f63740cf2e0 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.584+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637406a290 con 0x7f63740cf2e0 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.584+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f635c030910 con 0x7f63740cf2e0 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.585+0000 7f636affd640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 0x7f635406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.585+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f635c0ac8d0 con 0x7f63740cf2e0 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.585+0000 7f637259c640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 0x7f635406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.585+0000 7f637259c640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 0x7f635406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f636c00aac0 tx=0x7f636c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:19.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.589+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f635c07a740 con 0x7f63740cf2e0 2024-04-26T21:12:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.812+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f637406a290 con 0x7f63740cf2e0 2024-04-26T21:12:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.813+0000 7f636affd640 1 -- 172.21.15.40:0/1972665776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f635c07ea40 con 0x7f63740cf2e0 2024-04-26T21:12:19.814 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 msgr2=0x7f635406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 0x7f635406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f636c00aac0 tx=0x7f636c008040 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 msgr2=0x7f63741f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f635c0090b0 tx=0x7f635c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 shutdown_connections 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f635406c920 0x7f635406ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63740d11b0 0x7f637409d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cf2e0 0x7f63741f53f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 --2- 172.21.15.40:0/1972665776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f637406e2c0 0x7f63741f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 >> 172.21.15.40:0/1972665776 conn(0x7f63740b4a60 msgr2=0x7f63740c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 shutdown_connections 2024-04-26T21:12:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19.815+0000 7f6378fc1640 1 -- 172.21.15.40:0/1972665776 wait complete. 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:12:19.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:12:19.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:12:19.828 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:12:19.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:19.828 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:12:19.845 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:12:19.845 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:20.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:20.047 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:12:20.072 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:12:20.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:20.073 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:12:20.149 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:12:20.149 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:12:20.149 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:12:20.149 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:12:20.149 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:12:20.150 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:20.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:20.436 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:12:20.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:20.453 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-26T21:12:20.509 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:20.509 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:20.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:20.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:12:20.509 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:20.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:20.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:20.629 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:12:20.629 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:20.629 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:20.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:12:20.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:12:20.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:12:20.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:12:20.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:12:20.630 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:20.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:20.760 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:20.760 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:20.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:12:20.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:12:20.760 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:20.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:20.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:20.838 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:20.854 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:12:20.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:20.854 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:20.910 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:12:20.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:20.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:12:21.005 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:12:21.005 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:12:21.005 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:21.113 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:12:21.113 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:12:21.113 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:21.113 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:21.113 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:21.128 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:12:21.129 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:12:21.130 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:12:21.130 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:12:21.130 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:12:21.130 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:12:21.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:12:21.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:21.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:12:21.157 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:21.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:21.260 INFO:teuthology.orchestra.run.smithi040.stdout:client4603 2024-04-26T21:12:21.260 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:21.260 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:21.276 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:21.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:21.379 INFO:teuthology.orchestra.run.smithi040.stdout:client4603 2024-04-26T21:12:21.380 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:21.442 INFO:teuthology.orchestra.run.smithi040.stdout:client4603 2024-04-26T21:12:21.442 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eee7569e-7e8d-4976-9855-170ac833730d.client4603/status') 2024-04-26T21:12:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:21.505 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4603 (3)172.21.15.40:0/390315358 2024-04-26T21:12:21.505 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-04-26T21:12:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-04-26T21:12:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-04-26T21:12:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:69 bytes copied, 2.3143e-05 s, 3.0 MB/s 2024-04-26T21:12:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:21.513 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-26T21:12:21.513 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:12:21.513 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:12:21.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:12:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.614+0000 7fc59a87b640 1 Processor -- start 2024-04-26T21:12:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.614+0000 7fc59a87b640 1 -- start start 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.614+0000 7fc59a87b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc5940d11b0 0x7fc59406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.614+0000 7fc59a87b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc59406e2c0 0x7fc59415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.614+0000 7fc59a87b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5940d1740 con 0x7fc5940cf2e0 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc59a87b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc59406e710 con 0x7fc59406e2c0 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc59a87b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc5940d1a20 con 0x7fc5940d11b0 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc598e18640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc59406e2c0 0x7fc59415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc593fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc5940cf2e0 0x7fc59406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59028/0 (socket says 172.21.15.40:59028) 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc593fff640 1 -- 172.21.15.40:0/1136081484 learned_addr learned my addr 172.21.15.40:0/1136081484 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc5937fe640 1 --2- 172.21.15.40:0/1136081484 >> v2:172.21.15.184:3300/0 conn(0x7fc5940d11b0 0x7fc59406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc598e18640 1 -- 172.21.15.40:0/1136081484 >> v2:172.21.15.184:3300/0 conn(0x7fc5940d11b0 msgr2=0x7fc59406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc598e18640 1 --2- 172.21.15.40:0/1136081484 >> v2:172.21.15.184:3300/0 conn(0x7fc5940d11b0 0x7fc59406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc598e18640 1 -- 172.21.15.40:0/1136081484 >> v1:172.21.15.40:6789/0 conn(0x7fc5940cf2e0 legacy=0x7fc59406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc592ffd640 1 -- 172.21.15.40:0/1136081484 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878025352 0 0) 0x7fc5940d1740 con 0x7fc5940cf2e0 2024-04-26T21:12:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.615+0000 7fc598e18640 1 -- 172.21.15.40:0/1136081484 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5940d1d00 con 0x7fc59406e2c0 2024-04-26T21:12:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc598e18640 1 --2- 172.21.15.40:0/1136081484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc59406e2c0 0x7fc59415ebe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009d90 tx=0x7fc58c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9fb54b3ff3df97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc592ffd640 1 -- 172.21.15.40:0/1136081484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c007740 con 0x7fc59406e2c0 2024-04-26T21:12:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc592ffd640 1 -- 172.21.15.40:0/1136081484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c0078e0 con 0x7fc59406e2c0 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc592ffd640 1 -- 172.21.15.40:0/1136081484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c007cb0 con 0x7fc59406e2c0 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 -- 172.21.15.40:0/1136081484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc59406e2c0 msgr2=0x7fc59415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 --2- 172.21.15.40:0/1136081484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc59406e2c0 0x7fc59415ebe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009d90 tx=0x7fc58c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 -- 172.21.15.40:0/1136081484 shutdown_connections 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 --2- 172.21.15.40:0/1136081484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc59406e2c0 0x7fc59415ebe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 --2- 172.21.15.40:0/1136081484 >> v2:172.21.15.184:3300/0 conn(0x7fc5940d11b0 0x7fc59406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 -- 172.21.15.40:0/1136081484 >> 172.21.15.40:0/1136081484 conn(0x7fc5940b4a60 msgr2=0x7fc5940b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 -- 172.21.15.40:0/1136081484 shutdown_connections 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.616+0000 7fc59a87b640 1 -- 172.21.15.40:0/1136081484 wait complete. 2024-04-26T21:12:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.618+0000 7fc59a87b640 1 Processor -- start 2024-04-26T21:12:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.618+0000 7fc59a87b640 1 -- start start 2024-04-26T21:12:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc59406e2c0 0x7fc5940afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5940d11b0 0x7fc5940b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc5940cf6c0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5940d22c0 con 0x7fc59406e2c0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5940cf840 con 0x7fc5940d11b0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37842/0 (socket says 172.21.15.40:37842) 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 -- 172.21.15.40:0/3057053910 learned_addr learned my addr 172.21.15.40:0/3057053910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc593fff640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc59406e2c0 0x7fc5940afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc598e18640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5940d11b0 0x7fc5940b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 -- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5940d11b0 msgr2=0x7fc5940b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5940d11b0 0x7fc5940b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 -- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc59406e2c0 msgr2=0x7fc5940afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc59406e2c0 0x7fc5940afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 -- 172.21.15.40:0/3057053910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5940b1ca0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc5937fe640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d5f0 tx=0x7fc58800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880125f0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc588019070 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5940b1f80 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.619+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880129d0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.620+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5940b0260 con 0x7fc5940cf2e0 2024-04-26T21:12:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.620+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59406a290 con 0x7fc5940cf2e0 2024-04-26T21:12:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.621+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc588012b70 con 0x7fc5940cf2e0 2024-04-26T21:12:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.621+0000 7fc590ff9640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 0x7fc58006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.621+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc58808bd70 con 0x7fc5940cf2e0 2024-04-26T21:12:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.622+0000 7fc593fff640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 0x7fc58006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.622+0000 7fc593fff640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 0x7fc58006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc584007fd0 tx=0x7fc584007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.626+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc588059be0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.859+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc5940b8a80 con 0x7fc5940cf2e0 2024-04-26T21:12:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.859+0000 7fc590ff9640 1 -- 172.21.15.40:0/3057053910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fc58805dee0 con 0x7fc5940cf2e0 2024-04-26T21:12:21.860 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 msgr2=0x7fc58006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 0x7fc58006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc584007fd0 tx=0x7fc584007de0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 msgr2=0x7fc5941603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d5f0 tx=0x7fc58800dab0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 shutdown_connections 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc58006c8f0 0x7fc58006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5940d11b0 0x7fc5940b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5940cf2e0 0x7fc5941603b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 --2- 172.21.15.40:0/3057053910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc59406e2c0 0x7fc5940afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.861+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 >> 172.21.15.40:0/3057053910 conn(0x7fc5940b4a60 msgr2=0x7fc594160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.862+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 shutdown_connections 2024-04-26T21:12:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:21.862+0000 7fc59a87b640 1 -- 172.21.15.40:0/3057053910 wait complete. 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:12:21.873 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:12:21.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:21.874 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:12:21.889 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:21.889 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:12:21.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:12:21.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link tentative 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:21.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:21.892 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:12:21.953 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:12:21.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:21.955 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:12:22.011 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:12:22.011 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:12:22.011 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:12:22.011 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:12:22.011 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:12:22.011 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:22.011 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:22.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:12:22.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:12:22.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:12:22.012 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:22.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:22.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.136 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:12:22.151 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:12:22.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.152 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-26T21:12:22.207 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T21:12:22.207 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:22.207 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:22.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:12:22.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T21:12:22.207 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:22.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:12:22.347 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 06:68:0c:ad:12:77 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::468:cff:fead:1277/64 scope link 2024-04-26T21:12:22.348 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 06:68:0c:ad:12:77 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.414 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::468:cff:fead:1277/64 scope link 2024-04-26T21:12:22.415 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:22.416 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T21:12:22.416 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:22.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:22.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:22.566 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:22.566 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:12:22.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2024-04-26T21:12:22.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:12:22.566 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:12:22.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:22.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:22.643 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T21:12:22.672 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:12:22.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:22.672 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:12:22.726 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:12:22.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T21:12:22.809 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T21:12:22.809 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T21:12:22.809 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:22.818 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T21:12:22.818 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T21:12:22.818 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:22.818 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:22.818 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:22.825 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T21:12:22.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:12:22.826 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:12:22.827 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:12:22.827 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:12:22.828 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:12:22.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:22.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:12:22.887 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:22.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:22.984 INFO:teuthology.orchestra.run.smithi040.stdout:client4612 2024-04-26T21:12:22.984 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:22.985 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:23.000 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:23.097 INFO:teuthology.orchestra.run.smithi040.stdout:client4612 2024-04-26T21:12:23.097 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:23.153 INFO:teuthology.orchestra.run.smithi040.stdout:client4612 2024-04-26T21:12:23.154 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eee7569e-7e8d-4976-9855-170ac833730d.client4612/status') 2024-04-26T21:12:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:23.212 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4612 (3)172.21.15.40:0/3685803032 2024-04-26T21:12:23.212 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-04-26T21:12:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-04-26T21:12:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-04-26T21:12:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.6316e-05 s, 2.7 MB/s 2024-04-26T21:12:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:23.220 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T21:12:23.220 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:12:23.220 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:12:23.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:12:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.319+0000 7fa8f4746640 1 Processor -- start 2024-04-26T21:12:23.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.319+0000 7fa8f4746640 1 -- start start 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8ec13ff70 0x7fa8ec146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 0x7fa8ec1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec0d05b0 con 0x7fa8ec0cf810 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec0d0890 con 0x7fa8ec13ff70 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.320+0000 7fa8f4746640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec06e450 con 0x7fa8ec140350 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37856/0 (socket says 172.21.15.40:37856) 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 -- 172.21.15.40:0/3214090262 learned_addr learned my addr 172.21.15.40:0/3214090262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f1ce1640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 0x7fa8ec1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f24e2640 1 --2- 172.21.15.40:0/3214090262 >> v2:172.21.15.184:3300/0 conn(0x7fa8ec13ff70 0x7fa8ec146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 -- 172.21.15.40:0/3214090262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 msgr2=0x7fa8ec1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 0x7fa8ec1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 -- 172.21.15.40:0/3214090262 >> v2:172.21.15.184:3300/0 conn(0x7fa8ec13ff70 msgr2=0x7fa8ec146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 --2- 172.21.15.40:0/3214090262 >> v2:172.21.15.184:3300/0 conn(0x7fa8ec13ff70 0x7fa8ec146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f2ce3640 1 -- 172.21.15.40:0/3214090262 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec06e730 con 0x7fa8ec0cf810 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.321+0000 7fa8f1ce1640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 0x7fa8ec1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f2ce3640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec074200 tx=0x7fa8dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0bab4ea80b5f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f14e0640 1 -- 172.21.15.40:0/3214090262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc036070 con 0x7fa8ec0cf810 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f14e0640 1 -- 172.21.15.40:0/3214090262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc031600 con 0x7fa8ec0cf810 2024-04-26T21:12:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f14e0640 1 -- 172.21.15.40:0/3214090262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0319d0 con 0x7fa8ec0cf810 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 -- 172.21.15.40:0/3214090262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec0cf810 msgr2=0x7fa8ec0cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec074200 tx=0x7fa8dc02a610 comp rx=0 tx=0).stop 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 -- 172.21.15.40:0/3214090262 shutdown_connections 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec0cf810 0x7fa8ec0cfc80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 --2- 172.21.15.40:0/3214090262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec140350 0x7fa8ec1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 --2- 172.21.15.40:0/3214090262 >> v2:172.21.15.184:3300/0 conn(0x7fa8ec13ff70 0x7fa8ec146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.322+0000 7fa8f4746640 1 -- 172.21.15.40:0/3214090262 >> 172.21.15.40:0/3214090262 conn(0x7fa8ec0d0d30 msgr2=0x7fa8ec0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.323+0000 7fa8f4746640 1 -- 172.21.15.40:0/3214090262 shutdown_connections 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.323+0000 7fa8f4746640 1 -- 172.21.15.40:0/3214090262 wait complete. 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 Processor -- start 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 -- start start 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 0x7fa8ec13b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ec140350 0x7fa8ec1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec142a40 con 0x7fa8ec13ff70 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1428c0 con 0x7fa8ec140350 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.324+0000 7fa8f4746640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec142bc0 con 0x7fa8ec0cf810 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37866/0 (socket says 172.21.15.40:37866) 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 -- 172.21.15.40:0/1392318733 learned_addr learned my addr 172.21.15.40:0/1392318733 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f24e2640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 0x7fa8ec13b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f2ce3640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ec140350 0x7fa8ec1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 -- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 msgr2=0x7fa8ec13b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 0x7fa8ec13b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 -- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ec140350 msgr2=0x7fa8ec1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ec140350 0x7fa8ec1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 -- 172.21.15.40:0/1392318733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec1f6ce0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f24e2640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 0x7fa8ec13b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f1ce1640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d7d0 tx=0x7fa8e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004030 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e00041d0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000b6c0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec1f6fc0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.325+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec1f7580 con 0x7fa8ec13ff70 2024-04-26T21:12:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.326+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec06a290 con 0x7fa8ec13ff70 2024-04-26T21:12:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.327+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8e000b860 con 0x7fa8ec13ff70 2024-04-26T21:12:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.327+0000 7fa8eaffd640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.327+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa8e008be70 con 0x7fa8ec13ff70 2024-04-26T21:12:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.327+0000 7fa8f24e2640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.328+0000 7fa8f24e2640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 0x7fa8d806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006fd0 tx=0x7fa8e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:23.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.332+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8e005a2a0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.555+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa8ec13c1a0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.556+0000 7fa8eaffd640 1 -- 172.21.15.40:0/1392318733 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fa8e0059ba0 con 0x7fa8ec13ff70 2024-04-26T21:12:23.557 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 msgr2=0x7fa8d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 0x7fa8d806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006fd0 tx=0x7fa8e4008040 comp rx=0 tx=0).stop 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 msgr2=0x7fa8ec0b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d7d0 tx=0x7fa8e000dc90 comp rx=0 tx=0).stop 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 shutdown_connections 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ec140350 0x7fa8ec1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8ec13ff70 0x7fa8ec0b6770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 --2- 172.21.15.40:0/1392318733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ec0cf810 0x7fa8ec13b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:23.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 >> 172.21.15.40:0/1392318733 conn(0x7fa8ec0d0d30 msgr2=0x7fa8ec06eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:23.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 shutdown_connections 2024-04-26T21:12:23.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:23.557+0000 7fa8f4746640 1 -- 172.21.15.40:0/1392318733 wait complete. 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:12:23.569 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:12:23.570 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:12:23.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:23.570 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:12:23.588 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:23.588 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:23.588 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:23.588 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:23.589 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:12:23.590 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-04-26T21:12:23.590 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:12:23.590 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:23.590 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2024-04-26T21:12:23.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:12:23.591 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:23.685 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:23.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:23.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:23.859 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:12:23.884 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:12:23.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:23.886 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:12:23.999 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:12:24.000 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:12:24.000 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:12:24.000 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:12:24.000 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:12:24.000 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:24.000 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:24.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:12:24.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:12:24.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:12:24.001 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:24.048 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:24.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:24.234 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:12:24.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:24.249 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-04-26T21:12:24.304 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:24.304 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:24.304 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:12:24.305 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T21:12:24.305 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:24.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:24.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:24.420 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T21:12:24.420 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:24.420 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:24.420 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T21:12:24.420 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:12:24.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T21:12:24.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T21:12:24.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T21:12:24.421 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:24.468 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:24.538 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:24.543 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:24.544 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:24.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:12:24.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:12:24.544 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:24.591 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:24.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:24.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:24.618 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:12:24.633 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:12:24.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:24.633 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:12:24.684 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:12:24.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:24.684 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:12:24.779 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:12:24.779 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:12:24.780 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:24.879 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:12:24.879 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:12:24.879 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:24.879 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:24.879 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:24.884 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:12:24.884 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:12:24.884 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:24.884 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:24.884 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:12:24.885 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:12:24.886 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:12:24.886 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:12:24.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:12:24.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:24.887 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:12:24.910 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:24.999 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:25.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:25.006 INFO:teuthology.orchestra.run.smithi184.stdout:client4618 2024-04-26T21:12:25.006 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:25.006 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:25.022 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:25.110 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:25.111 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:25.117 INFO:teuthology.orchestra.run.smithi184.stdout:client4618 2024-04-26T21:12:25.117 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:25.165 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:25.166 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:25.172 INFO:teuthology.orchestra.run.smithi184.stdout:client4618 2024-04-26T21:12:25.172 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eee7569e-7e8d-4976-9855-170ac833730d.client4618/status') 2024-04-26T21:12:25.220 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:25.229 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4618 (3)192.168.0.1:0/3586394271 2024-04-26T21:12:25.229 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-04-26T21:12:25.229 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-04-26T21:12:25.229 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-04-26T21:12:25.229 INFO:teuthology.orchestra.run.smithi184.stderr:69 bytes copied, 2.2209e-05 s, 3.1 MB/s 2024-04-26T21:12:25.230 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:25.237 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-26T21:12:25.237 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:12:25.237 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:12:25.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:12:25.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.348+0000 7fd00dd78640 1 Processor -- start 2024-04-26T21:12:25.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 -- start start 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd008145980 0x7fd008160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0080b61a0 0x7fd0080720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0080b4260 con 0x7fd0080b3de0 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0080b5140 con 0x7fd008145980 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.349+0000 7fd00dd78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0080b4530 con 0x7fd0080b61a0 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd008145980 0x7fd008160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd008145980 0x7fd008160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37878/0 (socket says 172.21.15.40:37878) 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 -- 172.21.15.40:0/3651825496 learned_addr learned my addr 172.21.15.40:0/3651825496 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd007fff640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0080b61a0 0x7fd0080720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 -- 172.21.15.40:0/3651825496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0080b61a0 msgr2=0x7fd0080720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0080b61a0 0x7fd0080720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 -- 172.21.15.40:0/3651825496 >> v1:172.21.15.184:6789/0 conn(0x7fd0080b3de0 legacy=0x7fd0080b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.350+0000 7fd0077fe640 1 -- 172.21.15.40:0/3651825496 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0080b4810 con 0x7fd008145980 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd0077fe640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd008145980 0x7fd008160630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009870 tx=0x7fcff802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d73af35b65774c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd0067fc640 1 -- 172.21.15.40:0/3651825496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8037070 con 0x7fd008145980 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd0067fc640 1 -- 172.21.15.40:0/3651825496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8032440 con 0x7fd008145980 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd0067fc640 1 -- 172.21.15.40:0/3651825496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8032810 con 0x7fd008145980 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 -- 172.21.15.40:0/3651825496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd008145980 msgr2=0x7fd008160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd008145980 0x7fd008160630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009870 tx=0x7fcff802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 -- 172.21.15.40:0/3651825496 shutdown_connections 2024-04-26T21:12:25.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0080b61a0 0x7fd0080720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 --2- 172.21.15.40:0/3651825496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd008145980 0x7fd008160630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.351+0000 7fd00dd78640 1 -- 172.21.15.40:0/3651825496 >> 172.21.15.40:0/3651825496 conn(0x7fd0080d0d30 msgr2=0x7fd0080d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:25.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.352+0000 7fd00dd78640 1 -- 172.21.15.40:0/3651825496 shutdown_connections 2024-04-26T21:12:25.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.352+0000 7fd00dd78640 1 -- 172.21.15.40:0/3651825496 wait complete. 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.353+0000 7fd00dd78640 1 Processor -- start 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.353+0000 7fd00dd78640 1 -- start start 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 0x7fd00815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd008145980 0x7fd00813ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd008072610 con 0x7fd0080b61a0 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b4dd0 con 0x7fd0080b3de0 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd00dd78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd008072790 con 0x7fd008145980 2024-04-26T21:12:25.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37884/0 (socket says 172.21.15.40:37884) 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 -- 172.21.15.40:0/895378079 learned_addr learned my addr 172.21.15.40:0/895378079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd0077fe640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 0x7fd00815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd007fff640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd008145980 0x7fd00813ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 -- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd008145980 msgr2=0x7fd00813ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd008145980 0x7fd00813ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 -- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 msgr2=0x7fd00815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 0x7fd00815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 -- 172.21.15.40:0/895378079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00813f620 con 0x7fd0080b61a0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd0077fe640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 0x7fd00815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.354+0000 7fd006ffd640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c400 tx=0x7fcffc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.355+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0052d0 con 0x7fd0080b61a0 2024-04-26T21:12:25.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.355+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc005490 con 0x7fd0080b61a0 2024-04-26T21:12:25.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.355+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00813f900 con 0x7fd0080b61a0 2024-04-26T21:12:25.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.355+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007260 con 0x7fd0080b61a0 2024-04-26T21:12:25.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.355+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00813fec0 con 0x7fd0080b61a0 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.356+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00806a290 con 0x7fd0080b61a0 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.356+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcffc007400 con 0x7fd0080b61a0 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.357+0000 7fcfeffff640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.357+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcffc08b050 con 0x7fd0080b61a0 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.357+0000 7fd0077fe640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:25.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.357+0000 7fd0077fe640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 0x7fcff406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcff8034000 tx=0x7fcff802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:25.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.361+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcffc058e40 con 0x7fd0080b61a0 2024-04-26T21:12:25.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.590+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd008073cc0 con 0x7fd0080b61a0 2024-04-26T21:12:25.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.590+0000 7fcfeffff640 1 -- 172.21.15.40:0/895378079 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fcffc05d140 con 0x7fd0080b61a0 2024-04-26T21:12:25.591 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 msgr2=0x7fcff406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 0x7fcff406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcff8034000 tx=0x7fcff802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 msgr2=0x7fd0081f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c400 tx=0x7fcffc00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 shutdown_connections 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd008145980 0x7fd00813ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b61a0 0x7fd0081f2d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 --2- 172.21.15.40:0/895378079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0080b3de0 0x7fd00815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.592+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 >> 172.21.15.40:0/895378079 conn(0x7fd0080d0d30 msgr2=0x7fd00815b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.593+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 shutdown_connections 2024-04-26T21:12:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:25.593+0000 7fd00dd78640 1 -- 172.21.15.40:0/895378079 wait complete. 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:12:25.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:12:25.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:12:25.605 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:12:25.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:25.605 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:12:25.620 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:25.620 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:25.620 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:25.621 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link tentative 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.622 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:12:25.623 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:12:25.623 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link tentative 2024-04-26T21:12:25.623 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:25.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:25.623 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:12:25.683 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:12:25.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:25.685 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:12:25.740 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:12:25.740 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:12:25.740 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:12:25.740 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:12:25.740 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:12:25.740 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:25.741 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:25.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:12:25.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:12:25.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:12:25.741 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:25.828 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:25.859 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:25.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:25.865 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:12:25.880 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:12:25.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:25.880 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2024-04-26T21:12:25.934 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T21:12:25.934 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:25.934 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:25.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:12:25.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T21:12:25.935 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:26.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:26.044 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:26.050 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:26.073 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether f6:54:6a:8c:1a:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:12:26.074 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:12:26.074 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.074 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::f454:6aff:fe8c:1af3/64 scope link 2024-04-26T21:12:26.074 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:26.075 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:12:26.137 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether f6:54:6a:8c:1a:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::f454:6aff:fe8c:1af3/64 scope link 2024-04-26T21:12:26.138 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:12:26.139 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T21:12:26.140 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:26.227 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:26.280 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.286 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:12:26.286 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:12:26.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2024-04-26T21:12:26.287 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:12:26.287 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:12:26.334 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:26.361 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T21:12:26.375 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:12:26.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:12:26.375 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:12:26.430 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:12:26.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:26.430 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T21:12:26.511 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T21:12:26.511 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T21:12:26.512 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:26.520 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T21:12:26.520 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T21:12:26.520 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:26.520 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:26.520 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:12:26.525 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T21:12:26.526 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:12:26.526 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:12:26.526 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:12:26.526 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:12:26.527 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:12:26.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:26.527 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:12:26.591 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:26.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:26.680 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.686 INFO:teuthology.orchestra.run.smithi184.stdout:client4496 2024-04-26T21:12:26.686 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:26.687 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:26.702 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:26.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:26.791 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.796 INFO:teuthology.orchestra.run.smithi184.stdout:client4496 2024-04-26T21:12:26.797 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:12:26.844 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:12:26.846 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.851 INFO:teuthology.orchestra.run.smithi184.stdout:client4496 2024-04-26T21:12:26.852 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eee7569e-7e8d-4976-9855-170ac833730d.client4496/status') 2024-04-26T21:12:26.899 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:12:26.908 INFO:teuthology.orchestra.run.smithi184.stdout:instance: client.4496 (3)192.168.0.2:0/2893157278 2024-04-26T21:12:26.908 INFO:teuthology.orchestra.run.smithi184.stdout:blocklisted: false 2024-04-26T21:12:26.909 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records in 2024-04-26T21:12:26.909 INFO:teuthology.orchestra.run.smithi184.stderr:0+1 records out 2024-04-26T21:12:26.909 INFO:teuthology.orchestra.run.smithi184.stderr:69 bytes copied, 2.2454e-05 s, 3.1 MB/s 2024-04-26T21:12:26.909 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:12:26.915 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T21:12:26.931 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:27.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.043+0000 7f8f90efc640 1 Processor -- start 2024-04-26T21:12:27.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.043+0000 7f8f90efc640 1 -- start start 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.044+0000 7f8f90efc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f8c15f370 0x7f8f8c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.044+0000 7f8f90efc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f8c0d3ee0 0x7f8f8c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.044+0000 7f8f90efc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f8c0d03a0 con 0x7f8f8c15f750 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.044+0000 7f8f90efc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8c0b4d10 con 0x7f8f8c15f370 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.044+0000 7f8f90efc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8c0d0680 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f8c15f370 0x7f8f8c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f8c15f370 0x7f8f8c0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37886/0 (socket says 172.21.15.40:37886) 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f89d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8f8c15f750 0x7f8f8c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59034/0 (socket says 172.21.15.40:59034) 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 -- 172.21.15.40:0/37916814 learned_addr learned my addr 172.21.15.40:0/37916814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8ad9d640 1 --2- 172.21.15.40:0/37916814 >> v2:172.21.15.184:3300/0 conn(0x7f8f8c0d3ee0 0x7f8f8c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 -- 172.21.15.40:0/37916814 >> v2:172.21.15.184:3300/0 conn(0x7f8f8c0d3ee0 msgr2=0x7f8f8c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 --2- 172.21.15.40:0/37916814 >> v2:172.21.15.184:3300/0 conn(0x7f8f8c0d3ee0 0x7f8f8c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 -- 172.21.15.40:0/37916814 >> v1:172.21.15.40:6789/0 conn(0x7f8f8c15f750 legacy=0x7f8f8c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.045+0000 7f8f8a59c640 1 -- 172.21.15.40:0/37916814 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c0d0960 con 0x7f8f8c15f370 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f8a59c640 1 --2- 172.21.15.40:0/37916814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c15f370 0x7f8f8c0cfd70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0c8cf0 tx=0x7f8f8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9e17ad41d2b9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f8959a640 1 -- 172.21.15.40:0/37916814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84037070 con 0x7f8f8c15f370 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f8959a640 1 -- 172.21.15.40:0/37916814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f84032440 con 0x7f8f8c15f370 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f8959a640 1 -- 172.21.15.40:0/37916814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84032810 con 0x7f8f8c15f370 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f90efc640 1 -- 172.21.15.40:0/37916814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c15f370 msgr2=0x7f8f8c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.046+0000 7f8f90efc640 1 --2- 172.21.15.40:0/37916814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c15f370 0x7f8f8c0cfd70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0c8cf0 tx=0x7f8f8402a270 comp rx=0 tx=0).stop 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 -- 172.21.15.40:0/37916814 shutdown_connections 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 --2- 172.21.15.40:0/37916814 >> v2:172.21.15.184:3300/0 conn(0x7f8f8c0d3ee0 0x7f8f8c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 --2- 172.21.15.40:0/37916814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c15f370 0x7f8f8c0cfd70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 -- 172.21.15.40:0/37916814 >> 172.21.15.40:0/37916814 conn(0x7f8f8c08ec10 msgr2=0x7f8f8c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 -- 172.21.15.40:0/37916814 shutdown_connections 2024-04-26T21:12:27.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.047+0000 7f8f90efc640 1 -- 172.21.15.40:0/37916814 wait complete. 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.048+0000 7f8f90efc640 1 Processor -- start 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.048+0000 7f8f90efc640 1 -- start start 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f8c15f370 0x7f8f8c1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f8c15f750 0x7f8f8c14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c0b51c0 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c0d0f20 con 0x7f8f8c15f370 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f90efc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8c0b5340 con 0x7f8f8c15f750 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37896/0 (socket says 172.21.15.40:37896) 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 -- 172.21.15.40:0/4113851616 learned_addr learned my addr 172.21.15.40:0/4113851616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8ad9d640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f8c15f750 0x7f8f8c14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f89d9b640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f8c15f370 0x7f8f8c1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 -- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f8c15f750 msgr2=0x7f8f8c14c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f8c15f750 0x7f8f8c14c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 -- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f8c15f370 msgr2=0x7f8f8c1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f8c15f370 0x7f8f8c1f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 -- 172.21.15.40:0/4113851616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c14f490 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f8a59c640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f8402a2b0 tx=0x7f8f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.049+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84041030 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.050+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f84030440 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.050+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c0b7df0 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.050+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84037040 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.050+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c0b8370 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.051+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c06a290 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.051+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f84044070 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.051+0000 7f8f7affd640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 0x7f8f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.052+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8f840ac1d0 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.052+0000 7f8f89d9b640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 0x7f8f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.052+0000 7f8f89d9b640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 0x7f8f6c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0d1400 tx=0x7f8f80007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.056+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f8c14f490 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.242+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f8c0d3a90 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.243+0000 7f8f7affd640 1 -- 172.21.15.40:0/4113851616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f8f8407a720 con 0x7f8f8c0d3ee0 2024-04-26T21:12:27.245 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:27.245 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 msgr2=0x7f8f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 0x7f8f6c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0d1400 tx=0x7f8f80007cc0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 msgr2=0x7f8f8c14ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f8402a2b0 tx=0x7f8f840046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 shutdown_connections 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f6c06c8f0 0x7f8f6c06edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f8c15f750 0x7f8f8c14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f8c15f370 0x7f8f8c1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 --2- 172.21.15.40:0/4113851616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c0d3ee0 0x7f8f8c14ef50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 >> 172.21.15.40:0/4113851616 conn(0x7f8f8c08ec10 msgr2=0x7f8f8c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.245+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 shutdown_connections 2024-04-26T21:12:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.246+0000 7f8f90efc640 1 -- 172.21.15.40:0/4113851616 wait complete. 2024-04-26T21:12:27.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:27.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:27.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 Processor -- start 2024-04-26T21:12:27.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 -- start start 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f10180d3820 0x7f10180b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10180d3c90 con 0x7f10180b60b0 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10180d3f50 con 0x7f101815f680 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.362+0000 7f101e591640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f101815fa60 con 0x7f10180d3820 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f101cb2e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10180b60b0 0x7f10180d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59046/0 (socket says 172.21.15.40:59046) 2024-04-26T21:12:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f101cb2e640 1 -- 172.21.15.40:0/1120670196 learned_addr learned my addr 172.21.15.40:0/1120670196 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f10177fe640 1 --2- 172.21.15.40:0/1120670196 >> v2:172.21.15.184:3300/0 conn(0x7f10180d3820 0x7f10180b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f1016ffd640 1 -- 172.21.15.40:0/1120670196 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363865118 0 0) 0x7f10180d3c90 con 0x7f10180b60b0 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f1016ffd640 1 -- 172.21.15.40:0/1120670196 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1000003520 con 0x7f10180b60b0 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f10177fe640 1 -- 172.21.15.40:0/1120670196 >> v1:172.21.15.184:6789/0 conn(0x7f101815f680 legacy=0x7f1018163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f10177fe640 1 -- 172.21.15.40:0/1120670196 >> v1:172.21.15.40:6789/0 conn(0x7f10180b60b0 legacy=0x7f10180d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.363+0000 7f10177fe640 1 -- 172.21.15.40:0/1120670196 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180b5090 con 0x7f10180d3820 2024-04-26T21:12:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f10177fe640 1 --2- 172.21.15.40:0/1120670196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f10180b5b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f100c0097a0 tx=0x7f100c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8750cc16f5cd19fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f1016ffd640 1 -- 172.21.15.40:0/1120670196 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c037070 con 0x7f10180d3820 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f1016ffd640 1 -- 172.21.15.40:0/1120670196 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c032440 con 0x7f10180d3820 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f1016ffd640 1 -- 172.21.15.40:0/1120670196 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0327f0 con 0x7f10180d3820 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f101e591640 1 -- 172.21.15.40:0/1120670196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 msgr2=0x7f10180b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f101e591640 1 --2- 172.21.15.40:0/1120670196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f10180b5b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f100c0097a0 tx=0x7f100c02a230 comp rx=0 tx=0).stop 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f101e591640 1 -- 172.21.15.40:0/1120670196 shutdown_connections 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f101e591640 1 --2- 172.21.15.40:0/1120670196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f10180b5b70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.364+0000 7f101e591640 1 -- 172.21.15.40:0/1120670196 >> 172.21.15.40:0/1120670196 conn(0x7f101808eca0 msgr2=0x7f101809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.365+0000 7f101e591640 1 -- 172.21.15.40:0/1120670196 shutdown_connections 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.365+0000 7f101e591640 1 -- 172.21.15.40:0/1120670196 wait complete. 2024-04-26T21:12:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.366+0000 7f101e591640 1 Processor -- start 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.366+0000 7f101e591640 1 -- start start 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10180b60b0 0x7f10181f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f1018161f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10180d57b0 con 0x7f101815f680 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10180d5630 con 0x7f10180d3820 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10180d5930 con 0x7f10180b60b0 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37910/0 (socket says 172.21.15.40:37910) 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 -- 172.21.15.40:0/3650897983 learned_addr learned my addr 172.21.15.40:0/3650897983 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f1017fff640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10180b60b0 0x7f10181f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f10177fe640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f1018161f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 -- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10180b60b0 msgr2=0x7f10181f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10180b60b0 0x7f10181f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 -- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 msgr2=0x7f1018161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f1018161f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 -- 172.21.15.40:0/3650897983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180cc340 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101cb2e640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f101000cbe0 tx=0x7f101000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010017070 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101000ad40 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010012600 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.367+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10181f6180 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.368+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10180cc8c0 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.369+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101806a290 con 0x7f101815f680 2024-04-26T21:12:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.369+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101001a070 con 0x7f101815f680 2024-04-26T21:12:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.369+0000 7f1014ff9640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.370+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f101008b810 con 0x7f101815f680 2024-04-26T21:12:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.370+0000 7f1017fff640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.370+0000 7f1017fff640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10080016e0 tx=0x7f1008009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.374+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10100595f0 con 0x7f101815f680 2024-04-26T21:12:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.556+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10180bac00 con 0x7f101815f680 2024-04-26T21:12:27.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.556+0000 7f1014ff9640 1 -- 172.21.15.40:0/3650897983 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f101005d8f0 con 0x7f101815f680 2024-04-26T21:12:27.558 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.558+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 msgr2=0x7f100406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.558+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10080016e0 tx=0x7f1008009040 comp rx=0 tx=0).stop 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.558+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 msgr2=0x7f10181624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.558+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f101000cbe0 tx=0x7f101000a430 comp rx=0 tx=0).stop 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 shutdown_connections 2024-04-26T21:12:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f101815f680 0x7f10181624c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10180d3820 0x7f1018161f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 --2- 172.21.15.40:0/3650897983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10180b60b0 0x7f10181f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 >> 172.21.15.40:0/3650897983 conn(0x7f101808eca0 msgr2=0x7f101806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 shutdown_connections 2024-04-26T21:12:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.559+0000 7f101e591640 1 -- 172.21.15.40:0/3650897983 wait complete. 2024-04-26T21:12:27.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.673+0000 7f4590637640 1 Processor -- start 2024-04-26T21:12:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.673+0000 7f4590637640 1 -- start start 2024-04-26T21:12:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f4590637640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45881631f0 0x7f45880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f4590637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 0x7f45880b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f4590637640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458809c790 con 0x7f45880d1170 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f4590637640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f458806eec0 con 0x7f45881631f0 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f4590637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45880d15f0 con 0x7f458806ea30 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458e3d3640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45881631f0 0x7f45880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458ebd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 0x7f45880b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458dbd2640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f45880d1170 0x7f45881635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:38046/0 (socket says 172.21.15.40:38046) 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458e3d3640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45881631f0 0x7f45880b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:56376/0 (socket says 172.21.15.40:56376) 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458ebd4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 0x7f45880b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51826/0 (socket says 172.21.15.40:51826) 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.674+0000 7f458dbd2640 1 -- 172.21.15.40:0/3745941319 learned_addr learned my addr 172.21.15.40:0/3745941319 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458e3d3640 1 -- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 msgr2=0x7f45880b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458e3d3640 1 --2- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 0x7f45880b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458e3d3640 1 -- 172.21.15.40:0/3745941319 >> v1:172.21.15.184:6789/0 conn(0x7f45880d1170 legacy=0x7f45881635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458e3d3640 1 -- 172.21.15.40:0/3745941319 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4588163ec0 con 0x7f45881631f0 2024-04-26T21:12:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458d3d1640 1 -- 172.21.15.40:0/3745941319 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595713567 0 0) 0x7f458809c790 con 0x7f45880d1170 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.675+0000 7f458e3d3640 1 --2- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45881631f0 0x7f45880b2000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4578009dc0 tx=0x7f457802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f39da74c1bb5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f458d3d1640 1 -- 172.21.15.40:0/3745941319 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578007740 con 0x7f45881631f0 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f458d3d1640 1 -- 172.21.15.40:0/3745941319 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45780078e0 con 0x7f45881631f0 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f458d3d1640 1 -- 172.21.15.40:0/3745941319 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578007cb0 con 0x7f45881631f0 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 -- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45881631f0 msgr2=0x7f45880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 --2- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45881631f0 0x7f45880b2000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4578009dc0 tx=0x7f457802a5f0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 -- 172.21.15.40:0/3745941319 shutdown_connections 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 --2- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f458806ea30 0x7f45880b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 --2- 172.21.15.40:0/3745941319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45881631f0 0x7f45880b2000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 -- 172.21.15.40:0/3745941319 >> 172.21.15.40:0/3745941319 conn(0x7f458809ba40 msgr2=0x7f458809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 -- 172.21.15.40:0/3745941319 shutdown_connections 2024-04-26T21:12:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.676+0000 7f4590637640 1 -- 172.21.15.40:0/3745941319 wait complete. 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.677+0000 7f4590637640 1 Processor -- start 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.677+0000 7f4590637640 1 -- start start 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458806ea30 0x7f458814dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45880d1170 0x7f45881f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45880d2000 con 0x7f45881631f0 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45880d1d20 con 0x7f458806ea30 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f4590637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45880d0230 con 0x7f45880d1170 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f458ebd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f458ebd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37926/0 (socket says 172.21.15.40:37926) 2024-04-26T21:12:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f458ebd4640 1 -- 172.21.15.40:0/2002749507 learned_addr learned my addr 172.21.15.40:0/2002749507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f458dbd2640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45880d1170 0x7f45881f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.678+0000 7f458e3d3640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458806ea30 0x7f458814dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 -- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45880d1170 msgr2=0x7f45881f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45880d1170 0x7f45881f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 -- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458806ea30 msgr2=0x7f458814dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458806ea30 0x7f458814dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 -- 172.21.15.40:0/2002749507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45881f9650 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f458ebd4640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f458000e930 tx=0x7f458000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580017070 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458000c980 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458000cd70 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45881f9930 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.679+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458815c170 con 0x7f45881631f0 2024-04-26T21:12:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.680+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458815c370 con 0x7f45881631f0 2024-04-26T21:12:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.680+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f458001a070 con 0x7f45881631f0 2024-04-26T21:12:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.681+0000 7f4586ffd640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 0x7f457406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.681+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f458008be70 con 0x7f45881631f0 2024-04-26T21:12:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.681+0000 7f458e3d3640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 0x7f457406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.681+0000 7f458e3d3640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 0x7f457406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4578034000 tx=0x7f457802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.685+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f458815c370 con 0x7f45881631f0 2024-04-26T21:12:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.868+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f458806a290 con 0x7f45881631f0 2024-04-26T21:12:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.868+0000 7f4586ffd640 1 -- 172.21.15.40:0/2002749507 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f4580059c80 con 0x7f45881631f0 2024-04-26T21:12:27.869 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 msgr2=0x7f457406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 0x7f457406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4578034000 tx=0x7f457802f000 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 msgr2=0x7f45881eaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f458000e930 tx=0x7f458000edf0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 shutdown_connections 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f457406c8f0 0x7f457406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45881631f0 0x7f45881eaf70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45880d1170 0x7f45881f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 --2- 172.21.15.40:0/2002749507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f458806ea30 0x7f458814dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 >> 172.21.15.40:0/2002749507 conn(0x7f458809ba40 msgr2=0x7f4588070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 shutdown_connections 2024-04-26T21:12:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.870+0000 7f4590637640 1 -- 172.21.15.40:0/2002749507 wait complete. 2024-04-26T21:12:27.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:27.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.985+0000 7f18ad099640 1 Processor -- start 2024-04-26T21:12:27.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 -- start start 2024-04-26T21:12:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18a815d520 0x7f18a8161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f18a80cfd40 0x7f18a8073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a806f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f18a8161dd0 con 0x7f18a815d520 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f18a8070b00 con 0x7f18a80cfd40 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18ad099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18a8070de0 con 0x7f18a80735d0 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18a6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18a815d520 0x7f18a8161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18a6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18a815d520 0x7f18a8161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37940/0 (socket says 172.21.15.40:37940) 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.986+0000 7f18a6d9d640 1 -- 172.21.15.40:0/499019674 learned_addr learned my addr 172.21.15.40:0/499019674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 -- 172.21.15.40:0/499019674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 msgr2=0x7f18a806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 --2- 172.21.15.40:0/499019674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 -- 172.21.15.40:0/499019674 >> v2:172.21.15.184:3300/0 conn(0x7f18a80cfd40 msgr2=0x7f18a8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 --2- 172.21.15.40:0/499019674 >> v2:172.21.15.184:3300/0 conn(0x7f18a80cfd40 0x7f18a8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 -- 172.21.15.40:0/499019674 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a806dd60 con 0x7f18a815d520 2024-04-26T21:12:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a6d9d640 1 --2- 172.21.15.40:0/499019674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a815d520 0x7f18a8161890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ce7a0 tx=0x7f189802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0015460eec6856 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.987+0000 7f18a659c640 1 -- 172.21.15.40:0/499019674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898037070 con 0x7f18a815d520 2024-04-26T21:12:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18a659c640 1 -- 172.21.15.40:0/499019674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898032440 con 0x7f18a815d520 2024-04-26T21:12:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18a659c640 1 -- 172.21.15.40:0/499019674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898032810 con 0x7f18a815d520 2024-04-26T21:12:27.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 -- 172.21.15.40:0/499019674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a815d520 msgr2=0x7f18a8161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 --2- 172.21.15.40:0/499019674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a815d520 0x7f18a8161890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ce7a0 tx=0x7f189802a140 comp rx=0 tx=0).stop 2024-04-26T21:12:27.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 -- 172.21.15.40:0/499019674 shutdown_connections 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 --2- 172.21.15.40:0/499019674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 --2- 172.21.15.40:0/499019674 >> v2:172.21.15.184:3300/0 conn(0x7f18a80cfd40 0x7f18a8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 --2- 172.21.15.40:0/499019674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a815d520 0x7f18a8161890 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 -- 172.21.15.40:0/499019674 >> 172.21.15.40:0/499019674 conn(0x7f18a80d1e40 msgr2=0x7f18a80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 -- 172.21.15.40:0/499019674 shutdown_connections 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.988+0000 7f18ad099640 1 -- 172.21.15.40:0/499019674 wait complete. 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.989+0000 7f18ad099640 1 Processor -- start 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.989+0000 7f18ad099640 1 -- start start 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a80b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18a80cfd40 0x7f18a815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f18a80d18a0 con 0x7f18a80b76a0 2024-04-26T21:12:27.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8070790 con 0x7f18a80cfd40 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18ad099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f18a806f560 con 0x7f18a80735d0 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18a759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18a759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37952/0 (socket says 172.21.15.40:37952) 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.990+0000 7f18a759e640 1 -- 172.21.15.40:0/569498048 learned_addr learned my addr 172.21.15.40:0/569498048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 -- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 msgr2=0x7f18a80b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a80b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 -- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18a80cfd40 msgr2=0x7f18a815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18a80cfd40 0x7f18a815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 -- 172.21.15.40:0/569498048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a80b6600 con 0x7f18a80b76a0 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18a759e640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b970 tx=0x7f18a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0019070 con 0x7f18a80b76a0 2024-04-26T21:12:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a000cb60 con 0x7f18a80b76a0 2024-04-26T21:12:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0014440 con 0x7f18a80b76a0 2024-04-26T21:12:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a80b68e0 con 0x7f18a80b76a0 2024-04-26T21:12:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.991+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a80b6ea0 con 0x7f18a80b76a0 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.992+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a80693d0 con 0x7f18a80b76a0 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.992+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18a001c070 con 0x7f18a80b76a0 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.993+0000 7f189f7fe640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.993+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f18a008b880 con 0x7f18a80b76a0 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.993+0000 7f18a6d9d640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.994+0000 7f18a6d9d640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 0x7f188806ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:27.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:27.997+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18a0059cb0 con 0x7f18a80b76a0 2024-04-26T21:12:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.181+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18a80d1280 con 0x7f18a80b76a0 2024-04-26T21:12:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.182+0000 7f189f7fe640 1 -- 172.21.15.40:0/569498048 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f18a00595b0 con 0x7f18a80b76a0 2024-04-26T21:12:28.183 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 msgr2=0x7f188806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 0x7f188806ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802c000 comp rx=0 tx=0).stop 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 msgr2=0x7f18a815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b970 tx=0x7f18a000be30 comp rx=0 tx=0).stop 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 shutdown_connections 2024-04-26T21:12:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f188806c920 0x7f188806ede0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a80b76a0 0x7f18a815fa50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18a80cfd40 0x7f18a815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 --2- 172.21.15.40:0/569498048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18a80735d0 0x7f18a80b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 >> 172.21.15.40:0/569498048 conn(0x7f18a80d1e40 msgr2=0x7f18a80b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 shutdown_connections 2024-04-26T21:12:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.184+0000 7f18ad099640 1 -- 172.21.15.40:0/569498048 wait complete. 2024-04-26T21:12:28.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.296+0000 7f7585420640 1 Processor -- start 2024-04-26T21:12:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.296+0000 7f7585420640 1 -- start start 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7585420640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f758015f720 0x7f75800b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7585420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f75800d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7585420640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75800b2510 con 0x7f758015f340 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7585420640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f75800b27f0 con 0x7f758015f720 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7585420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75800b2ad0 con 0x7f75800d5610 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f758015f340 0x7f75800d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59048/0 (socket says 172.21.15.40:59048) 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757effd640 1 -- 172.21.15.40:0/309014008 learned_addr learned my addr 172.21.15.40:0/309014008 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757f7fe640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f75800d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382651265 0 0) 0x7f75800b2510 con 0x7f758015f340 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f7577fff640 1 --2- 172.21.15.40:0/309014008 >> v2:172.21.15.184:3300/0 conn(0x7f758015f720 0x7f75800b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f756c003520 con 0x7f758015f340 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 292484805 0 0) 0x7f756c003520 con 0x7f758015f340 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 msgr2=0x7f75800d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f75800d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 >> v2:172.21.15.184:3300/0 conn(0x7f758015f720 msgr2=0x7f75800b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 --2- 172.21.15.40:0/309014008 >> v2:172.21.15.184:3300/0 conn(0x7f758015f720 0x7f75800b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75800b2db0 con 0x7f758015f340 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7570001670 con 0x7f758015f340 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 >> v1:172.21.15.40:6789/0 conn(0x7f758015f340 legacy=0x7f75800d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f756c0039b0 0x7f756c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f756c004800 0x7f756c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f756c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75800b27f0 con 0x7f756c004800 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75800b2ad0 con 0x7f756c0039b0 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.297+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f756c00ba10 con 0x7f75800d5610 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f756c004800 0x7f756c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f757effd640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f756c0039b0 0x7f756c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f757f7fe640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f756c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 -- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 msgr2=0x7f756c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75800d5610 0x7f756c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 -- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f756c0039b0 msgr2=0x7f756c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f756c0039b0 0x7f756c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.298+0000 7f7577fff640 1 -- 172.21.15.40:0/309014008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c003520 con 0x7f756c004800 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7577fff640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f756c004800 0x7f756c004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ac0 tx=0x7f756c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874ce8e0f768ca5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568036070 con 0x7f756c004800 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568031400 con 0x7f756c004800 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f757e7fc640 1 -- 172.21.15.40:0/309014008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568009690 con 0x7f756c004800 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 -- 172.21.15.40:0/309014008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f756c004800 msgr2=0x7f756c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 --2- 172.21.15.40:0/309014008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f756c004800 0x7f756c004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ac0 tx=0x7f756c005830 comp rx=0 tx=0).stop 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f757f7fe640 1 -- 172.21.15.40:0/309014008 reap_dead start 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 -- 172.21.15.40:0/309014008 shutdown_connections 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 -- 172.21.15.40:0/309014008 >> 172.21.15.40:0/309014008 conn(0x7f758008ec70 msgr2=0x7f758009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 -- 172.21.15.40:0/309014008 shutdown_connections 2024-04-26T21:12:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.299+0000 7f7585420640 1 -- 172.21.15.40:0/309014008 wait complete. 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.300+0000 7f7585420640 1 Processor -- start 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 -- start start 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 0x7f758009ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7580155250 0x7f7580155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75800d40d0 con 0x7f75800cc320 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f75800d3f50 con 0x7f7580155250 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7585420640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f75800b35b0 con 0x7f756c004800 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7577fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7577fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37960/0 (socket says 172.21.15.40:37960) 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.301+0000 7f7577fff640 1 -- 172.21.15.40:0/3888669002 learned_addr learned my addr 172.21.15.40:0/3888669002 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f757f7fe640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7580155250 0x7f7580155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f757effd640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 0x7f758009ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 -- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 msgr2=0x7f758009ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 0x7f758009ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 -- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7580155250 msgr2=0x7f7580155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7580155250 0x7f7580155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 -- 172.21.15.40:0/3888669002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580155bd0 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f757effd640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 0x7f758009ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7577fff640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7568034040 tx=0x7f7568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568041020 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756803c470 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75800cd480 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568036040 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.302+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75800ccb50 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.303+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758006a290 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.303+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7568044020 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.304+0000 7f75777fe640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 0x7f756006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.304+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f75680ac350 con 0x7f75800cc320 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.304+0000 7f757effd640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 0x7f756006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.305+0000 7f757effd640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 0x7f756006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7570001eb0 tx=0x7f7570009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.308+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f756804b030 con 0x7f75800cc320 2024-04-26T21:12:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.494+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75800d3020 con 0x7f75800cc320 2024-04-26T21:12:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.495+0000 7f75777fe640 1 -- 172.21.15.40:0/3888669002 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f756807a780 con 0x7f75800cc320 2024-04-26T21:12:28.496 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:28.497 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:28.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 msgr2=0x7f756006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 0x7f756006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7570001eb0 tx=0x7f7570009040 comp rx=0 tx=0).stop 2024-04-26T21:12:28.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 msgr2=0x7f75800ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7568034040 tx=0x7f7568004730 comp rx=0 tx=0).stop 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 shutdown_connections 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f756006c920 0x7f756006ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7580155250 0x7f7580155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75800cc320 0x7f75800ca040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 --2- 172.21.15.40:0/3888669002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f756c004800 0x7f758009ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 >> 172.21.15.40:0/3888669002 conn(0x7f758008ec70 msgr2=0x7f75800d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 shutdown_connections 2024-04-26T21:12:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.497+0000 7f7585420640 1 -- 172.21.15.40:0/3888669002 wait complete. 2024-04-26T21:12:28.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:28.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.612+0000 7fc8ea074640 1 Processor -- start 2024-04-26T21:12:28.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.612+0000 7fc8ea074640 1 -- start start 2024-04-26T21:12:28.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8ea074640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8e406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8ea074640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc8e40b33e0 0x7fc8e4163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8ea074640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e409c700 con 0x7fc8e40d0010 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8ea074640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e40b4480 con 0x7fc8e40b33e0 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8ea074640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e406e050 con 0x7fc8e40b3c20 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8e40d0010 0x7fc8e40b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59062/0 (socket says 172.21.15.40:59062) 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e37fe640 1 -- 172.21.15.40:0/3462946731 learned_addr learned my addr 172.21.15.40:0/3462946731 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088879321 0 0) 0x7fc8e409c700 con 0x7fc8e40d0010 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8cc003520 con 0x7fc8e40d0010 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e3fff640 1 --2- 172.21.15.40:0/3462946731 >> v2:172.21.15.184:3300/0 conn(0x7fc8e40b33e0 0x7fc8e4163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e2ffd640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8e406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2143333995 0 0) 0x7fc8cc003520 con 0x7fc8e40d0010 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 msgr2=0x7fc8e406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8e406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 >> v2:172.21.15.184:3300/0 conn(0x7fc8e40b33e0 msgr2=0x7fc8e4163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.613+0000 7fc8e27fc640 1 --2- 172.21.15.40:0/3462946731 >> v2:172.21.15.184:3300/0 conn(0x7fc8e40b33e0 0x7fc8e4163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e40d1c50 con 0x7fc8e40d0010 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc8d8001670 con 0x7fc8e40d0010 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 >> v1:172.21.15.40:6789/0 conn(0x7fc8e40d0010 legacy=0x7fc8e40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8cc004800 0x7fc8cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b4480 con 0x7fc8cc0039b0 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e406e050 con 0x7fc8cc004800 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc00b9c0 con 0x7fc8e40b3c20 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e3fff640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e2ffd640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8cc004800 0x7fc8cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 -- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 msgr2=0x7fc8cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e40b3c20 0x7fc8cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 -- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8cc004800 msgr2=0x7fc8cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8cc004800 0x7fc8cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.614+0000 7fc8e37fe640 1 -- 172.21.15.40:0/3462946731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc003520 con 0x7fc8cc0039b0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8cc003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8001050 tx=0x7fc8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9505ebd11e98dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d802e070 con 0x7fc8cc0039b0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d8005af0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8e27fc640 1 -- 172.21.15.40:0/3462946731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8007d70 con 0x7fc8cc0039b0 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 -- 172.21.15.40:0/3462946731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 msgr2=0x7fc8cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 --2- 172.21.15.40:0/3462946731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8cc003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8001050 tx=0x7fc8cc005830 comp rx=0 tx=0).stop 2024-04-26T21:12:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8e3fff640 1 -- 172.21.15.40:0/3462946731 reap_dead start 2024-04-26T21:12:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 -- 172.21.15.40:0/3462946731 shutdown_connections 2024-04-26T21:12:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 -- 172.21.15.40:0/3462946731 >> 172.21.15.40:0/3462946731 conn(0x7fc8e409ba40 msgr2=0x7fc8e409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 -- 172.21.15.40:0/3462946731 shutdown_connections 2024-04-26T21:12:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.615+0000 7fc8ea074640 1 -- 172.21.15.40:0/3462946731 wait complete. 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.617+0000 7fc8ea074640 1 Processor -- start 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.617+0000 7fc8ea074640 1 -- start start 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.617+0000 7fc8ea074640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.617+0000 7fc8ea074640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e41f54c0 0x7fc8e41f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 0x7fc8e41f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b39f0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b3870 con 0x7fc8e41f95d0 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e406db10 con 0x7fc8e41f54c0 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37986/0 (socket says 172.21.15.40:37986) 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 -- 172.21.15.40:0/1810366413 learned_addr learned my addr 172.21.15.40:0/1810366413 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e2ffd640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e41f54c0 0x7fc8e41f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e3fff640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 0x7fc8e41f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 -- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e41f54c0 msgr2=0x7fc8e41f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e41f54c0 0x7fc8e41f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 -- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 msgr2=0x7fc8e41f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 0x7fc8e41f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 -- 172.21.15.40:0/1810366413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41eb4d0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e3fff640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 0x7fc8e41f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8e37fe640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8034040 tx=0x7fc8d8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8042070 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d8007480 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e409cfd0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8007d50 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.618+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e4151790 con 0x7fc8cc0039b0 2024-04-26T21:12:28.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.619+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e406a290 con 0x7fc8cc0039b0 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.620+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d8045070 con 0x7fc8cc0039b0 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.620+0000 7fc8cbfff640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 0x7fc8d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.621+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc8d80ac8e0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.621+0000 7fc8e2ffd640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 0x7fc8d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.621+0000 7fc8e2ffd640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 0x7fc8d006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8e41f58a0 tx=0x7fc8d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.625+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8d8005ac0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.807+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8e40b2550 con 0x7fc8cc0039b0 2024-04-26T21:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.807+0000 7fc8cbfff640 1 -- 172.21.15.40:0/1810366413 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fc8d807aec0 con 0x7fc8cc0039b0 2024-04-26T21:12:28.809 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 msgr2=0x7fc8d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 0x7fc8d006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8e41f58a0 tx=0x7fc8d4009190 comp rx=0 tx=0).stop 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 msgr2=0x7fc8e4151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8034040 tx=0x7fc8d8000fe0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 shutdown_connections 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8d006c8f0 0x7fc8d006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8e41f95d0 0x7fc8e41f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8e41f54c0 0x7fc8e41f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 --2- 172.21.15.40:0/1810366413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8cc0039b0 0x7fc8e4151250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 >> 172.21.15.40:0/1810366413 conn(0x7fc8e409ba40 msgr2=0x7fc8e40b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 shutdown_connections 2024-04-26T21:12:28.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.809+0000 7fc8ea074640 1 -- 172.21.15.40:0/1810366413 wait complete. 2024-04-26T21:12:28.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.932+0000 7f6429c93640 1 Processor -- start 2024-04-26T21:12:28.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.932+0000 7f6429c93640 1 -- start start 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6429c93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64240d1be0 0x7f64240d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6429c93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64240d4240 con 0x7f642415f490 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6429c93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64240d31a0 con 0x7f64240ba3a0 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6429c93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64240d3480 con 0x7f64240d1be0 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6422ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64240d1be0 0x7f64240d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f64237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f642415f490 0x7f64240d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59064/0 (socket says 172.21.15.40:59064) 2024-04-26T21:12:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f64237fe640 1 -- 172.21.15.40:0/603765866 learned_addr learned my addr 172.21.15.40:0/603765866 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6422ffd640 1 --2- 172.21.15.40:0/603765866 >> v2:172.21.15.40:3300/0 conn(0x7f64240d1be0 0x7f64240d2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37992/0 (socket says 172.21.15.40:37992) 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6422ffd640 1 -- 172.21.15.40:0/603765866 >> v1:172.21.15.184:6789/0 conn(0x7f64240ba3a0 legacy=0x7f64240ba7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6422ffd640 1 -- 172.21.15.40:0/603765866 >> v1:172.21.15.40:6789/0 conn(0x7f642415f490 legacy=0x7f64240d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f6422ffd640 1 -- 172.21.15.40:0/603765866 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64240d3760 con 0x7f64240d1be0 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.933+0000 7f64227fc640 1 -- 172.21.15.40:0/603765866 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374532839 0 0) 0x7f64240d4240 con 0x7f642415f490 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6422ffd640 1 --2- 172.21.15.40:0/603765866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64240d1be0 0x7f64240d2070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0c257c7cb4ee4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f64227fc640 1 -- 172.21.15.40:0/603765866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418037070 con 0x7f64240d1be0 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f64227fc640 1 -- 172.21.15.40:0/603765866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6418032440 con 0x7f64240d1be0 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f64227fc640 1 -- 172.21.15.40:0/603765866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418032810 con 0x7f64240d1be0 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 -- 172.21.15.40:0/603765866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64240d1be0 msgr2=0x7f64240d2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 --2- 172.21.15.40:0/603765866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64240d1be0 0x7f64240d2070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 -- 172.21.15.40:0/603765866 shutdown_connections 2024-04-26T21:12:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 --2- 172.21.15.40:0/603765866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64240d1be0 0x7f64240d2070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 -- 172.21.15.40:0/603765866 >> 172.21.15.40:0/603765866 conn(0x7f642409e530 msgr2=0x7f64240d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 -- 172.21.15.40:0/603765866 shutdown_connections 2024-04-26T21:12:28.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.934+0000 7f6429c93640 1 -- 172.21.15.40:0/603765866 wait complete. 2024-04-26T21:12:28.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 Processor -- start 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 -- start start 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64240ba3a0 0x7f64240cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64240d1be0 0x7f64240c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64240d11c0 con 0x7f642415f490 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64240d3d20 con 0x7f64240ba3a0 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.936+0000 7f6429c93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f642415f870 con 0x7f64240d1be0 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38002/0 (socket says 172.21.15.40:38002) 2024-04-26T21:12:28.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 -- 172.21.15.40:0/3946638687 learned_addr learned my addr 172.21.15.40:0/3946638687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6422ffd640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64240d1be0 0x7f64240c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f64237fe640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64240ba3a0 0x7f64240cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 -- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64240d1be0 msgr2=0x7f64240c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64240d1be0 0x7f64240c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 -- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64240ba3a0 msgr2=0x7f64240cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64240ba3a0 0x7f64240cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 -- 172.21.15.40:0/3946638687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64240cdc60 con 0x7f642415f490 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6423fff640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f641c00b990 tx=0x7f641c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c019070 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c00cba0 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c014440 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64240cdf40 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.937+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64240ce500 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.938+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642406a290 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.939+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f641c01c070 con 0x7f642415f490 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.939+0000 7f640bfff640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 0x7f641006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:28.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.939+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f641c08b8b0 con 0x7f642415f490 2024-04-26T21:12:28.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.940+0000 7f64237fe640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 0x7f641006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:28.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.940+0000 7f64237fe640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 0x7f641006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6414007fd0 tx=0x7f6414007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:28.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:28.944+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f641c059600 con 0x7f642415f490 2024-04-26T21:12:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.132+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f642415ec90 con 0x7f642415f490 2024-04-26T21:12:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.132+0000 7f640bfff640 1 -- 172.21.15.40:0/3946638687 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f641c05d900 con 0x7f642415f490 2024-04-26T21:12:29.134 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 msgr2=0x7f641006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 0x7f641006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6414007fd0 tx=0x7f6414007de0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 msgr2=0x7f642414ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f641c00b990 tx=0x7f641c00be50 comp rx=0 tx=0).stop 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 shutdown_connections 2024-04-26T21:12:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641006c8f0 0x7f641006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f642415f490 0x7f642414ad00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64240d1be0 0x7f64240c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 --2- 172.21.15.40:0/3946638687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64240ba3a0 0x7f64240cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.134+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 >> 172.21.15.40:0/3946638687 conn(0x7f642409e530 msgr2=0x7f64240d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.135+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 shutdown_connections 2024-04-26T21:12:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.135+0000 7f6429c93640 1 -- 172.21.15.40:0/3946638687 wait complete. 2024-04-26T21:12:29.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:29.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.252+0000 7f2251864640 1 Processor -- start 2024-04-26T21:12:29.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.252+0000 7f2251864640 1 -- start start 2024-04-26T21:12:29.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.253+0000 7f2251864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 0x7f224c142d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.253+0000 7f2251864640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.253+0000 7f2251864640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c1611d0 con 0x7f224c160c80 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.253+0000 7f2251864640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f224c06c5d0 con 0x7f224c0cef90 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.253+0000 7f2251864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f224c1614b0 con 0x7f224c15d800 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224affd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 0x7f224c142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:56454/0 (socket says 172.21.15.40:56454) 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224a7fc640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f224c160c80 0x7f224c0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:38062/0 (socket says 172.21.15.40:38062) 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224b7fe640 1 -- 172.21.15.40:0/2766063449 learned_addr learned my addr 172.21.15.40:0/2766063449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224affd640 1 -- 172.21.15.40:0/2766063449 >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 msgr2=0x7f224c143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224affd640 1 --2- 172.21.15.40:0/2766063449 >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224affd640 1 -- 172.21.15.40:0/2766063449 >> v1:172.21.15.184:6789/0 conn(0x7f224c160c80 legacy=0x7f224c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224affd640 1 -- 172.21.15.40:0/2766063449 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c161790 con 0x7f224c15d800 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.254+0000 7f224b7fe640 1 --2- 172.21.15.40:0/2766063449 >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f224affd640 1 --2- 172.21.15.40:0/2766063449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 0x7f224c142d30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f22440097a0 tx=0x7f224402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc20feb3e3e8c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2249ffb640 1 -- 172.21.15.40:0/2766063449 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244037070 con 0x7f224c15d800 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2249ffb640 1 -- 172.21.15.40:0/2766063449 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244032440 con 0x7f224c15d800 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2249ffb640 1 -- 172.21.15.40:0/2766063449 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244032810 con 0x7f224c15d800 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 -- 172.21.15.40:0/2766063449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 msgr2=0x7f224c142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 --2- 172.21.15.40:0/2766063449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 0x7f224c142d30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f22440097a0 tx=0x7f224402a180 comp rx=0 tx=0).stop 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 -- 172.21.15.40:0/2766063449 shutdown_connections 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 --2- 172.21.15.40:0/2766063449 >> v2:172.21.15.184:3300/0 conn(0x7f224c0cef90 0x7f224c143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 --2- 172.21.15.40:0/2766063449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c15d800 0x7f224c142d30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 -- 172.21.15.40:0/2766063449 >> 172.21.15.40:0/2766063449 conn(0x7f224c0b3b60 msgr2=0x7f224c0b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 -- 172.21.15.40:0/2766063449 shutdown_connections 2024-04-26T21:12:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.255+0000 7f2251864640 1 -- 172.21.15.40:0/2766063449 wait complete. 2024-04-26T21:12:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.257+0000 7f2251864640 1 Processor -- start 2024-04-26T21:12:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.257+0000 7f2251864640 1 -- start start 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.257+0000 7f2251864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c0cef90 0x7f224c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.257+0000 7f2251864640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.257+0000 7f2251864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f224c160c80 0x7f224c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2251864640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f224c15dbe0 con 0x7f224c15d800 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2251864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f224c161a70 con 0x7f224c160c80 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2251864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f224c15dd60 con 0x7f224c0cef90 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38008/0 (socket says 172.21.15.40:38008) 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 -- 172.21.15.40:0/2849139344 learned_addr learned my addr 172.21.15.40:0/2849139344 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224affd640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c0cef90 0x7f224c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224b7fe640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f224c160c80 0x7f224c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 -- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c0cef90 msgr2=0x7f224c155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c0cef90 0x7f224c155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 -- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f224c160c80 msgr2=0x7f224c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f224c160c80 0x7f224c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 -- 172.21.15.40:0/2849139344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c15ace0 con 0x7f224c15d800 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f224a7fc640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f223400c400 tx=0x7f223400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340074e0 con 0x7f224c15d800 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2234007680 con 0x7f224c15d800 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1f6860 con 0x7f224c15d800 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004a00 con 0x7f224c15d800 2024-04-26T21:12:29.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.258+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1f6e20 con 0x7f224c15d800 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.259+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c0693d0 con 0x7f224c15d800 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.260+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f223401a020 con 0x7f224c15d800 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.260+0000 7f2243fff640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 0x7f222c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.260+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f223408b230 con 0x7f224c15d800 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.261+0000 7f224affd640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 0x7f222c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.261+0000 7f224affd640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 0x7f222c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2244034000 tx=0x7f224402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.264+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22340596f0 con 0x7f224c15d800 2024-04-26T21:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.452+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f224c06e3c0 con 0x7f224c15d800 2024-04-26T21:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.452+0000 7f2243fff640 1 -- 172.21.15.40:0/2849139344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f2234058ff0 con 0x7f224c15d800 2024-04-26T21:12:29.454 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.454+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 msgr2=0x7f222c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.454+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 0x7f222c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2244034000 tx=0x7f224402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.454+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 msgr2=0x7f224c1f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.454+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f223400c400 tx=0x7f223400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 shutdown_connections 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f222c06c8f0 0x7f222c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f224c160c80 0x7f224c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f224c15d800 0x7f224c1f60c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 --2- 172.21.15.40:0/2849139344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f224c0cef90 0x7f224c155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 >> 172.21.15.40:0/2849139344 conn(0x7f224c0b3b60 msgr2=0x7f224c1601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 shutdown_connections 2024-04-26T21:12:29.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.455+0000 7f2251864640 1 -- 172.21.15.40:0/2849139344 wait complete. 2024-04-26T21:12:29.467 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.566+0000 7fa2654bd640 1 Processor -- start 2024-04-26T21:12:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.566+0000 7fa2654bd640 1 -- start start 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa2654bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa26013ff70 0x7fa260146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa2654bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2600d05b0 con 0x7fa2600cff20 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa2654bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2600d0890 con 0x7fa260140350 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa2654bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa26006e450 con 0x7fa26013ff70 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa25f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2600cff20 0x7fa26006ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59068/0 (socket says 172.21.15.40:59068) 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa25f7fe640 1 -- 172.21.15.40:0/1307183102 learned_addr learned my addr 172.21.15.40:0/1307183102 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa25effd640 1 --2- 172.21.15.40:0/1307183102 >> v2:172.21.15.184:3300/0 conn(0x7fa26013ff70 0x7fa260146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397147754 0 0) 0x7fa2600d05b0 con 0x7fa2600cff20 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.567+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa24c003520 con 0x7fa2600cff20 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.568+0000 7fa25effd640 1 -- 172.21.15.40:0/1307183102 >> v1:172.21.15.184:6789/0 conn(0x7fa260140350 legacy=0x7fa2600cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.568+0000 7fa25effd640 1 -- 172.21.15.40:0/1307183102 >> v1:172.21.15.40:6789/0 conn(0x7fa2600cff20 legacy=0x7fa26006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.568+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585971164 0 0) 0x7fa2600d0890 con 0x7fa260140350 2024-04-26T21:12:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.568+0000 7fa25effd640 1 -- 172.21.15.40:0/1307183102 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26006e730 con 0x7fa26013ff70 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.568+0000 7fa25effd640 1 --2- 172.21.15.40:0/1307183102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26013ff70 0x7fa260146080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097a0 tx=0x7fa25802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543baa80e180a94a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258037070 con 0x7fa26013ff70 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258032440 con 0x7fa26013ff70 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa25dffb640 1 -- 172.21.15.40:0/1307183102 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258032810 con 0x7fa26013ff70 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 -- 172.21.15.40:0/1307183102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26013ff70 msgr2=0x7fa260146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 --2- 172.21.15.40:0/1307183102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26013ff70 0x7fa260146080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097a0 tx=0x7fa25802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 -- 172.21.15.40:0/1307183102 shutdown_connections 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 --2- 172.21.15.40:0/1307183102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26013ff70 0x7fa260146080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 -- 172.21.15.40:0/1307183102 >> 172.21.15.40:0/1307183102 conn(0x7fa2600d0d30 msgr2=0x7fa2600d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 -- 172.21.15.40:0/1307183102 shutdown_connections 2024-04-26T21:12:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.569+0000 7fa2654bd640 1 -- 172.21.15.40:0/1307183102 wait complete. 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.570+0000 7fa2654bd640 1 Processor -- start 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.570+0000 7fa2654bd640 1 -- start start 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2600cff20 0x7fa2600ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 0x7fa26013ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa260146990 con 0x7fa26013ff70 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa260146810 con 0x7fa2600cff20 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa2654bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa26006f360 con 0x7fa260140350 2024-04-26T21:12:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa25e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa25e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38012/0 (socket says 172.21.15.40:38012) 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.571+0000 7fa25e7fc640 1 -- 172.21.15.40:0/4149475121 learned_addr learned my addr 172.21.15.40:0/4149475121 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25effd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2600cff20 0x7fa2600ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25f7fe640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 0x7fa26013ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 -- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 msgr2=0x7fa26013ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 0x7fa26013ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 -- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2600cff20 msgr2=0x7fa2600ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2600cff20 0x7fa2600ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 -- 172.21.15.40:0/4149475121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2601f84b0 con 0x7fa26013ff70 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25f7fe640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 0x7fa26013ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa25e7fc640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c410 tx=0x7fa24800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248017070 con 0x7fa26013ff70 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2480035f0 con 0x7fa26013ff70 2024-04-26T21:12:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2601f8790 con 0x7fa26013ff70 2024-04-26T21:12:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2480039e0 con 0x7fa26013ff70 2024-04-26T21:12:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.572+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2601f8d50 con 0x7fa26013ff70 2024-04-26T21:12:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.573+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26006a290 con 0x7fa26013ff70 2024-04-26T21:12:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.574+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa24801a070 con 0x7fa26013ff70 2024-04-26T21:12:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.574+0000 7fa2577fe640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 0x7fa24006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.574+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa24808ae20 con 0x7fa26013ff70 2024-04-26T21:12:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.574+0000 7fa25effd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 0x7fa24006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.575+0000 7fa25effd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 0x7fa24006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa258034000 tx=0x7fa25802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.579+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa248058c90 con 0x7fa26013ff70 2024-04-26T21:12:29.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.767+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2600b45e0 con 0x7fa26013ff70 2024-04-26T21:12:29.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.768+0000 7fa2577fe640 1 -- 172.21.15.40:0/4149475121 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7fa24805cf90 con 0x7fa26013ff70 2024-04-26T21:12:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:29.769 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 msgr2=0x7fa24006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 0x7fa24006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa258034000 tx=0x7fa25802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 msgr2=0x7fa260073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c410 tx=0x7fa24800c8d0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 shutdown_connections 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa24006c8f0 0x7fa24006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa260140350 0x7fa26013ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa26013ff70 0x7fa260073a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 --2- 172.21.15.40:0/4149475121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2600cff20 0x7fa2600ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 >> 172.21.15.40:0/4149475121 conn(0x7fa2600d0d30 msgr2=0x7fa2600b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 shutdown_connections 2024-04-26T21:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.770+0000 7fa2654bd640 1 -- 172.21.15.40:0/4149475121 wait complete. 2024-04-26T21:12:29.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:29.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.886+0000 7fc8c5607640 1 Processor -- start 2024-04-26T21:12:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.886+0000 7fc8c5607640 1 -- start start 2024-04-26T21:12:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8c5607640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8c00b4eb0 0x7fc8c00b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8c5607640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc8c00ceff0 0x7fc8c00cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8c5607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c015dca0 con 0x7fc8c0143840 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8c5607640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c0143c20 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8c5607640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c015df80 con 0x7fc8c00ceff0 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8be7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8c00b4eb0 0x7fc8c00b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8beffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8c0143840 0x7fc8c00b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59084/0 (socket says 172.21.15.40:59084) 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8beffd640 1 -- 172.21.15.40:0/1194614317 learned_addr learned my addr 172.21.15.40:0/1194614317 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8bf7fe640 1 --2- 172.21.15.40:0/1194614317 >> v2:172.21.15.184:3300/0 conn(0x7fc8c00ceff0 0x7fc8c00cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8be7fc640 1 -- 172.21.15.40:0/1194614317 >> v2:172.21.15.184:3300/0 conn(0x7fc8c00ceff0 msgr2=0x7fc8c00cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8be7fc640 1 --2- 172.21.15.40:0/1194614317 >> v2:172.21.15.184:3300/0 conn(0x7fc8c00ceff0 0x7fc8c00cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8be7fc640 1 -- 172.21.15.40:0/1194614317 >> v1:172.21.15.40:6789/0 conn(0x7fc8c0143840 legacy=0x7fc8c00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.887+0000 7fc8be7fc640 1 -- 172.21.15.40:0/1194614317 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c015e260 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.888+0000 7fc8be7fc640 1 --2- 172.21.15.40:0/1194614317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c00b4eb0 0x7fc8c00b5330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b70e0 tx=0x7fc8b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b95e0e42b684d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.888+0000 7fc8bdffb640 1 -- 172.21.15.40:0/1194614317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4037070 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.888+0000 7fc8bdffb640 1 -- 172.21.15.40:0/1194614317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b4032440 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.888+0000 7fc8bdffb640 1 -- 172.21.15.40:0/1194614317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4032810 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 -- 172.21.15.40:0/1194614317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c00b4eb0 msgr2=0x7fc8c00b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 --2- 172.21.15.40:0/1194614317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c00b4eb0 0x7fc8c00b5330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b70e0 tx=0x7fc8b402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 -- 172.21.15.40:0/1194614317 shutdown_connections 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 --2- 172.21.15.40:0/1194614317 >> v2:172.21.15.184:3300/0 conn(0x7fc8c00ceff0 0x7fc8c00cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 --2- 172.21.15.40:0/1194614317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c00b4eb0 0x7fc8c00b5330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 -- 172.21.15.40:0/1194614317 >> 172.21.15.40:0/1194614317 conn(0x7fc8c00cd8a0 msgr2=0x7fc8c00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 -- 172.21.15.40:0/1194614317 shutdown_connections 2024-04-26T21:12:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.889+0000 7fc8c5607640 1 -- 172.21.15.40:0/1194614317 wait complete. 2024-04-26T21:12:29.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.890+0000 7fc8c5607640 1 Processor -- start 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.890+0000 7fc8c5607640 1 -- start start 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 0x7fc8c013fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c00ceff0 0x7fc8c01456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c015d8c0 con 0x7fc8c0143840 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c00b46f0 con 0x7fc8c00ceff0 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8c5607640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c015da40 con 0x7fc8c00b4eb0 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38034/0 (socket says 172.21.15.40:38034) 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 -- 172.21.15.40:0/3353305868 learned_addr learned my addr 172.21.15.40:0/3353305868 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8beffd640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 0x7fc8c013fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8be7fc640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c00ceff0 0x7fc8c01456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 -- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 msgr2=0x7fc8c013fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 0x7fc8c013fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 -- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c00ceff0 msgr2=0x7fc8c01456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c00ceff0 0x7fc8c01456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.891+0000 7fc8bf7fe640 1 -- 172.21.15.40:0/3353305868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c0144600 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8beffd640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 0x7fc8c013fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8bf7fe640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007e40 tx=0x7fc8b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0014620 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0019070 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0014a20 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01448e0 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.892+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c0144ea0 con 0x7fc8c0143840 2024-04-26T21:12:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.893+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c006a290 con 0x7fc8c0143840 2024-04-26T21:12:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.893+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b0014bc0 con 0x7fc8c0143840 2024-04-26T21:12:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.894+0000 7fc8a77fe640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.894+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc8b008c4e0 con 0x7fc8c0143840 2024-04-26T21:12:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.894+0000 7fc8beffd640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.894+0000 7fc8beffd640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8000f60 tx=0x7fc8b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:29.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:29.898+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8b005a870 con 0x7fc8c0143840 2024-04-26T21:12:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.087+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8c00ce420 con 0x7fc8c0143840 2024-04-26T21:12:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.087+0000 7fc8a77fe640 1 -- 172.21.15.40:0/3353305868 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fc8b005a170 con 0x7fc8c0143840 2024-04-26T21:12:30.088 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 msgr2=0x7fc8ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8000f60 tx=0x7fc8b8009040 comp rx=0 tx=0).stop 2024-04-26T21:12:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 msgr2=0x7fc8c0143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0007e40 tx=0x7fc8b000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 shutdown_connections 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c0143840 0x7fc8c0143dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c00ceff0 0x7fc8c01456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 --2- 172.21.15.40:0/3353305868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8c00b4eb0 0x7fc8c013fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 >> 172.21.15.40:0/3353305868 conn(0x7fc8c00cd8a0 msgr2=0x7fc8c00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 shutdown_connections 2024-04-26T21:12:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.089+0000 7fc8c5607640 1 -- 172.21.15.40:0/3353305868 wait complete. 2024-04-26T21:12:30.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:30.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.203+0000 7febce5bf640 1 Processor -- start 2024-04-26T21:12:30.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.203+0000 7febce5bf640 1 -- start start 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febce5bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7febc815d340 0x7febc809c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febce5bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7febc80b2420 0x7febc80d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febce5bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc80d1b80 con 0x7febc80d1770 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febce5bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7febc80ce9d0 con 0x7febc815d340 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febce5bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7febc80cee60 con 0x7febc80b2420 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febc7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7febc815d340 0x7febc809c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febc7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7febc815d340 0x7febc809c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38046/0 (socket says 172.21.15.40:38046) 2024-04-26T21:12:30.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.204+0000 7febc7fff640 1 -- 172.21.15.40:0/3484165183 learned_addr learned my addr 172.21.15.40:0/3484165183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc77fe640 1 --2- 172.21.15.40:0/3484165183 >> v2:172.21.15.184:3300/0 conn(0x7febc80b2420 0x7febc80d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc7fff640 1 -- 172.21.15.40:0/3484165183 >> v2:172.21.15.184:3300/0 conn(0x7febc80b2420 msgr2=0x7febc80d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc7fff640 1 --2- 172.21.15.40:0/3484165183 >> v2:172.21.15.184:3300/0 conn(0x7febc80b2420 0x7febc80d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc7fff640 1 -- 172.21.15.40:0/3484165183 >> v1:172.21.15.40:6789/0 conn(0x7febc80d1770 legacy=0x7febc80cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc7fff640 1 -- 172.21.15.40:0/3484165183 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc80d0ff0 con 0x7febc815d340 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc6ffd640 1 -- 172.21.15.40:0/3484165183 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258345368 0 0) 0x7febc0001490 con 0x7febc80d1770 2024-04-26T21:12:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.205+0000 7febc7fff640 1 --2- 172.21.15.40:0/3484165183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc809c560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febc80d0aa0 tx=0x7febb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca6e3bfcecbd3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febc6ffd640 1 -- 172.21.15.40:0/3484165183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8037070 con 0x7febc815d340 2024-04-26T21:12:30.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febc6ffd640 1 -- 172.21.15.40:0/3484165183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb8032440 con 0x7febc815d340 2024-04-26T21:12:30.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febc6ffd640 1 -- 172.21.15.40:0/3484165183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8032810 con 0x7febc815d340 2024-04-26T21:12:30.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 -- 172.21.15.40:0/3484165183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 msgr2=0x7febc809c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 --2- 172.21.15.40:0/3484165183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc809c560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febc80d0aa0 tx=0x7febb802a270 comp rx=0 tx=0).stop 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 -- 172.21.15.40:0/3484165183 shutdown_connections 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 --2- 172.21.15.40:0/3484165183 >> v2:172.21.15.184:3300/0 conn(0x7febc80b2420 0x7febc80d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 --2- 172.21.15.40:0/3484165183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc809c560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 -- 172.21.15.40:0/3484165183 >> 172.21.15.40:0/3484165183 conn(0x7febc809ba40 msgr2=0x7febc809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 -- 172.21.15.40:0/3484165183 shutdown_connections 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.206+0000 7febce5bf640 1 -- 172.21.15.40:0/3484165183 wait complete. 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.207+0000 7febce5bf640 1 Processor -- start 2024-04-26T21:12:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.207+0000 7febce5bf640 1 -- start start 2024-04-26T21:12:30.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 0x7febc809dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 0x7febc81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7febc80712b0 con 0x7febc815d340 2024-04-26T21:12:30.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febc80ce660 con 0x7febc809ce30 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febce5bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7febc815d720 con 0x7febc80d1770 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febc77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febc77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38062/0 (socket says 172.21.15.40:38062) 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.208+0000 7febc77fe640 1 -- 172.21.15.40:0/842231599 learned_addr learned my addr 172.21.15.40:0/842231599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc7fff640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 0x7febc809dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 -- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 msgr2=0x7febc809dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febccb5c640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 0x7febc81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 0x7febc809dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 -- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 msgr2=0x7febc81552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 0x7febc81552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 -- 172.21.15.40:0/842231599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc809e5f0 con 0x7febc815d340 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc7fff640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 0x7febc809dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febccb5c640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 0x7febc81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc77fe640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d830 tx=0x7febbc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00b6c0 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc019070 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81566c0 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00ba80 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.209+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc8156960 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.210+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc80693d0 con 0x7febc815d340 2024-04-26T21:12:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.211+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febbc00bc20 con 0x7febc815d340 2024-04-26T21:12:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.211+0000 7febc4ff9640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.211+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7febbc08bf60 con 0x7febc815d340 2024-04-26T21:12:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.211+0000 7febc7fff640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.212+0000 7febc7fff640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 0x7febb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febb8034000 tx=0x7febb802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.215+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febbc01c360 con 0x7febc815d340 2024-04-26T21:12:30.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.398+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7febc80b0f60 con 0x7febc815d340 2024-04-26T21:12:30.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.398+0000 7febc4ff9640 1 -- 172.21.15.40:0/842231599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7febbc05a420 con 0x7febc815d340 2024-04-26T21:12:30.399 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 msgr2=0x7febb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 0x7febb406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febb8034000 tx=0x7febb802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 msgr2=0x7febc81f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d830 tx=0x7febbc00dcf0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 shutdown_connections 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc809ce30 0x7febc81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febc815d340 0x7febc81f3660 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 --2- 172.21.15.40:0/842231599 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febc80d1770 0x7febc809dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 >> 172.21.15.40:0/842231599 conn(0x7febc809ba40 msgr2=0x7febc80ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 shutdown_connections 2024-04-26T21:12:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.400+0000 7febce5bf640 1 -- 172.21.15.40:0/842231599 wait complete. 2024-04-26T21:12:30.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:30.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.515+0000 7f98da9b4640 1 Processor -- start 2024-04-26T21:12:30.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.515+0000 7f98da9b4640 1 -- start start 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98da9b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98d415b930 0x7f98d415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98da9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d4073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98da9b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d415c530 con 0x7f98d4074030 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98da9b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f98d415c810 con 0x7f98d415b930 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98da9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98d415caf0 con 0x7f98d415f4f0 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98d415b930 0x7f98d415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98d415b930 0x7f98d415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38064/0 (socket says 172.21.15.40:38064) 2024-04-26T21:12:30.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 -- 172.21.15.40:0/903682963 learned_addr learned my addr 172.21.15.40:0/903682963 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d37fe640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d4073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 -- 172.21.15.40:0/903682963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 msgr2=0x7f98d4073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d4073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 -- 172.21.15.40:0/903682963 >> v1:172.21.15.184:6789/0 conn(0x7f98d4074030 legacy=0x7f98d40720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d3fff640 1 -- 172.21.15.40:0/903682963 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d415cdd0 con 0x7f98d415b930 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.516+0000 7f98d37fe640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d4073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.517+0000 7f98d3fff640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d415bd10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009870 tx=0x7f98c402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ee1d5797da071b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.517+0000 7f98d2ffd640 1 -- 172.21.15.40:0/903682963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4037070 con 0x7f98d415b930 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.517+0000 7f98d2ffd640 1 -- 172.21.15.40:0/903682963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c4032440 con 0x7f98d415b930 2024-04-26T21:12:30.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.517+0000 7f98d2ffd640 1 -- 172.21.15.40:0/903682963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4032810 con 0x7f98d415b930 2024-04-26T21:12:30.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 -- 172.21.15.40:0/903682963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 msgr2=0x7f98d415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d415bd10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009870 tx=0x7f98c402a200 comp rx=0 tx=0).stop 2024-04-26T21:12:30.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 -- 172.21.15.40:0/903682963 shutdown_connections 2024-04-26T21:12:30.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d4073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 --2- 172.21.15.40:0/903682963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d415bd10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 -- 172.21.15.40:0/903682963 >> 172.21.15.40:0/903682963 conn(0x7f98d40cd8a0 msgr2=0x7f98d40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 -- 172.21.15.40:0/903682963 shutdown_connections 2024-04-26T21:12:30.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.518+0000 7f98da9b4640 1 -- 172.21.15.40:0/903682963 wait complete. 2024-04-26T21:12:30.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.519+0000 7f98da9b4640 1 Processor -- start 2024-04-26T21:12:30.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.519+0000 7f98da9b4640 1 -- start start 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d4074030 0x7f98d414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98d406fdd0 con 0x7f98d415b930 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98d415c250 con 0x7f98d4074030 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98da9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98d40d0270 con 0x7f98d415f4f0 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38076/0 (socket says 172.21.15.40:38076) 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 -- 172.21.15.40:0/1806683969 learned_addr learned my addr 172.21.15.40:0/1806683969 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d3fff640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d4074030 0x7f98d414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 -- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 msgr2=0x7f98d414f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d8f51640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d414f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 -- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d4074030 msgr2=0x7f98d414ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d4074030 0x7f98d414ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.520+0000 7f98d37fe640 1 -- 172.21.15.40:0/1806683969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d414f600 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98d37fe640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00eb40 tx=0x7f98bc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc007c10 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98bc007db0 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d415ad80 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc010680 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.521+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d414cc90 con 0x7f98d415b930 2024-04-26T21:12:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.522+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d406a290 con 0x7f98d415b930 2024-04-26T21:12:30.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.522+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98bc00b030 con 0x7f98d415b930 2024-04-26T21:12:30.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.523+0000 7f98d0ff9640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 0x7f98b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.523+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f98bc08df20 con 0x7f98d415b930 2024-04-26T21:12:30.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.523+0000 7f98d3fff640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 0x7f98b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.524+0000 7f98d3fff640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 0x7f98b406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98c4034000 tx=0x7f98c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.527+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98bc05c350 con 0x7f98d415b930 2024-04-26T21:12:30.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.711+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98d40cdeb0 con 0x7f98d415b930 2024-04-26T21:12:30.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.712+0000 7f98d0ff9640 1 -- 172.21.15.40:0/1806683969 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f98bc05bc50 con 0x7f98d415b930 2024-04-26T21:12:30.713 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.713+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 msgr2=0x7f98b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.713+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 0x7f98b406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98c4034000 tx=0x7f98c402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.713+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 msgr2=0x7f98d414efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.713+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00eb40 tx=0x7f98bc00cf70 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 shutdown_connections 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98b406c8f0 0x7f98b406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d415f4f0 0x7f98d414f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d415b930 0x7f98d414efa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 --2- 172.21.15.40:0/1806683969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d4074030 0x7f98d414ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 >> 172.21.15.40:0/1806683969 conn(0x7f98d40cd8a0 msgr2=0x7f98d40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 shutdown_connections 2024-04-26T21:12:30.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.714+0000 7f98da9b4640 1 -- 172.21.15.40:0/1806683969 wait complete. 2024-04-26T21:12:30.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:30.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.825+0000 7f10ec3e6640 1 Processor -- start 2024-04-26T21:12:30.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.825+0000 7f10ec3e6640 1 -- start start 2024-04-26T21:12:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f10e415f340 0x7f10e40b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10e40d4f00 0x7f10e40d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40b2510 con 0x7f10e40d4f00 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40b27f0 con 0x7f10e415f340 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ec3e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40b2ad0 con 0x7f10e415f720 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10e40d4f00 0x7f10e40d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10e40d4f00 0x7f10e40d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47380/0 (socket says 172.21.15.40:47380) 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 -- 172.21.15.40:0/1038289101 learned_addr learned my addr 172.21.15.40:0/1038289101 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea182640 1 --2- 172.21.15.40:0/1038289101 >> v2:172.21.15.184:3300/0 conn(0x7f10e415f340 0x7f10e40b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10e9981640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 -- 172.21.15.40:0/1038289101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 msgr2=0x7f10e40d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 -- 172.21.15.40:0/1038289101 >> v2:172.21.15.184:3300/0 conn(0x7f10e415f340 msgr2=0x7f10e40b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 --2- 172.21.15.40:0/1038289101 >> v2:172.21.15.184:3300/0 conn(0x7f10e415f340 0x7f10e40b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.826+0000 7f10ea983640 1 -- 172.21.15.40:0/1038289101 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40b2db0 con 0x7f10e40d4f00 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.827+0000 7f10e9981640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.827+0000 7f10ea983640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e40d4f00 0x7f10e40d5350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10e40d6000 tx=0x7f10dc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a617e003ba8b4312 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.827+0000 7f10e9180640 1 -- 172.21.15.40:0/1038289101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc037070 con 0x7f10e40d4f00 2024-04-26T21:12:30.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.827+0000 7f10e9180640 1 -- 172.21.15.40:0/1038289101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc032440 con 0x7f10e40d4f00 2024-04-26T21:12:30.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.827+0000 7f10e9180640 1 -- 172.21.15.40:0/1038289101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc032810 con 0x7f10e40d4f00 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 -- 172.21.15.40:0/1038289101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e40d4f00 msgr2=0x7f10e40d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e40d4f00 0x7f10e40d5350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10e40d6000 tx=0x7f10dc02a140 comp rx=0 tx=0).stop 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 -- 172.21.15.40:0/1038289101 shutdown_connections 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e40d4f00 0x7f10e40d5350 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/1038289101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/1038289101 >> v2:172.21.15.184:3300/0 conn(0x7f10e415f340 0x7f10e40b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 -- 172.21.15.40:0/1038289101 >> 172.21.15.40:0/1038289101 conn(0x7f10e408ec70 msgr2=0x7f10e409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 -- 172.21.15.40:0/1038289101 shutdown_connections 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.828+0000 7f10ec3e6640 1 -- 172.21.15.40:0/1038289101 wait complete. 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.829+0000 7f10ec3e6640 1 Processor -- start 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.829+0000 7f10ec3e6640 1 -- start start 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40d4f00 0x7f10e4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40b5040 con 0x7f10e415f340 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40b4ec0 con 0x7f10e40d4f00 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ec3e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40b51c0 con 0x7f10e415f720 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47388/0 (socket says 172.21.15.40:47388) 2024-04-26T21:12:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 -- 172.21.15.40:0/341588346 learned_addr learned my addr 172.21.15.40:0/341588346 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 -- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 msgr2=0x7f10e40ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 -- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40d4f00 msgr2=0x7f10e4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10ea182640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40d4f00 0x7f10e4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40d4f00 0x7f10e4158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.830+0000 7f10e9981640 1 -- 172.21.15.40:0/341588346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40ce8e0 con 0x7f10e415f340 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10e9981640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d7d0 tx=0x7f10d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8004030 con 0x7f10e415f340 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d80041d0 con 0x7f10e415f340 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e415fc90 con 0x7f10e415f340 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8009640 con 0x7f10e415f340 2024-04-26T21:12:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.831+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e4160250 con 0x7f10e415f340 2024-04-26T21:12:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.832+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e406a290 con 0x7f10e415f340 2024-04-26T21:12:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.832+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10d80097e0 con 0x7f10e415f340 2024-04-26T21:12:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.833+0000 7f10e2ffd640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 0x7f10d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.833+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f10d808bd20 con 0x7f10e415f340 2024-04-26T21:12:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.833+0000 7f10ea182640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 0x7f10d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.834+0000 7f10ea182640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 0x7f10d006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fd0 tx=0x7f10d400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:30.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:30.837+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10d805a150 con 0x7f10e415f340 2024-04-26T21:12:30.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:12:30.944 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:12:31.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.045+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10e40d3020 con 0x7f10e415f340 2024-04-26T21:12:31.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.046+0000 7f10e2ffd640 1 -- 172.21.15.40:0/341588346 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f10d8059a50 con 0x7f10e415f340 2024-04-26T21:12:31.047 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 msgr2=0x7f10d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 0x7f10d006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fd0 tx=0x7f10d400b040 comp rx=0 tx=0).stop 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 msgr2=0x7f10e40cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d7d0 tx=0x7f10d800dc90 comp rx=0 tx=0).stop 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 shutdown_connections 2024-04-26T21:12:31.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10d006c920 0x7f10d006ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e415f720 0x7f10e40ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e415f340 0x7f10e40cdd90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 --2- 172.21.15.40:0/341588346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40d4f00 0x7f10e4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 >> 172.21.15.40:0/341588346 conn(0x7f10e408ec70 msgr2=0x7f10e40d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 shutdown_connections 2024-04-26T21:12:31.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.048+0000 7f10ec3e6640 1 -- 172.21.15.40:0/341588346 wait complete. 2024-04-26T21:12:31.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:31.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:31.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 Processor -- start 2024-04-26T21:12:31.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 -- start start 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3040b2000 0x7fb30406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040d1170 0x7fb3040b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb30409c790 con 0x7fb3041631f0 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb3040d1600 con 0x7fb3040d1170 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.166+0000 7fb30b17f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb304163ec0 con 0x7fb3040b2000 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040d1170 0x7fb3040b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040d1170 0x7fb3040b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47392/0 (socket says 172.21.15.40:47392) 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb308f1b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3041631f0 0x7fb3041635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52306/0 (socket says 172.21.15.40:52306) 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 -- 172.21.15.40:0/3560650605 learned_addr learned my addr 172.21.15.40:0/3560650605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb2fbfff640 1 --2- 172.21.15.40:0/3560650605 >> v2:172.21.15.184:3300/0 conn(0x7fb3040b2000 0x7fb30406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 -- 172.21.15.40:0/3560650605 >> v2:172.21.15.184:3300/0 conn(0x7fb3040b2000 msgr2=0x7fb30406ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 --2- 172.21.15.40:0/3560650605 >> v2:172.21.15.184:3300/0 conn(0x7fb3040b2000 0x7fb30406ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 -- 172.21.15.40:0/3560650605 >> v1:172.21.15.40:6789/0 conn(0x7fb3041631f0 legacy=0x7fb3041635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb30971c640 1 -- 172.21.15.40:0/3560650605 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3040d1a40 con 0x7fb3040d1170 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.167+0000 7fb2fb7fe640 1 -- 172.21.15.40:0/3560650605 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942303158 0 0) 0x7fb30409c790 con 0x7fb3041631f0 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30971c640 1 --2- 172.21.15.40:0/3560650605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040d1170 0x7fb3040b3300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3040734b0 tx=0x7fb2fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6be92d3383b2238 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb2fb7fe640 1 -- 172.21.15.40:0/3560650605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc036070 con 0x7fb3040d1170 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb2fb7fe640 1 -- 172.21.15.40:0/3560650605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc031600 con 0x7fb3040d1170 2024-04-26T21:12:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb2fb7fe640 1 -- 172.21.15.40:0/3560650605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0319d0 con 0x7fb3040d1170 2024-04-26T21:12:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 -- 172.21.15.40:0/3560650605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040d1170 msgr2=0x7fb3040b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 --2- 172.21.15.40:0/3560650605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040d1170 0x7fb3040b3300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3040734b0 tx=0x7fb2fc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 -- 172.21.15.40:0/3560650605 shutdown_connections 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 --2- 172.21.15.40:0/3560650605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040d1170 0x7fb3040b3300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 --2- 172.21.15.40:0/3560650605 >> v2:172.21.15.184:3300/0 conn(0x7fb3040b2000 0x7fb30406ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 -- 172.21.15.40:0/3560650605 >> 172.21.15.40:0/3560650605 conn(0x7fb30409ba40 msgr2=0x7fb30409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 -- 172.21.15.40:0/3560650605 shutdown_connections 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.168+0000 7fb30b17f640 1 -- 172.21.15.40:0/3560650605 wait complete. 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.169+0000 7fb30b17f640 1 Processor -- start 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 -- start start 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 0x7fb3040ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 0x7fb30415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040d0230 con 0x7fb3040b2000 2024-04-26T21:12:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040d2000 con 0x7fb3040d1170 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.170+0000 7fb30b17f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb30414f250 con 0x7fb3041631f0 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47404/0 (socket says 172.21.15.40:47404) 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 -- 172.21.15.40:0/651345639 learned_addr learned my addr 172.21.15.40:0/651345639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb30971c640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 0x7fb30415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb2fbfff640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 0x7fb3040ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 -- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 msgr2=0x7fb30415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 0x7fb30415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 -- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 msgr2=0x7fb3040ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 0x7fb3040ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 -- 172.21.15.40:0/651345639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30414f4a0 con 0x7fb3040b2000 2024-04-26T21:12:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb30971c640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 0x7fb30415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb2fbfff640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 0x7fb3040ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb308f1b640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c430 tx=0x7fb30000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000074e0 con 0x7fb3040b2000 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3000047e0 con 0x7fb3040b2000 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30414f780 con 0x7fb3040b2000 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004bd0 con 0x7fb3040b2000 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.171+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30414fd40 con 0x7fb3040b2000 2024-04-26T21:12:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.173+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb304150120 con 0x7fb3040b2000 2024-04-26T21:12:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.173+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb300011070 con 0x7fb3040b2000 2024-04-26T21:12:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.173+0000 7fb2f97fa640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 0x7fb2f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.173+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb30008b830 con 0x7fb3040b2000 2024-04-26T21:12:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.174+0000 7fb2fbfff640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 0x7fb2f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.174+0000 7fb2fbfff640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 0x7fb2f006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3040ce910 tx=0x7fb2f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.178+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb304150120 con 0x7fb3040b2000 2024-04-26T21:12:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.363+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb30406a290 con 0x7fb3040b2000 2024-04-26T21:12:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.363+0000 7fb2f97fa640 1 -- 172.21.15.40:0/651345639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fb300059c60 con 0x7fb3040b2000 2024-04-26T21:12:31.364 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:31.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 msgr2=0x7fb2f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 0x7fb2f006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3040ce910 tx=0x7fb2f4008040 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 msgr2=0x7fb30414caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c430 tx=0x7fb30000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 shutdown_connections 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2f006c8f0 0x7fb2f006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3041631f0 0x7fb30415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3040d1170 0x7fb3040ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 --2- 172.21.15.40:0/651345639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040b2000 0x7fb30414caa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 >> 172.21.15.40:0/651345639 conn(0x7fb30409ba40 msgr2=0x7fb3040718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 shutdown_connections 2024-04-26T21:12:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.365+0000 7fb30b17f640 1 -- 172.21.15.40:0/651345639 wait complete. 2024-04-26T21:12:31.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:31.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.481+0000 7fcf2d4bd640 1 Processor -- start 2024-04-26T21:12:31.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.481+0000 7fcf2d4bd640 1 -- start start 2024-04-26T21:12:31.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf2d4bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf280d0b80 0x7fcf280d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf2d4bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf280cdce0 0x7fcf280ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf2d4bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2809bae0 con 0x7fcf280d1570 2024-04-26T21:12:31.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf2d4bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2809bdc0 con 0x7fcf280d0b80 2024-04-26T21:12:31.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf2d4bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2809c0a0 con 0x7fcf280cdce0 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf280d0b80 0x7fcf280d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf280d0b80 0x7fcf280d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47420/0 (socket says 172.21.15.40:47420) 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 -- 172.21.15.40:0/3496342089 learned_addr learned my addr 172.21.15.40:0/3496342089 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf277fe640 1 --2- 172.21.15.40:0/3496342089 >> v2:172.21.15.184:3300/0 conn(0x7fcf280cdce0 0x7fcf280ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 -- 172.21.15.40:0/3496342089 >> v2:172.21.15.184:3300/0 conn(0x7fcf280cdce0 msgr2=0x7fcf280ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 --2- 172.21.15.40:0/3496342089 >> v2:172.21.15.184:3300/0 conn(0x7fcf280cdce0 0x7fcf280ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 -- 172.21.15.40:0/3496342089 >> v1:172.21.15.184:6789/0 conn(0x7fcf280d1570 legacy=0x7fcf280d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.483+0000 7fcf267fc640 1 -- 172.21.15.40:0/3496342089 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2809c380 con 0x7fcf280d0b80 2024-04-26T21:12:31.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.484+0000 7fcf267fc640 1 --2- 172.21.15.40:0/3496342089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf280d0ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcf280735f0 tx=0x7fcf1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e7699189eda73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.484+0000 7fcf25ffb640 1 -- 172.21.15.40:0/3496342089 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18037070 con 0x7fcf280d0b80 2024-04-26T21:12:31.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.484+0000 7fcf25ffb640 1 -- 172.21.15.40:0/3496342089 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18032440 con 0x7fcf280d0b80 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.484+0000 7fcf25ffb640 1 -- 172.21.15.40:0/3496342089 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18032810 con 0x7fcf280d0b80 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/3496342089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 msgr2=0x7fcf280d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/3496342089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf280d0ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcf280735f0 tx=0x7fcf1802a270 comp rx=0 tx=0).stop 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/3496342089 shutdown_connections 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/3496342089 >> v2:172.21.15.184:3300/0 conn(0x7fcf280cdce0 0x7fcf280ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/3496342089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf280d0ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/3496342089 >> 172.21.15.40:0/3496342089 conn(0x7fcf280b4aa0 msgr2=0x7fcf280b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/3496342089 shutdown_connections 2024-04-26T21:12:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.485+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/3496342089 wait complete. 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.486+0000 7fcf2d4bd640 1 Processor -- start 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.486+0000 7fcf2d4bd640 1 -- start start 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 0x7fcf281f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf280d1570 0x7fcf28071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2815e6f0 con 0x7fcf280d0b80 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2815d680 con 0x7fcf280cdce0 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf2d4bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf280ce6d0 con 0x7fcf280d1570 2024-04-26T21:12:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47426/0 (socket says 172.21.15.40:47426) 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 -- 172.21.15.40:0/1327269379 learned_addr learned my addr 172.21.15.40:0/1327269379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf277fe640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf280d1570 0x7fcf28071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf26ffd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 0x7fcf281f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 -- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf280d1570 msgr2=0x7fcf28071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf280d1570 0x7fcf28071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 -- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 msgr2=0x7fcf281f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 0x7fcf281f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.487+0000 7fcf267fc640 1 -- 172.21.15.40:0/1327269379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf281f5e30 con 0x7fcf280d0b80 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf26ffd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 0x7fcf281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf267fc640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c00 tx=0x7fcf180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18041070 con 0x7fcf280d0b80 2024-04-26T21:12:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1803c870 con 0x7fcf280d0b80 2024-04-26T21:12:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18037040 con 0x7fcf280d0b80 2024-04-26T21:12:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf281f6110 con 0x7fcf280d0b80 2024-04-26T21:12:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.488+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf28070c50 con 0x7fcf280d0b80 2024-04-26T21:12:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.489+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf280693d0 con 0x7fcf280d0b80 2024-04-26T21:12:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.490+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf18044070 con 0x7fcf280d0b80 2024-04-26T21:12:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.490+0000 7fcf0f7fe640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 0x7fcf1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.490+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf180ac440 con 0x7fcf280d0b80 2024-04-26T21:12:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.490+0000 7fcf26ffd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 0x7fcf1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.491+0000 7fcf26ffd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 0x7fcf1406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c001660 tx=0x7fcf1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.494+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf1807a2b0 con 0x7fcf280d0b80 2024-04-26T21:12:31.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.680+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf2806d660 con 0x7fcf280d0b80 2024-04-26T21:12:31.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.680+0000 7fcf0f7fe640 1 -- 172.21.15.40:0/1327269379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fcf1807e5b0 con 0x7fcf280d0b80 2024-04-26T21:12:31.682 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 msgr2=0x7fcf1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 0x7fcf1406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c001660 tx=0x7fcf1c009040 comp rx=0 tx=0).stop 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 msgr2=0x7fcf2806d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c00 tx=0x7fcf180046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 shutdown_connections 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf1406c8f0 0x7fcf1406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf280d1570 0x7fcf28071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf280d0b80 0x7fcf2806d880 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 --2- 172.21.15.40:0/1327269379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf280cdce0 0x7fcf281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 >> 172.21.15.40:0/1327269379 conn(0x7fcf280b4aa0 msgr2=0x7fcf28158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 shutdown_connections 2024-04-26T21:12:31.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.683+0000 7fcf2d4bd640 1 -- 172.21.15.40:0/1327269379 wait complete. 2024-04-26T21:12:31.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.798+0000 7fb3eccd1640 1 Processor -- start 2024-04-26T21:12:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.798+0000 7fb3eccd1640 1 -- start start 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3eccd1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3e8147fb0 0x7fb3e806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3eccd1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3e81638b0 0x7fb3e806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3eccd1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e8148390 con 0x7fb3e81485f0 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3eccd1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e8163e20 con 0x7fb3e8147fb0 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3eccd1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e80b59f0 con 0x7fb3e81638b0 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3e8147fb0 0x7fb3e806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3e8147fb0 0x7fb3e806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47440/0 (socket says 172.21.15.40:47440) 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 -- 172.21.15.40:0/4044456462 learned_addr learned my addr 172.21.15.40:0/4044456462 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e5d9b640 1 --2- 172.21.15.40:0/4044456462 >> v2:172.21.15.184:3300/0 conn(0x7fb3e81638b0 0x7fb3e806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 -- 172.21.15.40:0/4044456462 >> v2:172.21.15.184:3300/0 conn(0x7fb3e81638b0 msgr2=0x7fb3e806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4044456462 >> v2:172.21.15.184:3300/0 conn(0x7fb3e81638b0 0x7fb3e806c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 -- 172.21.15.40:0/4044456462 >> v1:172.21.15.184:6789/0 conn(0x7fb3e81485f0 legacy=0x7fb3e8073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.799+0000 7fb3e659c640 1 -- 172.21.15.40:0/4044456462 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e806fe90 con 0x7fb3e8147fb0 2024-04-26T21:12:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.800+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4044456462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e8147fb0 0x7fb3e806c2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8074570 tx=0x7fb3d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eba8916363f554 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.800+0000 7fb3e559a640 1 -- 172.21.15.40:0/4044456462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8037070 con 0x7fb3e8147fb0 2024-04-26T21:12:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.800+0000 7fb3e559a640 1 -- 172.21.15.40:0/4044456462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8032440 con 0x7fb3e8147fb0 2024-04-26T21:12:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.800+0000 7fb3e559a640 1 -- 172.21.15.40:0/4044456462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8032810 con 0x7fb3e8147fb0 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4044456462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e8147fb0 msgr2=0x7fb3e806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4044456462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e8147fb0 0x7fb3e806c2a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8074570 tx=0x7fb3d802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4044456462 shutdown_connections 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4044456462 >> v2:172.21.15.184:3300/0 conn(0x7fb3e81638b0 0x7fb3e806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4044456462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e8147fb0 0x7fb3e806c2a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4044456462 >> 172.21.15.40:0/4044456462 conn(0x7fb3e80d1e90 msgr2=0x7fb3e80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4044456462 shutdown_connections 2024-04-26T21:12:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.801+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4044456462 wait complete. 2024-04-26T21:12:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.802+0000 7fb3eccd1640 1 Processor -- start 2024-04-26T21:12:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.802+0000 7fb3eccd1640 1 -- start start 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 0x7fb3e8160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e81485f0 0x7fb3e815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8148cd0 con 0x7fb3e81638b0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8070450 con 0x7fb3e8147fb0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3eccd1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e8148e50 con 0x7fb3e81485f0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47454/0 (socket says 172.21.15.40:47454) 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 -- 172.21.15.40:0/4259767903 learned_addr learned my addr 172.21.15.40:0/4259767903 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 0x7fb3e8160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e5d9b640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e81485f0 0x7fb3e815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 -- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e81485f0 msgr2=0x7fb3e815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e81485f0 0x7fb3e815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 -- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 msgr2=0x7fb3e8160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 0x7fb3e8160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e6d9d640 1 -- 172.21.15.40:0/4259767903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e8160f50 con 0x7fb3e81638b0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.803+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 0x7fb3e8160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3e6d9d640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c7a0 tx=0x7fb3e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0003b70 con 0x7fb3e81638b0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0003d30 con 0x7fb3e81638b0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e8161230 con 0x7fb3e81638b0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000aa90 con 0x7fb3e81638b0 2024-04-26T21:12:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.804+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e81617f0 con 0x7fb3e81638b0 2024-04-26T21:12:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.805+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e80693d0 con 0x7fb3e81638b0 2024-04-26T21:12:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.805+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3e000ac30 con 0x7fb3e81638b0 2024-04-26T21:12:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.806+0000 7fb3d6ffd640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.806+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb3e008ba40 con 0x7fb3e81638b0 2024-04-26T21:12:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.806+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:31.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.807+0000 7fb3e659c640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8034000 tx=0x7fb3d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.810+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3e0059e80 con 0x7fb3e81638b0 2024-04-26T21:12:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.995+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3e80b52e0 con 0x7fb3e81638b0 2024-04-26T21:12:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.995+0000 7fb3d6ffd640 1 -- 172.21.15.40:0/4259767903 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fb3e0059780 con 0x7fb3e81638b0 2024-04-26T21:12:31.997 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 msgr2=0x7fb3c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8034000 tx=0x7fb3d802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 msgr2=0x7fb3e80aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000c7a0 tx=0x7fb3e000cc60 comp rx=0 tx=0).stop 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 shutdown_connections 2024-04-26T21:12:31.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e81638b0 0x7fb3e80aea00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3e81485f0 0x7fb3e815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 --2- 172.21.15.40:0/4259767903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3e8147fb0 0x7fb3e8160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 >> 172.21.15.40:0/4259767903 conn(0x7fb3e80d1e90 msgr2=0x7fb3e8142350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 shutdown_connections 2024-04-26T21:12:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:31.997+0000 7fb3eccd1640 1 -- 172.21.15.40:0/4259767903 wait complete. 2024-04-26T21:12:32.021 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2024-04-26T21:12:32.021 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:12:32.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.122+0000 7f4643462640 1 Processor -- start 2024-04-26T21:12:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.122+0000 7f4643462640 1 -- start start 2024-04-26T21:12:32.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4643462640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f463c145980 0x7f463c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4643462640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c0b4260 con 0x7f463c0b3de0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4643462640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c0b4530 con 0x7f463c0b61a0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4643462640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f463c0b5140 con 0x7f463c145980 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f46409fd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f463c0b3de0 0x7f463c0b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52310/0 (socket says 172.21.15.40:52310) 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f46409fd640 1 -- 172.21.15.40:0/982745561 learned_addr learned my addr 172.21.15.40:0/982745561 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f46411fe640 1 --2- 172.21.15.40:0/982745561 >> v2:172.21.15.184:3300/0 conn(0x7f463c145980 0x7f463c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565094141 0 0) 0x7f463c0b4260 con 0x7f463c0b3de0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4628005730 con 0x7f463c0b3de0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1807903239 0 0) 0x7f4628005730 con 0x7f463c0b3de0 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 >> v2:172.21.15.184:3300/0 conn(0x7f463c145980 msgr2=0x7f463c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 --2- 172.21.15.40:0/982745561 >> v2:172.21.15.184:3300/0 conn(0x7f463c145980 0x7f463c160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 >> v1:172.21.15.184:6789/0 conn(0x7f463c0b61a0 legacy=0x7f463c0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.123+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c0b4810 con 0x7f463c0b3de0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f462c001670 con 0x7f463c0b3de0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 >> v1:172.21.15.40:6789/0 conn(0x7f463c0b3de0 legacy=0x7f463c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4628005bc0 0x7f4628005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4628006a10 0x7f4628006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f462800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f463c0b5140 con 0x7f46280073d0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f462800e370 con 0x7f4628006a10 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f462800e570 con 0x7f4628005bc0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f462800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46409fd640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4628006a10 0x7f4628006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46411fe640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4628005bc0 0x7f4628005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 -- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4628005bc0 msgr2=0x7f4628005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:32.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4628005bc0 0x7f4628005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 -- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4628006a10 msgr2=0x7f4628006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4628006a10 0x7f4628006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46419ff640 1 -- 172.21.15.40:0/982745561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628005730 con 0x7f46280073d0 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.124+0000 7f46409fd640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4628006a10 0x7f4628006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f46419ff640 1 -- 172.21.15.40:0/982745561 reap_dead start 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f46419ff640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f462800dbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4634001e50 tx=0x7f4628007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c1d178a637db3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634001a70 con 0x7f46280073d0 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4634003970 con 0x7f46280073d0 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f4633fff640 1 -- 172.21.15.40:0/982745561 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634003d60 con 0x7f46280073d0 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f4643462640 1 -- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 msgr2=0x7f462800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.125+0000 7f4643462640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f462800dbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4634001e50 tx=0x7f4628007a40 comp rx=0 tx=0).stop 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.126+0000 7f4643462640 1 -- 172.21.15.40:0/982745561 shutdown_connections 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.126+0000 7f4643462640 1 --2- 172.21.15.40:0/982745561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f462800dbd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.126+0000 7f4643462640 1 -- 172.21.15.40:0/982745561 >> 172.21.15.40:0/982745561 conn(0x7f463c0d0d30 msgr2=0x7f463c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.126+0000 7f4643462640 1 -- 172.21.15.40:0/982745561 shutdown_connections 2024-04-26T21:12:32.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.126+0000 7f4643462640 1 -- 172.21.15.40:0/982745561 wait complete. 2024-04-26T21:12:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.127+0000 7f4643462640 1 Processor -- start 2024-04-26T21:12:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.127+0000 7f4643462640 1 -- start start 2024-04-26T21:12:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f463c1f5a40 0x7f463c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c13be60 0x7f463c1f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f463c160630 con 0x7f46280073d0 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f463c0b4dd0 con 0x7f463c13be60 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f463c1607b0 con 0x7f463c1f5a40 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47462/0 (socket says 172.21.15.40:47462) 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 -- 172.21.15.40:0/3745067691 learned_addr learned my addr 172.21.15.40:0/3745067691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46409fd640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f463c1f5a40 0x7f463c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46419ff640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c13be60 0x7f463c1f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 -- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f463c1f5a40 msgr2=0x7f463c13b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f463c1f5a40 0x7f463c13b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 -- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c13be60 msgr2=0x7f463c1f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c13be60 0x7f463c1f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 -- 172.21.15.40:0/3745067691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1f8fc0 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f46411fe640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4638011540 tx=0x7f4638011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638015930 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638015ad0 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1f4f60 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.128+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800f460 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.129+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1f5520 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.129+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c06a290 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.130+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f463800f600 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.130+0000 7f4631ffb640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.130+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f463808c370 con 0x7f46280073d0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.131+0000 7f46409fd640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.131+0000 7f46409fd640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 0x7f462006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f462c001200 tx=0x7f462c007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.135+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4638019030 con 0x7f46280073d0 2024-04-26T21:12:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.324+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f463c073cc0 con 0x7f46280073d0 2024-04-26T21:12:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.325+0000 7f4631ffb640 1 -- 172.21.15.40:0/3745067691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f463805a7a0 con 0x7f46280073d0 2024-04-26T21:12:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:32.327 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 msgr2=0x7f462006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 0x7f462006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f462c001200 tx=0x7f462c007040 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 msgr2=0x7f463c0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4638011540 tx=0x7f4638011a00 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 shutdown_connections 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f462006c8f0 0x7f462006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f463c13be60 0x7f463c1f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f463c1f5a40 0x7f463c13b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 --2- 172.21.15.40:0/3745067691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46280073d0 0x7f463c0cf810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 >> 172.21.15.40:0/3745067691 conn(0x7f463c0d0d30 msgr2=0x7f463c0b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 shutdown_connections 2024-04-26T21:12:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.327+0000 7f4643462640 1 -- 172.21.15.40:0/3745067691 wait complete. 2024-04-26T21:12:32.339 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:32.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:32.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.442+0000 7f3b8966d640 1 Processor -- start 2024-04-26T21:12:32.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.442+0000 7f3b8966d640 1 -- start start 2024-04-26T21:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b8966d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b84162c60 0x7f3b840cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b8966d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d3f40 0x7f3b840d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b8966d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b84163040 con 0x7f3b840d2140 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b8966d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3b840d07b0 con 0x7f3b84162c60 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b8966d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b840d0a90 con 0x7f3b840d3f40 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b82ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b84162c60 0x7f3b840cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b837fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b840d2140 0x7f3b840d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52312/0 (socket says 172.21.15.40:52312) 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b82ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b84162c60 0x7f3b840cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47474/0 (socket says 172.21.15.40:47474) 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b837fe640 1 -- 172.21.15.40:0/4117735823 learned_addr learned my addr 172.21.15.40:0/4117735823 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.443+0000 7f3b827fc640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d3f40 0x7f3b840d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b82ffd640 1 -- 172.21.15.40:0/4117735823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d3f40 msgr2=0x7f3b840d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b82ffd640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d3f40 0x7f3b840d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b82ffd640 1 -- 172.21.15.40:0/4117735823 >> v1:172.21.15.40:6789/0 conn(0x7f3b840d2140 legacy=0x7f3b840d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b81ffb640 1 -- 172.21.15.40:0/4117735823 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627323364 0 0) 0x7f3b74001490 con 0x7f3b840d2140 2024-04-26T21:12:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b82ffd640 1 -- 172.21.15.40:0/4117735823 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b840d0d70 con 0x7f3b84162c60 2024-04-26T21:12:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b82ffd640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840cfe20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097a0 tx=0x7f3b7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f13da6564972be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b81ffb640 1 -- 172.21.15.40:0/4117735823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037070 con 0x7f3b84162c60 2024-04-26T21:12:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b81ffb640 1 -- 172.21.15.40:0/4117735823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c032440 con 0x7f3b84162c60 2024-04-26T21:12:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.444+0000 7f3b81ffb640 1 -- 172.21.15.40:0/4117735823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c032810 con 0x7f3b84162c60 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 -- 172.21.15.40:0/4117735823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 msgr2=0x7f3b840cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840cfe20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097a0 tx=0x7f3b7c02a210 comp rx=0 tx=0).stop 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 -- 172.21.15.40:0/4117735823 shutdown_connections 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d3f40 0x7f3b840d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 --2- 172.21.15.40:0/4117735823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840cfe20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 -- 172.21.15.40:0/4117735823 >> 172.21.15.40:0/4117735823 conn(0x7f3b8409e550 msgr2=0x7f3b840d5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 -- 172.21.15.40:0/4117735823 shutdown_connections 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.445+0000 7f3b8966d640 1 -- 172.21.15.40:0/4117735823 wait complete. 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.446+0000 7f3b8966d640 1 Processor -- start 2024-04-26T21:12:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.446+0000 7f3b8966d640 1 -- start start 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d2140 0x7f3b840ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 0x7f3b840a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b840d04e0 con 0x7f3b84162c60 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b840d0360 con 0x7f3b840d3f40 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b8966d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b840b9450 con 0x7f3b840d2140 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47480/0 (socket says 172.21.15.40:47480) 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b837fe640 1 -- 172.21.15.40:0/2544084035 learned_addr learned my addr 172.21.15.40:0/2544084035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.447+0000 7f3b827fc640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 0x7f3b840a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 -- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d2140 msgr2=0x7f3b840ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b82ffd640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d2140 0x7f3b840ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d2140 0x7f3b840ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 -- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 msgr2=0x7f3b840a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 0x7f3b840a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 -- 172.21.15.40:0/2544084035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8409efc0 con 0x7f3b84162c60 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b827fc640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 0x7f3b840a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b837fe640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c7f0 tx=0x7f3b7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74018070 con 0x7f3b84162c60 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7400a5e0 con 0x7f3b84162c60 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8409f2a0 con 0x7f3b84162c60 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400aa10 con 0x7f3b84162c60 2024-04-26T21:12:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.448+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8409f860 con 0x7f3b84162c60 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.449+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8406a290 con 0x7f3b84162c60 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.449+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b7401b070 con 0x7f3b84162c60 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.450+0000 7f3b7b7fe640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 0x7f3b6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.450+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3b7408b0d0 con 0x7f3b84162c60 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.450+0000 7f3b82ffd640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 0x7f3b6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.451+0000 7f3b82ffd640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 0x7f3b6406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.454+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b84163040 con 0x7f3b84162c60 2024-04-26T21:12:32.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.644+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b84075fb0 con 0x7f3b84162c60 2024-04-26T21:12:32.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.644+0000 7f3b7b7fe640 1 -- 172.21.15.40:0/2544084035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f3b74059590 con 0x7f3b84162c60 2024-04-26T21:12:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 msgr2=0x7f3b6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 0x7f3b6406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 msgr2=0x7f3b840a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c7f0 tx=0x7f3b7400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 shutdown_connections 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b6406c920 0x7f3b6406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b84162c60 0x7f3b840a1b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b840d3f40 0x7f3b840a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 --2- 172.21.15.40:0/2544084035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b840d2140 0x7f3b840ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 >> 172.21.15.40:0/2544084035 conn(0x7f3b8409e550 msgr2=0x7f3b8406d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 shutdown_connections 2024-04-26T21:12:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.646+0000 7f3b8966d640 1 -- 172.21.15.40:0/2544084035 wait complete. 2024-04-26T21:12:32.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:32.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.761+0000 7f95070d1640 1 Processor -- start 2024-04-26T21:12:32.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.761+0000 7f95070d1640 1 -- start start 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f95070d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950015f5b0 0x7f95000b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f95070d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95000b55b0 0x7f95000d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f95070d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95000b6aa0 con 0x7f950015f990 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f95070d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95000d3eb0 con 0x7f95000b55b0 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f95070d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95000d4190 con 0x7f950015f5b0 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f950566e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95000b55b0 0x7f95000d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f94fffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f950015f990 0x7f95000d0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52314/0 (socket says 172.21.15.40:52314) 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f94fffff640 1 -- 172.21.15.40:0/1614696743 learned_addr learned my addr 172.21.15.40:0/1614696743 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f9504e6d640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950015f5b0 0x7f95000b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f950566e640 1 -- 172.21.15.40:0/1614696743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950015f5b0 msgr2=0x7f95000b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f950566e640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950015f5b0 0x7f95000b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f950566e640 1 -- 172.21.15.40:0/1614696743 >> v1:172.21.15.40:6789/0 conn(0x7f950015f990 legacy=0x7f95000d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1614696743 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547043772 0 0) 0x7f95000b6aa0 con 0x7f950015f990 2024-04-26T21:12:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.762+0000 7f950566e640 1 -- 172.21.15.40:0/1614696743 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95000d0200 con 0x7f95000b55b0 2024-04-26T21:12:32.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.763+0000 7f950566e640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000b55b0 0x7f95000d3340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94f00097a0 tx=0x7f94f002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e20a31e8903a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.763+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1614696743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0037070 con 0x7f95000b55b0 2024-04-26T21:12:32.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.763+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1614696743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f002ae50 con 0x7f95000b55b0 2024-04-26T21:12:32.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.763+0000 7f94ff7fe640 1 -- 172.21.15.40:0/1614696743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0032660 con 0x7f95000b55b0 2024-04-26T21:12:32.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 -- 172.21.15.40:0/1614696743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000b55b0 msgr2=0x7f95000d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000b55b0 0x7f95000d3340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94f00097a0 tx=0x7f94f002a100 comp rx=0 tx=0).stop 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 -- 172.21.15.40:0/1614696743 shutdown_connections 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95000b55b0 0x7f95000d3340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 --2- 172.21.15.40:0/1614696743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950015f5b0 0x7f95000b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 -- 172.21.15.40:0/1614696743 >> 172.21.15.40:0/1614696743 conn(0x7f950008ec40 msgr2=0x7f950009e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 -- 172.21.15.40:0/1614696743 shutdown_connections 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.764+0000 7f95070d1640 1 -- 172.21.15.40:0/1614696743 wait complete. 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.765+0000 7f95070d1640 1 Processor -- start 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.765+0000 7f95070d1640 1 -- start start 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b55b0 0x7f950015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f950015f990 0x7f9500156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95000d1290 con 0x7f950015f5b0 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95000d1110 con 0x7f950015f990 2024-04-26T21:12:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f95070d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95000d3880 con 0x7f95000b55b0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47494/0 (socket says 172.21.15.40:47494) 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 -- 172.21.15.40:0/4217140681 learned_addr learned my addr 172.21.15.40:0/4217140681 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f9504e6d640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b55b0 0x7f950015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f950566e640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f950015f990 0x7f9500156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 -- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b55b0 msgr2=0x7f950015fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b55b0 0x7f950015fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 -- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f950015f990 msgr2=0x7f9500156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f950015f990 0x7f9500156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 -- 172.21.15.40:0/4217140681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9500160490 con 0x7f950015f5b0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.766+0000 7f94fffff640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c460 tx=0x7f94f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.767+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4018070 con 0x7f950015f5b0 2024-04-26T21:12:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.767+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f4007500 con 0x7f950015f5b0 2024-04-26T21:12:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.767+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9500160770 con 0x7f950015f5b0 2024-04-26T21:12:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.767+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004a20 con 0x7f950015f5b0 2024-04-26T21:12:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.767+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9500160d30 con 0x7f950015f5b0 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.768+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950006a290 con 0x7f950015f5b0 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.768+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94f401b070 con 0x7f950015f5b0 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.769+0000 7f94fd7fa640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 0x7f94ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.769+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f94f408ade0 con 0x7f950015f5b0 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.769+0000 7f9504e6d640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 0x7f94ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.769+0000 7f9504e6d640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 0x7f94ec06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f94f80113e0 tx=0x7f94f800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.773+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94f4058b30 con 0x7f950015f5b0 2024-04-26T21:12:32.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.959+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95001639f0 con 0x7f950015f5b0 2024-04-26T21:12:32.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.959+0000 7f94fd7fa640 1 -- 172.21.15.40:0/4217140681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f94f405ce30 con 0x7f950015f5b0 2024-04-26T21:12:32.960 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:32.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 msgr2=0x7f94ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 0x7f94ec06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f94f80113e0 tx=0x7f94f800b040 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 msgr2=0x7f95000b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c460 tx=0x7f94f400c920 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 shutdown_connections 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94ec06c8f0 0x7f94ec06edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f950015f990 0x7f9500156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950015f5b0 0x7f95000b7e00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 --2- 172.21.15.40:0/4217140681 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95000b55b0 0x7f950015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 >> 172.21.15.40:0/4217140681 conn(0x7f950008ec40 msgr2=0x7f9500163c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 shutdown_connections 2024-04-26T21:12:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:32.961+0000 7f95070d1640 1 -- 172.21.15.40:0/4217140681 wait complete. 2024-04-26T21:12:32.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.076+0000 7f17d4d14640 1 Processor -- start 2024-04-26T21:12:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.076+0000 7f17d4d14640 1 -- start start 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17d4d14640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17d015f680 0x7f17d00d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17d4d14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17d0163aa0 0x7f17d00b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17d4d14640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d00d3f50 con 0x7f17d00b60b0 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17d4d14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17d015fa60 con 0x7f17d0163aa0 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17d4d14640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f17d00b5090 con 0x7f17d015f680 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17cdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17d0163aa0 0x7f17d00b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17cdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17d0163aa0 0x7f17d00b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47496/0 (socket says 172.21.15.40:47496) 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17cdd9b640 1 -- 172.21.15.40:0/2035348133 learned_addr learned my addr 172.21.15.40:0/2035348133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.077+0000 7f17ce59c640 1 --2- 172.21.15.40:0/2035348133 >> v2:172.21.15.184:3300/0 conn(0x7f17d015f680 0x7f17d00d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cdd9b640 1 -- 172.21.15.40:0/2035348133 >> v2:172.21.15.184:3300/0 conn(0x7f17d015f680 msgr2=0x7f17d00d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/2035348133 >> v2:172.21.15.184:3300/0 conn(0x7f17d015f680 0x7f17d00d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cdd9b640 1 -- 172.21.15.40:0/2035348133 >> v1:172.21.15.184:6789/0 conn(0x7f17d00b60b0 legacy=0x7f17d00d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cdd9b640 1 -- 172.21.15.40:0/2035348133 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00b5370 con 0x7f17d0163aa0 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/2035348133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d0163aa0 0x7f17d00b5b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17d00d6370 tx=0x7f17b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a643cdd4c582ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.078+0000 7f17cd59a640 1 -- 172.21.15.40:0/2035348133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8037070 con 0x7f17d0163aa0 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17cd59a640 1 -- 172.21.15.40:0/2035348133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b8032440 con 0x7f17d0163aa0 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17cd59a640 1 -- 172.21.15.40:0/2035348133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8032810 con 0x7f17d0163aa0 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 -- 172.21.15.40:0/2035348133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d0163aa0 msgr2=0x7f17d00b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 --2- 172.21.15.40:0/2035348133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d0163aa0 0x7f17d00b5b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17d00d6370 tx=0x7f17b802a270 comp rx=0 tx=0).stop 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 -- 172.21.15.40:0/2035348133 shutdown_connections 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 --2- 172.21.15.40:0/2035348133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d0163aa0 0x7f17d00b5b70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 --2- 172.21.15.40:0/2035348133 >> v2:172.21.15.184:3300/0 conn(0x7f17d015f680 0x7f17d00d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 -- 172.21.15.40:0/2035348133 >> 172.21.15.40:0/2035348133 conn(0x7f17d008eca0 msgr2=0x7f17d009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 -- 172.21.15.40:0/2035348133 shutdown_connections 2024-04-26T21:12:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.079+0000 7f17d4d14640 1 -- 172.21.15.40:0/2035348133 wait complete. 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 Processor -- start 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 -- start start 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 0x7f17d01f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 0x7f17d01f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00d57b0 con 0x7f17d00b60b0 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00d5630 con 0x7f17d0163aa0 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17d4d14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17d00d5930 con 0x7f17d015f680 2024-04-26T21:12:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17ce59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47512/0 (socket says 172.21.15.40:47512) 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.081+0000 7f17ce59c640 1 -- 172.21.15.40:0/3323678427 learned_addr learned my addr 172.21.15.40:0/3323678427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 0x7f17d01f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ced9d640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 0x7f17d01f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 -- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 msgr2=0x7f17d01f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 0x7f17d01f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 -- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 msgr2=0x7f17d01f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 0x7f17d01f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 -- 172.21.15.40:0/3323678427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00cbb90 con 0x7f17d00b60b0 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 0x7f17d01f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ced9d640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 0x7f17d01f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17ce59c640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17c800d800 tx=0x7f17c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c800b6c0 con 0x7f17d00b60b0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c8019070 con 0x7f17d00b60b0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d01f68b0 con 0x7f17d00b60b0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c800ba80 con 0x7f17d00b60b0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.082+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01f9360 con 0x7f17d00b60b0 2024-04-26T21:12:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.083+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d006a290 con 0x7f17d00b60b0 2024-04-26T21:12:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.084+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17c800bc20 con 0x7f17d00b60b0 2024-04-26T21:12:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.084+0000 7f17c6ffd640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 0x7f17b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.084+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f17c808bf90 con 0x7f17d00b60b0 2024-04-26T21:12:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.084+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 0x7f17b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.085+0000 7f17cdd9b640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 0x7f17b006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17b8034000 tx=0x7f17b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.088+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17c801c360 con 0x7f17d00b60b0 2024-04-26T21:12:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.272+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17d00b26d0 con 0x7f17d00b60b0 2024-04-26T21:12:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.272+0000 7f17c6ffd640 1 -- 172.21.15.40:0/3323678427 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f17c805a3c0 con 0x7f17d00b60b0 2024-04-26T21:12:33.274 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 msgr2=0x7f17b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 0x7f17b006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17b8034000 tx=0x7f17b802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 msgr2=0x7f17d00d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17c800d800 tx=0x7f17c800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 shutdown_connections 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f17b006c920 0x7f17b006ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17d0163aa0 0x7f17d01f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17d015f680 0x7f17d01f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 --2- 172.21.15.40:0/3323678427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17d00b60b0 0x7f17d00d1180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 >> 172.21.15.40:0/3323678427 conn(0x7f17d008eca0 msgr2=0x7f17d015db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:33.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 shutdown_connections 2024-04-26T21:12:33.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.274+0000 7f17d4d14640 1 -- 172.21.15.40:0/3323678427 wait complete. 2024-04-26T21:12:33.286 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-04-26T21:12:33.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.385+0000 7fd561d78640 1 Processor -- start 2024-04-26T21:12:33.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd561d78640 1 -- start start 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd561d78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd561d78640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c0d1740 con 0x7fd55c0d11b0 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd561d78640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c0d1a20 con 0x7fd55c06e490 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd561d78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c0d1d00 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.386+0000 7fd55b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd55c0d11b0 0x7fd55c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52330/0 (socket says 172.21.15.40:52330) 2024-04-26T21:12:33.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47522/0 (socket says 172.21.15.40:47522) 2024-04-26T21:12:33.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55affd640 1 -- 172.21.15.40:0/4148706299 learned_addr learned my addr 172.21.15.40:0/4148706299 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:33.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55b7fe640 1 -- 172.21.15.40:0/4148706299 >> v1:172.21.15.184:6789/0 conn(0x7fd55c06e490 legacy=0x7fd55c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:33.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55b7fe640 1 -- 172.21.15.40:0/4148706299 >> v1:172.21.15.40:6789/0 conn(0x7fd55c0d11b0 legacy=0x7fd55c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55b7fe640 1 -- 172.21.15.40:0/4148706299 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c0d1fe0 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.387+0000 7fd55b7fe640 1 --2- 172.21.15.40:0/4148706299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0097d0 tx=0x7fd54c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bd5d585184adbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd55a7fc640 1 -- 172.21.15.40:0/4148706299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c037070 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd55a7fc640 1 -- 172.21.15.40:0/4148706299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c032440 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd55a7fc640 1 -- 172.21.15.40:0/4148706299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c032810 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 -- 172.21.15.40:0/4148706299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c0cf2e0 msgr2=0x7fd55c06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 --2- 172.21.15.40:0/4148706299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0097d0 tx=0x7fd54c02a210 comp rx=0 tx=0).stop 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 -- 172.21.15.40:0/4148706299 shutdown_connections 2024-04-26T21:12:33.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 --2- 172.21.15.40:0/4148706299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c06d4d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 -- 172.21.15.40:0/4148706299 >> 172.21.15.40:0/4148706299 conn(0x7fd55c0b4a60 msgr2=0x7fd55c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:33.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 -- 172.21.15.40:0/4148706299 shutdown_connections 2024-04-26T21:12:33.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.388+0000 7fd561d78640 1 -- 172.21.15.40:0/4148706299 wait complete. 2024-04-26T21:12:33.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.389+0000 7fd561d78640 1 Processor -- start 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 -- start start 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 0x7fd55c0b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0cf6c0 con 0x7fd55c06e490 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c06da10 con 0x7fd55c0cf2e0 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd561d78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c0cf840 con 0x7fd55c0d11b0 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47532/0 (socket says 172.21.15.40:47532) 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.390+0000 7fd55b7fe640 1 -- 172.21.15.40:0/1462855114 learned_addr learned my addr 172.21.15.40:0/1462855114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 -- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 msgr2=0x7fd55c0b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55bfff640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 0x7fd55c0b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55affd640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 0x7fd55c0b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 -- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 msgr2=0x7fd55c0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c0b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 -- 172.21.15.40:0/1462855114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c1ea450 con 0x7fd55c06e490 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55bfff640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 0x7fd55c0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55affd640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd55b7fe640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098f0 tx=0x7fd54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c037040 con 0x7fd55c06e490 2024-04-26T21:12:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c004250 con 0x7fd55c06e490 2024-04-26T21:12:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c1ea730 con 0x7fd55c06e490 2024-04-26T21:12:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c03c730 con 0x7fd55c06e490 2024-04-26T21:12:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.391+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c1f5710 con 0x7fd55c06e490 2024-04-26T21:12:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.392+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55c06a290 con 0x7fd55c06e490 2024-04-26T21:12:33.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.393+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd54c03a070 con 0x7fd55c06e490 2024-04-26T21:12:33.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.393+0000 7fd543fff640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:33.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.393+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd54c0ac470 con 0x7fd55c06e490 2024-04-26T21:12:33.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.393+0000 7fd55affd640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:33.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.394+0000 7fd55affd640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 0x7fd54806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0b86e0 tx=0x7fd550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:33.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.398+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd54c041020 con 0x7fd55c06e490 2024-04-26T21:12:33.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.653+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7fd55c0b8ef0 con 0x7fd55c06e490 2024-04-26T21:12:33.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.997+0000 7fd543fff640 1 -- 172.21.15.40:0/1462855114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v53) ==== 168+0+0 (secure 0 0 0) 0x7fd54c07a930 con 0x7fd55c06e490 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 msgr2=0x7fd54806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 0x7fd54806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0b86e0 tx=0x7fd550009040 comp rx=0 tx=0).stop 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 msgr2=0x7fd55c162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098f0 tx=0x7fd54c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 shutdown_connections 2024-04-26T21:12:34.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd54806c8f0 0x7fd54806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd55c0d11b0 0x7fd55c0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd55c0cf2e0 0x7fd55c0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 --2- 172.21.15.40:0/1462855114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd55c06e490 0x7fd55c162590 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 >> 172.21.15.40:0/1462855114 conn(0x7fd55c0b4a60 msgr2=0x7fd55c071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 shutdown_connections 2024-04-26T21:12:34.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:33.999+0000 7fd561d78640 1 -- 172.21.15.40:0/1462855114 wait complete. 2024-04-26T21:12:34.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:12:34.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.114+0000 7f04ec580640 1 Processor -- start 2024-04-26T21:12:34.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 -- start start 2024-04-26T21:12:34.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e415f470 0x7f04e40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04e40b47d0 0x7f04e406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e40aeb50 con 0x7f04e4069ae0 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f04e40aee30 con 0x7f04e415f470 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.115+0000 7f04ec580640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f04e40af110 con 0x7f04e40b47d0 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e415f470 0x7f04e40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04e415f470 0x7f04e40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47548/0 (socket says 172.21.15.40:47548) 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 -- 172.21.15.40:0/1369008795 learned_addr learned my addr 172.21.15.40:0/1369008795 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 -- 172.21.15.40:0/1369008795 >> v2:172.21.15.184:3300/0 conn(0x7f04e40b47d0 msgr2=0x7f04e406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04eab1d640 1 --2- 172.21.15.40:0/1369008795 >> v2:172.21.15.184:3300/0 conn(0x7f04e40b47d0 0x7f04e406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 --2- 172.21.15.40:0/1369008795 >> v2:172.21.15.184:3300/0 conn(0x7f04e40b47d0 0x7f04e406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 -- 172.21.15.40:0/1369008795 >> v1:172.21.15.184:6789/0 conn(0x7f04e4069ae0 legacy=0x7f04e40b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.116+0000 7f04ea31c640 1 -- 172.21.15.40:0/1369008795 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e40af3f0 con 0x7f04e415f470 2024-04-26T21:12:34.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04ea31c640 1 --2- 172.21.15.40:0/1369008795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e415f470 0x7f04e40af910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04e40710b0 tx=0x7f04dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576332650577a06e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04e931a640 1 -- 172.21.15.40:0/1369008795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc037070 con 0x7f04e415f470 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04e931a640 1 -- 172.21.15.40:0/1369008795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc032440 con 0x7f04e415f470 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04e931a640 1 -- 172.21.15.40:0/1369008795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc032810 con 0x7f04e415f470 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04ec580640 1 -- 172.21.15.40:0/1369008795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e415f470 msgr2=0x7f04e40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.117+0000 7f04ec580640 1 --2- 172.21.15.40:0/1369008795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e415f470 0x7f04e40af910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04e40710b0 tx=0x7f04dc02a230 comp rx=0 tx=0).stop 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 -- 172.21.15.40:0/1369008795 shutdown_connections 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 --2- 172.21.15.40:0/1369008795 >> v2:172.21.15.184:3300/0 conn(0x7f04e40b47d0 0x7f04e406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 --2- 172.21.15.40:0/1369008795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e415f470 0x7f04e40af910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 -- 172.21.15.40:0/1369008795 >> 172.21.15.40:0/1369008795 conn(0x7f04e4073910 msgr2=0x7f04e4073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 -- 172.21.15.40:0/1369008795 shutdown_connections 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.118+0000 7f04ec580640 1 -- 172.21.15.40:0/1369008795 wait complete. 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 Processor -- start 2024-04-26T21:12:34.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 -- start start 2024-04-26T21:12:34.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e40b47d0 0x7f04e414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 0x7f04e4152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40b63a0 con 0x7f04e4069ae0 2024-04-26T21:12:34.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.119+0000 7f04ec580640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40b6220 con 0x7f04e415f470 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ec580640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f04e40afe50 con 0x7f04e40b47d0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47556/0 (socket says 172.21.15.40:47556) 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 -- 172.21.15.40:0/118830336 learned_addr learned my addr 172.21.15.40:0/118830336 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04eab1d640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 0x7f04e4152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04e9b1b640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e40b47d0 0x7f04e414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 -- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e40b47d0 msgr2=0x7f04e414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e40b47d0 0x7f04e414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 -- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 msgr2=0x7f04e4152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 0x7f04e4152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 -- 172.21.15.40:0/118830336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41526f0 con 0x7f04e4069ae0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04eab1d640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 0x7f04e4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ea31c640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04dc000c00 tx=0x7f04dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc041020 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc030970 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e414f9d0 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc037040 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.120+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41529b0 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.121+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e40b06f0 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.122+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04dc044020 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.122+0000 7f04e2ffd640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.122+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f04dc0ac4e0 con 0x7f04e4069ae0 2024-04-26T21:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.122+0000 7f04e9b1b640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.123+0000 7f04e9b1b640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 0x7f04d006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.126+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04e40b06f0 con 0x7f04e4069ae0 2024-04-26T21:12:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.311+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04e406a290 con 0x7f04e4069ae0 2024-04-26T21:12:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.314+0000 7f04e2ffd640 1 -- 172.21.15.40:0/118830336 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f04dc04b030 con 0x7f04e4069ae0 2024-04-26T21:12:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 msgr2=0x7f04d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 0x7f04d006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8008040 comp rx=0 tx=0).stop 2024-04-26T21:12:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 msgr2=0x7f04e414e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04dc000c00 tx=0x7f04dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 shutdown_connections 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04e415f470 0x7f04e4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04e40b47d0 0x7f04e414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 --2- 172.21.15.40:0/118830336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04e4069ae0 0x7f04e414e590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 >> 172.21.15.40:0/118830336 conn(0x7f04e4073910 msgr2=0x7f04e40d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 shutdown_connections 2024-04-26T21:12:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.316+0000 7f04ec580640 1 -- 172.21.15.40:0/118830336 wait complete. 2024-04-26T21:12:34.328 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:12:34.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:12:34.369 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:12:34.411 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:12:34.411 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:34.411 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:34.427 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:34.427 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:34.482 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:34.482 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:34.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.677+0000 7fcc66339640 1 Processor -- start 2024-04-26T21:12:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.677+0000 7fcc66339640 1 -- start start 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc600cf2e0 0x7fcc6006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 0x7fcc6006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 0x7fcc6015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6006e750 con 0x7fcc600cf2e0 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcc600d1740 con 0x7fcc600d11b0 2024-04-26T21:12:34.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc66339640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc600d1a20 con 0x7fcc6006e2c0 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc600cf2e0 0x7fcc6006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc600cf2e0 0x7fcc6006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47558/0 (socket says 172.21.15.40:47558) 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 -- 172.21.15.40:0/3310354503 learned_addr learned my addr 172.21.15.40:0/3310354503 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 -- 172.21.15.40:0/3310354503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 msgr2=0x7fcc6015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5f7fe640 1 --2- 172.21.15.40:0/3310354503 >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 0x7fcc6006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc648d6640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 0x7fcc6015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 0x7fcc6015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 -- 172.21.15.40:0/3310354503 >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 msgr2=0x7fcc6006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3310354503 >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 0x7fcc6006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5ffff640 1 -- 172.21.15.40:0/3310354503 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc600d1d00 con 0x7fcc600cf2e0 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc5f7fe640 1 --2- 172.21.15.40:0/3310354503 >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 0x7fcc6006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.678+0000 7fcc648d6640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 0x7fcc6015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:34.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.679+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600cf2e0 0x7fcc6006d4d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8528b669d31f0b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.679+0000 7fcc5effd640 1 -- 172.21.15.40:0/3310354503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48037070 con 0x7fcc600cf2e0 2024-04-26T21:12:34.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.679+0000 7fcc5effd640 1 -- 172.21.15.40:0/3310354503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc48032440 con 0x7fcc600cf2e0 2024-04-26T21:12:34.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.679+0000 7fcc5effd640 1 -- 172.21.15.40:0/3310354503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48032810 con 0x7fcc600cf2e0 2024-04-26T21:12:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 -- 172.21.15.40:0/3310354503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600cf2e0 msgr2=0x7fcc6006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600cf2e0 0x7fcc6006d4d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802a210 comp rx=0 tx=0).stop 2024-04-26T21:12:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 -- 172.21.15.40:0/3310354503 shutdown_connections 2024-04-26T21:12:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc6006e2c0 0x7fcc6015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 --2- 172.21.15.40:0/3310354503 >> v2:172.21.15.184:3300/0 conn(0x7fcc600d11b0 0x7fcc6006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 --2- 172.21.15.40:0/3310354503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600cf2e0 0x7fcc6006d4d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 -- 172.21.15.40:0/3310354503 >> 172.21.15.40:0/3310354503 conn(0x7fcc600b4a60 msgr2=0x7fcc600b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 -- 172.21.15.40:0/3310354503 shutdown_connections 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.680+0000 7fcc66339640 1 -- 172.21.15.40:0/3310354503 wait complete. 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.681+0000 7fcc66339640 1 Processor -- start 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.681+0000 7fcc66339640 1 -- start start 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6006e2c0 0x7fcc601484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc600cf2e0 0x7fcc6015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6006da10 con 0x7fcc600d11b0 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc600d22c0 con 0x7fcc6006e2c0 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc66339640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc600cf6c0 con 0x7fcc600cf2e0 2024-04-26T21:12:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47560/0 (socket says 172.21.15.40:47560) 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 -- 172.21.15.40:0/3762026757 learned_addr learned my addr 172.21.15.40:0/3762026757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6006e2c0 0x7fcc601484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc5f7fe640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc600cf2e0 0x7fcc6015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 -- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc600cf2e0 msgr2=0x7fcc6015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc600cf2e0 0x7fcc6015bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 -- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6006e2c0 msgr2=0x7fcc601484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6006e2c0 0x7fcc601484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.682+0000 7fcc648d6640 1 -- 172.21.15.40:0/3762026757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc600b7ac0 con 0x7fcc600d11b0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc648d6640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800cf20 tx=0x7fcc5800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5801a070 con 0x7fcc600d11b0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5800d840 con 0x7fcc600d11b0 2024-04-26T21:12:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc600b7da0 con 0x7fcc600d11b0 2024-04-26T21:12:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800dc30 con 0x7fcc600d11b0 2024-04-26T21:12:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.683+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc600b8360 con 0x7fcc600d11b0 2024-04-26T21:12:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.684+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6006a290 con 0x7fcc600d11b0 2024-04-26T21:12:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.684+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc5801d070 con 0x7fcc600d11b0 2024-04-26T21:12:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.685+0000 7fcc5cff9640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 0x7fcc4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.685+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc5808c520 con 0x7fcc600d11b0 2024-04-26T21:12:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.685+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 0x7fcc4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.685+0000 7fcc5ffff640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 0x7fcc4006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcc48034000 tx=0x7fcc4802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:34.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.689+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc5805a820 con 0x7fcc600d11b0 2024-04-26T21:12:34.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.874+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc60071870 con 0x7fcc600d11b0 2024-04-26T21:12:34.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.875+0000 7fcc5cff9640 1 -- 172.21.15.40:0/3762026757 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7fcc5805a120 con 0x7fcc600d11b0 2024-04-26T21:12:34.876 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:34.876 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 msgr2=0x7fcc4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 0x7fcc4006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcc48034000 tx=0x7fcc4802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 msgr2=0x7fcc6015c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800cf20 tx=0x7fcc5800a7d0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 shutdown_connections 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc4006c8f0 0x7fcc4006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc600d11b0 0x7fcc6015c230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc600cf2e0 0x7fcc6015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 --2- 172.21.15.40:0/3762026757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc6006e2c0 0x7fcc601484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 >> 172.21.15.40:0/3762026757 conn(0x7fcc600b4a60 msgr2=0x7fcc600c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 shutdown_connections 2024-04-26T21:12:34.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.877+0000 7fcc66339640 1 -- 172.21.15.40:0/3762026757 wait complete. 2024-04-26T21:12:34.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:34.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.992+0000 7f328bb22640 1 Processor -- start 2024-04-26T21:12:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 -- start start 2024-04-26T21:12:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3284162ca0 0x7f3284070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3284070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32840d2470 con 0x7f3284071160 2024-04-26T21:12:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f32840715d0 con 0x7f3284162ca0 2024-04-26T21:12:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328bb22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f32840cdc40 con 0x7f32840d1c20 2024-04-26T21:12:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328a0bf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3284071160 0x7f32840cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52344/0 (socket says 172.21.15.40:52344) 2024-04-26T21:12:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.993+0000 7f328a0bf640 1 -- 172.21.15.40:0/2514995709 learned_addr learned my addr 172.21.15.40:0/2514995709 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32898be640 1 --2- 172.21.15.40:0/2514995709 >> v2:172.21.15.184:3300/0 conn(0x7f3284162ca0 0x7f3284070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32890bd640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3284070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789814496 0 0) 0x7f32840d2470 con 0x7f3284071160 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270003520 con 0x7f3284071160 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201503187 0 0) 0x7f3270003520 con 0x7f3284071160 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 msgr2=0x7f3284070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3284070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 >> v2:172.21.15.184:3300/0 conn(0x7f3284162ca0 msgr2=0x7f3284070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 --2- 172.21.15.40:0/2514995709 >> v2:172.21.15.184:3300/0 conn(0x7f3284162ca0 0x7f3284070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32840d2750 con 0x7f3284071160 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f327c0019f0 con 0x7f3284071160 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 >> v1:172.21.15.40:6789/0 conn(0x7f3284071160 legacy=0x7f32840cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f3270003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3270004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32840715d0 con 0x7f32700039b0 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32840cdc40 con 0x7f3270004d40 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f327000b9c0 con 0x7f32840d1c20 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32898be640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f3270003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f32890bd640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3270004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.994+0000 7f328a0bf640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 -- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 msgr2=0x7f3270004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840d1c20 0x7f3270004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 -- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3270004d40 msgr2=0x7f32700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 -- 172.21.15.40:0/2514995709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270003520 con 0x7f32700039b0 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f328a0bf640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32898be640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f3270003dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ac0 tx=0x7f3270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b48872a805744b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280036070 con 0x7f32700039b0 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280031400 con 0x7f32700039b0 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.995+0000 7f32888bc640 1 -- 172.21.15.40:0/2514995709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280009690 con 0x7f32700039b0 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 -- 172.21.15.40:0/2514995709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 msgr2=0x7f3270003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 --2- 172.21.15.40:0/2514995709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f3270003dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ac0 tx=0x7f3270005830 comp rx=0 tx=0).stop 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328a0bf640 1 -- 172.21.15.40:0/2514995709 reap_dead start 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 -- 172.21.15.40:0/2514995709 shutdown_connections 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 -- 172.21.15.40:0/2514995709 >> 172.21.15.40:0/2514995709 conn(0x7f328409ba40 msgr2=0x7f328409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 -- 172.21.15.40:0/2514995709 shutdown_connections 2024-04-26T21:12:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.996+0000 7f328bb22640 1 -- 172.21.15.40:0/2514995709 wait complete. 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.997+0000 7f328bb22640 1 Processor -- start 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.997+0000 7f328bb22640 1 -- start start 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.998+0000 7f328bb22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328bb22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328414c1f0 0x7f32841f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328bb22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840b03e0 0x7f32840b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328bb22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32840d2080 con 0x7f32700039b0 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328bb22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32840cd8d0 con 0x7f328414c1f0 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328bb22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f32840d2200 con 0x7f32840b03e0 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47582/0 (socket says 172.21.15.40:47582) 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 -- 172.21.15.40:0/1993693485 learned_addr learned my addr 172.21.15.40:0/1993693485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32890bd640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328414c1f0 0x7f32841f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f328a0bf640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840b03e0 0x7f32840b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 -- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840b03e0 msgr2=0x7f32840b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840b03e0 0x7f32840b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 -- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328414c1f0 msgr2=0x7f32841f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328414c1f0 0x7f32841f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 -- 172.21.15.40:0/1993693485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284146190 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:34.999+0000 7f32898be640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f328002abe0 tx=0x7f3280004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.000+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280040020 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.000+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280003890 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.000+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32840b1860 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.000+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280036040 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.000+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32840b12a0 con 0x7f32700039b0 2024-04-26T21:12:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.001+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32840693d0 con 0x7f32700039b0 2024-04-26T21:12:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.001+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3280043020 con 0x7f32700039b0 2024-04-26T21:12:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.002+0000 7f327a7fc640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 0x7f326806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.002+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f32800ab500 con 0x7f32700039b0 2024-04-26T21:12:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.002+0000 7f32890bd640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 0x7f326806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.002+0000 7f32890bd640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 0x7f326806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3284075d60 tx=0x7f3274009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.006+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3280079a00 con 0x7f32700039b0 2024-04-26T21:12:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.189+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32840b26e0 con 0x7f32700039b0 2024-04-26T21:12:35.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.190+0000 7f327a7fc640 1 -- 172.21.15.40:0/1993693485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f3280079300 con 0x7f32700039b0 2024-04-26T21:12:35.191 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 msgr2=0x7f326806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 0x7f326806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3284075d60 tx=0x7f3274009190 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 msgr2=0x7f32841f83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f328002abe0 tx=0x7f3280004730 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 shutdown_connections 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f326806c920 0x7f326806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32840b03e0 0x7f32840b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328414c1f0 0x7f32841f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 --2- 172.21.15.40:0/1993693485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32700039b0 0x7f32841f83b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 >> 172.21.15.40:0/1993693485 conn(0x7f328409ba40 msgr2=0x7f3284161470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 shutdown_connections 2024-04-26T21:12:35.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.192+0000 7f328bb22640 1 -- 172.21.15.40:0/1993693485 wait complete. 2024-04-26T21:12:35.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:35.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.306+0000 7fa6224a0640 1 Processor -- start 2024-04-26T21:12:35.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.306+0000 7fa6224a0640 1 -- start start 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 0x7fa61c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 0x7fa61c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c0b4480 con 0x7fa61c0b3c20 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c06e050 con 0x7fa61c0b3210 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa6224a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c09c700 con 0x7fa61c0d0010 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47586/0 (socket says 172.21.15.40:47586) 2024-04-26T21:12:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 -- 172.21.15.40:0/2614072545 learned_addr learned my addr 172.21.15.40:0/2614072545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa620a3d640 1 --2- 172.21.15.40:0/2614072545 >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 0x7fa61c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 -- 172.21.15.40:0/2614072545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 msgr2=0x7fa61c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61bfff640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 0x7fa61c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 0x7fa61c06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 -- 172.21.15.40:0/2614072545 >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 msgr2=0x7fa61c0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 --2- 172.21.15.40:0/2614072545 >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 0x7fa61c0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61b7fe640 1 -- 172.21.15.40:0/2614072545 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c0d1c50 con 0x7fa61c0b3c20 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa620a3d640 1 --2- 172.21.15.40:0/2614072545 >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 0x7fa61c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.307+0000 7fa61bfff640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 0x7fa61c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.308+0000 7fa61b7fe640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0741b0 tx=0x7fa60c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8628596eed6ffc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.308+0000 7fa61affd640 1 -- 172.21.15.40:0/2614072545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c037070 con 0x7fa61c0b3c20 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.308+0000 7fa61affd640 1 -- 172.21.15.40:0/2614072545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa60c032440 con 0x7fa61c0b3c20 2024-04-26T21:12:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.308+0000 7fa61affd640 1 -- 172.21.15.40:0/2614072545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c032810 con 0x7fa61c0b3c20 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 -- 172.21.15.40:0/2614072545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0b3c20 msgr2=0x7fa61c0b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0741b0 tx=0x7fa60c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 -- 172.21.15.40:0/2614072545 shutdown_connections 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2614072545 >> v2:172.21.15.184:3300/0 conn(0x7fa61c0b3210 0x7fa61c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0b3c20 0x7fa61c0b2cd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2614072545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0d0010 0x7fa61c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 -- 172.21.15.40:0/2614072545 >> 172.21.15.40:0/2614072545 conn(0x7fa61c09ba40 msgr2=0x7fa61c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 -- 172.21.15.40:0/2614072545 shutdown_connections 2024-04-26T21:12:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.309+0000 7fa6224a0640 1 -- 172.21.15.40:0/2614072545 wait complete. 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.310+0000 7fa6224a0640 1 Processor -- start 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.310+0000 7fa6224a0640 1 -- start start 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0b3210 0x7fa61c1e9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0b3c20 0x7fa61c151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c15fe90 con 0x7fa61c0d0010 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c15fd10 con 0x7fa61c0b3c20 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa6224a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c160010 con 0x7fa61c0b3210 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47594/0 (socket says 172.21.15.40:47594) 2024-04-26T21:12:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 -- 172.21.15.40:0/2779395646 learned_addr learned my addr 172.21.15.40:0/2779395646 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa61b7fe640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0b3c20 0x7fa61c151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa61bfff640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0b3210 0x7fa61c1e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 -- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0b3210 msgr2=0x7fa61c1e9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0b3210 0x7fa61c1e9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 -- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0b3c20 msgr2=0x7fa61c151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0b3c20 0x7fa61c151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 -- 172.21.15.40:0/2779395646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c0b73b0 con 0x7fa61c0d0010 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.311+0000 7fa620a3d640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa61400cef0 tx=0x7fa61400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.312+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401a070 con 0x7fa61c0d0010 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.312+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61400d860 con 0x7fa61c0d0010 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.312+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c0b7690 con 0x7fa61c0d0010 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.312+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400dc50 con 0x7fa61c0d0010 2024-04-26T21:12:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.312+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c0b7c50 con 0x7fa61c0d0010 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.313+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c06a290 con 0x7fa61c0d0010 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.313+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa61401d070 con 0x7fa61c0d0010 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.314+0000 7fa618ff9640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 0x7fa5fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.314+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa61408c390 con 0x7fa61c0d0010 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.314+0000 7fa61bfff640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 0x7fa5fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.314+0000 7fa61bfff640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 0x7fa5fc06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b7c0 tx=0x7fa604008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.320+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa61405a910 con 0x7fa61c0d0010 2024-04-26T21:12:35.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.504+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa61c068e00 con 0x7fa61c0d0010 2024-04-26T21:12:35.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.504+0000 7fa618ff9640 1 -- 172.21.15.40:0/2779395646 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa61405a210 con 0x7fa61c0d0010 2024-04-26T21:12:35.506 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.506+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 msgr2=0x7fa5fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.506+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 0x7fa5fc06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b7c0 tx=0x7fa604008040 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.506+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 msgr2=0x7fa61c1523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.506+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa61400cef0 tx=0x7fa61400a710 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 shutdown_connections 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa5fc06c8f0 0x7fa5fc06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c0d0010 0x7fa61c1523c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c0b3c20 0x7fa61c151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 --2- 172.21.15.40:0/2779395646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa61c0b3210 0x7fa61c1e9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 >> 172.21.15.40:0/2779395646 conn(0x7fa61c09ba40 msgr2=0x7fa61c15f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 shutdown_connections 2024-04-26T21:12:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.507+0000 7fa6224a0640 1 -- 172.21.15.40:0/2779395646 wait complete. 2024-04-26T21:12:35.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:35.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.622+0000 7f56c16cc640 1 Processor -- start 2024-04-26T21:12:35.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.622+0000 7f56c16cc640 1 -- start start 2024-04-26T21:12:35.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56c16cc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 0x7f56bc0d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56c16cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56bc0cfe20 0x7f56bc0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56c16cc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56bc0d5cd0 con 0x7f56bc0d36b0 2024-04-26T21:12:35.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56c16cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0d5a90 con 0x7f56bc0cfe20 2024-04-26T21:12:35.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56c16cc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0b50e0 con 0x7f56bc0d50c0 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56bc0cfe20 0x7f56bc0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56bc0cfe20 0x7f56bc0d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47608/0 (socket says 172.21.15.40:47608) 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 -- 172.21.15.40:0/4237443786 learned_addr learned my addr 172.21.15.40:0/4237443786 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56b3fff640 1 --2- 172.21.15.40:0/4237443786 >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 0x7f56bc0d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 -- 172.21.15.40:0/4237443786 >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 msgr2=0x7f56bc0d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 --2- 172.21.15.40:0/4237443786 >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 0x7f56bc0d5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 -- 172.21.15.40:0/4237443786 >> v1:172.21.15.184:6789/0 conn(0x7f56bc0d36b0 legacy=0x7f56bc0d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56bb7fe640 1 -- 172.21.15.40:0/4237443786 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc0b5f00 con 0x7f56bc0cfe20 2024-04-26T21:12:35.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.623+0000 7f56b3fff640 1 --2- 172.21.15.40:0/4237443786 >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 0x7f56bc0d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:35.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.624+0000 7f56bb7fe640 1 --2- 172.21.15.40:0/4237443786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0cfe20 0x7f56bc0d0250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f56bc08d600 tx=0x7f56b402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95633faf78499c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.624+0000 7f56ba7fc640 1 -- 172.21.15.40:0/4237443786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4036070 con 0x7f56bc0cfe20 2024-04-26T21:12:35.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.624+0000 7f56ba7fc640 1 -- 172.21.15.40:0/4237443786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b4031600 con 0x7f56bc0cfe20 2024-04-26T21:12:35.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.624+0000 7f56ba7fc640 1 -- 172.21.15.40:0/4237443786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40319d0 con 0x7f56bc0cfe20 2024-04-26T21:12:35.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.624+0000 7f56c16cc640 1 -- 172.21.15.40:0/4237443786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0cfe20 msgr2=0x7f56bc0d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 --2- 172.21.15.40:0/4237443786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0cfe20 0x7f56bc0d0250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f56bc08d600 tx=0x7f56b402a730 comp rx=0 tx=0).stop 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 -- 172.21.15.40:0/4237443786 shutdown_connections 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 --2- 172.21.15.40:0/4237443786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0cfe20 0x7f56bc0d0250 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 --2- 172.21.15.40:0/4237443786 >> v2:172.21.15.184:3300/0 conn(0x7f56bc0d50c0 0x7f56bc0d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 -- 172.21.15.40:0/4237443786 >> 172.21.15.40:0/4237443786 conn(0x7f56bc08ec30 msgr2=0x7f56bc09e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 -- 172.21.15.40:0/4237443786 shutdown_connections 2024-04-26T21:12:35.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.625+0000 7f56c16cc640 1 -- 172.21.15.40:0/4237443786 wait complete. 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.626+0000 7f56c16cc640 1 Processor -- start 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.626+0000 7f56c16cc640 1 -- start start 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.626+0000 7f56c16cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56bc0cfe20 0x7f56bc14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56bc0d36b0 0x7f56bc0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0b67a0 con 0x7f56bc0d50c0 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0b64c0 con 0x7f56bc0cfe20 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc15f750 con 0x7f56bc0d36b0 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47614/0 (socket says 172.21.15.40:47614) 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 -- 172.21.15.40:0/2537144866 learned_addr learned my addr 172.21.15.40:0/2537144866 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56baffd640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56bc0cfe20 0x7f56bc14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56b3fff640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56bc0d36b0 0x7f56bc0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 -- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56bc0d36b0 msgr2=0x7f56bc0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56bc0d36b0 0x7f56bc0cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 -- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56bc0cfe20 msgr2=0x7f56bc14b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56bc0cfe20 0x7f56bc14b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 -- 172.21.15.40:0/2537144866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc14b700 con 0x7f56bc0d50c0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56bb7fe640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034040 tx=0x7f56b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4036050 con 0x7f56bc0d50c0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b4042070 con 0x7f56bc0d50c0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc14ba80 con 0x7f56bc0d50c0 2024-04-26T21:12:35.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4009510 con 0x7f56bc0d50c0 2024-04-26T21:12:35.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.627+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc14c040 con 0x7f56bc0d50c0 2024-04-26T21:12:35.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.628+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc06a290 con 0x7f56bc0d50c0 2024-04-26T21:12:35.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.629+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56b4007bf0 con 0x7f56bc0d50c0 2024-04-26T21:12:35.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.629+0000 7f56b37fe640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 0x7f569c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.630+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f56b40aca00 con 0x7f56bc0d50c0 2024-04-26T21:12:35.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.630+0000 7f56baffd640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 0x7f569c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.630+0000 7f56baffd640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 0x7f569c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001640 tx=0x7f56ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.634+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56bc14b700 con 0x7f56bc0d50c0 2024-04-26T21:12:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.824+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56bc0b2eb0 con 0x7f56bc0d50c0 2024-04-26T21:12:35.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.824+0000 7f56b37fe640 1 -- 172.21.15.40:0/2537144866 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f56b407ada0 con 0x7f56bc0d50c0 2024-04-26T21:12:35.825 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 msgr2=0x7f569c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 0x7f569c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001640 tx=0x7f56ac009040 comp rx=0 tx=0).stop 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 msgr2=0x7f56bc0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034040 tx=0x7f56b4004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 shutdown_connections 2024-04-26T21:12:35.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f569c06c8f0 0x7f569c06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56bc0d50c0 0x7f56bc0cb600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56bc0d36b0 0x7f56bc0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 --2- 172.21.15.40:0/2537144866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56bc0cfe20 0x7f56bc14b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 >> 172.21.15.40:0/2537144866 conn(0x7f56bc08ec30 msgr2=0x7f56bc0b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 shutdown_connections 2024-04-26T21:12:35.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.826+0000 7f56c16cc640 1 -- 172.21.15.40:0/2537144866 wait complete. 2024-04-26T21:12:35.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:12:35.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.938+0000 7f1fbebec640 1 Processor -- start 2024-04-26T21:12:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.938+0000 7f1fbebec640 1 -- start start 2024-04-26T21:12:35.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbebec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1fb8140350 0x7f1fb80cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbebec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb8146790 0x7f1fb80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbebec640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80d05b0 con 0x7f1fb813ff70 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbebec640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb80d0890 con 0x7f1fb8140350 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbebec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb806e450 con 0x7f1fb8146790 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbc988640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1fb813ff70 0x7f1fb8146080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52358/0 (socket says 172.21.15.40:52358) 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fb7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1fb8140350 0x7f1fb80cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.939+0000 7f1fbc988640 1 -- 172.21.15.40:0/2818431922 learned_addr learned my addr 172.21.15.40:0/2818431922 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.940+0000 7f1fb7fff640 1 -- 172.21.15.40:0/2818431922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb8146790 msgr2=0x7f1fb80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.940+0000 7f1fb7fff640 1 --2- 172.21.15.40:0/2818431922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb8146790 0x7f1fb80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.940+0000 7f1fb7fff640 1 -- 172.21.15.40:0/2818431922 >> v1:172.21.15.40:6789/0 conn(0x7f1fb813ff70 legacy=0x7f1fb8146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.940+0000 7f1fb7fff640 1 -- 172.21.15.40:0/2818431922 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb806e730 con 0x7f1fb8140350 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.940+0000 7f1fb7fff640 1 --2- 172.21.15.40:0/2818431922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8140350 0x7f1fb80cf810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097a0 tx=0x7f1fac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265be1f49b3018d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fb77fe640 1 -- 172.21.15.40:0/2818431922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac037070 con 0x7f1fb8140350 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fb77fe640 1 -- 172.21.15.40:0/2818431922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac032440 con 0x7f1fb8140350 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fb77fe640 1 -- 172.21.15.40:0/2818431922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac02ae60 con 0x7f1fb8140350 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 -- 172.21.15.40:0/2818431922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8140350 msgr2=0x7f1fb80cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2818431922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8140350 0x7f1fb80cf810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097a0 tx=0x7f1fac02a110 comp rx=0 tx=0).stop 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 -- 172.21.15.40:0/2818431922 shutdown_connections 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2818431922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb8146790 0x7f1fb80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2818431922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8140350 0x7f1fb80cf810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 -- 172.21.15.40:0/2818431922 >> 172.21.15.40:0/2818431922 conn(0x7f1fb80d0d30 msgr2=0x7f1fb80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 -- 172.21.15.40:0/2818431922 shutdown_connections 2024-04-26T21:12:35.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.941+0000 7f1fbebec640 1 -- 172.21.15.40:0/2818431922 wait complete. 2024-04-26T21:12:35.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.942+0000 7f1fbebec640 1 Processor -- start 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.942+0000 7f1fbebec640 1 -- start start 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 0x7f1fb8142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb8140350 0x7f1fb80af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb806edd0 con 0x7f1fb8146790 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb806ec50 con 0x7f1fb8140350 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbebec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb806ef50 con 0x7f1fb813ff70 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbd189640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbd189640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47630/0 (socket says 172.21.15.40:47630) 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.943+0000 7f1fbd189640 1 -- 172.21.15.40:0/2616364164 learned_addr learned my addr 172.21.15.40:0/2616364164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbc988640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 0x7f1fb8142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fb7fff640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb8140350 0x7f1fb80af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 -- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 msgr2=0x7f1fb8142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 0x7f1fb8142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 -- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb8140350 msgr2=0x7f1fb80af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb8140350 0x7f1fb80af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 -- 172.21.15.40:0/2616364164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb8143560 con 0x7f1fb8146790 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbc988640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 0x7f1fb8142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbd189640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b410 tx=0x7f1fa800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8015020 con 0x7f1fb8146790 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa8011830 con 0x7f1fb8146790 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb8143840 con 0x7f1fb8146790 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8011c20 con 0x7f1fb8146790 2024-04-26T21:12:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.944+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb8143e00 con 0x7f1fb8146790 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.945+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb806a290 con 0x7f1fb8146790 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.946+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fa8018020 con 0x7f1fb8146790 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.946+0000 7f1fb57fa640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 0x7f1fa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.946+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1fa808baa0 con 0x7f1fb8146790 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.946+0000 7f1fbc988640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 0x7f1fa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.947+0000 7f1fbc988640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 0x7f1fa406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00012e0 tx=0x7f1fb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:35.950+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fa801f020 con 0x7f1fb8146790 2024-04-26T21:12:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.146+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1fb80b0af0 con 0x7f1fb8146790 2024-04-26T21:12:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.147+0000 7f1fb57fa640 1 -- 172.21.15.40:0/2616364164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f1fa80172c0 con 0x7f1fb8146790 2024-04-26T21:12:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 msgr2=0x7f1fa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 0x7f1fa406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00012e0 tx=0x7f1fb0009040 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 msgr2=0x7f1fb815da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800b410 tx=0x7f1fa800b8d0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 shutdown_connections 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1fa406c920 0x7f1fa406ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb8146790 0x7f1fb815da70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb8140350 0x7f1fb80af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 --2- 172.21.15.40:0/2616364164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1fb813ff70 0x7f1fb8142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 >> 172.21.15.40:0/2616364164 conn(0x7f1fb80d0d30 msgr2=0x7f1fb815c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 shutdown_connections 2024-04-26T21:12:36.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.149+0000 7f1fbebec640 1 -- 172.21.15.40:0/2616364164 wait complete. 2024-04-26T21:12:36.161 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:12:36.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:36.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:12:36.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:12:36.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:36.187 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:12:36.253 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:36.253 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:36.271 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:36.272 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:36.330 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:36.330 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:36.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:36.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.488+0000 7f37e60d9640 1 Processor -- start 2024-04-26T21:12:36.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.488+0000 7f37e60d9640 1 -- start start 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37e60d9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 0x7f37e00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37e60d9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37e015f750 0x7f37e00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37e60d9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00d03a0 con 0x7f37e00d37d0 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37e60d9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37e00b4d10 con 0x7f37e015f750 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37e60d9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f37e00d0680 con 0x7f37e015f370 2024-04-26T21:12:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37deffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37e015f750 0x7f37e00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37dffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37e00d37d0 0x7f37e00d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52366/0 (socket says 172.21.15.40:52366) 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.489+0000 7f37dffff640 1 -- 172.21.15.40:0/2471009841 learned_addr learned my addr 172.21.15.40:0/2471009841 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37df7fe640 1 --2- 172.21.15.40:0/2471009841 >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 0x7f37e00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37deffd640 1 -- 172.21.15.40:0/2471009841 >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 msgr2=0x7f37e00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37deffd640 1 --2- 172.21.15.40:0/2471009841 >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 0x7f37e00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37deffd640 1 -- 172.21.15.40:0/2471009841 >> v1:172.21.15.40:6789/0 conn(0x7f37e00d37d0 legacy=0x7f37e00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37de7fc640 1 -- 172.21.15.40:0/2471009841 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1816775061 0 0) 0x7f37e00d03a0 con 0x7f37e00d37d0 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37deffd640 1 -- 172.21.15.40:0/2471009841 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00d0960 con 0x7f37e015f750 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37df7fe640 1 --2- 172.21.15.40:0/2471009841 >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 0x7f37e00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.490+0000 7f37deffd640 1 --2- 172.21.15.40:0/2471009841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e015f750 0x7f37e00b47d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37e00d2d10 tx=0x7f37d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b16ae52afc4a37f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37de7fc640 1 -- 172.21.15.40:0/2471009841 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4037070 con 0x7f37e015f750 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37de7fc640 1 -- 172.21.15.40:0/2471009841 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4032440 con 0x7f37e015f750 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37de7fc640 1 -- 172.21.15.40:0/2471009841 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4032810 con 0x7f37e015f750 2024-04-26T21:12:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 -- 172.21.15.40:0/2471009841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e015f750 msgr2=0x7f37e00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 --2- 172.21.15.40:0/2471009841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e015f750 0x7f37e00b47d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37e00d2d10 tx=0x7f37d402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 -- 172.21.15.40:0/2471009841 shutdown_connections 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 --2- 172.21.15.40:0/2471009841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e015f750 0x7f37e00b47d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 --2- 172.21.15.40:0/2471009841 >> v2:172.21.15.184:3300/0 conn(0x7f37e015f370 0x7f37e00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 -- 172.21.15.40:0/2471009841 >> 172.21.15.40:0/2471009841 conn(0x7f37e008ec10 msgr2=0x7f37e009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 -- 172.21.15.40:0/2471009841 shutdown_connections 2024-04-26T21:12:36.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.491+0000 7f37e60d9640 1 -- 172.21.15.40:0/2471009841 wait complete. 2024-04-26T21:12:36.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.492+0000 7f37e60d9640 1 Processor -- start 2024-04-26T21:12:36.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 -- start start 2024-04-26T21:12:36.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e015f370 0x7f37e00cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e015f750 0x7f37e00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00b61d0 con 0x7f37e00d37d0 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00d0f20 con 0x7f37e015f370 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37e60d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00b6350 con 0x7f37e015f750 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47638/0 (socket says 172.21.15.40:47638) 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.493+0000 7f37df7fe640 1 -- 172.21.15.40:0/3312104933 learned_addr learned my addr 172.21.15.40:0/3312104933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37deffd640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e015f370 0x7f37e00cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37dffff640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e015f750 0x7f37e00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 -- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e015f750 msgr2=0x7f37e00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e015f750 0x7f37e00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 -- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e015f370 msgr2=0x7f37e00cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e015f370 0x7f37e00cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 -- 172.21.15.40:0/3312104933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00ce000 con 0x7f37e00d37d0 2024-04-26T21:12:36.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37df7fe640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c7f0 tx=0x7f37d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0015070 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d0007a30 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e00ce2e0 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0007e00 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.494+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e00ce8a0 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.495+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e006a290 con 0x7f37e00d37d0 2024-04-26T21:12:36.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.496+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37d0018070 con 0x7f37e00d37d0 2024-04-26T21:12:36.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.496+0000 7f37c7fff640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 0x7f37cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.496+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f37d008ba80 con 0x7f37e00d37d0 2024-04-26T21:12:36.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.496+0000 7f37deffd640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 0x7f37cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.497+0000 7f37deffd640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 0x7f37cc06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37d4034000 tx=0x7f37d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.500+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37d0059f40 con 0x7f37e00d37d0 2024-04-26T21:12:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.691+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37e00bb0b0 con 0x7f37e00d37d0 2024-04-26T21:12:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.692+0000 7f37c7fff640 1 -- 172.21.15.40:0/3312104933 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f37d0059840 con 0x7f37e00d37d0 2024-04-26T21:12:36.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:36.693 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 msgr2=0x7f37cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 0x7f37cc06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37d4034000 tx=0x7f37d402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 msgr2=0x7f37e01f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c7f0 tx=0x7f37d000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 shutdown_connections 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f37cc06c920 0x7f37cc06ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37e015f750 0x7f37e00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e015f370 0x7f37e00cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 --2- 172.21.15.40:0/3312104933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37e00d37d0 0x7f37e01f4700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 >> 172.21.15.40:0/3312104933 conn(0x7f37e008ec10 msgr2=0x7f37e006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 shutdown_connections 2024-04-26T21:12:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.694+0000 7f37e60d9640 1 -- 172.21.15.40:0/3312104933 wait complete. 2024-04-26T21:12:36.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:36.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:36.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.809+0000 7fb81cd5f640 1 Processor -- start 2024-04-26T21:12:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.809+0000 7fb81cd5f640 1 -- start start 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.810+0000 7fb81cd5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8180b2420 0x7fb81809c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.810+0000 7fb81cd5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8180d1940 0x7fb8180702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.810+0000 7fb81cd5f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8180cee60 con 0x7fb81815d340 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.810+0000 7fb81cd5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8180ce9d0 con 0x7fb8180b2420 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.810+0000 7fb81cd5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8180d0ff0 con 0x7fb8180d1940 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8180b2420 0x7fb81809c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8180b2420 0x7fb81809c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47640/0 (socket says 172.21.15.40:47640) 2024-04-26T21:12:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 -- 172.21.15.40:0/3011320512 learned_addr learned my addr 172.21.15.40:0/3011320512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb81659c640 1 --1- 172.21.15.40:0/3011320512 >> v1:172.21.15.40:6789/0 conn(0x7fb81815d340 0x7fb8180d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52372/0 (socket says 172.21.15.40:52372) 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb816d9d640 1 --2- 172.21.15.40:0/3011320512 >> v2:172.21.15.184:3300/0 conn(0x7fb8180d1940 0x7fb8180702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 -- 172.21.15.40:0/3011320512 >> v2:172.21.15.184:3300/0 conn(0x7fb8180d1940 msgr2=0x7fb8180702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 --2- 172.21.15.40:0/3011320512 >> v2:172.21.15.184:3300/0 conn(0x7fb8180d1940 0x7fb8180702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 -- 172.21.15.40:0/3011320512 >> v1:172.21.15.40:6789/0 conn(0x7fb81815d340 legacy=0x7fb8180d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.811+0000 7fb815d9b640 1 -- 172.21.15.40:0/3011320512 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180d0d10 con 0x7fb8180b2420 2024-04-26T21:12:36.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb815d9b640 1 --2- 172.21.15.40:0/3011320512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180b2420 0x7fb81809c560 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8180746b0 tx=0x7fb80c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cbafa755c7460d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81559a640 1 -- 172.21.15.40:0/3011320512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c037070 con 0x7fb8180b2420 2024-04-26T21:12:36.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81559a640 1 -- 172.21.15.40:0/3011320512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c032440 con 0x7fb8180b2420 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81559a640 1 -- 172.21.15.40:0/3011320512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c032810 con 0x7fb8180b2420 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 -- 172.21.15.40:0/3011320512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180b2420 msgr2=0x7fb81809c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/3011320512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180b2420 0x7fb81809c560 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8180746b0 tx=0x7fb80c02a270 comp rx=0 tx=0).stop 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 -- 172.21.15.40:0/3011320512 shutdown_connections 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/3011320512 >> v2:172.21.15.184:3300/0 conn(0x7fb8180d1940 0x7fb8180702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/3011320512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180b2420 0x7fb81809c560 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 -- 172.21.15.40:0/3011320512 >> 172.21.15.40:0/3011320512 conn(0x7fb81809ba40 msgr2=0x7fb81809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 -- 172.21.15.40:0/3011320512 shutdown_connections 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.812+0000 7fb81cd5f640 1 -- 172.21.15.40:0/3011320512 wait complete. 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 Processor -- start 2024-04-26T21:12:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 -- start start 2024-04-26T21:12:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb81815d340 0x7fb81814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb81814b6f0 0x7fb818149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180b2090 con 0x7fb8180d1940 2024-04-26T21:12:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb81815d720 con 0x7fb81814b6f0 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81cd5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180b2210 con 0x7fb81815d340 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.814+0000 7fb81659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47650/0 (socket says 172.21.15.40:47650) 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 -- 172.21.15.40:0/265853999 learned_addr learned my addr 172.21.15.40:0/265853999 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb816d9d640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb81814b6f0 0x7fb818149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb815d9b640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb81815d340 0x7fb81814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 -- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb81815d340 msgr2=0x7fb81814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb81815d340 0x7fb81814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 -- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb81814b6f0 msgr2=0x7fb818149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb81814b6f0 0x7fb818149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 -- 172.21.15.40:0/265853999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181f6620 con 0x7fb8180d1940 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81659c640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c3c0 tx=0x7fb80800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8080074e0 con 0x7fb8180d1940 2024-04-26T21:12:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb808007680 con 0x7fb8180d1940 2024-04-26T21:12:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81814aad0 con 0x7fb8180d1940 2024-04-26T21:12:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808004a00 con 0x7fb8180d1940 2024-04-26T21:12:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.815+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8181e6160 con 0x7fb8180d1940 2024-04-26T21:12:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.816+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8180693d0 con 0x7fb8180d1940 2024-04-26T21:12:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.817+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb80801a020 con 0x7fb8180d1940 2024-04-26T21:12:36.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.817+0000 7fb806ffd640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:36.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.818+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb80808b280 con 0x7fb8180d1940 2024-04-26T21:12:36.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.818+0000 7fb815d9b640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:36.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.818+0000 7fb815d9b640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 0x7fb7f806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb80c034000 tx=0x7fb80c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:36.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:36.822+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8181f6620 con 0x7fb8180d1940 2024-04-26T21:12:37.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.005+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8181e5f30 con 0x7fb8180d1940 2024-04-26T21:12:37.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.005+0000 7fb806ffd640 1 -- 172.21.15.40:0/265853999 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fb808059740 con 0x7fb8180d1940 2024-04-26T21:12:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:37.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 msgr2=0x7fb7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 0x7fb7f806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb80c034000 tx=0x7fb80c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 msgr2=0x7fb8181f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c3c0 tx=0x7fb80800c880 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 shutdown_connections 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.007+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb81814b6f0 0x7fb818149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.008+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb81815d340 0x7fb81814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.008+0000 7fb81cd5f640 1 --2- 172.21.15.40:0/265853999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8180d1940 0x7fb8181f60e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.008+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 >> 172.21.15.40:0/265853999 conn(0x7fb81809ba40 msgr2=0x7fb8180c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.008+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 shutdown_connections 2024-04-26T21:12:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.008+0000 7fb81cd5f640 1 -- 172.21.15.40:0/265853999 wait complete. 2024-04-26T21:12:37.020 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.122+0000 7f643b1da640 1 Processor -- start 2024-04-26T21:12:37.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.122+0000 7f643b1da640 1 -- start start 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f643b1da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643415d340 0x7f643409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f643b1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64340d1940 0x7f64340702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f643b1da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64340cee60 con 0x7f64340b2420 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f643b1da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64340ce9d0 con 0x7f643415d340 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f643b1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64340d0ff0 con 0x7f64340d1940 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643415d340 0x7f643409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f643415d340 0x7f643409c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47652/0 (socket says 172.21.15.40:47652) 2024-04-26T21:12:37.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6433fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64340b2420 0x7f64340d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52386/0 (socket says 172.21.15.40:52386) 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 -- 172.21.15.40:0/3628676111 learned_addr learned my addr 172.21.15.40:0/3628676111 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6439777640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64340d1940 0x7f64340702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 -- 172.21.15.40:0/3628676111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64340d1940 msgr2=0x7f64340702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64340d1940 0x7f64340702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 -- 172.21.15.40:0/3628676111 >> v1:172.21.15.40:6789/0 conn(0x7f64340b2420 legacy=0x7f64340d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f6438f76640 1 -- 172.21.15.40:0/3628676111 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64340d0d10 con 0x7f643415d340 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.123+0000 7f64337fe640 1 -- 172.21.15.40:0/3628676111 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062766243 0 0) 0x7f64340cee60 con 0x7f64340b2420 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.124+0000 7f6438f76640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643415d340 0x7f643409c560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6428009870 tx=0x7f642802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2146aad7c7da4e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.124+0000 7f64337fe640 1 -- 172.21.15.40:0/3628676111 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428037070 con 0x7f643415d340 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.124+0000 7f64337fe640 1 -- 172.21.15.40:0/3628676111 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6428032440 con 0x7f643415d340 2024-04-26T21:12:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.124+0000 7f64337fe640 1 -- 172.21.15.40:0/3628676111 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428032810 con 0x7f643415d340 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 -- 172.21.15.40:0/3628676111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643415d340 msgr2=0x7f643409c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643415d340 0x7f643409c560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6428009870 tx=0x7f642802a270 comp rx=0 tx=0).stop 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 -- 172.21.15.40:0/3628676111 shutdown_connections 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64340d1940 0x7f64340702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 --2- 172.21.15.40:0/3628676111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f643415d340 0x7f643409c560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 -- 172.21.15.40:0/3628676111 >> 172.21.15.40:0/3628676111 conn(0x7f643409ba40 msgr2=0x7f643409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 -- 172.21.15.40:0/3628676111 shutdown_connections 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.125+0000 7f643b1da640 1 -- 172.21.15.40:0/3628676111 wait complete. 2024-04-26T21:12:37.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.126+0000 7f643b1da640 1 Processor -- start 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.126+0000 7f643b1da640 1 -- start start 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 0x7f643414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 0x7f64341575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64340b2090 con 0x7f6434157ae0 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f643415d720 con 0x7f64340d1940 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f64340b2210 con 0x7f643415d340 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47658/0 (socket says 172.21.15.40:47658) 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 -- 172.21.15.40:0/1464404778 learned_addr learned my addr 172.21.15.40:0/1464404778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6438f76640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 0x7f643414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6433fff640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 0x7f64341575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 -- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 msgr2=0x7f64341575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 0x7f64341575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 -- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 msgr2=0x7f643414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 0x7f643414c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 -- 172.21.15.40:0/1464404778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643414c980 con 0x7f6434157ae0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6438f76640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 0x7f643414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6433fff640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 0x7f64341575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f6439777640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f642c007e60 tx=0x7f642c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c014620 con 0x7f6434157ae0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c019070 con 0x7f6434157ae0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c014a00 con 0x7f6434157ae0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6434156500 con 0x7f6434157ae0 2024-04-26T21:12:37.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.127+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6434156ac0 con 0x7f6434157ae0 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.129+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64340693d0 con 0x7f6434157ae0 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.129+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f642c014ba0 con 0x7f6434157ae0 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.129+0000 7f64317fa640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 0x7f641806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.129+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f642c08c3e0 con 0x7f6434157ae0 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.129+0000 7f6438f76640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 0x7f641806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.130+0000 7f6438f76640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 0x7f641806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6428034000 tx=0x7f642802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.133+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f642c05a810 con 0x7f6434157ae0 2024-04-26T21:12:37.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.318+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6434070e50 con 0x7f6434157ae0 2024-04-26T21:12:37.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.318+0000 7f64317fa640 1 -- 172.21.15.40:0/1464404778 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f642c05a110 con 0x7f6434157ae0 2024-04-26T21:12:37.319 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.320+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 msgr2=0x7f641806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.320+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 0x7f641806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6428034000 tx=0x7f642802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.320+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 msgr2=0x7f643414ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.320+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f642c007e60 tx=0x7f642c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 shutdown_connections 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f641806c920 0x7f641806ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6434157ae0 0x7f643414ceb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f643415d340 0x7f64341575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 --2- 172.21.15.40:0/1464404778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64340d1940 0x7f643414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 >> 172.21.15.40:0/1464404778 conn(0x7f643409ba40 msgr2=0x7f6434071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 shutdown_connections 2024-04-26T21:12:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.321+0000 7f643b1da640 1 -- 172.21.15.40:0/1464404778 wait complete. 2024-04-26T21:12:37.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:37.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.435+0000 7fbfb9a6b640 1 Processor -- start 2024-04-26T21:12:37.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.435+0000 7fbfb9a6b640 1 -- start start 2024-04-26T21:12:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb9a6b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbfb415f5b0 0x7fbfb40b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb9a6b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbfb415f990 0x7fbfb40d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb9a6b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfb40b6aa0 con 0x7fbfb40b55b0 2024-04-26T21:12:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb9a6b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb40d3eb0 con 0x7fbfb415f5b0 2024-04-26T21:12:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb9a6b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb40d4190 con 0x7fbfb415f990 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbfb415f5b0 0x7fbfb40b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbfb415f5b0 0x7fbfb40b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47674/0 (socket says 172.21.15.40:47674) 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 -- 172.21.15.40:0/748536439 learned_addr learned my addr 172.21.15.40:0/748536439 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb2ffd640 1 --2- 172.21.15.40:0/748536439 >> v2:172.21.15.184:3300/0 conn(0x7fbfb415f990 0x7fbfb40d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 -- 172.21.15.40:0/748536439 >> v2:172.21.15.184:3300/0 conn(0x7fbfb415f990 msgr2=0x7fbfb40d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 --2- 172.21.15.40:0/748536439 >> v2:172.21.15.184:3300/0 conn(0x7fbfb415f990 0x7fbfb40d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 -- 172.21.15.40:0/748536439 >> v1:172.21.15.184:6789/0 conn(0x7fbfb40b55b0 legacy=0x7fbfb40d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.436+0000 7fbfb37fe640 1 -- 172.21.15.40:0/748536439 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb40d0200 con 0x7fbfb415f5b0 2024-04-26T21:12:37.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb37fe640 1 --2- 172.21.15.40:0/748536439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40b5070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbfb408ebd0 tx=0x7fbfa402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21422a4f78c74d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb27fc640 1 -- 172.21.15.40:0/748536439 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4037070 con 0x7fbfb415f5b0 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb27fc640 1 -- 172.21.15.40:0/748536439 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa4032440 con 0x7fbfb415f5b0 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb27fc640 1 -- 172.21.15.40:0/748536439 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4032810 con 0x7fbfb415f5b0 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/748536439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 msgr2=0x7fbfb40b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/748536439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40b5070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbfb408ebd0 tx=0x7fbfa402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/748536439 shutdown_connections 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/748536439 >> v2:172.21.15.184:3300/0 conn(0x7fbfb415f990 0x7fbfb40d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/748536439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40b5070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.437+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/748536439 >> 172.21.15.40:0/748536439 conn(0x7fbfb408ec40 msgr2=0x7fbfb409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.438+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/748536439 shutdown_connections 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.438+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/748536439 wait complete. 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.439+0000 7fbfb9a6b640 1 Processor -- start 2024-04-26T21:12:37.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.439+0000 7fbfb9a6b640 1 -- start start 2024-04-26T21:12:37.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb40b55b0 0x7fbfb4153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 0x7fbfb40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb40d3bd0 con 0x7fbfb415f5b0 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb40d3a50 con 0x7fbfb40b55b0 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb40d0f40 con 0x7fbfb415f990 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47686/0 (socket says 172.21.15.40:47686) 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 -- 172.21.15.40:0/3980703030 learned_addr learned my addr 172.21.15.40:0/3980703030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb37fe640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb40b55b0 0x7fbfb4153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb3fff640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 0x7fbfb40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 -- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 msgr2=0x7fbfb40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 0x7fbfb40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 -- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb40b55b0 msgr2=0x7fbfb4153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb40b55b0 0x7fbfb4153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 -- 172.21.15.40:0/3980703030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb4153570 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb3fff640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 0x7fbfb40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb2ffd640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e8a0 tx=0x7fbfa800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8017070 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa800c940 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb4153a40 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cd30 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.440+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb4160250 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.441+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb415aba0 con 0x7fbfb415f5b0 2024-04-26T21:12:37.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.442+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfa801a070 con 0x7fbfb415f5b0 2024-04-26T21:12:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.442+0000 7fbf9bfff640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 0x7fbfa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.442+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbfa808b980 con 0x7fbfb415f5b0 2024-04-26T21:12:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.442+0000 7fbfb37fe640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 0x7fbfa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.443+0000 7fbfb37fe640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 0x7fbfa006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4034000 tx=0x7fbfa402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.447+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbfa8059760 con 0x7fbfb415f5b0 2024-04-26T21:12:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.632+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfb406a290 con 0x7fbfb415f5b0 2024-04-26T21:12:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.633+0000 7fbf9bfff640 1 -- 172.21.15.40:0/3980703030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fbfa805da60 con 0x7fbfb415f5b0 2024-04-26T21:12:37.634 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 msgr2=0x7fbfa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 0x7fbfa006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4034000 tx=0x7fbfa402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 msgr2=0x7fbfb40ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e8a0 tx=0x7fbfa800ed60 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 shutdown_connections 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbfa006c8f0 0x7fbfa006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbfb415f990 0x7fbfb40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfb415f5b0 0x7fbfb40ce600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 --2- 172.21.15.40:0/3980703030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb40b55b0 0x7fbfb4153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 >> 172.21.15.40:0/3980703030 conn(0x7fbfb408ec40 msgr2=0x7fbfb40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.634+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 shutdown_connections 2024-04-26T21:12:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.635+0000 7fbfb9a6b640 1 -- 172.21.15.40:0/3980703030 wait complete. 2024-04-26T21:12:37.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:12:37.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.747+0000 7f34cde20640 1 Processor -- start 2024-04-26T21:12:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.747+0000 7f34cde20640 1 -- start start 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34cde20640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f34c815f720 0x7f34c80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34cde20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34cde20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c80b2510 con 0x7f34c815f340 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34cde20640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f34c80b27f0 con 0x7f34c815f720 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34cde20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80b2ad0 con 0x7f34c80d5610 2024-04-26T21:12:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f34c815f340 0x7f34c80d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52396/0 (socket says 172.21.15.40:52396) 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c77fe640 1 -- 172.21.15.40:0/3448266676 learned_addr learned my addr 172.21.15.40:0/3448266676 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/3448266676 >> v2:172.21.15.184:3300/0 conn(0x7f34c815f720 0x7f34c80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c7fff640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389511554 0 0) 0x7f34c80b2510 con 0x7f34c815f340 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.748+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b0003520 con 0x7f34c815f340 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2472176702 0 0) 0x7f34b0003520 con 0x7f34c815f340 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c6ffd640 1 -- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 msgr2=0x7f34c80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c6ffd640 1 -- 172.21.15.40:0/3448266676 >> v1:172.21.15.40:6789/0 conn(0x7f34c815f340 legacy=0x7f34c80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c6ffd640 1 -- 172.21.15.40:0/3448266676 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c80b2db0 con 0x7f34c815f720 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.749+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c80b1e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0097a0 tx=0x7f34bc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4379592799b447ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc037070 con 0x7f34c815f720 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc02ab10 con 0x7f34c815f720 2024-04-26T21:12:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34c67fc640 1 -- 172.21.15.40:0/3448266676 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8001670 con 0x7f34c815f720 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 -- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 msgr2=0x7f34c80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c80b1e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0097a0 tx=0x7f34bc02a080 comp rx=0 tx=0).stop 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 -- 172.21.15.40:0/3448266676 shutdown_connections 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 --2- 172.21.15.40:0/3448266676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c80b1e80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 -- 172.21.15.40:0/3448266676 >> 172.21.15.40:0/3448266676 conn(0x7f34c808ec70 msgr2=0x7f34c809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 -- 172.21.15.40:0/3448266676 shutdown_connections 2024-04-26T21:12:37.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.750+0000 7f34cde20640 1 -- 172.21.15.40:0/3448266676 wait complete. 2024-04-26T21:12:37.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.751+0000 7f34cde20640 1 Processor -- start 2024-04-26T21:12:37.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 -- start start 2024-04-26T21:12:37.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c8162640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80b3940 con 0x7f34c815f340 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80b37c0 con 0x7f34c815f720 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34cde20640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80b3ac0 con 0x7f34c80d5610 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34c6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34c6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47692/0 (socket says 172.21.15.40:47692) 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.752+0000 7f34c6ffd640 1 -- 172.21.15.40:0/944370447 learned_addr learned my addr 172.21.15.40:0/944370447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 -- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 msgr2=0x7f34c8162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c77fe640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c8162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c8162640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 -- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 msgr2=0x7f34c815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c7fff640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c815fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 -- 172.21.15.40:0/944370447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c8160510 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34c6ffd640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f34bc035040 tx=0x7f34bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc037050 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81607f0 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc043070 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc006e10 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.753+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c80d1c70 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.754+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c806a290 con 0x7f34c815f340 2024-04-26T21:12:37.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.754+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34bc03ca10 con 0x7f34c815f340 2024-04-26T21:12:37.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.755+0000 7f34affff640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:37.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.755+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f34bc0ac6f0 con 0x7f34c815f340 2024-04-26T21:12:37.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.755+0000 7f34c77fe640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:37.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.756+0000 7f34c77fe640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 0x7f34b406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34b8005880 tx=0x7f34b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.759+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34bc07abc0 con 0x7f34c815f340 2024-04-26T21:12:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.958+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34c80d3020 con 0x7f34c815f340 2024-04-26T21:12:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.959+0000 7f34affff640 1 -- 172.21.15.40:0/944370447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f34bc07a4c0 con 0x7f34c815f340 2024-04-26T21:12:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:12:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 msgr2=0x7f34b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 0x7f34b406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34b8005880 tx=0x7f34b8009040 comp rx=0 tx=0).stop 2024-04-26T21:12:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 msgr2=0x7f34c80d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f34bc035040 tx=0x7f34bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 shutdown_connections 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34c815f720 0x7f34c815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34c815f340 0x7f34c80d1730 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 --2- 172.21.15.40:0/944370447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34c80d5610 0x7f34c8162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 >> 172.21.15.40:0/944370447 conn(0x7f34c808ec70 msgr2=0x7f34c80d4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 shutdown_connections 2024-04-26T21:12:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:37.961+0000 7f34cde20640 1 -- 172.21.15.40:0/944370447 wait complete. 2024-04-26T21:12:37.973 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:12:37.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:37.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T21:12:38.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:12:38.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:38.004 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:12:38.167 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:38.167 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:38.183 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:38.184 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:38.240 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:12:38.240 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:38.296 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.437+0000 7f795ae2a640 1 Processor -- start 2024-04-26T21:12:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.437+0000 7f795ae2a640 1 -- start start 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f795ae2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f795415f720 0x7f79540b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f795ae2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f795ae2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79540b2510 con 0x7f795415f340 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f795ae2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f79540b27f0 con 0x7f795415f720 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f795ae2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79540b2ad0 con 0x7f79540d5610 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f7953fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f795415f720 0x7f79540b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f79593c7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f79593c7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58894/0 (socket says 172.21.15.40:58894) 2024-04-26T21:12:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.438+0000 7f79593c7640 1 -- 172.21.15.40:0/674059384 learned_addr learned my addr 172.21.15.40:0/674059384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.439+0000 7f79593c7640 1 -- 172.21.15.40:0/674059384 >> v2:172.21.15.184:3300/0 conn(0x7f795415f720 msgr2=0x7f79540b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.439+0000 7f79593c7640 1 --2- 172.21.15.40:0/674059384 >> v2:172.21.15.184:3300/0 conn(0x7f795415f720 0x7f79540b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.439+0000 7f79593c7640 1 -- 172.21.15.40:0/674059384 >> v1:172.21.15.184:6789/0 conn(0x7f795415f340 legacy=0x7f79540d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.439+0000 7f79593c7640 1 -- 172.21.15.40:0/674059384 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540b2db0 con 0x7f79540d5610 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f79593c7640 1 --2- 172.21.15.40:0/674059384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca7f85ffca9ba74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f79537fe640 1 -- 172.21.15.40:0/674059384 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c036070 con 0x7f79540d5610 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f79537fe640 1 -- 172.21.15.40:0/674059384 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c031600 con 0x7f79540d5610 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f79537fe640 1 -- 172.21.15.40:0/674059384 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0319d0 con 0x7f79540d5610 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 -- 172.21.15.40:0/674059384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 msgr2=0x7f79540d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 --2- 172.21.15.40:0/674059384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c02a680 comp rx=0 tx=0).stop 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 -- 172.21.15.40:0/674059384 shutdown_connections 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 --2- 172.21.15.40:0/674059384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540d3d40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 --2- 172.21.15.40:0/674059384 >> v2:172.21.15.184:3300/0 conn(0x7f795415f720 0x7f79540b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 -- 172.21.15.40:0/674059384 >> 172.21.15.40:0/674059384 conn(0x7f795408ec70 msgr2=0x7f795409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 -- 172.21.15.40:0/674059384 shutdown_connections 2024-04-26T21:12:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.440+0000 7f795ae2a640 1 -- 172.21.15.40:0/674059384 wait complete. 2024-04-26T21:12:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 Processor -- start 2024-04-26T21:12:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 -- start start 2024-04-26T21:12:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795415f340 0x7f79541586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79540b3940 con 0x7f795415f720 2024-04-26T21:12:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79540b37c0 con 0x7f795415f340 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.442+0000 7f795ae2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79540b3ac0 con 0x7f79540d5610 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47706/0 (socket says 172.21.15.40:47706) 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 -- 172.21.15.40:0/1105299063 learned_addr learned my addr 172.21.15.40:0/1105299063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f7953fff640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795415f340 0x7f79541586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f7958bc6640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 -- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 msgr2=0x7f79540b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 -- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795415f340 msgr2=0x7f79541586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795415f340 0x7f79541586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 -- 172.21.15.40:0/1105299063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540c8e20 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79593c7640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f794c034040 tx=0x7f794c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c036050 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c042070 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79540c9100 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c009510 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.443+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79540b8380 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.444+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795406a290 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.445+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794c007c70 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.445+0000 7f79517fa640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 0x7f794006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.445+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f794c0ac820 con 0x7f795415f720 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.445+0000 7f7958bc6640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 0x7f794006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.446+0000 7f7958bc6640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 0x7f794006ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7944001780 tx=0x7f7944009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.449+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f794c07a600 con 0x7f795415f720 2024-04-26T21:12:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.638+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79540b7a30 con 0x7f795415f720 2024-04-26T21:12:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.639+0000 7f79517fa640 1 -- 172.21.15.40:0/1105299063 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7f794c07e900 con 0x7f795415f720 2024-04-26T21:12:38.641 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:38.641 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.641+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 msgr2=0x7f794006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.641+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 0x7f794006ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7944001780 tx=0x7f7944009040 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.641+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 msgr2=0x7f79540cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.641+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f794c034040 tx=0x7f794c004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 shutdown_connections 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f794006c920 0x7f794006ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795415f720 0x7f79540cde70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795415f340 0x7f79541586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 --2- 172.21.15.40:0/1105299063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79540d5610 0x7f79540b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 >> 172.21.15.40:0/1105299063 conn(0x7f795408ec70 msgr2=0x7f79540b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 shutdown_connections 2024-04-26T21:12:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.642+0000 7f795ae2a640 1 -- 172.21.15.40:0/1105299063 wait complete. 2024-04-26T21:12:38.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:38.655 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.757+0000 7f0e00e7b640 1 Processor -- start 2024-04-26T21:12:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.757+0000 7f0e00e7b640 1 -- start start 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.758+0000 7f0e00e7b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.758+0000 7f0e00e7b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 0x7f0dfc0d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.758+0000 7f0e00e7b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc0d3c90 con 0x7f0dfc15f680 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.758+0000 7f0e00e7b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0dfc15fa60 con 0x7f0dfc0d3820 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.758+0000 7f0e00e7b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc0d3f50 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0df9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0df9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:38524/0 (socket says 172.21.15.40:38524) 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0dfad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 0x7f0dfc0d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0df9d9b640 1 -- 172.21.15.40:0/2748164279 learned_addr learned my addr 172.21.15.40:0/2748164279 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0dfad9d640 1 -- 172.21.15.40:0/2748164279 >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 msgr2=0x7f0dfc0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0dfad9d640 1 --2- 172.21.15.40:0/2748164279 >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0dfad9d640 1 -- 172.21.15.40:0/2748164279 >> v1:172.21.15.184:6789/0 conn(0x7f0dfc15f680 legacy=0x7f0dfc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0dfad9d640 1 -- 172.21.15.40:0/2748164279 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0b5090 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.759+0000 7f0df9d9b640 1 --2- 172.21.15.40:0/2748164279 >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.760+0000 7f0dfad9d640 1 --2- 172.21.15.40:0/2748164279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 0x7f0dfc0d4f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009d90 tx=0x7f0dec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804e2701148f1871 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.760+0000 7f0df959a640 1 -- 172.21.15.40:0/2748164279 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec007740 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.760+0000 7f0df959a640 1 -- 172.21.15.40:0/2748164279 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec0078e0 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.760+0000 7f0df959a640 1 -- 172.21.15.40:0/2748164279 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec007cb0 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 -- 172.21.15.40:0/2748164279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 msgr2=0x7f0dfc0d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/2748164279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 0x7f0dfc0d4f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009d90 tx=0x7f0dec02a660 comp rx=0 tx=0).stop 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 -- 172.21.15.40:0/2748164279 shutdown_connections 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/2748164279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc0b60b0 0x7f0dfc0d4f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/2748164279 >> v2:172.21.15.184:3300/0 conn(0x7f0dfc0d3820 0x7f0dfc0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 -- 172.21.15.40:0/2748164279 >> 172.21.15.40:0/2748164279 conn(0x7f0dfc08eca0 msgr2=0x7f0dfc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 -- 172.21.15.40:0/2748164279 shutdown_connections 2024-04-26T21:12:38.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.761+0000 7f0e00e7b640 1 -- 172.21.15.40:0/2748164279 wait complete. 2024-04-26T21:12:38.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.762+0000 7f0e00e7b640 1 Processor -- start 2024-04-26T21:12:38.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.762+0000 7f0e00e7b640 1 -- start start 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 0x7f0dfc162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 0x7f0dfc0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0d55e0 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc0d5460 con 0x7f0dfc0d3820 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0e00e7b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc0d5760 con 0x7f0dfc15f680 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47710/0 (socket says 172.21.15.40:47710) 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 -- 172.21.15.40:0/406817012 learned_addr learned my addr 172.21.15.40:0/406817012 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0df9d9b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 0x7f0dfc162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfad9d640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 0x7f0dfc0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 -- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 msgr2=0x7f0dfc0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 0x7f0dfc0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 -- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 msgr2=0x7f0dfc162520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 0x7f0dfc162520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 -- 172.21.15.40:0/406817012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0d0750 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0df9d9b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 0x7f0dfc162520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfad9d640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 0x7f0dfc0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0dfa59c640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0df400c340 tx=0x7f0df400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.763+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4017070 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.764+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df40052d0 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.764+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc0cc340 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.764+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4003850 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.764+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc0cc900 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.765+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc06a290 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.765+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0df401a070 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.765+0000 7f0deaffd640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 0x7f0ddc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.765+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0df408ae20 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.766+0000 7f0df9d9b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 0x7f0ddc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.766+0000 7f0df9d9b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 0x7f0ddc06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009630 tx=0x7f0df0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:38.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.770+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0df4059250 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.953+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0dfc0b4cd0 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.953+0000 7f0deaffd640 1 -- 172.21.15.40:0/406817012 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f0df4058b50 con 0x7f0dfc0b60b0 2024-04-26T21:12:38.955 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 msgr2=0x7f0ddc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 0x7f0ddc06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009630 tx=0x7f0df0008040 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 msgr2=0x7f0dfc153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0df400c340 tx=0x7f0df400c800 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 shutdown_connections 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ddc06c920 0x7f0ddc06ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0dfc15f680 0x7f0dfc0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dfc0d3820 0x7f0dfc162520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 --2- 172.21.15.40:0/406817012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dfc0b60b0 0x7f0dfc153350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 >> 172.21.15.40:0/406817012 conn(0x7f0dfc08eca0 msgr2=0x7f0dfc0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 shutdown_connections 2024-04-26T21:12:38.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:38.955+0000 7f0e00e7b640 1 -- 172.21.15.40:0/406817012 wait complete. 2024-04-26T21:12:38.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.070+0000 7f6d873c9640 1 Processor -- start 2024-04-26T21:12:39.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 -- start start 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d8015c4c0 0x7f6d8006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d8006d8f0 0x7f6d8015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8006f730 con 0x7f6d8015c0e0 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6d80160aa0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.071+0000 7f6d873c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8006dd20 con 0x7f6d8006d8f0 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d8015c4c0 0x7f6d8006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d8015c4c0 0x7f6d8006d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47718/0 (socket says 172.21.15.40:47718) 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 -- 172.21.15.40:0/2068905748 learned_addr learned my addr 172.21.15.40:0/2068905748 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d85966640 1 --2- 172.21.15.40:0/2068905748 >> v2:172.21.15.184:3300/0 conn(0x7f6d8006d8f0 0x7f6d8015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 -- 172.21.15.40:0/2068905748 >> v2:172.21.15.184:3300/0 conn(0x7f6d8006d8f0 msgr2=0x7f6d8015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 --2- 172.21.15.40:0/2068905748 >> v2:172.21.15.184:3300/0 conn(0x7f6d8006d8f0 0x7f6d8015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 -- 172.21.15.40:0/2068905748 >> v1:172.21.15.184:6789/0 conn(0x7f6d8015c0e0 legacy=0x7f6d8006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 -- 172.21.15.40:0/2068905748 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800cff00 con 0x7f6d8015c4c0 2024-04-26T21:12:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.072+0000 7f6d84964640 1 --2- 172.21.15.40:0/2068905748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d8006d3b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d80073020 tx=0x7f6d7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1c2f7c98752b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d77fff640 1 -- 172.21.15.40:0/2068905748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70037070 con 0x7f6d8015c4c0 2024-04-26T21:12:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d77fff640 1 -- 172.21.15.40:0/2068905748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d70032440 con 0x7f6d8015c4c0 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d77fff640 1 -- 172.21.15.40:0/2068905748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70032810 con 0x7f6d8015c4c0 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 -- 172.21.15.40:0/2068905748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 msgr2=0x7f6d8006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 --2- 172.21.15.40:0/2068905748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d8006d3b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d80073020 tx=0x7f6d7002a230 comp rx=0 tx=0).stop 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 -- 172.21.15.40:0/2068905748 shutdown_connections 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 --2- 172.21.15.40:0/2068905748 >> v2:172.21.15.184:3300/0 conn(0x7f6d8006d8f0 0x7f6d8015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 --2- 172.21.15.40:0/2068905748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d8006d3b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 -- 172.21.15.40:0/2068905748 >> 172.21.15.40:0/2068905748 conn(0x7f6d800d1e40 msgr2=0x7f6d800d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 -- 172.21.15.40:0/2068905748 shutdown_connections 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.073+0000 7f6d873c9640 1 -- 172.21.15.40:0/2068905748 wait complete. 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.074+0000 7f6d873c9640 1 Processor -- start 2024-04-26T21:12:39.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.074+0000 7f6d873c9640 1 -- start start 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d8006d8f0 0x7f6d800cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 0x7f6d800b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80160450 con 0x7f6d8015c4c0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80160170 con 0x7f6d8006d8f0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d873c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d80160730 con 0x7f6d8015c0e0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d85966640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d85966640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47734/0 (socket says 172.21.15.40:47734) 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.075+0000 7f6d85966640 1 -- 172.21.15.40:0/3811921546 learned_addr learned my addr 172.21.15.40:0/3811921546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d84964640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 0x7f6d800b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85165640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d8006d8f0 0x7f6d800cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 -- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 msgr2=0x7f6d800b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 0x7f6d800b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 -- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d8006d8f0 msgr2=0x7f6d800cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d8006d8f0 0x7f6d800cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 -- 172.21.15.40:0/3811921546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800c57e0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d84964640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 0x7f6d800b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d85966640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007e60 tx=0x7f6d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78014420 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d78019070 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8015a670 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78014820 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.076+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d80151a70 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.077+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d80070440 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.077+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d780149c0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.078+0000 7f6d75ffb640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 0x7f6d6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.078+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6d7808c2f0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.078+0000 7f6d85165640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 0x7f6d6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.079+0000 7f6d85165640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 0x7f6d6406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003d90 tx=0x7f6d7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.082+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d80070440 con 0x7f6d8015c4c0 2024-04-26T21:12:39.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.268+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d800693d0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.268+0000 7f6d75ffb640 1 -- 172.21.15.40:0/3811921546 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f6d7805a7b0 con 0x7f6d8015c4c0 2024-04-26T21:12:39.269 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:39.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 msgr2=0x7f6d6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 0x7f6d6406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c003d90 tx=0x7f6d7c009040 comp rx=0 tx=0).stop 2024-04-26T21:12:39.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 msgr2=0x7f6d800b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007e60 tx=0x7f6d7800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 shutdown_connections 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d6406c920 0x7f6d6406ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d8015c4c0 0x7f6d800b6d30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d8015c0e0 0x7f6d800b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 --2- 172.21.15.40:0/3811921546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d8006d8f0 0x7f6d800cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 >> 172.21.15.40:0/3811921546 conn(0x7f6d800d1e40 msgr2=0x7f6d8006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 shutdown_connections 2024-04-26T21:12:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.270+0000 7f6d873c9640 1 -- 172.21.15.40:0/3811921546 wait complete. 2024-04-26T21:12:39.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.384+0000 7f3907904640 1 Processor -- start 2024-04-26T21:12:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.384+0000 7f3907904640 1 -- start start 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3907904640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39000d0010 0x7f390006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3907904640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 0x7f3900163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3907904640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390009c700 con 0x7f39000b3c20 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3907904640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f39000b4480 con 0x7f39000d0010 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3907904640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f390006e050 con 0x7f39000b33e0 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39000d0010 0x7f390006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3904e9f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f39000b3c20 0x7f39000b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52408/0 (socket says 172.21.15.40:52408) 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39000d0010 0x7f390006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47746/0 (socket says 172.21.15.40:47746) 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3904e9f640 1 -- 172.21.15.40:0/4201361792 learned_addr learned my addr 172.21.15.40:0/4201361792 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3905ea1640 1 --2- 172.21.15.40:0/4201361792 >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 0x7f3900163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 -- 172.21.15.40:0/4201361792 >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 msgr2=0x7f3900163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 --2- 172.21.15.40:0/4201361792 >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 0x7f3900163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 -- 172.21.15.40:0/4201361792 >> v1:172.21.15.40:6789/0 conn(0x7f39000b3c20 legacy=0x7f39000b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f39056a0640 1 -- 172.21.15.40:0/4201361792 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39000d1c50 con 0x7f39000d0010 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f38f7fff640 1 -- 172.21.15.40:0/4201361792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664338500 0 0) 0x7f390009c700 con 0x7f39000b3c20 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.385+0000 7f3905ea1640 1 --2- 172.21.15.40:0/4201361792 >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 0x7f3900163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.386+0000 7f39056a0640 1 --2- 172.21.15.40:0/4201361792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000d0010 0x7f390006db10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3900073bb0 tx=0x7f38fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de54fa6840b75041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.386+0000 7f38f7fff640 1 -- 172.21.15.40:0/4201361792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc037070 con 0x7f39000d0010 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.386+0000 7f38f7fff640 1 -- 172.21.15.40:0/4201361792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc032440 con 0x7f39000d0010 2024-04-26T21:12:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.386+0000 7f38f7fff640 1 -- 172.21.15.40:0/4201361792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc032810 con 0x7f39000d0010 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 -- 172.21.15.40:0/4201361792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000d0010 msgr2=0x7f390006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 --2- 172.21.15.40:0/4201361792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000d0010 0x7f390006db10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3900073bb0 tx=0x7f38fc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 -- 172.21.15.40:0/4201361792 shutdown_connections 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 --2- 172.21.15.40:0/4201361792 >> v2:172.21.15.184:3300/0 conn(0x7f39000b33e0 0x7f3900163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 --2- 172.21.15.40:0/4201361792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000d0010 0x7f390006db10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 -- 172.21.15.40:0/4201361792 >> 172.21.15.40:0/4201361792 conn(0x7f390009ba40 msgr2=0x7f390009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 -- 172.21.15.40:0/4201361792 shutdown_connections 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.387+0000 7f3907904640 1 -- 172.21.15.40:0/4201361792 wait complete. 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.388+0000 7f3907904640 1 Processor -- start 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.388+0000 7f3907904640 1 -- start start 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 0x7f39000a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39000d0010 0x7f39000b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39000b3990 con 0x7f39000b3c20 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39000b3810 con 0x7f39000d0010 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3907904640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39001637c0 con 0x7f39000b33e0 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47760/0 (socket says 172.21.15.40:47760) 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 -- 172.21.15.40:0/408888693 learned_addr learned my addr 172.21.15.40:0/408888693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3905ea1640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39000d0010 0x7f39000b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f39056a0640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 0x7f39000a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 -- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 msgr2=0x7f39000a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 0x7f39000a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 -- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39000d0010 msgr2=0x7f39000b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39000d0010 0x7f39000b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 -- 172.21.15.40:0/408888693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39000b4680 con 0x7f39000b3c20 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f39056a0640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 0x7f39000a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f3904e9f640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c4d0 tx=0x7f38f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0018070 con 0x7f39000b3c20 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.389+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f0007500 con 0x7f39000b3c20 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.390+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3900151160 con 0x7f39000b3c20 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.390+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0004a20 con 0x7f39000b3c20 2024-04-26T21:12:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.390+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001519a0 con 0x7f39000b3c20 2024-04-26T21:12:39.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.391+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390006a290 con 0x7f39000b3c20 2024-04-26T21:12:39.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.391+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38f001b070 con 0x7f39000b3c20 2024-04-26T21:12:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.391+0000 7f38f5ffb640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 0x7f38e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.392+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f38f008ae40 con 0x7f39000b3c20 2024-04-26T21:12:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.392+0000 7f39056a0640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 0x7f38e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.392+0000 7f39056a0640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 0x7f38e406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38fc034000 tx=0x7f38fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.396+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3900151160 con 0x7f39000b3c20 2024-04-26T21:12:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.581+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3900068e00 con 0x7f39000b3c20 2024-04-26T21:12:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.582+0000 7f38f5ffb640 1 -- 172.21.15.40:0/408888693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f38f0059270 con 0x7f39000b3c20 2024-04-26T21:12:39.583 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 msgr2=0x7f38e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 0x7f38e406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38fc034000 tx=0x7f38fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 msgr2=0x7f39000d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c4d0 tx=0x7f38f000c990 comp rx=0 tx=0).stop 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 shutdown_connections 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38e406c920 0x7f38e406ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39000d0010 0x7f39000b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39000b3c20 0x7f39000d1100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 --2- 172.21.15.40:0/408888693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39000b33e0 0x7f39000a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 >> 172.21.15.40:0/408888693 conn(0x7f390009ba40 msgr2=0x7f39000afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 shutdown_connections 2024-04-26T21:12:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.584+0000 7f3907904640 1 -- 172.21.15.40:0/408888693 wait complete. 2024-04-26T21:12:39.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:12:39.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.696+0000 7f775159e640 1 Processor -- start 2024-04-26T21:12:39.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.696+0000 7f775159e640 1 -- start start 2024-04-26T21:12:39.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f775159e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162780 0x7f774c0b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f775159e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f774c0d57e0 0x7f774c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f775159e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c0b6840 con 0x7f774c162b60 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f775159e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f774c0d3b30 con 0x7f774c0d57e0 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f775159e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0b6b20 con 0x7f774c162780 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f774c0d57e0 0x7f774c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f774c0d57e0 0x7f774c0b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47762/0 (socket says 172.21.15.40:47762) 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 -- 172.21.15.40:0/19940468 learned_addr learned my addr 172.21.15.40:0/19940468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f774bfff640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162780 0x7f774c0b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 -- 172.21.15.40:0/19940468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162780 msgr2=0x7f774c0b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162780 0x7f774c0b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 -- 172.21.15.40:0/19940468 >> v1:172.21.15.40:6789/0 conn(0x7f774c162b60 legacy=0x7f774c0d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f7750d9d640 1 -- 172.21.15.40:0/19940468 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0b1f70 con 0x7f774c0d57e0 2024-04-26T21:12:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.697+0000 7f774affd640 1 -- 172.21.15.40:0/19940468 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138096961 0 0) 0x7f774c0b6840 con 0x7f774c162b60 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.698+0000 7f7750d9d640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c0d57e0 0x7f774c0b6130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7744009e50 tx=0x7f774402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc93bbe69c19913 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.698+0000 7f774affd640 1 -- 172.21.15.40:0/19940468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744036070 con 0x7f774c0d57e0 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.698+0000 7f774affd640 1 -- 172.21.15.40:0/19940468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7744031600 con 0x7f774c0d57e0 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.698+0000 7f774affd640 1 -- 172.21.15.40:0/19940468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440319d0 con 0x7f774c0d57e0 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 -- 172.21.15.40:0/19940468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c0d57e0 msgr2=0x7f774c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c0d57e0 0x7f774c0b6130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7744009e50 tx=0x7f774402a710 comp rx=0 tx=0).stop 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 -- 172.21.15.40:0/19940468 shutdown_connections 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c0d57e0 0x7f774c0b6130 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 --2- 172.21.15.40:0/19940468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162780 0x7f774c0b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 -- 172.21.15.40:0/19940468 >> 172.21.15.40:0/19940468 conn(0x7f774c08ed60 msgr2=0x7f774c09e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 -- 172.21.15.40:0/19940468 shutdown_connections 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.699+0000 7f775159e640 1 -- 172.21.15.40:0/19940468 wait complete. 2024-04-26T21:12:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.700+0000 7f775159e640 1 Processor -- start 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.700+0000 7f775159e640 1 -- start start 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774c0d57e0 0x7f774c0d2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162b60 0x7f774c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0b2a50 con 0x7f774c162780 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0b2530 con 0x7f774c0d57e0 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f775159e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0b2bd0 con 0x7f774c162b60 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47772/0 (socket says 172.21.15.40:47772) 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 -- 172.21.15.40:0/4133885762 learned_addr learned my addr 172.21.15.40:0/4133885762 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f7750d9d640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162b60 0x7f774c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774bfff640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774c0d57e0 0x7f774c0d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 -- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162b60 msgr2=0x7f774c14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162b60 0x7f774c14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 -- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774c0d57e0 msgr2=0x7f774c0d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774c0d57e0 0x7f774c0d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.701+0000 7f774b7fe640 1 -- 172.21.15.40:0/4133885762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0c95e0 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f774b7fe640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c440 tx=0x7f773c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017070 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773c005480 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c0c98c0 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c003830 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.702+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c0c9e80 con 0x7f774c162780 2024-04-26T21:12:39.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.703+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c0a32f0 con 0x7f774c162780 2024-04-26T21:12:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.703+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f773c01a070 con 0x7f774c162780 2024-04-26T21:12:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.704+0000 7f7748ff9640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 0x7f772c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.704+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f773c08ada0 con 0x7f774c162780 2024-04-26T21:12:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.704+0000 7f774bfff640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 0x7f772c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.705+0000 7f774bfff640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 0x7f772c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7734006fd0 tx=0x7f7734008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.708+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f773c058b80 con 0x7f774c162780 2024-04-26T21:12:39.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.907+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f774c0507a0 con 0x7f774c162780 2024-04-26T21:12:39.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.907+0000 7f7748ff9640 1 -- 172.21.15.40:0/4133885762 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f773c05ce80 con 0x7f774c162780 2024-04-26T21:12:39.908 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 msgr2=0x7f772c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 0x7f772c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7734006fd0 tx=0x7f7734008040 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 msgr2=0x7f774c0c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c440 tx=0x7f773c00c900 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 shutdown_connections 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772c06c8f0 0x7f772c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f774c162b60 0x7f774c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774c162780 0x7f774c0c8e70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 --2- 172.21.15.40:0/4133885762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f774c0d57e0 0x7f774c0d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 >> 172.21.15.40:0/4133885762 conn(0x7f774c08ed60 msgr2=0x7f774c06e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 shutdown_connections 2024-04-26T21:12:39.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:39.909+0000 7f775159e640 1 -- 172.21.15.40:0/4133885762 wait complete. 2024-04-26T21:12:39.921 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T21:12:39.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:39.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T21:12:39.947 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:12:39.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:39.947 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T21:12:40.003 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:40.003 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:40.022 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:40.022 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:40.081 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:12:40.081 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:12:40.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:40.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.239+0000 7fd158d6a640 1 Processor -- start 2024-04-26T21:12:40.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd158d6a640 1 -- start start 2024-04-26T21:12:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd158d6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1540cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd158d6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1540d03a0 con 0x7fd15415f370 2024-04-26T21:12:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd158d6a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1540d0680 con 0x7fd15415f750 2024-04-26T21:12:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd158d6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1540d0960 con 0x7fd1540d3ee0 2024-04-26T21:12:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd15259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd15415f370 0x7fd1540d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52430/0 (socket says 172.21.15.40:52430) 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.240+0000 7fd15259c640 1 -- 172.21.15.40:0/556837039 learned_addr learned my addr 172.21.15.40:0/556837039 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd14bfff640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1540cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922933971 0 0) 0x7fd1540d03a0 con 0x7fd15415f370 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd140003520 con 0x7fd15415f370 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1739264426 0 0) 0x7fd140003520 con 0x7fd15415f370 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 msgr2=0x7fd1540cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1540cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 >> v1:172.21.15.184:6789/0 conn(0x7fd15415f750 legacy=0x7fd1540b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1540d0c40 con 0x7fd15415f370 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd144001670 con 0x7fd15415f370 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 >> v1:172.21.15.40:6789/0 conn(0x7fd15415f370 legacy=0x7fd1540d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1400039e0 0x7fd140003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd140004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1400050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540d0960 con 0x7fd140004390 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd14000b9c0 con 0x7fd1400039e0 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd14000bbc0 con 0x7fd1540d3ee0 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd14bfff640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd140004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd15259c640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1400039e0 0x7fd140003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.241+0000 7fd152d9d640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1400050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 -- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 msgr2=0x7fd1400050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1400050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 -- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1400039e0 msgr2=0x7fd140003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1400039e0 0x7fd140003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 -- 172.21.15.40:0/556837039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd140003520 con 0x7fd140004390 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd152d9d640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1540d3ee0 0x7fd1400050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd14bfff640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd140004860 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ac0 tx=0x7fd140005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327f7c7b558de54c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c036070 con 0x7fd140004390 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c031400 con 0x7fd140004390 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.242+0000 7fd151d9b640 1 -- 172.21.15.40:0/556837039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c009690 con 0x7fd140004390 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 -- 172.21.15.40:0/556837039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 msgr2=0x7fd140004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 --2- 172.21.15.40:0/556837039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd140004860 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ac0 tx=0x7fd140005830 comp rx=0 tx=0).stop 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd152d9d640 1 -- 172.21.15.40:0/556837039 reap_dead start 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 -- 172.21.15.40:0/556837039 shutdown_connections 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 -- 172.21.15.40:0/556837039 >> 172.21.15.40:0/556837039 conn(0x7fd15408ec10 msgr2=0x7fd15409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 -- 172.21.15.40:0/556837039 shutdown_connections 2024-04-26T21:12:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.243+0000 7fd158d6a640 1 -- 172.21.15.40:0/556837039 wait complete. 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.244+0000 7fd158d6a640 1 Processor -- start 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.244+0000 7fd158d6a640 1 -- start start 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1541625f0 0x7fd1541f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1540cd350 0x7fd15406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540cfef0 con 0x7fd140004390 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540cfd70 con 0x7fd1540cd350 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd158d6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1540d0070 con 0x7fd1541625f0 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47788/0 (socket says 172.21.15.40:47788) 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 -- 172.21.15.40:0/3715342888 learned_addr learned my addr 172.21.15.40:0/3715342888 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd14bfff640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1541625f0 0x7fd1541f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd152d9d640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1540cd350 0x7fd15406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 -- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1541625f0 msgr2=0x7fd1541f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1541625f0 0x7fd1541f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 -- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1540cd350 msgr2=0x7fd15406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1540cd350 0x7fd15406cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.245+0000 7fd15259c640 1 -- 172.21.15.40:0/3715342888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1541f5970 con 0x7fd140004390 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd15259c640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd1440072c0 tx=0x7fd144007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14401a070 con 0x7fd140004390 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14400f960 con 0x7fd140004390 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14400fd10 con 0x7fd140004390 2024-04-26T21:12:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1541f5c50 con 0x7fd140004390 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.246+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541f5f30 con 0x7fd140004390 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.247+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15406a290 con 0x7fd140004390 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.247+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd14401d070 con 0x7fd140004390 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.248+0000 7fd14affd640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 0x7fd13406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.248+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd14408b620 con 0x7fd140004390 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.248+0000 7fd14bfff640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 0x7fd13406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.248+0000 7fd14bfff640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 0x7fd13406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ac0 tx=0x7fd13c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.252+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd144059a50 con 0x7fd140004390 2024-04-26T21:12:40.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.439+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1540d5440 con 0x7fd140004390 2024-04-26T21:12:40.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.440+0000 7fd14affd640 1 -- 172.21.15.40:0/3715342888 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16605 (secure 0 0 0) 0x7fd144059350 con 0x7fd140004390 2024-04-26T21:12:40.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:40.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:02.753655+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 msgr2=0x7fd13406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 0x7fd13406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ac0 tx=0x7fd13c008030 comp rx=0 tx=0).stop 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 msgr2=0x7fd15415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd1440072c0 tx=0x7fd144007780 comp rx=0 tx=0).stop 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 shutdown_connections 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd13406c8f0 0x7fd13406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1540cd350 0x7fd15406cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1541625f0 0x7fd1541f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 --2- 172.21.15.40:0/3715342888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd140004390 0x7fd15415b810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 >> 172.21.15.40:0/3715342888 conn(0x7fd15408ec10 msgr2=0x7fd1540b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:40.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 shutdown_connections 2024-04-26T21:12:40.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.442+0000 7fd158d6a640 1 -- 172.21.15.40:0/3715342888 wait complete. 2024-04-26T21:12:40.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:40.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.557+0000 7f7241ec8640 1 Processor -- start 2024-04-26T21:12:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.557+0000 7f7241ec8640 1 -- start start 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f7241ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15f2b0 0x7f723c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f7241ec8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f723c06c9b0 0x7f723c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f7241ec8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c070b30 con 0x7f723c0706a0 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f7241ec8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f723c0af610 con 0x7f723c06c9b0 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f7241ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0700b0 con 0x7f723c15f2b0 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f723c06c9b0 0x7f723c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f723c06c9b0 0x7f723c0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47794/0 (socket says 172.21.15.40:47794) 2024-04-26T21:12:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 -- 172.21.15.40:0/3239579077 learned_addr learned my addr 172.21.15.40:0/3239579077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723b7fe640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15f2b0 0x7f723c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 -- 172.21.15.40:0/3239579077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15f2b0 msgr2=0x7f723c06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15f2b0 0x7f723c06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 -- 172.21.15.40:0/3239579077 >> v1:172.21.15.184:6789/0 conn(0x7f723c0706a0 legacy=0x7f723c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.558+0000 7f723bfff640 1 -- 172.21.15.40:0/3239579077 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c070e10 con 0x7f723c06c9b0 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.559+0000 7f723bfff640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c0afa80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e50 tx=0x7f723402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b828e20e2832c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.559+0000 7f723a7fc640 1 -- 172.21.15.40:0/3239579077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234036070 con 0x7f723c06c9b0 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.559+0000 7f723a7fc640 1 -- 172.21.15.40:0/3239579077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7234031600 con 0x7f723c06c9b0 2024-04-26T21:12:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.559+0000 7f723a7fc640 1 -- 172.21.15.40:0/3239579077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340319d0 con 0x7f723c06c9b0 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 -- 172.21.15.40:0/3239579077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 msgr2=0x7f723c0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c0afa80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e50 tx=0x7f723402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 -- 172.21.15.40:0/3239579077 shutdown_connections 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c0afa80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 --2- 172.21.15.40:0/3239579077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15f2b0 0x7f723c06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 -- 172.21.15.40:0/3239579077 >> 172.21.15.40:0/3239579077 conn(0x7f723c06f050 msgr2=0x7f723c06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 -- 172.21.15.40:0/3239579077 shutdown_connections 2024-04-26T21:12:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.560+0000 7f7241ec8640 1 -- 172.21.15.40:0/3239579077 wait complete. 2024-04-26T21:12:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.561+0000 7f7241ec8640 1 Processor -- start 2024-04-26T21:12:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.561+0000 7f7241ec8640 1 -- start start 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f723c15f2b0 0x7f723c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15c270 0x7f723c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0b4f40 con 0x7f723c06c9b0 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0713d0 con 0x7f723c15f2b0 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f7241ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0b1890 con 0x7f723c15c270 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47798/0 (socket says 172.21.15.40:47798) 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 -- 172.21.15.40:0/1544559209 learned_addr learned my addr 172.21.15.40:0/1544559209 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 -- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15c270 msgr2=0x7f723c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15c270 0x7f723c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 -- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f723c15f2b0 msgr2=0x7f723c1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f723c15f2b0 0x7f723c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 -- 172.21.15.40:0/1544559209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c15a8f0 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.562+0000 7f723b7fe640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f722c00d5f0 tx=0x7f722c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.563+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c0125f0 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.563+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c019070 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.563+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c0129d0 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.563+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c15a610 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.563+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c1610b0 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.564+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c06a290 con 0x7f723c06c9b0 2024-04-26T21:12:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.564+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f722c012b70 con 0x7f723c06c9b0 2024-04-26T21:12:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.565+0000 7f7223fff640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.565+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f722c08bec0 con 0x7f723c06c9b0 2024-04-26T21:12:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.565+0000 7f723affd640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.566+0000 7f723affd640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 0x7f722806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f723c160bb0 tx=0x7f72300079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.569+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f722c05a380 con 0x7f723c06c9b0 2024-04-26T21:12:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.755+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f723c0738f0 con 0x7f723c06c9b0 2024-04-26T21:12:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.755+0000 7f7223fff640 1 -- 172.21.15.40:0/1544559209 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f722c059c80 con 0x7f723c06c9b0 2024-04-26T21:12:40.756 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 msgr2=0x7f722806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 0x7f722806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f723c160bb0 tx=0x7f72300079b0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 msgr2=0x7f723c1f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f722c00d5f0 tx=0x7f722c00dab0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 shutdown_connections 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f723c15c270 0x7f723c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f723c15f2b0 0x7f723c1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 --2- 172.21.15.40:0/1544559209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723c06c9b0 0x7f723c1f4eb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 >> 172.21.15.40:0/1544559209 conn(0x7f723c06f050 msgr2=0x7f723c0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 shutdown_connections 2024-04-26T21:12:40.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.757+0000 7f7241ec8640 1 -- 172.21.15.40:0/1544559209 wait complete. 2024-04-26T21:12:40.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:40.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.872+0000 7ff5ab7e4640 1 Processor -- start 2024-04-26T21:12:40.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.872+0000 7ff5ab7e4640 1 -- start start 2024-04-26T21:12:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a406f340 0x7ff5a40b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5a40d0300 0x7ff5a406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40b3e50 0x7ff5a40720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a406fc60 con 0x7ff5a40b3e50 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a40b46a0 con 0x7ff5a40d0300 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5ab7e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a40b5db0 con 0x7ff5a406f340 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40b3e50 0x7ff5a40720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40b3e50 0x7ff5a40720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54416/0 (socket says 172.21.15.40:54416) 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 -- 172.21.15.40:0/2569596860 learned_addr learned my addr 172.21.15.40:0/2569596860 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 -- 172.21.15.40:0/2569596860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a406f340 msgr2=0x7ff5a40b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 --2- 172.21.15.40:0/2569596860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a406f340 0x7ff5a40b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 -- 172.21.15.40:0/2569596860 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40d0300 msgr2=0x7ff5a406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a8d7f640 1 --2- 172.21.15.40:0/2569596860 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40d0300 0x7ff5a406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 --2- 172.21.15.40:0/2569596860 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40d0300 0x7ff5a406f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.873+0000 7ff5a9d81640 1 -- 172.21.15.40:0/2569596860 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a40b6090 con 0x7ff5a40b3e50 2024-04-26T21:12:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.874+0000 7ff5a9d81640 1 --2- 172.21.15.40:0/2569596860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a40720f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40b3c10 tx=0x7ff59402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91231441e784017f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.874+0000 7ff5a3fff640 1 -- 172.21.15.40:0/2569596860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594036070 con 0x7ff5a40b3e50 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.874+0000 7ff5a3fff640 1 -- 172.21.15.40:0/2569596860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff594031600 con 0x7ff5a40b3e50 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.874+0000 7ff5a3fff640 1 -- 172.21.15.40:0/2569596860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940319d0 con 0x7ff5a40b3e50 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2569596860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 msgr2=0x7ff5a40720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2569596860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a40720f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40b3c10 tx=0x7ff59402a570 comp rx=0 tx=0).stop 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2569596860 shutdown_connections 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2569596860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a40720f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2569596860 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40d0300 0x7ff5a406f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2569596860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a406f340 0x7ff5a40b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2569596860 >> 172.21.15.40:0/2569596860 conn(0x7ff5a40cd8a0 msgr2=0x7ff5a40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2569596860 shutdown_connections 2024-04-26T21:12:40.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.875+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2569596860 wait complete. 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.876+0000 7ff5ab7e4640 1 Processor -- start 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.876+0000 7ff5ab7e4640 1 -- start start 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406f340 0x7ff5a41f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 0x7ff5a41e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40b42e0 con 0x7ff5a40b3e50 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40b6650 con 0x7ff5a406f340 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5ab7e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a40b4460 con 0x7ff5a40d0300 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54428/0 (socket says 172.21.15.40:54428) 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 -- 172.21.15.40:0/2099306690 learned_addr learned my addr 172.21.15.40:0/2099306690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a9580640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406f340 0x7ff5a41f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a9d81640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 0x7ff5a41e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 -- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 msgr2=0x7ff5a41e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 0x7ff5a41e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 -- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406f340 msgr2=0x7ff5a41f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406f340 0x7ff5a41f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 -- 172.21.15.40:0/2099306690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a41e6020 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a9d81640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 0x7ff5a41e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.877+0000 7ff5a8d7f640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c820 tx=0x7ff59800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.878+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598015070 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.878+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598004020 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.878+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a41e6300 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.878+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980043f0 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.878+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41e68c0 con 0x7ff5a40b3e50 2024-04-26T21:12:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.879+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a40d0870 con 0x7ff5a40b3e50 2024-04-26T21:12:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.879+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff598007830 con 0x7ff5a40b3e50 2024-04-26T21:12:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.880+0000 7ff5a1ffb640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.880+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff59808b960 con 0x7ff5a40b3e50 2024-04-26T21:12:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.880+0000 7ff5a9580640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:40.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.880+0000 7ff5a9580640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4145090 tx=0x7ff59c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:40.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:40.884+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5a40d0870 con 0x7ff5a40b3e50 2024-04-26T21:12:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.074+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff5a406a290 con 0x7ff5a40b3e50 2024-04-26T21:12:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.074+0000 7ff5a1ffb640 1 -- 172.21.15.40:0/2099306690 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff598059d90 con 0x7ff5a40b3e50 2024-04-26T21:12:41.075 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 msgr2=0x7ff59006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4145090 tx=0x7ff59c008040 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 msgr2=0x7ff5a41e56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c820 tx=0x7ff59800cce0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 shutdown_connections 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d0300 0x7ff5a41e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40b3e50 0x7ff5a41e56b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 --2- 172.21.15.40:0/2099306690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406f340 0x7ff5a41f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 >> 172.21.15.40:0/2099306690 conn(0x7ff5a40cd8a0 msgr2=0x7ff5a4140020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 shutdown_connections 2024-04-26T21:12:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.076+0000 7ff5ab7e4640 1 -- 172.21.15.40:0/2099306690 wait complete. 2024-04-26T21:12:41.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.191+0000 7fd5c4683640 1 Processor -- start 2024-04-26T21:12:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.191+0000 7fd5c4683640 1 -- start start 2024-04-26T21:12:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c4683640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5bc15f470 0x7fd5bc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c4683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc0b47d0 0x7fd5bc06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c4683640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5bc0aeb50 con 0x7fd5bc069ae0 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c4683640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd5bc0aee30 con 0x7fd5bc15f470 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c4683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc0af110 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5bc15f470 0x7fd5bc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c1c1e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5bc069ae0 0x7fd5bc0b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34942/0 (socket says 172.21.15.40:34942) 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c1c1e640 1 -- 172.21.15.40:0/2947913722 learned_addr learned my addr 172.21.15.40:0/2947913722 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5bc15f470 0x7fd5bc0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54440/0 (socket says 172.21.15.40:54440) 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c2c20640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc0b47d0 0x7fd5bc06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 -- 172.21.15.40:0/2947913722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc0b47d0 msgr2=0x7fd5bc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc0b47d0 0x7fd5bc06e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 -- 172.21.15.40:0/2947913722 >> v1:172.21.15.40:6789/0 conn(0x7fd5bc069ae0 legacy=0x7fd5bc0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c241f640 1 -- 172.21.15.40:0/2947913722 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0af3f0 con 0x7fd5bc15f470 2024-04-26T21:12:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.192+0000 7fd5c141d640 1 -- 172.21.15.40:0/2947913722 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211400531 0 0) 0x7fd5bc0aeb50 con 0x7fd5bc069ae0 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.193+0000 7fd5c241f640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc15f470 0x7fd5bc0af910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35037f2a1d15f9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.193+0000 7fd5c141d640 1 -- 172.21.15.40:0/2947913722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8037070 con 0x7fd5bc15f470 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.193+0000 7fd5c141d640 1 -- 172.21.15.40:0/2947913722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b8032440 con 0x7fd5bc15f470 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.193+0000 7fd5c141d640 1 -- 172.21.15.40:0/2947913722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8032810 con 0x7fd5bc15f470 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 -- 172.21.15.40:0/2947913722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc15f470 msgr2=0x7fd5bc0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc15f470 0x7fd5bc0af910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b802a180 comp rx=0 tx=0).stop 2024-04-26T21:12:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 -- 172.21.15.40:0/2947913722 shutdown_connections 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc0b47d0 0x7fd5bc06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 --2- 172.21.15.40:0/2947913722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc15f470 0x7fd5bc0af910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 -- 172.21.15.40:0/2947913722 >> 172.21.15.40:0/2947913722 conn(0x7fd5bc073910 msgr2=0x7fd5bc073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 -- 172.21.15.40:0/2947913722 shutdown_connections 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.194+0000 7fd5c4683640 1 -- 172.21.15.40:0/2947913722 wait complete. 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.195+0000 7fd5c4683640 1 Processor -- start 2024-04-26T21:12:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.195+0000 7fd5c4683640 1 -- start start 2024-04-26T21:12:41.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 0x7fd5bc09a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5bc15f470 0x7fd5bc14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b63a0 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b6220 con 0x7fd5bc15f470 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c4683640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc0afe50 con 0x7fd5bc069ae0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54452/0 (socket says 172.21.15.40:54452) 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 -- 172.21.15.40:0/3061736922 learned_addr learned my addr 172.21.15.40:0/3061736922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c241f640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 0x7fd5bc09a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c2c20640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5bc15f470 0x7fd5bc14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 -- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 msgr2=0x7fd5bc09a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 0x7fd5bc09a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 -- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5bc15f470 msgr2=0x7fd5bc14eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5bc15f470 0x7fd5bc14eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 -- 172.21.15.40:0/3061736922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc1518c0 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c241f640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 0x7fd5bc09a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5c1c1e640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00c4d0 tx=0x7fd5ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.196+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac018070 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.197+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ac007500 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.197+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac004a20 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.197+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc151ba0 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.197+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc142160 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.198+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc06a290 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.198+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5ac004bc0 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.198+0000 7fd5b2ffd640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 0x7fd5a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.199+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5854+0+0 (secure 0 0 0) 0x7fd5ac08af00 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.199+0000 7fd5c241f640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 0x7fd5a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.199+0000 7fd5c241f640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 0x7fd5a006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.203+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5ac059220 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.388+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5bc09aa80 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.388+0000 7fd5b2ffd640 1 -- 172.21.15.40:0/3061736922 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd5ac058b20 con 0x7fd5bc0b47d0 2024-04-26T21:12:41.390 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:41.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 msgr2=0x7fd5a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 0x7fd5a006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 msgr2=0x7fd5bc14e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00c4d0 tx=0x7fd5ac00c990 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 shutdown_connections 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5a006c8f0 0x7fd5a006edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5bc15f470 0x7fd5bc14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5bc0b47d0 0x7fd5bc14e980 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 --2- 172.21.15.40:0/3061736922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5bc069ae0 0x7fd5bc09a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 >> 172.21.15.40:0/3061736922 conn(0x7fd5bc073910 msgr2=0x7fd5bc06ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 shutdown_connections 2024-04-26T21:12:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.390+0000 7fd5c4683640 1 -- 172.21.15.40:0/3061736922 wait complete. 2024-04-26T21:12:41.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:12:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 Processor -- start 2024-04-26T21:12:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 -- start start 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd19806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd19806e2c0 0x7fd19815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1980d1740 con 0x7fd1980cf2e0 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd19806e750 con 0x7fd19806e2c0 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.503+0000 7fd19ed14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980d1a20 con 0x7fd1980d11b0 2024-04-26T21:12:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd19806e2c0 0x7fd19815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd19806e2c0 0x7fd19815ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54466/0 (socket says 172.21.15.40:54466) 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19cab0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd1980cf2e0 0x7fd19806d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34950/0 (socket says 172.21.15.40:34950) 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 -- 172.21.15.40:0/3964215651 learned_addr learned my addr 172.21.15.40:0/3964215651 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd197fff640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd19806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 -- 172.21.15.40:0/3964215651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 msgr2=0x7fd19806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd19806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 -- 172.21.15.40:0/3964215651 >> v1:172.21.15.40:6789/0 conn(0x7fd1980cf2e0 legacy=0x7fd19806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd1977fe640 1 -- 172.21.15.40:0/3964215651 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 146572508 0 0) 0x7fd1980d1740 con 0x7fd1980cf2e0 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd19d2b1640 1 -- 172.21.15.40:0/3964215651 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980d1d00 con 0x7fd19806e2c0 2024-04-26T21:12:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.504+0000 7fd197fff640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd19806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19d2b1640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd19806e2c0 0x7fd19815ebe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd190009e50 tx=0x7fd19002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a7608c16dca80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd1977fe640 1 -- 172.21.15.40:0/3964215651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190036070 con 0x7fd19806e2c0 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd1977fe640 1 -- 172.21.15.40:0/3964215651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190031600 con 0x7fd19806e2c0 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd1977fe640 1 -- 172.21.15.40:0/3964215651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900319d0 con 0x7fd19806e2c0 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 -- 172.21.15.40:0/3964215651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd19806e2c0 msgr2=0x7fd19815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd19806e2c0 0x7fd19815ebe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd190009e50 tx=0x7fd19002a710 comp rx=0 tx=0).stop 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 -- 172.21.15.40:0/3964215651 shutdown_connections 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd19806e2c0 0x7fd19815ebe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 --2- 172.21.15.40:0/3964215651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd19806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 -- 172.21.15.40:0/3964215651 >> 172.21.15.40:0/3964215651 conn(0x7fd1980b4a60 msgr2=0x7fd1980b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 -- 172.21.15.40:0/3964215651 shutdown_connections 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.505+0000 7fd19ed14640 1 -- 172.21.15.40:0/3964215651 wait complete. 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.506+0000 7fd19ed14640 1 Processor -- start 2024-04-26T21:12:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 -- start start 2024-04-26T21:12:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd19806e2c0 0x7fd198160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd1981f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980cf6c0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980d22c0 con 0x7fd19806e2c0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.507+0000 7fd19ed14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980cf840 con 0x7fd1980d11b0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54474/0 (socket says 172.21.15.40:54474) 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 -- 172.21.15.40:0/2436885501 learned_addr learned my addr 172.21.15.40:0/2436885501 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd19d2b1640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd1981f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd19cab0640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd19806e2c0 0x7fd198160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 -- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 msgr2=0x7fd1981f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd1981f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 -- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd19806e2c0 msgr2=0x7fd198160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd19806e2c0 0x7fd198160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 -- 172.21.15.40:0/2436885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981e8200 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd19d2b1640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd1981f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd197fff640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7f0 tx=0x7fd18400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184015070 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd184007820 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981e84e0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007bd0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.508+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981e8aa0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.509+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19806a290 con 0x7fd1980cf2e0 2024-04-26T21:12:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.510+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd184018070 con 0x7fd1980cf2e0 2024-04-26T21:12:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.510+0000 7fd1957fa640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.510+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5854+0+0 (secure 0 0 0) 0x7fd18408bd50 con 0x7fd1980cf2e0 2024-04-26T21:12:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.510+0000 7fd19cab0640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.511+0000 7fd19cab0640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 0x7fd17c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000f60 tx=0x7fd18c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.515+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd18408f030 con 0x7fd1980cf2e0 2024-04-26T21:12:41.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.711+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1980b8ef0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.711+0000 7fd1957fa640 1 -- 172.21.15.40:0/2436885501 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fd18405a0f0 con 0x7fd1980cf2e0 2024-04-26T21:12:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.713+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 msgr2=0x7fd17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.713+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 0x7fd17c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000f60 tx=0x7fd18c007c00 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.713+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 msgr2=0x7fd1981e7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.713+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7f0 tx=0x7fd18400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 shutdown_connections 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1980d11b0 0x7fd1981f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1980cf2e0 0x7fd1981e7890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 --2- 172.21.15.40:0/2436885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd19806e2c0 0x7fd198160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 >> 172.21.15.40:0/2436885501 conn(0x7fd1980b4a60 msgr2=0x7fd1980c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 shutdown_connections 2024-04-26T21:12:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.714+0000 7fd19ed14640 1 -- 172.21.15.40:0/2436885501 wait complete. 2024-04-26T21:12:41.725 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T21:12:41.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:41.726 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T21:12:41.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:12:41.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:12:41.757 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T21:12:41.814 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:12:41.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.914+0000 7fd6ac68c640 1 Processor -- start 2024-04-26T21:12:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.914+0000 7fd6ac68c640 1 -- start start 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.915+0000 7fd6ac68c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 0x7fd6a40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.915+0000 7fd6ac68c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.915+0000 7fd6ac68c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a40d03a0 con 0x7fd6a40d37d0 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.915+0000 7fd6ac68c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a40b4d10 con 0x7fd6a415f370 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.915+0000 7fd6ac68c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a40d0680 con 0x7fd6a415f750 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6aa428640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 0x7fd6a40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:55420/0 (socket says 172.21.15.40:55420) 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 -- 172.21.15.40:0/3432452075 learned_addr learned my addr 172.21.15.40:0/3432452075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6aac29640 1 --1- 172.21.15.40:0/3432452075 >> v1:172.21.15.184:6789/0 conn(0x7fd6a40d37d0 0x7fd6a40d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:53866/0 (socket says 172.21.15.40:53866) 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 -- 172.21.15.40:0/3432452075 >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 msgr2=0x7fd6a40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 --2- 172.21.15.40:0/3432452075 >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 0x7fd6a40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 -- 172.21.15.40:0/3432452075 >> v1:172.21.15.184:6789/0 conn(0x7fd6a40d37d0 legacy=0x7fd6a40d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6a9c27640 1 -- 172.21.15.40:0/3432452075 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a40d0960 con 0x7fd6a415f750 2024-04-26T21:12:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.916+0000 7fd6aa428640 1 --2- 172.21.15.40:0/3432452075 >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 0x7fd6a40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6a9c27640 1 --2- 172.21.15.40:0/3432452075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd698009870 tx=0x7fd69802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a5baae2472afa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6a9426640 1 -- 172.21.15.40:0/3432452075 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698037070 con 0x7fd6a415f750 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6a9426640 1 -- 172.21.15.40:0/3432452075 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698032440 con 0x7fd6a415f750 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6a9426640 1 -- 172.21.15.40:0/3432452075 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698032810 con 0x7fd6a415f750 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6ac68c640 1 -- 172.21.15.40:0/3432452075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 msgr2=0x7fd6a40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.917+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/3432452075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd698009870 tx=0x7fd69802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 -- 172.21.15.40:0/3432452075 shutdown_connections 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/3432452075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f750 0x7fd6a40b47d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/3432452075 >> v2:172.21.15.184:3300/0 conn(0x7fd6a415f370 0x7fd6a40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 -- 172.21.15.40:0/3432452075 >> 172.21.15.40:0/3432452075 conn(0x7fd6a408ec10 msgr2=0x7fd6a409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 -- 172.21.15.40:0/3432452075 shutdown_connections 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.918+0000 7fd6ac68c640 1 -- 172.21.15.40:0/3432452075 wait complete. 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 Processor -- start 2024-04-26T21:12:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 -- start start 2024-04-26T21:12:41.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f370 0x7fd6a41545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6a415f750 0x7fd6a40b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40b51c0 con 0x7fd6a40d37d0 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40d0f20 con 0x7fd6a415f750 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.919+0000 7fd6ac68c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a40b5340 con 0x7fd6a415f370 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54476/0 (socket says 172.21.15.40:54476) 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 -- 172.21.15.40:0/2060940500 learned_addr learned my addr 172.21.15.40:0/2060940500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6a9c27640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f370 0x7fd6a41545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aac29640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6a415f750 0x7fd6a40b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 -- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f370 msgr2=0x7fd6a41545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f370 0x7fd6a41545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 -- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6a415f750 msgr2=0x7fd6a40b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:41.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6a415f750 0x7fd6a40b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 -- 172.21.15.40:0/2060940500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a40b8930 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6aa428640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00efc0 tx=0x7fd69c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c018020 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c014680 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a40b8c10 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c014a50 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.920+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a40b91d0 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.921+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a406a290 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.922+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd69c005d10 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.922+0000 7fd6a2ffd640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.922+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5854+0+0 (secure 0 0 0) 0x7fd69c08ca80 con 0x7fd6a40d37d0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.923+0000 7fd6a9c27640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.923+0000 7fd6a9c27640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 0x7fd69006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd698034000 tx=0x7fd69802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:41.927+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd69c08ee20 con 0x7fd6a40d37d0 2024-04-26T21:12:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.154+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a406d320 con 0x7fd6a40d37d0 2024-04-26T21:12:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.155+0000 7fd6a2ffd640 1 -- 172.21.15.40:0/2060940500 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7fd69c022030 con 0x7fd6a40d37d0 2024-04-26T21:12:42.156 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:42.156 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 msgr2=0x7fd69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 0x7fd69006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd698034000 tx=0x7fd69802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 msgr2=0x7fd6a40cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00efc0 tx=0x7fd69c00c490 comp rx=0 tx=0).stop 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 shutdown_connections 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd69006c8f0 0x7fd69006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6a415f750 0x7fd6a40b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6a415f370 0x7fd6a41545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 --2- 172.21.15.40:0/2060940500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a40d37d0 0x7fd6a40cecf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 >> 172.21.15.40:0/2060940500 conn(0x7fd6a408ec10 msgr2=0x7fd6a406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 shutdown_connections 2024-04-26T21:12:42.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.157+0000 7fd6ac68c640 1 -- 172.21.15.40:0/2060940500 wait complete. 2024-04-26T21:12:42.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.270+0000 7f31944c5640 1 Processor -- start 2024-04-26T21:12:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.270+0000 7f31944c5640 1 -- start start 2024-04-26T21:12:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f31944c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f318c15f370 0x7f318c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f31944c5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 0x7f318c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f31944c5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c0d03a0 con 0x7f318c0d37d0 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f31944c5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f318c0b4d10 con 0x7f318c15f370 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f31944c5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f318c0d0680 con 0x7f318c15f750 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f3192261640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f318c15f370 0x7f318c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f3192261640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f318c15f370 0x7f318c0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54484/0 (socket says 172.21.15.40:54484) 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f3192261640 1 -- 172.21.15.40:0/423014027 learned_addr learned my addr 172.21.15.40:0/423014027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.271+0000 7f3192a62640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f318c0d37d0 0x7f318c0d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34960/0 (socket says 172.21.15.40:34960) 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3191a60640 1 --2- 172.21.15.40:0/423014027 >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 0x7f318c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3192261640 1 -- 172.21.15.40:0/423014027 >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 msgr2=0x7f318c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3192261640 1 --2- 172.21.15.40:0/423014027 >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 0x7f318c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3192261640 1 -- 172.21.15.40:0/423014027 >> v1:172.21.15.40:6789/0 conn(0x7f318c0d37d0 legacy=0x7f318c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3192261640 1 -- 172.21.15.40:0/423014027 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c0d0960 con 0x7f318c15f370 2024-04-26T21:12:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f319125f640 1 -- 172.21.15.40:0/423014027 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764613426 0 0) 0x7f318c0d03a0 con 0x7f318c0d37d0 2024-04-26T21:12:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3191a60640 1 --2- 172.21.15.40:0/423014027 >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 0x7f318c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.272+0000 7f3192261640 1 --2- 172.21.15.40:0/423014027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c15f370 0x7f318c0cfd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f318c0d02b0 tx=0x7f317c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6362de4455f4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f319125f640 1 -- 172.21.15.40:0/423014027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c036070 con 0x7f318c15f370 2024-04-26T21:12:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f319125f640 1 -- 172.21.15.40:0/423014027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c031600 con 0x7f318c15f370 2024-04-26T21:12:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f319125f640 1 -- 172.21.15.40:0/423014027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c0319d0 con 0x7f318c15f370 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 -- 172.21.15.40:0/423014027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c15f370 msgr2=0x7f318c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 --2- 172.21.15.40:0/423014027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c15f370 0x7f318c0cfd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f318c0d02b0 tx=0x7f317c02a730 comp rx=0 tx=0).stop 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 -- 172.21.15.40:0/423014027 shutdown_connections 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 --2- 172.21.15.40:0/423014027 >> v2:172.21.15.184:3300/0 conn(0x7f318c15f750 0x7f318c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 --2- 172.21.15.40:0/423014027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c15f370 0x7f318c0cfd70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 -- 172.21.15.40:0/423014027 >> 172.21.15.40:0/423014027 conn(0x7f318c08ec10 msgr2=0x7f318c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 -- 172.21.15.40:0/423014027 shutdown_connections 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.273+0000 7f31944c5640 1 -- 172.21.15.40:0/423014027 wait complete. 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.274+0000 7f31944c5640 1 Processor -- start 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.274+0000 7f31944c5640 1 -- start start 2024-04-26T21:12:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c15f370 0x7f318c0b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 0x7f318c0ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0b61d0 con 0x7f318c0d37d0 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0d0f20 con 0x7f318c15f370 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f31944c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0b6350 con 0x7f318c15f750 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54494/0 (socket says 172.21.15.40:54494) 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 -- 172.21.15.40:0/781481302 learned_addr learned my addr 172.21.15.40:0/781481302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3191a60640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c15f370 0x7f318c0b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192a62640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 0x7f318c0ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 -- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 msgr2=0x7f318c0ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 0x7f318c0ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 -- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c15f370 msgr2=0x7f318c0b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.275+0000 7f3192261640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c15f370 0x7f318c0b9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f3192261640 1 -- 172.21.15.40:0/781481302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c1f59c0 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f3192a62640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 0x7f318c0ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f3192261640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f317c000f80 tx=0x7f317c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c041020 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c037040 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c0b8fa0 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c036050 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.276+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c0b9560 con 0x7f318c0d37d0 2024-04-26T21:12:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.277+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c06a290 con 0x7f318c0d37d0 2024-04-26T21:12:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.277+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f317c007c70 con 0x7f318c0d37d0 2024-04-26T21:12:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.278+0000 7f318affd640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.278+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f317c0ac470 con 0x7f318c0d37d0 2024-04-26T21:12:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.278+0000 7f3191a60640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.279+0000 7f3191a60640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f318c09e960 tx=0x7f3180008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.282+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f317c04b060 con 0x7f318c0d37d0 2024-04-26T21:12:42.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.466+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f318c0b8d70 con 0x7f318c0d37d0 2024-04-26T21:12:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.468+0000 7f318affd640 1 -- 172.21.15.40:0/781481302 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16901 (secure 0 0 0) 0x7f317c07a8a0 con 0x7f318c0d37d0 2024-04-26T21:12:42.469 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:42.469 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:42.019368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 msgr2=0x7f317806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f318c09e960 tx=0x7f3180008040 comp rx=0 tx=0).stop 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 msgr2=0x7f318c15e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f317c000f80 tx=0x7f317c004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 shutdown_connections 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c15f750 0x7f318c0ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c15f370 0x7f318c0b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 --2- 172.21.15.40:0/781481302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f318c0d37d0 0x7f318c15e0a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 >> 172.21.15.40:0/781481302 conn(0x7f318c08ec10 msgr2=0x7f318c06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 shutdown_connections 2024-04-26T21:12:42.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.470+0000 7f31944c5640 1 -- 172.21.15.40:0/781481302 wait complete. 2024-04-26T21:12:42.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:11:58.923694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:00.058755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:42.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.585+0000 7f586e852640 1 Processor -- start 2024-04-26T21:12:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 -- start start 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58680d3820 0x7f58680b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 0x7f58680d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680d3c90 con 0x7f586815f680 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f586815fa60 con 0x7f58680d3820 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.586+0000 7f586e852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58680d3f50 con 0x7f58680b60b0 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f586cdef640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 0x7f58680d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f58677fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58680d3820 0x7f58680b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f5867fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f586815f680 0x7f5868163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:53868/0 (socket says 172.21.15.40:53868) 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f5867fff640 1 -- 172.21.15.40:0/3484857107 learned_addr learned my addr 172.21.15.40:0/3484857107 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f58677fe640 1 -- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 msgr2=0x7f58680d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f58677fe640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 0x7f58680d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f58677fe640 1 -- 172.21.15.40:0/3484857107 >> v1:172.21.15.184:6789/0 conn(0x7f586815f680 legacy=0x7f5868163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f5866ffd640 1 -- 172.21.15.40:0/3484857107 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473850700 0 0) 0x7f58680d3c90 con 0x7f586815f680 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f58677fe640 1 -- 172.21.15.40:0/3484857107 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680b5090 con 0x7f58680d3820 2024-04-26T21:12:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.587+0000 7f586cdef640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 0x7f58680d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.588+0000 7f58677fe640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680d3820 0x7f58680b5b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f585c0097a0 tx=0x7f585c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2fe9be8879c3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.588+0000 7f5866ffd640 1 -- 172.21.15.40:0/3484857107 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c037070 con 0x7f58680d3820 2024-04-26T21:12:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.588+0000 7f5866ffd640 1 -- 172.21.15.40:0/3484857107 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c032440 con 0x7f58680d3820 2024-04-26T21:12:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.588+0000 7f5866ffd640 1 -- 172.21.15.40:0/3484857107 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c032810 con 0x7f58680d3820 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 -- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680d3820 msgr2=0x7f58680b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680d3820 0x7f58680b5b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f585c0097a0 tx=0x7f585c02a180 comp rx=0 tx=0).stop 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 -- 172.21.15.40:0/3484857107 shutdown_connections 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680b60b0 0x7f58680d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 --2- 172.21.15.40:0/3484857107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680d3820 0x7f58680b5b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 -- 172.21.15.40:0/3484857107 >> 172.21.15.40:0/3484857107 conn(0x7f586808eca0 msgr2=0x7f586809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 -- 172.21.15.40:0/3484857107 shutdown_connections 2024-04-26T21:12:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.589+0000 7f586e852640 1 -- 172.21.15.40:0/3484857107 wait complete. 2024-04-26T21:12:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.590+0000 7f586e852640 1 Processor -- start 2024-04-26T21:12:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.590+0000 7f586e852640 1 -- start start 2024-04-26T21:12:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680b60b0 0x7f58681536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680d3820 0x7f5868160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58680d55e0 con 0x7f586815f680 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58680d5460 con 0x7f58680b60b0 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586e852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58680d5760 con 0x7f58680d3820 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54496/0 (socket says 172.21.15.40:54496) 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 -- 172.21.15.40:0/3874718820 learned_addr learned my addr 172.21.15.40:0/3874718820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f58677fe640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680d3820 0x7f5868160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f5867fff640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680b60b0 0x7f58681536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 -- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680d3820 msgr2=0x7f5868160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680d3820 0x7f5868160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 -- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680b60b0 msgr2=0x7f58681536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680b60b0 0x7f58681536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 -- 172.21.15.40:0/3874718820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680cde70 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.591+0000 7f586cdef640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009e60 tx=0x7f586000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.592+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860018030 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.592+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586000cdf0 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.592+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58680ce150 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.592+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860003bb0 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.592+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58680ce710 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.593+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586806a290 con 0x7f586815f680 2024-04-26T21:12:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.593+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5860005d10 con 0x7f586815f680 2024-04-26T21:12:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.594+0000 7f5864ff9640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.594+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f586008c590 con 0x7f586815f680 2024-04-26T21:12:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.594+0000 7f5867fff640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.594+0000 7f5867fff640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 0x7f585406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.598+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5860022070 con 0x7f586815f680 2024-04-26T21:12:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.783+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f586815ecb0 con 0x7f586815f680 2024-04-26T21:12:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.783+0000 7f5864ff9640 1 -- 172.21.15.40:0/3874718820 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f5860010040 con 0x7f586815f680 2024-04-26T21:12:42.784 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 msgr2=0x7f585406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 0x7f585406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).stop 2024-04-26T21:12:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 msgr2=0x7f5868161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009e60 tx=0x7f586000c490 comp rx=0 tx=0).stop 2024-04-26T21:12:42.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 shutdown_connections 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f586815f680 0x7f5868161fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58680d3820 0x7f5868160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 --2- 172.21.15.40:0/3874718820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58680b60b0 0x7f58681536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 >> 172.21.15.40:0/3874718820 conn(0x7f586808eca0 msgr2=0x7f58680d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 shutdown_connections 2024-04-26T21:12:42.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.785+0000 7f586e852640 1 -- 172.21.15.40:0/3874718820 wait complete. 2024-04-26T21:12:42.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:12:42.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.899+0000 7f1dbeb71640 1 Processor -- start 2024-04-26T21:12:42.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.899+0000 7f1dbeb71640 1 -- start start 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbeb71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 0x7f1db80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbeb71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1db80d1170 0x7f1db81635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbeb71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db809c790 con 0x7f1db80b3300 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbeb71640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1db80d15f0 con 0x7f1db80d1170 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbeb71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8163ec0 con 0x7f1db81631f0 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1db80d1170 0x7f1db81635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1db80d1170 0x7f1db81635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54500/0 (socket says 172.21.15.40:54500) 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 -- 172.21.15.40:0/2219293549 learned_addr learned my addr 172.21.15.40:0/2219293549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbd10e640 1 --1- 172.21.15.40:0/2219293549 >> v1:172.21.15.40:6789/0 conn(0x7f1db80b3300 0x7f1db80b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34974/0 (socket says 172.21.15.40:34974) 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbc90d640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 0x7f1db80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 -- 172.21.15.40:0/2219293549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 msgr2=0x7f1db80b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 0x7f1db80b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 -- 172.21.15.40:0/2219293549 >> v1:172.21.15.40:6789/0 conn(0x7f1db80b3300 legacy=0x7f1db80b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1db7fff640 1 -- 172.21.15.40:0/2219293549 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80d1a40 con 0x7f1db80d1170 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.900+0000 7f1dbc90d640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 0x7f1db80b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.901+0000 7f1db7fff640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80d1170 0x7f1db81635d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009870 tx=0x7f1da402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e6d43ec1575344 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.901+0000 7f1db77fe640 1 -- 172.21.15.40:0/2219293549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4037070 con 0x7f1db80d1170 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.901+0000 7f1db77fe640 1 -- 172.21.15.40:0/2219293549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4032440 con 0x7f1db80d1170 2024-04-26T21:12:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.901+0000 7f1db77fe640 1 -- 172.21.15.40:0/2219293549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032810 con 0x7f1db80d1170 2024-04-26T21:12:42.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 -- 172.21.15.40:0/2219293549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80d1170 msgr2=0x7f1db81635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80d1170 0x7f1db81635d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009870 tx=0x7f1da402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 -- 172.21.15.40:0/2219293549 shutdown_connections 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80d1170 0x7f1db81635d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/2219293549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db81631f0 0x7f1db80b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 -- 172.21.15.40:0/2219293549 >> 172.21.15.40:0/2219293549 conn(0x7f1db809ba40 msgr2=0x7f1db809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 -- 172.21.15.40:0/2219293549 shutdown_connections 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.902+0000 7f1dbeb71640 1 -- 172.21.15.40:0/2219293549 wait complete. 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.903+0000 7f1dbeb71640 1 Processor -- start 2024-04-26T21:12:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.903+0000 7f1dbeb71640 1 -- start start 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 0x7f1db8162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 0x7f1db80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d0230 con 0x7f1db80b3300 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d2000 con 0x7f1db81631f0 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbeb71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8161a00 con 0x7f1db80d1170 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54510/0 (socket says 172.21.15.40:54510) 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 -- 172.21.15.40:0/3969122108 learned_addr learned my addr 172.21.15.40:0/3969122108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1db7fff640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 0x7f1db8162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbd10e640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 0x7f1db80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 -- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 msgr2=0x7f1db8162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 0x7f1db8162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 -- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 msgr2=0x7f1db80b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 0x7f1db80b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbc90d640 1 -- 172.21.15.40:0/3969122108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8160b50 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1db7fff640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 0x7f1db8162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.904+0000 7f1dbd10e640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 0x7f1db80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1dbc90d640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c7f0 tx=0x7f1dac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac015070 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dac007820 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac007bd0 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db8160df0 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.905+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81613b0 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.906+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db806a290 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.906+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dac018070 con 0x7f1db80b3300 2024-04-26T21:12:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.907+0000 7f1db57fa640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:42.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.907+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f1dac08bd50 con 0x7f1db80b3300 2024-04-26T21:12:42.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.907+0000 7f1db7fff640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:42.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.908+0000 7f1db7fff640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1da4034000 tx=0x7f1da402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:42.911+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dac08f030 con 0x7f1db80b3300 2024-04-26T21:12:43.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.099+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1db809c570 con 0x7f1db80b3300 2024-04-26T21:12:43.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.099+0000 7f1db57fa640 1 -- 172.21.15.40:0/3969122108 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f1dac05a060 con 0x7f1db80b3300 2024-04-26T21:12:43.100 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:12:43.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 msgr2=0x7f1d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1da4034000 tx=0x7f1da402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 msgr2=0x7f1db8151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c7f0 tx=0x7f1dac00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 shutdown_connections 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1db81631f0 0x7f1db80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1db80d1170 0x7f1db8162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 --2- 172.21.15.40:0/3969122108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1db80b3300 0x7f1db8151410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 >> 172.21.15.40:0/3969122108 conn(0x7f1db809ba40 msgr2=0x7f1db806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 shutdown_connections 2024-04-26T21:12:43.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.101+0000 7f1dbeb71640 1 -- 172.21.15.40:0/3969122108 wait complete. 2024-04-26T21:12:43.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:12:43.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.215+0000 7fc37b39d640 1 Processor -- start 2024-04-26T21:12:43.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.215+0000 7fc37b39d640 1 -- start start 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc37b39d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc374162ca0 0x7fc374070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc37b39d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 0x7fc3740ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc37b39d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3740d2470 con 0x7fc3740d1c20 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc37b39d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc3740cdc40 con 0x7fc374162ca0 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc37b39d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3740d2750 con 0x7fc374071330 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc379139640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc374162ca0 0x7fc374070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc378938640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc3740d1c20 0x7fc374070c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34980/0 (socket says 172.21.15.40:34980) 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.216+0000 7fc378938640 1 -- 172.21.15.40:0/1956039322 learned_addr learned my addr 172.21.15.40:0/1956039322 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc37993a640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 0x7fc3740ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc379139640 1 -- 172.21.15.40:0/1956039322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 msgr2=0x7fc3740ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc379139640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 0x7fc3740ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc379139640 1 -- 172.21.15.40:0/1956039322 >> v1:172.21.15.40:6789/0 conn(0x7fc3740d1c20 legacy=0x7fc374070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc373fff640 1 -- 172.21.15.40:0/1956039322 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132235901 0 0) 0x7fc3740d2470 con 0x7fc3740d1c20 2024-04-26T21:12:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc379139640 1 -- 172.21.15.40:0/1956039322 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740cd5f0 con 0x7fc374162ca0 2024-04-26T21:12:43.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc37993a640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 0x7fc3740ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.217+0000 7fc379139640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374162ca0 0x7fc374070640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0097a0 tx=0x7fc36c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cabc7ff9101ea31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc373fff640 1 -- 172.21.15.40:0/1956039322 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c037070 con 0x7fc374162ca0 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc373fff640 1 -- 172.21.15.40:0/1956039322 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c032440 con 0x7fc374162ca0 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc373fff640 1 -- 172.21.15.40:0/1956039322 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c032810 con 0x7fc374162ca0 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 -- 172.21.15.40:0/1956039322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374162ca0 msgr2=0x7fc374070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374162ca0 0x7fc374070640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0097a0 tx=0x7fc36c02a180 comp rx=0 tx=0).stop 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 -- 172.21.15.40:0/1956039322 shutdown_connections 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc374071330 0x7fc3740ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1956039322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374162ca0 0x7fc374070640 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 -- 172.21.15.40:0/1956039322 >> 172.21.15.40:0/1956039322 conn(0x7fc37409ba40 msgr2=0x7fc37409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 -- 172.21.15.40:0/1956039322 shutdown_connections 2024-04-26T21:12:43.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.218+0000 7fc37b39d640 1 -- 172.21.15.40:0/1956039322 wait complete. 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.219+0000 7fc37b39d640 1 Processor -- start 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 -- start start 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 0x7fc37415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 0x7fc37415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3740d2230 con 0x7fc374071330 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3740d20b0 con 0x7fc374162ca0 2024-04-26T21:12:43.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc37b39d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc3740ce610 con 0x7fc3740d1c20 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc379139640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc379139640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54514/0 (socket says 172.21.15.40:54514) 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.220+0000 7fc379139640 1 -- 172.21.15.40:0/1748708402 learned_addr learned my addr 172.21.15.40:0/1748708402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc378938640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 0x7fc37415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 -- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 msgr2=0x7fc37415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc37993a640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 0x7fc37415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 0x7fc37415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 -- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 msgr2=0x7fc37415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 0x7fc37415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 -- 172.21.15.40:0/1748708402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740ccff0 con 0x7fc374071330 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc378938640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 0x7fc37415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc37993a640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 0x7fc37415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:43.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc379139640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc36c035040 tx=0x7fc36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c037050 con 0x7fc374071330 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c038040 con 0x7fc374071330 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741519b0 con 0x7fc374071330 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c004470 con 0x7fc374071330 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.221+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc374160b50 con 0x7fc374071330 2024-04-26T21:12:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.222+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3740693d0 con 0x7fc374071330 2024-04-26T21:12:43.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.223+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc36c030980 con 0x7fc374071330 2024-04-26T21:12:43.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.223+0000 7fc371ffb640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 0x7fc36006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.223+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7fc36c0ac7e0 con 0x7fc374071330 2024-04-26T21:12:43.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.223+0000 7fc378938640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 0x7fc36006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.224+0000 7fc378938640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 0x7fc36006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc37415c250 tx=0x7fc368009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.227+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc3740ccff0 con 0x7fc374071330 2024-04-26T21:12:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.414+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc37406ec10 con 0x7fc374071330 2024-04-26T21:12:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.415+0000 7fc371ffb640 1 -- 172.21.15.40:0/1748708402 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc36c07ab80 con 0x7fc374071330 2024-04-26T21:12:43.416 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:12:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 msgr2=0x7fc36006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 0x7fc36006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc37415c250 tx=0x7fc368009040 comp rx=0 tx=0).stop 2024-04-26T21:12:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 msgr2=0x7fc374151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc36c035040 tx=0x7fc36c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 shutdown_connections 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc36006c8f0 0x7fc36006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc374162ca0 0x7fc37415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3740d1c20 0x7fc37415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 --2- 172.21.15.40:0/1748708402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc374071330 0x7fc374151470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 >> 172.21.15.40:0/1748708402 conn(0x7fc37409ba40 msgr2=0x7fc3740cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 shutdown_connections 2024-04-26T21:12:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.417+0000 7fc37b39d640 1 -- 172.21.15.40:0/1748708402 wait complete. 2024-04-26T21:12:43.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.528+0000 7fb2a9cb8640 1 Processor -- start 2024-04-26T21:12:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.528+0000 7fb2a9cb8640 1 -- start start 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a9cb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2a413ff70 0x7fb2a4146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a9cb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 0x7fb2a41465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a9cb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a40d05b0 con 0x7fb2a4140350 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a9cb8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a40d0890 con 0x7fb2a413ff70 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a9cb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a406e450 con 0x7fb2a40cff20 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2a413ff70 0x7fb2a4146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2a413ff70 0x7fb2a4146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54528/0 (socket says 172.21.15.40:54528) 2024-04-26T21:12:43.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 -- 172.21.15.40:0/1595096373 learned_addr learned my addr 172.21.15.40:0/1595096373 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a3fff640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 0x7fb2a41465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 -- 172.21.15.40:0/1595096373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 msgr2=0x7fb2a41465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 0x7fb2a41465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 -- 172.21.15.40:0/1595096373 >> v1:172.21.15.184:6789/0 conn(0x7fb2a4140350 legacy=0x7fb2a40cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a37fe640 1 -- 172.21.15.40:0/1595096373 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a406e730 con 0x7fb2a413ff70 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.529+0000 7fb2a3fff640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 0x7fb2a41465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.530+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4146080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb294009870 tx=0x7fb29402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fd88481a0b1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.530+0000 7fb2a27fc640 1 -- 172.21.15.40:0/1595096373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294037070 con 0x7fb2a413ff70 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.530+0000 7fb2a27fc640 1 -- 172.21.15.40:0/1595096373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294032440 con 0x7fb2a413ff70 2024-04-26T21:12:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.530+0000 7fb2a27fc640 1 -- 172.21.15.40:0/1595096373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294032810 con 0x7fb2a413ff70 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/1595096373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 msgr2=0x7fb2a4146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4146080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb294009870 tx=0x7fb29402a180 comp rx=0 tx=0).stop 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/1595096373 shutdown_connections 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a40cff20 0x7fb2a41465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/1595096373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4146080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/1595096373 >> 172.21.15.40:0/1595096373 conn(0x7fb2a40d0d30 msgr2=0x7fb2a40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/1595096373 shutdown_connections 2024-04-26T21:12:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.531+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/1595096373 wait complete. 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.532+0000 7fb2a9cb8640 1 Processor -- start 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.532+0000 7fb2a9cb8640 1 -- start start 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 0x7fb2a413b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a4140350 0x7fb2a4145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a406efe0 con 0x7fb2a413ff70 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a406ee60 con 0x7fb2a40cff20 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a9cb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a406f160 con 0x7fb2a4140350 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54538/0 (socket says 172.21.15.40:54538) 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 -- 172.21.15.40:0/3357712326 learned_addr learned my addr 172.21.15.40:0/3357712326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a3fff640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a4140350 0x7fb2a4145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 0x7fb2a413b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 -- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a4140350 msgr2=0x7fb2a4145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a4140350 0x7fb2a4145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 -- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 msgr2=0x7fb2a413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 0x7fb2a413b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 -- 172.21.15.40:0/3357712326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4158350 con 0x7fb2a413ff70 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 0x7fb2a413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.533+0000 7fb2a2ffd640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b420 tx=0x7fb29800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.534+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298011830 con 0x7fb2a413ff70 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.534+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb298016070 con 0x7fb2a413ff70 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.534+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a4142730 con 0x7fb2a413ff70 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.534+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298011c10 con 0x7fb2a413ff70 2024-04-26T21:12:43.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.534+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a413cd20 con 0x7fb2a413ff70 2024-04-26T21:12:43.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.535+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a40b54b0 con 0x7fb2a413ff70 2024-04-26T21:12:43.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.535+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb29801b460 con 0x7fb2a413ff70 2024-04-26T21:12:43.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.536+0000 7fb28bfff640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 0x7fb29006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.536+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7fb29808b7b0 con 0x7fb2a413ff70 2024-04-26T21:12:43.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.536+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 0x7fb29006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.536+0000 7fb2a37fe640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 0x7fb29006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb294034000 tx=0x7fb29402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.540+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2a40b54b0 con 0x7fb2a413ff70 2024-04-26T21:12:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.763+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2a406a290 con 0x7fb2a413ff70 2024-04-26T21:12:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.763+0000 7fb28bfff640 1 -- 172.21.15.40:0/3357712326 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7fb298059be0 con 0x7fb2a413ff70 2024-04-26T21:12:43.765 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:43.765 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:43.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 msgr2=0x7fb29006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 0x7fb29006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb294034000 tx=0x7fb29402e000 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 msgr2=0x7fb2a4157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b420 tx=0x7fb29800b8e0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 shutdown_connections 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb29006c8f0 0x7fb29006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a4140350 0x7fb2a4145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a413ff70 0x7fb2a4157cc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 --2- 172.21.15.40:0/3357712326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a40cff20 0x7fb2a413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 >> 172.21.15.40:0/3357712326 conn(0x7fb2a40d0d30 msgr2=0x7fb2a4140730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 shutdown_connections 2024-04-26T21:12:43.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.765+0000 7fb2a9cb8640 1 -- 172.21.15.40:0/3357712326 wait complete. 2024-04-26T21:12:43.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.877+0000 7fd9af371640 1 Processor -- start 2024-04-26T21:12:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.877+0000 7fd9af371640 1 -- start start 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9af371640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9af371640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9a80d5610 0x7fd9a80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9af371640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a80b2510 con 0x7fd9a815f340 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9af371640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a80b27f0 con 0x7fd9a80d5610 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9af371640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a80b2ad0 con 0x7fd9a815f720 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9a80d5610 0x7fd9a80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9a80d5610 0x7fd9a80d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54550/0 (socket says 172.21.15.40:54550) 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 -- 172.21.15.40:0/3031632752 learned_addr learned my addr 172.21.15.40:0/3031632752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ac90c640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 -- 172.21.15.40:0/3031632752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 msgr2=0x7fd9a80b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 -- 172.21.15.40:0/3031632752 >> v1:172.21.15.184:6789/0 conn(0x7fd9a815f340 legacy=0x7fd9a80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ad90e640 1 -- 172.21.15.40:0/3031632752 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a80b2db0 con 0x7fd9a80d5610 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.878+0000 7fd9ac90c640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd9ad90e640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a80d5610 0x7fd9a80d3d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8d5114db5aa2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd99ffff640 1 -- 172.21.15.40:0/3031632752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0036070 con 0x7fd9a80d5610 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd99ffff640 1 -- 172.21.15.40:0/3031632752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0031600 con 0x7fd9a80d5610 2024-04-26T21:12:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd99ffff640 1 -- 172.21.15.40:0/3031632752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00319d0 con 0x7fd9a80d5610 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd9af371640 1 -- 172.21.15.40:0/3031632752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a80d5610 msgr2=0x7fd9a80d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.879+0000 7fd9af371640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a80d5610 0x7fd9a80d3d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009e50 tx=0x7fd9a002a630 comp rx=0 tx=0).stop 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 -- 172.21.15.40:0/3031632752 shutdown_connections 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a80d5610 0x7fd9a80d3d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 --2- 172.21.15.40:0/3031632752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 -- 172.21.15.40:0/3031632752 >> 172.21.15.40:0/3031632752 conn(0x7fd9a808ec70 msgr2=0x7fd9a809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 -- 172.21.15.40:0/3031632752 shutdown_connections 2024-04-26T21:12:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.880+0000 7fd9af371640 1 -- 172.21.15.40:0/3031632752 wait complete. 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.881+0000 7fd9af371640 1 Processor -- start 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.881+0000 7fd9af371640 1 -- start start 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9a80d5610 0x7fd9a815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80b3980 con 0x7fd9a815f340 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80b3800 con 0x7fd9a80d5610 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a80b3b00 con 0x7fd9a815f720 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54562/0 (socket says 172.21.15.40:54562) 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 -- 172.21.15.40:0/2007228297 learned_addr learned my addr 172.21.15.40:0/2007228297 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ad90e640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ad10d640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9a80d5610 0x7fd9a815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 -- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 msgr2=0x7fd9a80cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 -- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9a80d5610 msgr2=0x7fd9a815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9a80d5610 0x7fd9a815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 -- 172.21.15.40:0/2007228297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a80b93e0 con 0x7fd9a815f340 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9ac90c640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c7f0 tx=0x7fd99800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998015070 con 0x7fd9a815f340 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998007820 con 0x7fd9a815f340 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998007bd0 con 0x7fd9a815f340 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.882+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a80b96c0 con 0x7fd9a815f340 2024-04-26T21:12:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.883+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a80b9c80 con 0x7fd9a815f340 2024-04-26T21:12:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.884+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a806a290 con 0x7fd9a815f340 2024-04-26T21:12:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.884+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd998018070 con 0x7fd9a815f340 2024-04-26T21:12:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.884+0000 7fd99dffb640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 0x7fd98c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.884+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7fd99808bbb0 con 0x7fd9a815f340 2024-04-26T21:12:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.885+0000 7fd9ad10d640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 0x7fd98c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.885+0000 7fd9ad10d640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 0x7fd98c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4004f60 tx=0x7fd9a4007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:43.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:43.889+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd998059990 con 0x7fd9a815f340 2024-04-26T21:12:44.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.073+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9a80d3020 con 0x7fd9a815f340 2024-04-26T21:12:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.075+0000 7fd99dffb640 1 -- 172.21.15.40:0/2007228297 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16901 (secure 0 0 0) 0x7fd99805ee20 con 0x7fd9a815f340 2024-04-26T21:12:44.076 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:44.076 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:42.019368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 msgr2=0x7fd98c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 0x7fd98c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4004f60 tx=0x7fd9a4007e30 comp rx=0 tx=0).stop 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 msgr2=0x7fd9a81f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c7f0 tx=0x7fd99800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 shutdown_connections 2024-04-26T21:12:44.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd98c06c8f0 0x7fd98c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9a815f720 0x7fd9a80cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a815f340 0x7fd9a81f5160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 --2- 172.21.15.40:0/2007228297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9a80d5610 0x7fd9a815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 >> 172.21.15.40:0/2007228297 conn(0x7fd9a808ec70 msgr2=0x7fd9a80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 shutdown_connections 2024-04-26T21:12:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.077+0000 7fd9af371640 1 -- 172.21.15.40:0/2007228297 wait complete. 2024-04-26T21:12:44.089 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:12:44.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.189+0000 7f151f1f0640 1 Processor -- start 2024-04-26T21:12:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.189+0000 7f151f1f0640 1 -- start start 2024-04-26T21:12:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151f1f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15180cf2e0 0x7f151806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151f1f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15180d11b0 0x7f151806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151f1f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15180d1740 con 0x7f151806e2c0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151f1f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f151806e710 con 0x7f15180d11b0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151f1f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f15180d1a20 con 0x7f15180cf2e0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f1517fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15180d11b0 0x7f151806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151d78d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f151806e2c0 0x7f151815ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34990/0 (socket says 172.21.15.40:34990) 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.190+0000 7f151d78d640 1 -- 172.21.15.40:0/3235539649 learned_addr learned my addr 172.21.15.40:0/3235539649 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f1517fff640 1 -- 172.21.15.40:0/3235539649 >> v2:172.21.15.184:3300/0 conn(0x7f15180cf2e0 msgr2=0x7f151806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f151cf8c640 1 --2- 172.21.15.40:0/3235539649 >> v2:172.21.15.184:3300/0 conn(0x7f15180cf2e0 0x7f151806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f1517fff640 1 --2- 172.21.15.40:0/3235539649 >> v2:172.21.15.184:3300/0 conn(0x7f15180cf2e0 0x7f151806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f15177fe640 1 -- 172.21.15.40:0/3235539649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329895313 0 0) 0x7f15180d1740 con 0x7f151806e2c0 2024-04-26T21:12:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f1517fff640 1 -- 172.21.15.40:0/3235539649 >> v1:172.21.15.40:6789/0 conn(0x7f151806e2c0 legacy=0x7f151815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f1517fff640 1 -- 172.21.15.40:0/3235539649 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180d1d00 con 0x7f15180d11b0 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.191+0000 7f1517fff640 1 --2- 172.21.15.40:0/3235539649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f151806dd80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15080097a0 tx=0x7f150802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d052011e332c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f15177fe640 1 -- 172.21.15.40:0/3235539649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508037070 con 0x7f15180d11b0 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f15177fe640 1 -- 172.21.15.40:0/3235539649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508032440 con 0x7f15180d11b0 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f15177fe640 1 -- 172.21.15.40:0/3235539649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508032810 con 0x7f15180d11b0 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 -- 172.21.15.40:0/3235539649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 msgr2=0x7f151806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 --2- 172.21.15.40:0/3235539649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f151806dd80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15080097a0 tx=0x7f150802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 -- 172.21.15.40:0/3235539649 shutdown_connections 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 --2- 172.21.15.40:0/3235539649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f151806dd80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 --2- 172.21.15.40:0/3235539649 >> v2:172.21.15.184:3300/0 conn(0x7f15180cf2e0 0x7f151806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 -- 172.21.15.40:0/3235539649 >> 172.21.15.40:0/3235539649 conn(0x7f15180b4a60 msgr2=0x7f15180b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 -- 172.21.15.40:0/3235539649 shutdown_connections 2024-04-26T21:12:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.192+0000 7f151f1f0640 1 -- 172.21.15.40:0/3235539649 wait complete. 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.193+0000 7f151f1f0640 1 Processor -- start 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.193+0000 7f151f1f0640 1 -- start start 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f151806e2c0 0x7f151815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 0x7f15180a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f151806da10 con 0x7f15180d11b0 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15180d22c0 con 0x7f15180cf2e0 2024-04-26T21:12:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151f1f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15180cf6c0 con 0x7f151806e2c0 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151d78d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151d78d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54586/0 (socket says 172.21.15.40:54586) 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151d78d640 1 -- 172.21.15.40:0/603967526 learned_addr learned my addr 172.21.15.40:0/603967526 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f151cf8c640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f151806e2c0 0x7f151815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.194+0000 7f1517fff640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 0x7f15180a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 -- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f151806e2c0 msgr2=0x7f151815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f151806e2c0 0x7f151815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 -- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 msgr2=0x7f15180a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 0x7f15180a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 -- 172.21.15.40:0/603967526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181f8de0 con 0x7f15180d11b0 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f1517fff640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 0x7f15180a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151d78d640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f150000c450 tx=0x7f150000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500018070 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500007500 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15181f90c0 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500004a20 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.195+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151809fdb0 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.196+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151806a290 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.197+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f150001b070 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.197+0000 7f15157fa640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 0x7f14f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.197+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f150008ada0 con 0x7f15180d11b0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.197+0000 7f151cf8c640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 0x7f14f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.198+0000 7f151cf8c640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 0x7f14f806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f151000f510 tx=0x7f15100077b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.201+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1500058c40 con 0x7f15180d11b0 2024-04-26T21:12:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.424+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15180ce600 con 0x7f15180d11b0 2024-04-26T21:12:44.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.425+0000 7f15157fa640 1 -- 172.21.15.40:0/603967526 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f150005cf40 con 0x7f15180d11b0 2024-04-26T21:12:44.586 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:44.587 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 msgr2=0x7f14f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 0x7f14f806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f151000f510 tx=0x7f15100077b0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 msgr2=0x7f15181f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f150000c450 tx=0x7f150000c910 comp rx=0 tx=0).stop 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 shutdown_connections 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f14f806c8f0 0x7f14f806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15180d11b0 0x7f15181f8270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15180cf2e0 0x7f15180a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 --2- 172.21.15.40:0/603967526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f151806e2c0 0x7f151815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 >> 172.21.15.40:0/603967526 conn(0x7f15180b4a60 msgr2=0x7f1518071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 shutdown_connections 2024-04-26T21:12:44.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.427+0000 7f151f1f0640 1 -- 172.21.15.40:0/603967526 wait complete. 2024-04-26T21:12:44.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:44.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 Processor -- start 2024-04-26T21:12:44.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 -- start start 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa9815f370 0x7ffa980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffa980d3ee0 0x7ffa980b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa980d03a0 con 0x7ffa9815f750 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffa980b4d10 con 0x7ffa9815f370 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.701+0000 7ffa9dfaf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffa980d0680 con 0x7ffa980d3ee0 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa9815f370 0x7ffa980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa9815f370 0x7ffa980cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54588/0 (socket says 172.21.15.40:54588) 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 -- 172.21.15.40:0/466087400 learned_addr learned my addr 172.21.15.40:0/466087400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 -- 172.21.15.40:0/466087400 >> v2:172.21.15.184:3300/0 conn(0x7ffa980d3ee0 msgr2=0x7ffa980b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 --2- 172.21.15.40:0/466087400 >> v2:172.21.15.184:3300/0 conn(0x7ffa980d3ee0 0x7ffa980b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 -- 172.21.15.40:0/466087400 >> v1:172.21.15.184:6789/0 conn(0x7ffa9815f750 legacy=0x7ffa980d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 -- 172.21.15.40:0/466087400 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa980d0960 con 0x7ffa9815f370 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.702+0000 7ffa977fe640 1 --2- 172.21.15.40:0/466087400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f370 0x7ffa980cfd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffa980c8cf0 tx=0x7ffa8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed6367ca7d4507 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa967fc640 1 -- 172.21.15.40:0/466087400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88037070 con 0x7ffa9815f370 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa967fc640 1 -- 172.21.15.40:0/466087400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88032440 con 0x7ffa9815f370 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa967fc640 1 -- 172.21.15.40:0/466087400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88032810 con 0x7ffa9815f370 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/466087400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f370 msgr2=0x7ffa980cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/466087400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f370 0x7ffa980cfd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffa980c8cf0 tx=0x7ffa8802a230 comp rx=0 tx=0).stop 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/466087400 shutdown_connections 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/466087400 >> v2:172.21.15.184:3300/0 conn(0x7ffa980d3ee0 0x7ffa980b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/466087400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f370 0x7ffa980cfd70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/466087400 >> 172.21.15.40:0/466087400 conn(0x7ffa9808ec10 msgr2=0x7ffa9809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/466087400 shutdown_connections 2024-04-26T21:12:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.703+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/466087400 wait complete. 2024-04-26T21:12:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.704+0000 7ffa9dfaf640 1 Processor -- start 2024-04-26T21:12:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 -- start start 2024-04-26T21:12:44.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa980d3ee0 0x7ffa9815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9815f370 0x7ffa980a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa980b51c0 con 0x7ffa9815f750 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa980d0f20 con 0x7ffa9815f370 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa9dfaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa980b5340 con 0x7ffa980d3ee0 2024-04-26T21:12:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.705+0000 7ffa97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54600/0 (socket says 172.21.15.40:54600) 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 -- 172.21.15.40:0/1152843718 learned_addr learned my addr 172.21.15.40:0/1152843718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa96ffd640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9815f370 0x7ffa980a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa977fe640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa980d3ee0 0x7ffa9815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 -- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa980d3ee0 msgr2=0x7ffa9815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa980d3ee0 0x7ffa9815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 -- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9815f370 msgr2=0x7ffa980a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9815f370 0x7ffa980a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 -- 172.21.15.40:0/1152843718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9815c180 con 0x7ffa9815f750 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa97fff640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b970 tx=0x7ffa9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90019070 con 0x7ffa9815f750 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9000cb60 con 0x7ffa9815f750 2024-04-26T21:12:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa981f5360 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90014440 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.706+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa981f5920 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.707+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9806a290 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.708+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa9001c070 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.708+0000 7ffa8ffff640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 0x7ffa7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.708+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7ffa9008b8f0 con 0x7ffa9815f750 2024-04-26T21:12:44.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.708+0000 7ffa977fe640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 0x7ffa7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.709+0000 7ffa977fe640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 0x7ffa7806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa88034000 tx=0x7ffa8802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.712+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa90059c90 con 0x7ffa9815f750 2024-04-26T21:12:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.936+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa980d3ab0 con 0x7ffa9815f750 2024-04-26T21:12:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.936+0000 7ffa8ffff640 1 -- 172.21.15.40:0/1152843718 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7ffa9008b070 con 0x7ffa9815f750 2024-04-26T21:12:44.937 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2994732898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2994732898},{"type":"v1","addr":"172.21.15.184:6836","nonce":2994732898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2818017058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2818017058},{"type":"v1","addr":"172.21.15.40:6833","nonce":2818017058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2576066961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2576066961},{"type":"v1","addr":"172.21.15.40:6835","nonce":2576066961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:01.103924+0000","modified":"2024-04-26T21:12:03.117727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2363826920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2363826920},{"type":"v1","addr":"172.21.15.184:6837","nonce":2363826920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-26T21:12:44.938 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 msgr2=0x7ffa7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 0x7ffa7806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa88034000 tx=0x7ffa8802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 msgr2=0x7ffa980a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b970 tx=0x7ffa9000be30 comp rx=0 tx=0).stop 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 shutdown_connections 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa7806c8f0 0x7ffa7806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa9815f750 0x7ffa980a2be0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9815f370 0x7ffa980a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 --2- 172.21.15.40:0/1152843718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa980d3ee0 0x7ffa9815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 >> 172.21.15.40:0/1152843718 conn(0x7ffa9808ec10 msgr2=0x7ffa9809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.938+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 shutdown_connections 2024-04-26T21:12:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:44.939+0000 7ffa9dfaf640 1 -- 172.21.15.40:0/1152843718 wait complete. 2024-04-26T21:12:44.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:45.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.051+0000 7fe5a5e64640 1 Processor -- start 2024-04-26T21:12:45.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.051+0000 7fe5a5e64640 1 -- start start 2024-04-26T21:12:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe5a5e64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5a015f340 0x7fe5a00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe5a5e64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 0x7fe5a00d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe5a5e64640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a00b2510 con 0x7fe5a015f720 2024-04-26T21:12:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe5a5e64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a00b27f0 con 0x7fe5a015f340 2024-04-26T21:12:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe5a5e64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a00b2ad0 con 0x7fe5a00d5610 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe5a015f340 0x7fe5a00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5a015f720 0x7fe5a00d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35006/0 (socket says 172.21.15.40:35006) 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59effd640 1 -- 172.21.15.40:0/2889802542 learned_addr learned my addr 172.21.15.40:0/2889802542 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59ffff640 1 --2- 172.21.15.40:0/2889802542 >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 0x7fe5a00d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59f7fe640 1 -- 172.21.15.40:0/2889802542 >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 msgr2=0x7fe5a00d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/2889802542 >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 0x7fe5a00d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59f7fe640 1 -- 172.21.15.40:0/2889802542 >> v1:172.21.15.40:6789/0 conn(0x7fe5a015f720 legacy=0x7fe5a00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59f7fe640 1 -- 172.21.15.40:0/2889802542 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a00b2db0 con 0x7fe5a015f340 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.052+0000 7fe59ffff640 1 --2- 172.21.15.40:0/2889802542 >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 0x7fe5a00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.053+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/2889802542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f340 0x7fe5a00b1e80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d4c20 tx=0x7fe59002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa410dae00d0ba20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.053+0000 7fe59e7fc640 1 -- 172.21.15.40:0/2889802542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590037070 con 0x7fe5a015f340 2024-04-26T21:12:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.053+0000 7fe59e7fc640 1 -- 172.21.15.40:0/2889802542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe590032440 con 0x7fe5a015f340 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.053+0000 7fe59e7fc640 1 -- 172.21.15.40:0/2889802542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59002ae60 con 0x7fe5a015f340 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 -- 172.21.15.40:0/2889802542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f340 msgr2=0x7fe5a00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/2889802542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f340 0x7fe5a00b1e80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d4c20 tx=0x7fe59002a110 comp rx=0 tx=0).stop 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 -- 172.21.15.40:0/2889802542 shutdown_connections 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/2889802542 >> v2:172.21.15.184:3300/0 conn(0x7fe5a00d5610 0x7fe5a00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/2889802542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f340 0x7fe5a00b1e80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 -- 172.21.15.40:0/2889802542 >> 172.21.15.40:0/2889802542 conn(0x7fe5a008ec70 msgr2=0x7fe5a009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 -- 172.21.15.40:0/2889802542 shutdown_connections 2024-04-26T21:12:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.054+0000 7fe5a5e64640 1 -- 172.21.15.40:0/2889802542 wait complete. 2024-04-26T21:12:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.055+0000 7fe5a5e64640 1 Processor -- start 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.055+0000 7fe5a5e64640 1 -- start start 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 0x7fe5a0151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a015f340 0x7fe5a01f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00b3940 con 0x7fe5a015f720 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00b37c0 con 0x7fe5a015f340 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe5a5e64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a00b3ac0 con 0x7fe5a00d5610 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54612/0 (socket says 172.21.15.40:54612) 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 -- 172.21.15.40:0/3256181137 learned_addr learned my addr 172.21.15.40:0/3256181137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59effd640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a015f340 0x7fe5a01f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 0x7fe5a0151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 -- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 msgr2=0x7fe5a0151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 0x7fe5a0151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 -- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a015f340 msgr2=0x7fe5a01f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a015f340 0x7fe5a01f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 -- 172.21.15.40:0/3256181137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a00c9b50 con 0x7fe5a015f720 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 0x7fe5a0151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.056+0000 7fe59ffff640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe59800cea0 tx=0x7fe59800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.057+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598003b90 con 0x7fe5a015f720 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.057+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe598003d50 con 0x7fe5a015f720 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.057+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a00c9e30 con 0x7fe5a015f720 2024-04-26T21:12:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.057+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598004260 con 0x7fe5a015f720 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.057+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a00ca3f0 con 0x7fe5a015f720 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.058+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a00b60c0 con 0x7fe5a015f720 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.058+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe598004400 con 0x7fe5a015f720 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.058+0000 7fe587fff640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.059+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7fe59808c5d0 con 0x7fe5a015f720 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.059+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.059+0000 7fe59f7fe640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 0x7fe58c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe590034000 tx=0x7fe59002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.063+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5a00b60c0 con 0x7fe5a015f720 2024-04-26T21:12:45.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.248+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5a006a290 con 0x7fe5a015f720 2024-04-26T21:12:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.249+0000 7fe587fff640 1 -- 172.21.15.40:0/3256181137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16901 (secure 0 0 0) 0x7fe59805a980 con 0x7fe5a015f720 2024-04-26T21:12:45.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:45.251 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:42.019368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:11:58.923694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:00.058755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 msgr2=0x7fe58c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 0x7fe58c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe590034000 tx=0x7fe59002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 msgr2=0x7fe5a01f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe59800cea0 tx=0x7fe59800cf90 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 shutdown_connections 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5a015f720 0x7fe5a01f7f40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a015f340 0x7fe5a01f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 --2- 172.21.15.40:0/3256181137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5a00d5610 0x7fe5a0151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 >> 172.21.15.40:0/3256181137 conn(0x7fe5a008ec70 msgr2=0x7fe5a00b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 shutdown_connections 2024-04-26T21:12:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.251+0000 7fe5a5e64640 1 -- 172.21.15.40:0/3256181137 wait complete. 2024-04-26T21:12:45.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:12:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.364+0000 7f573e322640 1 Processor -- start 2024-04-26T21:12:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.364+0000 7f573e322640 1 -- start start 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573e322640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815f720 0x7f57380b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573e322640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57380d5610 0x7f57380d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573e322640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57380b2510 con 0x7f573815f340 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573e322640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f57380b27f0 con 0x7f57380d5610 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573e322640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57380b2ad0 con 0x7f573815f720 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f5737fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f573815f340 0x7f57380d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35016/0 (socket says 172.21.15.40:35016) 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f5737fff640 1 -- 172.21.15.40:0/4233300016 learned_addr learned my addr 172.21.15.40:0/4233300016 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f573c8bf640 1 --2- 172.21.15.40:0/4233300016 >> v2:172.21.15.40:3300/0 conn(0x7f57380d5610 0x7f57380d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.365+0000 7f57377fe640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815f720 0x7f57380b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f573c8bf640 1 -- 172.21.15.40:0/4233300016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815f720 msgr2=0x7f57380b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f573c8bf640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815f720 0x7f57380b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f573c8bf640 1 -- 172.21.15.40:0/4233300016 >> v1:172.21.15.40:6789/0 conn(0x7f573815f340 legacy=0x7f57380d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f573c8bf640 1 -- 172.21.15.40:0/4233300016 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57380b2db0 con 0x7f57380d5610 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f5736ffd640 1 -- 172.21.15.40:0/4233300016 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884034360 0 0) 0x7f57380b2510 con 0x7f573815f340 2024-04-26T21:12:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f573c8bf640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57380d5610 0x7f57380d3d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5730009d90 tx=0x7f573002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bbd592a89867f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.366+0000 7f5736ffd640 1 -- 172.21.15.40:0/4233300016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730031600 con 0x7f57380d5610 2024-04-26T21:12:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f5736ffd640 1 -- 172.21.15.40:0/4233300016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57300317a0 con 0x7f57380d5610 2024-04-26T21:12:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f5736ffd640 1 -- 172.21.15.40:0/4233300016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730031b70 con 0x7f57380d5610 2024-04-26T21:12:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 -- 172.21.15.40:0/4233300016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57380d5610 msgr2=0x7f57380d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57380d5610 0x7f57380d3d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5730009d90 tx=0x7f573002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 -- 172.21.15.40:0/4233300016 shutdown_connections 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57380d5610 0x7f57380d3d40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 --2- 172.21.15.40:0/4233300016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815f720 0x7f57380b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 -- 172.21.15.40:0/4233300016 >> 172.21.15.40:0/4233300016 conn(0x7f573808ec70 msgr2=0x7f573809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 -- 172.21.15.40:0/4233300016 shutdown_connections 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.367+0000 7f573e322640 1 -- 172.21.15.40:0/4233300016 wait complete. 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.368+0000 7f573e322640 1 Processor -- start 2024-04-26T21:12:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.368+0000 7f573e322640 1 -- start start 2024-04-26T21:12:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380d5610 0x7f57380c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f573815f340 0x7f57380ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57380b3980 con 0x7f573815f720 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57380b3800 con 0x7f573815f340 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573e322640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57380b3b00 con 0x7f57380d5610 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573c8bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573c8bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54634/0 (socket says 172.21.15.40:54634) 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f573c8bf640 1 -- 172.21.15.40:0/2729673659 learned_addr learned my addr 172.21.15.40:0/2729673659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f5737fff640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380d5610 0x7f57380c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.369+0000 7f57377fe640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f573815f340 0x7f57380ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 -- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380d5610 msgr2=0x7f57380c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380d5610 0x7f57380c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 -- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f573815f340 msgr2=0x7f57380ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f573815f340 0x7f57380ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 -- 172.21.15.40:0/2729673659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57380cf260 con 0x7f573815f720 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573c8bf640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5730009eb0 tx=0x7f5730004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730040070 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5730030b60 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57380ca540 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730009450 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.370+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57380cab00 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.371+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573806a290 con 0x7f573815f720 2024-04-26T21:12:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.372+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5730039040 con 0x7f573815f720 2024-04-26T21:12:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.372+0000 7f5734ff9640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.372+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f57300ad180 con 0x7f573815f720 2024-04-26T21:12:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.372+0000 7f5737fff640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.373+0000 7f5737fff640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 0x7f572406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f572c000f60 tx=0x7f572c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.376+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57380cf260 con 0x7f573815f720 2024-04-26T21:12:45.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.600+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f57380d3020 con 0x7f573815f720 2024-04-26T21:12:45.602 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:12:45.600+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:12:45.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.602+0000 7f5734ff9640 1 -- 172.21.15.40:0/2729673659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f573007aed0 con 0x7f573815f720 2024-04-26T21:12:45.603 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 msgr2=0x7f572406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 0x7f572406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f572c000f60 tx=0x7f572c007d50 comp rx=0 tx=0).stop 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 msgr2=0x7f57380cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5730009eb0 tx=0x7f5730004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 shutdown_connections 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f573815f720 0x7f57380cb5e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f573815f340 0x7f57380ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 --2- 172.21.15.40:0/2729673659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380d5610 0x7f57380c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 >> 172.21.15.40:0/2729673659 conn(0x7f573808ec70 msgr2=0x7f57380b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:45.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 shutdown_connections 2024-04-26T21:12:45.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.604+0000 7f573e322640 1 -- 172.21.15.40:0/2729673659 wait complete. 2024-04-26T21:12:45.609 INFO:tasks.ceph.mds.b.smithi184.stderr: -8735> 2024-04-26T21:11:54.429+0000 7fddedc06640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:12:45.611 INFO:tasks.ceph.mds.b.smithi184.stderr: -7530> 2024-04-26T21:12:05.429+0000 7fdde7bfa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:12:45.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:12:45.645 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:12:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.718+0000 7fae83283640 1 Processor -- start 2024-04-26T21:12:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 -- start start 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae7c0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae7c163aa0 0x7fae7c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c0d3f50 con 0x7fae7c0b60b0 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fae7c15fa60 con 0x7fae7c163aa0 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae83283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c0b5090 con 0x7fae7c15f680 2024-04-26T21:12:45.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae81820640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae7c0b60b0 0x7fae7c0d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35028/0 (socket says 172.21.15.40:35028) 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.719+0000 7fae81820640 1 -- 172.21.15.40:0/1163024104 learned_addr learned my addr 172.21.15.40:0/1163024104 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> v2:172.21.15.184:3300/0 conn(0x7fae7c163aa0 0x7fae7c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae8101f640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae7c0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803789153 0 0) 0x7fae7c0d3f50 con 0x7fae7c0b60b0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae7c0b60b0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2985902816 0 0) 0x7fae64003520 con 0x7fae7c0b60b0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 msgr2=0x7fae7c0d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae7c0d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 >> v2:172.21.15.184:3300/0 conn(0x7fae7c163aa0 msgr2=0x7fae7c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 --2- 172.21.15.40:0/1163024104 >> v2:172.21.15.184:3300/0 conn(0x7fae7c163aa0 0x7fae7c0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c0b5370 con 0x7fae7c0b60b0 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> v2:172.21.15.184:3300/0 conn(0x7fae7c163aa0 0x7fae7c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fae6c0019f0 con 0x7fae7c0b60b0 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 >> v1:172.21.15.40:6789/0 conn(0x7fae7c0b60b0 legacy=0x7fae7c0d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae640039b0 0x7fae64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c15fa60 con 0x7fae64004800 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0b5090 con 0x7fae640039b0 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400b9c0 con 0x7fae7c15f680 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae8101f640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae640039b0 0x7fae64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.720+0000 7fae81820640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 -- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 msgr2=0x7fae640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c15f680 0x7fae640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 -- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae640039b0 msgr2=0x7fae64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae640039b0 0x7fae64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 -- 172.21.15.40:0/1163024104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003520 con 0x7fae64004800 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae8081e640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae64004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae70009c90 tx=0x7fae64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c078fc138bfa2e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70036070 con 0x7fae64004800 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae70031400 con 0x7fae64004800 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.721+0000 7fae7bfff640 1 -- 172.21.15.40:0/1163024104 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70009860 con 0x7fae64004800 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 -- 172.21.15.40:0/1163024104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 msgr2=0x7fae64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 --2- 172.21.15.40:0/1163024104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae64004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae70009c90 tx=0x7fae64005830 comp rx=0 tx=0).stop 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae81820640 1 -- 172.21.15.40:0/1163024104 reap_dead start 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 -- 172.21.15.40:0/1163024104 shutdown_connections 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 -- 172.21.15.40:0/1163024104 >> 172.21.15.40:0/1163024104 conn(0x7fae7c08eca0 msgr2=0x7fae7c09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 -- 172.21.15.40:0/1163024104 shutdown_connections 2024-04-26T21:12:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.722+0000 7fae83283640 1 -- 172.21.15.40:0/1163024104 wait complete. 2024-04-26T21:12:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.723+0000 7fae83283640 1 Processor -- start 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.723+0000 7fae83283640 1 -- start start 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c0c9b60 0x7fae7c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 0x7fae7c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0d57b0 con 0x7fae64004800 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0d5630 con 0x7fae7c0cd8d0 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae83283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c0d5930 con 0x7fae7c0c9b60 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54646/0 (socket says 172.21.15.40:54646) 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 -- 172.21.15.40:0/622508055 learned_addr learned my addr 172.21.15.40:0/622508055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8081e640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c0c9b60 0x7fae7c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae81820640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 0x7fae7c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 -- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c0c9b60 msgr2=0x7fae7c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c0c9b60 0x7fae7c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 -- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 msgr2=0x7fae7c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 0x7fae7c0cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae8101f640 1 -- 172.21.15.40:0/622508055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c153070 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.724+0000 7fae81820640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 0x7fae7c0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae8101f640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae740118e0 tx=0x7fae74011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7400faf0 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7400fc90 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c0ce520 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74014440 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.725+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c0ceae0 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.726+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c0cb820 con 0x7fae64004800 2024-04-26T21:12:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.726+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae74012040 con 0x7fae64004800 2024-04-26T21:12:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.727+0000 7fae79ffb640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.727+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5940+0+0 (secure 0 0 0) 0x7fae7408c110 con 0x7fae64004800 2024-04-26T21:12:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.727+0000 7fae8081e640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.728+0000 7fae8081e640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 0x7fae6806ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fae70033000 tx=0x7fae70008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.731+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae74019030 con 0x7fae64004800 2024-04-26T21:12:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:45.956+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fae7c06a290 con 0x7fae64004800 2024-04-26T21:12:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.604+0000 7fae79ffb640 1 -- 172.21.15.40:0/622508055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7fae7405a4e0 con 0x7fae64004800 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.606+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 msgr2=0x7fae6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.606+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 0x7fae6806ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fae70033000 tx=0x7fae70008030 comp rx=0 tx=0).stop 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.606+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 msgr2=0x7fae7c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.606+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae740118e0 tx=0x7fae74011da0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 shutdown_connections 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae7c0cd8d0 0x7fae7c0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae7c0c9b60 0x7fae7c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 --2- 172.21.15.40:0/622508055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae64004800 0x7fae7c0c9620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 >> 172.21.15.40:0/622508055 conn(0x7fae7c08eca0 msgr2=0x7fae7c163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 shutdown_connections 2024-04-26T21:12:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.607+0000 7fae83283640 1 -- 172.21.15.40:0/622508055 wait complete. 2024-04-26T21:12:46.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:12:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.731+0000 7fc06a26d640 1 Processor -- start 2024-04-26T21:12:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.731+0000 7fc06a26d640 1 -- start start 2024-04-26T21:12:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc06a26d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc0640d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc06a26d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0640d33a0 con 0x7fc064162a10 2024-04-26T21:12:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc06a26d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0640d3680 con 0x7fc0640d2140 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc06a26d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0640b4970 con 0x7fc0640b6020 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc063fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc064162a10 0x7fc0640d1a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35044/0 (socket says 172.21.15.40:35044) 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc063fff640 1 -- 172.21.15.40:0/201780186 learned_addr learned my addr 172.21.15.40:0/201780186 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.732+0000 7fc06880a640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc0640d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894276227 0 0) 0x7fc0640d33a0 con 0x7fc064162a10 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04c003520 con 0x7fc064162a10 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3312315879 0 0) 0x7fc04c003520 con 0x7fc064162a10 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 msgr2=0x7fc0640d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc0640d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 >> v1:172.21.15.184:6789/0 conn(0x7fc0640d2140 legacy=0x7fc0640cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0640d3960 con 0x7fc064162a10 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc058001670 con 0x7fc064162a10 2024-04-26T21:12:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 >> v1:172.21.15.40:6789/0 conn(0x7fc064162a10 legacy=0x7fc0640d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc04c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc04c004390 0x7fc04c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc04c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00b9c0 con 0x7fc04c0039e0 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00bb40 con 0x7fc04c004390 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c00bd40 con 0x7fc0640b6020 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc04c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc0637fe640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc04c004390 0x7fc04c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc06880a640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc04c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 -- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 msgr2=0x7fc04c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0640b6020 0x7fc04c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 -- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc04c004390 msgr2=0x7fc04c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc04c004390 0x7fc04c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.733+0000 7fc063fff640 1 -- 172.21.15.40:0/201780186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c003520 con 0x7fc04c0039e0 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.734+0000 7fc063fff640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc04c003dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c290 tx=0x7fc04c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a39668ad412dcdc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.734+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058007d50 con 0x7fc04c0039e0 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.734+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc058005af0 con 0x7fc04c0039e0 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.734+0000 7fc062ffd640 1 -- 172.21.15.40:0/201780186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058007070 con 0x7fc04c0039e0 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 -- 172.21.15.40:0/201780186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 msgr2=0x7fc04c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 --2- 172.21.15.40:0/201780186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc04c003dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c290 tx=0x7fc04c005830 comp rx=0 tx=0).stop 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06880a640 1 -- 172.21.15.40:0/201780186 reap_dead start 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 -- 172.21.15.40:0/201780186 shutdown_connections 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 -- 172.21.15.40:0/201780186 >> 172.21.15.40:0/201780186 conn(0x7fc06409e530 msgr2=0x7fc0640d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 -- 172.21.15.40:0/201780186 shutdown_connections 2024-04-26T21:12:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.735+0000 7fc06a26d640 1 -- 172.21.15.40:0/201780186 wait complete. 2024-04-26T21:12:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.736+0000 7fc06a26d640 1 Processor -- start 2024-04-26T21:12:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.736+0000 7fc06a26d640 1 -- start start 2024-04-26T21:12:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 0x7fc0640c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 0x7fc0641f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc064162df0 con 0x7fc04c0039e0 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0640d3f20 con 0x7fc0641f8720 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06a26d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0640b6600 con 0x7fc0641f92c0 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc063fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc063fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54652/0 (socket says 172.21.15.40:54652) 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc063fff640 1 -- 172.21.15.40:0/1454012053 learned_addr learned my addr 172.21.15.40:0/1454012053 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc06880a640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 0x7fc0641f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.737+0000 7fc0637fe640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 0x7fc0640c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 -- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 msgr2=0x7fc0640c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 0x7fc0640c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 -- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 msgr2=0x7fc0641f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 0x7fc0641f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 -- 172.21.15.40:0/1454012053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0640cb160 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc06880a640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 0x7fc0641f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc0637fe640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 0x7fc0640c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc063fff640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c380 tx=0x7fc058001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058042070 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc058007480 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0640cb3d0 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058007d50 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.738+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0640cb990 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.739+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc064160000 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.740+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc058045070 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.740+0000 7fc060ff9640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.740+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5940+0+0 (secure 0 0 0) 0x7fc0580ac7c0 con 0x7fc04c0039e0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.740+0000 7fc0637fe640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.741+0000 7fc0637fe640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc054000c00 tx=0x7fc054009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:46.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.744+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc05807a4b0 con 0x7fc04c0039e0 2024-04-26T21:12:46.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:46.929+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc06406a290 con 0x7fc04c0039e0 2024-04-26T21:12:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.611+0000 7fc060ff9640 1 -- 172.21.15.40:0/1454012053 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) ==== 181+0+0 (secure 0 0 0) 0x7fc05807e7b0 con 0x7fc04c0039e0 2024-04-26T21:12:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:12:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.613+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 msgr2=0x7fc05006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.613+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 0x7fc05006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc054000c00 tx=0x7fc054009040 comp rx=0 tx=0).stop 2024-04-26T21:12:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.613+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 msgr2=0x7fc0640ba650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.613+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c380 tx=0x7fc058001a40 comp rx=0 tx=0).stop 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 shutdown_connections 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc05006c8f0 0x7fc05006edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0641f8720 0x7fc0641f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0641f92c0 0x7fc0640c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 --2- 172.21.15.40:0/1454012053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc04c0039e0 0x7fc0640ba650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 >> 172.21.15.40:0/1454012053 conn(0x7fc06409e530 msgr2=0x7fc0640cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 shutdown_connections 2024-04-26T21:12:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.614+0000 7fc06a26d640 1 -- 172.21.15.40:0/1454012053 wait complete. 2024-04-26T21:12:47.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:12:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.740+0000 7fb43867e640 1 Processor -- start 2024-04-26T21:12:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.740+0000 7fb43867e640 1 -- start start 2024-04-26T21:12:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb430162a10 0x7fb4300d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb4300d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4300d1f70 0x7fb430163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb4300b4970 con 0x7fb4300d1f70 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4300d33a0 con 0x7fb430162a10 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43867e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4300d3680 con 0x7fb4300b6020 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb436c1b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4300d1f70 0x7fb430163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb436c1b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4300d1f70 0x7fb430163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54664/0 (socket says 172.21.15.40:54664) 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb436c1b640 1 -- 172.21.15.40:0/811385542 learned_addr learned my addr 172.21.15.40:0/811385542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb435c19640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb4300d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.741+0000 7fb43641a640 1 --2- 172.21.15.40:0/811385542 >> v2:172.21.15.184:3300/0 conn(0x7fb430162a10 0x7fb4300d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 -- 172.21.15.40:0/811385542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 msgr2=0x7fb4300d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb4300d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 -- 172.21.15.40:0/811385542 >> v2:172.21.15.184:3300/0 conn(0x7fb430162a10 msgr2=0x7fb4300d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 --2- 172.21.15.40:0/811385542 >> v2:172.21.15.184:3300/0 conn(0x7fb430162a10 0x7fb4300d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 -- 172.21.15.40:0/811385542 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4300d3960 con 0x7fb4300d1f70 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb436c1b640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb430163100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb43008eda0 tx=0x7fb42802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5391b83db9e891 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.742+0000 7fb435418640 1 -- 172.21.15.40:0/811385542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428036070 con 0x7fb4300d1f70 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb435418640 1 -- 172.21.15.40:0/811385542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb428031600 con 0x7fb4300d1f70 2024-04-26T21:12:47.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb435418640 1 -- 172.21.15.40:0/811385542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280319d0 con 0x7fb4300d1f70 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 -- 172.21.15.40:0/811385542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 msgr2=0x7fb430163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb430163100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb43008eda0 tx=0x7fb42802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 -- 172.21.15.40:0/811385542 shutdown_connections 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb430163100 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 --2- 172.21.15.40:0/811385542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb4300d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 --2- 172.21.15.40:0/811385542 >> v2:172.21.15.184:3300/0 conn(0x7fb430162a10 0x7fb4300d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 -- 172.21.15.40:0/811385542 >> 172.21.15.40:0/811385542 conn(0x7fb43009e530 msgr2=0x7fb4300d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 -- 172.21.15.40:0/811385542 shutdown_connections 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.743+0000 7fb43867e640 1 -- 172.21.15.40:0/811385542 wait complete. 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.744+0000 7fb43867e640 1 Processor -- start 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.744+0000 7fb43867e640 1 -- start start 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb430159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 0x7fb43015c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb430162df0 con 0x7fb4300d1f70 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4300d3f20 con 0x7fb430162a10 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43867e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4300b6600 con 0x7fb4300b6020 2024-04-26T21:12:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb435c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb435c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54670/0 (socket says 172.21.15.40:54670) 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb435c19640 1 -- 172.21.15.40:0/186517054 learned_addr learned my addr 172.21.15.40:0/186517054 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb43641a640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb430159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.745+0000 7fb436c1b640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 0x7fb43015c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 -- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 msgr2=0x7fb430159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb430159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 -- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 msgr2=0x7fb43015c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 0x7fb43015c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 -- 172.21.15.40:0/186517054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4300ccc60 con 0x7fb4300d1f70 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb43641a640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb430159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb436c1b640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 0x7fb43015c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb435c19640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb42000d5f0 tx=0x7fb42000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200125f0 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb420019070 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4300ccf40 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200129d0 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.746+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4300b8770 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.747+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43006a290 con 0x7fb4300d1f70 2024-04-26T21:12:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.747+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb420012b70 con 0x7fb4300d1f70 2024-04-26T21:12:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.748+0000 7fb426ffd640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 0x7fb41406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.748+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5502+0+0 (secure 0 0 0) 0x7fb42008be90 con 0x7fb4300d1f70 2024-04-26T21:12:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.748+0000 7fb43641a640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 0x7fb41406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.749+0000 7fb43641a640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 0x7fb41406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006fd0 tx=0x7fb42c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.752+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb42005a390 con 0x7fb4300d1f70 2024-04-26T21:12:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:47.943+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb4300b6ac0 con 0x7fb4300d1f70 2024-04-26T21:12:48.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.612+0000 7fb426ffd640 1 -- 172.21.15.40:0/186517054 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) ==== 169+0+0 (secure 0 0 0) 0x7fb420059c90 con 0x7fb4300d1f70 2024-04-26T21:12:48.614 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:12:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.616+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 msgr2=0x7fb41406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.616+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 0x7fb41406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006fd0 tx=0x7fb42c008040 comp rx=0 tx=0).stop 2024-04-26T21:12:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.616+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 msgr2=0x7fb4300cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:48.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.616+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb42000d5f0 tx=0x7fb42000dab0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 shutdown_connections 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb41406c8f0 0x7fb41406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb430162a10 0x7fb43015c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4300d1f70 0x7fb4300cc2c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 --2- 172.21.15.40:0/186517054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300b6020 0x7fb430159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.620+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 >> 172.21.15.40:0/186517054 conn(0x7fb43009e530 msgr2=0x7fb4300d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:48.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.621+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 shutdown_connections 2024-04-26T21:12:48.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.621+0000 7fb43867e640 1 -- 172.21.15.40:0/186517054 wait complete. 2024-04-26T21:12:48.640 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:12:48.640 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:12:48.640 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:12:48.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:12:48.641 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:12:48.641 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:12:48.641 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:12:48.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:12:48.641 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:12:48.641 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:12:48.641 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:12:48.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:12:48.642 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:12:48.642 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:12:48.642 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:12:48.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:12:48.651 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:12:48.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:12:48.652 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:12:48.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:12:48.653 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:12:48.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:12:48.654 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:12:48.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:12:48.694 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:12:48.695 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:48.695 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:12:48.696 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:48.697 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:12:48.697 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:48.697 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:12:48.697 DEBUG:teuthology.parallel:result is None 2024-04-26T21:12:48.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:12:48.773 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:12:48.774 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:12:48.775 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:12:48.775 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:12:48.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.814+0000 7f7fe98d1640 1 Processor -- start 2024-04-26T21:12:48.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.814+0000 7f7fe98d1640 1 -- start start 2024-04-26T21:12:48.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe98d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7fe415f370 0x7f7fe40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe98d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe98d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe40d03a0 con 0x7f7fe415f750 2024-04-26T21:12:48.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe98d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe40b4d10 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe98d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe40d0680 con 0x7f7fe415f370 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54720/0 (socket says 172.21.15.40:54720) 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 -- 172.21.15.40:0/2232548000 learned_addr learned my addr 172.21.15.40:0/2232548000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 -- 172.21.15.40:0/2232548000 >> v2:172.21.15.184:3300/0 conn(0x7f7fe415f370 msgr2=0x7f7fe40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 --2- 172.21.15.40:0/2232548000 >> v2:172.21.15.184:3300/0 conn(0x7f7fe415f370 0x7f7fe40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 -- 172.21.15.40:0/2232548000 >> v1:172.21.15.184:6789/0 conn(0x7f7fe415f750 legacy=0x7f7fe40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:48.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.815+0000 7f7fe37fe640 1 -- 172.21.15.40:0/2232548000 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe40d0960 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.816+0000 7f7fe37fe640 1 --2- 172.21.15.40:0/2232548000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40d2d10 tx=0x7f7fdc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad2583503f25423 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:48.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.816+0000 7f7fe1ffb640 1 -- 172.21.15.40:0/2232548000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc036070 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.816+0000 7f7fe1ffb640 1 -- 172.21.15.40:0/2232548000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc031600 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.816+0000 7f7fe1ffb640 1 -- 172.21.15.40:0/2232548000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0319d0 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 -- 172.21.15.40:0/2232548000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe40d3ee0 msgr2=0x7f7fe40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/2232548000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40d2d10 tx=0x7f7fdc02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 -- 172.21.15.40:0/2232548000 shutdown_connections 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/2232548000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe40d3ee0 0x7f7fe40b47d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/2232548000 >> v2:172.21.15.184:3300/0 conn(0x7f7fe415f370 0x7f7fe40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 -- 172.21.15.40:0/2232548000 >> 172.21.15.40:0/2232548000 conn(0x7f7fe408ec10 msgr2=0x7f7fe409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 -- 172.21.15.40:0/2232548000 shutdown_connections 2024-04-26T21:12:48.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.817+0000 7f7fe98d1640 1 -- 172.21.15.40:0/2232548000 wait complete. 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.818+0000 7f7fe98d1640 1 Processor -- start 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.818+0000 7f7fe98d1640 1 -- start start 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 0x7f7fe41f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 0x7f7fe40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe40b51c0 con 0x7f7fe415f370 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe40d0f20 con 0x7f7fe415f750 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe98d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe40b5340 con 0x7f7fe40d3ee0 2024-04-26T21:12:48.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54730/0 (socket says 172.21.15.40:54730) 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 -- 172.21.15.40:0/329553164 learned_addr learned my addr 172.21.15.40:0/329553164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe2ffd640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 0x7f7fe41f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe37fe640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 0x7f7fe40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 -- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 msgr2=0x7f7fe41f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 0x7f7fe41f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 -- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 msgr2=0x7f7fe40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 0x7f7fe40ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 -- 172.21.15.40:0/329553164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41f95c0 con 0x7f7fe415f370 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe2ffd640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 0x7f7fe41f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe37fe640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 0x7f7fe40ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.819+0000 7f7fe27fc640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b3e0 tx=0x7f7fd400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:48.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.820+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4015020 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.820+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4011850 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.820+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011c40 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.820+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41f5190 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.820+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41f5750 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.821+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe406a290 con 0x7f7fe415f370 2024-04-26T21:12:48.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.821+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fd4018020 con 0x7f7fe415f370 2024-04-26T21:12:48.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.822+0000 7f7fcbfff640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 0x7f7fd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:48.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.822+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4943+0+0 (secure 0 0 0) 0x7f7fd408b6a0 con 0x7f7fe415f370 2024-04-26T21:12:48.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.822+0000 7f7fe2ffd640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 0x7f7fd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:48.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.822+0000 7f7fe2ffd640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 0x7f7fd006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009930 tx=0x7f7fd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:48.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:48.826+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fd401f020 con 0x7f7fe415f370 2024-04-26T21:12:49.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.023+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7fe40cf690 con 0x7f7fe415f370 2024-04-26T21:12:49.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.630+0000 7f7fcbfff640 1 -- 172.21.15.40:0/329553164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7f7fe41f95c0 con 0x7f7fe415f370 2024-04-26T21:12:49.631 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 msgr2=0x7f7fd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 0x7f7fd006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009930 tx=0x7f7fd8008040 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 msgr2=0x7f7fe415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b3e0 tx=0x7f7fd400b8a0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 shutdown_connections 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fd006c920 0x7f7fd006ede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe415f750 0x7f7fe40ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe415f370 0x7f7fe415d090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 --2- 172.21.15.40:0/329553164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fe40d3ee0 0x7f7fe41f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.633+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 >> 172.21.15.40:0/329553164 conn(0x7f7fe408ec10 msgr2=0x7f7fe415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.634+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 shutdown_connections 2024-04-26T21:12:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.634+0000 7f7fe98d1640 1 -- 172.21.15.40:0/329553164 wait complete. 2024-04-26T21:12:49.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:12:49.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.755+0000 7fcda0bb4640 1 Processor -- start 2024-04-26T21:12:49.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.755+0000 7fcda0bb4640 1 -- start start 2024-04-26T21:12:49.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd9c13ff70 0x7fcd9c146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd9c140350 0x7fcd9c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0cf810 0x7fcd9c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c0d05b0 con 0x7fcd9c13ff70 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c0d0890 con 0x7fcd9c140350 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcda0bb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c06e450 con 0x7fcd9c0cf810 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd9c13ff70 0x7fcd9c146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd9c13ff70 0x7fcd9c146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54736/0 (socket says 172.21.15.40:54736) 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 -- 172.21.15.40:0/1270059443 learned_addr learned my addr 172.21.15.40:0/1270059443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9ad9d640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0cf810 0x7fcd9c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd93fff640 1 --2- 172.21.15.40:0/1270059443 >> v2:172.21.15.184:3300/0 conn(0x7fcd9c140350 0x7fcd9c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 -- 172.21.15.40:0/1270059443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0cf810 msgr2=0x7fcd9c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0cf810 0x7fcd9c0cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 -- 172.21.15.40:0/1270059443 >> v2:172.21.15.184:3300/0 conn(0x7fcd9c140350 msgr2=0x7fcd9c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1270059443 >> v2:172.21.15.184:3300/0 conn(0x7fcd9c140350 0x7fcd9c1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.756+0000 7fcd9a59c640 1 -- 172.21.15.40:0/1270059443 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c06e730 con 0x7fcd9c13ff70 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.757+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c146080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0cf1c0 tx=0x7fcd8c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965266ff1e88a872 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.757+0000 7fcd99d9b640 1 -- 172.21.15.40:0/1270059443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c037070 con 0x7fcd9c13ff70 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.757+0000 7fcd99d9b640 1 -- 172.21.15.40:0/1270059443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c032440 con 0x7fcd9c13ff70 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.757+0000 7fcd99d9b640 1 -- 172.21.15.40:0/1270059443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c032810 con 0x7fcd9c13ff70 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1270059443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 msgr2=0x7fcd9c146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c146080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0cf1c0 tx=0x7fcd8c02a140 comp rx=0 tx=0).stop 2024-04-26T21:12:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1270059443 shutdown_connections 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c0cf810 0x7fcd9c0cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1270059443 >> v2:172.21.15.184:3300/0 conn(0x7fcd9c140350 0x7fcd9c1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1270059443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c146080 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1270059443 >> 172.21.15.40:0/1270059443 conn(0x7fcd9c0d0d30 msgr2=0x7fcd9c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1270059443 shutdown_connections 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.758+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1270059443 wait complete. 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.759+0000 7fcda0bb4640 1 Processor -- start 2024-04-26T21:12:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.759+0000 7fcda0bb4640 1 -- start start 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0cf810 0x7fcd9c1f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c140350 0x7fcd9c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c142a40 con 0x7fcd9c13ff70 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c1428c0 con 0x7fcd9c0cf810 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcda0bb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c142bc0 con 0x7fcd9c140350 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54738/0 (socket says 172.21.15.40:54738) 2024-04-26T21:12:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 -- 172.21.15.40:0/1840034494 learned_addr learned my addr 172.21.15.40:0/1840034494 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0cf810 0x7fcd9c1f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd9ad9d640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c140350 0x7fcd9c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 -- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c140350 msgr2=0x7fcd9c073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c140350 0x7fcd9c073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 -- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0cf810 msgr2=0x7fcd9c1f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0cf810 0x7fcd9c1f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.760+0000 7fcd93fff640 1 -- 172.21.15.40:0/1840034494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c0ae8b0 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcd93fff640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e870 tx=0x7fcd8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84017070 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8400c940 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c1f60b0 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400cd30 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.761+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c0af110 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.762+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c06a290 con 0x7fcd9c13ff70 2024-04-26T21:12:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.762+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd8401a070 con 0x7fcd9c13ff70 2024-04-26T21:12:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.763+0000 7fcd92ffd640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.763+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fcd8408b590 con 0x7fcd9c13ff70 2024-04-26T21:12:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.763+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.764+0000 7fcd9a59c640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:49.767+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd840596c0 con 0x7fcd9c13ff70 2024-04-26T21:12:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.023+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcd9c140c60 con 0x7fcd9c13ff70 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.024+0000 7fcd92ffd640 1 -- 172.21.15.40:0/1840034494 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7fcd8405d9c0 con 0x7fcd9c13ff70 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:12:50.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:12:50.027 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.030 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.031 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:12:50.034 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:12:50.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmmAHKLhAAIoJ8gvdWnvGqY6QRSi2GgA==", 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmiQrKLhAAwInk0lq32yLSetItbAEFlQ==", 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmABLKLhAAU6JEAVETpXNtO5ti9tYXEg==", 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T21:12:50.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmNRnKLhAAqLys5OqKzpUxEa5OGxpJBA==", 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmjiDKLhAAlaGjYVCuifluOGMp/oiaIA==", 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T21:12:50.038 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCIGCxmIyjKLhAAfgflKFGq41IiPDbAVmD1bQ==", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:12:50.040 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 msgr2=0x7fcd7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 msgr2=0x7fcd9c0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e870 tx=0x7fcd8400ed30 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 shutdown_connections 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd7c06c8f0 0x7fcd7c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd9c140350 0x7fcd9c073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd9c13ff70 0x7fcd9c0afbf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 --2- 172.21.15.40:0/1840034494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd9c0cf810 0x7fcd9c1f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 >> 172.21.15.40:0/1840034494 conn(0x7fcd9c0d0d30 msgr2=0x7fcd9c06f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 shutdown_connections 2024-04-26T21:12:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.026+0000 7fcda0bb4640 1 -- 172.21.15.40:0/1840034494 wait complete. 2024-04-26T21:12:50.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T21:12:50.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.143+0000 7f199dcbd640 1 Processor -- start 2024-04-26T21:12:50.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.143+0000 7f199dcbd640 1 -- start start 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f199dcbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f199dcbd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1998160d50 0x7f19981611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f199dcbd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19980d0be0 con 0x7f199815d380 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f199dcbd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f19980d0ec0 con 0x7f1998160d50 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f199dcbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19980d11a0 con 0x7f19980b56d0 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f1996ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f1997fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1998160d50 0x7f19981611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f1996ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:55684/0 (socket says 172.21.15.40:55684) 2024-04-26T21:12:50.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.144+0000 7f1996ffd640 1 -- 172.21.15.40:0/1937081220 learned_addr learned my addr 172.21.15.40:0/1937081220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1997fff640 1 -- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 msgr2=0x7f19980d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1997fff640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1997fff640 1 -- 172.21.15.40:0/1937081220 >> v1:172.21.15.184:6789/0 conn(0x7f199815d380 legacy=0x7f19980ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1997fff640 1 -- 172.21.15.40:0/1937081220 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19980d1480 con 0x7f1998160d50 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1996ffd640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.145+0000 7f1997fff640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1998160d50 0x7f19981611e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f199002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3c4a01834e94bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f19967fc640 1 -- 172.21.15.40:0/1937081220 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990007740 con 0x7f1998160d50 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f19967fc640 1 -- 172.21.15.40:0/1937081220 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19900078e0 con 0x7f1998160d50 2024-04-26T21:12:50.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f19967fc640 1 -- 172.21.15.40:0/1937081220 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990007cb0 con 0x7f1998160d50 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 -- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1998160d50 msgr2=0x7f19981611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1998160d50 0x7f19981611e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f199002a660 comp rx=0 tx=0).stop 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 -- 172.21.15.40:0/1937081220 shutdown_connections 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1998160d50 0x7f19981611e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1937081220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19980b56d0 0x7f19980d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 -- 172.21.15.40:0/1937081220 >> 172.21.15.40:0/1937081220 conn(0x7f19980b4a60 msgr2=0x7f19980b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 -- 172.21.15.40:0/1937081220 shutdown_connections 2024-04-26T21:12:50.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.146+0000 7f199dcbd640 1 -- 172.21.15.40:0/1937081220 wait complete. 2024-04-26T21:12:50.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.147+0000 7f199dcbd640 1 Processor -- start 2024-04-26T21:12:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 -- start start 2024-04-26T21:12:50.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19980b56d0 0x7f19981522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199815d380 0x7f199806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19980b5dc0 con 0x7f1998160d50 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19980cedc0 con 0x7f19980b56d0 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f199dcbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19980b5f40 con 0x7f199815d380 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f1997fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f1997fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54748/0 (socket says 172.21.15.40:54748) 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.148+0000 7f1997fff640 1 -- 172.21.15.40:0/1814274502 learned_addr learned my addr 172.21.15.40:0/1814274502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1996ffd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199815d380 0x7f199806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f19977fe640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19980b56d0 0x7f19981522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 -- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199815d380 msgr2=0x7f199806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199815d380 0x7f199806ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 -- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19980b56d0 msgr2=0x7f19981522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19980b56d0 0x7f19981522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 -- 172.21.15.40:0/1814274502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1998152810 con 0x7f1998160d50 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f1997fff640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f1990004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990040070 con 0x7f1998160d50 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199815dbb0 con 0x7f1998160d50 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990031bc0 con 0x7f1998160d50 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990009450 con 0x7f1998160d50 2024-04-26T21:12:50.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.149+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199815e170 con 0x7f1998160d50 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.150+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199806a290 con 0x7f1998160d50 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.151+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1990039040 con 0x7f1998160d50 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.151+0000 7f197ffff640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.151+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f19900ac780 con 0x7f1998160d50 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.151+0000 7f19977fe640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.152+0000 7f19977fe640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 0x7f198406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f198c001780 tx=0x7f198c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.155+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f199007a830 con 0x7f1998160d50 2024-04-26T21:12:50.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.411+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f1998076090 con 0x7f1998160d50 2024-04-26T21:12:50.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.413+0000 7f197ffff640 1 -- 172.21.15.40:0/1814274502 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f199007eb30 con 0x7f1998160d50 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 msgr2=0x7f198406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 0x7f198406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f198c001780 tx=0x7f198c009040 comp rx=0 tx=0).stop 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 msgr2=0x7f1998071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1990009d90 tx=0x7f1990004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 shutdown_connections 2024-04-26T21:12:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998160d50 0x7f1998071380 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f199815d380 0x7f199806ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 --2- 172.21.15.40:0/1814274502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19980b56d0 0x7f19981522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 >> 172.21.15.40:0/1814274502 conn(0x7f19980b4a60 msgr2=0x7f199815aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 shutdown_connections 2024-04-26T21:12:50.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.415+0000 7f199dcbd640 1 -- 172.21.15.40:0/1814274502 wait complete. 2024-04-26T21:12:50.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T21:12:50.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.527+0000 7f6e136c4640 1 Processor -- start 2024-04-26T21:12:50.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 -- start start 2024-04-26T21:12:50.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 0x7f6e0c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e0c162f30 0x7f6e0c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e0c06cbc0 con 0x7f6e0c15fa20 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0c163df0 con 0x7f6e0c162f30 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e136c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c163830 con 0x7f6e0c06c2a0 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.528+0000 7f6e11c61640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e0c162f30 0x7f6e0c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e0c162f30 0x7f6e0c06fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54764/0 (socket says 172.21.15.40:54764) 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 -- 172.21.15.40:0/3759693249 learned_addr learned my addr 172.21.15.40:0/3759693249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 -- 172.21.15.40:0/3759693249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 msgr2=0x7f6e0c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11460640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 0x7f6e0c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 0x7f6e0c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 -- 172.21.15.40:0/3759693249 >> v1:172.21.15.184:6789/0 conn(0x7f6e0c15fa20 legacy=0x7f6e0c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11c61640 1 -- 172.21.15.40:0/3759693249 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c163b10 con 0x7f6e0c162f30 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.529+0000 7f6e11460640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 0x7f6e0c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e11c61640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c06fb40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009e50 tx=0x7f6dfc02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b5184a07c9370f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e0bfff640 1 -- 172.21.15.40:0/3759693249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc036070 con 0x7f6e0c162f30 2024-04-26T21:12:50.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e0bfff640 1 -- 172.21.15.40:0/3759693249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc031600 con 0x7f6e0c162f30 2024-04-26T21:12:50.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e0bfff640 1 -- 172.21.15.40:0/3759693249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0319d0 con 0x7f6e0c162f30 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 -- 172.21.15.40:0/3759693249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 msgr2=0x7f6e0c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c06fb40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009e50 tx=0x7f6dfc02a5b0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 -- 172.21.15.40:0/3759693249 shutdown_connections 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c06fb40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3759693249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c06c2a0 0x7f6e0c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 -- 172.21.15.40:0/3759693249 >> 172.21.15.40:0/3759693249 conn(0x7f6e0c0b3b10 msgr2=0x7f6e0c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 -- 172.21.15.40:0/3759693249 shutdown_connections 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.530+0000 7f6e136c4640 1 -- 172.21.15.40:0/3759693249 wait complete. 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.531+0000 7f6e136c4640 1 Processor -- start 2024-04-26T21:12:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 -- start start 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c06c2a0 0x7f6e0c15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c15fa20 0x7f6e0c155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c1606a0 con 0x7f6e0c162f30 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0cec30 con 0x7f6e0c06c2a0 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e136c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c160820 con 0x7f6e0c15fa20 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e11c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e11c61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54768/0 (socket says 172.21.15.40:54768) 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.532+0000 7f6e11c61640 1 -- 172.21.15.40:0/3217618395 learned_addr learned my addr 172.21.15.40:0/3217618395 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11460640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c06c2a0 0x7f6e0c15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e10c5f640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c15fa20 0x7f6e0c155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 -- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c15fa20 msgr2=0x7f6e0c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c15fa20 0x7f6e0c155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 -- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c06c2a0 msgr2=0x7f6e0c15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c06c2a0 0x7f6e0c15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 -- 172.21.15.40:0/3217618395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0b6990 con 0x7f6e0c162f30 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e11c61640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009e50 tx=0x7f6dfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc041020 con 0x7f6e0c162f30 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc0077a0 con 0x7f6e0c162f30 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0b58f0 con 0x7f6e0c162f30 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc036040 con 0x7f6e0c162f30 2024-04-26T21:12:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.533+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c0b5eb0 con 0x7f6e0c162f30 2024-04-26T21:12:50.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.534+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c0693d0 con 0x7f6e0c162f30 2024-04-26T21:12:50.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.535+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dfc044020 con 0x7f6e0c162f30 2024-04-26T21:12:50.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.535+0000 7f6e09ffb640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 0x7f6df806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.535+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f6dfc0ac790 con 0x7f6e0c162f30 2024-04-26T21:12:50.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.535+0000 7f6e11460640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 0x7f6df806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.536+0000 7f6e11460640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 0x7f6df806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c071120 tx=0x7f6e04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.540+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dfc04b030 con 0x7f6e0c162f30 2024-04-26T21:12:50.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.795+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f6e0c0d2a70 con 0x7f6e0c162f30 2024-04-26T21:12:50.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.799+0000 7f6e09ffb640 1 -- 172.21.15.40:0/3217618395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f6dfc07afd0 con 0x7f6e0c162f30 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 msgr2=0x7f6df806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 0x7f6df806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c071120 tx=0x7f6e04008040 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 msgr2=0x7f6e0c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009e50 tx=0x7f6dfc004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 shutdown_connections 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6df806c8f0 0x7f6df806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e0c162f30 0x7f6e0c156510 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e0c15fa20 0x7f6e0c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 --2- 172.21.15.40:0/3217618395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c06c2a0 0x7f6e0c15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 >> 172.21.15.40:0/3217618395 conn(0x7f6e0c0b3b10 msgr2=0x7f6e0c0d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 shutdown_connections 2024-04-26T21:12:50.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.801+0000 7f6e136c4640 1 -- 172.21.15.40:0/3217618395 wait complete. 2024-04-26T21:12:50.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T21:12:50.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.914+0000 7f074b699640 1 Processor -- start 2024-04-26T21:12:50.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.914+0000 7f074b699640 1 -- start start 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f074b699640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 0x7f074406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f074b699640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f074406e490 0x7f074415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f074b699640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07440d1740 con 0x7f07440d11b0 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f074b699640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f07440d1a20 con 0x7f074406e490 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f074b699640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07440d1d00 con 0x7f07440cf2e0 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f074406e490 0x7f074415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f074406e490 0x7f074415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37860/0 (socket says 172.21.15.40:37860) 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 -- 172.21.15.40:0/375957140 learned_addr learned my addr 172.21.15.40:0/375957140 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 -- 172.21.15.40:0/375957140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 msgr2=0x7f074406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749435640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 0x7f074406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 0x7f074406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 -- 172.21.15.40:0/375957140 >> v1:172.21.15.184:6789/0 conn(0x7f07440d11b0 legacy=0x7f074406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749c36640 1 -- 172.21.15.40:0/375957140 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07440d1fe0 con 0x7f074406e490 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.915+0000 7f0749435640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 0x7f074406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f0749c36640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074406e490 0x7f074415ebe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f073c009e50 tx=0x7f073c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ea2966b2e37c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f073bfff640 1 -- 172.21.15.40:0/375957140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c036070 con 0x7f074406e490 2024-04-26T21:12:50.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f073bfff640 1 -- 172.21.15.40:0/375957140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073c031600 con 0x7f074406e490 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f073bfff640 1 -- 172.21.15.40:0/375957140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c0319d0 con 0x7f074406e490 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f074b699640 1 -- 172.21.15.40:0/375957140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074406e490 msgr2=0x7f074415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f074b699640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074406e490 0x7f074415ebe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f073c009e50 tx=0x7f073c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f074b699640 1 -- 172.21.15.40:0/375957140 shutdown_connections 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.916+0000 7f074b699640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074406e490 0x7f074415ebe0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.917+0000 7f074b699640 1 --2- 172.21.15.40:0/375957140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440cf2e0 0x7f074406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.917+0000 7f074b699640 1 -- 172.21.15.40:0/375957140 >> 172.21.15.40:0/375957140 conn(0x7f07440b4a60 msgr2=0x7f07440b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:50.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.917+0000 7f074b699640 1 -- 172.21.15.40:0/375957140 shutdown_connections 2024-04-26T21:12:50.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.917+0000 7f074b699640 1 -- 172.21.15.40:0/375957140 wait complete. 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 Processor -- start 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 -- start start 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074406e490 0x7f07440a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440d11b0 0x7f074409ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07440cf6c0 con 0x7f07440cf2e0 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f074406da10 con 0x7f074406e490 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.918+0000 7f074b699640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f07440cf840 con 0x7f07440d11b0 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37862/0 (socket says 172.21.15.40:37862) 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 -- 172.21.15.40:0/2315747638 learned_addr learned my addr 172.21.15.40:0/2315747638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0749435640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074406e490 0x7f07440a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0749c36640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440d11b0 0x7f074409ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 -- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440d11b0 msgr2=0x7f074409ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440d11b0 0x7f074409ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 -- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074406e490 msgr2=0x7f07440a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074406e490 0x7f07440a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 -- 172.21.15.40:0/2315747638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07440a0dd0 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0748c34640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f073400b3f0 tx=0x7f073400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734015020 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0734011830 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074409cf40 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.919+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734011c20 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.920+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07440c9630 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.921+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07440cc760 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.921+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0734018020 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.922+0000 7f0739ffb640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 0x7f072806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.922+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f073408b390 con 0x7f07440cf2e0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.922+0000 7f0749435640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 0x7f072806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.922+0000 7f0749435640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 0x7f072806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f074000aac0 tx=0x7f0740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:50.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:50.926+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07440cc760 con 0x7f07440cf2e0 2024-04-26T21:12:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.183+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f0744163920 con 0x7f07440cf2e0 2024-04-26T21:12:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.186+0000 7f0739ffb640 1 -- 172.21.15.40:0/2315747638 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f073401f030 con 0x7f07440cf2e0 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 msgr2=0x7f072806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 0x7f072806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f074000aac0 tx=0x7f0740008040 comp rx=0 tx=0).stop 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 msgr2=0x7f07440c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f073400b3f0 tx=0x7f073400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 shutdown_connections 2024-04-26T21:12:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f072806c8f0 0x7f072806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07440d11b0 0x7f074409ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07440cf2e0 0x7f07440c8870 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 --2- 172.21.15.40:0/2315747638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f074406e490 0x7f07440a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 >> 172.21.15.40:0/2315747638 conn(0x7f07440b4a60 msgr2=0x7f07440cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 shutdown_connections 2024-04-26T21:12:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.188+0000 7f074b699640 1 -- 172.21.15.40:0/2315747638 wait complete. 2024-04-26T21:12:51.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T21:12:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.300+0000 7fd1ed0f1640 1 Processor -- start 2024-04-26T21:12:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.300+0000 7fd1ed0f1640 1 -- start start 2024-04-26T21:12:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1ed0f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1e80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1ed0f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e80b2510 con 0x7fd1e815f720 2024-04-26T21:12:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1ed0f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e80b27f0 con 0x7fd1e815f340 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1ed0f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e80b2ad0 con 0x7fd1e80d5610 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd1e815f720 0x7fd1e80b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42738/0 (socket says 172.21.15.40:42738) 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e659c640 1 -- 172.21.15.40:0/864855793 learned_addr learned my addr 172.21.15.40:0/864855793 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e759e640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1e80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204004736 0 0) 0x7fd1e80b2510 con 0x7fd1e815f720 2024-04-26T21:12:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1d0003520 con 0x7fd1e815f720 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4005358114 0 0) 0x7fd1d0003520 con 0x7fd1e815f720 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 msgr2=0x7fd1e80b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1e80b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 >> v1:172.21.15.184:6789/0 conn(0x7fd1e815f340 legacy=0x7fd1e80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.301+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e80b2db0 con 0x7fd1e815f720 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd1dc001670 con 0x7fd1e815f720 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 >> v1:172.21.15.40:6789/0 conn(0x7fd1e815f720 legacy=0x7fd1e80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d00039e0 0x7fd1d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e80b2ad0 con 0x7fd1d0004800 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d000baf0 con 0x7fd1d00039e0 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d000bcf0 con 0x7fd1e80d5610 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e6d9d640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d00039e0 0x7fd1d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e759e640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 -- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 msgr2=0x7fd1d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80d5610 0x7fd1d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 -- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d00039e0 msgr2=0x7fd1d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d00039e0 0x7fd1d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e659c640 1 -- 172.21.15.40:0/864855793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d0003520 con 0x7fd1d0004800 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.302+0000 7fd1e6d9d640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d00039e0 0x7fd1d0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1e659c640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c210 tx=0x7fd1d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ada59a9ff14ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007d10 con 0x7fd1d0004800 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0058d0 con 0x7fd1d0004800 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1e5d9b640 1 -- 172.21.15.40:0/864855793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005d00 con 0x7fd1d0004800 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/864855793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d0004800 msgr2=0x7fd1d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/864855793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c210 tx=0x7fd1d0005830 comp rx=0 tx=0).stop 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1e759e640 1 -- 172.21.15.40:0/864855793 reap_dead start 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/864855793 shutdown_connections 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/864855793 >> 172.21.15.40:0/864855793 conn(0x7fd1e808ec70 msgr2=0x7fd1e809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.303+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/864855793 shutdown_connections 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.304+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/864855793 wait complete. 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.305+0000 7fd1ed0f1640 1 Processor -- start 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.305+0000 7fd1ed0f1640 1 -- start start 2024-04-26T21:12:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d0004800 0x7fd1e814d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e815dd90 0x7fd1e81e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e80b2210 con 0x7fd1e81e7480 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e80b2090 con 0x7fd1d0004800 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1ed0f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e80d3d40 con 0x7fd1e815dd90 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37888/0 (socket says 172.21.15.40:37888) 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 -- 172.21.15.40:0/3362857904 learned_addr learned my addr 172.21.15.40:0/3362857904 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e6d9d640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d0004800 0x7fd1e814d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e659c640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e815dd90 0x7fd1e81e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 -- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e815dd90 msgr2=0x7fd1e81e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e815dd90 0x7fd1e81e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 -- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d0004800 msgr2=0x7fd1e814d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d0004800 0x7fd1e814d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 -- 172.21.15.40:0/3362857904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e81e7900 con 0x7fd1e81e7480 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.306+0000 7fd1e759e640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80d5aa0 tx=0x7fd1d800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.307+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003990 con 0x7fd1e81e7480 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.307+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d801c070 con 0x7fd1e81e7480 2024-04-26T21:12:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.307+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003d50 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.307+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e81e7be0 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.307+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e81e81a0 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.308+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e806a290 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.308+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1d8004030 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.309+0000 7fd1cf7fe640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 0x7fd1d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.309+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fd1d808c0a0 con 0x7fd1e81e7480 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.309+0000 7fd1e6d9d640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 0x7fd1d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.309+0000 7fd1e6d9d640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 0x7fd1d406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007fd0 tx=0x7fd1e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.313+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1d8010040 con 0x7fd1e81e7480 2024-04-26T21:12:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.579+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd1e80ba880 con 0x7fd1e81e7480 2024-04-26T21:12:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.582+0000 7fd1cf7fe640 1 -- 172.21.15.40:0/3362857904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fd1d805a1d0 con 0x7fd1e81e7480 2024-04-26T21:12:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 msgr2=0x7fd1d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 0x7fd1d406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007fd0 tx=0x7fd1e0009040 comp rx=0 tx=0).stop 2024-04-26T21:12:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 msgr2=0x7fd1e81e92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80d5aa0 tx=0x7fd1d800f580 comp rx=0 tx=0).stop 2024-04-26T21:12:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 shutdown_connections 2024-04-26T21:12:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1d406c8f0 0x7fd1d406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1e81e7480 0x7fd1e81e92e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e815dd90 0x7fd1e81e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 --2- 172.21.15.40:0/3362857904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d0004800 0x7fd1e814d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 >> 172.21.15.40:0/3362857904 conn(0x7fd1e808ec70 msgr2=0x7fd1e80d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 shutdown_connections 2024-04-26T21:12:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.584+0000 7fd1ed0f1640 1 -- 172.21.15.40:0/3362857904 wait complete. 2024-04-26T21:12:51.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T21:12:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.699+0000 7f9ad1b24640 1 Processor -- start 2024-04-26T21:12:51.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.699+0000 7f9ad1b24640 1 -- start start 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 0x7f9acc0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0d4f00 0x7f9acc0d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc0b2510 con 0x7f9acc0d4f00 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc0b27f0 con 0x7f9acc15f340 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9ad1b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0b2ad0 con 0x7f9acc15f720 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0d4f00 0x7f9acc0d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0d4f00 0x7f9acc0d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37902/0 (socket says 172.21.15.40:37902) 2024-04-26T21:12:51.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 -- 172.21.15.40:0/1588498298 learned_addr learned my addr 172.21.15.40:0/1588498298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/1588498298 >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 0x7f9acc0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acaffd640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 -- 172.21.15.40:0/1588498298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 msgr2=0x7f9acc0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 -- 172.21.15.40:0/1588498298 >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 msgr2=0x7f9acc0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 --2- 172.21.15.40:0/1588498298 >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 0x7f9acc0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acbfff640 1 -- 172.21.15.40:0/1588498298 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc0b2db0 con 0x7f9acc0d4f00 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/1588498298 >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 0x7f9acc0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.700+0000 7f9acaffd640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.701+0000 7f9acbfff640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0d4f00 0x7f9acc0d5350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0d6000 tx=0x7f9ac402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac4ec82e1f5c0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.701+0000 7f9aca7fc640 1 -- 172.21.15.40:0/1588498298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4036070 con 0x7f9acc0d4f00 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.701+0000 7f9aca7fc640 1 -- 172.21.15.40:0/1588498298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac4031600 con 0x7f9acc0d4f00 2024-04-26T21:12:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.701+0000 7f9aca7fc640 1 -- 172.21.15.40:0/1588498298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40319d0 con 0x7f9acc0d4f00 2024-04-26T21:12:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 -- 172.21.15.40:0/1588498298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0d4f00 msgr2=0x7f9acc0d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0d4f00 0x7f9acc0d5350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0d6000 tx=0x7f9ac402a610 comp rx=0 tx=0).stop 2024-04-26T21:12:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 -- 172.21.15.40:0/1588498298 shutdown_connections 2024-04-26T21:12:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0d4f00 0x7f9acc0d5350 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/1588498298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/1588498298 >> v2:172.21.15.184:3300/0 conn(0x7f9acc15f340 0x7f9acc0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 -- 172.21.15.40:0/1588498298 >> 172.21.15.40:0/1588498298 conn(0x7f9acc08ec70 msgr2=0x7f9acc09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 -- 172.21.15.40:0/1588498298 shutdown_connections 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.702+0000 7f9ad1b24640 1 -- 172.21.15.40:0/1588498298 wait complete. 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.703+0000 7f9ad1b24640 1 Processor -- start 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.703+0000 7f9ad1b24640 1 -- start start 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9acc0d4f00 0x7f9acc158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0b5040 con 0x7f9acc15f340 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0b4ec0 con 0x7f9acc0d4f00 2024-04-26T21:12:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ad1b24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0b51c0 con 0x7f9acc15f720 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37912/0 (socket says 172.21.15.40:37912) 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 -- 172.21.15.40:0/2146916587 learned_addr learned my addr 172.21.15.40:0/2146916587 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9acc0d4f00 0x7f9acc158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acbfff640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 -- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 msgr2=0x7f9acc0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 -- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9acc0d4f00 msgr2=0x7f9acc158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9acc0d4f00 0x7f9acc158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 -- 172.21.15.40:0/2146916587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc0ce8e0 con 0x7f9acc15f340 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9acaffd640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d7d0 tx=0x7f9abc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc004030 con 0x7f9acc15f340 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.704+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc0041d0 con 0x7f9acc15f340 2024-04-26T21:12:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.705+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc15fc90 con 0x7f9acc15f340 2024-04-26T21:12:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.705+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc009640 con 0x7f9acc15f340 2024-04-26T21:12:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.705+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc160250 con 0x7f9acc15f340 2024-04-26T21:12:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.705+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc06a290 con 0x7f9acc15f340 2024-04-26T21:12:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.706+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9abc0097e0 con 0x7f9acc15f340 2024-04-26T21:12:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.706+0000 7f9ab3fff640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 0x7f9ab806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.707+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f9abc08b8d0 con 0x7f9acc15f340 2024-04-26T21:12:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.707+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 0x7f9ab806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.707+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 0x7f9ab806ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009950 tx=0x7f9ac0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:51.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.711+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9abc059a00 con 0x7f9acc15f340 2024-04-26T21:12:51.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.968+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f9acc0b5f20 con 0x7f9acc15f340 2024-04-26T21:12:51.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.970+0000 7f9ab3fff640 1 -- 172.21.15.40:0/2146916587 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f9abc05dd00 con 0x7f9acc15f340 2024-04-26T21:12:51.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 msgr2=0x7f9ab806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 0x7f9ab806ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009950 tx=0x7f9ac0008040 comp rx=0 tx=0).stop 2024-04-26T21:12:51.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 msgr2=0x7f9acc0cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d7d0 tx=0x7f9abc00dc90 comp rx=0 tx=0).stop 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 shutdown_connections 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ab806c920 0x7f9ab806ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9acc15f720 0x7f9acc0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc15f340 0x7f9acc0cdd90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 --2- 172.21.15.40:0/2146916587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9acc0d4f00 0x7f9acc158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 >> 172.21.15.40:0/2146916587 conn(0x7f9acc08ec70 msgr2=0x7f9acc0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 shutdown_connections 2024-04-26T21:12:51.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:51.972+0000 7f9ad1b24640 1 -- 172.21.15.40:0/2146916587 wait complete. 2024-04-26T21:12:51.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T21:12:52.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.084+0000 7f234b5a4640 1 Processor -- start 2024-04-26T21:12:52.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.084+0000 7f234b5a4640 1 -- start start 2024-04-26T21:12:52.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f234b5a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2344146790 0x7f23440cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f234b5a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23440d05b0 con 0x7f2344140350 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f234b5a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23440d0890 con 0x7f234413ff70 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f234b5a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f234406e450 con 0x7f2344146790 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f2348b3f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2344140350 0x7f23440cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42746/0 (socket says 172.21.15.40:42746) 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f2348b3f640 1 -- 172.21.15.40:0/1801261121 learned_addr learned my addr 172.21.15.40:0/1801261121 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f2349b41640 1 --2- 172.21.15.40:0/1801261121 >> v2:172.21.15.184:3300/0 conn(0x7f2344146790 0x7f23440cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3799233153 0 0) 0x7f23440d05b0 con 0x7f2344140350 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2330003520 con 0x7f2344140350 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.085+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3460555753 0 0) 0x7f2330003520 con 0x7f2344140350 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 >> v2:172.21.15.184:3300/0 conn(0x7f2344146790 msgr2=0x7f23440cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 --2- 172.21.15.40:0/1801261121 >> v2:172.21.15.184:3300/0 conn(0x7f2344146790 0x7f23440cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 >> v1:172.21.15.184:6789/0 conn(0x7f234413ff70 legacy=0x7f2344146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234406e730 con 0x7f2344140350 2024-04-26T21:12:52.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f2334001670 con 0x7f2344140350 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 >> v1:172.21.15.40:6789/0 conn(0x7f2344140350 legacy=0x7f23440cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2330003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330005210 0x7f233000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f234406e450 con 0x7f23300039e0 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f233000c140 con 0x7f2330005210 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f233000c340 con 0x7f2330004800 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2330003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349b41640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330005210 0x7f233000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2348b3f640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 -- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330004800 msgr2=0x7f2330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 -- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330005210 msgr2=0x7f233000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2330005210 0x7f233000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349340640 1 -- 172.21.15.40:0/1801261121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2330003520 con 0x7f23300039e0 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.086+0000 7f2349b41640 1 -- 172.21.15.40:0/1801261121 reap_dead start 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f2349340640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2330003e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2340001750 tx=0x7f2330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df04b8b1e1e3eeb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340001370 con 0x7f23300039e0 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23400052d0 con 0x7f23300039e0 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f233bfff640 1 -- 172.21.15.40:0/1801261121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234002abc0 con 0x7f23300039e0 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f234b5a4640 1 -- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 msgr2=0x7f2330003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f234b5a4640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2330003e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2340001750 tx=0x7f2330005830 comp rx=0 tx=0).stop 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f234b5a4640 1 -- 172.21.15.40:0/1801261121 shutdown_connections 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f234b5a4640 1 --2- 172.21.15.40:0/1801261121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2330003e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.087+0000 7f234b5a4640 1 -- 172.21.15.40:0/1801261121 >> 172.21.15.40:0/1801261121 conn(0x7f23440d0d30 msgr2=0x7f23440d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.088+0000 7f234b5a4640 1 -- 172.21.15.40:0/1801261121 shutdown_connections 2024-04-26T21:12:52.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.088+0000 7f234b5a4640 1 -- 172.21.15.40:0/1801261121 wait complete. 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.089+0000 7f234b5a4640 1 Processor -- start 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.089+0000 7f234b5a4640 1 -- start start 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.089+0000 7f234b5a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.089+0000 7f234b5a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 0x7f2344140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f234b5a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f234415a070 0x7f234415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f234b5a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2344146410 con 0x7f23300039e0 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f234b5a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2344146290 con 0x7f2344140b70 2024-04-26T21:12:52.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f234b5a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2344146590 con 0x7f234415a070 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37930/0 (socket says 172.21.15.40:37930) 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 -- 172.21.15.40:0/910051539 learned_addr learned my addr 172.21.15.40:0/910051539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2348b3f640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 0x7f2344140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349b41640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f234415a070 0x7f234415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 -- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f234415a070 msgr2=0x7f234415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f234415a070 0x7f234415a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 -- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 msgr2=0x7f2344140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 0x7f2344140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 -- 172.21.15.40:0/910051539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23441f2cf0 con 0x7f23300039e0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2348b3f640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 0x7f2344140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2349340640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f234002a570 tx=0x7f2340001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234003f030 con 0x7f23300039e0 2024-04-26T21:12:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2340005480 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340037040 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.090+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23441f2fd0 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.091+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23441f3590 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.092+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234406a290 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.092+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2340042070 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.092+0000 7f2339ffb640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 0x7f232806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.093+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f23400ab1a0 con 0x7f23300039e0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.093+0000 7f2348b3f640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 0x7f232806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.093+0000 7f2348b3f640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 0x7f232806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f234415ead0 tx=0x7f2334009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.097+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2340079920 con 0x7f23300039e0 2024-04-26T21:12:52.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.350+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f23440b45e0 con 0x7f23300039e0 2024-04-26T21:12:52.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.352+0000 7f2339ffb640 1 -- 172.21.15.40:0/910051539 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f2340079220 con 0x7f23300039e0 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 msgr2=0x7f232806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 0x7f232806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f234415ead0 tx=0x7f2334009040 comp rx=0 tx=0).stop 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 msgr2=0x7f2344158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f234002a570 tx=0x7f2340001120 comp rx=0 tx=0).stop 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 shutdown_connections 2024-04-26T21:12:52.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f232806c920 0x7f232806ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f234415a070 0x7f234415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2344140b70 0x7f2344140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 --2- 172.21.15.40:0/910051539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23300039e0 0x7f2344158c40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.354+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 >> 172.21.15.40:0/910051539 conn(0x7f23440d0d30 msgr2=0x7f2344160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.355+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 shutdown_connections 2024-04-26T21:12:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.355+0000 7f234b5a4640 1 -- 172.21.15.40:0/910051539 wait complete. 2024-04-26T21:12:52.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:52.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.466+0000 7f45a2c3e640 1 Processor -- start 2024-04-26T21:12:52.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.466+0000 7f45a2c3e640 1 -- start start 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a2c3e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0cf2e0 0x7f459c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a2c3e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f459c06e490 0x7f459c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a2c3e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459c0d1740 con 0x7f459c0d11b0 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a2c3e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f459c0d1a20 con 0x7f459c06e490 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a2c3e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0d1d00 con 0x7f459c0cf2e0 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f459c06e490 0x7f459c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f459c06e490 0x7f459c15ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37942/0 (socket says 172.21.15.40:37942) 2024-04-26T21:12:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 -- 172.21.15.40:0/289067887 learned_addr learned my addr 172.21.15.40:0/289067887 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 -- 172.21.15.40:0/289067887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0cf2e0 msgr2=0x7f459c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a09da640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0cf2e0 0x7f459c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0cf2e0 0x7f459c06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 -- 172.21.15.40:0/289067887 >> v1:172.21.15.184:6789/0 conn(0x7f459c0d11b0 legacy=0x7f459c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.467+0000 7f45a11db640 1 -- 172.21.15.40:0/289067887 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0d1fe0 con 0x7f459c06e490 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.468+0000 7f45a11db640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c15ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4594009e50 tx=0x7f459402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de57fbe0cb566699 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.468+0000 7f459b7fe640 1 -- 172.21.15.40:0/289067887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594036070 con 0x7f459c06e490 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.468+0000 7f459b7fe640 1 -- 172.21.15.40:0/289067887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4594031600 con 0x7f459c06e490 2024-04-26T21:12:52.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.468+0000 7f459b7fe640 1 -- 172.21.15.40:0/289067887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940319d0 con 0x7f459c06e490 2024-04-26T21:12:52.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 -- 172.21.15.40:0/289067887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 msgr2=0x7f459c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c15ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4594009e50 tx=0x7f459402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 -- 172.21.15.40:0/289067887 shutdown_connections 2024-04-26T21:12:52.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c15ebe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/289067887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0cf2e0 0x7f459c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 -- 172.21.15.40:0/289067887 >> 172.21.15.40:0/289067887 conn(0x7f459c0b4a60 msgr2=0x7f459c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 -- 172.21.15.40:0/289067887 shutdown_connections 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.469+0000 7f45a2c3e640 1 -- 172.21.15.40:0/289067887 wait complete. 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.470+0000 7f45a2c3e640 1 Processor -- start 2024-04-26T21:12:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.470+0000 7f45a2c3e640 1 -- start start 2024-04-26T21:12:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 0x7f459c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0d11b0 0x7f459c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0cf6c0 con 0x7f459c06e490 2024-04-26T21:12:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f459c06da10 con 0x7f459c0cf2e0 2024-04-26T21:12:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a2c3e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0cf840 con 0x7f459c0d11b0 2024-04-26T21:12:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37950/0 (socket says 172.21.15.40:37950) 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 -- 172.21.15.40:0/246629589 learned_addr learned my addr 172.21.15.40:0/246629589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a11db640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0d11b0 0x7f459c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f459bfff640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 0x7f459c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 -- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0d11b0 msgr2=0x7f459c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0d11b0 0x7f459c0c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 -- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 msgr2=0x7f459c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.471+0000 7f45a09da640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 0x7f459c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45a09da640 1 -- 172.21.15.40:0/246629589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0cb230 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f459bfff640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 0x7f459c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45a09da640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c7f0 tx=0x7f458c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c015070 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c004220 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c0cb510 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c007620 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.472+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c0cbad0 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.473+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c06a290 con 0x7f459c06e490 2024-04-26T21:12:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.473+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f458c0079d0 con 0x7f459c06e490 2024-04-26T21:12:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.474+0000 7f45997fa640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 0x7f458806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.474+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f458c08b750 con 0x7f459c06e490 2024-04-26T21:12:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.474+0000 7f459bfff640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 0x7f458806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.475+0000 7f459bfff640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 0x7f458806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.478+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f458c059880 con 0x7f459c06e490 2024-04-26T21:12:52.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.664+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f459c0cd3d0 con 0x7f459c06e490 2024-04-26T21:12:52.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.665+0000 7f45997fa640 1 -- 172.21.15.40:0/246629589 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+12891 (secure 0 0 0) 0x7f458c05db80 con 0x7f459c06e490 2024-04-26T21:12:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:52.666 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:49.638367+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 msgr2=0x7f458806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 0x7f458806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590008040 comp rx=0 tx=0).stop 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 msgr2=0x7f459c0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c7f0 tx=0x7f458c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 shutdown_connections 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f458806c920 0x7f458806ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f459c0d11b0 0x7f459c0c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f459c0cf2e0 0x7f459c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 --2- 172.21.15.40:0/246629589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f459c06e490 0x7f459c0c9840 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 >> 172.21.15.40:0/246629589 conn(0x7f459c0b4a60 msgr2=0x7f459c0cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 shutdown_connections 2024-04-26T21:12:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.667+0000 7f45a2c3e640 1 -- 172.21.15.40:0/246629589 wait complete. 2024-04-26T21:12:52.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:12:52.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:12:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.781+0000 7fb16d0bd640 1 Processor -- start 2024-04-26T21:12:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.781+0000 7fb16d0bd640 1 -- start start 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.782+0000 7fb16d0bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 0x7fb16806ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.782+0000 7fb16d0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.782+0000 7fb16d0bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16809c790 con 0x7fb1681631f0 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.782+0000 7fb16d0bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680d1600 con 0x7fb1680b2000 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.782+0000 7fb16d0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb168163ec0 con 0x7fb1680d1170 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16659c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 0x7fb16806ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56006/0 (socket says 172.21.15.40:56006) 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb166d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb1681631f0 0x7fb1681635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41890/0 (socket says 172.21.15.40:41890) 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 -- 172.21.15.40:0/2488115095 learned_addr learned my addr 172.21.15.40:0/2488115095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb165d9b640 1 -- 172.21.15.40:0/2488115095 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720529529 0 0) 0x7fb16809c790 con 0x7fb1681631f0 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb165d9b640 1 -- 172.21.15.40:0/2488115095 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb154003520 con 0x7fb1681631f0 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 -- 172.21.15.40:0/2488115095 >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 msgr2=0x7fb16806ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 --2- 172.21.15.40:0/2488115095 >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 0x7fb16806ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 -- 172.21.15.40:0/2488115095 >> v1:172.21.15.184:6789/0 conn(0x7fb1681631f0 legacy=0x7fb1681635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16759e640 1 -- 172.21.15.40:0/2488115095 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680d1a40 con 0x7fb1680d1170 2024-04-26T21:12:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.783+0000 7fb16659c640 1 --2- 172.21.15.40:0/2488115095 >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 0x7fb16806ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16759e640 1 --2- 172.21.15.40:0/2488115095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb158009e50 tx=0x7fb15802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f08315a89c3cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb165d9b640 1 -- 172.21.15.40:0/2488115095 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158036070 con 0x7fb1680d1170 2024-04-26T21:12:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb165d9b640 1 -- 172.21.15.40:0/2488115095 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158031600 con 0x7fb1680d1170 2024-04-26T21:12:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb165d9b640 1 -- 172.21.15.40:0/2488115095 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580319b0 con 0x7fb1680d1170 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 -- 172.21.15.40:0/2488115095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 msgr2=0x7fb1680b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/2488115095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb158009e50 tx=0x7fb15802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 -- 172.21.15.40:0/2488115095 shutdown_connections 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/2488115095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1680d1170 0x7fb1680b3300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/2488115095 >> v2:172.21.15.184:3300/0 conn(0x7fb1680b2000 0x7fb16806ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 -- 172.21.15.40:0/2488115095 >> 172.21.15.40:0/2488115095 conn(0x7fb16809ba40 msgr2=0x7fb16809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 -- 172.21.15.40:0/2488115095 shutdown_connections 2024-04-26T21:12:52.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.784+0000 7fb16d0bd640 1 -- 172.21.15.40:0/2488115095 wait complete. 2024-04-26T21:12:52.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.786+0000 7fb16d0bd640 1 Processor -- start 2024-04-26T21:12:52.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.786+0000 7fb16d0bd640 1 -- start start 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1680d1170 0x7fb1681f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1681631f0 0x7fb1681ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680d0230 con 0x7fb1680b2000 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680d2000 con 0x7fb1680d1170 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb1680cb4a0 con 0x7fb1681631f0 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37966/0 (socket says 172.21.15.40:37966) 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 -- 172.21.15.40:0/4280749706 learned_addr learned my addr 172.21.15.40:0/4280749706 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16759e640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1681631f0 0x7fb1681ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16659c640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1680d1170 0x7fb1681f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 -- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1681631f0 msgr2=0x7fb1681ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1681631f0 0x7fb1681ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 -- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1680d1170 msgr2=0x7fb1681f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1680d1170 0x7fb1681f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 -- 172.21.15.40:0/4280749706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680cb740 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb166d9d640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c390 tx=0x7fb16000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600074e0 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1600047e0 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.787+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1680cba80 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.788+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160004bd0 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.788+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16815d7a0 con 0x7fb1680b2000 2024-04-26T21:12:52.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.789+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16806a290 con 0x7fb1680b2000 2024-04-26T21:12:52.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.789+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb16001a020 con 0x7fb1680b2000 2024-04-26T21:12:52.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.789+0000 7fb15f7fe640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:52.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.790+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fb16008af70 con 0x7fb1680b2000 2024-04-26T21:12:52.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.790+0000 7fb16659c640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:52.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.790+0000 7fb16659c640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb16815f1e0 tx=0x7fb150009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:52.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.794+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1680cba80 con 0x7fb1680b2000 2024-04-26T21:12:52.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.979+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1680d2ae0 con 0x7fb1680b2000 2024-04-26T21:12:52.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.979+0000 7fb15f7fe640 1 -- 172.21.15.40:0/4280749706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7fb1600590c0 con 0x7fb1680b2000 2024-04-26T21:12:52.981 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:12:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 msgr2=0x7fb14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb16815f1e0 tx=0x7fb150009190 comp rx=0 tx=0).stop 2024-04-26T21:12:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 msgr2=0x7fb16815d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c390 tx=0x7fb16000c850 comp rx=0 tx=0).stop 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 shutdown_connections 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1681631f0 0x7fb1681ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1680d1170 0x7fb1681f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 --2- 172.21.15.40:0/4280749706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1680b2000 0x7fb16815d260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 >> 172.21.15.40:0/4280749706 conn(0x7fb16809ba40 msgr2=0x7fb1680b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 shutdown_connections 2024-04-26T21:12:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:52.981+0000 7fb16d0bd640 1 -- 172.21.15.40:0/4280749706 wait complete. 2024-04-26T21:12:52.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:12:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.093+0000 7fa0db712640 1 Processor -- start 2024-04-26T21:12:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.093+0000 7fa0db712640 1 -- start start 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0db712640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0d4160d50 0x7fa0d41611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0db712640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d40d0be0 con 0x7fa0d40b56d0 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0db712640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d40d0ec0 con 0x7fa0d415d380 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0db712640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d41616e0 con 0x7fa0d4160d50 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d8cad640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa0d40b56d0 0x7fa0d40d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42758/0 (socket says 172.21.15.40:42758) 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d8cad640 1 -- 172.21.15.40:0/3699828744 learned_addr learned my addr 172.21.15.40:0/3699828744 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749377484 0 0) 0x7fa0d40d0be0 con 0x7fa0d40b56d0 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/3699828744 >> v2:172.21.15.184:3300/0 conn(0x7fa0d4160d50 0x7fa0d41611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0bc003520 con 0x7fa0d40b56d0 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3019333851 0 0) 0x7fa0bc003520 con 0x7fa0d40b56d0 2024-04-26T21:12:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 >> v2:172.21.15.184:3300/0 conn(0x7fa0d4160d50 msgr2=0x7fa0d41611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 --2- 172.21.15.40:0/3699828744 >> v2:172.21.15.184:3300/0 conn(0x7fa0d4160d50 0x7fa0d41611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 >> v1:172.21.15.184:6789/0 conn(0x7fa0d415d380 legacy=0x7fa0d40ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d40d11a0 con 0x7fa0d40b56d0 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa0c8001670 con 0x7fa0d40b56d0 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 >> v1:172.21.15.40:6789/0 conn(0x7fa0d40b56d0 legacy=0x7fa0d40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc0039b0 0x7fa0bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc004800 0x7fa0bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 0x7fa0bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41616e0 con 0x7fa0bc0051c0 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc00c160 con 0x7fa0bc004800 2024-04-26T21:12:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.094+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0bc00c360 con 0x7fa0bc0039b0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d8cad640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc004800 0x7fa0bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 0x7fa0bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d94ae640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc0039b0 0x7fa0bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 -- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc0039b0 msgr2=0x7fa0bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc0039b0 0x7fa0bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 -- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc004800 msgr2=0x7fa0bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc004800 0x7fa0bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 -- 172.21.15.40:0/3699828744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc003520 con 0x7fa0bc0051c0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.095+0000 7fa0d9caf640 1 -- 172.21.15.40:0/3699828744 reap_dead start 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 0x7fa0bc00b9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009fc0 tx=0x7fa0bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83bfb0a12d00cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc035020 con 0x7fa0bc0051c0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc003b90 con 0x7fa0bc0051c0 2024-04-26T21:12:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0d3fff640 1 -- 172.21.15.40:0/3699828744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc009c10 con 0x7fa0bc0051c0 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 -- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 msgr2=0x7fa0bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 0x7fa0bc00b9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009fc0 tx=0x7fa0bc005830 comp rx=0 tx=0).stop 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 -- 172.21.15.40:0/3699828744 shutdown_connections 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 --2- 172.21.15.40:0/3699828744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0bc0051c0 0x7fa0bc00b9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 -- 172.21.15.40:0/3699828744 >> 172.21.15.40:0/3699828744 conn(0x7fa0d40b4a60 msgr2=0x7fa0d40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 -- 172.21.15.40:0/3699828744 shutdown_connections 2024-04-26T21:12:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.096+0000 7fa0db712640 1 -- 172.21.15.40:0/3699828744 wait complete. 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.097+0000 7fa0db712640 1 Processor -- start 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 -- start start 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 0x7fa0d4154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 0x7fa0d415fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41611a0 con 0x7fa0d40b7390 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d40cedc0 con 0x7fa0bc0051c0 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0db712640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d4161320 con 0x7fa0d415f9b0 2024-04-26T21:12:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0d8cad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0d8cad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37988/0 (socket says 172.21.15.40:37988) 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.098+0000 7fa0d8cad640 1 -- 172.21.15.40:0/4101155062 learned_addr learned my addr 172.21.15.40:0/4101155062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 0x7fa0d415fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 -- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 msgr2=0x7fa0d415fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d94ae640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 0x7fa0d4154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 0x7fa0d415fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 -- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 msgr2=0x7fa0d4154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 0x7fa0d4154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 -- 172.21.15.40:0/4101155062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d41f34e0 con 0x7fa0d40b7390 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d94ae640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 0x7fa0d4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d9caf640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 0x7fa0d415fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d8cad640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80072c0 tx=0x7fa0c8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800f8e0 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c8018070 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d41f37c0 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800fca0 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.099+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41f3d80 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.100+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d406a290 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.101+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0c801d460 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.101+0000 7fa0d1ffb640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 0x7fa0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.101+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa0c808b7e0 con 0x7fa0d40b7390 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.101+0000 7fa0d94ae640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 0x7fa0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.102+0000 7fa0d94ae640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 0x7fa0c006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa0d41606f0 tx=0x7fa0c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.105+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0c8059910 con 0x7fa0d40b7390 2024-04-26T21:12:53.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.295+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa0d41f3f50 con 0x7fa0d40b7390 2024-04-26T21:12:53.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.295+0000 7fa0d1ffb640 1 -- 172.21.15.40:0/4101155062 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7fa0c805dc10 con 0x7fa0d40b7390 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:12:53.297 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:12:53.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 msgr2=0x7fa0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 0x7fa0c006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa0d41606f0 tx=0x7fa0c4009040 comp rx=0 tx=0).stop 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 msgr2=0x7fa0d40b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80072c0 tx=0x7fa0c8007780 comp rx=0 tx=0).stop 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 shutdown_connections 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa0c006c8f0 0x7fa0c006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0d415f9b0 0x7fa0d415fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0d40b7390 0x7fa0d40b7810 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 --2- 172.21.15.40:0/4101155062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc0051c0 0x7fa0d4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 >> 172.21.15.40:0/4101155062 conn(0x7fa0d40b4a60 msgr2=0x7fa0d415c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:53.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 shutdown_connections 2024-04-26T21:12:53.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.297+0000 7fa0db712640 1 -- 172.21.15.40:0/4101155062 wait complete. 2024-04-26T21:12:53.309 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:12:53.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:12:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.412+0000 7fdfbbac4640 1 Processor -- start 2024-04-26T21:12:53.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.412+0000 7fdfbbac4640 1 -- start start 2024-04-26T21:12:53.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfbbac4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdfb416a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfbbac4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb4163080 con 0x7fdfb4148cb0 2024-04-26T21:12:53.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfbbac4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb4163360 con 0x7fdfb4148050 2024-04-26T21:12:53.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfbbac4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb4163c90 con 0x7fdfb40b5360 2024-04-26T21:12:53.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfb905f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdfb4148cb0 0x7fdfb40b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42762/0 (socket says 172.21.15.40:42762) 2024-04-26T21:12:53.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.413+0000 7fdfb905f640 1 -- 172.21.15.40:0/4287960598 learned_addr learned my addr 172.21.15.40:0/4287960598 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfba061640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdfb416a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781511980 0 0) 0x7fdfb4163080 con 0x7fdfb4148cb0 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf9c003520 con 0x7fdfb4148cb0 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047220000 0 0) 0x7fdfb4163360 con 0x7fdfb4148050 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb4163080 con 0x7fdfb4148050 2024-04-26T21:12:53.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283313696 0 0) 0x7fdf9c003520 con 0x7fdfb4148cb0 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 msgr2=0x7fdfb416a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdfb416a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 >> v1:172.21.15.184:6789/0 conn(0x7fdfb4148050 legacy=0x7fdfb40d38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb4163640 con 0x7fdfb4148cb0 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fdfa8001670 con 0x7fdfb4148cb0 2024-04-26T21:12:53.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 >> v1:172.21.15.40:6789/0 conn(0x7fdfb4148cb0 legacy=0x7fdfb40b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf9c003b70 0x7fdf9c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdf9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4163c90 con 0x7fdf9c004800 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c00baa0 con 0x7fdf9c003b70 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c00bca0 con 0x7fdfb40b5360 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfb9860640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf9c003b70 0x7fdf9c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.414+0000 7fdfba061640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdf9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 -- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 msgr2=0x7fdf9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdf9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 -- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf9c003b70 msgr2=0x7fdf9c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf9c003b70 0x7fdf9c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 -- 172.21.15.40:0/4287960598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c003520 con 0x7fdf9c004800 2024-04-26T21:12:53.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfba061640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb40b5360 0x7fdf9c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8001eb0 tx=0x7fdf9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b76a0c0e48f01ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8007ce0 con 0x7fdf9c004800 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa8005ad0 con 0x7fdf9c004800 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.415+0000 7fdfb885e640 1 -- 172.21.15.40:0/4287960598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8007070 con 0x7fdf9c004800 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4287960598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 msgr2=0x7fdf9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4287960598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdf9c004c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8001eb0 tx=0x7fdf9c005830 comp rx=0 tx=0).stop 2024-04-26T21:12:53.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfba061640 1 -- 172.21.15.40:0/4287960598 reap_dead start 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4287960598 shutdown_connections 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4287960598 >> 172.21.15.40:0/4287960598 conn(0x7fdfb406c960 msgr2=0x7fdfb40b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4287960598 shutdown_connections 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.416+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4287960598 wait complete. 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.417+0000 7fdfbbac4640 1 Processor -- start 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.417+0000 7fdfbbac4640 1 -- start start 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfb4160cd0 0x7fdfb413f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb4162470 0x7fdfb4145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb406e8e0 con 0x7fdf9c004800 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4148430 con 0x7fdfb4160cd0 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfbbac4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb406ea60 con 0x7fdfb4162470 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38004/0 (socket says 172.21.15.40:38004) 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 -- 172.21.15.40:0/4031508682 learned_addr learned my addr 172.21.15.40:0/4031508682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 -- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb4162470 msgr2=0x7fdfb4145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb4162470 0x7fdfb4145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 -- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfb4160cd0 msgr2=0x7fdfb413f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfb4160cd0 0x7fdfb413f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 -- 172.21.15.40:0/4031508682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb4161150 con 0x7fdf9c004800 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb9860640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400fb40 tx=0x7fdfa40021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.418+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4003b80 con 0x7fdf9c004800 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.419+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4018070 con 0x7fdf9c004800 2024-04-26T21:12:53.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.419+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb41613d0 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.419+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400d7d0 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.419+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb4161990 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.420+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb40693d0 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.420+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfa400d970 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.420+0000 7fdfb27fc640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.421+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fdfa408b970 con 0x7fdf9c004800 2024-04-26T21:12:53.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.421+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.421+0000 7fdfb905f640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 0x7fdfa006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8032000 tx=0x7fdfa8007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.425+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdfb4161150 con 0x7fdf9c004800 2024-04-26T21:12:53.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.614+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdfb40b6030 con 0x7fdf9c004800 2024-04-26T21:12:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.808+0000 7fdfb27fc640 1 -- 172.21.15.40:0/4031508682 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7fdfa405a180 con 0x7fdf9c004800 2024-04-26T21:12:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:12:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 msgr2=0x7fdfa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 0x7fdfa006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8032000 tx=0x7fdfa8007000 comp rx=0 tx=0).stop 2024-04-26T21:12:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 msgr2=0x7fdfb4154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400fb40 tx=0x7fdfa40021e0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 shutdown_connections 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfb4162470 0x7fdfb4145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfb4160cd0 0x7fdfb413f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 --2- 172.21.15.40:0/4031508682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf9c004800 0x7fdfb4154cd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.810+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 >> 172.21.15.40:0/4031508682 conn(0x7fdfb406c960 msgr2=0x7fdfb4142560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.811+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 shutdown_connections 2024-04-26T21:12:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.811+0000 7fdfbbac4640 1 -- 172.21.15.40:0/4031508682 wait complete. 2024-04-26T21:12:53.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:12:53.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.933+0000 7f21d0bb6640 1 Processor -- start 2024-04-26T21:12:53.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.933+0000 7f21d0bb6640 1 -- start start 2024-04-26T21:12:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21d0bb6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 0x7f21cc0d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21d0bb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21d0bb6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc0b52d0 con 0x7f21cc0b46b0 2024-04-26T21:12:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21d0bb6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc15faa0 con 0x7f21cc15f6c0 2024-04-26T21:12:53.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21d0bb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0b5080 con 0x7f21cc0b40f0 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.934+0000 7f21c9d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56074/0 (socket says 172.21.15.40:56074) 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 -- 172.21.15.40:0/1299192050 learned_addr learned my addr 172.21.15.40:0/1299192050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21ca59c640 1 --2- 172.21.15.40:0/1299192050 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 0x7f21cc0d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 -- 172.21.15.40:0/1299192050 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 msgr2=0x7f21cc0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 --2- 172.21.15.40:0/1299192050 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 0x7f21cc0d37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 -- 172.21.15.40:0/1299192050 >> v1:172.21.15.184:6789/0 conn(0x7f21cc0b46b0 legacy=0x7f21cc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c959a640 1 -- 172.21.15.40:0/1299192050 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635410 0 0) 0x7f21cc0b52d0 con 0x7f21cc0b46b0 2024-04-26T21:12:53.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21c9d9b640 1 -- 172.21.15.40:0/1299192050 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0b55b0 con 0x7f21cc0b40f0 2024-04-26T21:12:53.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.935+0000 7f21ca59c640 1 --2- 172.21.15.40:0/1299192050 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 0x7f21cc0d37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21c9d9b640 1 --2- 172.21.15.40:0/1299192050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b438c04522edf8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21c959a640 1 -- 172.21.15.40:0/1299192050 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4037070 con 0x7f21cc0b40f0 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21c959a640 1 -- 172.21.15.40:0/1299192050 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4032440 con 0x7f21cc0b40f0 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21c959a640 1 -- 172.21.15.40:0/1299192050 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4032810 con 0x7f21cc0b40f0 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21d0bb6640 1 -- 172.21.15.40:0/1299192050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 msgr2=0x7f21cc0d3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.936+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/1299192050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b402a270 comp rx=0 tx=0).stop 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 -- 172.21.15.40:0/1299192050 shutdown_connections 2024-04-26T21:12:53.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/1299192050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b40f0 0x7f21cc0d3d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/1299192050 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15f6c0 0x7f21cc0d37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 -- 172.21.15.40:0/1299192050 >> 172.21.15.40:0/1299192050 conn(0x7f21cc08ec10 msgr2=0x7f21cc09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 -- 172.21.15.40:0/1299192050 shutdown_connections 2024-04-26T21:12:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.937+0000 7f21d0bb6640 1 -- 172.21.15.40:0/1299192050 wait complete. 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.938+0000 7f21d0bb6640 1 Processor -- start 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.938+0000 7f21d0bb6640 1 -- start start 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b46b0 0x7f21cc1f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21cc15f6c0 0x7f21cc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0d0d80 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0b5b70 con 0x7f21cc15f6c0 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21d0bb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0b6090 con 0x7f21cc0b46b0 2024-04-26T21:12:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38018/0 (socket says 172.21.15.40:38018) 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 -- 172.21.15.40:0/572647470 learned_addr learned my addr 172.21.15.40:0/572647470 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21c9d9b640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21cc15f6c0 0x7f21cc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21ca59c640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b46b0 0x7f21cc1f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 -- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b46b0 msgr2=0x7f21cc1f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b46b0 0x7f21cc1f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 -- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21cc15f6c0 msgr2=0x7f21cc0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21cc15f6c0 0x7f21cc0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.939+0000 7f21cad9d640 1 -- 172.21.15.40:0/572647470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc1f5900 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21cad9d640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c780 tx=0x7f21bc00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc003b70 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc003d10 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc1f5be0 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc005270 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.940+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1f27a0 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.941+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc06a290 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.941+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21bc005410 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.942+0000 7f21c2ffd640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.942+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7f21bc08b7d0 con 0x7f21cc1f4eb0 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.942+0000 7f21ca59c640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.943+0000 7f21ca59c640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 0x7f21ac06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21cc158420 tx=0x7f21c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:53.946+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21bc059e70 con 0x7f21cc1f4eb0 2024-04-26T21:12:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.131+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f21cc1f5db0 con 0x7f21cc1f4eb0 2024-04-26T21:12:54.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.814+0000 7f21c2ffd640 1 -- 172.21.15.40:0/572647470 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7f21bc059770 con 0x7f21cc1f4eb0 2024-04-26T21:12:54.815 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:12:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 msgr2=0x7f21ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 0x7f21ac06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21cc158420 tx=0x7f21c4008040 comp rx=0 tx=0).stop 2024-04-26T21:12:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 msgr2=0x7f21cc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c780 tx=0x7f21bc00cb80 comp rx=0 tx=0).stop 2024-04-26T21:12:54.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 shutdown_connections 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21cc1f4eb0 0x7f21cc06d9e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21cc15f6c0 0x7f21cc0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 --2- 172.21.15.40:0/572647470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc0b46b0 0x7f21cc1f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 >> 172.21.15.40:0/572647470 conn(0x7f21cc08ec10 msgr2=0x7f21cc0b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 shutdown_connections 2024-04-26T21:12:54.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.817+0000 7f21d0bb6640 1 -- 172.21.15.40:0/572647470 wait complete. 2024-04-26T21:12:54.843 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:12:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.951+0000 7fab20cdc640 1 Processor -- start 2024-04-26T21:12:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.951+0000 7fab20cdc640 1 -- start start 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab20cdc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 0x7fab1c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab20cdc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab1c0ceff0 0x7fab1c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab20cdc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab1c15dca0 con 0x7fab1c143840 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab20cdc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fab1c143c20 con 0x7fab1c0ceff0 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab20cdc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fab1c15df80 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab1c0ceff0 0x7fab1c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab1c0ceff0 0x7fab1c0cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38022/0 (socket says 172.21.15.40:38022) 2024-04-26T21:12:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 -- 172.21.15.40:0/741013497 learned_addr learned my addr 172.21.15.40:0/741013497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab13fff640 1 --2- 172.21.15.40:0/741013497 >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 0x7fab1c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 -- 172.21.15.40:0/741013497 >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 msgr2=0x7fab1c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 --2- 172.21.15.40:0/741013497 >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 0x7fab1c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 -- 172.21.15.40:0/741013497 >> v1:172.21.15.184:6789/0 conn(0x7fab1c143840 legacy=0x7fab1c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab1ad9d640 1 -- 172.21.15.40:0/741013497 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c15e260 con 0x7fab1c0ceff0 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.952+0000 7fab13fff640 1 --2- 172.21.15.40:0/741013497 >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 0x7fab1c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.953+0000 7fab1ad9d640 1 --2- 172.21.15.40:0/741013497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0ceff0 0x7fab1c0cf480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0b0a90 tx=0x7fab1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae86572631706331 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.953+0000 7fab19d9b640 1 -- 172.21.15.40:0/741013497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14036070 con 0x7fab1c0ceff0 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.953+0000 7fab19d9b640 1 -- 172.21.15.40:0/741013497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14031600 con 0x7fab1c0ceff0 2024-04-26T21:12:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.953+0000 7fab19d9b640 1 -- 172.21.15.40:0/741013497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140319d0 con 0x7fab1c0ceff0 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 -- 172.21.15.40:0/741013497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0ceff0 msgr2=0x7fab1c0cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 --2- 172.21.15.40:0/741013497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0ceff0 0x7fab1c0cf480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0b0a90 tx=0x7fab1402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 -- 172.21.15.40:0/741013497 shutdown_connections 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 --2- 172.21.15.40:0/741013497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0ceff0 0x7fab1c0cf480 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 --2- 172.21.15.40:0/741013497 >> v2:172.21.15.184:3300/0 conn(0x7fab1c0b4eb0 0x7fab1c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 -- 172.21.15.40:0/741013497 >> 172.21.15.40:0/741013497 conn(0x7fab1c0cd8a0 msgr2=0x7fab1c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 -- 172.21.15.40:0/741013497 shutdown_connections 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.954+0000 7fab20cdc640 1 -- 172.21.15.40:0/741013497 wait complete. 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.955+0000 7fab20cdc640 1 Processor -- start 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.955+0000 7fab20cdc640 1 -- start start 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 0x7fab1c141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1c143840 0x7fab1c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c15d8c0 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c0b46f0 con 0x7fab1c0ceff0 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab20cdc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c15da40 con 0x7fab1c143840 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38032/0 (socket says 172.21.15.40:38032) 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 -- 172.21.15.40:0/1230677312 learned_addr learned my addr 172.21.15.40:0/1230677312 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab13fff640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 0x7fab1c141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1ad9d640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1c143840 0x7fab1c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 -- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1c143840 msgr2=0x7fab1c0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1c143840 0x7fab1c0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 -- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 msgr2=0x7fab1c141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 0x7fab1c141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 -- 172.21.15.40:0/1230677312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c132400 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab13fff640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 0x7fab1c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.956+0000 7fab1a59c640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b3f0 tx=0x7fab0c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.957+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c015020 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.957+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c011830 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.957+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c011c20 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.957+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c1357a0 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.957+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c140960 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.958+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c06a290 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.958+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab0c018020 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.959+0000 7fab12ffd640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.959+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7fab0c08b6f0 con 0x7fab1c0b4eb0 2024-04-26T21:12:54.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.959+0000 7fab13fff640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.959+0000 7fab13fff640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 0x7faafc06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0d0570 tx=0x7fab04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:54.963+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab0c01f030 con 0x7fab1c0b4eb0 2024-04-26T21:12:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.188+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fab1c0ce420 con 0x7fab1c0b4eb0 2024-04-26T21:12:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.815+0000 7fab12ffd640 1 -- 172.21.15.40:0/1230677312 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7fab1c132400 con 0x7fab1c0b4eb0 2024-04-26T21:12:55.817 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 msgr2=0x7faafc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 0x7faafc06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0d0570 tx=0x7fab04008040 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 msgr2=0x7fab1c131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b3f0 tx=0x7fab0c00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 shutdown_connections 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab1c143840 0x7fab1c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab1c0ceff0 0x7fab1c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 --2- 172.21.15.40:0/1230677312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab1c0b4eb0 0x7fab1c131ec0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b3f0 tx=0x7fab0c00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.819+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 >> 172.21.15.40:0/1230677312 conn(0x7fab1c0cd8a0 msgr2=0x7fab1c0b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.820+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 shutdown_connections 2024-04-26T21:12:55.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.820+0000 7fab20cdc640 1 -- 172.21.15.40:0/1230677312 wait complete. 2024-04-26T21:12:55.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.955+0000 7f13c4c45640 1 Processor -- start 2024-04-26T21:12:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.955+0000 7f13c4c45640 1 -- start start 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13c4c45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 0x7f13c0073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13c4c45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13c4c45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c015e2e0 con 0x7f13c015d6b0 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13c4c45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13c015e5c0 con 0x7f13c0146980 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13c4c45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13c01613c0 con 0x7f13c0161850 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.956+0000 7f13bdd9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:56096/0 (socket says 172.21.15.40:56096) 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13be59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 0x7f13c0073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 -- 172.21.15.40:0/261742369 learned_addr learned my addr 172.21.15.40:0/261742369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 -- 172.21.15.40:0/261742369 >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 msgr2=0x7f13c0073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 --2- 172.21.15.40:0/261742369 >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 0x7f13c0073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 -- 172.21.15.40:0/261742369 >> v1:172.21.15.184:6789/0 conn(0x7f13c015d6b0 legacy=0x7f13c006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13bdd9b640 1 -- 172.21.15.40:0/261742369 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0160d70 con 0x7f13c0161850 2024-04-26T21:12:55.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.957+0000 7f13be59c640 1 --2- 172.21.15.40:0/261742369 >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 0x7f13c0073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13bdd9b640 1 --2- 172.21.15.40:0/261742369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009870 tx=0x7f13a802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91e29d692243a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13bd59a640 1 -- 172.21.15.40:0/261742369 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8037070 con 0x7f13c0161850 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13bd59a640 1 -- 172.21.15.40:0/261742369 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a8032440 con 0x7f13c0161850 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13bd59a640 1 -- 172.21.15.40:0/261742369 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8032810 con 0x7f13c0161850 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 -- 172.21.15.40:0/261742369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 msgr2=0x7f13c00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 --2- 172.21.15.40:0/261742369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009870 tx=0x7f13a802a270 comp rx=0 tx=0).stop 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 -- 172.21.15.40:0/261742369 shutdown_connections 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 --2- 172.21.15.40:0/261742369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c00740f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 --2- 172.21.15.40:0/261742369 >> v2:172.21.15.184:3300/0 conn(0x7f13c0146980 0x7f13c0073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.958+0000 7f13c4c45640 1 -- 172.21.15.40:0/261742369 >> 172.21.15.40:0/261742369 conn(0x7f13c00ce8b0 msgr2=0x7f13c00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.959+0000 7f13c4c45640 1 -- 172.21.15.40:0/261742369 shutdown_connections 2024-04-26T21:12:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.959+0000 7f13c4c45640 1 -- 172.21.15.40:0/261742369 wait complete. 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.960+0000 7f13c4c45640 1 Processor -- start 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.960+0000 7f13c4c45640 1 -- start start 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c015d6b0 0x7f13c015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c013b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0146d60 con 0x7f13c0146980 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13c00b5120 con 0x7f13c015d6b0 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13c015db40 con 0x7f13c0161850 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38048/0 (socket says 172.21.15.40:38048) 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 -- 172.21.15.40:0/2801888397 learned_addr learned my addr 172.21.15.40:0/2801888397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13bed9d640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c013b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13bdd9b640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c015d6b0 0x7f13c015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 -- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 msgr2=0x7f13c013b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c013b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 -- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c015d6b0 msgr2=0x7f13c015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c015d6b0 0x7f13c015fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 -- 172.21.15.40:0/2801888397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0142950 con 0x7f13c0146980 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13be59c640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13b800ec10 tx=0x7f13b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8017070 con 0x7f13c0146980 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b800cc30 con 0x7f13c0146980 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c01f6ac0 con 0x7f13c0146980 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8013470 con 0x7f13c0146980 2024-04-26T21:12:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.961+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c01f7080 con 0x7f13c0146980 2024-04-26T21:12:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.962+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c006a290 con 0x7f13c0146980 2024-04-26T21:12:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.963+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13b801a070 con 0x7f13c0146980 2024-04-26T21:12:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.963+0000 7f13b6ffd640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.963+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f13b808b980 con 0x7f13c0146980 2024-04-26T21:12:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.964+0000 7f13bdd9b640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.964+0000 7f13bdd9b640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 0x7f13a006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13a8034000 tx=0x7f13a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:55.968+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13b80597f0 con 0x7f13c0146980 2024-04-26T21:12:56.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.154+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c00b1740 con 0x7f13c0146980 2024-04-26T21:12:56.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.155+0000 7f13b6ffd640 1 -- 172.21.15.40:0/2801888397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16682 (secure 0 0 0) 0x7f13b805daf0 con 0x7f13c0146980 2024-04-26T21:12:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:56.156 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:55.189886+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:56.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 msgr2=0x7f13a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 0x7f13a006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13a8034000 tx=0x7f13a802e000 comp rx=0 tx=0).stop 2024-04-26T21:12:56.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 msgr2=0x7f13c0142410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13b800ec10 tx=0x7f13b800c520 comp rx=0 tx=0).stop 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 shutdown_connections 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13c0161850 0x7f13c013b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13c015d6b0 0x7f13c015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 --2- 172.21.15.40:0/2801888397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0146980 0x7f13c0142410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 >> 172.21.15.40:0/2801888397 conn(0x7f13c00ce8b0 msgr2=0x7f13c0070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 shutdown_connections 2024-04-26T21:12:56.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.157+0000 7f13c4c45640 1 -- 172.21.15.40:0/2801888397 wait complete. 2024-04-26T21:12:56.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:56.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.268+0000 7f3f44baf640 1 Processor -- start 2024-04-26T21:12:56.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.268+0000 7f3f44baf640 1 -- start start 2024-04-26T21:12:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 0x7f3f40073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 0x7f3f401636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f40163be0 0x7f3f400b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f400cfa80 con 0x7f3f40163be0 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f400cfd60 con 0x7f3f40160640 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f44baf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f400d0040 con 0x7f3f40148a30 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f3ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f40163be0 0x7f3f400b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f3ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f40163be0 0x7f3f400b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38060/0 (socket says 172.21.15.40:38060) 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2350320353 learned_addr learned my addr 172.21.15.40:0/2350320353 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f3dd9b640 1 --2- 172.21.15.40:0/2350320353 >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 0x7f3f401636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.269+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 0x7f3f40073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2350320353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 msgr2=0x7f3f40073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 0x7f3f40073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2350320353 >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 msgr2=0x7f3f401636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2350320353 >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 0x7f3f401636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2350320353 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f40162ea0 con 0x7f3f40163be0 2024-04-26T21:12:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3dd9b640 1 --2- 172.21.15.40:0/2350320353 >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 0x7f3f401636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 0x7f3f40073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f400b3af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3f400721c0 tx=0x7f3f3002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172df829b9479392 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3d59a640 1 -- 172.21.15.40:0/2350320353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036070 con 0x7f3f40163be0 2024-04-26T21:12:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3d59a640 1 -- 172.21.15.40:0/2350320353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30031600 con 0x7f3f40163be0 2024-04-26T21:12:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.270+0000 7f3f3d59a640 1 -- 172.21.15.40:0/2350320353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300319d0 con 0x7f3f40163be0 2024-04-26T21:12:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 -- 172.21.15.40:0/2350320353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 msgr2=0x7f3f400b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f400b3af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3f400721c0 tx=0x7f3f3002a610 comp rx=0 tx=0).stop 2024-04-26T21:12:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 -- 172.21.15.40:0/2350320353 shutdown_connections 2024-04-26T21:12:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f400b3af0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2350320353 >> v2:172.21.15.184:3300/0 conn(0x7f3f40160640 0x7f3f401636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2350320353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40148a30 0x7f3f40073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 -- 172.21.15.40:0/2350320353 >> 172.21.15.40:0/2350320353 conn(0x7f3f400ce8b0 msgr2=0x7f3f400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 -- 172.21.15.40:0/2350320353 shutdown_connections 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.271+0000 7f3f44baf640 1 -- 172.21.15.40:0/2350320353 wait complete. 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 Processor -- start 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 -- start start 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 0x7f3f400af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 0x7f3f400af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40074590 con 0x7f3f40163be0 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40074410 con 0x7f3f40148a30 2024-04-26T21:12:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f44baf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f40148e10 con 0x7f3f40160640 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.273+0000 7f3f3ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38068/0 (socket says 172.21.15.40:38068) 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2220618368 learned_addr learned my addr 172.21.15.40:0/2220618368 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3dd9b640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 0x7f3f400af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 0x7f3f400af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 msgr2=0x7f3f400af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 0x7f3f400af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 msgr2=0x7f3f400af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 0x7f3f400af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 -- 172.21.15.40:0/2220618368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f400b0ac0 con 0x7f3f40163be0 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3dd9b640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 0x7f3f400af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 0x7f3f400af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f3ed9d640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009f40 tx=0x7f3f30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30041020 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30007780 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036040 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f400b02d0 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.274+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f40134e80 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.275+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4006a290 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.276+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f30044020 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.276+0000 7f3f36ffd640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 0x7f3f2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.276+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3f300ac810 con 0x7f3f40163be0 2024-04-26T21:12:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.276+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 0x7f3f2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.277+0000 7f3f3e59c640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 0x7f3f2006ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.280+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f3004b030 con 0x7f3f40163be0 2024-04-26T21:12:56.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.465+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f400739f0 con 0x7f3f40163be0 2024-04-26T21:12:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.466+0000 7f3f36ffd640 1 -- 172.21.15.40:0/2220618368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16682 (secure 0 0 0) 0x7f3f400b0ac0 con 0x7f3f40163be0 2024-04-26T21:12:56.467 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:56.467 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:55.189886+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 msgr2=0x7f3f2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 0x7f3f2006ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-04-26T21:12:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 msgr2=0x7f3f401369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009f40 tx=0x7f3f30004a40 comp rx=0 tx=0).stop 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 shutdown_connections 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f2006c920 0x7f3f2006ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f40163be0 0x7f3f401369e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f40160640 0x7f3f400af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 --2- 172.21.15.40:0/2220618368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f40148a30 0x7f3f400af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 >> 172.21.15.40:0/2220618368 conn(0x7f3f400ce8b0 msgr2=0x7f3f400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 shutdown_connections 2024-04-26T21:12:56.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.468+0000 7f3f44baf640 1 -- 172.21.15.40:0/2220618368 wait complete. 2024-04-26T21:12:56.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:12:56.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.583+0000 7f579ba64640 1 Processor -- start 2024-04-26T21:12:56.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.583+0000 7f579ba64640 1 -- start start 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579ba64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f579415f490 0x7f57940d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579ba64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d22f0 0x7f57940ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579ba64640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57940d4240 con 0x7f57940d0ff0 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579ba64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f57940d31a0 con 0x7f579415f490 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579ba64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57940d3480 con 0x7f57940d22f0 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f5799800640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f579415f490 0x7f57940d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f5799800640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f579415f490 0x7f57940d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:37516/0 (socket says 172.21.15.40:37516) 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f5799800640 1 -- 172.21.15.40:0/3223797840 learned_addr learned my addr 172.21.15.40:0/3223797840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f5798fff640 1 --1- 172.21.15.40:0/3223797840 >> v1:172.21.15.184:6789/0 conn(0x7f57940d0ff0 0x7f57940d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41960/0 (socket says 172.21.15.40:41960) 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.584+0000 7f579a001640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d22f0 0x7f57940ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f5799800640 1 -- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d22f0 msgr2=0x7f57940ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f5799800640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d22f0 0x7f57940ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f5799800640 1 -- 172.21.15.40:0/3223797840 >> v1:172.21.15.184:6789/0 conn(0x7f57940d0ff0 legacy=0x7f57940d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f578bfff640 1 -- 172.21.15.40:0/3223797840 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816059955 0 0) 0x7f57940d4240 con 0x7f57940d0ff0 2024-04-26T21:12:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f5799800640 1 -- 172.21.15.40:0/3223797840 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57940d3760 con 0x7f579415f490 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.585+0000 7f5799800640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f579415f490 0x7f57940d0ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb98db4cb28e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f578bfff640 1 -- 172.21.15.40:0/3223797840 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790037070 con 0x7f579415f490 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f578bfff640 1 -- 172.21.15.40:0/3223797840 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790032440 con 0x7f579415f490 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f578bfff640 1 -- 172.21.15.40:0/3223797840 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790032810 con 0x7f579415f490 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 -- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f579415f490 msgr2=0x7f57940d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f579415f490 0x7f57940d0ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579002a270 comp rx=0 tx=0).stop 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 -- 172.21.15.40:0/3223797840 shutdown_connections 2024-04-26T21:12:56.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57940d22f0 0x7f57940ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 --2- 172.21.15.40:0/3223797840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f579415f490 0x7f57940d0ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.586+0000 7f579ba64640 1 -- 172.21.15.40:0/3223797840 >> 172.21.15.40:0/3223797840 conn(0x7f579409e530 msgr2=0x7f57940d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.587+0000 7f579ba64640 1 -- 172.21.15.40:0/3223797840 shutdown_connections 2024-04-26T21:12:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.587+0000 7f579ba64640 1 -- 172.21.15.40:0/3223797840 wait complete. 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.588+0000 7f579ba64640 1 Processor -- start 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.588+0000 7f579ba64640 1 -- start start 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940d0ff0 0x7f57941536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f579415f490 0x7f57940ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f579415f870 con 0x7f57940d22f0 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57940d3d20 con 0x7f57940d0ff0 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57940d2dc0 con 0x7f579415f490 2024-04-26T21:12:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38080/0 (socket says 172.21.15.40:38080) 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 -- 172.21.15.40:0/1111512442 learned_addr learned my addr 172.21.15.40:0/1111512442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5799800640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940d0ff0 0x7f57941536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579a001640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f579415f490 0x7f57940ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 -- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f579415f490 msgr2=0x7f57940ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f579415f490 0x7f57940ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 -- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940d0ff0 msgr2=0x7f57941536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940d0ff0 0x7f57941536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 -- 172.21.15.40:0/1111512442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579415da50 con 0x7f57940d22f0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5798fff640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f578400c460 tx=0x7f578400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784018070 con 0x7f57940d22f0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5784007500 con 0x7f57940d22f0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57940c9730 con 0x7f57940d22f0 2024-04-26T21:12:56.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.589+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784004a20 con 0x7f57940d22f0 2024-04-26T21:12:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.590+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57940c9cf0 con 0x7f57940d22f0 2024-04-26T21:12:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.591+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57940d41b0 con 0x7f57940d22f0 2024-04-26T21:12:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.591+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f578401b070 con 0x7f57940d22f0 2024-04-26T21:12:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.591+0000 7f5789ffb640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.592+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f578408acc0 con 0x7f57940d22f0 2024-04-26T21:12:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.592+0000 7f5799800640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.592+0000 7f5799800640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 0x7f577806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:56.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.596+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57940d41b0 con 0x7f57940d22f0 2024-04-26T21:12:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:56.826+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f57940c8560 con 0x7f57940d22f0 2024-04-26T21:12:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.581+0000 7f5789ffb640 1 -- 172.21.15.40:0/1111512442 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f579415da50 con 0x7f57940d22f0 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 msgr2=0x7f577806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 0x7f577806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 msgr2=0x7f57940ba220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f578400c460 tx=0x7f578400c920 comp rx=0 tx=0).stop 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 shutdown_connections 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f577806c8f0 0x7f577806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f579415f490 0x7f57940ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57940d22f0 0x7f57940ba220 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 --2- 172.21.15.40:0/1111512442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57940d0ff0 0x7f57941536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 >> 172.21.15.40:0/1111512442 conn(0x7f579409e530 msgr2=0x7f5794162ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 shutdown_connections 2024-04-26T21:12:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.584+0000 7f579ba64640 1 -- 172.21.15.40:0/1111512442 wait complete. 2024-04-26T21:12:57.598 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:12:57.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:12:57.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.707+0000 7ff187722640 1 Processor -- start 2024-04-26T21:12:57.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff187722640 1 -- start start 2024-04-26T21:12:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff187722640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff180162ca0 0x7ff180070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff187722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800d2470 con 0x7ff1800d1c20 2024-04-26T21:12:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff187722640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800d2750 con 0x7ff180071330 2024-04-26T21:12:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff187722640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff1800cdc40 con 0x7ff180162ca0 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff184cbd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff1800d1c20 0x7ff180070c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42774/0 (socket says 172.21.15.40:42774) 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.708+0000 7ff184cbd640 1 -- 172.21.15.40:0/568693728 learned_addr learned my addr 172.21.15.40:0/568693728 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573040521 0 0) 0x7ff1800d2470 con 0x7ff1800d1c20 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff1854be640 1 --2- 172.21.15.40:0/568693728 >> v2:172.21.15.184:3300/0 conn(0x7ff180162ca0 0x7ff180070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff168003520 con 0x7ff1800d1c20 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4147640816 0 0) 0x7ff168003520 con 0x7ff1800d1c20 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 >> v2:172.21.15.184:3300/0 conn(0x7ff180162ca0 msgr2=0x7ff180070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 --2- 172.21.15.40:0/568693728 >> v2:172.21.15.184:3300/0 conn(0x7ff180162ca0 0x7ff180070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 >> v1:172.21.15.184:6789/0 conn(0x7ff180071330 legacy=0x7ff1800ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800cd5f0 con 0x7ff1800d1c20 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff174001670 con 0x7ff1800d1c20 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 >> v1:172.21.15.40:6789/0 conn(0x7ff1800d1c20 legacy=0x7ff180070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff168004800 0x7ff168004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1680051c0 0x7ff16800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800cdc40 con 0x7ff1680039b0 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff16800c060 con 0x7ff1680051c0 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff16800c260 con 0x7ff168004800 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff1854be640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff184cbd640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff168004800 0x7ff168004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff185cbf640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1680051c0 0x7ff16800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.709+0000 7ff1854be640 1 -- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff168004800 msgr2=0x7ff168004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff1854be640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff168004800 0x7ff168004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff1854be640 1 -- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1680051c0 msgr2=0x7ff16800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff1854be640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1680051c0 0x7ff16800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff1854be640 1 -- 172.21.15.40:0/568693728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff168003520 con 0x7ff1680039b0 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff185cbf640 1 -- 172.21.15.40:0/568693728 reap_dead start 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff1854be640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff168003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff170009ac0 tx=0x7ff168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6810bbf4a6c4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170036070 con 0x7ff1680039b0 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff170031400 con 0x7ff1680039b0 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.710+0000 7ff17ffff640 1 -- 172.21.15.40:0/568693728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170009690 con 0x7ff1680039b0 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 -- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 msgr2=0x7ff168003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff168003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff170009ac0 tx=0x7ff168005830 comp rx=0 tx=0).stop 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 -- 172.21.15.40:0/568693728 shutdown_connections 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 --2- 172.21.15.40:0/568693728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff168003dd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 -- 172.21.15.40:0/568693728 >> 172.21.15.40:0/568693728 conn(0x7ff18009ba40 msgr2=0x7ff18009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 -- 172.21.15.40:0/568693728 shutdown_connections 2024-04-26T21:12:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.711+0000 7ff187722640 1 -- 172.21.15.40:0/568693728 wait complete. 2024-04-26T21:12:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.712+0000 7ff187722640 1 Processor -- start 2024-04-26T21:12:57.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.712+0000 7ff187722640 1 -- start start 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1800b0bd0 0x7ff1801f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1801f5b00 0x7ff1801f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800d2230 con 0x7ff1680039b0 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800d20b0 con 0x7ff1800b0bd0 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff187722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff180070640 con 0x7ff1801f5b00 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38088/0 (socket says 172.21.15.40:38088) 2024-04-26T21:12:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 -- 172.21.15.40:0/437193664 learned_addr learned my addr 172.21.15.40:0/437193664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 -- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1801f5b00 msgr2=0x7ff1801f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1801f5b00 0x7ff1801f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 -- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1800b0bd0 msgr2=0x7ff1801f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1800b0bd0 0x7ff1801f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 -- 172.21.15.40:0/437193664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18009d0f0 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.713+0000 7ff1854be640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff17002abb0 tx=0x7ff170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.714+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17003f020 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.714+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff170037040 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.714+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170036070 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.714+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1800b1230 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.714+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18009ee80 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.715+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1801610c0 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.715+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff170042020 con 0x7ff1680039b0 2024-04-26T21:12:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.716+0000 7ff17dffb640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 0x7ff16c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.716+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7ff1700abc00 con 0x7ff1680039b0 2024-04-26T21:12:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.716+0000 7ff184cbd640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 0x7ff16c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.716+0000 7ff184cbd640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 0x7ff16c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff174000f60 tx=0x7ff174000ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.720+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1801610c0 con 0x7ff1680039b0 2024-04-26T21:12:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:57.947+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff1800693d0 con 0x7ff1680039b0 2024-04-26T21:12:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.845+0000 7ff17dffb640 1 -- 172.21.15.40:0/437193664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7ff170049030 con 0x7ff1680039b0 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.847+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 msgr2=0x7ff16c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.847+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 0x7ff16c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff174000f60 tx=0x7ff174000ef0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.847+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 msgr2=0x7ff1800b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.847+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff17002abb0 tx=0x7ff170004730 comp rx=0 tx=0).stop 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 shutdown_connections 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff16c06c920 0x7ff16c06ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1801f5b00 0x7ff1801f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1800b0bd0 0x7ff1801f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 --2- 172.21.15.40:0/437193664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1680039b0 0x7ff1800b0690 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 >> 172.21.15.40:0/437193664 conn(0x7ff18009ba40 msgr2=0x7ff18006e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 shutdown_connections 2024-04-26T21:12:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.848+0000 7ff187722640 1 -- 172.21.15.40:0/437193664 wait complete. 2024-04-26T21:12:58.862 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:12:58.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:12:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.969+0000 7f437d237640 1 Processor -- start 2024-04-26T21:12:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.969+0000 7f437d237640 1 -- start start 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437d237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43780b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437d237640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43780b2510 con 0x7f437815f720 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437d237640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43780b27f0 con 0x7f437815f340 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437d237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43780b2ad0 con 0x7f43780d5610 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f436ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f437815f720 0x7f43780b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42776/0 (socket says 172.21.15.40:42776) 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f436ffff640 1 -- 172.21.15.40:0/1515855049 learned_addr learned my addr 172.21.15.40:0/1515855049 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437759e640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43780b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292999656 0 0) 0x7f43780b2510 con 0x7f437815f720 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003520 con 0x7f437815f720 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371500721 0 0) 0x7f43780b27f0 con 0x7f437815f340 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43780b2510 con 0x7f437815f340 2024-04-26T21:12:58.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2633916574 0 0) 0x7f4364003520 con 0x7f437815f720 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 msgr2=0x7f43780b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43780b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 >> v1:172.21.15.184:6789/0 conn(0x7f437815f340 legacy=0x7f43780d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780b2db0 con 0x7f437815f720 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f4360001670 con 0x7f437815f720 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437759e640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43780b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.970+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 >> v1:172.21.15.40:6789/0 conn(0x7f437815f720 legacy=0x7f43780b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f4364003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43640051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b2ad0 con 0x7f4364003b70 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f436400bad0 con 0x7f4364004800 2024-04-26T21:12:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f436400bcd0 con 0x7f43780d5610 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f4364003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f436ffff640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f437759e640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43640051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 -- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 msgr2=0x7f43640051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43780d5610 0x7f43640051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 -- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4364004800 msgr2=0x7f4364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.971+0000 7f4376d9d640 1 -- 172.21.15.40:0/1515855049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364003520 con 0x7f4364003b70 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f4376d9d640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f4364003f70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43680018d0 tx=0x7f4364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5886eebca35dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680014f0 con 0x7f4364003b70 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43680052d0 con 0x7f4364003b70 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437659c640 1 -- 172.21.15.40:0/1515855049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436802aca0 con 0x7f4364003b70 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 -- 172.21.15.40:0/1515855049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 msgr2=0x7f4364003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 --2- 172.21.15.40:0/1515855049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f4364003f70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43680018d0 tx=0x7f4364005830 comp rx=0 tx=0).stop 2024-04-26T21:12:58.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437759e640 1 -- 172.21.15.40:0/1515855049 reap_dead start 2024-04-26T21:12:58.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 -- 172.21.15.40:0/1515855049 shutdown_connections 2024-04-26T21:12:58.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 -- 172.21.15.40:0/1515855049 >> 172.21.15.40:0/1515855049 conn(0x7f437808ec70 msgr2=0x7f437809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:58.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 -- 172.21.15.40:0/1515855049 shutdown_connections 2024-04-26T21:12:58.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.972+0000 7f437d237640 1 -- 172.21.15.40:0/1515855049 wait complete. 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.974+0000 7f437d237640 1 Processor -- start 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.974+0000 7f437d237640 1 -- start start 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 0x7f437815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 0x7f4378162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b2210 con 0x7f4364003b70 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b2090 con 0x7f43780cb610 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437d237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43780d3d40 con 0x7f4378161bc0 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38116/0 (socket says 172.21.15.40:38116) 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 -- 172.21.15.40:0/3865270421 learned_addr learned my addr 172.21.15.40:0/3865270421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437759e640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 0x7f4378162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f436ffff640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 0x7f437815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 -- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 msgr2=0x7f4378162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 0x7f4378162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 -- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 msgr2=0x7f437815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 0x7f437815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 -- 172.21.15.40:0/3865270421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780c9d70 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f437759e640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 0x7f4378162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f436ffff640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 0x7f437815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.975+0000 7f4376d9d640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4368000c00 tx=0x7f43680012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.976+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368037040 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.976+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368006e50 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.976+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43780ca050 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.976+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680014f0 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.976+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43780ca610 con 0x7f4364003b70 2024-04-26T21:12:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.977+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437806a290 con 0x7f4364003b70 2024-04-26T21:12:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.977+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4368044020 con 0x7f4364003b70 2024-04-26T21:12:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.978+0000 7f436f7fe640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 0x7f435806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.978+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f43680ab7f0 con 0x7f4364003b70 2024-04-26T21:12:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.978+0000 7f436ffff640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 0x7f435806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.979+0000 7f436ffff640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 0x7f435806ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43780d1c70 tx=0x7f4360009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:58.982+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f436804b020 con 0x7f4364003b70 2024-04-26T21:12:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.209+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43780d3020 con 0x7f4364003b70 2024-04-26T21:12:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.209+0000 7f436f7fe640 1 -- 172.21.15.40:0/3865270421 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f4368079b80 con 0x7f4364003b70 2024-04-26T21:12:59.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:59.211 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:12:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 msgr2=0x7f435806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 0x7f435806ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43780d1c70 tx=0x7f4360009040 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 msgr2=0x7f43780c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4368000c00 tx=0x7f43680012d0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 shutdown_connections 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435806c920 0x7f435806ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4378161bc0 0x7f4378162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43780cb610 0x7f437815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 --2- 172.21.15.40:0/3865270421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4364003b70 0x7f43780c9790 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 >> 172.21.15.40:0/3865270421 conn(0x7f437808ec70 msgr2=0x7f437806e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 shutdown_connections 2024-04-26T21:12:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.211+0000 7f437d237640 1 -- 172.21.15.40:0/3865270421 wait complete. 2024-04-26T21:12:59.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:12:59.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.323+0000 7f9f5c047640 1 Processor -- start 2024-04-26T21:12:59.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.323+0000 7f9f5c047640 1 -- start start 2024-04-26T21:12:59.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5c047640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 0x7f9f540b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5c047640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f540d5610 0x7f9f540d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5c047640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f540b2510 con 0x7f9f5415f720 2024-04-26T21:12:59.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5c047640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9f540b27f0 con 0x7f9f540d5610 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5c047640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9f540b2ad0 con 0x7f9f5415f340 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5a5e4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f540d5610 0x7f9f540d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f595e2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9f5415f720 0x7f9f540d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42786/0 (socket says 172.21.15.40:42786) 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f595e2640 1 -- 172.21.15.40:0/2964328554 learned_addr learned my addr 172.21.15.40:0/2964328554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f59de3640 1 --2- 172.21.15.40:0/2964328554 >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 0x7f9f540b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5a5e4640 1 -- 172.21.15.40:0/2964328554 >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 msgr2=0x7f9f540b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.324+0000 7f9f5a5e4640 1 --2- 172.21.15.40:0/2964328554 >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 0x7f9f540b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f5a5e4640 1 -- 172.21.15.40:0/2964328554 >> v1:172.21.15.40:6789/0 conn(0x7f9f5415f720 legacy=0x7f9f540d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f58de1640 1 -- 172.21.15.40:0/2964328554 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1211743566 0 0) 0x7f9f540b2510 con 0x7f9f5415f720 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f5a5e4640 1 -- 172.21.15.40:0/2964328554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540b2db0 con 0x7f9f540d5610 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f59de3640 1 --2- 172.21.15.40:0/2964328554 >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 0x7f9f540b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:12:59.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f5a5e4640 1 --2- 172.21.15.40:0/2964328554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540d3d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f540d6000 tx=0x7f9f4c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5806c0c2e698e242 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f58de1640 1 -- 172.21.15.40:0/2964328554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007740 con 0x7f9f540d5610 2024-04-26T21:12:59.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f58de1640 1 -- 172.21.15.40:0/2964328554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0078e0 con 0x7f9f540d5610 2024-04-26T21:12:59.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.325+0000 7f9f58de1640 1 -- 172.21.15.40:0/2964328554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007cb0 con 0x7f9f540d5610 2024-04-26T21:12:59.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 -- 172.21.15.40:0/2964328554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 msgr2=0x7f9f540d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 --2- 172.21.15.40:0/2964328554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540d3d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f540d6000 tx=0x7f9f4c02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 -- 172.21.15.40:0/2964328554 shutdown_connections 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 --2- 172.21.15.40:0/2964328554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540d3d40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 --2- 172.21.15.40:0/2964328554 >> v2:172.21.15.184:3300/0 conn(0x7f9f5415f340 0x7f9f540b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 -- 172.21.15.40:0/2964328554 >> 172.21.15.40:0/2964328554 conn(0x7f9f5408ec70 msgr2=0x7f9f5409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 -- 172.21.15.40:0/2964328554 shutdown_connections 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.326+0000 7f9f5c047640 1 -- 172.21.15.40:0/2964328554 wait complete. 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.327+0000 7f9f5c047640 1 Processor -- start 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.327+0000 7f9f5c047640 1 -- start start 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 0x7f9f541f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f5415f720 0x7f9f540cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540b3940 con 0x7f9f540d5610 2024-04-26T21:12:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540b37c0 con 0x7f9f5415f340 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f5c047640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540b3ac0 con 0x7f9f5415f720 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38146/0 (socket says 172.21.15.40:38146) 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 -- 172.21.15.40:0/3968269972 learned_addr learned my addr 172.21.15.40:0/3968269972 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 -- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f5415f720 msgr2=0x7f9f540cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f5415f720 0x7f9f540cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 -- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 msgr2=0x7f9f541f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f595e2640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 0x7f9f541f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 0x7f9f541f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f59de3640 1 -- 172.21.15.40:0/3968269972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540d03b0 con 0x7f9f540d5610 2024-04-26T21:12:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.328+0000 7f9f595e2640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 0x7f9f541f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f59de3640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000e7e0 tx=0x7f9f5000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000f040 con 0x7f9f540d5610 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f50004030 con 0x7f9f540d5610 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f540d0690 con 0x7f9f540d5610 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004400 con 0x7f9f540d5610 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.329+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f540d0c50 con 0x7f9f540d5610 2024-04-26T21:12:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.330+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5406a290 con 0x7f9f540d5610 2024-04-26T21:12:59.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.330+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f5000c6d0 con 0x7f9f540d5610 2024-04-26T21:12:59.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.331+0000 7f9f4a7fc640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 0x7f9f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.331+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9f5008bca0 con 0x7f9f540d5610 2024-04-26T21:12:59.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.331+0000 7f9f595e2640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 0x7f9f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.332+0000 7f9f595e2640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 0x7f9f3806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cbda0 tx=0x7f9f44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.335+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f50017020 con 0x7f9f540d5610 2024-04-26T21:12:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.520+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f540d3020 con 0x7f9f540d5610 2024-04-26T21:12:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.521+0000 7f9f4a7fc640 1 -- 172.21.15.40:0/3968269972 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7f9f5005a0c0 con 0x7f9f540d5610 2024-04-26T21:12:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:59.523 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:12:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 msgr2=0x7f9f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 0x7f9f3806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cbda0 tx=0x7f9f44009040 comp rx=0 tx=0).stop 2024-04-26T21:12:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 msgr2=0x7f9f540cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000e7e0 tx=0x7f9f5000eca0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 shutdown_connections 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f3806c8f0 0x7f9f3806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f5415f720 0x7f9f540cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5415f340 0x7f9f541f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 --2- 172.21.15.40:0/3968269972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f540d5610 0x7f9f540cfd20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 >> 172.21.15.40:0/3968269972 conn(0x7f9f5408ec70 msgr2=0x7f9f540b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 shutdown_connections 2024-04-26T21:12:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.523+0000 7f9f5c047640 1 -- 172.21.15.40:0/3968269972 wait complete. 2024-04-26T21:12:59.535 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:12:59.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:12:59.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 Processor -- start 2024-04-26T21:12:59.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 -- start start 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f370 0x7fc0ec0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0ec15f750 0x7fc0ec0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec0d03a0 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ec0b4d10 con 0x7fc0ec15f750 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0f2789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec0d0680 con 0x7fc0ec15f370 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.635+0000 7fc0eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0ec15f750 0x7fc0ec0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0ec15f750 0x7fc0ec0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38160/0 (socket says 172.21.15.40:38160) 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 -- 172.21.15.40:0/2383174861 learned_addr learned my addr 172.21.15.40:0/2383174861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0ebfff640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f370 0x7fc0ec0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 -- 172.21.15.40:0/2383174861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f370 msgr2=0x7fc0ec0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:12:59.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f370 0x7fc0ec0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 -- 172.21.15.40:0/2383174861 >> v1:172.21.15.184:6789/0 conn(0x7fc0ec0d37d0 legacy=0x7fc0ec0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.636+0000 7fc0eb7fe640 1 -- 172.21.15.40:0/2383174861 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0d0960 con 0x7fc0ec15f750 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0eb7fe640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec15f750 0x7fc0ec0b47d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009870 tx=0x7fc0e002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecefc70f30757c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0eaffd640 1 -- 172.21.15.40:0/2383174861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0037070 con 0x7fc0ec15f750 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0eaffd640 1 -- 172.21.15.40:0/2383174861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e0032440 con 0x7fc0ec15f750 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0eaffd640 1 -- 172.21.15.40:0/2383174861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0032810 con 0x7fc0ec15f750 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 -- 172.21.15.40:0/2383174861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec15f750 msgr2=0x7fc0ec0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec15f750 0x7fc0ec0b47d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009870 tx=0x7fc0e002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 -- 172.21.15.40:0/2383174861 shutdown_connections 2024-04-26T21:12:59.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec15f750 0x7fc0ec0b47d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 --2- 172.21.15.40:0/2383174861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f370 0x7fc0ec0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 -- 172.21.15.40:0/2383174861 >> 172.21.15.40:0/2383174861 conn(0x7fc0ec08ec10 msgr2=0x7fc0ec09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 -- 172.21.15.40:0/2383174861 shutdown_connections 2024-04-26T21:12:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.637+0000 7fc0f2789640 1 -- 172.21.15.40:0/2383174861 wait complete. 2024-04-26T21:12:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.638+0000 7fc0f2789640 1 Processor -- start 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 -- start start 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec15f370 0x7fc0ec0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f750 0x7fc0ec0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0b51c0 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0d0f20 con 0x7fc0ec15f370 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0f2789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec0b5340 con 0x7fc0ec15f750 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.639+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38176/0 (socket says 172.21.15.40:38176) 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 -- 172.21.15.40:0/818942865 learned_addr learned my addr 172.21.15.40:0/818942865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0eb7fe640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec15f370 0x7fc0ec0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0f0d26640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f750 0x7fc0ec0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 -- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f750 msgr2=0x7fc0ec0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f750 0x7fc0ec0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 -- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec15f370 msgr2=0x7fc0ec0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec15f370 0x7fc0ec0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 -- 172.21.15.40:0/818942865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0cc800 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0ebfff640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00d800 tx=0x7fc0dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00b660 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc019070 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec0ccaa0 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00ba60 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.640+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec0cd060 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.641+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec06a290 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.642+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0dc00bc00 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.642+0000 7fc0e8ff9640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.642+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc0dc08c000 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.642+0000 7fc0eb7fe640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:12:59.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.643+0000 7fc0eb7fe640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 0x7fc0d806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:12:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.647+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0dc05a430 con 0x7fc0ec0d37d0 2024-04-26T21:12:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.828+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0ec0d3530 con 0x7fc0d806c8f0 2024-04-26T21:12:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.835+0000 7fc0e8ff9640 1 -- 172.21.15.40:0/818942865 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251287 (secure 0 0 0) 0x7fc0ec0d3530 con 0x7fc0d806c8f0 2024-04-26T21:12:59.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:12:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.837+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 msgr2=0x7fc0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.837+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 0x7fc0d806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).stop 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.837+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 msgr2=0x7fc0ec162ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.837+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00d800 tx=0x7fc0dc00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 shutdown_connections 2024-04-26T21:12:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0ec15f750 0x7fc0ec0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0ec15f370 0x7fc0ec0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 --2- 172.21.15.40:0/818942865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0d37d0 0x7fc0ec162ae0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 >> 172.21.15.40:0/818942865 conn(0x7fc0ec08ec10 msgr2=0x7fc0ec0d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 shutdown_connections 2024-04-26T21:12:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.838+0000 7fc0f2789640 1 -- 172.21.15.40:0/818942865 wait complete. 2024-04-26T21:12:59.850 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T21:12:58.931964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220496,"kb_used_data":4308,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4411392,"data_stored":2421212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121725"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835219+0000","last_change":"2024-04-26T21:12:55.834874+0000","last_active":"2024-04-26T21:12:56.835219+0000","last_peered":"2024-04-26T21:12:56.835219+0000","last_clean":"2024-04-26T21:12:56.835219+0000","last_became_active":"2024-04-26T21:12:55.834593+0000","last_became_peered":"2024-04-26T21:12:55.834593+0000","last_unstale":"2024-04-26T21:12:56.835219+0000","last_undegraded":"2024-04-26T21:12:56.835219+0000","last_fullsized":"2024-04-26T21:12:56.835219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:30.540994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018250+0000","last_change":"2024-04-26T21:12:55.843688+0000","last_active":"2024-04-26T21:12:57.018250+0000","last_peered":"2024-04-26T21:12:57.018250+0000","last_clean":"2024-04-26T21:12:57.018250+0000","last_became_active":"2024-04-26T21:12:55.843298+0000","last_became_peered":"2024-04-26T21:12:55.843298+0000","last_unstale":"2024-04-26T21:12:57.018250+0000","last_undegraded":"2024-04-26T21:12:57.018250+0000","last_fullsized":"2024-04-26T21:12:57.018250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835869+0000","last_change":"2024-04-26T21:12:55.847059+0000","last_active":"2024-04-26T21:12:56.835869+0000","last_peered":"2024-04-26T21:12:56.835869+0000","last_clean":"2024-04-26T21:12:56.835869+0000","last_became_active":"2024-04-26T21:12:55.846609+0000","last_became_peered":"2024-04-26T21:12:55.846609+0000","last_unstale":"2024-04-26T21:12:56.835869+0000","last_undegraded":"2024-04-26T21:12:56.835869+0000","last_fullsized":"2024-04-26T21:12:56.835869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.826216+0000","last_change":"2024-04-26T21:12:54.826216+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.826216+0000","last_undegraded":"2024-04-26T21:12:54.826216+0000","last_fullsized":"2024-04-26T21:12:54.826216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829415+0000","last_change":"2024-04-26T21:12:55.845666+0000","last_active":"2024-04-26T21:12:56.829415+0000","last_peered":"2024-04-26T21:12:56.829415+0000","last_clean":"2024-04-26T21:12:56.829415+0000","last_became_active":"2024-04-26T21:12:55.845327+0000","last_became_peered":"2024-04-26T21:12:55.845327+0000","last_unstale":"2024-04-26T21:12:56.829415+0000","last_undegraded":"2024-04-26T21:12:56.829415+0000","last_fullsized":"2024-04-26T21:12:56.829415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:20.848985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018048+0000","last_change":"2024-04-26T21:12:55.843903+0000","last_active":"2024-04-26T21:12:57.018048+0000","last_peered":"2024-04-26T21:12:57.018048+0000","last_clean":"2024-04-26T21:12:57.018048+0000","last_became_active":"2024-04-26T21:12:55.842999+0000","last_became_peered":"2024-04-26T21:12:55.842999+0000","last_unstale":"2024-04-26T21:12:57.018048+0000","last_undegraded":"2024-04-26T21:12:57.018048+0000","last_fullsized":"2024-04-26T21:12:57.018048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018110+0000","last_change":"2024-04-26T21:12:55.829330+0000","last_active":"2024-04-26T21:12:57.018110+0000","last_peered":"2024-04-26T21:12:57.018110+0000","last_clean":"2024-04-26T21:12:57.018110+0000","last_became_active":"2024-04-26T21:12:55.828931+0000","last_became_peered":"2024-04-26T21:12:55.828931+0000","last_unstale":"2024-04-26T21:12:57.018110+0000","last_undegraded":"2024-04-26T21:12:57.018110+0000","last_fullsized":"2024-04-26T21:12:57.018110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.829652+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.829377+0000","last_became_peered":"2024-04-26T21:12:55.829377+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:29.407920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018385+0000","last_change":"2024-04-26T21:12:55.839415+0000","last_active":"2024-04-26T21:12:57.018385+0000","last_peered":"2024-04-26T21:12:57.018385+0000","last_clean":"2024-04-26T21:12:57.018385+0000","last_became_active":"2024-04-26T21:12:55.839122+0000","last_became_peered":"2024-04-26T21:12:55.839122+0000","last_unstale":"2024-04-26T21:12:57.018385+0000","last_undegraded":"2024-04-26T21:12:57.018385+0000","last_fullsized":"2024-04-26T21:12:57.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:28.305031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825703+0000","last_change":"2024-04-26T21:12:54.825703+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825703+0000","last_undegraded":"2024-04-26T21:12:54.825703+0000","last_fullsized":"2024-04-26T21:12:54.825703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018636+0000","last_change":"2024-04-26T21:12:55.845593+0000","last_active":"2024-04-26T21:12:57.018636+0000","last_peered":"2024-04-26T21:12:57.018636+0000","last_clean":"2024-04-26T21:12:57.018636+0000","last_became_active":"2024-04-26T21:12:55.845268+0000","last_became_peered":"2024-04-26T21:12:55.845268+0000","last_unstale":"2024-04-26T21:12:57.018636+0000","last_undegraded":"2024-04-26T21:12:57.018636+0000","last_fullsized":"2024-04-26T21:12:57.018636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835098+0000","last_change":"2024-04-26T21:12:55.845017+0000","last_active":"2024-04-26T21:12:56.835098+0000","last_peered":"2024-04-26T21:12:56.835098+0000","last_clean":"2024-04-26T21:12:56.835098+0000","last_became_active":"2024-04-26T21:12:55.844675+0000","last_became_peered":"2024-04-26T21:12:55.844675+0000","last_unstale":"2024-04-26T21:12:56.835098+0000","last_undegraded":"2024-04-26T21:12:56.835098+0000","last_fullsized":"2024-04-26T21:12:56.835098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829449+0000","last_change":"2024-04-26T21:12:55.848662+0000","last_active":"2024-04-26T21:12:56.829449+0000","last_peered":"2024-04-26T21:12:56.829449+0000","last_clean":"2024-04-26T21:12:56.829449+0000","last_became_active":"2024-04-26T21:12:55.848361+0000","last_became_peered":"2024-04-26T21:12:55.848361+0000","last_unstale":"2024-04-26T21:12:56.829449+0000","last_undegraded":"2024-04-26T21:12:56.829449+0000","last_fullsized":"2024-04-26T21:12:56.829449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:27.246380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018174+0000","last_change":"2024-04-26T21:12:55.851294+0000","last_active":"2024-04-26T21:12:57.018174+0000","last_peered":"2024-04-26T21:12:57.018174+0000","last_clean":"2024-04-26T21:12:57.018174+0000","last_became_active":"2024-04-26T21:12:55.850964+0000","last_became_peered":"2024-04-26T21:12:55.850964+0000","last_unstale":"2024-04-26T21:12:57.018174+0000","last_undegraded":"2024-04-26T21:12:57.018174+0000","last_fullsized":"2024-04-26T21:12:57.018174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835931+0000","last_change":"2024-04-26T21:12:55.827567+0000","last_active":"2024-04-26T21:12:56.835931+0000","last_peered":"2024-04-26T21:12:56.835931+0000","last_clean":"2024-04-26T21:12:56.835931+0000","last_became_active":"2024-04-26T21:12:55.827292+0000","last_became_peered":"2024-04-26T21:12:55.827292+0000","last_unstale":"2024-04-26T21:12:56.835931+0000","last_undegraded":"2024-04-26T21:12:56.835931+0000","last_fullsized":"2024-04-26T21:12:56.835931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017985+0000","last_change":"2024-04-26T21:12:55.848969+0000","last_active":"2024-04-26T21:12:57.017985+0000","last_peered":"2024-04-26T21:12:57.017985+0000","last_clean":"2024-04-26T21:12:57.017985+0000","last_became_active":"2024-04-26T21:12:55.848716+0000","last_became_peered":"2024-04-26T21:12:55.848716+0000","last_unstale":"2024-04-26T21:12:57.017985+0000","last_undegraded":"2024-04-26T21:12:57.017985+0000","last_fullsized":"2024-04-26T21:12:57.017985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018861+0000","last_change":"2024-04-26T21:12:55.834371+0000","last_active":"2024-04-26T21:12:57.018861+0000","last_peered":"2024-04-26T21:12:57.018861+0000","last_clean":"2024-04-26T21:12:57.018861+0000","last_became_active":"2024-04-26T21:12:55.834069+0000","last_became_peered":"2024-04-26T21:12:55.834069+0000","last_unstale":"2024-04-26T21:12:57.018861+0000","last_undegraded":"2024-04-26T21:12:57.018861+0000","last_fullsized":"2024-04-26T21:12:57.018861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018317+0000","last_change":"2024-04-26T21:12:55.852224+0000","last_active":"2024-04-26T21:12:57.018317+0000","last_peered":"2024-04-26T21:12:57.018317+0000","last_clean":"2024-04-26T21:12:57.018317+0000","last_became_active":"2024-04-26T21:12:55.851851+0000","last_became_peered":"2024-04-26T21:12:55.851851+0000","last_unstale":"2024-04-26T21:12:57.018317+0000","last_undegraded":"2024-04-26T21:12:57.018317+0000","last_fullsized":"2024-04-26T21:12:57.018317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:18.378959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017681+0000","last_change":"2024-04-26T21:12:55.852436+0000","last_active":"2024-04-26T21:12:57.017681+0000","last_peered":"2024-04-26T21:12:57.017681+0000","last_clean":"2024-04-26T21:12:57.017681+0000","last_became_active":"2024-04-26T21:12:55.852148+0000","last_became_peered":"2024-04-26T21:12:55.852148+0000","last_unstale":"2024-04-26T21:12:57.017681+0000","last_undegraded":"2024-04-26T21:12:57.017681+0000","last_fullsized":"2024-04-26T21:12:57.017681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829635+0000","last_change":"2024-04-26T21:12:55.833830+0000","last_active":"2024-04-26T21:12:56.829635+0000","last_peered":"2024-04-26T21:12:56.829635+0000","last_clean":"2024-04-26T21:12:56.829635+0000","last_became_active":"2024-04-26T21:12:55.833323+0000","last_became_peered":"2024-04-26T21:12:55.833323+0000","last_unstale":"2024-04-26T21:12:56.829635+0000","last_undegraded":"2024-04-26T21:12:56.829635+0000","last_fullsized":"2024-04-26T21:12:56.829635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:18.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018226+0000","last_change":"2024-04-26T21:12:55.831331+0000","last_active":"2024-04-26T21:12:57.018226+0000","last_peered":"2024-04-26T21:12:57.018226+0000","last_clean":"2024-04-26T21:12:57.018226+0000","last_became_active":"2024-04-26T21:12:55.830490+0000","last_became_peered":"2024-04-26T21:12:55.830490+0000","last_unstale":"2024-04-26T21:12:57.018226+0000","last_undegraded":"2024-04-26T21:12:57.018226+0000","last_fullsized":"2024-04-26T21:12:57.018226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:40.323141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018258+0000","last_change":"2024-04-26T21:12:55.848889+0000","last_active":"2024-04-26T21:12:57.018258+0000","last_peered":"2024-04-26T21:12:57.018258+0000","last_clean":"2024-04-26T21:12:57.018258+0000","last_became_active":"2024-04-26T21:12:55.848607+0000","last_became_peered":"2024-04-26T21:12:55.848607+0000","last_unstale":"2024-04-26T21:12:57.018258+0000","last_undegraded":"2024-04-26T21:12:57.018258+0000","last_fullsized":"2024-04-26T21:12:57.018258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017553+0000","last_change":"2024-04-26T21:12:55.834830+0000","last_active":"2024-04-26T21:12:57.017553+0000","last_peered":"2024-04-26T21:12:57.017553+0000","last_clean":"2024-04-26T21:12:57.017553+0000","last_became_active":"2024-04-26T21:12:55.834541+0000","last_became_peered":"2024-04-26T21:12:55.834541+0000","last_unstale":"2024-04-26T21:12:57.017553+0000","last_undegraded":"2024-04-26T21:12:57.017553+0000","last_fullsized":"2024-04-26T21:12:57.017553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835083+0000","last_change":"2024-04-26T21:12:55.829794+0000","last_active":"2024-04-26T21:12:56.835083+0000","last_peered":"2024-04-26T21:12:56.835083+0000","last_clean":"2024-04-26T21:12:56.835083+0000","last_became_active":"2024-04-26T21:12:55.829261+0000","last_became_peered":"2024-04-26T21:12:55.829261+0000","last_unstale":"2024-04-26T21:12:56.835083+0000","last_undegraded":"2024-04-26T21:12:56.835083+0000","last_fullsized":"2024-04-26T21:12:56.835083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824632+0000","last_change":"2024-04-26T21:12:54.824632+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824632+0000","last_undegraded":"2024-04-26T21:12:54.824632+0000","last_fullsized":"2024-04-26T21:12:54.824632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018383+0000","last_change":"2024-04-26T21:12:55.847342+0000","last_active":"2024-04-26T21:12:57.018383+0000","last_peered":"2024-04-26T21:12:57.018383+0000","last_clean":"2024-04-26T21:12:57.018383+0000","last_became_active":"2024-04-26T21:12:55.846476+0000","last_became_peered":"2024-04-26T21:12:55.846476+0000","last_unstale":"2024-04-26T21:12:57.018383+0000","last_undegraded":"2024-04-26T21:12:57.018383+0000","last_fullsized":"2024-04-26T21:12:57.018383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:29.560745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825269+0000","last_change":"2024-04-26T21:12:54.825269+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825269+0000","last_undegraded":"2024-04-26T21:12:54.825269+0000","last_fullsized":"2024-04-26T21:12:54.825269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835252+0000","last_change":"2024-04-26T21:12:55.833992+0000","last_active":"2024-04-26T21:12:56.835252+0000","last_peered":"2024-04-26T21:12:56.835252+0000","last_clean":"2024-04-26T21:12:56.835252+0000","last_became_active":"2024-04-26T21:12:55.833496+0000","last_became_peered":"2024-04-26T21:12:55.833496+0000","last_unstale":"2024-04-26T21:12:56.835252+0000","last_undegraded":"2024-04-26T21:12:56.835252+0000","last_fullsized":"2024-04-26T21:12:56.835252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018305+0000","last_change":"2024-04-26T21:12:55.846206+0000","last_active":"2024-04-26T21:12:57.018305+0000","last_peered":"2024-04-26T21:12:57.018305+0000","last_clean":"2024-04-26T21:12:57.018305+0000","last_became_active":"2024-04-26T21:12:55.845924+0000","last_became_peered":"2024-04-26T21:12:55.845924+0000","last_unstale":"2024-04-26T21:12:57.018305+0000","last_undegraded":"2024-04-26T21:12:57.018305+0000","last_fullsized":"2024-04-26T21:12:57.018305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:12.137923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018052+0000","last_change":"2024-04-26T21:12:55.833028+0000","last_active":"2024-04-26T21:12:57.018052+0000","last_peered":"2024-04-26T21:12:57.018052+0000","last_clean":"2024-04-26T21:12:57.018052+0000","last_became_active":"2024-04-26T21:12:55.832783+0000","last_became_peered":"2024-04-26T21:12:55.832783+0000","last_unstale":"2024-04-26T21:12:57.018052+0000","last_undegraded":"2024-04-26T21:12:57.018052+0000","last_fullsized":"2024-04-26T21:12:57.018052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835242+0000","last_change":"2024-04-26T21:12:55.829964+0000","last_active":"2024-04-26T21:12:56.835242+0000","last_peered":"2024-04-26T21:12:56.835242+0000","last_clean":"2024-04-26T21:12:56.835242+0000","last_became_active":"2024-04-26T21:12:55.828615+0000","last_became_peered":"2024-04-26T21:12:55.828615+0000","last_unstale":"2024-04-26T21:12:56.835242+0000","last_undegraded":"2024-04-26T21:12:56.835242+0000","last_fullsized":"2024-04-26T21:12:56.835242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:33.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017806+0000","last_change":"2024-04-26T21:12:55.835339+0000","last_active":"2024-04-26T21:12:57.017806+0000","last_peered":"2024-04-26T21:12:57.017806+0000","last_clean":"2024-04-26T21:12:57.017806+0000","last_became_active":"2024-04-26T21:12:55.834346+0000","last_became_peered":"2024-04-26T21:12:55.834346+0000","last_unstale":"2024-04-26T21:12:57.017806+0000","last_undegraded":"2024-04-26T21:12:57.017806+0000","last_fullsized":"2024-04-26T21:12:57.017806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017889+0000","last_change":"2024-04-26T21:12:55.835570+0000","last_active":"2024-04-26T21:12:57.017889+0000","last_peered":"2024-04-26T21:12:57.017889+0000","last_clean":"2024-04-26T21:12:57.017889+0000","last_became_active":"2024-04-26T21:12:55.835290+0000","last_became_peered":"2024-04-26T21:12:55.835290+0000","last_unstale":"2024-04-26T21:12:57.017889+0000","last_undegraded":"2024-04-26T21:12:57.017889+0000","last_fullsized":"2024-04-26T21:12:57.017889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835504+0000","last_change":"2024-04-26T21:12:54.836243+0000","last_active":"2024-04-26T21:12:56.835504+0000","last_peered":"2024-04-26T21:12:56.835504+0000","last_clean":"2024-04-26T21:12:56.835504+0000","last_became_active":"2024-04-26T21:12:54.835926+0000","last_became_peered":"2024-04-26T21:12:54.835926+0000","last_unstale":"2024-04-26T21:12:56.835504+0000","last_undegraded":"2024-04-26T21:12:56.835504+0000","last_fullsized":"2024-04-26T21:12:56.835504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835388+0000","last_change":"2024-04-26T21:12:55.833085+0000","last_active":"2024-04-26T21:12:56.835388+0000","last_peered":"2024-04-26T21:12:56.835388+0000","last_clean":"2024-04-26T21:12:56.835388+0000","last_became_active":"2024-04-26T21:12:55.832730+0000","last_became_peered":"2024-04-26T21:12:55.832730+0000","last_unstale":"2024-04-26T21:12:56.835388+0000","last_undegraded":"2024-04-26T21:12:56.835388+0000","last_fullsized":"2024-04-26T21:12:56.835388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018001+0000","last_change":"2024-04-26T21:12:54.828997+0000","last_active":"2024-04-26T21:12:57.018001+0000","last_peered":"2024-04-26T21:12:57.018001+0000","last_clean":"2024-04-26T21:12:57.018001+0000","last_became_active":"2024-04-26T21:12:54.828635+0000","last_became_peered":"2024-04-26T21:12:54.828635+0000","last_unstale":"2024-04-26T21:12:57.018001+0000","last_undegraded":"2024-04-26T21:12:57.018001+0000","last_fullsized":"2024-04-26T21:12:57.018001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018665+0000","last_change":"2024-04-26T21:12:55.846106+0000","last_active":"2024-04-26T21:12:57.018665+0000","last_peered":"2024-04-26T21:12:57.018665+0000","last_clean":"2024-04-26T21:12:57.018665+0000","last_became_active":"2024-04-26T21:12:55.845816+0000","last_became_peered":"2024-04-26T21:12:55.845816+0000","last_unstale":"2024-04-26T21:12:57.018665+0000","last_undegraded":"2024-04-26T21:12:57.018665+0000","last_fullsized":"2024-04-26T21:12:57.018665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:44.371719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017598+0000","last_change":"2024-04-26T21:12:54.841177+0000","last_active":"2024-04-26T21:12:57.017598+0000","last_peered":"2024-04-26T21:12:57.017598+0000","last_clean":"2024-04-26T21:12:57.017598+0000","last_became_active":"2024-04-26T21:12:54.840803+0000","last_became_peered":"2024-04-26T21:12:54.840803+0000","last_unstale":"2024-04-26T21:12:57.017598+0000","last_undegraded":"2024-04-26T21:12:57.017598+0000","last_fullsized":"2024-04-26T21:12:57.017598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.922188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824483+0000","last_change":"2024-04-26T21:12:54.824483+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824483+0000","last_undegraded":"2024-04-26T21:12:54.824483+0000","last_fullsized":"2024-04-26T21:12:54.824483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829652+0000","last_change":"2024-04-26T21:12:54.833964+0000","last_active":"2024-04-26T21:12:56.829652+0000","last_peered":"2024-04-26T21:12:56.829652+0000","last_clean":"2024-04-26T21:12:56.829652+0000","last_became_active":"2024-04-26T21:12:54.833662+0000","last_became_peered":"2024-04-26T21:12:54.833662+0000","last_unstale":"2024-04-26T21:12:56.829652+0000","last_undegraded":"2024-04-26T21:12:56.829652+0000","last_fullsized":"2024-04-26T21:12:56.829652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:11.073082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018468+0000","last_change":"2024-04-26T21:12:55.839174+0000","last_active":"2024-04-26T21:12:57.018468+0000","last_peered":"2024-04-26T21:12:57.018468+0000","last_clean":"2024-04-26T21:12:57.018468+0000","last_became_active":"2024-04-26T21:12:55.838902+0000","last_became_peered":"2024-04-26T21:12:55.838902+0000","last_unstale":"2024-04-26T21:12:57.018468+0000","last_undegraded":"2024-04-26T21:12:57.018468+0000","last_fullsized":"2024-04-26T21:12:57.018468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018348+0000","last_change":"2024-04-26T21:12:54.838674+0000","last_active":"2024-04-26T21:12:57.018348+0000","last_peered":"2024-04-26T21:12:57.018348+0000","last_clean":"2024-04-26T21:12:57.018348+0000","last_became_active":"2024-04-26T21:12:54.838357+0000","last_became_peered":"2024-04-26T21:12:54.838357+0000","last_unstale":"2024-04-26T21:12:57.018348+0000","last_undegraded":"2024-04-26T21:12:57.018348+0000","last_fullsized":"2024-04-26T21:12:57.018348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835420+0000","last_change":"2024-04-26T21:12:55.833557+0000","last_active":"2024-04-26T21:12:56.835420+0000","last_peered":"2024-04-26T21:12:56.835420+0000","last_clean":"2024-04-26T21:12:56.835420+0000","last_became_active":"2024-04-26T21:12:55.833295+0000","last_became_peered":"2024-04-26T21:12:55.833295+0000","last_unstale":"2024-04-26T21:12:56.835420+0000","last_undegraded":"2024-04-26T21:12:56.835420+0000","last_fullsized":"2024-04-26T21:12:56.835420+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018262+0000","last_change":"2024-04-26T21:12:54.837240+0000","last_active":"2024-04-26T21:12:57.018262+0000","last_peered":"2024-04-26T21:12:57.018262+0000","last_clean":"2024-04-26T21:12:57.018262+0000","last_became_active":"2024-04-26T21:12:54.837022+0000","last_became_peered":"2024-04-26T21:12:54.837022+0000","last_unstale":"2024-04-26T21:12:57.018262+0000","last_undegraded":"2024-04-26T21:12:57.018262+0000","last_fullsized":"2024-04-26T21:12:57.018262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:20.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017917+0000","last_change":"2024-04-26T21:12:55.828118+0000","last_active":"2024-04-26T21:12:57.017917+0000","last_peered":"2024-04-26T21:12:57.017917+0000","last_clean":"2024-04-26T21:12:57.017917+0000","last_became_active":"2024-04-26T21:12:55.827805+0000","last_became_peered":"2024-04-26T21:12:55.827805+0000","last_unstale":"2024-04-26T21:12:57.017917+0000","last_undegraded":"2024-04-26T21:12:57.017917+0000","last_fullsized":"2024-04-26T21:12:57.017917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017693+0000","last_change":"2024-04-26T21:12:54.835535+0000","last_active":"2024-04-26T21:12:57.017693+0000","last_peered":"2024-04-26T21:12:57.017693+0000","last_clean":"2024-04-26T21:12:57.017693+0000","last_became_active":"2024-04-26T21:12:54.835258+0000","last_became_peered":"2024-04-26T21:12:54.835258+0000","last_unstale":"2024-04-26T21:12:57.017693+0000","last_undegraded":"2024-04-26T21:12:57.017693+0000","last_fullsized":"2024-04-26T21:12:57.017693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.823608+0000","last_change":"2024-04-26T21:12:54.823608+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.823608+0000","last_undegraded":"2024-04-26T21:12:54.823608+0000","last_fullsized":"2024-04-26T21:12:54.823608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017760+0000","last_change":"2024-04-26T21:12:54.833665+0000","last_active":"2024-04-26T21:12:57.017760+0000","last_peered":"2024-04-26T21:12:57.017760+0000","last_clean":"2024-04-26T21:12:57.017760+0000","last_became_active":"2024-04-26T21:12:54.833394+0000","last_became_peered":"2024-04-26T21:12:54.833394+0000","last_unstale":"2024-04-26T21:12:57.017760+0000","last_undegraded":"2024-04-26T21:12:57.017760+0000","last_fullsized":"2024-04-26T21:12:57.017760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:18.903622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.836233+0000","last_change":"2024-04-26T21:12:55.844791+0000","last_active":"2024-04-26T21:12:56.836233+0000","last_peered":"2024-04-26T21:12:56.836233+0000","last_clean":"2024-04-26T21:12:56.836233+0000","last_became_active":"2024-04-26T21:12:55.844567+0000","last_became_peered":"2024-04-26T21:12:55.844567+0000","last_unstale":"2024-04-26T21:12:56.836233+0000","last_undegraded":"2024-04-26T21:12:56.836233+0000","last_fullsized":"2024-04-26T21:12:56.836233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017815+0000","last_change":"2024-04-26T21:12:54.842287+0000","last_active":"2024-04-26T21:12:57.017815+0000","last_peered":"2024-04-26T21:12:57.017815+0000","last_clean":"2024-04-26T21:12:57.017815+0000","last_became_active":"2024-04-26T21:12:54.841969+0000","last_became_peered":"2024-04-26T21:12:54.841969+0000","last_unstale":"2024-04-26T21:12:57.017815+0000","last_undegraded":"2024-04-26T21:12:57.017815+0000","last_fullsized":"2024-04-26T21:12:57.017815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.740507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017709+0000","last_change":"2024-04-26T21:12:55.852140+0000","last_active":"2024-04-26T21:12:57.017709+0000","last_peered":"2024-04-26T21:12:57.017709+0000","last_clean":"2024-04-26T21:12:57.017709+0000","last_became_active":"2024-04-26T21:12:55.851753+0000","last_became_peered":"2024-04-26T21:12:55.851753+0000","last_unstale":"2024-04-26T21:12:57.017709+0000","last_undegraded":"2024-04-26T21:12:57.017709+0000","last_fullsized":"2024-04-26T21:12:57.017709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:54.832871+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:54.832587+0000","last_became_peered":"2024-04-26T21:12:54.832587+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017990+0000","last_change":"2024-04-26T21:12:55.851660+0000","last_active":"2024-04-26T21:12:57.017990+0000","last_peered":"2024-04-26T21:12:57.017990+0000","last_clean":"2024-04-26T21:12:57.017990+0000","last_became_active":"2024-04-26T21:12:55.850979+0000","last_became_peered":"2024-04-26T21:12:55.850979+0000","last_unstale":"2024-04-26T21:12:57.017990+0000","last_undegraded":"2024-04-26T21:12:57.017990+0000","last_fullsized":"2024-04-26T21:12:57.017990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835911+0000","last_change":"2024-04-26T21:12:54.830694+0000","last_active":"2024-04-26T21:12:56.835911+0000","last_peered":"2024-04-26T21:12:56.835911+0000","last_clean":"2024-04-26T21:12:56.835911+0000","last_became_active":"2024-04-26T21:12:54.830373+0000","last_became_peered":"2024-04-26T21:12:54.830373+0000","last_unstale":"2024-04-26T21:12:56.835911+0000","last_undegraded":"2024-04-26T21:12:56.835911+0000","last_fullsized":"2024-04-26T21:12:56.835911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835683+0000","last_change":"2024-04-26T21:12:55.846932+0000","last_active":"2024-04-26T21:12:56.835683+0000","last_peered":"2024-04-26T21:12:56.835683+0000","last_clean":"2024-04-26T21:12:56.835683+0000","last_became_active":"2024-04-26T21:12:55.846616+0000","last_became_peered":"2024-04-26T21:12:55.846616+0000","last_unstale":"2024-04-26T21:12:56.835683+0000","last_undegraded":"2024-04-26T21:12:56.835683+0000","last_fullsized":"2024-04-26T21:12:56.835683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017971+0000","last_change":"2024-04-26T21:12:54.840841+0000","last_active":"2024-04-26T21:12:57.017971+0000","last_peered":"2024-04-26T21:12:57.017971+0000","last_clean":"2024-04-26T21:12:57.017971+0000","last_became_active":"2024-04-26T21:12:54.840336+0000","last_became_peered":"2024-04-26T21:12:54.840336+0000","last_unstale":"2024-04-26T21:12:57.017971+0000","last_undegraded":"2024-04-26T21:12:57.017971+0000","last_fullsized":"2024-04-26T21:12:57.017971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824074+0000","last_change":"2024-04-26T21:12:54.824074+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824074+0000","last_undegraded":"2024-04-26T21:12:54.824074+0000","last_fullsized":"2024-04-26T21:12:54.824074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018411+0000","last_change":"2024-04-26T21:12:54.833969+0000","last_active":"2024-04-26T21:12:57.018411+0000","last_peered":"2024-04-26T21:12:57.018411+0000","last_clean":"2024-04-26T21:12:57.018411+0000","last_became_active":"2024-04-26T21:12:54.833628+0000","last_became_peered":"2024-04-26T21:12:54.833628+0000","last_unstale":"2024-04-26T21:12:57.018411+0000","last_undegraded":"2024-04-26T21:12:57.018411+0000","last_fullsized":"2024-04-26T21:12:57.018411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.844863+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.844486+0000","last_became_peered":"2024-04-26T21:12:55.844486+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.836902+0000","last_change":"2024-04-26T21:12:54.831096+0000","last_active":"2024-04-26T21:12:54.836902+0000","last_peered":"2024-04-26T21:12:54.836902+0000","last_clean":"2024-04-26T21:12:54.836902+0000","last_became_active":"2024-04-26T21:12:54.830785+0000","last_became_peered":"2024-04-26T21:12:54.830785+0000","last_unstale":"2024-04-26T21:12:54.836902+0000","last_undegraded":"2024-04-26T21:12:54.836902+0000","last_fullsized":"2024-04-26T21:12:54.836902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:19.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018645+0000","last_change":"2024-04-26T21:12:55.834991+0000","last_active":"2024-04-26T21:12:57.018645+0000","last_peered":"2024-04-26T21:12:57.018645+0000","last_clean":"2024-04-26T21:12:57.018645+0000","last_became_active":"2024-04-26T21:12:55.834663+0000","last_became_peered":"2024-04-26T21:12:55.834663+0000","last_unstale":"2024-04-26T21:12:57.018645+0000","last_undegraded":"2024-04-26T21:12:57.018645+0000","last_fullsized":"2024-04-26T21:12:57.018645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017474+0000","last_change":"2024-04-26T21:12:54.842208+0000","last_active":"2024-04-26T21:12:57.017474+0000","last_peered":"2024-04-26T21:12:57.017474+0000","last_clean":"2024-04-26T21:12:57.017474+0000","last_became_active":"2024-04-26T21:12:54.841932+0000","last_became_peered":"2024-04-26T21:12:54.841932+0000","last_unstale":"2024-04-26T21:12:57.017474+0000","last_undegraded":"2024-04-26T21:12:57.017474+0000","last_fullsized":"2024-04-26T21:12:57.017474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:29.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835302+0000","last_change":"2024-04-26T21:12:55.830389+0000","last_active":"2024-04-26T21:12:56.835302+0000","last_peered":"2024-04-26T21:12:56.835302+0000","last_clean":"2024-04-26T21:12:56.835302+0000","last_became_active":"2024-04-26T21:12:55.830080+0000","last_became_peered":"2024-04-26T21:12:55.830080+0000","last_unstale":"2024-04-26T21:12:56.835302+0000","last_undegraded":"2024-04-26T21:12:56.835302+0000","last_fullsized":"2024-04-26T21:12:56.835302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017777+0000","last_change":"2024-04-26T21:12:54.833724+0000","last_active":"2024-04-26T21:12:57.017777+0000","last_peered":"2024-04-26T21:12:57.017777+0000","last_clean":"2024-04-26T21:12:57.017777+0000","last_became_active":"2024-04-26T21:12:54.833458+0000","last_became_peered":"2024-04-26T21:12:54.833458+0000","last_unstale":"2024-04-26T21:12:57.017777+0000","last_undegraded":"2024-04-26T21:12:57.017777+0000","last_fullsized":"2024-04-26T21:12:57.017777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:26.631042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:55.835158+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:55.834796+0000","last_became_peered":"2024-04-26T21:12:55.834796+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840552+0000","last_change":"2024-04-26T21:12:54.837631+0000","last_active":"2024-04-26T21:12:54.840552+0000","last_peered":"2024-04-26T21:12:54.840552+0000","last_clean":"2024-04-26T21:12:54.840552+0000","last_became_active":"2024-04-26T21:12:54.837117+0000","last_became_peered":"2024-04-26T21:12:54.837117+0000","last_unstale":"2024-04-26T21:12:54.840552+0000","last_undegraded":"2024-04-26T21:12:54.840552+0000","last_fullsized":"2024-04-26T21:12:54.840552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:00.880559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835652+0000","last_change":"2024-04-26T21:12:55.837208+0000","last_active":"2024-04-26T21:12:56.835652+0000","last_peered":"2024-04-26T21:12:56.835652+0000","last_clean":"2024-04-26T21:12:56.835652+0000","last_became_active":"2024-04-26T21:12:55.836934+0000","last_became_peered":"2024-04-26T21:12:55.836934+0000","last_unstale":"2024-04-26T21:12:56.835652+0000","last_undegraded":"2024-04-26T21:12:56.835652+0000","last_fullsized":"2024-04-26T21:12:56.835652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018106+0000","last_change":"2024-04-26T21:12:54.833728+0000","last_active":"2024-04-26T21:12:57.018106+0000","last_peered":"2024-04-26T21:12:57.018106+0000","last_clean":"2024-04-26T21:12:57.018106+0000","last_became_active":"2024-04-26T21:12:54.833388+0000","last_became_peered":"2024-04-26T21:12:54.833388+0000","last_unstale":"2024-04-26T21:12:57.018106+0000","last_undegraded":"2024-04-26T21:12:57.018106+0000","last_fullsized":"2024-04-26T21:12:57.018106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825701+0000","last_change":"2024-04-26T21:12:54.825701+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825701+0000","last_undegraded":"2024-04-26T21:12:54.825701+0000","last_fullsized":"2024-04-26T21:12:54.825701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018083+0000","last_change":"2024-04-26T21:12:54.832422+0000","last_active":"2024-04-26T21:12:57.018083+0000","last_peered":"2024-04-26T21:12:57.018083+0000","last_clean":"2024-04-26T21:12:57.018083+0000","last_became_active":"2024-04-26T21:12:54.832171+0000","last_became_peered":"2024-04-26T21:12:54.832171+0000","last_unstale":"2024-04-26T21:12:57.018083+0000","last_undegraded":"2024-04-26T21:12:57.018083+0000","last_fullsized":"2024-04-26T21:12:57.018083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:33.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829689+0000","last_change":"2024-04-26T21:12:55.840206+0000","last_active":"2024-04-26T21:12:56.829689+0000","last_peered":"2024-04-26T21:12:56.829689+0000","last_clean":"2024-04-26T21:12:56.829689+0000","last_became_active":"2024-04-26T21:12:55.839915+0000","last_became_peered":"2024-04-26T21:12:55.839915+0000","last_unstale":"2024-04-26T21:12:56.829689+0000","last_undegraded":"2024-04-26T21:12:56.829689+0000","last_fullsized":"2024-04-26T21:12:56.829689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.934501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018739+0000","last_change":"2024-04-26T21:12:54.838655+0000","last_active":"2024-04-26T21:12:57.018739+0000","last_peered":"2024-04-26T21:12:57.018739+0000","last_clean":"2024-04-26T21:12:57.018739+0000","last_became_active":"2024-04-26T21:12:54.838244+0000","last_became_peered":"2024-04-26T21:12:54.838244+0000","last_unstale":"2024-04-26T21:12:57.018739+0000","last_undegraded":"2024-04-26T21:12:57.018739+0000","last_fullsized":"2024-04-26T21:12:57.018739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:26.055217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.834792+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.834492+0000","last_became_peered":"2024-04-26T21:12:55.834492+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:12.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018066+0000","last_change":"2024-04-26T21:12:54.834125+0000","last_active":"2024-04-26T21:12:57.018066+0000","last_peered":"2024-04-26T21:12:57.018066+0000","last_clean":"2024-04-26T21:12:57.018066+0000","last_became_active":"2024-04-26T21:12:54.833830+0000","last_became_peered":"2024-04-26T21:12:54.833830+0000","last_unstale":"2024-04-26T21:12:57.018066+0000","last_undegraded":"2024-04-26T21:12:57.018066+0000","last_fullsized":"2024-04-26T21:12:57.018066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:17.724781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829576+0000","last_change":"2024-04-26T21:12:55.829614+0000","last_active":"2024-04-26T21:12:56.829576+0000","last_peered":"2024-04-26T21:12:56.829576+0000","last_clean":"2024-04-26T21:12:56.829576+0000","last_became_active":"2024-04-26T21:12:55.829102+0000","last_became_peered":"2024-04-26T21:12:55.829102+0000","last_unstale":"2024-04-26T21:12:56.829576+0000","last_undegraded":"2024-04-26T21:12:56.829576+0000","last_fullsized":"2024-04-26T21:12:56.829576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.539573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835517+0000","last_change":"2024-04-26T21:12:54.833584+0000","last_active":"2024-04-26T21:12:56.835517+0000","last_peered":"2024-04-26T21:12:56.835517+0000","last_clean":"2024-04-26T21:12:56.835517+0000","last_became_active":"2024-04-26T21:12:54.833302+0000","last_became_peered":"2024-04-26T21:12:54.833302+0000","last_unstale":"2024-04-26T21:12:56.835517+0000","last_undegraded":"2024-04-26T21:12:56.835517+0000","last_fullsized":"2024-04-26T21:12:56.835517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.829378+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.829065+0000","last_became_peered":"2024-04-26T21:12:55.829065+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:01.592713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017725+0000","last_change":"2024-04-26T21:12:54.834382+0000","last_active":"2024-04-26T21:12:57.017725+0000","last_peered":"2024-04-26T21:12:57.017725+0000","last_clean":"2024-04-26T21:12:57.017725+0000","last_became_active":"2024-04-26T21:12:54.833899+0000","last_became_peered":"2024-04-26T21:12:54.833899+0000","last_unstale":"2024-04-26T21:12:57.017725+0000","last_undegraded":"2024-04-26T21:12:57.017725+0000","last_fullsized":"2024-04-26T21:12:57.017725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:11.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018255+0000","last_change":"2024-04-26T21:12:55.831832+0000","last_active":"2024-04-26T21:12:57.018255+0000","last_peered":"2024-04-26T21:12:57.018255+0000","last_clean":"2024-04-26T21:12:57.018255+0000","last_became_active":"2024-04-26T21:12:55.831550+0000","last_became_peered":"2024-04-26T21:12:55.831550+0000","last_unstale":"2024-04-26T21:12:57.018255+0000","last_undegraded":"2024-04-26T21:12:57.018255+0000","last_fullsized":"2024-04-26T21:12:57.018255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835186+0000","last_change":"2024-04-26T21:12:54.830666+0000","last_active":"2024-04-26T21:12:56.835186+0000","last_peered":"2024-04-26T21:12:56.835186+0000","last_clean":"2024-04-26T21:12:56.835186+0000","last_became_active":"2024-04-26T21:12:54.830338+0000","last_became_peered":"2024-04-26T21:12:54.830338+0000","last_unstale":"2024-04-26T21:12:56.835186+0000","last_undegraded":"2024-04-26T21:12:56.835186+0000","last_fullsized":"2024-04-26T21:12:56.835186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829680+0000","last_change":"2024-04-26T21:12:55.834082+0000","last_active":"2024-04-26T21:12:56.829680+0000","last_peered":"2024-04-26T21:12:56.829680+0000","last_clean":"2024-04-26T21:12:56.829680+0000","last_became_active":"2024-04-26T21:12:55.833778+0000","last_became_peered":"2024-04-26T21:12:55.833778+0000","last_unstale":"2024-04-26T21:12:56.829680+0000","last_undegraded":"2024-04-26T21:12:56.829680+0000","last_fullsized":"2024-04-26T21:12:56.829680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:57.707945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840467+0000","last_change":"2024-04-26T21:12:54.837054+0000","last_active":"2024-04-26T21:12:54.840467+0000","last_peered":"2024-04-26T21:12:54.840467+0000","last_clean":"2024-04-26T21:12:54.840467+0000","last_became_active":"2024-04-26T21:12:54.836690+0000","last_became_peered":"2024-04-26T21:12:54.836690+0000","last_unstale":"2024-04-26T21:12:54.840467+0000","last_undegraded":"2024-04-26T21:12:54.840467+0000","last_fullsized":"2024-04-26T21:12:54.840467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:53.292509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835819+0000","last_change":"2024-04-26T21:12:55.840516+0000","last_active":"2024-04-26T21:12:56.835819+0000","last_peered":"2024-04-26T21:12:56.835819+0000","last_clean":"2024-04-26T21:12:56.835819+0000","last_became_active":"2024-04-26T21:12:55.840212+0000","last_became_peered":"2024-04-26T21:12:55.840212+0000","last_unstale":"2024-04-26T21:12:56.835819+0000","last_undegraded":"2024-04-26T21:12:56.835819+0000","last_fullsized":"2024-04-26T21:12:56.835819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018109+0000","last_change":"2024-04-26T21:12:54.834650+0000","last_active":"2024-04-26T21:12:57.018109+0000","last_peered":"2024-04-26T21:12:57.018109+0000","last_clean":"2024-04-26T21:12:57.018109+0000","last_became_active":"2024-04-26T21:12:54.834365+0000","last_became_peered":"2024-04-26T21:12:54.834365+0000","last_unstale":"2024-04-26T21:12:57.018109+0000","last_undegraded":"2024-04-26T21:12:57.018109+0000","last_fullsized":"2024-04-26T21:12:57.018109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017612+0000","last_change":"2024-04-26T21:12:55.846844+0000","last_active":"2024-04-26T21:12:57.017612+0000","last_peered":"2024-04-26T21:12:57.017612+0000","last_clean":"2024-04-26T21:12:57.017612+0000","last_became_active":"2024-04-26T21:12:55.846587+0000","last_became_peered":"2024-04-26T21:12:55.846587+0000","last_unstale":"2024-04-26T21:12:57.017612+0000","last_undegraded":"2024-04-26T21:12:57.017612+0000","last_fullsized":"2024-04-26T21:12:57.017612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839967+0000","last_change":"2024-04-26T21:12:54.833945+0000","last_active":"2024-04-26T21:12:54.839967+0000","last_peered":"2024-04-26T21:12:54.839967+0000","last_clean":"2024-04-26T21:12:54.839967+0000","last_became_active":"2024-04-26T21:12:54.833186+0000","last_became_peered":"2024-04-26T21:12:54.833186+0000","last_unstale":"2024-04-26T21:12:54.839967+0000","last_undegraded":"2024-04-26T21:12:54.839967+0000","last_fullsized":"2024-04-26T21:12:54.839967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:47.875355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835471+0000","last_change":"2024-04-26T21:12:55.831340+0000","last_active":"2024-04-26T21:12:56.835471+0000","last_peered":"2024-04-26T21:12:56.835471+0000","last_clean":"2024-04-26T21:12:56.835471+0000","last_became_active":"2024-04-26T21:12:55.831068+0000","last_became_peered":"2024-04-26T21:12:55.831068+0000","last_unstale":"2024-04-26T21:12:56.835471+0000","last_undegraded":"2024-04-26T21:12:56.835471+0000","last_fullsized":"2024-04-26T21:12:56.835471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018242+0000","last_change":"2024-04-26T21:12:54.825357+0000","last_active":"2024-04-26T21:12:57.018242+0000","last_peered":"2024-04-26T21:12:57.018242+0000","last_clean":"2024-04-26T21:12:57.018242+0000","last_became_active":"2024-04-26T21:12:54.824948+0000","last_became_peered":"2024-04-26T21:12:54.824948+0000","last_unstale":"2024-04-26T21:12:57.018242+0000","last_undegraded":"2024-04-26T21:12:57.018242+0000","last_fullsized":"2024-04-26T21:12:57.018242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:56.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017961+0000","last_change":"2024-04-26T21:12:55.832196+0000","last_active":"2024-04-26T21:12:57.017961+0000","last_peered":"2024-04-26T21:12:57.017961+0000","last_clean":"2024-04-26T21:12:57.017961+0000","last_became_active":"2024-04-26T21:12:55.831921+0000","last_became_peered":"2024-04-26T21:12:55.831921+0000","last_unstale":"2024-04-26T21:12:57.017961+0000","last_undegraded":"2024-04-26T21:12:57.017961+0000","last_fullsized":"2024-04-26T21:12:57.017961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839796+0000","last_change":"2024-04-26T21:12:54.833203+0000","last_active":"2024-04-26T21:12:54.839796+0000","last_peered":"2024-04-26T21:12:54.839796+0000","last_clean":"2024-04-26T21:12:54.839796+0000","last_became_active":"2024-04-26T21:12:54.832884+0000","last_became_peered":"2024-04-26T21:12:54.832884+0000","last_unstale":"2024-04-26T21:12:54.839796+0000","last_undegraded":"2024-04-26T21:12:54.839796+0000","last_fullsized":"2024-04-26T21:12:54.839796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:31.880236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017397+0000","last_change":"2024-04-26T21:12:55.848208+0000","last_active":"2024-04-26T21:12:57.017397+0000","last_peered":"2024-04-26T21:12:57.017397+0000","last_clean":"2024-04-26T21:12:57.017397+0000","last_became_active":"2024-04-26T21:12:55.847911+0000","last_became_peered":"2024-04-26T21:12:55.847911+0000","last_unstale":"2024-04-26T21:12:57.017397+0000","last_undegraded":"2024-04-26T21:12:57.017397+0000","last_fullsized":"2024-04-26T21:12:57.017397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018179+0000","last_change":"2024-04-26T21:12:54.829043+0000","last_active":"2024-04-26T21:12:57.018179+0000","last_peered":"2024-04-26T21:12:57.018179+0000","last_clean":"2024-04-26T21:12:57.018179+0000","last_became_active":"2024-04-26T21:12:54.828752+0000","last_became_peered":"2024-04-26T21:12:54.828752+0000","last_unstale":"2024-04-26T21:12:57.018179+0000","last_undegraded":"2024-04-26T21:12:57.018179+0000","last_fullsized":"2024-04-26T21:12:57.018179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017744+0000","last_change":"2024-04-26T21:12:55.846026+0000","last_active":"2024-04-26T21:12:57.017744+0000","last_peered":"2024-04-26T21:12:57.017744+0000","last_clean":"2024-04-26T21:12:57.017744+0000","last_became_active":"2024-04-26T21:12:55.845739+0000","last_became_peered":"2024-04-26T21:12:55.845739+0000","last_unstale":"2024-04-26T21:12:57.017744+0000","last_undegraded":"2024-04-26T21:12:57.017744+0000","last_fullsized":"2024-04-26T21:12:57.017744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017377+0000","last_change":"2024-04-26T21:12:54.842196+0000","last_active":"2024-04-26T21:12:57.017377+0000","last_peered":"2024-04-26T21:12:57.017377+0000","last_clean":"2024-04-26T21:12:57.017377+0000","last_became_active":"2024-04-26T21:12:54.841890+0000","last_became_peered":"2024-04-26T21:12:54.841890+0000","last_unstale":"2024-04-26T21:12:57.017377+0000","last_undegraded":"2024-04-26T21:12:57.017377+0000","last_fullsized":"2024-04-26T21:12:57.017377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:46.928170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018560+0000","last_change":"2024-04-26T21:12:55.829002+0000","last_active":"2024-04-26T21:12:57.018560+0000","last_peered":"2024-04-26T21:12:57.018560+0000","last_clean":"2024-04-26T21:12:57.018560+0000","last_became_active":"2024-04-26T21:12:55.828722+0000","last_became_peered":"2024-04-26T21:12:55.828722+0000","last_unstale":"2024-04-26T21:12:57.018560+0000","last_undegraded":"2024-04-26T21:12:57.018560+0000","last_fullsized":"2024-04-26T21:12:57.018560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.834952+0000","last_change":"2024-04-26T21:12:54.833670+0000","last_active":"2024-04-26T21:12:56.834952+0000","last_peered":"2024-04-26T21:12:56.834952+0000","last_clean":"2024-04-26T21:12:56.834952+0000","last_became_active":"2024-04-26T21:12:54.833382+0000","last_became_peered":"2024-04-26T21:12:54.833382+0000","last_unstale":"2024-04-26T21:12:56.834952+0000","last_undegraded":"2024-04-26T21:12:56.834952+0000","last_fullsized":"2024-04-26T21:12:56.834952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018043+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:57.018043+0000","last_peered":"2024-04-26T21:12:57.018043+0000","last_clean":"2024-04-26T21:12:57.018043+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:57.018043+0000","last_undegraded":"2024-04-26T21:12:57.018043+0000","last_fullsized":"2024-04-26T21:12:57.018043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":747537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":157149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:12:59.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:13:00.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.989+0000 7f5b44724640 1 Processor -- start 2024-04-26T21:13:00.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.989+0000 7f5b44724640 1 -- start start 2024-04-26T21:13:00.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.990+0000 7f5b44724640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b3c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.990+0000 7f5b44724640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3c0cda60 con 0x7f5b3c15d360 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.990+0000 7f5b44724640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3c0d1100 con 0x7f5b3c0d1510 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.990+0000 7f5b44724640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3c15d740 con 0x7f5b3c0d0b20 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b424c0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5b3c15d360 0x7f5b3c09c450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42798/0 (socket says 172.21.15.40:42798) 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b424c0640 1 -- 172.21.15.40:0/853025081 learned_addr learned my addr 172.21.15.40:0/853025081 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b41cbf640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b3c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154706012 0 0) 0x7f5b3c0cda60 con 0x7f5b3c15d360 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b24003520 con 0x7f5b3c15d360 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1878920008 0 0) 0x7f5b24003520 con 0x7f5b3c15d360 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 msgr2=0x7f5b3c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b3c0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 >> v1:172.21.15.184:6789/0 conn(0x7f5b3c0d1510 legacy=0x7f5b3c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c0cdd00 con 0x7f5b3c15d360 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5b34001670 con 0x7f5b3c15d360 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 >> v1:172.21.15.40:6789/0 conn(0x7f5b3c15d360 legacy=0x7f5b3c09c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b240039e0 0x7f5b24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b24004d40 0x7f5b240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c15d740 con 0x7f5b240039e0 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2400ba70 con 0x7f5b24004d40 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2400bc70 con 0x7f5b3c0d0b20 2024-04-26T21:13:00.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b424c0640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b240039e0 0x7f5b24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b41cbf640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.991+0000 7f5b42cc1640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b24004d40 0x7f5b240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 -- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 msgr2=0x7f5b24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b3c0d0b20 0x7f5b24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 -- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b24004d40 msgr2=0x7f5b240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b24004d40 0x7f5b240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 -- 172.21.15.40:0/853025081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b24003520 con 0x7f5b240039e0 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b42cc1640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b24004d40 0x7f5b240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.992+0000 7f5b424c0640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b240039e0 0x7f5b24003e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b34001eb0 tx=0x7f5b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e40aec3ba085d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34007d30 con 0x7f5b240039e0 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b340058d0 con 0x7f5b240039e0 2024-04-26T21:13:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b414be640 1 -- 172.21.15.40:0/853025081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34005ce0 con 0x7f5b240039e0 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 -- 172.21.15.40:0/853025081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b240039e0 msgr2=0x7f5b24003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 --2- 172.21.15.40:0/853025081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b240039e0 0x7f5b24003e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b34001eb0 tx=0x7f5b24005830 comp rx=0 tx=0).stop 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b42cc1640 1 -- 172.21.15.40:0/853025081 reap_dead start 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 -- 172.21.15.40:0/853025081 shutdown_connections 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 -- 172.21.15.40:0/853025081 >> 172.21.15.40:0/853025081 conn(0x7f5b3c0b4a40 msgr2=0x7f5b3c0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 -- 172.21.15.40:0/853025081 shutdown_connections 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.993+0000 7f5b44724640 1 -- 172.21.15.40:0/853025081 wait complete. 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.994+0000 7f5b44724640 1 Processor -- start 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.994+0000 7f5b44724640 1 -- start start 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 0x7f5b3c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 0x7f5b3c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c070ba0 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c070a20 con 0x7f5b3c15f550 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b44724640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3c070d20 con 0x7f5b240039e0 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b42cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b42cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38184/0 (socket says 172.21.15.40:38184) 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b42cc1640 1 -- 172.21.15.40:0/2030222311 learned_addr learned my addr 172.21.15.40:0/2030222311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b424c0640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 0x7f5b3c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.995+0000 7f5b41cbf640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 0x7f5b3c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 -- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 msgr2=0x7f5b3c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 0x7f5b3c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 -- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 msgr2=0x7f5b3c06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 0x7f5b3c06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 -- 172.21.15.40:0/2030222311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c0c7f30 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b424c0640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 0x7f5b3c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b41cbf640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 0x7f5b3c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b42cc1640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f80 tx=0x7f5b2c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c006b50 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c006d10 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c0c8210 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c003970 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.996+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c0c87d0 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.997+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c0693d0 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.997+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b2c01a020 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.998+0000 7f5b3affd640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 0x7f5b2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.998+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f5b2c08b180 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.998+0000 7f5b424c0640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 0x7f5b2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:12:59.999+0000 7f5b424c0640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 0x7f5b2806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b34033000 tx=0x7f5b3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.002+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b2c0595a0 con 0x7f5b3c0b0da0 2024-04-26T21:13:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.185+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b3c06f3b0 con 0x7f5b2806c920 2024-04-26T21:13:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.192+0000 7f5b3affd640 1 -- 172.21.15.40:0/2030222311 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251287 (secure 0 0 0) 0x7f5b34001530 con 0x7f5b2806c920 2024-04-26T21:13:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:00.196 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.194+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 msgr2=0x7f5b2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.194+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 0x7f5b2806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5b34033000 tx=0x7f5b3402e000 comp rx=0 tx=0).stop 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.194+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 msgr2=0x7f5b3c0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.194+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f80 tx=0x7f5b2c00eb90 comp rx=0 tx=0).stop 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 shutdown_connections 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5b2806c920 0x7f5b2806ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b3c0b0da0 0x7f5b3c0b1250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b3c15f550 0x7f5b3c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 --2- 172.21.15.40:0/2030222311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240039e0 0x7f5b3c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 >> 172.21.15.40:0/2030222311 conn(0x7f5b3c0b4a40 msgr2=0x7f5b3c15c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 shutdown_connections 2024-04-26T21:13:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.195+0000 7f5b44724640 1 -- 172.21.15.40:0/2030222311 wait complete. 2024-04-26T21:13:00.206 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T21:12:58.931964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220496,"kb_used_data":4308,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4411392,"data_stored":2421212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121725"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835219+0000","last_change":"2024-04-26T21:12:55.834874+0000","last_active":"2024-04-26T21:12:56.835219+0000","last_peered":"2024-04-26T21:12:56.835219+0000","last_clean":"2024-04-26T21:12:56.835219+0000","last_became_active":"2024-04-26T21:12:55.834593+0000","last_became_peered":"2024-04-26T21:12:55.834593+0000","last_unstale":"2024-04-26T21:12:56.835219+0000","last_undegraded":"2024-04-26T21:12:56.835219+0000","last_fullsized":"2024-04-26T21:12:56.835219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:30.540994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018250+0000","last_change":"2024-04-26T21:12:55.843688+0000","last_active":"2024-04-26T21:12:57.018250+0000","last_peered":"2024-04-26T21:12:57.018250+0000","last_clean":"2024-04-26T21:12:57.018250+0000","last_became_active":"2024-04-26T21:12:55.843298+0000","last_became_peered":"2024-04-26T21:12:55.843298+0000","last_unstale":"2024-04-26T21:12:57.018250+0000","last_undegraded":"2024-04-26T21:12:57.018250+0000","last_fullsized":"2024-04-26T21:12:57.018250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835869+0000","last_change":"2024-04-26T21:12:55.847059+0000","last_active":"2024-04-26T21:12:56.835869+0000","last_peered":"2024-04-26T21:12:56.835869+0000","last_clean":"2024-04-26T21:12:56.835869+0000","last_became_active":"2024-04-26T21:12:55.846609+0000","last_became_peered":"2024-04-26T21:12:55.846609+0000","last_unstale":"2024-04-26T21:12:56.835869+0000","last_undegraded":"2024-04-26T21:12:56.835869+0000","last_fullsized":"2024-04-26T21:12:56.835869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.826216+0000","last_change":"2024-04-26T21:12:54.826216+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.826216+0000","last_undegraded":"2024-04-26T21:12:54.826216+0000","last_fullsized":"2024-04-26T21:12:54.826216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829415+0000","last_change":"2024-04-26T21:12:55.845666+0000","last_active":"2024-04-26T21:12:56.829415+0000","last_peered":"2024-04-26T21:12:56.829415+0000","last_clean":"2024-04-26T21:12:56.829415+0000","last_became_active":"2024-04-26T21:12:55.845327+0000","last_became_peered":"2024-04-26T21:12:55.845327+0000","last_unstale":"2024-04-26T21:12:56.829415+0000","last_undegraded":"2024-04-26T21:12:56.829415+0000","last_fullsized":"2024-04-26T21:12:56.829415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:20.848985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018048+0000","last_change":"2024-04-26T21:12:55.843903+0000","last_active":"2024-04-26T21:12:57.018048+0000","last_peered":"2024-04-26T21:12:57.018048+0000","last_clean":"2024-04-26T21:12:57.018048+0000","last_became_active":"2024-04-26T21:12:55.842999+0000","last_became_peered":"2024-04-26T21:12:55.842999+0000","last_unstale":"2024-04-26T21:12:57.018048+0000","last_undegraded":"2024-04-26T21:12:57.018048+0000","last_fullsized":"2024-04-26T21:12:57.018048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018110+0000","last_change":"2024-04-26T21:12:55.829330+0000","last_active":"2024-04-26T21:12:57.018110+0000","last_peered":"2024-04-26T21:12:57.018110+0000","last_clean":"2024-04-26T21:12:57.018110+0000","last_became_active":"2024-04-26T21:12:55.828931+0000","last_became_peered":"2024-04-26T21:12:55.828931+0000","last_unstale":"2024-04-26T21:12:57.018110+0000","last_undegraded":"2024-04-26T21:12:57.018110+0000","last_fullsized":"2024-04-26T21:12:57.018110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.829652+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.829377+0000","last_became_peered":"2024-04-26T21:12:55.829377+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:29.407920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018385+0000","last_change":"2024-04-26T21:12:55.839415+0000","last_active":"2024-04-26T21:12:57.018385+0000","last_peered":"2024-04-26T21:12:57.018385+0000","last_clean":"2024-04-26T21:12:57.018385+0000","last_became_active":"2024-04-26T21:12:55.839122+0000","last_became_peered":"2024-04-26T21:12:55.839122+0000","last_unstale":"2024-04-26T21:12:57.018385+0000","last_undegraded":"2024-04-26T21:12:57.018385+0000","last_fullsized":"2024-04-26T21:12:57.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:28.305031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825703+0000","last_change":"2024-04-26T21:12:54.825703+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825703+0000","last_undegraded":"2024-04-26T21:12:54.825703+0000","last_fullsized":"2024-04-26T21:12:54.825703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018636+0000","last_change":"2024-04-26T21:12:55.845593+0000","last_active":"2024-04-26T21:12:57.018636+0000","last_peered":"2024-04-26T21:12:57.018636+0000","last_clean":"2024-04-26T21:12:57.018636+0000","last_became_active":"2024-04-26T21:12:55.845268+0000","last_became_peered":"2024-04-26T21:12:55.845268+0000","last_unstale":"2024-04-26T21:12:57.018636+0000","last_undegraded":"2024-04-26T21:12:57.018636+0000","last_fullsized":"2024-04-26T21:12:57.018636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835098+0000","last_change":"2024-04-26T21:12:55.845017+0000","last_active":"2024-04-26T21:12:56.835098+0000","last_peered":"2024-04-26T21:12:56.835098+0000","last_clean":"2024-04-26T21:12:56.835098+0000","last_became_active":"2024-04-26T21:12:55.844675+0000","last_became_peered":"2024-04-26T21:12:55.844675+0000","last_unstale":"2024-04-26T21:12:56.835098+0000","last_undegraded":"2024-04-26T21:12:56.835098+0000","last_fullsized":"2024-04-26T21:12:56.835098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829449+0000","last_change":"2024-04-26T21:12:55.848662+0000","last_active":"2024-04-26T21:12:56.829449+0000","last_peered":"2024-04-26T21:12:56.829449+0000","last_clean":"2024-04-26T21:12:56.829449+0000","last_became_active":"2024-04-26T21:12:55.848361+0000","last_became_peered":"2024-04-26T21:12:55.848361+0000","last_unstale":"2024-04-26T21:12:56.829449+0000","last_undegraded":"2024-04-26T21:12:56.829449+0000","last_fullsized":"2024-04-26T21:12:56.829449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:27.246380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018174+0000","last_change":"2024-04-26T21:12:55.851294+0000","last_active":"2024-04-26T21:12:57.018174+0000","last_peered":"2024-04-26T21:12:57.018174+0000","last_clean":"2024-04-26T21:12:57.018174+0000","last_became_active":"2024-04-26T21:12:55.850964+0000","last_became_peered":"2024-04-26T21:12:55.850964+0000","last_unstale":"2024-04-26T21:12:57.018174+0000","last_undegraded":"2024-04-26T21:12:57.018174+0000","last_fullsized":"2024-04-26T21:12:57.018174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835931+0000","last_change":"2024-04-26T21:12:55.827567+0000","last_active":"2024-04-26T21:12:56.835931+0000","last_peered":"2024-04-26T21:12:56.835931+0000","last_clean":"2024-04-26T21:12:56.835931+0000","last_became_active":"2024-04-26T21:12:55.827292+0000","last_became_peered":"2024-04-26T21:12:55.827292+0000","last_unstale":"2024-04-26T21:12:56.835931+0000","last_undegraded":"2024-04-26T21:12:56.835931+0000","last_fullsized":"2024-04-26T21:12:56.835931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017985+0000","last_change":"2024-04-26T21:12:55.848969+0000","last_active":"2024-04-26T21:12:57.017985+0000","last_peered":"2024-04-26T21:12:57.017985+0000","last_clean":"2024-04-26T21:12:57.017985+0000","last_became_active":"2024-04-26T21:12:55.848716+0000","last_became_peered":"2024-04-26T21:12:55.848716+0000","last_unstale":"2024-04-26T21:12:57.017985+0000","last_undegraded":"2024-04-26T21:12:57.017985+0000","last_fullsized":"2024-04-26T21:12:57.017985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018861+0000","last_change":"2024-04-26T21:12:55.834371+0000","last_active":"2024-04-26T21:12:57.018861+0000","last_peered":"2024-04-26T21:12:57.018861+0000","last_clean":"2024-04-26T21:12:57.018861+0000","last_became_active":"2024-04-26T21:12:55.834069+0000","last_became_peered":"2024-04-26T21:12:55.834069+0000","last_unstale":"2024-04-26T21:12:57.018861+0000","last_undegraded":"2024-04-26T21:12:57.018861+0000","last_fullsized":"2024-04-26T21:12:57.018861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018317+0000","last_change":"2024-04-26T21:12:55.852224+0000","last_active":"2024-04-26T21:12:57.018317+0000","last_peered":"2024-04-26T21:12:57.018317+0000","last_clean":"2024-04-26T21:12:57.018317+0000","last_became_active":"2024-04-26T21:12:55.851851+0000","last_became_peered":"2024-04-26T21:12:55.851851+0000","last_unstale":"2024-04-26T21:12:57.018317+0000","last_undegraded":"2024-04-26T21:12:57.018317+0000","last_fullsized":"2024-04-26T21:12:57.018317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:18.378959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017681+0000","last_change":"2024-04-26T21:12:55.852436+0000","last_active":"2024-04-26T21:12:57.017681+0000","last_peered":"2024-04-26T21:12:57.017681+0000","last_clean":"2024-04-26T21:12:57.017681+0000","last_became_active":"2024-04-26T21:12:55.852148+0000","last_became_peered":"2024-04-26T21:12:55.852148+0000","last_unstale":"2024-04-26T21:12:57.017681+0000","last_undegraded":"2024-04-26T21:12:57.017681+0000","last_fullsized":"2024-04-26T21:12:57.017681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829635+0000","last_change":"2024-04-26T21:12:55.833830+0000","last_active":"2024-04-26T21:12:56.829635+0000","last_peered":"2024-04-26T21:12:56.829635+0000","last_clean":"2024-04-26T21:12:56.829635+0000","last_became_active":"2024-04-26T21:12:55.833323+0000","last_became_peered":"2024-04-26T21:12:55.833323+0000","last_unstale":"2024-04-26T21:12:56.829635+0000","last_undegraded":"2024-04-26T21:12:56.829635+0000","last_fullsized":"2024-04-26T21:12:56.829635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:18.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018226+0000","last_change":"2024-04-26T21:12:55.831331+0000","last_active":"2024-04-26T21:12:57.018226+0000","last_peered":"2024-04-26T21:12:57.018226+0000","last_clean":"2024-04-26T21:12:57.018226+0000","last_became_active":"2024-04-26T21:12:55.830490+0000","last_became_peered":"2024-04-26T21:12:55.830490+0000","last_unstale":"2024-04-26T21:12:57.018226+0000","last_undegraded":"2024-04-26T21:12:57.018226+0000","last_fullsized":"2024-04-26T21:12:57.018226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:40.323141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018258+0000","last_change":"2024-04-26T21:12:55.848889+0000","last_active":"2024-04-26T21:12:57.018258+0000","last_peered":"2024-04-26T21:12:57.018258+0000","last_clean":"2024-04-26T21:12:57.018258+0000","last_became_active":"2024-04-26T21:12:55.848607+0000","last_became_peered":"2024-04-26T21:12:55.848607+0000","last_unstale":"2024-04-26T21:12:57.018258+0000","last_undegraded":"2024-04-26T21:12:57.018258+0000","last_fullsized":"2024-04-26T21:12:57.018258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017553+0000","last_change":"2024-04-26T21:12:55.834830+0000","last_active":"2024-04-26T21:12:57.017553+0000","last_peered":"2024-04-26T21:12:57.017553+0000","last_clean":"2024-04-26T21:12:57.017553+0000","last_became_active":"2024-04-26T21:12:55.834541+0000","last_became_peered":"2024-04-26T21:12:55.834541+0000","last_unstale":"2024-04-26T21:12:57.017553+0000","last_undegraded":"2024-04-26T21:12:57.017553+0000","last_fullsized":"2024-04-26T21:12:57.017553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835083+0000","last_change":"2024-04-26T21:12:55.829794+0000","last_active":"2024-04-26T21:12:56.835083+0000","last_peered":"2024-04-26T21:12:56.835083+0000","last_clean":"2024-04-26T21:12:56.835083+0000","last_became_active":"2024-04-26T21:12:55.829261+0000","last_became_peered":"2024-04-26T21:12:55.829261+0000","last_unstale":"2024-04-26T21:12:56.835083+0000","last_undegraded":"2024-04-26T21:12:56.835083+0000","last_fullsized":"2024-04-26T21:12:56.835083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824632+0000","last_change":"2024-04-26T21:12:54.824632+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824632+0000","last_undegraded":"2024-04-26T21:12:54.824632+0000","last_fullsized":"2024-04-26T21:12:54.824632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018383+0000","last_change":"2024-04-26T21:12:55.847342+0000","last_active":"2024-04-26T21:12:57.018383+0000","last_peered":"2024-04-26T21:12:57.018383+0000","last_clean":"2024-04-26T21:12:57.018383+0000","last_became_active":"2024-04-26T21:12:55.846476+0000","last_became_peered":"2024-04-26T21:12:55.846476+0000","last_unstale":"2024-04-26T21:12:57.018383+0000","last_undegraded":"2024-04-26T21:12:57.018383+0000","last_fullsized":"2024-04-26T21:12:57.018383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:29.560745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825269+0000","last_change":"2024-04-26T21:12:54.825269+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825269+0000","last_undegraded":"2024-04-26T21:12:54.825269+0000","last_fullsized":"2024-04-26T21:12:54.825269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835252+0000","last_change":"2024-04-26T21:12:55.833992+0000","last_active":"2024-04-26T21:12:56.835252+0000","last_peered":"2024-04-26T21:12:56.835252+0000","last_clean":"2024-04-26T21:12:56.835252+0000","last_became_active":"2024-04-26T21:12:55.833496+0000","last_became_peered":"2024-04-26T21:12:55.833496+0000","last_unstale":"2024-04-26T21:12:56.835252+0000","last_undegraded":"2024-04-26T21:12:56.835252+0000","last_fullsized":"2024-04-26T21:12:56.835252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018305+0000","last_change":"2024-04-26T21:12:55.846206+0000","last_active":"2024-04-26T21:12:57.018305+0000","last_peered":"2024-04-26T21:12:57.018305+0000","last_clean":"2024-04-26T21:12:57.018305+0000","last_became_active":"2024-04-26T21:12:55.845924+0000","last_became_peered":"2024-04-26T21:12:55.845924+0000","last_unstale":"2024-04-26T21:12:57.018305+0000","last_undegraded":"2024-04-26T21:12:57.018305+0000","last_fullsized":"2024-04-26T21:12:57.018305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:12.137923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018052+0000","last_change":"2024-04-26T21:12:55.833028+0000","last_active":"2024-04-26T21:12:57.018052+0000","last_peered":"2024-04-26T21:12:57.018052+0000","last_clean":"2024-04-26T21:12:57.018052+0000","last_became_active":"2024-04-26T21:12:55.832783+0000","last_became_peered":"2024-04-26T21:12:55.832783+0000","last_unstale":"2024-04-26T21:12:57.018052+0000","last_undegraded":"2024-04-26T21:12:57.018052+0000","last_fullsized":"2024-04-26T21:12:57.018052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835242+0000","last_change":"2024-04-26T21:12:55.829964+0000","last_active":"2024-04-26T21:12:56.835242+0000","last_peered":"2024-04-26T21:12:56.835242+0000","last_clean":"2024-04-26T21:12:56.835242+0000","last_became_active":"2024-04-26T21:12:55.828615+0000","last_became_peered":"2024-04-26T21:12:55.828615+0000","last_unstale":"2024-04-26T21:12:56.835242+0000","last_undegraded":"2024-04-26T21:12:56.835242+0000","last_fullsized":"2024-04-26T21:12:56.835242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:33.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017806+0000","last_change":"2024-04-26T21:12:55.835339+0000","last_active":"2024-04-26T21:12:57.017806+0000","last_peered":"2024-04-26T21:12:57.017806+0000","last_clean":"2024-04-26T21:12:57.017806+0000","last_became_active":"2024-04-26T21:12:55.834346+0000","last_became_peered":"2024-04-26T21:12:55.834346+0000","last_unstale":"2024-04-26T21:12:57.017806+0000","last_undegraded":"2024-04-26T21:12:57.017806+0000","last_fullsized":"2024-04-26T21:12:57.017806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017889+0000","last_change":"2024-04-26T21:12:55.835570+0000","last_active":"2024-04-26T21:12:57.017889+0000","last_peered":"2024-04-26T21:12:57.017889+0000","last_clean":"2024-04-26T21:12:57.017889+0000","last_became_active":"2024-04-26T21:12:55.835290+0000","last_became_peered":"2024-04-26T21:12:55.835290+0000","last_unstale":"2024-04-26T21:12:57.017889+0000","last_undegraded":"2024-04-26T21:12:57.017889+0000","last_fullsized":"2024-04-26T21:12:57.017889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835504+0000","last_change":"2024-04-26T21:12:54.836243+0000","last_active":"2024-04-26T21:12:56.835504+0000","last_peered":"2024-04-26T21:12:56.835504+0000","last_clean":"2024-04-26T21:12:56.835504+0000","last_became_active":"2024-04-26T21:12:54.835926+0000","last_became_peered":"2024-04-26T21:12:54.835926+0000","last_unstale":"2024-04-26T21:12:56.835504+0000","last_undegraded":"2024-04-26T21:12:56.835504+0000","last_fullsized":"2024-04-26T21:12:56.835504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835388+0000","last_change":"2024-04-26T21:12:55.833085+0000","last_active":"2024-04-26T21:12:56.835388+0000","last_peered":"2024-04-26T21:12:56.835388+0000","last_clean":"2024-04-26T21:12:56.835388+0000","last_became_active":"2024-04-26T21:12:55.832730+0000","last_became_peered":"2024-04-26T21:12:55.832730+0000","last_unstale":"2024-04-26T21:12:56.835388+0000","last_undegraded":"2024-04-26T21:12:56.835388+0000","last_fullsized":"2024-04-26T21:12:56.835388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018001+0000","last_change":"2024-04-26T21:12:54.828997+0000","last_active":"2024-04-26T21:12:57.018001+0000","last_peered":"2024-04-26T21:12:57.018001+0000","last_clean":"2024-04-26T21:12:57.018001+0000","last_became_active":"2024-04-26T21:12:54.828635+0000","last_became_peered":"2024-04-26T21:12:54.828635+0000","last_unstale":"2024-04-26T21:12:57.018001+0000","last_undegraded":"2024-04-26T21:12:57.018001+0000","last_fullsized":"2024-04-26T21:12:57.018001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018665+0000","last_change":"2024-04-26T21:12:55.846106+0000","last_active":"2024-04-26T21:12:57.018665+0000","last_peered":"2024-04-26T21:12:57.018665+0000","last_clean":"2024-04-26T21:12:57.018665+0000","last_became_active":"2024-04-26T21:12:55.845816+0000","last_became_peered":"2024-04-26T21:12:55.845816+0000","last_unstale":"2024-04-26T21:12:57.018665+0000","last_undegraded":"2024-04-26T21:12:57.018665+0000","last_fullsized":"2024-04-26T21:12:57.018665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:44.371719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017598+0000","last_change":"2024-04-26T21:12:54.841177+0000","last_active":"2024-04-26T21:12:57.017598+0000","last_peered":"2024-04-26T21:12:57.017598+0000","last_clean":"2024-04-26T21:12:57.017598+0000","last_became_active":"2024-04-26T21:12:54.840803+0000","last_became_peered":"2024-04-26T21:12:54.840803+0000","last_unstale":"2024-04-26T21:12:57.017598+0000","last_undegraded":"2024-04-26T21:12:57.017598+0000","last_fullsized":"2024-04-26T21:12:57.017598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.922188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824483+0000","last_change":"2024-04-26T21:12:54.824483+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824483+0000","last_undegraded":"2024-04-26T21:12:54.824483+0000","last_fullsized":"2024-04-26T21:12:54.824483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829652+0000","last_change":"2024-04-26T21:12:54.833964+0000","last_active":"2024-04-26T21:12:56.829652+0000","last_peered":"2024-04-26T21:12:56.829652+0000","last_clean":"2024-04-26T21:12:56.829652+0000","last_became_active":"2024-04-26T21:12:54.833662+0000","last_became_peered":"2024-04-26T21:12:54.833662+0000","last_unstale":"2024-04-26T21:12:56.829652+0000","last_undegraded":"2024-04-26T21:12:56.829652+0000","last_fullsized":"2024-04-26T21:12:56.829652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:11.073082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018468+0000","last_change":"2024-04-26T21:12:55.839174+0000","last_active":"2024-04-26T21:12:57.018468+0000","last_peered":"2024-04-26T21:12:57.018468+0000","last_clean":"2024-04-26T21:12:57.018468+0000","last_became_active":"2024-04-26T21:12:55.838902+0000","last_became_peered":"2024-04-26T21:12:55.838902+0000","last_unstale":"2024-04-26T21:12:57.018468+0000","last_undegraded":"2024-04-26T21:12:57.018468+0000","last_fullsized":"2024-04-26T21:12:57.018468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018348+0000","last_change":"2024-04-26T21:12:54.838674+0000","last_active":"2024-04-26T21:12:57.018348+0000","last_peered":"2024-04-26T21:12:57.018348+0000","last_clean":"2024-04-26T21:12:57.018348+0000","last_became_active":"2024-04-26T21:12:54.838357+0000","last_became_peered":"2024-04-26T21:12:54.838357+0000","last_unstale":"2024-04-26T21:12:57.018348+0000","last_undegraded":"2024-04-26T21:12:57.018348+0000","last_fullsized":"2024-04-26T21:12:57.018348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835420+0000","last_change":"2024-04-26T21:12:55.833557+0000","last_active":"2024-04-26T21:12:56.835420+0000","last_peered":"2024-04-26T21:12:56.835420+0000","last_clean":"2024-04-26T21:12:56.835420+0000","last_became_active":"2024-04-26T21:12:55.833295+0000","last_became_peered":"2024-04-26T21:12:55.833295+0000","last_unstale":"2024-04-26T21:12:56.835420+0000","last_undegraded":"2024-04-26T21:12:56.835420+0000","last_fullsized":"2024-04-26T21:12:56.835420+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018262+0000","last_change":"2024-04-26T21:12:54.837240+0000","last_active":"2024-04-26T21:12:57.018262+0000","last_peered":"2024-04-26T21:12:57.018262+0000","last_clean":"2024-04-26T21:12:57.018262+0000","last_became_active":"2024-04-26T21:12:54.837022+0000","last_became_peered":"2024-04-26T21:12:54.837022+0000","last_unstale":"2024-04-26T21:12:57.018262+0000","last_undegraded":"2024-04-26T21:12:57.018262+0000","last_fullsized":"2024-04-26T21:12:57.018262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:20.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017917+0000","last_change":"2024-04-26T21:12:55.828118+0000","last_active":"2024-04-26T21:12:57.017917+0000","last_peered":"2024-04-26T21:12:57.017917+0000","last_clean":"2024-04-26T21:12:57.017917+0000","last_became_active":"2024-04-26T21:12:55.827805+0000","last_became_peered":"2024-04-26T21:12:55.827805+0000","last_unstale":"2024-04-26T21:12:57.017917+0000","last_undegraded":"2024-04-26T21:12:57.017917+0000","last_fullsized":"2024-04-26T21:12:57.017917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017693+0000","last_change":"2024-04-26T21:12:54.835535+0000","last_active":"2024-04-26T21:12:57.017693+0000","last_peered":"2024-04-26T21:12:57.017693+0000","last_clean":"2024-04-26T21:12:57.017693+0000","last_became_active":"2024-04-26T21:12:54.835258+0000","last_became_peered":"2024-04-26T21:12:54.835258+0000","last_unstale":"2024-04-26T21:12:57.017693+0000","last_undegraded":"2024-04-26T21:12:57.017693+0000","last_fullsized":"2024-04-26T21:12:57.017693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.823608+0000","last_change":"2024-04-26T21:12:54.823608+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.823608+0000","last_undegraded":"2024-04-26T21:12:54.823608+0000","last_fullsized":"2024-04-26T21:12:54.823608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017760+0000","last_change":"2024-04-26T21:12:54.833665+0000","last_active":"2024-04-26T21:12:57.017760+0000","last_peered":"2024-04-26T21:12:57.017760+0000","last_clean":"2024-04-26T21:12:57.017760+0000","last_became_active":"2024-04-26T21:12:54.833394+0000","last_became_peered":"2024-04-26T21:12:54.833394+0000","last_unstale":"2024-04-26T21:12:57.017760+0000","last_undegraded":"2024-04-26T21:12:57.017760+0000","last_fullsized":"2024-04-26T21:12:57.017760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:18.903622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.836233+0000","last_change":"2024-04-26T21:12:55.844791+0000","last_active":"2024-04-26T21:12:56.836233+0000","last_peered":"2024-04-26T21:12:56.836233+0000","last_clean":"2024-04-26T21:12:56.836233+0000","last_became_active":"2024-04-26T21:12:55.844567+0000","last_became_peered":"2024-04-26T21:12:55.844567+0000","last_unstale":"2024-04-26T21:12:56.836233+0000","last_undegraded":"2024-04-26T21:12:56.836233+0000","last_fullsized":"2024-04-26T21:12:56.836233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017815+0000","last_change":"2024-04-26T21:12:54.842287+0000","last_active":"2024-04-26T21:12:57.017815+0000","last_peered":"2024-04-26T21:12:57.017815+0000","last_clean":"2024-04-26T21:12:57.017815+0000","last_became_active":"2024-04-26T21:12:54.841969+0000","last_became_peered":"2024-04-26T21:12:54.841969+0000","last_unstale":"2024-04-26T21:12:57.017815+0000","last_undegraded":"2024-04-26T21:12:57.017815+0000","last_fullsized":"2024-04-26T21:12:57.017815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.740507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017709+0000","last_change":"2024-04-26T21:12:55.852140+0000","last_active":"2024-04-26T21:12:57.017709+0000","last_peered":"2024-04-26T21:12:57.017709+0000","last_clean":"2024-04-26T21:12:57.017709+0000","last_became_active":"2024-04-26T21:12:55.851753+0000","last_became_peered":"2024-04-26T21:12:55.851753+0000","last_unstale":"2024-04-26T21:12:57.017709+0000","last_undegraded":"2024-04-26T21:12:57.017709+0000","last_fullsized":"2024-04-26T21:12:57.017709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:54.832871+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:54.832587+0000","last_became_peered":"2024-04-26T21:12:54.832587+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017990+0000","last_change":"2024-04-26T21:12:55.851660+0000","last_active":"2024-04-26T21:12:57.017990+0000","last_peered":"2024-04-26T21:12:57.017990+0000","last_clean":"2024-04-26T21:12:57.017990+0000","last_became_active":"2024-04-26T21:12:55.850979+0000","last_became_peered":"2024-04-26T21:12:55.850979+0000","last_unstale":"2024-04-26T21:12:57.017990+0000","last_undegraded":"2024-04-26T21:12:57.017990+0000","last_fullsized":"2024-04-26T21:12:57.017990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835911+0000","last_change":"2024-04-26T21:12:54.830694+0000","last_active":"2024-04-26T21:12:56.835911+0000","last_peered":"2024-04-26T21:12:56.835911+0000","last_clean":"2024-04-26T21:12:56.835911+0000","last_became_active":"2024-04-26T21:12:54.830373+0000","last_became_peered":"2024-04-26T21:12:54.830373+0000","last_unstale":"2024-04-26T21:12:56.835911+0000","last_undegraded":"2024-04-26T21:12:56.835911+0000","last_fullsized":"2024-04-26T21:12:56.835911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835683+0000","last_change":"2024-04-26T21:12:55.846932+0000","last_active":"2024-04-26T21:12:56.835683+0000","last_peered":"2024-04-26T21:12:56.835683+0000","last_clean":"2024-04-26T21:12:56.835683+0000","last_became_active":"2024-04-26T21:12:55.846616+0000","last_became_peered":"2024-04-26T21:12:55.846616+0000","last_unstale":"2024-04-26T21:12:56.835683+0000","last_undegraded":"2024-04-26T21:12:56.835683+0000","last_fullsized":"2024-04-26T21:12:56.835683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017971+0000","last_change":"2024-04-26T21:12:54.840841+0000","last_active":"2024-04-26T21:12:57.017971+0000","last_peered":"2024-04-26T21:12:57.017971+0000","last_clean":"2024-04-26T21:12:57.017971+0000","last_became_active":"2024-04-26T21:12:54.840336+0000","last_became_peered":"2024-04-26T21:12:54.840336+0000","last_unstale":"2024-04-26T21:12:57.017971+0000","last_undegraded":"2024-04-26T21:12:57.017971+0000","last_fullsized":"2024-04-26T21:12:57.017971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824074+0000","last_change":"2024-04-26T21:12:54.824074+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824074+0000","last_undegraded":"2024-04-26T21:12:54.824074+0000","last_fullsized":"2024-04-26T21:12:54.824074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018411+0000","last_change":"2024-04-26T21:12:54.833969+0000","last_active":"2024-04-26T21:12:57.018411+0000","last_peered":"2024-04-26T21:12:57.018411+0000","last_clean":"2024-04-26T21:12:57.018411+0000","last_became_active":"2024-04-26T21:12:54.833628+0000","last_became_peered":"2024-04-26T21:12:54.833628+0000","last_unstale":"2024-04-26T21:12:57.018411+0000","last_undegraded":"2024-04-26T21:12:57.018411+0000","last_fullsized":"2024-04-26T21:12:57.018411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.844863+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.844486+0000","last_became_peered":"2024-04-26T21:12:55.844486+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.836902+0000","last_change":"2024-04-26T21:12:54.831096+0000","last_active":"2024-04-26T21:12:54.836902+0000","last_peered":"2024-04-26T21:12:54.836902+0000","last_clean":"2024-04-26T21:12:54.836902+0000","last_became_active":"2024-04-26T21:12:54.830785+0000","last_became_peered":"2024-04-26T21:12:54.830785+0000","last_unstale":"2024-04-26T21:12:54.836902+0000","last_undegraded":"2024-04-26T21:12:54.836902+0000","last_fullsized":"2024-04-26T21:12:54.836902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:19.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018645+0000","last_change":"2024-04-26T21:12:55.834991+0000","last_active":"2024-04-26T21:12:57.018645+0000","last_peered":"2024-04-26T21:12:57.018645+0000","last_clean":"2024-04-26T21:12:57.018645+0000","last_became_active":"2024-04-26T21:12:55.834663+0000","last_became_peered":"2024-04-26T21:12:55.834663+0000","last_unstale":"2024-04-26T21:12:57.018645+0000","last_undegraded":"2024-04-26T21:12:57.018645+0000","last_fullsized":"2024-04-26T21:12:57.018645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017474+0000","last_change":"2024-04-26T21:12:54.842208+0000","last_active":"2024-04-26T21:12:57.017474+0000","last_peered":"2024-04-26T21:12:57.017474+0000","last_clean":"2024-04-26T21:12:57.017474+0000","last_became_active":"2024-04-26T21:12:54.841932+0000","last_became_peered":"2024-04-26T21:12:54.841932+0000","last_unstale":"2024-04-26T21:12:57.017474+0000","last_undegraded":"2024-04-26T21:12:57.017474+0000","last_fullsized":"2024-04-26T21:12:57.017474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:29.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835302+0000","last_change":"2024-04-26T21:12:55.830389+0000","last_active":"2024-04-26T21:12:56.835302+0000","last_peered":"2024-04-26T21:12:56.835302+0000","last_clean":"2024-04-26T21:12:56.835302+0000","last_became_active":"2024-04-26T21:12:55.830080+0000","last_became_peered":"2024-04-26T21:12:55.830080+0000","last_unstale":"2024-04-26T21:12:56.835302+0000","last_undegraded":"2024-04-26T21:12:56.835302+0000","last_fullsized":"2024-04-26T21:12:56.835302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017777+0000","last_change":"2024-04-26T21:12:54.833724+0000","last_active":"2024-04-26T21:12:57.017777+0000","last_peered":"2024-04-26T21:12:57.017777+0000","last_clean":"2024-04-26T21:12:57.017777+0000","last_became_active":"2024-04-26T21:12:54.833458+0000","last_became_peered":"2024-04-26T21:12:54.833458+0000","last_unstale":"2024-04-26T21:12:57.017777+0000","last_undegraded":"2024-04-26T21:12:57.017777+0000","last_fullsized":"2024-04-26T21:12:57.017777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:26.631042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:55.835158+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:55.834796+0000","last_became_peered":"2024-04-26T21:12:55.834796+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840552+0000","last_change":"2024-04-26T21:12:54.837631+0000","last_active":"2024-04-26T21:12:54.840552+0000","last_peered":"2024-04-26T21:12:54.840552+0000","last_clean":"2024-04-26T21:12:54.840552+0000","last_became_active":"2024-04-26T21:12:54.837117+0000","last_became_peered":"2024-04-26T21:12:54.837117+0000","last_unstale":"2024-04-26T21:12:54.840552+0000","last_undegraded":"2024-04-26T21:12:54.840552+0000","last_fullsized":"2024-04-26T21:12:54.840552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:00.880559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835652+0000","last_change":"2024-04-26T21:12:55.837208+0000","last_active":"2024-04-26T21:12:56.835652+0000","last_peered":"2024-04-26T21:12:56.835652+0000","last_clean":"2024-04-26T21:12:56.835652+0000","last_became_active":"2024-04-26T21:12:55.836934+0000","last_became_peered":"2024-04-26T21:12:55.836934+0000","last_unstale":"2024-04-26T21:12:56.835652+0000","last_undegraded":"2024-04-26T21:12:56.835652+0000","last_fullsized":"2024-04-26T21:12:56.835652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018106+0000","last_change":"2024-04-26T21:12:54.833728+0000","last_active":"2024-04-26T21:12:57.018106+0000","last_peered":"2024-04-26T21:12:57.018106+0000","last_clean":"2024-04-26T21:12:57.018106+0000","last_became_active":"2024-04-26T21:12:54.833388+0000","last_became_peered":"2024-04-26T21:12:54.833388+0000","last_unstale":"2024-04-26T21:12:57.018106+0000","last_undegraded":"2024-04-26T21:12:57.018106+0000","last_fullsized":"2024-04-26T21:12:57.018106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825701+0000","last_change":"2024-04-26T21:12:54.825701+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825701+0000","last_undegraded":"2024-04-26T21:12:54.825701+0000","last_fullsized":"2024-04-26T21:12:54.825701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018083+0000","last_change":"2024-04-26T21:12:54.832422+0000","last_active":"2024-04-26T21:12:57.018083+0000","last_peered":"2024-04-26T21:12:57.018083+0000","last_clean":"2024-04-26T21:12:57.018083+0000","last_became_active":"2024-04-26T21:12:54.832171+0000","last_became_peered":"2024-04-26T21:12:54.832171+0000","last_unstale":"2024-04-26T21:12:57.018083+0000","last_undegraded":"2024-04-26T21:12:57.018083+0000","last_fullsized":"2024-04-26T21:12:57.018083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:33.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829689+0000","last_change":"2024-04-26T21:12:55.840206+0000","last_active":"2024-04-26T21:12:56.829689+0000","last_peered":"2024-04-26T21:12:56.829689+0000","last_clean":"2024-04-26T21:12:56.829689+0000","last_became_active":"2024-04-26T21:12:55.839915+0000","last_became_peered":"2024-04-26T21:12:55.839915+0000","last_unstale":"2024-04-26T21:12:56.829689+0000","last_undegraded":"2024-04-26T21:12:56.829689+0000","last_fullsized":"2024-04-26T21:12:56.829689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.934501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018739+0000","last_change":"2024-04-26T21:12:54.838655+0000","last_active":"2024-04-26T21:12:57.018739+0000","last_peered":"2024-04-26T21:12:57.018739+0000","last_clean":"2024-04-26T21:12:57.018739+0000","last_became_active":"2024-04-26T21:12:54.838244+0000","last_became_peered":"2024-04-26T21:12:54.838244+0000","last_unstale":"2024-04-26T21:12:57.018739+0000","last_undegraded":"2024-04-26T21:12:57.018739+0000","last_fullsized":"2024-04-26T21:12:57.018739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:26.055217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.834792+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.834492+0000","last_became_peered":"2024-04-26T21:12:55.834492+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:12.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018066+0000","last_change":"2024-04-26T21:12:54.834125+0000","last_active":"2024-04-26T21:12:57.018066+0000","last_peered":"2024-04-26T21:12:57.018066+0000","last_clean":"2024-04-26T21:12:57.018066+0000","last_became_active":"2024-04-26T21:12:54.833830+0000","last_became_peered":"2024-04-26T21:12:54.833830+0000","last_unstale":"2024-04-26T21:12:57.018066+0000","last_undegraded":"2024-04-26T21:12:57.018066+0000","last_fullsized":"2024-04-26T21:12:57.018066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:17.724781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829576+0000","last_change":"2024-04-26T21:12:55.829614+0000","last_active":"2024-04-26T21:12:56.829576+0000","last_peered":"2024-04-26T21:12:56.829576+0000","last_clean":"2024-04-26T21:12:56.829576+0000","last_became_active":"2024-04-26T21:12:55.829102+0000","last_became_peered":"2024-04-26T21:12:55.829102+0000","last_unstale":"2024-04-26T21:12:56.829576+0000","last_undegraded":"2024-04-26T21:12:56.829576+0000","last_fullsized":"2024-04-26T21:12:56.829576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.539573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835517+0000","last_change":"2024-04-26T21:12:54.833584+0000","last_active":"2024-04-26T21:12:56.835517+0000","last_peered":"2024-04-26T21:12:56.835517+0000","last_clean":"2024-04-26T21:12:56.835517+0000","last_became_active":"2024-04-26T21:12:54.833302+0000","last_became_peered":"2024-04-26T21:12:54.833302+0000","last_unstale":"2024-04-26T21:12:56.835517+0000","last_undegraded":"2024-04-26T21:12:56.835517+0000","last_fullsized":"2024-04-26T21:12:56.835517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.829378+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.829065+0000","last_became_peered":"2024-04-26T21:12:55.829065+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:01.592713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017725+0000","last_change":"2024-04-26T21:12:54.834382+0000","last_active":"2024-04-26T21:12:57.017725+0000","last_peered":"2024-04-26T21:12:57.017725+0000","last_clean":"2024-04-26T21:12:57.017725+0000","last_became_active":"2024-04-26T21:12:54.833899+0000","last_became_peered":"2024-04-26T21:12:54.833899+0000","last_unstale":"2024-04-26T21:12:57.017725+0000","last_undegraded":"2024-04-26T21:12:57.017725+0000","last_fullsized":"2024-04-26T21:12:57.017725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:11.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018255+0000","last_change":"2024-04-26T21:12:55.831832+0000","last_active":"2024-04-26T21:12:57.018255+0000","last_peered":"2024-04-26T21:12:57.018255+0000","last_clean":"2024-04-26T21:12:57.018255+0000","last_became_active":"2024-04-26T21:12:55.831550+0000","last_became_peered":"2024-04-26T21:12:55.831550+0000","last_unstale":"2024-04-26T21:12:57.018255+0000","last_undegraded":"2024-04-26T21:12:57.018255+0000","last_fullsized":"2024-04-26T21:12:57.018255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835186+0000","last_change":"2024-04-26T21:12:54.830666+0000","last_active":"2024-04-26T21:12:56.835186+0000","last_peered":"2024-04-26T21:12:56.835186+0000","last_clean":"2024-04-26T21:12:56.835186+0000","last_became_active":"2024-04-26T21:12:54.830338+0000","last_became_peered":"2024-04-26T21:12:54.830338+0000","last_unstale":"2024-04-26T21:12:56.835186+0000","last_undegraded":"2024-04-26T21:12:56.835186+0000","last_fullsized":"2024-04-26T21:12:56.835186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829680+0000","last_change":"2024-04-26T21:12:55.834082+0000","last_active":"2024-04-26T21:12:56.829680+0000","last_peered":"2024-04-26T21:12:56.829680+0000","last_clean":"2024-04-26T21:12:56.829680+0000","last_became_active":"2024-04-26T21:12:55.833778+0000","last_became_peered":"2024-04-26T21:12:55.833778+0000","last_unstale":"2024-04-26T21:12:56.829680+0000","last_undegraded":"2024-04-26T21:12:56.829680+0000","last_fullsized":"2024-04-26T21:12:56.829680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:57.707945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840467+0000","last_change":"2024-04-26T21:12:54.837054+0000","last_active":"2024-04-26T21:12:54.840467+0000","last_peered":"2024-04-26T21:12:54.840467+0000","last_clean":"2024-04-26T21:12:54.840467+0000","last_became_active":"2024-04-26T21:12:54.836690+0000","last_became_peered":"2024-04-26T21:12:54.836690+0000","last_unstale":"2024-04-26T21:12:54.840467+0000","last_undegraded":"2024-04-26T21:12:54.840467+0000","last_fullsized":"2024-04-26T21:12:54.840467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:53.292509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835819+0000","last_change":"2024-04-26T21:12:55.840516+0000","last_active":"2024-04-26T21:12:56.835819+0000","last_peered":"2024-04-26T21:12:56.835819+0000","last_clean":"2024-04-26T21:12:56.835819+0000","last_became_active":"2024-04-26T21:12:55.840212+0000","last_became_peered":"2024-04-26T21:12:55.840212+0000","last_unstale":"2024-04-26T21:12:56.835819+0000","last_undegraded":"2024-04-26T21:12:56.835819+0000","last_fullsized":"2024-04-26T21:12:56.835819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018109+0000","last_change":"2024-04-26T21:12:54.834650+0000","last_active":"2024-04-26T21:12:57.018109+0000","last_peered":"2024-04-26T21:12:57.018109+0000","last_clean":"2024-04-26T21:12:57.018109+0000","last_became_active":"2024-04-26T21:12:54.834365+0000","last_became_peered":"2024-04-26T21:12:54.834365+0000","last_unstale":"2024-04-26T21:12:57.018109+0000","last_undegraded":"2024-04-26T21:12:57.018109+0000","last_fullsized":"2024-04-26T21:12:57.018109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017612+0000","last_change":"2024-04-26T21:12:55.846844+0000","last_active":"2024-04-26T21:12:57.017612+0000","last_peered":"2024-04-26T21:12:57.017612+0000","last_clean":"2024-04-26T21:12:57.017612+0000","last_became_active":"2024-04-26T21:12:55.846587+0000","last_became_peered":"2024-04-26T21:12:55.846587+0000","last_unstale":"2024-04-26T21:12:57.017612+0000","last_undegraded":"2024-04-26T21:12:57.017612+0000","last_fullsized":"2024-04-26T21:12:57.017612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839967+0000","last_change":"2024-04-26T21:12:54.833945+0000","last_active":"2024-04-26T21:12:54.839967+0000","last_peered":"2024-04-26T21:12:54.839967+0000","last_clean":"2024-04-26T21:12:54.839967+0000","last_became_active":"2024-04-26T21:12:54.833186+0000","last_became_peered":"2024-04-26T21:12:54.833186+0000","last_unstale":"2024-04-26T21:12:54.839967+0000","last_undegraded":"2024-04-26T21:12:54.839967+0000","last_fullsized":"2024-04-26T21:12:54.839967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:47.875355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835471+0000","last_change":"2024-04-26T21:12:55.831340+0000","last_active":"2024-04-26T21:12:56.835471+0000","last_peered":"2024-04-26T21:12:56.835471+0000","last_clean":"2024-04-26T21:12:56.835471+0000","last_became_active":"2024-04-26T21:12:55.831068+0000","last_became_peered":"2024-04-26T21:12:55.831068+0000","last_unstale":"2024-04-26T21:12:56.835471+0000","last_undegraded":"2024-04-26T21:12:56.835471+0000","last_fullsized":"2024-04-26T21:12:56.835471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018242+0000","last_change":"2024-04-26T21:12:54.825357+0000","last_active":"2024-04-26T21:12:57.018242+0000","last_peered":"2024-04-26T21:12:57.018242+0000","last_clean":"2024-04-26T21:12:57.018242+0000","last_became_active":"2024-04-26T21:12:54.824948+0000","last_became_peered":"2024-04-26T21:12:54.824948+0000","last_unstale":"2024-04-26T21:12:57.018242+0000","last_undegraded":"2024-04-26T21:12:57.018242+0000","last_fullsized":"2024-04-26T21:12:57.018242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:56.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017961+0000","last_change":"2024-04-26T21:12:55.832196+0000","last_active":"2024-04-26T21:12:57.017961+0000","last_peered":"2024-04-26T21:12:57.017961+0000","last_clean":"2024-04-26T21:12:57.017961+0000","last_became_active":"2024-04-26T21:12:55.831921+0000","last_became_peered":"2024-04-26T21:12:55.831921+0000","last_unstale":"2024-04-26T21:12:57.017961+0000","last_undegraded":"2024-04-26T21:12:57.017961+0000","last_fullsized":"2024-04-26T21:12:57.017961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839796+0000","last_change":"2024-04-26T21:12:54.833203+0000","last_active":"2024-04-26T21:12:54.839796+0000","last_peered":"2024-04-26T21:12:54.839796+0000","last_clean":"2024-04-26T21:12:54.839796+0000","last_became_active":"2024-04-26T21:12:54.832884+0000","last_became_peered":"2024-04-26T21:12:54.832884+0000","last_unstale":"2024-04-26T21:12:54.839796+0000","last_undegraded":"2024-04-26T21:12:54.839796+0000","last_fullsized":"2024-04-26T21:12:54.839796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:31.880236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017397+0000","last_change":"2024-04-26T21:12:55.848208+0000","last_active":"2024-04-26T21:12:57.017397+0000","last_peered":"2024-04-26T21:12:57.017397+0000","last_clean":"2024-04-26T21:12:57.017397+0000","last_became_active":"2024-04-26T21:12:55.847911+0000","last_became_peered":"2024-04-26T21:12:55.847911+0000","last_unstale":"2024-04-26T21:12:57.017397+0000","last_undegraded":"2024-04-26T21:12:57.017397+0000","last_fullsized":"2024-04-26T21:12:57.017397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018179+0000","last_change":"2024-04-26T21:12:54.829043+0000","last_active":"2024-04-26T21:12:57.018179+0000","last_peered":"2024-04-26T21:12:57.018179+0000","last_clean":"2024-04-26T21:12:57.018179+0000","last_became_active":"2024-04-26T21:12:54.828752+0000","last_became_peered":"2024-04-26T21:12:54.828752+0000","last_unstale":"2024-04-26T21:12:57.018179+0000","last_undegraded":"2024-04-26T21:12:57.018179+0000","last_fullsized":"2024-04-26T21:12:57.018179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017744+0000","last_change":"2024-04-26T21:12:55.846026+0000","last_active":"2024-04-26T21:12:57.017744+0000","last_peered":"2024-04-26T21:12:57.017744+0000","last_clean":"2024-04-26T21:12:57.017744+0000","last_became_active":"2024-04-26T21:12:55.845739+0000","last_became_peered":"2024-04-26T21:12:55.845739+0000","last_unstale":"2024-04-26T21:12:57.017744+0000","last_undegraded":"2024-04-26T21:12:57.017744+0000","last_fullsized":"2024-04-26T21:12:57.017744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017377+0000","last_change":"2024-04-26T21:12:54.842196+0000","last_active":"2024-04-26T21:12:57.017377+0000","last_peered":"2024-04-26T21:12:57.017377+0000","last_clean":"2024-04-26T21:12:57.017377+0000","last_became_active":"2024-04-26T21:12:54.841890+0000","last_became_peered":"2024-04-26T21:12:54.841890+0000","last_unstale":"2024-04-26T21:12:57.017377+0000","last_undegraded":"2024-04-26T21:12:57.017377+0000","last_fullsized":"2024-04-26T21:12:57.017377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:46.928170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018560+0000","last_change":"2024-04-26T21:12:55.829002+0000","last_active":"2024-04-26T21:12:57.018560+0000","last_peered":"2024-04-26T21:12:57.018560+0000","last_clean":"2024-04-26T21:12:57.018560+0000","last_became_active":"2024-04-26T21:12:55.828722+0000","last_became_peered":"2024-04-26T21:12:55.828722+0000","last_unstale":"2024-04-26T21:12:57.018560+0000","last_undegraded":"2024-04-26T21:12:57.018560+0000","last_fullsized":"2024-04-26T21:12:57.018560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.834952+0000","last_change":"2024-04-26T21:12:54.833670+0000","last_active":"2024-04-26T21:12:56.834952+0000","last_peered":"2024-04-26T21:12:56.834952+0000","last_clean":"2024-04-26T21:12:56.834952+0000","last_became_active":"2024-04-26T21:12:54.833382+0000","last_became_peered":"2024-04-26T21:12:54.833382+0000","last_unstale":"2024-04-26T21:12:56.834952+0000","last_undegraded":"2024-04-26T21:12:56.834952+0000","last_fullsized":"2024-04-26T21:12:56.834952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018043+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:57.018043+0000","last_peered":"2024-04-26T21:12:57.018043+0000","last_clean":"2024-04-26T21:12:57.018043+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:57.018043+0000","last_undegraded":"2024-04-26T21:12:57.018043+0000","last_fullsized":"2024-04-26T21:12:57.018043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":747537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":157149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:13:00.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:13:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.350+0000 7fa77f959640 1 Processor -- start 2024-04-26T21:13:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 -- start start 2024-04-26T21:13:00.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa77815f340 0x7fa7780b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa77815f720 0x7fa7780d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7780b2510 con 0x7fa7780d4f00 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa7780b27f0 con 0x7fa77815f720 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.351+0000 7fa77f959640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780b2ad0 con 0x7fa77815f340 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77def6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7780d4f00 0x7fa7780d5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42800/0 (socket says 172.21.15.40:42800) 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77def6640 1 -- 172.21.15.40:0/2963255573 learned_addr learned my addr 172.21.15.40:0/2963255573 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa77815f340 0x7fa7780b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77cef4640 1 --2- 172.21.15.40:0/2963255573 >> v2:172.21.15.184:3300/0 conn(0x7fa77815f720 0x7fa7780d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733866522 0 0) 0x7fa7780b2510 con 0x7fa7780d4f00 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa760003520 con 0x7fa7780d4f00 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2542533902 0 0) 0x7fa760003520 con 0x7fa7780d4f00 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa77815f340 msgr2=0x7fa7780b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa77815f340 0x7fa7780b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 >> v2:172.21.15.184:3300/0 conn(0x7fa77815f720 msgr2=0x7fa7780d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 --2- 172.21.15.40:0/2963255573 >> v2:172.21.15.184:3300/0 conn(0x7fa77815f720 0x7fa7780d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780b2db0 con 0x7fa7780d4f00 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa7680019f0 con 0x7fa7780d4f00 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 >> v1:172.21.15.40:6789/0 conn(0x7fa7780d4f00 legacy=0x7fa7780d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7600039b0 0x7fa760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b27f0 con 0x7fa760004800 2024-04-26T21:13:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b2ad0 con 0x7fa7600039b0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa76000bf30 con 0x7fa7600051c0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa77815f340 0x7fa7780b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.352+0000 7fa77cef4640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77def6640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7600039b0 0x7fa760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 -- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7600051c0 msgr2=0x7fa76000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 -- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7600039b0 msgr2=0x7fa760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7600039b0 0x7fa760003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 -- 172.21.15.40:0/2963255573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa760003520 con 0x7fa760004800 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7600039b0 0x7fa760003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.353+0000 7fa77cef4640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa760004800 0x7fa760004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009ac0 tx=0x7fa760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477b18d02748ec0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c036070 con 0x7fa760004800 2024-04-26T21:13:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c031400 con 0x7fa760004800 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa777fff640 1 -- 172.21.15.40:0/2963255573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c009690 con 0x7fa760004800 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 -- 172.21.15.40:0/2963255573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa760004800 msgr2=0x7fa760004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 --2- 172.21.15.40:0/2963255573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa760004800 0x7fa760004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009ac0 tx=0x7fa760005830 comp rx=0 tx=0).stop 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77def6640 1 -- 172.21.15.40:0/2963255573 reap_dead start 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 -- 172.21.15.40:0/2963255573 shutdown_connections 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 -- 172.21.15.40:0/2963255573 >> 172.21.15.40:0/2963255573 conn(0x7fa77808ec70 msgr2=0x7fa77809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 -- 172.21.15.40:0/2963255573 shutdown_connections 2024-04-26T21:13:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.354+0000 7fa77f959640 1 -- 172.21.15.40:0/2963255573 wait complete. 2024-04-26T21:13:00.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.355+0000 7fa77f959640 1 Processor -- start 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 -- start start 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7781555c0 0x7fa7780c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7780ca480 0x7fa7780ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b2290 con 0x7fa7781559a0 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b2110 con 0x7fa7781555c0 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77f959640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780b35b0 con 0x7fa7780ca480 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77def6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77def6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38206/0 (socket says 172.21.15.40:38206) 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.356+0000 7fa77def6640 1 -- 172.21.15.40:0/3588948253 learned_addr learned my addr 172.21.15.40:0/3588948253 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7781555c0 0x7fa7780c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77cef4640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7780ca480 0x7fa7780ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 -- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7780ca480 msgr2=0x7fa7780ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7780ca480 0x7fa7780ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 -- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7781555c0 msgr2=0x7fa7780c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7781555c0 0x7fa7780c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 -- 172.21.15.40:0/3588948253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780ba230 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77def6640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7780c8d50 tx=0x7fa76800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680133f0 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa768019070 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680137b0 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7780ba510 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.357+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7780baad0 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.358+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77806a290 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.359+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa768013950 con 0x7fa7781559a0 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.359+0000 7fa775ffb640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.359+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa76808c4b0 con 0x7fa7781559a0 2024-04-26T21:13:00.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.359+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.360+0000 7fa77d6f5640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7780d6000 tx=0x7fa770008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.363+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa76805a8e0 con 0x7fa7781559a0 2024-04-26T21:13:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.544+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7780d3020 con 0x7fa76406c8f0 2024-04-26T21:13:00.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.550+0000 7fa775ffb640 1 -- 172.21.15.40:0/3588948253 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251287 (secure 0 0 0) 0x7fa7780d3020 con 0x7fa76406c8f0 2024-04-26T21:13:00.554 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:00.555 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:13:00.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 msgr2=0x7fa76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7780d6000 tx=0x7fa770008040 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 msgr2=0x7fa7780cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7780c8d50 tx=0x7fa76800a750 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 shutdown_connections 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7781559a0 0x7fa7780cec50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7780ca480 0x7fa7780ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 --2- 172.21.15.40:0/3588948253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7781555c0 0x7fa7780c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 >> 172.21.15.40:0/3588948253 conn(0x7fa77808ec70 msgr2=0x7fa7780b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 shutdown_connections 2024-04-26T21:13:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.553+0000 7fa77f959640 1 -- 172.21.15.40:0/3588948253 wait complete. 2024-04-26T21:13:00.565 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T21:12:58.931964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220496,"kb_used_data":4308,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4411392,"data_stored":2421212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121725"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835219+0000","last_change":"2024-04-26T21:12:55.834874+0000","last_active":"2024-04-26T21:12:56.835219+0000","last_peered":"2024-04-26T21:12:56.835219+0000","last_clean":"2024-04-26T21:12:56.835219+0000","last_became_active":"2024-04-26T21:12:55.834593+0000","last_became_peered":"2024-04-26T21:12:55.834593+0000","last_unstale":"2024-04-26T21:12:56.835219+0000","last_undegraded":"2024-04-26T21:12:56.835219+0000","last_fullsized":"2024-04-26T21:12:56.835219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:30.540994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018250+0000","last_change":"2024-04-26T21:12:55.843688+0000","last_active":"2024-04-26T21:12:57.018250+0000","last_peered":"2024-04-26T21:12:57.018250+0000","last_clean":"2024-04-26T21:12:57.018250+0000","last_became_active":"2024-04-26T21:12:55.843298+0000","last_became_peered":"2024-04-26T21:12:55.843298+0000","last_unstale":"2024-04-26T21:12:57.018250+0000","last_undegraded":"2024-04-26T21:12:57.018250+0000","last_fullsized":"2024-04-26T21:12:57.018250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835869+0000","last_change":"2024-04-26T21:12:55.847059+0000","last_active":"2024-04-26T21:12:56.835869+0000","last_peered":"2024-04-26T21:12:56.835869+0000","last_clean":"2024-04-26T21:12:56.835869+0000","last_became_active":"2024-04-26T21:12:55.846609+0000","last_became_peered":"2024-04-26T21:12:55.846609+0000","last_unstale":"2024-04-26T21:12:56.835869+0000","last_undegraded":"2024-04-26T21:12:56.835869+0000","last_fullsized":"2024-04-26T21:12:56.835869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.826216+0000","last_change":"2024-04-26T21:12:54.826216+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.826216+0000","last_undegraded":"2024-04-26T21:12:54.826216+0000","last_fullsized":"2024-04-26T21:12:54.826216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829415+0000","last_change":"2024-04-26T21:12:55.845666+0000","last_active":"2024-04-26T21:12:56.829415+0000","last_peered":"2024-04-26T21:12:56.829415+0000","last_clean":"2024-04-26T21:12:56.829415+0000","last_became_active":"2024-04-26T21:12:55.845327+0000","last_became_peered":"2024-04-26T21:12:55.845327+0000","last_unstale":"2024-04-26T21:12:56.829415+0000","last_undegraded":"2024-04-26T21:12:56.829415+0000","last_fullsized":"2024-04-26T21:12:56.829415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:20.848985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018048+0000","last_change":"2024-04-26T21:12:55.843903+0000","last_active":"2024-04-26T21:12:57.018048+0000","last_peered":"2024-04-26T21:12:57.018048+0000","last_clean":"2024-04-26T21:12:57.018048+0000","last_became_active":"2024-04-26T21:12:55.842999+0000","last_became_peered":"2024-04-26T21:12:55.842999+0000","last_unstale":"2024-04-26T21:12:57.018048+0000","last_undegraded":"2024-04-26T21:12:57.018048+0000","last_fullsized":"2024-04-26T21:12:57.018048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018110+0000","last_change":"2024-04-26T21:12:55.829330+0000","last_active":"2024-04-26T21:12:57.018110+0000","last_peered":"2024-04-26T21:12:57.018110+0000","last_clean":"2024-04-26T21:12:57.018110+0000","last_became_active":"2024-04-26T21:12:55.828931+0000","last_became_peered":"2024-04-26T21:12:55.828931+0000","last_unstale":"2024-04-26T21:12:57.018110+0000","last_undegraded":"2024-04-26T21:12:57.018110+0000","last_fullsized":"2024-04-26T21:12:57.018110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.829652+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.829377+0000","last_became_peered":"2024-04-26T21:12:55.829377+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:29.407920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018385+0000","last_change":"2024-04-26T21:12:55.839415+0000","last_active":"2024-04-26T21:12:57.018385+0000","last_peered":"2024-04-26T21:12:57.018385+0000","last_clean":"2024-04-26T21:12:57.018385+0000","last_became_active":"2024-04-26T21:12:55.839122+0000","last_became_peered":"2024-04-26T21:12:55.839122+0000","last_unstale":"2024-04-26T21:12:57.018385+0000","last_undegraded":"2024-04-26T21:12:57.018385+0000","last_fullsized":"2024-04-26T21:12:57.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:28.305031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825703+0000","last_change":"2024-04-26T21:12:54.825703+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825703+0000","last_undegraded":"2024-04-26T21:12:54.825703+0000","last_fullsized":"2024-04-26T21:12:54.825703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018636+0000","last_change":"2024-04-26T21:12:55.845593+0000","last_active":"2024-04-26T21:12:57.018636+0000","last_peered":"2024-04-26T21:12:57.018636+0000","last_clean":"2024-04-26T21:12:57.018636+0000","last_became_active":"2024-04-26T21:12:55.845268+0000","last_became_peered":"2024-04-26T21:12:55.845268+0000","last_unstale":"2024-04-26T21:12:57.018636+0000","last_undegraded":"2024-04-26T21:12:57.018636+0000","last_fullsized":"2024-04-26T21:12:57.018636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835098+0000","last_change":"2024-04-26T21:12:55.845017+0000","last_active":"2024-04-26T21:12:56.835098+0000","last_peered":"2024-04-26T21:12:56.835098+0000","last_clean":"2024-04-26T21:12:56.835098+0000","last_became_active":"2024-04-26T21:12:55.844675+0000","last_became_peered":"2024-04-26T21:12:55.844675+0000","last_unstale":"2024-04-26T21:12:56.835098+0000","last_undegraded":"2024-04-26T21:12:56.835098+0000","last_fullsized":"2024-04-26T21:12:56.835098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829449+0000","last_change":"2024-04-26T21:12:55.848662+0000","last_active":"2024-04-26T21:12:56.829449+0000","last_peered":"2024-04-26T21:12:56.829449+0000","last_clean":"2024-04-26T21:12:56.829449+0000","last_became_active":"2024-04-26T21:12:55.848361+0000","last_became_peered":"2024-04-26T21:12:55.848361+0000","last_unstale":"2024-04-26T21:12:56.829449+0000","last_undegraded":"2024-04-26T21:12:56.829449+0000","last_fullsized":"2024-04-26T21:12:56.829449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:27.246380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018174+0000","last_change":"2024-04-26T21:12:55.851294+0000","last_active":"2024-04-26T21:12:57.018174+0000","last_peered":"2024-04-26T21:12:57.018174+0000","last_clean":"2024-04-26T21:12:57.018174+0000","last_became_active":"2024-04-26T21:12:55.850964+0000","last_became_peered":"2024-04-26T21:12:55.850964+0000","last_unstale":"2024-04-26T21:12:57.018174+0000","last_undegraded":"2024-04-26T21:12:57.018174+0000","last_fullsized":"2024-04-26T21:12:57.018174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835931+0000","last_change":"2024-04-26T21:12:55.827567+0000","last_active":"2024-04-26T21:12:56.835931+0000","last_peered":"2024-04-26T21:12:56.835931+0000","last_clean":"2024-04-26T21:12:56.835931+0000","last_became_active":"2024-04-26T21:12:55.827292+0000","last_became_peered":"2024-04-26T21:12:55.827292+0000","last_unstale":"2024-04-26T21:12:56.835931+0000","last_undegraded":"2024-04-26T21:12:56.835931+0000","last_fullsized":"2024-04-26T21:12:56.835931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017985+0000","last_change":"2024-04-26T21:12:55.848969+0000","last_active":"2024-04-26T21:12:57.017985+0000","last_peered":"2024-04-26T21:12:57.017985+0000","last_clean":"2024-04-26T21:12:57.017985+0000","last_became_active":"2024-04-26T21:12:55.848716+0000","last_became_peered":"2024-04-26T21:12:55.848716+0000","last_unstale":"2024-04-26T21:12:57.017985+0000","last_undegraded":"2024-04-26T21:12:57.017985+0000","last_fullsized":"2024-04-26T21:12:57.017985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018861+0000","last_change":"2024-04-26T21:12:55.834371+0000","last_active":"2024-04-26T21:12:57.018861+0000","last_peered":"2024-04-26T21:12:57.018861+0000","last_clean":"2024-04-26T21:12:57.018861+0000","last_became_active":"2024-04-26T21:12:55.834069+0000","last_became_peered":"2024-04-26T21:12:55.834069+0000","last_unstale":"2024-04-26T21:12:57.018861+0000","last_undegraded":"2024-04-26T21:12:57.018861+0000","last_fullsized":"2024-04-26T21:12:57.018861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018317+0000","last_change":"2024-04-26T21:12:55.852224+0000","last_active":"2024-04-26T21:12:57.018317+0000","last_peered":"2024-04-26T21:12:57.018317+0000","last_clean":"2024-04-26T21:12:57.018317+0000","last_became_active":"2024-04-26T21:12:55.851851+0000","last_became_peered":"2024-04-26T21:12:55.851851+0000","last_unstale":"2024-04-26T21:12:57.018317+0000","last_undegraded":"2024-04-26T21:12:57.018317+0000","last_fullsized":"2024-04-26T21:12:57.018317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:18.378959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017681+0000","last_change":"2024-04-26T21:12:55.852436+0000","last_active":"2024-04-26T21:12:57.017681+0000","last_peered":"2024-04-26T21:12:57.017681+0000","last_clean":"2024-04-26T21:12:57.017681+0000","last_became_active":"2024-04-26T21:12:55.852148+0000","last_became_peered":"2024-04-26T21:12:55.852148+0000","last_unstale":"2024-04-26T21:12:57.017681+0000","last_undegraded":"2024-04-26T21:12:57.017681+0000","last_fullsized":"2024-04-26T21:12:57.017681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829635+0000","last_change":"2024-04-26T21:12:55.833830+0000","last_active":"2024-04-26T21:12:56.829635+0000","last_peered":"2024-04-26T21:12:56.829635+0000","last_clean":"2024-04-26T21:12:56.829635+0000","last_became_active":"2024-04-26T21:12:55.833323+0000","last_became_peered":"2024-04-26T21:12:55.833323+0000","last_unstale":"2024-04-26T21:12:56.829635+0000","last_undegraded":"2024-04-26T21:12:56.829635+0000","last_fullsized":"2024-04-26T21:12:56.829635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:18.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018226+0000","last_change":"2024-04-26T21:12:55.831331+0000","last_active":"2024-04-26T21:12:57.018226+0000","last_peered":"2024-04-26T21:12:57.018226+0000","last_clean":"2024-04-26T21:12:57.018226+0000","last_became_active":"2024-04-26T21:12:55.830490+0000","last_became_peered":"2024-04-26T21:12:55.830490+0000","last_unstale":"2024-04-26T21:12:57.018226+0000","last_undegraded":"2024-04-26T21:12:57.018226+0000","last_fullsized":"2024-04-26T21:12:57.018226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:40.323141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018258+0000","last_change":"2024-04-26T21:12:55.848889+0000","last_active":"2024-04-26T21:12:57.018258+0000","last_peered":"2024-04-26T21:12:57.018258+0000","last_clean":"2024-04-26T21:12:57.018258+0000","last_became_active":"2024-04-26T21:12:55.848607+0000","last_became_peered":"2024-04-26T21:12:55.848607+0000","last_unstale":"2024-04-26T21:12:57.018258+0000","last_undegraded":"2024-04-26T21:12:57.018258+0000","last_fullsized":"2024-04-26T21:12:57.018258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017553+0000","last_change":"2024-04-26T21:12:55.834830+0000","last_active":"2024-04-26T21:12:57.017553+0000","last_peered":"2024-04-26T21:12:57.017553+0000","last_clean":"2024-04-26T21:12:57.017553+0000","last_became_active":"2024-04-26T21:12:55.834541+0000","last_became_peered":"2024-04-26T21:12:55.834541+0000","last_unstale":"2024-04-26T21:12:57.017553+0000","last_undegraded":"2024-04-26T21:12:57.017553+0000","last_fullsized":"2024-04-26T21:12:57.017553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835083+0000","last_change":"2024-04-26T21:12:55.829794+0000","last_active":"2024-04-26T21:12:56.835083+0000","last_peered":"2024-04-26T21:12:56.835083+0000","last_clean":"2024-04-26T21:12:56.835083+0000","last_became_active":"2024-04-26T21:12:55.829261+0000","last_became_peered":"2024-04-26T21:12:55.829261+0000","last_unstale":"2024-04-26T21:12:56.835083+0000","last_undegraded":"2024-04-26T21:12:56.835083+0000","last_fullsized":"2024-04-26T21:12:56.835083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824632+0000","last_change":"2024-04-26T21:12:54.824632+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824632+0000","last_undegraded":"2024-04-26T21:12:54.824632+0000","last_fullsized":"2024-04-26T21:12:54.824632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018383+0000","last_change":"2024-04-26T21:12:55.847342+0000","last_active":"2024-04-26T21:12:57.018383+0000","last_peered":"2024-04-26T21:12:57.018383+0000","last_clean":"2024-04-26T21:12:57.018383+0000","last_became_active":"2024-04-26T21:12:55.846476+0000","last_became_peered":"2024-04-26T21:12:55.846476+0000","last_unstale":"2024-04-26T21:12:57.018383+0000","last_undegraded":"2024-04-26T21:12:57.018383+0000","last_fullsized":"2024-04-26T21:12:57.018383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:29.560745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825269+0000","last_change":"2024-04-26T21:12:54.825269+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825269+0000","last_undegraded":"2024-04-26T21:12:54.825269+0000","last_fullsized":"2024-04-26T21:12:54.825269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835252+0000","last_change":"2024-04-26T21:12:55.833992+0000","last_active":"2024-04-26T21:12:56.835252+0000","last_peered":"2024-04-26T21:12:56.835252+0000","last_clean":"2024-04-26T21:12:56.835252+0000","last_became_active":"2024-04-26T21:12:55.833496+0000","last_became_peered":"2024-04-26T21:12:55.833496+0000","last_unstale":"2024-04-26T21:12:56.835252+0000","last_undegraded":"2024-04-26T21:12:56.835252+0000","last_fullsized":"2024-04-26T21:12:56.835252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018305+0000","last_change":"2024-04-26T21:12:55.846206+0000","last_active":"2024-04-26T21:12:57.018305+0000","last_peered":"2024-04-26T21:12:57.018305+0000","last_clean":"2024-04-26T21:12:57.018305+0000","last_became_active":"2024-04-26T21:12:55.845924+0000","last_became_peered":"2024-04-26T21:12:55.845924+0000","last_unstale":"2024-04-26T21:12:57.018305+0000","last_undegraded":"2024-04-26T21:12:57.018305+0000","last_fullsized":"2024-04-26T21:12:57.018305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:12.137923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018052+0000","last_change":"2024-04-26T21:12:55.833028+0000","last_active":"2024-04-26T21:12:57.018052+0000","last_peered":"2024-04-26T21:12:57.018052+0000","last_clean":"2024-04-26T21:12:57.018052+0000","last_became_active":"2024-04-26T21:12:55.832783+0000","last_became_peered":"2024-04-26T21:12:55.832783+0000","last_unstale":"2024-04-26T21:12:57.018052+0000","last_undegraded":"2024-04-26T21:12:57.018052+0000","last_fullsized":"2024-04-26T21:12:57.018052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835242+0000","last_change":"2024-04-26T21:12:55.829964+0000","last_active":"2024-04-26T21:12:56.835242+0000","last_peered":"2024-04-26T21:12:56.835242+0000","last_clean":"2024-04-26T21:12:56.835242+0000","last_became_active":"2024-04-26T21:12:55.828615+0000","last_became_peered":"2024-04-26T21:12:55.828615+0000","last_unstale":"2024-04-26T21:12:56.835242+0000","last_undegraded":"2024-04-26T21:12:56.835242+0000","last_fullsized":"2024-04-26T21:12:56.835242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:33.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017806+0000","last_change":"2024-04-26T21:12:55.835339+0000","last_active":"2024-04-26T21:12:57.017806+0000","last_peered":"2024-04-26T21:12:57.017806+0000","last_clean":"2024-04-26T21:12:57.017806+0000","last_became_active":"2024-04-26T21:12:55.834346+0000","last_became_peered":"2024-04-26T21:12:55.834346+0000","last_unstale":"2024-04-26T21:12:57.017806+0000","last_undegraded":"2024-04-26T21:12:57.017806+0000","last_fullsized":"2024-04-26T21:12:57.017806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017889+0000","last_change":"2024-04-26T21:12:55.835570+0000","last_active":"2024-04-26T21:12:57.017889+0000","last_peered":"2024-04-26T21:12:57.017889+0000","last_clean":"2024-04-26T21:12:57.017889+0000","last_became_active":"2024-04-26T21:12:55.835290+0000","last_became_peered":"2024-04-26T21:12:55.835290+0000","last_unstale":"2024-04-26T21:12:57.017889+0000","last_undegraded":"2024-04-26T21:12:57.017889+0000","last_fullsized":"2024-04-26T21:12:57.017889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835504+0000","last_change":"2024-04-26T21:12:54.836243+0000","last_active":"2024-04-26T21:12:56.835504+0000","last_peered":"2024-04-26T21:12:56.835504+0000","last_clean":"2024-04-26T21:12:56.835504+0000","last_became_active":"2024-04-26T21:12:54.835926+0000","last_became_peered":"2024-04-26T21:12:54.835926+0000","last_unstale":"2024-04-26T21:12:56.835504+0000","last_undegraded":"2024-04-26T21:12:56.835504+0000","last_fullsized":"2024-04-26T21:12:56.835504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835388+0000","last_change":"2024-04-26T21:12:55.833085+0000","last_active":"2024-04-26T21:12:56.835388+0000","last_peered":"2024-04-26T21:12:56.835388+0000","last_clean":"2024-04-26T21:12:56.835388+0000","last_became_active":"2024-04-26T21:12:55.832730+0000","last_became_peered":"2024-04-26T21:12:55.832730+0000","last_unstale":"2024-04-26T21:12:56.835388+0000","last_undegraded":"2024-04-26T21:12:56.835388+0000","last_fullsized":"2024-04-26T21:12:56.835388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018001+0000","last_change":"2024-04-26T21:12:54.828997+0000","last_active":"2024-04-26T21:12:57.018001+0000","last_peered":"2024-04-26T21:12:57.018001+0000","last_clean":"2024-04-26T21:12:57.018001+0000","last_became_active":"2024-04-26T21:12:54.828635+0000","last_became_peered":"2024-04-26T21:12:54.828635+0000","last_unstale":"2024-04-26T21:12:57.018001+0000","last_undegraded":"2024-04-26T21:12:57.018001+0000","last_fullsized":"2024-04-26T21:12:57.018001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018665+0000","last_change":"2024-04-26T21:12:55.846106+0000","last_active":"2024-04-26T21:12:57.018665+0000","last_peered":"2024-04-26T21:12:57.018665+0000","last_clean":"2024-04-26T21:12:57.018665+0000","last_became_active":"2024-04-26T21:12:55.845816+0000","last_became_peered":"2024-04-26T21:12:55.845816+0000","last_unstale":"2024-04-26T21:12:57.018665+0000","last_undegraded":"2024-04-26T21:12:57.018665+0000","last_fullsized":"2024-04-26T21:12:57.018665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:44.371719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017598+0000","last_change":"2024-04-26T21:12:54.841177+0000","last_active":"2024-04-26T21:12:57.017598+0000","last_peered":"2024-04-26T21:12:57.017598+0000","last_clean":"2024-04-26T21:12:57.017598+0000","last_became_active":"2024-04-26T21:12:54.840803+0000","last_became_peered":"2024-04-26T21:12:54.840803+0000","last_unstale":"2024-04-26T21:12:57.017598+0000","last_undegraded":"2024-04-26T21:12:57.017598+0000","last_fullsized":"2024-04-26T21:12:57.017598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.922188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824483+0000","last_change":"2024-04-26T21:12:54.824483+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824483+0000","last_undegraded":"2024-04-26T21:12:54.824483+0000","last_fullsized":"2024-04-26T21:12:54.824483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829652+0000","last_change":"2024-04-26T21:12:54.833964+0000","last_active":"2024-04-26T21:12:56.829652+0000","last_peered":"2024-04-26T21:12:56.829652+0000","last_clean":"2024-04-26T21:12:56.829652+0000","last_became_active":"2024-04-26T21:12:54.833662+0000","last_became_peered":"2024-04-26T21:12:54.833662+0000","last_unstale":"2024-04-26T21:12:56.829652+0000","last_undegraded":"2024-04-26T21:12:56.829652+0000","last_fullsized":"2024-04-26T21:12:56.829652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:11.073082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018468+0000","last_change":"2024-04-26T21:12:55.839174+0000","last_active":"2024-04-26T21:12:57.018468+0000","last_peered":"2024-04-26T21:12:57.018468+0000","last_clean":"2024-04-26T21:12:57.018468+0000","last_became_active":"2024-04-26T21:12:55.838902+0000","last_became_peered":"2024-04-26T21:12:55.838902+0000","last_unstale":"2024-04-26T21:12:57.018468+0000","last_undegraded":"2024-04-26T21:12:57.018468+0000","last_fullsized":"2024-04-26T21:12:57.018468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018348+0000","last_change":"2024-04-26T21:12:54.838674+0000","last_active":"2024-04-26T21:12:57.018348+0000","last_peered":"2024-04-26T21:12:57.018348+0000","last_clean":"2024-04-26T21:12:57.018348+0000","last_became_active":"2024-04-26T21:12:54.838357+0000","last_became_peered":"2024-04-26T21:12:54.838357+0000","last_unstale":"2024-04-26T21:12:57.018348+0000","last_undegraded":"2024-04-26T21:12:57.018348+0000","last_fullsized":"2024-04-26T21:12:57.018348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835420+0000","last_change":"2024-04-26T21:12:55.833557+0000","last_active":"2024-04-26T21:12:56.835420+0000","last_peered":"2024-04-26T21:12:56.835420+0000","last_clean":"2024-04-26T21:12:56.835420+0000","last_became_active":"2024-04-26T21:12:55.833295+0000","last_became_peered":"2024-04-26T21:12:55.833295+0000","last_unstale":"2024-04-26T21:12:56.835420+0000","last_undegraded":"2024-04-26T21:12:56.835420+0000","last_fullsized":"2024-04-26T21:12:56.835420+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018262+0000","last_change":"2024-04-26T21:12:54.837240+0000","last_active":"2024-04-26T21:12:57.018262+0000","last_peered":"2024-04-26T21:12:57.018262+0000","last_clean":"2024-04-26T21:12:57.018262+0000","last_became_active":"2024-04-26T21:12:54.837022+0000","last_became_peered":"2024-04-26T21:12:54.837022+0000","last_unstale":"2024-04-26T21:12:57.018262+0000","last_undegraded":"2024-04-26T21:12:57.018262+0000","last_fullsized":"2024-04-26T21:12:57.018262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:20.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017917+0000","last_change":"2024-04-26T21:12:55.828118+0000","last_active":"2024-04-26T21:12:57.017917+0000","last_peered":"2024-04-26T21:12:57.017917+0000","last_clean":"2024-04-26T21:12:57.017917+0000","last_became_active":"2024-04-26T21:12:55.827805+0000","last_became_peered":"2024-04-26T21:12:55.827805+0000","last_unstale":"2024-04-26T21:12:57.017917+0000","last_undegraded":"2024-04-26T21:12:57.017917+0000","last_fullsized":"2024-04-26T21:12:57.017917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017693+0000","last_change":"2024-04-26T21:12:54.835535+0000","last_active":"2024-04-26T21:12:57.017693+0000","last_peered":"2024-04-26T21:12:57.017693+0000","last_clean":"2024-04-26T21:12:57.017693+0000","last_became_active":"2024-04-26T21:12:54.835258+0000","last_became_peered":"2024-04-26T21:12:54.835258+0000","last_unstale":"2024-04-26T21:12:57.017693+0000","last_undegraded":"2024-04-26T21:12:57.017693+0000","last_fullsized":"2024-04-26T21:12:57.017693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.823608+0000","last_change":"2024-04-26T21:12:54.823608+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.823608+0000","last_undegraded":"2024-04-26T21:12:54.823608+0000","last_fullsized":"2024-04-26T21:12:54.823608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017760+0000","last_change":"2024-04-26T21:12:54.833665+0000","last_active":"2024-04-26T21:12:57.017760+0000","last_peered":"2024-04-26T21:12:57.017760+0000","last_clean":"2024-04-26T21:12:57.017760+0000","last_became_active":"2024-04-26T21:12:54.833394+0000","last_became_peered":"2024-04-26T21:12:54.833394+0000","last_unstale":"2024-04-26T21:12:57.017760+0000","last_undegraded":"2024-04-26T21:12:57.017760+0000","last_fullsized":"2024-04-26T21:12:57.017760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:18.903622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.836233+0000","last_change":"2024-04-26T21:12:55.844791+0000","last_active":"2024-04-26T21:12:56.836233+0000","last_peered":"2024-04-26T21:12:56.836233+0000","last_clean":"2024-04-26T21:12:56.836233+0000","last_became_active":"2024-04-26T21:12:55.844567+0000","last_became_peered":"2024-04-26T21:12:55.844567+0000","last_unstale":"2024-04-26T21:12:56.836233+0000","last_undegraded":"2024-04-26T21:12:56.836233+0000","last_fullsized":"2024-04-26T21:12:56.836233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017815+0000","last_change":"2024-04-26T21:12:54.842287+0000","last_active":"2024-04-26T21:12:57.017815+0000","last_peered":"2024-04-26T21:12:57.017815+0000","last_clean":"2024-04-26T21:12:57.017815+0000","last_became_active":"2024-04-26T21:12:54.841969+0000","last_became_peered":"2024-04-26T21:12:54.841969+0000","last_unstale":"2024-04-26T21:12:57.017815+0000","last_undegraded":"2024-04-26T21:12:57.017815+0000","last_fullsized":"2024-04-26T21:12:57.017815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.740507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017709+0000","last_change":"2024-04-26T21:12:55.852140+0000","last_active":"2024-04-26T21:12:57.017709+0000","last_peered":"2024-04-26T21:12:57.017709+0000","last_clean":"2024-04-26T21:12:57.017709+0000","last_became_active":"2024-04-26T21:12:55.851753+0000","last_became_peered":"2024-04-26T21:12:55.851753+0000","last_unstale":"2024-04-26T21:12:57.017709+0000","last_undegraded":"2024-04-26T21:12:57.017709+0000","last_fullsized":"2024-04-26T21:12:57.017709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:54.832871+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:54.832587+0000","last_became_peered":"2024-04-26T21:12:54.832587+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017990+0000","last_change":"2024-04-26T21:12:55.851660+0000","last_active":"2024-04-26T21:12:57.017990+0000","last_peered":"2024-04-26T21:12:57.017990+0000","last_clean":"2024-04-26T21:12:57.017990+0000","last_became_active":"2024-04-26T21:12:55.850979+0000","last_became_peered":"2024-04-26T21:12:55.850979+0000","last_unstale":"2024-04-26T21:12:57.017990+0000","last_undegraded":"2024-04-26T21:12:57.017990+0000","last_fullsized":"2024-04-26T21:12:57.017990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835911+0000","last_change":"2024-04-26T21:12:54.830694+0000","last_active":"2024-04-26T21:12:56.835911+0000","last_peered":"2024-04-26T21:12:56.835911+0000","last_clean":"2024-04-26T21:12:56.835911+0000","last_became_active":"2024-04-26T21:12:54.830373+0000","last_became_peered":"2024-04-26T21:12:54.830373+0000","last_unstale":"2024-04-26T21:12:56.835911+0000","last_undegraded":"2024-04-26T21:12:56.835911+0000","last_fullsized":"2024-04-26T21:12:56.835911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835683+0000","last_change":"2024-04-26T21:12:55.846932+0000","last_active":"2024-04-26T21:12:56.835683+0000","last_peered":"2024-04-26T21:12:56.835683+0000","last_clean":"2024-04-26T21:12:56.835683+0000","last_became_active":"2024-04-26T21:12:55.846616+0000","last_became_peered":"2024-04-26T21:12:55.846616+0000","last_unstale":"2024-04-26T21:12:56.835683+0000","last_undegraded":"2024-04-26T21:12:56.835683+0000","last_fullsized":"2024-04-26T21:12:56.835683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017971+0000","last_change":"2024-04-26T21:12:54.840841+0000","last_active":"2024-04-26T21:12:57.017971+0000","last_peered":"2024-04-26T21:12:57.017971+0000","last_clean":"2024-04-26T21:12:57.017971+0000","last_became_active":"2024-04-26T21:12:54.840336+0000","last_became_peered":"2024-04-26T21:12:54.840336+0000","last_unstale":"2024-04-26T21:12:57.017971+0000","last_undegraded":"2024-04-26T21:12:57.017971+0000","last_fullsized":"2024-04-26T21:12:57.017971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.824074+0000","last_change":"2024-04-26T21:12:54.824074+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.824074+0000","last_undegraded":"2024-04-26T21:12:54.824074+0000","last_fullsized":"2024-04-26T21:12:54.824074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018411+0000","last_change":"2024-04-26T21:12:54.833969+0000","last_active":"2024-04-26T21:12:57.018411+0000","last_peered":"2024-04-26T21:12:57.018411+0000","last_clean":"2024-04-26T21:12:57.018411+0000","last_became_active":"2024-04-26T21:12:54.833628+0000","last_became_peered":"2024-04-26T21:12:54.833628+0000","last_unstale":"2024-04-26T21:12:57.018411+0000","last_undegraded":"2024-04-26T21:12:57.018411+0000","last_fullsized":"2024-04-26T21:12:57.018411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.844863+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.844486+0000","last_became_peered":"2024-04-26T21:12:55.844486+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.836902+0000","last_change":"2024-04-26T21:12:54.831096+0000","last_active":"2024-04-26T21:12:54.836902+0000","last_peered":"2024-04-26T21:12:54.836902+0000","last_clean":"2024-04-26T21:12:54.836902+0000","last_became_active":"2024-04-26T21:12:54.830785+0000","last_became_peered":"2024-04-26T21:12:54.830785+0000","last_unstale":"2024-04-26T21:12:54.836902+0000","last_undegraded":"2024-04-26T21:12:54.836902+0000","last_fullsized":"2024-04-26T21:12:54.836902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:19.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018645+0000","last_change":"2024-04-26T21:12:55.834991+0000","last_active":"2024-04-26T21:12:57.018645+0000","last_peered":"2024-04-26T21:12:57.018645+0000","last_clean":"2024-04-26T21:12:57.018645+0000","last_became_active":"2024-04-26T21:12:55.834663+0000","last_became_peered":"2024-04-26T21:12:55.834663+0000","last_unstale":"2024-04-26T21:12:57.018645+0000","last_undegraded":"2024-04-26T21:12:57.018645+0000","last_fullsized":"2024-04-26T21:12:57.018645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017474+0000","last_change":"2024-04-26T21:12:54.842208+0000","last_active":"2024-04-26T21:12:57.017474+0000","last_peered":"2024-04-26T21:12:57.017474+0000","last_clean":"2024-04-26T21:12:57.017474+0000","last_became_active":"2024-04-26T21:12:54.841932+0000","last_became_peered":"2024-04-26T21:12:54.841932+0000","last_unstale":"2024-04-26T21:12:57.017474+0000","last_undegraded":"2024-04-26T21:12:57.017474+0000","last_fullsized":"2024-04-26T21:12:57.017474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:29.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835302+0000","last_change":"2024-04-26T21:12:55.830389+0000","last_active":"2024-04-26T21:12:56.835302+0000","last_peered":"2024-04-26T21:12:56.835302+0000","last_clean":"2024-04-26T21:12:56.835302+0000","last_became_active":"2024-04-26T21:12:55.830080+0000","last_became_peered":"2024-04-26T21:12:55.830080+0000","last_unstale":"2024-04-26T21:12:56.835302+0000","last_undegraded":"2024-04-26T21:12:56.835302+0000","last_fullsized":"2024-04-26T21:12:56.835302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017777+0000","last_change":"2024-04-26T21:12:54.833724+0000","last_active":"2024-04-26T21:12:57.017777+0000","last_peered":"2024-04-26T21:12:57.017777+0000","last_clean":"2024-04-26T21:12:57.017777+0000","last_became_active":"2024-04-26T21:12:54.833458+0000","last_became_peered":"2024-04-26T21:12:54.833458+0000","last_unstale":"2024-04-26T21:12:57.017777+0000","last_undegraded":"2024-04-26T21:12:57.017777+0000","last_fullsized":"2024-04-26T21:12:57.017777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:26.631042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:55.835158+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:55.834796+0000","last_became_peered":"2024-04-26T21:12:55.834796+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840552+0000","last_change":"2024-04-26T21:12:54.837631+0000","last_active":"2024-04-26T21:12:54.840552+0000","last_peered":"2024-04-26T21:12:54.840552+0000","last_clean":"2024-04-26T21:12:54.840552+0000","last_became_active":"2024-04-26T21:12:54.837117+0000","last_became_peered":"2024-04-26T21:12:54.837117+0000","last_unstale":"2024-04-26T21:12:54.840552+0000","last_undegraded":"2024-04-26T21:12:54.840552+0000","last_fullsized":"2024-04-26T21:12:54.840552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:00.880559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835652+0000","last_change":"2024-04-26T21:12:55.837208+0000","last_active":"2024-04-26T21:12:56.835652+0000","last_peered":"2024-04-26T21:12:56.835652+0000","last_clean":"2024-04-26T21:12:56.835652+0000","last_became_active":"2024-04-26T21:12:55.836934+0000","last_became_peered":"2024-04-26T21:12:55.836934+0000","last_unstale":"2024-04-26T21:12:56.835652+0000","last_undegraded":"2024-04-26T21:12:56.835652+0000","last_fullsized":"2024-04-26T21:12:56.835652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018106+0000","last_change":"2024-04-26T21:12:54.833728+0000","last_active":"2024-04-26T21:12:57.018106+0000","last_peered":"2024-04-26T21:12:57.018106+0000","last_clean":"2024-04-26T21:12:57.018106+0000","last_became_active":"2024-04-26T21:12:54.833388+0000","last_became_peered":"2024-04-26T21:12:54.833388+0000","last_unstale":"2024-04-26T21:12:57.018106+0000","last_undegraded":"2024-04-26T21:12:57.018106+0000","last_fullsized":"2024-04-26T21:12:57.018106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-04-26T21:12:54.825701+0000","last_change":"2024-04-26T21:12:54.825701+0000","last_active":"2024-04-26T21:12:54.809395+0000","last_peered":"2024-04-26T21:12:54.809395+0000","last_clean":"2024-04-26T21:12:54.809395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:12:54.825701+0000","last_undegraded":"2024-04-26T21:12:54.825701+0000","last_fullsized":"2024-04-26T21:12:54.825701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018083+0000","last_change":"2024-04-26T21:12:54.832422+0000","last_active":"2024-04-26T21:12:57.018083+0000","last_peered":"2024-04-26T21:12:57.018083+0000","last_clean":"2024-04-26T21:12:57.018083+0000","last_became_active":"2024-04-26T21:12:54.832171+0000","last_became_peered":"2024-04-26T21:12:54.832171+0000","last_unstale":"2024-04-26T21:12:57.018083+0000","last_undegraded":"2024-04-26T21:12:57.018083+0000","last_fullsized":"2024-04-26T21:12:57.018083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:33.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829689+0000","last_change":"2024-04-26T21:12:55.840206+0000","last_active":"2024-04-26T21:12:56.829689+0000","last_peered":"2024-04-26T21:12:56.829689+0000","last_clean":"2024-04-26T21:12:56.829689+0000","last_became_active":"2024-04-26T21:12:55.839915+0000","last_became_peered":"2024-04-26T21:12:55.839915+0000","last_unstale":"2024-04-26T21:12:56.829689+0000","last_undegraded":"2024-04-26T21:12:56.829689+0000","last_fullsized":"2024-04-26T21:12:56.829689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.934501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018739+0000","last_change":"2024-04-26T21:12:54.838655+0000","last_active":"2024-04-26T21:12:57.018739+0000","last_peered":"2024-04-26T21:12:57.018739+0000","last_clean":"2024-04-26T21:12:57.018739+0000","last_became_active":"2024-04-26T21:12:54.838244+0000","last_became_peered":"2024-04-26T21:12:54.838244+0000","last_unstale":"2024-04-26T21:12:57.018739+0000","last_undegraded":"2024-04-26T21:12:57.018739+0000","last_fullsized":"2024-04-26T21:12:57.018739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:26.055217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.834792+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.834492+0000","last_became_peered":"2024-04-26T21:12:55.834492+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:12.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018066+0000","last_change":"2024-04-26T21:12:54.834125+0000","last_active":"2024-04-26T21:12:57.018066+0000","last_peered":"2024-04-26T21:12:57.018066+0000","last_clean":"2024-04-26T21:12:57.018066+0000","last_became_active":"2024-04-26T21:12:54.833830+0000","last_became_peered":"2024-04-26T21:12:54.833830+0000","last_unstale":"2024-04-26T21:12:57.018066+0000","last_undegraded":"2024-04-26T21:12:57.018066+0000","last_fullsized":"2024-04-26T21:12:57.018066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:17.724781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829576+0000","last_change":"2024-04-26T21:12:55.829614+0000","last_active":"2024-04-26T21:12:56.829576+0000","last_peered":"2024-04-26T21:12:56.829576+0000","last_clean":"2024-04-26T21:12:56.829576+0000","last_became_active":"2024-04-26T21:12:55.829102+0000","last_became_peered":"2024-04-26T21:12:55.829102+0000","last_unstale":"2024-04-26T21:12:56.829576+0000","last_undegraded":"2024-04-26T21:12:56.829576+0000","last_fullsized":"2024-04-26T21:12:56.829576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.539573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835517+0000","last_change":"2024-04-26T21:12:54.833584+0000","last_active":"2024-04-26T21:12:56.835517+0000","last_peered":"2024-04-26T21:12:56.835517+0000","last_clean":"2024-04-26T21:12:56.835517+0000","last_became_active":"2024-04-26T21:12:54.833302+0000","last_became_peered":"2024-04-26T21:12:54.833302+0000","last_unstale":"2024-04-26T21:12:56.835517+0000","last_undegraded":"2024-04-26T21:12:56.835517+0000","last_fullsized":"2024-04-26T21:12:56.835517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.829378+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.829065+0000","last_became_peered":"2024-04-26T21:12:55.829065+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:01.592713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017725+0000","last_change":"2024-04-26T21:12:54.834382+0000","last_active":"2024-04-26T21:12:57.017725+0000","last_peered":"2024-04-26T21:12:57.017725+0000","last_clean":"2024-04-26T21:12:57.017725+0000","last_became_active":"2024-04-26T21:12:54.833899+0000","last_became_peered":"2024-04-26T21:12:54.833899+0000","last_unstale":"2024-04-26T21:12:57.017725+0000","last_undegraded":"2024-04-26T21:12:57.017725+0000","last_fullsized":"2024-04-26T21:12:57.017725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:11.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018255+0000","last_change":"2024-04-26T21:12:55.831832+0000","last_active":"2024-04-26T21:12:57.018255+0000","last_peered":"2024-04-26T21:12:57.018255+0000","last_clean":"2024-04-26T21:12:57.018255+0000","last_became_active":"2024-04-26T21:12:55.831550+0000","last_became_peered":"2024-04-26T21:12:55.831550+0000","last_unstale":"2024-04-26T21:12:57.018255+0000","last_undegraded":"2024-04-26T21:12:57.018255+0000","last_fullsized":"2024-04-26T21:12:57.018255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835186+0000","last_change":"2024-04-26T21:12:54.830666+0000","last_active":"2024-04-26T21:12:56.835186+0000","last_peered":"2024-04-26T21:12:56.835186+0000","last_clean":"2024-04-26T21:12:56.835186+0000","last_became_active":"2024-04-26T21:12:54.830338+0000","last_became_peered":"2024-04-26T21:12:54.830338+0000","last_unstale":"2024-04-26T21:12:56.835186+0000","last_undegraded":"2024-04-26T21:12:56.835186+0000","last_fullsized":"2024-04-26T21:12:56.835186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829680+0000","last_change":"2024-04-26T21:12:55.834082+0000","last_active":"2024-04-26T21:12:56.829680+0000","last_peered":"2024-04-26T21:12:56.829680+0000","last_clean":"2024-04-26T21:12:56.829680+0000","last_became_active":"2024-04-26T21:12:55.833778+0000","last_became_peered":"2024-04-26T21:12:55.833778+0000","last_unstale":"2024-04-26T21:12:56.829680+0000","last_undegraded":"2024-04-26T21:12:56.829680+0000","last_fullsized":"2024-04-26T21:12:56.829680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:57.707945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.840467+0000","last_change":"2024-04-26T21:12:54.837054+0000","last_active":"2024-04-26T21:12:54.840467+0000","last_peered":"2024-04-26T21:12:54.840467+0000","last_clean":"2024-04-26T21:12:54.840467+0000","last_became_active":"2024-04-26T21:12:54.836690+0000","last_became_peered":"2024-04-26T21:12:54.836690+0000","last_unstale":"2024-04-26T21:12:54.840467+0000","last_undegraded":"2024-04-26T21:12:54.840467+0000","last_fullsized":"2024-04-26T21:12:54.840467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:53.292509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835819+0000","last_change":"2024-04-26T21:12:55.840516+0000","last_active":"2024-04-26T21:12:56.835819+0000","last_peered":"2024-04-26T21:12:56.835819+0000","last_clean":"2024-04-26T21:12:56.835819+0000","last_became_active":"2024-04-26T21:12:55.840212+0000","last_became_peered":"2024-04-26T21:12:55.840212+0000","last_unstale":"2024-04-26T21:12:56.835819+0000","last_undegraded":"2024-04-26T21:12:56.835819+0000","last_fullsized":"2024-04-26T21:12:56.835819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018109+0000","last_change":"2024-04-26T21:12:54.834650+0000","last_active":"2024-04-26T21:12:57.018109+0000","last_peered":"2024-04-26T21:12:57.018109+0000","last_clean":"2024-04-26T21:12:57.018109+0000","last_became_active":"2024-04-26T21:12:54.834365+0000","last_became_peered":"2024-04-26T21:12:54.834365+0000","last_unstale":"2024-04-26T21:12:57.018109+0000","last_undegraded":"2024-04-26T21:12:57.018109+0000","last_fullsized":"2024-04-26T21:12:57.018109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017612+0000","last_change":"2024-04-26T21:12:55.846844+0000","last_active":"2024-04-26T21:12:57.017612+0000","last_peered":"2024-04-26T21:12:57.017612+0000","last_clean":"2024-04-26T21:12:57.017612+0000","last_became_active":"2024-04-26T21:12:55.846587+0000","last_became_peered":"2024-04-26T21:12:55.846587+0000","last_unstale":"2024-04-26T21:12:57.017612+0000","last_undegraded":"2024-04-26T21:12:57.017612+0000","last_fullsized":"2024-04-26T21:12:57.017612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839967+0000","last_change":"2024-04-26T21:12:54.833945+0000","last_active":"2024-04-26T21:12:54.839967+0000","last_peered":"2024-04-26T21:12:54.839967+0000","last_clean":"2024-04-26T21:12:54.839967+0000","last_became_active":"2024-04-26T21:12:54.833186+0000","last_became_peered":"2024-04-26T21:12:54.833186+0000","last_unstale":"2024-04-26T21:12:54.839967+0000","last_undegraded":"2024-04-26T21:12:54.839967+0000","last_fullsized":"2024-04-26T21:12:54.839967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:47.875355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835471+0000","last_change":"2024-04-26T21:12:55.831340+0000","last_active":"2024-04-26T21:12:56.835471+0000","last_peered":"2024-04-26T21:12:56.835471+0000","last_clean":"2024-04-26T21:12:56.835471+0000","last_became_active":"2024-04-26T21:12:55.831068+0000","last_became_peered":"2024-04-26T21:12:55.831068+0000","last_unstale":"2024-04-26T21:12:56.835471+0000","last_undegraded":"2024-04-26T21:12:56.835471+0000","last_fullsized":"2024-04-26T21:12:56.835471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018242+0000","last_change":"2024-04-26T21:12:54.825357+0000","last_active":"2024-04-26T21:12:57.018242+0000","last_peered":"2024-04-26T21:12:57.018242+0000","last_clean":"2024-04-26T21:12:57.018242+0000","last_became_active":"2024-04-26T21:12:54.824948+0000","last_became_peered":"2024-04-26T21:12:54.824948+0000","last_unstale":"2024-04-26T21:12:57.018242+0000","last_undegraded":"2024-04-26T21:12:57.018242+0000","last_fullsized":"2024-04-26T21:12:57.018242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:56.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017961+0000","last_change":"2024-04-26T21:12:55.832196+0000","last_active":"2024-04-26T21:12:57.017961+0000","last_peered":"2024-04-26T21:12:57.017961+0000","last_clean":"2024-04-26T21:12:57.017961+0000","last_became_active":"2024-04-26T21:12:55.831921+0000","last_became_peered":"2024-04-26T21:12:55.831921+0000","last_unstale":"2024-04-26T21:12:57.017961+0000","last_undegraded":"2024-04-26T21:12:57.017961+0000","last_fullsized":"2024-04-26T21:12:57.017961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:12:54.839796+0000","last_change":"2024-04-26T21:12:54.833203+0000","last_active":"2024-04-26T21:12:54.839796+0000","last_peered":"2024-04-26T21:12:54.839796+0000","last_clean":"2024-04-26T21:12:54.839796+0000","last_became_active":"2024-04-26T21:12:54.832884+0000","last_became_peered":"2024-04-26T21:12:54.832884+0000","last_unstale":"2024-04-26T21:12:54.839796+0000","last_undegraded":"2024-04-26T21:12:54.839796+0000","last_fullsized":"2024-04-26T21:12:54.839796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:31.880236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017397+0000","last_change":"2024-04-26T21:12:55.848208+0000","last_active":"2024-04-26T21:12:57.017397+0000","last_peered":"2024-04-26T21:12:57.017397+0000","last_clean":"2024-04-26T21:12:57.017397+0000","last_became_active":"2024-04-26T21:12:55.847911+0000","last_became_peered":"2024-04-26T21:12:55.847911+0000","last_unstale":"2024-04-26T21:12:57.017397+0000","last_undegraded":"2024-04-26T21:12:57.017397+0000","last_fullsized":"2024-04-26T21:12:57.017397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018179+0000","last_change":"2024-04-26T21:12:54.829043+0000","last_active":"2024-04-26T21:12:57.018179+0000","last_peered":"2024-04-26T21:12:57.018179+0000","last_clean":"2024-04-26T21:12:57.018179+0000","last_became_active":"2024-04-26T21:12:54.828752+0000","last_became_peered":"2024-04-26T21:12:54.828752+0000","last_unstale":"2024-04-26T21:12:57.018179+0000","last_undegraded":"2024-04-26T21:12:57.018179+0000","last_fullsized":"2024-04-26T21:12:57.018179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017744+0000","last_change":"2024-04-26T21:12:55.846026+0000","last_active":"2024-04-26T21:12:57.017744+0000","last_peered":"2024-04-26T21:12:57.017744+0000","last_clean":"2024-04-26T21:12:57.017744+0000","last_became_active":"2024-04-26T21:12:55.845739+0000","last_became_peered":"2024-04-26T21:12:55.845739+0000","last_unstale":"2024-04-26T21:12:57.017744+0000","last_undegraded":"2024-04-26T21:12:57.017744+0000","last_fullsized":"2024-04-26T21:12:57.017744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017377+0000","last_change":"2024-04-26T21:12:54.842196+0000","last_active":"2024-04-26T21:12:57.017377+0000","last_peered":"2024-04-26T21:12:57.017377+0000","last_clean":"2024-04-26T21:12:57.017377+0000","last_became_active":"2024-04-26T21:12:54.841890+0000","last_became_peered":"2024-04-26T21:12:54.841890+0000","last_unstale":"2024-04-26T21:12:57.017377+0000","last_undegraded":"2024-04-26T21:12:57.017377+0000","last_fullsized":"2024-04-26T21:12:57.017377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:46.928170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018560+0000","last_change":"2024-04-26T21:12:55.829002+0000","last_active":"2024-04-26T21:12:57.018560+0000","last_peered":"2024-04-26T21:12:57.018560+0000","last_clean":"2024-04-26T21:12:57.018560+0000","last_became_active":"2024-04-26T21:12:55.828722+0000","last_became_peered":"2024-04-26T21:12:55.828722+0000","last_unstale":"2024-04-26T21:12:57.018560+0000","last_undegraded":"2024-04-26T21:12:57.018560+0000","last_fullsized":"2024-04-26T21:12:57.018560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.834952+0000","last_change":"2024-04-26T21:12:54.833670+0000","last_active":"2024-04-26T21:12:56.834952+0000","last_peered":"2024-04-26T21:12:56.834952+0000","last_clean":"2024-04-26T21:12:56.834952+0000","last_became_active":"2024-04-26T21:12:54.833382+0000","last_became_peered":"2024-04-26T21:12:54.833382+0000","last_unstale":"2024-04-26T21:12:56.834952+0000","last_undegraded":"2024-04-26T21:12:56.834952+0000","last_fullsized":"2024-04-26T21:12:56.834952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018043+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:57.018043+0000","last_peered":"2024-04-26T21:12:57.018043+0000","last_clean":"2024-04-26T21:12:57.018043+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:57.018043+0000","last_undegraded":"2024-04-26T21:12:57.018043+0000","last_fullsized":"2024-04-26T21:12:57.018043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":747537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":157149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:13:00.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-26T21:13:00.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.826216+0000', 'last_change': '2024-04-26T21:12:54.826216+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.826216+0000', 'last_undegraded': '2024-04-26T21:12:54.826216+0000', 'last_fullsized': '2024-04-26T21:12:54.826216+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.825703+0000', 'last_change': '2024-04-26T21:12:54.825703+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.825703+0000', 'last_undegraded': '2024-04-26T21:12:54.825703+0000', 'last_fullsized': '2024-04-26T21:12:54.825703+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.824632+0000', 'last_change': '2024-04-26T21:12:54.824632+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.824632+0000', 'last_undegraded': '2024-04-26T21:12:54.824632+0000', 'last_fullsized': '2024-04-26T21:12:54.824632+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.825269+0000', 'last_change': '2024-04-26T21:12:54.825269+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.825269+0000', 'last_undegraded': '2024-04-26T21:12:54.825269+0000', 'last_fullsized': '2024-04-26T21:12:54.825269+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.824483+0000', 'last_change': '2024-04-26T21:12:54.824483+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.824483+0000', 'last_undegraded': '2024-04-26T21:12:54.824483+0000', 'last_fullsized': '2024-04-26T21:12:54.824483+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.823608+0000', 'last_change': '2024-04-26T21:12:54.823608+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.823608+0000', 'last_undegraded': '2024-04-26T21:12:54.823608+0000', 'last_fullsized': '2024-04-26T21:12:54.823608+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T21:13:00.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.824074+0000', 'last_change': '2024-04-26T21:12:54.824074+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.824074+0000', 'last_undegraded': '2024-04-26T21:12:54.824074+0000', 'last_fullsized': '2024-04-26T21:12:54.824074+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T21:13:00.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:12:54.825701+0000', 'last_change': '2024-04-26T21:12:54.825701+0000', 'last_active': '2024-04-26T21:12:54.809395+0000', 'last_peered': '2024-04-26T21:12:54.809395+0000', 'last_clean': '2024-04-26T21:12:54.809395+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:12:54.825701+0000', 'last_undegraded': '2024-04-26T21:12:54.825701+0000', 'last_fullsized': '2024-04-26T21:12:54.825701+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'last_clean_scrub_stamp': '2024-04-26T21:12:54.809395+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:13:00.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:13:00.640 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:13:00.639+0000 7fbdf581d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:13:00.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.703+0000 7f9c8e378640 1 Processor -- start 2024-04-26T21:13:00.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.703+0000 7f9c8e378640 1 -- start start 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c8e378640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c88073fc0 0x7f9c88072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c8e378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88162e50 0x7f9c88163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c8e378640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c88072560 con 0x7f9c880d0b90 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c8e378640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9c88163910 con 0x7f9c88073fc0 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c8e378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c88163bf0 con 0x7f9c88162e50 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c88073fc0 0x7f9c88072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c88073fc0 0x7f9c88072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37628/0 (socket says 172.21.15.40:37628) 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 -- 172.21.15.40:0/103143724 learned_addr learned my addr 172.21.15.40:0/103143724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c877fe640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88162e50 0x7f9c88163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 -- 172.21.15.40:0/103143724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88162e50 msgr2=0x7f9c88163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88162e50 0x7f9c88163290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.704+0000 7f9c87fff640 1 -- 172.21.15.40:0/103143724 >> v1:172.21.15.184:6789/0 conn(0x7f9c880d0b90 legacy=0x7f9c880d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.705+0000 7f9c87fff640 1 -- 172.21.15.40:0/103143724 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8806ea50 con 0x7f9c88073fc0 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.705+0000 7f9c87fff640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88073fc0 0x7f9c88072020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a968749f6be20f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.705+0000 7f9c86ffd640 1 -- 172.21.15.40:0/103143724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037070 con 0x7f9c88073fc0 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.705+0000 7f9c86ffd640 1 -- 172.21.15.40:0/103143724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7002ae50 con 0x7f9c88073fc0 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.705+0000 7f9c86ffd640 1 -- 172.21.15.40:0/103143724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70032860 con 0x7f9c88073fc0 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 -- 172.21.15.40:0/103143724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88073fc0 msgr2=0x7f9c88072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88073fc0 0x7f9c88072020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002a300 comp rx=0 tx=0).stop 2024-04-26T21:13:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 -- 172.21.15.40:0/103143724 shutdown_connections 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88162e50 0x7f9c88163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 --2- 172.21.15.40:0/103143724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88073fc0 0x7f9c88072020 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 -- 172.21.15.40:0/103143724 >> 172.21.15.40:0/103143724 conn(0x7f9c880cd830 msgr2=0x7f9c880cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 -- 172.21.15.40:0/103143724 shutdown_connections 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.706+0000 7f9c8e378640 1 -- 172.21.15.40:0/103143724 wait complete. 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.707+0000 7f9c8e378640 1 Processor -- start 2024-04-26T21:13:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.707+0000 7f9c8e378640 1 -- start start 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88073fc0 0x7f9c881e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c880d0b90 0x7f9c880b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c880743a0 con 0x7f9c88162e50 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88148de0 con 0x7f9c880d0b90 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8e378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c880b5f80 con 0x7f9c88073fc0 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37638/0 (socket says 172.21.15.40:37638) 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 -- 172.21.15.40:0/583650945 learned_addr learned my addr 172.21.15.40:0/583650945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c877fe640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c880d0b90 0x7f9c880b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c87fff640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88073fc0 0x7f9c881e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 -- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88073fc0 msgr2=0x7f9c881e9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88073fc0 0x7f9c881e9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 -- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c880d0b90 msgr2=0x7f9c880b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c880d0b90 0x7f9c880b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.708+0000 7f9c8c915640 1 -- 172.21.15.40:0/583650945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c880b5690 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c8c915640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c760 tx=0x7f9c8000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80003b70 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80003d30 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000aa90 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881e8f50 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.709+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c880b6950 con 0x7f9c88162e50 2024-04-26T21:13:00.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.710+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c880b5a40 con 0x7f9c88162e50 2024-04-26T21:13:00.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.710+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c8000ac30 con 0x7f9c88162e50 2024-04-26T21:13:00.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.711+0000 7f9c84ff9640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 0x7f9c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:00.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.711+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9c8008b900 con 0x7f9c88162e50 2024-04-26T21:13:00.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.711+0000 7f9c87fff640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 0x7f9c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:00.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.712+0000 7f9c87fff640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 0x7f9c6806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:00.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.715+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c80059640 con 0x7f9c88162e50 2024-04-26T21:13:00.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.975+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9c8806a290 con 0x7f9c88162e50 2024-04-26T21:13:00.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.975+0000 7f9c84ff9640 1 -- 172.21.15.40:0/583650945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1646 (secure 0 0 0) 0x7f9c8008b070 con 0x7f9c88162e50 2024-04-26T21:13:00.976 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:00.976 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592756,"bytes_used":225787904,"bytes_avail":772868325376,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2599,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5145}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-26T21:12:48.929835+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 msgr2=0x7f9c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 0x7f9c6806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002e000 comp rx=0 tx=0).stop 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 msgr2=0x7f9c88160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c760 tx=0x7f9c8000cc20 comp rx=0 tx=0).stop 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 shutdown_connections 2024-04-26T21:13:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c6806c8f0 0x7f9c6806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88162e50 0x7f9c88160c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c880d0b90 0x7f9c880b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 --2- 172.21.15.40:0/583650945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c88073fc0 0x7f9c881e9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 >> 172.21.15.40:0/583650945 conn(0x7f9c880cd830 msgr2=0x7f9c8806f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 shutdown_connections 2024-04-26T21:13:00.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:00.977+0000 7f9c8e378640 1 -- 172.21.15.40:0/583650945 wait complete. 2024-04-26T21:13:00.989 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:13:00.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:13:00.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:13:01.032 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:13:01.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.129+0000 7f9c8a1c6640 1 Processor -- start 2024-04-26T21:13:01.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.129+0000 7f9c8a1c6640 1 -- start start 2024-04-26T21:13:01.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c8a1c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c840cf2e0 0x7f9c8406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c8a1c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c840d11b0 0x7f9c8406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c8a1c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c840d1740 con 0x7f9c8406e2c0 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c8a1c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8406e750 con 0x7f9c840d11b0 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c8a1c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840d1a20 con 0x7f9c840cf2e0 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c840d11b0 0x7f9c8406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c840d11b0 0x7f9c8406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37646/0 (socket says 172.21.15.40:37646) 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 -- 172.21.15.40:0/2878069321 learned_addr learned my addr 172.21.15.40:0/2878069321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c837fe640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c840cf2e0 0x7f9c8406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 -- 172.21.15.40:0/2878069321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c840cf2e0 msgr2=0x7f9c8406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c840cf2e0 0x7f9c8406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 -- 172.21.15.40:0/2878069321 >> v1:172.21.15.184:6789/0 conn(0x7f9c8406e2c0 legacy=0x7f9c8415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.130+0000 7f9c82ffd640 1 -- 172.21.15.40:0/2878069321 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c840d1d00 con 0x7f9c840d11b0 2024-04-26T21:13:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.131+0000 7f9c82ffd640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8406dd80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009870 tx=0x7f9c7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52afa1efc6a70ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.131+0000 7f9c827fc640 1 -- 172.21.15.40:0/2878069321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78037070 con 0x7f9c840d11b0 2024-04-26T21:13:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.131+0000 7f9c827fc640 1 -- 172.21.15.40:0/2878069321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c78032440 con 0x7f9c840d11b0 2024-04-26T21:13:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.131+0000 7f9c827fc640 1 -- 172.21.15.40:0/2878069321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78032810 con 0x7f9c840d11b0 2024-04-26T21:13:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/2878069321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 msgr2=0x7f9c8406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8406dd80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009870 tx=0x7f9c7802a270 comp rx=0 tx=0).stop 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/2878069321 shutdown_connections 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8406dd80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/2878069321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c840cf2e0 0x7f9c8406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/2878069321 >> 172.21.15.40:0/2878069321 conn(0x7f9c840b4a60 msgr2=0x7f9c840b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/2878069321 shutdown_connections 2024-04-26T21:13:01.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.132+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/2878069321 wait complete. 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.133+0000 7f9c8a1c6640 1 Processor -- start 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.133+0000 7f9c8a1c6640 1 -- start start 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c8406e2c0 0x7f9c84160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 0x7f9c8409eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8406da10 con 0x7f9c840d11b0 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840d22c0 con 0x7f9c840cf2e0 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c8a1c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840cf6c0 con 0x7f9c8406e2c0 2024-04-26T21:13:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37658/0 (socket says 172.21.15.40:37658) 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 -- 172.21.15.40:0/702290605 learned_addr learned my addr 172.21.15.40:0/702290605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c837fe640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c8406e2c0 0x7f9c84160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c82ffd640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 0x7f9c8409eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 -- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c8406e2c0 msgr2=0x7f9c84160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c8406e2c0 0x7f9c84160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 -- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 msgr2=0x7f9c8409eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 0x7f9c8409eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c83fff640 1 -- 172.21.15.40:0/702290605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c840c9c70 con 0x7f9c840d11b0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.134+0000 7f9c82ffd640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 0x7f9c8409eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c83fff640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c7d0 tx=0x7f9c7c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c003b70 con 0x7f9c840d11b0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c003d30 con 0x7f9c840d11b0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8409d0b0 con 0x7f9c840d11b0 2024-04-26T21:13:01.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00aa90 con 0x7f9c840d11b0 2024-04-26T21:13:01.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.135+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8409d670 con 0x7f9c840d11b0 2024-04-26T21:13:01.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.136+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8409cea0 con 0x7f9c840d11b0 2024-04-26T21:13:01.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.136+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c7c00ac30 con 0x7f9c840d11b0 2024-04-26T21:13:01.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.137+0000 7f9c6bfff640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:01.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.137+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7c08ba20 con 0x7f9c840d11b0 2024-04-26T21:13:01.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.137+0000 7f9c837fe640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:01.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.138+0000 7f9c837fe640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 0x7f9c7006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.141+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c7c059ee0 con 0x7f9c840d11b0 2024-04-26T21:13:01.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.324+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c840ca0d0 con 0x7f9c7006c8f0 2024-04-26T21:13:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.331+0000 7f9c6bfff640 1 -- 172.21.15.40:0/702290605 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f9c840ca0d0 con 0x7f9c7006c8f0 2024-04-26T21:13:01.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:13:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.333+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 msgr2=0x7f9c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.333+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 0x7f9c7006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74008040 comp rx=0 tx=0).stop 2024-04-26T21:13:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.333+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 msgr2=0x7f9c8409f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.333+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c7d0 tx=0x7f9c7c00cc90 comp rx=0 tx=0).stop 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 shutdown_connections 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c840d11b0 0x7f9c8409f0c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c840cf2e0 0x7f9c8409eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 --2- 172.21.15.40:0/702290605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c8406e2c0 0x7f9c84160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 >> 172.21.15.40:0/702290605 conn(0x7f9c840b4a60 msgr2=0x7f9c840b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 shutdown_connections 2024-04-26T21:13:01.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:01.334+0000 7f9c8a1c6640 1 -- 172.21.15.40:0/702290605 wait complete. 2024-04-26T21:13:01.345 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-26T21:13:00.932556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220548,"kb_used_data":4360,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754172,"statfs":{"total":773094113280,"available":772868272128,"internally_reserved":0,"allocated":4464640,"data_stored":2442188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.114061"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835219+0000","last_change":"2024-04-26T21:12:55.834874+0000","last_active":"2024-04-26T21:12:56.835219+0000","last_peered":"2024-04-26T21:12:56.835219+0000","last_clean":"2024-04-26T21:12:56.835219+0000","last_became_active":"2024-04-26T21:12:55.834593+0000","last_became_peered":"2024-04-26T21:12:55.834593+0000","last_unstale":"2024-04-26T21:12:56.835219+0000","last_undegraded":"2024-04-26T21:12:56.835219+0000","last_fullsized":"2024-04-26T21:12:56.835219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:30.540994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018250+0000","last_change":"2024-04-26T21:12:55.843688+0000","last_active":"2024-04-26T21:12:57.018250+0000","last_peered":"2024-04-26T21:12:57.018250+0000","last_clean":"2024-04-26T21:12:57.018250+0000","last_became_active":"2024-04-26T21:12:55.843298+0000","last_became_peered":"2024-04-26T21:12:55.843298+0000","last_unstale":"2024-04-26T21:12:57.018250+0000","last_undegraded":"2024-04-26T21:12:57.018250+0000","last_fullsized":"2024-04-26T21:12:57.018250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835869+0000","last_change":"2024-04-26T21:12:55.847059+0000","last_active":"2024-04-26T21:12:56.835869+0000","last_peered":"2024-04-26T21:12:56.835869+0000","last_clean":"2024-04-26T21:12:56.835869+0000","last_became_active":"2024-04-26T21:12:55.846609+0000","last_became_peered":"2024-04-26T21:12:55.846609+0000","last_unstale":"2024-04-26T21:12:56.835869+0000","last_undegraded":"2024-04-26T21:12:56.835869+0000","last_fullsized":"2024-04-26T21:12:56.835869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019004+0000","last_change":"2024-04-26T21:12:55.838731+0000","last_active":"2024-04-26T21:12:57.019004+0000","last_peered":"2024-04-26T21:12:57.019004+0000","last_clean":"2024-04-26T21:12:57.019004+0000","last_became_active":"2024-04-26T21:12:55.838444+0000","last_became_peered":"2024-04-26T21:12:55.838444+0000","last_unstale":"2024-04-26T21:12:57.019004+0000","last_undegraded":"2024-04-26T21:12:57.019004+0000","last_fullsized":"2024-04-26T21:12:57.019004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:54.237329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829415+0000","last_change":"2024-04-26T21:12:55.845666+0000","last_active":"2024-04-26T21:12:56.829415+0000","last_peered":"2024-04-26T21:12:56.829415+0000","last_clean":"2024-04-26T21:12:56.829415+0000","last_became_active":"2024-04-26T21:12:55.845327+0000","last_became_peered":"2024-04-26T21:12:55.845327+0000","last_unstale":"2024-04-26T21:12:56.829415+0000","last_undegraded":"2024-04-26T21:12:56.829415+0000","last_fullsized":"2024-04-26T21:12:56.829415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:20.848985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018048+0000","last_change":"2024-04-26T21:12:55.843903+0000","last_active":"2024-04-26T21:12:57.018048+0000","last_peered":"2024-04-26T21:12:57.018048+0000","last_clean":"2024-04-26T21:12:57.018048+0000","last_became_active":"2024-04-26T21:12:55.842999+0000","last_became_peered":"2024-04-26T21:12:55.842999+0000","last_unstale":"2024-04-26T21:12:57.018048+0000","last_undegraded":"2024-04-26T21:12:57.018048+0000","last_fullsized":"2024-04-26T21:12:57.018048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018110+0000","last_change":"2024-04-26T21:12:55.829330+0000","last_active":"2024-04-26T21:12:57.018110+0000","last_peered":"2024-04-26T21:12:57.018110+0000","last_clean":"2024-04-26T21:12:57.018110+0000","last_became_active":"2024-04-26T21:12:55.828931+0000","last_became_peered":"2024-04-26T21:12:55.828931+0000","last_unstale":"2024-04-26T21:12:57.018110+0000","last_undegraded":"2024-04-26T21:12:57.018110+0000","last_fullsized":"2024-04-26T21:12:57.018110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.829652+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.829377+0000","last_became_peered":"2024-04-26T21:12:55.829377+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:29.407920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018385+0000","last_change":"2024-04-26T21:12:55.839415+0000","last_active":"2024-04-26T21:12:57.018385+0000","last_peered":"2024-04-26T21:12:57.018385+0000","last_clean":"2024-04-26T21:12:57.018385+0000","last_became_active":"2024-04-26T21:12:55.839122+0000","last_became_peered":"2024-04-26T21:12:55.839122+0000","last_unstale":"2024-04-26T21:12:57.018385+0000","last_undegraded":"2024-04-26T21:12:57.018385+0000","last_fullsized":"2024-04-26T21:12:57.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:28.305031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018096+0000","last_change":"2024-04-26T21:12:55.846118+0000","last_active":"2024-04-26T21:12:57.018096+0000","last_peered":"2024-04-26T21:12:57.018096+0000","last_clean":"2024-04-26T21:12:57.018096+0000","last_became_active":"2024-04-26T21:12:55.845845+0000","last_became_peered":"2024-04-26T21:12:55.845845+0000","last_unstale":"2024-04-26T21:12:57.018096+0000","last_undegraded":"2024-04-26T21:12:57.018096+0000","last_fullsized":"2024-04-26T21:12:57.018096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:10:25.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018636+0000","last_change":"2024-04-26T21:12:55.845593+0000","last_active":"2024-04-26T21:12:57.018636+0000","last_peered":"2024-04-26T21:12:57.018636+0000","last_clean":"2024-04-26T21:12:57.018636+0000","last_became_active":"2024-04-26T21:12:55.845268+0000","last_became_peered":"2024-04-26T21:12:55.845268+0000","last_unstale":"2024-04-26T21:12:57.018636+0000","last_undegraded":"2024-04-26T21:12:57.018636+0000","last_fullsized":"2024-04-26T21:12:57.018636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835098+0000","last_change":"2024-04-26T21:12:55.845017+0000","last_active":"2024-04-26T21:12:56.835098+0000","last_peered":"2024-04-26T21:12:56.835098+0000","last_clean":"2024-04-26T21:12:56.835098+0000","last_became_active":"2024-04-26T21:12:55.844675+0000","last_became_peered":"2024-04-26T21:12:55.844675+0000","last_unstale":"2024-04-26T21:12:56.835098+0000","last_undegraded":"2024-04-26T21:12:56.835098+0000","last_fullsized":"2024-04-26T21:12:56.835098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829449+0000","last_change":"2024-04-26T21:12:55.848662+0000","last_active":"2024-04-26T21:12:56.829449+0000","last_peered":"2024-04-26T21:12:56.829449+0000","last_clean":"2024-04-26T21:12:56.829449+0000","last_became_active":"2024-04-26T21:12:55.848361+0000","last_became_peered":"2024-04-26T21:12:55.848361+0000","last_unstale":"2024-04-26T21:12:56.829449+0000","last_undegraded":"2024-04-26T21:12:56.829449+0000","last_fullsized":"2024-04-26T21:12:56.829449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:27.246380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018174+0000","last_change":"2024-04-26T21:12:55.851294+0000","last_active":"2024-04-26T21:12:57.018174+0000","last_peered":"2024-04-26T21:12:57.018174+0000","last_clean":"2024-04-26T21:12:57.018174+0000","last_became_active":"2024-04-26T21:12:55.850964+0000","last_became_peered":"2024-04-26T21:12:55.850964+0000","last_unstale":"2024-04-26T21:12:57.018174+0000","last_undegraded":"2024-04-26T21:12:57.018174+0000","last_fullsized":"2024-04-26T21:12:57.018174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835931+0000","last_change":"2024-04-26T21:12:55.827567+0000","last_active":"2024-04-26T21:12:56.835931+0000","last_peered":"2024-04-26T21:12:56.835931+0000","last_clean":"2024-04-26T21:12:56.835931+0000","last_became_active":"2024-04-26T21:12:55.827292+0000","last_became_peered":"2024-04-26T21:12:55.827292+0000","last_unstale":"2024-04-26T21:12:56.835931+0000","last_undegraded":"2024-04-26T21:12:56.835931+0000","last_fullsized":"2024-04-26T21:12:56.835931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017985+0000","last_change":"2024-04-26T21:12:55.848969+0000","last_active":"2024-04-26T21:12:57.017985+0000","last_peered":"2024-04-26T21:12:57.017985+0000","last_clean":"2024-04-26T21:12:57.017985+0000","last_became_active":"2024-04-26T21:12:55.848716+0000","last_became_peered":"2024-04-26T21:12:55.848716+0000","last_unstale":"2024-04-26T21:12:57.017985+0000","last_undegraded":"2024-04-26T21:12:57.017985+0000","last_fullsized":"2024-04-26T21:12:57.017985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018861+0000","last_change":"2024-04-26T21:12:55.834371+0000","last_active":"2024-04-26T21:12:57.018861+0000","last_peered":"2024-04-26T21:12:57.018861+0000","last_clean":"2024-04-26T21:12:57.018861+0000","last_became_active":"2024-04-26T21:12:55.834069+0000","last_became_peered":"2024-04-26T21:12:55.834069+0000","last_unstale":"2024-04-26T21:12:57.018861+0000","last_undegraded":"2024-04-26T21:12:57.018861+0000","last_fullsized":"2024-04-26T21:12:57.018861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018317+0000","last_change":"2024-04-26T21:12:55.852224+0000","last_active":"2024-04-26T21:12:57.018317+0000","last_peered":"2024-04-26T21:12:57.018317+0000","last_clean":"2024-04-26T21:12:57.018317+0000","last_became_active":"2024-04-26T21:12:55.851851+0000","last_became_peered":"2024-04-26T21:12:55.851851+0000","last_unstale":"2024-04-26T21:12:57.018317+0000","last_undegraded":"2024-04-26T21:12:57.018317+0000","last_fullsized":"2024-04-26T21:12:57.018317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:18.378959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017681+0000","last_change":"2024-04-26T21:12:55.852436+0000","last_active":"2024-04-26T21:12:57.017681+0000","last_peered":"2024-04-26T21:12:57.017681+0000","last_clean":"2024-04-26T21:12:57.017681+0000","last_became_active":"2024-04-26T21:12:55.852148+0000","last_became_peered":"2024-04-26T21:12:55.852148+0000","last_unstale":"2024-04-26T21:12:57.017681+0000","last_undegraded":"2024-04-26T21:12:57.017681+0000","last_fullsized":"2024-04-26T21:12:57.017681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829635+0000","last_change":"2024-04-26T21:12:55.833830+0000","last_active":"2024-04-26T21:12:56.829635+0000","last_peered":"2024-04-26T21:12:56.829635+0000","last_clean":"2024-04-26T21:12:56.829635+0000","last_became_active":"2024-04-26T21:12:55.833323+0000","last_became_peered":"2024-04-26T21:12:55.833323+0000","last_unstale":"2024-04-26T21:12:56.829635+0000","last_undegraded":"2024-04-26T21:12:56.829635+0000","last_fullsized":"2024-04-26T21:12:56.829635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:18.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018226+0000","last_change":"2024-04-26T21:12:55.831331+0000","last_active":"2024-04-26T21:12:57.018226+0000","last_peered":"2024-04-26T21:12:57.018226+0000","last_clean":"2024-04-26T21:12:57.018226+0000","last_became_active":"2024-04-26T21:12:55.830490+0000","last_became_peered":"2024-04-26T21:12:55.830490+0000","last_unstale":"2024-04-26T21:12:57.018226+0000","last_undegraded":"2024-04-26T21:12:57.018226+0000","last_fullsized":"2024-04-26T21:12:57.018226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:40.323141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018258+0000","last_change":"2024-04-26T21:12:55.848889+0000","last_active":"2024-04-26T21:12:57.018258+0000","last_peered":"2024-04-26T21:12:57.018258+0000","last_clean":"2024-04-26T21:12:57.018258+0000","last_became_active":"2024-04-26T21:12:55.848607+0000","last_became_peered":"2024-04-26T21:12:55.848607+0000","last_unstale":"2024-04-26T21:12:57.018258+0000","last_undegraded":"2024-04-26T21:12:57.018258+0000","last_fullsized":"2024-04-26T21:12:57.018258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017553+0000","last_change":"2024-04-26T21:12:55.834830+0000","last_active":"2024-04-26T21:12:57.017553+0000","last_peered":"2024-04-26T21:12:57.017553+0000","last_clean":"2024-04-26T21:12:57.017553+0000","last_became_active":"2024-04-26T21:12:55.834541+0000","last_became_peered":"2024-04-26T21:12:55.834541+0000","last_unstale":"2024-04-26T21:12:57.017553+0000","last_undegraded":"2024-04-26T21:12:57.017553+0000","last_fullsized":"2024-04-26T21:12:57.017553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835083+0000","last_change":"2024-04-26T21:12:55.829794+0000","last_active":"2024-04-26T21:12:56.835083+0000","last_peered":"2024-04-26T21:12:56.835083+0000","last_clean":"2024-04-26T21:12:56.835083+0000","last_became_active":"2024-04-26T21:12:55.829261+0000","last_became_peered":"2024-04-26T21:12:55.829261+0000","last_unstale":"2024-04-26T21:12:56.835083+0000","last_undegraded":"2024-04-26T21:12:56.835083+0000","last_fullsized":"2024-04-26T21:12:56.835083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018536+0000","last_change":"2024-04-26T21:12:55.839085+0000","last_active":"2024-04-26T21:12:57.018536+0000","last_peered":"2024-04-26T21:12:57.018536+0000","last_clean":"2024-04-26T21:12:57.018536+0000","last_became_active":"2024-04-26T21:12:55.838774+0000","last_became_peered":"2024-04-26T21:12:55.838774+0000","last_unstale":"2024-04-26T21:12:57.018536+0000","last_undegraded":"2024-04-26T21:12:57.018536+0000","last_fullsized":"2024-04-26T21:12:57.018536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:56.996573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018383+0000","last_change":"2024-04-26T21:12:55.847342+0000","last_active":"2024-04-26T21:12:57.018383+0000","last_peered":"2024-04-26T21:12:57.018383+0000","last_clean":"2024-04-26T21:12:57.018383+0000","last_became_active":"2024-04-26T21:12:55.846476+0000","last_became_peered":"2024-04-26T21:12:55.846476+0000","last_unstale":"2024-04-26T21:12:57.018383+0000","last_undegraded":"2024-04-26T21:12:57.018383+0000","last_fullsized":"2024-04-26T21:12:57.018383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:29.560745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018605+0000","last_change":"2024-04-26T21:12:55.842867+0000","last_active":"2024-04-26T21:12:57.018605+0000","last_peered":"2024-04-26T21:12:57.018605+0000","last_clean":"2024-04-26T21:12:57.018605+0000","last_became_active":"2024-04-26T21:12:55.842387+0000","last_became_peered":"2024-04-26T21:12:55.842387+0000","last_unstale":"2024-04-26T21:12:57.018605+0000","last_undegraded":"2024-04-26T21:12:57.018605+0000","last_fullsized":"2024-04-26T21:12:57.018605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:51.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835252+0000","last_change":"2024-04-26T21:12:55.833992+0000","last_active":"2024-04-26T21:12:56.835252+0000","last_peered":"2024-04-26T21:12:56.835252+0000","last_clean":"2024-04-26T21:12:56.835252+0000","last_became_active":"2024-04-26T21:12:55.833496+0000","last_became_peered":"2024-04-26T21:12:55.833496+0000","last_unstale":"2024-04-26T21:12:56.835252+0000","last_undegraded":"2024-04-26T21:12:56.835252+0000","last_fullsized":"2024-04-26T21:12:56.835252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018305+0000","last_change":"2024-04-26T21:12:55.846206+0000","last_active":"2024-04-26T21:12:57.018305+0000","last_peered":"2024-04-26T21:12:57.018305+0000","last_clean":"2024-04-26T21:12:57.018305+0000","last_became_active":"2024-04-26T21:12:55.845924+0000","last_became_peered":"2024-04-26T21:12:55.845924+0000","last_unstale":"2024-04-26T21:12:57.018305+0000","last_undegraded":"2024-04-26T21:12:57.018305+0000","last_fullsized":"2024-04-26T21:12:57.018305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:12.137923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018052+0000","last_change":"2024-04-26T21:12:55.833028+0000","last_active":"2024-04-26T21:12:57.018052+0000","last_peered":"2024-04-26T21:12:57.018052+0000","last_clean":"2024-04-26T21:12:57.018052+0000","last_became_active":"2024-04-26T21:12:55.832783+0000","last_became_peered":"2024-04-26T21:12:55.832783+0000","last_unstale":"2024-04-26T21:12:57.018052+0000","last_undegraded":"2024-04-26T21:12:57.018052+0000","last_fullsized":"2024-04-26T21:12:57.018052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835242+0000","last_change":"2024-04-26T21:12:55.829964+0000","last_active":"2024-04-26T21:12:56.835242+0000","last_peered":"2024-04-26T21:12:56.835242+0000","last_clean":"2024-04-26T21:12:56.835242+0000","last_became_active":"2024-04-26T21:12:55.828615+0000","last_became_peered":"2024-04-26T21:12:55.828615+0000","last_unstale":"2024-04-26T21:12:56.835242+0000","last_undegraded":"2024-04-26T21:12:56.835242+0000","last_fullsized":"2024-04-26T21:12:56.835242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:33.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017806+0000","last_change":"2024-04-26T21:12:55.835339+0000","last_active":"2024-04-26T21:12:57.017806+0000","last_peered":"2024-04-26T21:12:57.017806+0000","last_clean":"2024-04-26T21:12:57.017806+0000","last_became_active":"2024-04-26T21:12:55.834346+0000","last_became_peered":"2024-04-26T21:12:55.834346+0000","last_unstale":"2024-04-26T21:12:57.017806+0000","last_undegraded":"2024-04-26T21:12:57.017806+0000","last_fullsized":"2024-04-26T21:12:57.017806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017889+0000","last_change":"2024-04-26T21:12:55.835570+0000","last_active":"2024-04-26T21:12:57.017889+0000","last_peered":"2024-04-26T21:12:57.017889+0000","last_clean":"2024-04-26T21:12:57.017889+0000","last_became_active":"2024-04-26T21:12:55.835290+0000","last_became_peered":"2024-04-26T21:12:55.835290+0000","last_unstale":"2024-04-26T21:12:57.017889+0000","last_undegraded":"2024-04-26T21:12:57.017889+0000","last_fullsized":"2024-04-26T21:12:57.017889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835504+0000","last_change":"2024-04-26T21:12:54.836243+0000","last_active":"2024-04-26T21:12:56.835504+0000","last_peered":"2024-04-26T21:12:56.835504+0000","last_clean":"2024-04-26T21:12:56.835504+0000","last_became_active":"2024-04-26T21:12:54.835926+0000","last_became_peered":"2024-04-26T21:12:54.835926+0000","last_unstale":"2024-04-26T21:12:56.835504+0000","last_undegraded":"2024-04-26T21:12:56.835504+0000","last_fullsized":"2024-04-26T21:12:56.835504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835388+0000","last_change":"2024-04-26T21:12:55.833085+0000","last_active":"2024-04-26T21:12:56.835388+0000","last_peered":"2024-04-26T21:12:56.835388+0000","last_clean":"2024-04-26T21:12:56.835388+0000","last_became_active":"2024-04-26T21:12:55.832730+0000","last_became_peered":"2024-04-26T21:12:55.832730+0000","last_unstale":"2024-04-26T21:12:56.835388+0000","last_undegraded":"2024-04-26T21:12:56.835388+0000","last_fullsized":"2024-04-26T21:12:56.835388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018001+0000","last_change":"2024-04-26T21:12:54.828997+0000","last_active":"2024-04-26T21:12:57.018001+0000","last_peered":"2024-04-26T21:12:57.018001+0000","last_clean":"2024-04-26T21:12:57.018001+0000","last_became_active":"2024-04-26T21:12:54.828635+0000","last_became_peered":"2024-04-26T21:12:54.828635+0000","last_unstale":"2024-04-26T21:12:57.018001+0000","last_undegraded":"2024-04-26T21:12:57.018001+0000","last_fullsized":"2024-04-26T21:12:57.018001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018665+0000","last_change":"2024-04-26T21:12:55.846106+0000","last_active":"2024-04-26T21:12:57.018665+0000","last_peered":"2024-04-26T21:12:57.018665+0000","last_clean":"2024-04-26T21:12:57.018665+0000","last_became_active":"2024-04-26T21:12:55.845816+0000","last_became_peered":"2024-04-26T21:12:55.845816+0000","last_unstale":"2024-04-26T21:12:57.018665+0000","last_undegraded":"2024-04-26T21:12:57.018665+0000","last_fullsized":"2024-04-26T21:12:57.018665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:44.371719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017598+0000","last_change":"2024-04-26T21:12:54.841177+0000","last_active":"2024-04-26T21:12:57.017598+0000","last_peered":"2024-04-26T21:12:57.017598+0000","last_clean":"2024-04-26T21:12:57.017598+0000","last_became_active":"2024-04-26T21:12:54.840803+0000","last_became_peered":"2024-04-26T21:12:54.840803+0000","last_unstale":"2024-04-26T21:12:57.017598+0000","last_undegraded":"2024-04-26T21:12:57.017598+0000","last_fullsized":"2024-04-26T21:12:57.017598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.922188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018748+0000","last_change":"2024-04-26T21:12:55.842589+0000","last_active":"2024-04-26T21:12:57.018748+0000","last_peered":"2024-04-26T21:12:57.018748+0000","last_clean":"2024-04-26T21:12:57.018748+0000","last_became_active":"2024-04-26T21:12:55.841589+0000","last_became_peered":"2024-04-26T21:12:55.841589+0000","last_unstale":"2024-04-26T21:12:57.018748+0000","last_undegraded":"2024-04-26T21:12:57.018748+0000","last_fullsized":"2024-04-26T21:12:57.018748+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:57.131512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829652+0000","last_change":"2024-04-26T21:12:54.833964+0000","last_active":"2024-04-26T21:12:56.829652+0000","last_peered":"2024-04-26T21:12:56.829652+0000","last_clean":"2024-04-26T21:12:56.829652+0000","last_became_active":"2024-04-26T21:12:54.833662+0000","last_became_peered":"2024-04-26T21:12:54.833662+0000","last_unstale":"2024-04-26T21:12:56.829652+0000","last_undegraded":"2024-04-26T21:12:56.829652+0000","last_fullsized":"2024-04-26T21:12:56.829652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:11.073082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018468+0000","last_change":"2024-04-26T21:12:55.839174+0000","last_active":"2024-04-26T21:12:57.018468+0000","last_peered":"2024-04-26T21:12:57.018468+0000","last_clean":"2024-04-26T21:12:57.018468+0000","last_became_active":"2024-04-26T21:12:55.838902+0000","last_became_peered":"2024-04-26T21:12:55.838902+0000","last_unstale":"2024-04-26T21:12:57.018468+0000","last_undegraded":"2024-04-26T21:12:57.018468+0000","last_fullsized":"2024-04-26T21:12:57.018468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018348+0000","last_change":"2024-04-26T21:12:54.838674+0000","last_active":"2024-04-26T21:12:57.018348+0000","last_peered":"2024-04-26T21:12:57.018348+0000","last_clean":"2024-04-26T21:12:57.018348+0000","last_became_active":"2024-04-26T21:12:54.838357+0000","last_became_peered":"2024-04-26T21:12:54.838357+0000","last_unstale":"2024-04-26T21:12:57.018348+0000","last_undegraded":"2024-04-26T21:12:57.018348+0000","last_fullsized":"2024-04-26T21:12:57.018348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835420+0000","last_change":"2024-04-26T21:12:55.833557+0000","last_active":"2024-04-26T21:12:56.835420+0000","last_peered":"2024-04-26T21:12:56.835420+0000","last_clean":"2024-04-26T21:12:56.835420+0000","last_became_active":"2024-04-26T21:12:55.833295+0000","last_became_peered":"2024-04-26T21:12:55.833295+0000","last_unstale":"2024-04-26T21:12:56.835420+0000","last_undegraded":"2024-04-26T21:12:56.835420+0000","last_fullsized":"2024-04-26T21:12:56.835420+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018262+0000","last_change":"2024-04-26T21:12:54.837240+0000","last_active":"2024-04-26T21:12:57.018262+0000","last_peered":"2024-04-26T21:12:57.018262+0000","last_clean":"2024-04-26T21:12:57.018262+0000","last_became_active":"2024-04-26T21:12:54.837022+0000","last_became_peered":"2024-04-26T21:12:54.837022+0000","last_unstale":"2024-04-26T21:12:57.018262+0000","last_undegraded":"2024-04-26T21:12:57.018262+0000","last_fullsized":"2024-04-26T21:12:57.018262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:20.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017917+0000","last_change":"2024-04-26T21:12:55.828118+0000","last_active":"2024-04-26T21:12:57.017917+0000","last_peered":"2024-04-26T21:12:57.017917+0000","last_clean":"2024-04-26T21:12:57.017917+0000","last_became_active":"2024-04-26T21:12:55.827805+0000","last_became_peered":"2024-04-26T21:12:55.827805+0000","last_unstale":"2024-04-26T21:12:57.017917+0000","last_undegraded":"2024-04-26T21:12:57.017917+0000","last_fullsized":"2024-04-26T21:12:57.017917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017693+0000","last_change":"2024-04-26T21:12:54.835535+0000","last_active":"2024-04-26T21:12:57.017693+0000","last_peered":"2024-04-26T21:12:57.017693+0000","last_clean":"2024-04-26T21:12:57.017693+0000","last_became_active":"2024-04-26T21:12:54.835258+0000","last_became_peered":"2024-04-26T21:12:54.835258+0000","last_unstale":"2024-04-26T21:12:57.017693+0000","last_undegraded":"2024-04-26T21:12:57.017693+0000","last_fullsized":"2024-04-26T21:12:57.017693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017840+0000","last_change":"2024-04-26T21:12:55.840788+0000","last_active":"2024-04-26T21:12:57.017840+0000","last_peered":"2024-04-26T21:12:57.017840+0000","last_clean":"2024-04-26T21:12:57.017840+0000","last_became_active":"2024-04-26T21:12:55.840492+0000","last_became_peered":"2024-04-26T21:12:55.840492+0000","last_unstale":"2024-04-26T21:12:57.017840+0000","last_undegraded":"2024-04-26T21:12:57.017840+0000","last_fullsized":"2024-04-26T21:12:57.017840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:46.344935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017760+0000","last_change":"2024-04-26T21:12:54.833665+0000","last_active":"2024-04-26T21:12:57.017760+0000","last_peered":"2024-04-26T21:12:57.017760+0000","last_clean":"2024-04-26T21:12:57.017760+0000","last_became_active":"2024-04-26T21:12:54.833394+0000","last_became_peered":"2024-04-26T21:12:54.833394+0000","last_unstale":"2024-04-26T21:12:57.017760+0000","last_undegraded":"2024-04-26T21:12:57.017760+0000","last_fullsized":"2024-04-26T21:12:57.017760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:18.903622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.836233+0000","last_change":"2024-04-26T21:12:55.844791+0000","last_active":"2024-04-26T21:12:56.836233+0000","last_peered":"2024-04-26T21:12:56.836233+0000","last_clean":"2024-04-26T21:12:56.836233+0000","last_became_active":"2024-04-26T21:12:55.844567+0000","last_became_peered":"2024-04-26T21:12:55.844567+0000","last_unstale":"2024-04-26T21:12:56.836233+0000","last_undegraded":"2024-04-26T21:12:56.836233+0000","last_fullsized":"2024-04-26T21:12:56.836233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017815+0000","last_change":"2024-04-26T21:12:54.842287+0000","last_active":"2024-04-26T21:12:57.017815+0000","last_peered":"2024-04-26T21:12:57.017815+0000","last_clean":"2024-04-26T21:12:57.017815+0000","last_became_active":"2024-04-26T21:12:54.841969+0000","last_became_peered":"2024-04-26T21:12:54.841969+0000","last_unstale":"2024-04-26T21:12:57.017815+0000","last_undegraded":"2024-04-26T21:12:57.017815+0000","last_fullsized":"2024-04-26T21:12:57.017815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.740507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017709+0000","last_change":"2024-04-26T21:12:55.852140+0000","last_active":"2024-04-26T21:12:57.017709+0000","last_peered":"2024-04-26T21:12:57.017709+0000","last_clean":"2024-04-26T21:12:57.017709+0000","last_became_active":"2024-04-26T21:12:55.851753+0000","last_became_peered":"2024-04-26T21:12:55.851753+0000","last_unstale":"2024-04-26T21:12:57.017709+0000","last_undegraded":"2024-04-26T21:12:57.017709+0000","last_fullsized":"2024-04-26T21:12:57.017709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:54.832871+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:54.832587+0000","last_became_peered":"2024-04-26T21:12:54.832587+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017990+0000","last_change":"2024-04-26T21:12:55.851660+0000","last_active":"2024-04-26T21:12:57.017990+0000","last_peered":"2024-04-26T21:12:57.017990+0000","last_clean":"2024-04-26T21:12:57.017990+0000","last_became_active":"2024-04-26T21:12:55.850979+0000","last_became_peered":"2024-04-26T21:12:55.850979+0000","last_unstale":"2024-04-26T21:12:57.017990+0000","last_undegraded":"2024-04-26T21:12:57.017990+0000","last_fullsized":"2024-04-26T21:12:57.017990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835911+0000","last_change":"2024-04-26T21:12:54.830694+0000","last_active":"2024-04-26T21:12:56.835911+0000","last_peered":"2024-04-26T21:12:56.835911+0000","last_clean":"2024-04-26T21:12:56.835911+0000","last_became_active":"2024-04-26T21:12:54.830373+0000","last_became_peered":"2024-04-26T21:12:54.830373+0000","last_unstale":"2024-04-26T21:12:56.835911+0000","last_undegraded":"2024-04-26T21:12:56.835911+0000","last_fullsized":"2024-04-26T21:12:56.835911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835683+0000","last_change":"2024-04-26T21:12:55.846932+0000","last_active":"2024-04-26T21:12:56.835683+0000","last_peered":"2024-04-26T21:12:56.835683+0000","last_clean":"2024-04-26T21:12:56.835683+0000","last_became_active":"2024-04-26T21:12:55.846616+0000","last_became_peered":"2024-04-26T21:12:55.846616+0000","last_unstale":"2024-04-26T21:12:56.835683+0000","last_undegraded":"2024-04-26T21:12:56.835683+0000","last_fullsized":"2024-04-26T21:12:56.835683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017971+0000","last_change":"2024-04-26T21:12:54.840841+0000","last_active":"2024-04-26T21:12:57.017971+0000","last_peered":"2024-04-26T21:12:57.017971+0000","last_clean":"2024-04-26T21:12:57.017971+0000","last_became_active":"2024-04-26T21:12:54.840336+0000","last_became_peered":"2024-04-26T21:12:54.840336+0000","last_unstale":"2024-04-26T21:12:57.017971+0000","last_undegraded":"2024-04-26T21:12:57.017971+0000","last_fullsized":"2024-04-26T21:12:57.017971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018332+0000","last_change":"2024-04-26T21:12:55.843140+0000","last_active":"2024-04-26T21:12:57.018332+0000","last_peered":"2024-04-26T21:12:57.018332+0000","last_clean":"2024-04-26T21:12:57.018332+0000","last_became_active":"2024-04-26T21:12:55.842833+0000","last_became_peered":"2024-04-26T21:12:55.842833+0000","last_unstale":"2024-04-26T21:12:57.018332+0000","last_undegraded":"2024-04-26T21:12:57.018332+0000","last_fullsized":"2024-04-26T21:12:57.018332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:46.288609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018411+0000","last_change":"2024-04-26T21:12:54.833969+0000","last_active":"2024-04-26T21:12:57.018411+0000","last_peered":"2024-04-26T21:12:57.018411+0000","last_clean":"2024-04-26T21:12:57.018411+0000","last_became_active":"2024-04-26T21:12:54.833628+0000","last_became_peered":"2024-04-26T21:12:54.833628+0000","last_unstale":"2024-04-26T21:12:57.018411+0000","last_undegraded":"2024-04-26T21:12:57.018411+0000","last_fullsized":"2024-04-26T21:12:57.018411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.844863+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.844486+0000","last_became_peered":"2024-04-26T21:12:55.844486+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018263+0000","last_change":"2024-04-26T21:12:54.831096+0000","last_active":"2024-04-26T21:12:57.018263+0000","last_peered":"2024-04-26T21:12:57.018263+0000","last_clean":"2024-04-26T21:12:57.018263+0000","last_became_active":"2024-04-26T21:12:54.830785+0000","last_became_peered":"2024-04-26T21:12:54.830785+0000","last_unstale":"2024-04-26T21:12:57.018263+0000","last_undegraded":"2024-04-26T21:12:57.018263+0000","last_fullsized":"2024-04-26T21:12:57.018263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:45.252155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018645+0000","last_change":"2024-04-26T21:12:55.834991+0000","last_active":"2024-04-26T21:12:57.018645+0000","last_peered":"2024-04-26T21:12:57.018645+0000","last_clean":"2024-04-26T21:12:57.018645+0000","last_became_active":"2024-04-26T21:12:55.834663+0000","last_became_peered":"2024-04-26T21:12:55.834663+0000","last_unstale":"2024-04-26T21:12:57.018645+0000","last_undegraded":"2024-04-26T21:12:57.018645+0000","last_fullsized":"2024-04-26T21:12:57.018645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017474+0000","last_change":"2024-04-26T21:12:54.842208+0000","last_active":"2024-04-26T21:12:57.017474+0000","last_peered":"2024-04-26T21:12:57.017474+0000","last_clean":"2024-04-26T21:12:57.017474+0000","last_became_active":"2024-04-26T21:12:54.841932+0000","last_became_peered":"2024-04-26T21:12:54.841932+0000","last_unstale":"2024-04-26T21:12:57.017474+0000","last_undegraded":"2024-04-26T21:12:57.017474+0000","last_fullsized":"2024-04-26T21:12:57.017474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:29.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835302+0000","last_change":"2024-04-26T21:12:55.830389+0000","last_active":"2024-04-26T21:12:56.835302+0000","last_peered":"2024-04-26T21:12:56.835302+0000","last_clean":"2024-04-26T21:12:56.835302+0000","last_became_active":"2024-04-26T21:12:55.830080+0000","last_became_peered":"2024-04-26T21:12:55.830080+0000","last_unstale":"2024-04-26T21:12:56.835302+0000","last_undegraded":"2024-04-26T21:12:56.835302+0000","last_fullsized":"2024-04-26T21:12:56.835302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017777+0000","last_change":"2024-04-26T21:12:54.833724+0000","last_active":"2024-04-26T21:12:57.017777+0000","last_peered":"2024-04-26T21:12:57.017777+0000","last_clean":"2024-04-26T21:12:57.017777+0000","last_became_active":"2024-04-26T21:12:54.833458+0000","last_became_peered":"2024-04-26T21:12:54.833458+0000","last_unstale":"2024-04-26T21:12:57.017777+0000","last_undegraded":"2024-04-26T21:12:57.017777+0000","last_fullsized":"2024-04-26T21:12:57.017777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:26.631042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:55.835158+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:55.834796+0000","last_became_peered":"2024-04-26T21:12:55.834796+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018703+0000","last_change":"2024-04-26T21:12:54.837631+0000","last_active":"2024-04-26T21:12:57.018703+0000","last_peered":"2024-04-26T21:12:57.018703+0000","last_clean":"2024-04-26T21:12:57.018703+0000","last_became_active":"2024-04-26T21:12:54.837117+0000","last_became_peered":"2024-04-26T21:12:54.837117+0000","last_unstale":"2024-04-26T21:12:57.018703+0000","last_undegraded":"2024-04-26T21:12:57.018703+0000","last_fullsized":"2024-04-26T21:12:57.018703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:45.775778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835652+0000","last_change":"2024-04-26T21:12:55.837208+0000","last_active":"2024-04-26T21:12:56.835652+0000","last_peered":"2024-04-26T21:12:56.835652+0000","last_clean":"2024-04-26T21:12:56.835652+0000","last_became_active":"2024-04-26T21:12:55.836934+0000","last_became_peered":"2024-04-26T21:12:55.836934+0000","last_unstale":"2024-04-26T21:12:56.835652+0000","last_undegraded":"2024-04-26T21:12:56.835652+0000","last_fullsized":"2024-04-26T21:12:56.835652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018106+0000","last_change":"2024-04-26T21:12:54.833728+0000","last_active":"2024-04-26T21:12:57.018106+0000","last_peered":"2024-04-26T21:12:57.018106+0000","last_clean":"2024-04-26T21:12:57.018106+0000","last_became_active":"2024-04-26T21:12:54.833388+0000","last_became_peered":"2024-04-26T21:12:54.833388+0000","last_unstale":"2024-04-26T21:12:57.018106+0000","last_undegraded":"2024-04-26T21:12:57.018106+0000","last_fullsized":"2024-04-26T21:12:57.018106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019012+0000","last_change":"2024-04-26T21:12:55.838065+0000","last_active":"2024-04-26T21:12:57.019012+0000","last_peered":"2024-04-26T21:12:57.019012+0000","last_clean":"2024-04-26T21:12:57.019012+0000","last_became_active":"2024-04-26T21:12:55.837784+0000","last_became_peered":"2024-04-26T21:12:55.837784+0000","last_unstale":"2024-04-26T21:12:57.019012+0000","last_undegraded":"2024-04-26T21:12:57.019012+0000","last_fullsized":"2024-04-26T21:12:57.019012+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:13.088171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018083+0000","last_change":"2024-04-26T21:12:54.832422+0000","last_active":"2024-04-26T21:12:57.018083+0000","last_peered":"2024-04-26T21:12:57.018083+0000","last_clean":"2024-04-26T21:12:57.018083+0000","last_became_active":"2024-04-26T21:12:54.832171+0000","last_became_peered":"2024-04-26T21:12:54.832171+0000","last_unstale":"2024-04-26T21:12:57.018083+0000","last_undegraded":"2024-04-26T21:12:57.018083+0000","last_fullsized":"2024-04-26T21:12:57.018083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:33.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829689+0000","last_change":"2024-04-26T21:12:55.840206+0000","last_active":"2024-04-26T21:12:56.829689+0000","last_peered":"2024-04-26T21:12:56.829689+0000","last_clean":"2024-04-26T21:12:56.829689+0000","last_became_active":"2024-04-26T21:12:55.839915+0000","last_became_peered":"2024-04-26T21:12:55.839915+0000","last_unstale":"2024-04-26T21:12:56.829689+0000","last_undegraded":"2024-04-26T21:12:56.829689+0000","last_fullsized":"2024-04-26T21:12:56.829689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.934501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018739+0000","last_change":"2024-04-26T21:12:54.838655+0000","last_active":"2024-04-26T21:12:57.018739+0000","last_peered":"2024-04-26T21:12:57.018739+0000","last_clean":"2024-04-26T21:12:57.018739+0000","last_became_active":"2024-04-26T21:12:54.838244+0000","last_became_peered":"2024-04-26T21:12:54.838244+0000","last_unstale":"2024-04-26T21:12:57.018739+0000","last_undegraded":"2024-04-26T21:12:57.018739+0000","last_fullsized":"2024-04-26T21:12:57.018739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:26.055217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.834792+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.834492+0000","last_became_peered":"2024-04-26T21:12:55.834492+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:12.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018066+0000","last_change":"2024-04-26T21:12:54.834125+0000","last_active":"2024-04-26T21:12:57.018066+0000","last_peered":"2024-04-26T21:12:57.018066+0000","last_clean":"2024-04-26T21:12:57.018066+0000","last_became_active":"2024-04-26T21:12:54.833830+0000","last_became_peered":"2024-04-26T21:12:54.833830+0000","last_unstale":"2024-04-26T21:12:57.018066+0000","last_undegraded":"2024-04-26T21:12:57.018066+0000","last_fullsized":"2024-04-26T21:12:57.018066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:17.724781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829576+0000","last_change":"2024-04-26T21:12:55.829614+0000","last_active":"2024-04-26T21:12:56.829576+0000","last_peered":"2024-04-26T21:12:56.829576+0000","last_clean":"2024-04-26T21:12:56.829576+0000","last_became_active":"2024-04-26T21:12:55.829102+0000","last_became_peered":"2024-04-26T21:12:55.829102+0000","last_unstale":"2024-04-26T21:12:56.829576+0000","last_undegraded":"2024-04-26T21:12:56.829576+0000","last_fullsized":"2024-04-26T21:12:56.829576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.539573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835517+0000","last_change":"2024-04-26T21:12:54.833584+0000","last_active":"2024-04-26T21:12:56.835517+0000","last_peered":"2024-04-26T21:12:56.835517+0000","last_clean":"2024-04-26T21:12:56.835517+0000","last_became_active":"2024-04-26T21:12:54.833302+0000","last_became_peered":"2024-04-26T21:12:54.833302+0000","last_unstale":"2024-04-26T21:12:56.835517+0000","last_undegraded":"2024-04-26T21:12:56.835517+0000","last_fullsized":"2024-04-26T21:12:56.835517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.829378+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.829065+0000","last_became_peered":"2024-04-26T21:12:55.829065+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:01.592713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017725+0000","last_change":"2024-04-26T21:12:54.834382+0000","last_active":"2024-04-26T21:12:57.017725+0000","last_peered":"2024-04-26T21:12:57.017725+0000","last_clean":"2024-04-26T21:12:57.017725+0000","last_became_active":"2024-04-26T21:12:54.833899+0000","last_became_peered":"2024-04-26T21:12:54.833899+0000","last_unstale":"2024-04-26T21:12:57.017725+0000","last_undegraded":"2024-04-26T21:12:57.017725+0000","last_fullsized":"2024-04-26T21:12:57.017725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:11.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018255+0000","last_change":"2024-04-26T21:12:55.831832+0000","last_active":"2024-04-26T21:12:57.018255+0000","last_peered":"2024-04-26T21:12:57.018255+0000","last_clean":"2024-04-26T21:12:57.018255+0000","last_became_active":"2024-04-26T21:12:55.831550+0000","last_became_peered":"2024-04-26T21:12:55.831550+0000","last_unstale":"2024-04-26T21:12:57.018255+0000","last_undegraded":"2024-04-26T21:12:57.018255+0000","last_fullsized":"2024-04-26T21:12:57.018255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835186+0000","last_change":"2024-04-26T21:12:54.830666+0000","last_active":"2024-04-26T21:12:56.835186+0000","last_peered":"2024-04-26T21:12:56.835186+0000","last_clean":"2024-04-26T21:12:56.835186+0000","last_became_active":"2024-04-26T21:12:54.830338+0000","last_became_peered":"2024-04-26T21:12:54.830338+0000","last_unstale":"2024-04-26T21:12:56.835186+0000","last_undegraded":"2024-04-26T21:12:56.835186+0000","last_fullsized":"2024-04-26T21:12:56.835186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829680+0000","last_change":"2024-04-26T21:12:55.834082+0000","last_active":"2024-04-26T21:12:56.829680+0000","last_peered":"2024-04-26T21:12:56.829680+0000","last_clean":"2024-04-26T21:12:56.829680+0000","last_became_active":"2024-04-26T21:12:55.833778+0000","last_became_peered":"2024-04-26T21:12:55.833778+0000","last_unstale":"2024-04-26T21:12:56.829680+0000","last_undegraded":"2024-04-26T21:12:56.829680+0000","last_fullsized":"2024-04-26T21:12:56.829680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:57.707945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018803+0000","last_change":"2024-04-26T21:12:54.837054+0000","last_active":"2024-04-26T21:12:57.018803+0000","last_peered":"2024-04-26T21:12:57.018803+0000","last_clean":"2024-04-26T21:12:57.018803+0000","last_became_active":"2024-04-26T21:12:54.836690+0000","last_became_peered":"2024-04-26T21:12:54.836690+0000","last_unstale":"2024-04-26T21:12:57.018803+0000","last_undegraded":"2024-04-26T21:12:57.018803+0000","last_fullsized":"2024-04-26T21:12:57.018803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:18.816955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835819+0000","last_change":"2024-04-26T21:12:55.840516+0000","last_active":"2024-04-26T21:12:56.835819+0000","last_peered":"2024-04-26T21:12:56.835819+0000","last_clean":"2024-04-26T21:12:56.835819+0000","last_became_active":"2024-04-26T21:12:55.840212+0000","last_became_peered":"2024-04-26T21:12:55.840212+0000","last_unstale":"2024-04-26T21:12:56.835819+0000","last_undegraded":"2024-04-26T21:12:56.835819+0000","last_fullsized":"2024-04-26T21:12:56.835819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018109+0000","last_change":"2024-04-26T21:12:54.834650+0000","last_active":"2024-04-26T21:12:57.018109+0000","last_peered":"2024-04-26T21:12:57.018109+0000","last_clean":"2024-04-26T21:12:57.018109+0000","last_became_active":"2024-04-26T21:12:54.834365+0000","last_became_peered":"2024-04-26T21:12:54.834365+0000","last_unstale":"2024-04-26T21:12:57.018109+0000","last_undegraded":"2024-04-26T21:12:57.018109+0000","last_fullsized":"2024-04-26T21:12:57.018109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017612+0000","last_change":"2024-04-26T21:12:55.846844+0000","last_active":"2024-04-26T21:12:57.017612+0000","last_peered":"2024-04-26T21:12:57.017612+0000","last_clean":"2024-04-26T21:12:57.017612+0000","last_became_active":"2024-04-26T21:12:55.846587+0000","last_became_peered":"2024-04-26T21:12:55.846587+0000","last_unstale":"2024-04-26T21:12:57.017612+0000","last_undegraded":"2024-04-26T21:12:57.017612+0000","last_fullsized":"2024-04-26T21:12:57.017612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019113+0000","last_change":"2024-04-26T21:12:54.833945+0000","last_active":"2024-04-26T21:12:57.019113+0000","last_peered":"2024-04-26T21:12:57.019113+0000","last_clean":"2024-04-26T21:12:57.019113+0000","last_became_active":"2024-04-26T21:12:54.833186+0000","last_became_peered":"2024-04-26T21:12:54.833186+0000","last_unstale":"2024-04-26T21:12:57.019113+0000","last_undegraded":"2024-04-26T21:12:57.019113+0000","last_fullsized":"2024-04-26T21:12:57.019113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:10.115123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835471+0000","last_change":"2024-04-26T21:12:55.831340+0000","last_active":"2024-04-26T21:12:56.835471+0000","last_peered":"2024-04-26T21:12:56.835471+0000","last_clean":"2024-04-26T21:12:56.835471+0000","last_became_active":"2024-04-26T21:12:55.831068+0000","last_became_peered":"2024-04-26T21:12:55.831068+0000","last_unstale":"2024-04-26T21:12:56.835471+0000","last_undegraded":"2024-04-26T21:12:56.835471+0000","last_fullsized":"2024-04-26T21:12:56.835471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018242+0000","last_change":"2024-04-26T21:12:54.825357+0000","last_active":"2024-04-26T21:12:57.018242+0000","last_peered":"2024-04-26T21:12:57.018242+0000","last_clean":"2024-04-26T21:12:57.018242+0000","last_became_active":"2024-04-26T21:12:54.824948+0000","last_became_peered":"2024-04-26T21:12:54.824948+0000","last_unstale":"2024-04-26T21:12:57.018242+0000","last_undegraded":"2024-04-26T21:12:57.018242+0000","last_fullsized":"2024-04-26T21:12:57.018242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:56.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017961+0000","last_change":"2024-04-26T21:12:55.832196+0000","last_active":"2024-04-26T21:12:57.017961+0000","last_peered":"2024-04-26T21:12:57.017961+0000","last_clean":"2024-04-26T21:12:57.017961+0000","last_became_active":"2024-04-26T21:12:55.831921+0000","last_became_peered":"2024-04-26T21:12:55.831921+0000","last_unstale":"2024-04-26T21:12:57.017961+0000","last_undegraded":"2024-04-26T21:12:57.017961+0000","last_fullsized":"2024-04-26T21:12:57.017961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018382+0000","last_change":"2024-04-26T21:12:54.833203+0000","last_active":"2024-04-26T21:12:57.018382+0000","last_peered":"2024-04-26T21:12:57.018382+0000","last_clean":"2024-04-26T21:12:57.018382+0000","last_became_active":"2024-04-26T21:12:54.832884+0000","last_became_peered":"2024-04-26T21:12:54.832884+0000","last_unstale":"2024-04-26T21:12:57.018382+0000","last_undegraded":"2024-04-26T21:12:57.018382+0000","last_fullsized":"2024-04-26T21:12:57.018382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:36.900099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017397+0000","last_change":"2024-04-26T21:12:55.848208+0000","last_active":"2024-04-26T21:12:57.017397+0000","last_peered":"2024-04-26T21:12:57.017397+0000","last_clean":"2024-04-26T21:12:57.017397+0000","last_became_active":"2024-04-26T21:12:55.847911+0000","last_became_peered":"2024-04-26T21:12:55.847911+0000","last_unstale":"2024-04-26T21:12:57.017397+0000","last_undegraded":"2024-04-26T21:12:57.017397+0000","last_fullsized":"2024-04-26T21:12:57.017397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018179+0000","last_change":"2024-04-26T21:12:54.829043+0000","last_active":"2024-04-26T21:12:57.018179+0000","last_peered":"2024-04-26T21:12:57.018179+0000","last_clean":"2024-04-26T21:12:57.018179+0000","last_became_active":"2024-04-26T21:12:54.828752+0000","last_became_peered":"2024-04-26T21:12:54.828752+0000","last_unstale":"2024-04-26T21:12:57.018179+0000","last_undegraded":"2024-04-26T21:12:57.018179+0000","last_fullsized":"2024-04-26T21:12:57.018179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017744+0000","last_change":"2024-04-26T21:12:55.846026+0000","last_active":"2024-04-26T21:12:57.017744+0000","last_peered":"2024-04-26T21:12:57.017744+0000","last_clean":"2024-04-26T21:12:57.017744+0000","last_became_active":"2024-04-26T21:12:55.845739+0000","last_became_peered":"2024-04-26T21:12:55.845739+0000","last_unstale":"2024-04-26T21:12:57.017744+0000","last_undegraded":"2024-04-26T21:12:57.017744+0000","last_fullsized":"2024-04-26T21:12:57.017744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017377+0000","last_change":"2024-04-26T21:12:54.842196+0000","last_active":"2024-04-26T21:12:57.017377+0000","last_peered":"2024-04-26T21:12:57.017377+0000","last_clean":"2024-04-26T21:12:57.017377+0000","last_became_active":"2024-04-26T21:12:54.841890+0000","last_became_peered":"2024-04-26T21:12:54.841890+0000","last_unstale":"2024-04-26T21:12:57.017377+0000","last_undegraded":"2024-04-26T21:12:57.017377+0000","last_fullsized":"2024-04-26T21:12:57.017377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:46.928170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018560+0000","last_change":"2024-04-26T21:12:55.829002+0000","last_active":"2024-04-26T21:12:57.018560+0000","last_peered":"2024-04-26T21:12:57.018560+0000","last_clean":"2024-04-26T21:12:57.018560+0000","last_became_active":"2024-04-26T21:12:55.828722+0000","last_became_peered":"2024-04-26T21:12:55.828722+0000","last_unstale":"2024-04-26T21:12:57.018560+0000","last_undegraded":"2024-04-26T21:12:57.018560+0000","last_fullsized":"2024-04-26T21:12:57.018560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.834952+0000","last_change":"2024-04-26T21:12:54.833670+0000","last_active":"2024-04-26T21:12:56.834952+0000","last_peered":"2024-04-26T21:12:56.834952+0000","last_clean":"2024-04-26T21:12:56.834952+0000","last_became_active":"2024-04-26T21:12:54.833382+0000","last_became_peered":"2024-04-26T21:12:54.833382+0000","last_unstale":"2024-04-26T21:12:56.834952+0000","last_undegraded":"2024-04-26T21:12:56.834952+0000","last_fullsized":"2024-04-26T21:12:56.834952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018043+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:57.018043+0000","last_peered":"2024-04-26T21:12:57.018043+0000","last_clean":"2024-04-26T21:12:57.018043+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:57.018043+0000","last_undegraded":"2024-04-26T21:12:57.018043+0000","last_fullsized":"2024-04-26T21:12:57.018043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":747537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":157149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:13:04.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:13:04.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.462+0000 7f6f7ebb7640 1 Processor -- start 2024-04-26T21:13:04.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.462+0000 7f6f7ebb7640 1 -- start start 2024-04-26T21:13:04.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7ebb7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7ebb7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f780d11b0 0x7f6f7806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7ebb7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f780d1740 con 0x7f6f7806e2c0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7ebb7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7806e750 con 0x7f6f780d11b0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7ebb7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780d1a20 con 0x7f6f780cf2e0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7c953640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f780d11b0 0x7f6f7806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f780d11b0 0x7f6f7806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:34328/0 (socket says 172.21.15.40:34328) 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7c953640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:60664/0 (socket says 172.21.15.40:60664) 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 -- 172.21.15.40:0/3271184710 learned_addr learned my addr 172.21.15.40:0/3271184710 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f7d154640 1 --1- 172.21.15.40:0/3271184710 >> v1:172.21.15.184:6789/0 conn(0x7f6f7806e2c0 0x7f6f7815ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:40050/0 (socket says 172.21.15.40:40050) 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 -- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 msgr2=0x7f6f7806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.463+0000 7f6f77fff640 1 -- 172.21.15.40:0/3271184710 >> v1:172.21.15.184:6789/0 conn(0x7f6f7806e2c0 legacy=0x7f6f7815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f77fff640 1 -- 172.21.15.40:0/3271184710 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f780d1d00 con 0x7f6f780d11b0 2024-04-26T21:13:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f7c953640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f77fff640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780d11b0 0x7f6f7806dd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009870 tx=0x7f6f6402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=179b90d86e1701f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f777fe640 1 -- 172.21.15.40:0/3271184710 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64037070 con 0x7f6f780d11b0 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f777fe640 1 -- 172.21.15.40:0/3271184710 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f64032440 con 0x7f6f780d11b0 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.464+0000 7f6f777fe640 1 -- 172.21.15.40:0/3271184710 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64032810 con 0x7f6f780d11b0 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780d11b0 msgr2=0x7f6f7806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780d11b0 0x7f6f7806dd80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009870 tx=0x7f6f6402a250 comp rx=0 tx=0).stop 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/3271184710 shutdown_connections 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780d11b0 0x7f6f7806dd80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/3271184710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f780cf2e0 0x7f6f7806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/3271184710 >> 172.21.15.40:0/3271184710 conn(0x7f6f780b4a60 msgr2=0x7f6f780b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/3271184710 shutdown_connections 2024-04-26T21:13:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.465+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/3271184710 wait complete. 2024-04-26T21:13:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.466+0000 7f6f7ebb7640 1 Processor -- start 2024-04-26T21:13:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.466+0000 7f6f7ebb7640 1 -- start start 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f7806e2c0 0x7f6f780c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780cf2e0 0x7f6f780b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7806da10 con 0x7f6f780d11b0 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780d22c0 con 0x7f6f780cf2e0 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7ebb7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780cf6c0 con 0x7f6f7806e2c0 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7d154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7d154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37668/0 (socket says 172.21.15.40:37668) 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.467+0000 7f6f7d154640 1 -- 172.21.15.40:0/2472158075 learned_addr learned my addr 172.21.15.40:0/2472158075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7c953640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f7806e2c0 0x7f6f780c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f77fff640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780cf2e0 0x7f6f780b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 -- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f7806e2c0 msgr2=0x7f6f780c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f7806e2c0 0x7f6f780c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 -- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780cf2e0 msgr2=0x7f6f780b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780cf2e0 0x7f6f780b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 -- 172.21.15.40:0/2472158075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f780b0ca0 con 0x7f6f780d11b0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7d154640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c780 tx=0x7f6f7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70017070 con 0x7f6f780d11b0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7000a640 con 0x7f6f780d11b0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000aa30 con 0x7f6f780d11b0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f780c7d00 con 0x7f6f780d11b0 2024-04-26T21:13:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.468+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f780c82c0 con 0x7f6f780d11b0 2024-04-26T21:13:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.469+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f78068e90 con 0x7f6f780d11b0 2024-04-26T21:13:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.470+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f7001a070 con 0x7f6f780d11b0 2024-04-26T21:13:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.470+0000 7f6f757fa640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.470+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f6f7008b070 con 0x7f6f780d11b0 2024-04-26T21:13:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.471+0000 7f6f7c953640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.471+0000 7f6f7c953640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009560 tx=0x7f6f6c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.475+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f78068e90 con 0x7f6f780d11b0 2024-04-26T21:13:04.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.655+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f7806a290 con 0x7f6f5c06c8f0 2024-04-26T21:13:04.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.662+0000 7f6f757fa640 1 -- 172.21.15.40:0/2472158075 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f6f7806a290 con 0x7f6f5c06c8f0 2024-04-26T21:13:04.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:04.666 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 msgr2=0x7f6f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009560 tx=0x7f6f6c009190 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 msgr2=0x7f6f7809e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c780 tx=0x7f6f7000cc40 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 shutdown_connections 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780d11b0 0x7f6f7809e250 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780cf2e0 0x7f6f780b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 --2- 172.21.15.40:0/2472158075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f7806e2c0 0x7f6f780c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 >> 172.21.15.40:0/2472158075 conn(0x7f6f780b4a60 msgr2=0x7f6f780b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 shutdown_connections 2024-04-26T21:13:04.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.664+0000 7f6f7ebb7640 1 -- 172.21.15.40:0/2472158075 wait complete. 2024-04-26T21:13:04.676 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-26T21:13:02.932896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220548,"kb_used_data":4360,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754172,"statfs":{"total":773094113280,"available":772868272128,"internally_reserved":0,"allocated":4464640,"data_stored":2442188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001735"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835219+0000","last_change":"2024-04-26T21:12:55.834874+0000","last_active":"2024-04-26T21:12:56.835219+0000","last_peered":"2024-04-26T21:12:56.835219+0000","last_clean":"2024-04-26T21:12:56.835219+0000","last_became_active":"2024-04-26T21:12:55.834593+0000","last_became_peered":"2024-04-26T21:12:55.834593+0000","last_unstale":"2024-04-26T21:12:56.835219+0000","last_undegraded":"2024-04-26T21:12:56.835219+0000","last_fullsized":"2024-04-26T21:12:56.835219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:30.540994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018250+0000","last_change":"2024-04-26T21:12:55.843688+0000","last_active":"2024-04-26T21:12:57.018250+0000","last_peered":"2024-04-26T21:12:57.018250+0000","last_clean":"2024-04-26T21:12:57.018250+0000","last_became_active":"2024-04-26T21:12:55.843298+0000","last_became_peered":"2024-04-26T21:12:55.843298+0000","last_unstale":"2024-04-26T21:12:57.018250+0000","last_undegraded":"2024-04-26T21:12:57.018250+0000","last_fullsized":"2024-04-26T21:12:57.018250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835869+0000","last_change":"2024-04-26T21:12:55.847059+0000","last_active":"2024-04-26T21:12:56.835869+0000","last_peered":"2024-04-26T21:12:56.835869+0000","last_clean":"2024-04-26T21:12:56.835869+0000","last_became_active":"2024-04-26T21:12:55.846609+0000","last_became_peered":"2024-04-26T21:12:55.846609+0000","last_unstale":"2024-04-26T21:12:56.835869+0000","last_undegraded":"2024-04-26T21:12:56.835869+0000","last_fullsized":"2024-04-26T21:12:56.835869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019004+0000","last_change":"2024-04-26T21:12:55.838731+0000","last_active":"2024-04-26T21:12:57.019004+0000","last_peered":"2024-04-26T21:12:57.019004+0000","last_clean":"2024-04-26T21:12:57.019004+0000","last_became_active":"2024-04-26T21:12:55.838444+0000","last_became_peered":"2024-04-26T21:12:55.838444+0000","last_unstale":"2024-04-26T21:12:57.019004+0000","last_undegraded":"2024-04-26T21:12:57.019004+0000","last_fullsized":"2024-04-26T21:12:57.019004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:54.237329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829415+0000","last_change":"2024-04-26T21:12:55.845666+0000","last_active":"2024-04-26T21:12:56.829415+0000","last_peered":"2024-04-26T21:12:56.829415+0000","last_clean":"2024-04-26T21:12:56.829415+0000","last_became_active":"2024-04-26T21:12:55.845327+0000","last_became_peered":"2024-04-26T21:12:55.845327+0000","last_unstale":"2024-04-26T21:12:56.829415+0000","last_undegraded":"2024-04-26T21:12:56.829415+0000","last_fullsized":"2024-04-26T21:12:56.829415+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:20.848985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018048+0000","last_change":"2024-04-26T21:12:55.843903+0000","last_active":"2024-04-26T21:12:57.018048+0000","last_peered":"2024-04-26T21:12:57.018048+0000","last_clean":"2024-04-26T21:12:57.018048+0000","last_became_active":"2024-04-26T21:12:55.842999+0000","last_became_peered":"2024-04-26T21:12:55.842999+0000","last_unstale":"2024-04-26T21:12:57.018048+0000","last_undegraded":"2024-04-26T21:12:57.018048+0000","last_fullsized":"2024-04-26T21:12:57.018048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018110+0000","last_change":"2024-04-26T21:12:55.829330+0000","last_active":"2024-04-26T21:12:57.018110+0000","last_peered":"2024-04-26T21:12:57.018110+0000","last_clean":"2024-04-26T21:12:57.018110+0000","last_became_active":"2024-04-26T21:12:55.828931+0000","last_became_peered":"2024-04-26T21:12:55.828931+0000","last_unstale":"2024-04-26T21:12:57.018110+0000","last_undegraded":"2024-04-26T21:12:57.018110+0000","last_fullsized":"2024-04-26T21:12:57.018110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.829652+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.829377+0000","last_became_peered":"2024-04-26T21:12:55.829377+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:29.407920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018385+0000","last_change":"2024-04-26T21:12:55.839415+0000","last_active":"2024-04-26T21:12:57.018385+0000","last_peered":"2024-04-26T21:12:57.018385+0000","last_clean":"2024-04-26T21:12:57.018385+0000","last_became_active":"2024-04-26T21:12:55.839122+0000","last_became_peered":"2024-04-26T21:12:55.839122+0000","last_unstale":"2024-04-26T21:12:57.018385+0000","last_undegraded":"2024-04-26T21:12:57.018385+0000","last_fullsized":"2024-04-26T21:12:57.018385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:01:28.305031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018096+0000","last_change":"2024-04-26T21:12:55.846118+0000","last_active":"2024-04-26T21:12:57.018096+0000","last_peered":"2024-04-26T21:12:57.018096+0000","last_clean":"2024-04-26T21:12:57.018096+0000","last_became_active":"2024-04-26T21:12:55.845845+0000","last_became_peered":"2024-04-26T21:12:55.845845+0000","last_unstale":"2024-04-26T21:12:57.018096+0000","last_undegraded":"2024-04-26T21:12:57.018096+0000","last_fullsized":"2024-04-26T21:12:57.018096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:10:25.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018636+0000","last_change":"2024-04-26T21:12:55.845593+0000","last_active":"2024-04-26T21:12:57.018636+0000","last_peered":"2024-04-26T21:12:57.018636+0000","last_clean":"2024-04-26T21:12:57.018636+0000","last_became_active":"2024-04-26T21:12:55.845268+0000","last_became_peered":"2024-04-26T21:12:55.845268+0000","last_unstale":"2024-04-26T21:12:57.018636+0000","last_undegraded":"2024-04-26T21:12:57.018636+0000","last_fullsized":"2024-04-26T21:12:57.018636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:11.595640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835098+0000","last_change":"2024-04-26T21:12:55.845017+0000","last_active":"2024-04-26T21:12:56.835098+0000","last_peered":"2024-04-26T21:12:56.835098+0000","last_clean":"2024-04-26T21:12:56.835098+0000","last_became_active":"2024-04-26T21:12:55.844675+0000","last_became_peered":"2024-04-26T21:12:55.844675+0000","last_unstale":"2024-04-26T21:12:56.835098+0000","last_undegraded":"2024-04-26T21:12:56.835098+0000","last_fullsized":"2024-04-26T21:12:56.835098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829449+0000","last_change":"2024-04-26T21:12:55.848662+0000","last_active":"2024-04-26T21:12:56.829449+0000","last_peered":"2024-04-26T21:12:56.829449+0000","last_clean":"2024-04-26T21:12:56.829449+0000","last_became_active":"2024-04-26T21:12:55.848361+0000","last_became_peered":"2024-04-26T21:12:55.848361+0000","last_unstale":"2024-04-26T21:12:56.829449+0000","last_undegraded":"2024-04-26T21:12:56.829449+0000","last_fullsized":"2024-04-26T21:12:56.829449+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:27.246380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018174+0000","last_change":"2024-04-26T21:12:55.851294+0000","last_active":"2024-04-26T21:12:57.018174+0000","last_peered":"2024-04-26T21:12:57.018174+0000","last_clean":"2024-04-26T21:12:57.018174+0000","last_became_active":"2024-04-26T21:12:55.850964+0000","last_became_peered":"2024-04-26T21:12:55.850964+0000","last_unstale":"2024-04-26T21:12:57.018174+0000","last_undegraded":"2024-04-26T21:12:57.018174+0000","last_fullsized":"2024-04-26T21:12:57.018174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835931+0000","last_change":"2024-04-26T21:12:55.827567+0000","last_active":"2024-04-26T21:12:56.835931+0000","last_peered":"2024-04-26T21:12:56.835931+0000","last_clean":"2024-04-26T21:12:56.835931+0000","last_became_active":"2024-04-26T21:12:55.827292+0000","last_became_peered":"2024-04-26T21:12:55.827292+0000","last_unstale":"2024-04-26T21:12:56.835931+0000","last_undegraded":"2024-04-26T21:12:56.835931+0000","last_fullsized":"2024-04-26T21:12:56.835931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017985+0000","last_change":"2024-04-26T21:12:55.848969+0000","last_active":"2024-04-26T21:12:57.017985+0000","last_peered":"2024-04-26T21:12:57.017985+0000","last_clean":"2024-04-26T21:12:57.017985+0000","last_became_active":"2024-04-26T21:12:55.848716+0000","last_became_peered":"2024-04-26T21:12:55.848716+0000","last_unstale":"2024-04-26T21:12:57.017985+0000","last_undegraded":"2024-04-26T21:12:57.017985+0000","last_fullsized":"2024-04-26T21:12:57.017985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:22.846822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018861+0000","last_change":"2024-04-26T21:12:55.834371+0000","last_active":"2024-04-26T21:12:57.018861+0000","last_peered":"2024-04-26T21:12:57.018861+0000","last_clean":"2024-04-26T21:12:57.018861+0000","last_became_active":"2024-04-26T21:12:55.834069+0000","last_became_peered":"2024-04-26T21:12:55.834069+0000","last_unstale":"2024-04-26T21:12:57.018861+0000","last_undegraded":"2024-04-26T21:12:57.018861+0000","last_fullsized":"2024-04-26T21:12:57.018861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018317+0000","last_change":"2024-04-26T21:12:55.852224+0000","last_active":"2024-04-26T21:12:57.018317+0000","last_peered":"2024-04-26T21:12:57.018317+0000","last_clean":"2024-04-26T21:12:57.018317+0000","last_became_active":"2024-04-26T21:12:55.851851+0000","last_became_peered":"2024-04-26T21:12:55.851851+0000","last_unstale":"2024-04-26T21:12:57.018317+0000","last_undegraded":"2024-04-26T21:12:57.018317+0000","last_fullsized":"2024-04-26T21:12:57.018317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:18.378959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017681+0000","last_change":"2024-04-26T21:12:55.852436+0000","last_active":"2024-04-26T21:12:57.017681+0000","last_peered":"2024-04-26T21:12:57.017681+0000","last_clean":"2024-04-26T21:12:57.017681+0000","last_became_active":"2024-04-26T21:12:55.852148+0000","last_became_peered":"2024-04-26T21:12:55.852148+0000","last_unstale":"2024-04-26T21:12:57.017681+0000","last_undegraded":"2024-04-26T21:12:57.017681+0000","last_fullsized":"2024-04-26T21:12:57.017681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829635+0000","last_change":"2024-04-26T21:12:55.833830+0000","last_active":"2024-04-26T21:12:56.829635+0000","last_peered":"2024-04-26T21:12:56.829635+0000","last_clean":"2024-04-26T21:12:56.829635+0000","last_became_active":"2024-04-26T21:12:55.833323+0000","last_became_peered":"2024-04-26T21:12:55.833323+0000","last_unstale":"2024-04-26T21:12:56.829635+0000","last_undegraded":"2024-04-26T21:12:56.829635+0000","last_fullsized":"2024-04-26T21:12:56.829635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:18.731112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018226+0000","last_change":"2024-04-26T21:12:55.831331+0000","last_active":"2024-04-26T21:12:57.018226+0000","last_peered":"2024-04-26T21:12:57.018226+0000","last_clean":"2024-04-26T21:12:57.018226+0000","last_became_active":"2024-04-26T21:12:55.830490+0000","last_became_peered":"2024-04-26T21:12:55.830490+0000","last_unstale":"2024-04-26T21:12:57.018226+0000","last_undegraded":"2024-04-26T21:12:57.018226+0000","last_fullsized":"2024-04-26T21:12:57.018226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:40.323141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018258+0000","last_change":"2024-04-26T21:12:55.848889+0000","last_active":"2024-04-26T21:12:57.018258+0000","last_peered":"2024-04-26T21:12:57.018258+0000","last_clean":"2024-04-26T21:12:57.018258+0000","last_became_active":"2024-04-26T21:12:55.848607+0000","last_became_peered":"2024-04-26T21:12:55.848607+0000","last_unstale":"2024-04-26T21:12:57.018258+0000","last_undegraded":"2024-04-26T21:12:57.018258+0000","last_fullsized":"2024-04-26T21:12:57.018258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.386686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017553+0000","last_change":"2024-04-26T21:12:55.834830+0000","last_active":"2024-04-26T21:12:57.017553+0000","last_peered":"2024-04-26T21:12:57.017553+0000","last_clean":"2024-04-26T21:12:57.017553+0000","last_became_active":"2024-04-26T21:12:55.834541+0000","last_became_peered":"2024-04-26T21:12:55.834541+0000","last_unstale":"2024-04-26T21:12:57.017553+0000","last_undegraded":"2024-04-26T21:12:57.017553+0000","last_fullsized":"2024-04-26T21:12:57.017553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835083+0000","last_change":"2024-04-26T21:12:55.829794+0000","last_active":"2024-04-26T21:12:56.835083+0000","last_peered":"2024-04-26T21:12:56.835083+0000","last_clean":"2024-04-26T21:12:56.835083+0000","last_became_active":"2024-04-26T21:12:55.829261+0000","last_became_peered":"2024-04-26T21:12:55.829261+0000","last_unstale":"2024-04-26T21:12:56.835083+0000","last_undegraded":"2024-04-26T21:12:56.835083+0000","last_fullsized":"2024-04-26T21:12:56.835083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018536+0000","last_change":"2024-04-26T21:12:55.839085+0000","last_active":"2024-04-26T21:12:57.018536+0000","last_peered":"2024-04-26T21:12:57.018536+0000","last_clean":"2024-04-26T21:12:57.018536+0000","last_became_active":"2024-04-26T21:12:55.838774+0000","last_became_peered":"2024-04-26T21:12:55.838774+0000","last_unstale":"2024-04-26T21:12:57.018536+0000","last_undegraded":"2024-04-26T21:12:57.018536+0000","last_fullsized":"2024-04-26T21:12:57.018536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:56.996573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018383+0000","last_change":"2024-04-26T21:12:55.847342+0000","last_active":"2024-04-26T21:12:57.018383+0000","last_peered":"2024-04-26T21:12:57.018383+0000","last_clean":"2024-04-26T21:12:57.018383+0000","last_became_active":"2024-04-26T21:12:55.846476+0000","last_became_peered":"2024-04-26T21:12:55.846476+0000","last_unstale":"2024-04-26T21:12:57.018383+0000","last_undegraded":"2024-04-26T21:12:57.018383+0000","last_fullsized":"2024-04-26T21:12:57.018383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:29.560745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018605+0000","last_change":"2024-04-26T21:12:55.842867+0000","last_active":"2024-04-26T21:12:57.018605+0000","last_peered":"2024-04-26T21:12:57.018605+0000","last_clean":"2024-04-26T21:12:57.018605+0000","last_became_active":"2024-04-26T21:12:55.842387+0000","last_became_peered":"2024-04-26T21:12:55.842387+0000","last_unstale":"2024-04-26T21:12:57.018605+0000","last_undegraded":"2024-04-26T21:12:57.018605+0000","last_fullsized":"2024-04-26T21:12:57.018605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:51.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835252+0000","last_change":"2024-04-26T21:12:55.833992+0000","last_active":"2024-04-26T21:12:56.835252+0000","last_peered":"2024-04-26T21:12:56.835252+0000","last_clean":"2024-04-26T21:12:56.835252+0000","last_became_active":"2024-04-26T21:12:55.833496+0000","last_became_peered":"2024-04-26T21:12:55.833496+0000","last_unstale":"2024-04-26T21:12:56.835252+0000","last_undegraded":"2024-04-26T21:12:56.835252+0000","last_fullsized":"2024-04-26T21:12:56.835252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018305+0000","last_change":"2024-04-26T21:12:55.846206+0000","last_active":"2024-04-26T21:12:57.018305+0000","last_peered":"2024-04-26T21:12:57.018305+0000","last_clean":"2024-04-26T21:12:57.018305+0000","last_became_active":"2024-04-26T21:12:55.845924+0000","last_became_peered":"2024-04-26T21:12:55.845924+0000","last_unstale":"2024-04-26T21:12:57.018305+0000","last_undegraded":"2024-04-26T21:12:57.018305+0000","last_fullsized":"2024-04-26T21:12:57.018305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:12.137923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018052+0000","last_change":"2024-04-26T21:12:55.833028+0000","last_active":"2024-04-26T21:12:57.018052+0000","last_peered":"2024-04-26T21:12:57.018052+0000","last_clean":"2024-04-26T21:12:57.018052+0000","last_became_active":"2024-04-26T21:12:55.832783+0000","last_became_peered":"2024-04-26T21:12:55.832783+0000","last_unstale":"2024-04-26T21:12:57.018052+0000","last_undegraded":"2024-04-26T21:12:57.018052+0000","last_fullsized":"2024-04-26T21:12:57.018052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835242+0000","last_change":"2024-04-26T21:12:55.829964+0000","last_active":"2024-04-26T21:12:56.835242+0000","last_peered":"2024-04-26T21:12:56.835242+0000","last_clean":"2024-04-26T21:12:56.835242+0000","last_became_active":"2024-04-26T21:12:55.828615+0000","last_became_peered":"2024-04-26T21:12:55.828615+0000","last_unstale":"2024-04-26T21:12:56.835242+0000","last_undegraded":"2024-04-26T21:12:56.835242+0000","last_fullsized":"2024-04-26T21:12:56.835242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:33.472157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017806+0000","last_change":"2024-04-26T21:12:55.835339+0000","last_active":"2024-04-26T21:12:57.017806+0000","last_peered":"2024-04-26T21:12:57.017806+0000","last_clean":"2024-04-26T21:12:57.017806+0000","last_became_active":"2024-04-26T21:12:55.834346+0000","last_became_peered":"2024-04-26T21:12:55.834346+0000","last_unstale":"2024-04-26T21:12:57.017806+0000","last_undegraded":"2024-04-26T21:12:57.017806+0000","last_fullsized":"2024-04-26T21:12:57.017806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:31.780527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017889+0000","last_change":"2024-04-26T21:12:55.835570+0000","last_active":"2024-04-26T21:12:57.017889+0000","last_peered":"2024-04-26T21:12:57.017889+0000","last_clean":"2024-04-26T21:12:57.017889+0000","last_became_active":"2024-04-26T21:12:55.835290+0000","last_became_peered":"2024-04-26T21:12:55.835290+0000","last_unstale":"2024-04-26T21:12:57.017889+0000","last_undegraded":"2024-04-26T21:12:57.017889+0000","last_fullsized":"2024-04-26T21:12:57.017889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835504+0000","last_change":"2024-04-26T21:12:54.836243+0000","last_active":"2024-04-26T21:12:56.835504+0000","last_peered":"2024-04-26T21:12:56.835504+0000","last_clean":"2024-04-26T21:12:56.835504+0000","last_became_active":"2024-04-26T21:12:54.835926+0000","last_became_peered":"2024-04-26T21:12:54.835926+0000","last_unstale":"2024-04-26T21:12:56.835504+0000","last_undegraded":"2024-04-26T21:12:56.835504+0000","last_fullsized":"2024-04-26T21:12:56.835504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835388+0000","last_change":"2024-04-26T21:12:55.833085+0000","last_active":"2024-04-26T21:12:56.835388+0000","last_peered":"2024-04-26T21:12:56.835388+0000","last_clean":"2024-04-26T21:12:56.835388+0000","last_became_active":"2024-04-26T21:12:55.832730+0000","last_became_peered":"2024-04-26T21:12:55.832730+0000","last_unstale":"2024-04-26T21:12:56.835388+0000","last_undegraded":"2024-04-26T21:12:56.835388+0000","last_fullsized":"2024-04-26T21:12:56.835388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018001+0000","last_change":"2024-04-26T21:12:54.828997+0000","last_active":"2024-04-26T21:12:57.018001+0000","last_peered":"2024-04-26T21:12:57.018001+0000","last_clean":"2024-04-26T21:12:57.018001+0000","last_became_active":"2024-04-26T21:12:54.828635+0000","last_became_peered":"2024-04-26T21:12:54.828635+0000","last_unstale":"2024-04-26T21:12:57.018001+0000","last_undegraded":"2024-04-26T21:12:57.018001+0000","last_fullsized":"2024-04-26T21:12:57.018001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018665+0000","last_change":"2024-04-26T21:12:55.846106+0000","last_active":"2024-04-26T21:12:57.018665+0000","last_peered":"2024-04-26T21:12:57.018665+0000","last_clean":"2024-04-26T21:12:57.018665+0000","last_became_active":"2024-04-26T21:12:55.845816+0000","last_became_peered":"2024-04-26T21:12:55.845816+0000","last_unstale":"2024-04-26T21:12:57.018665+0000","last_undegraded":"2024-04-26T21:12:57.018665+0000","last_fullsized":"2024-04-26T21:12:57.018665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:36:44.371719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017598+0000","last_change":"2024-04-26T21:12:54.841177+0000","last_active":"2024-04-26T21:12:57.017598+0000","last_peered":"2024-04-26T21:12:57.017598+0000","last_clean":"2024-04-26T21:12:57.017598+0000","last_became_active":"2024-04-26T21:12:54.840803+0000","last_became_peered":"2024-04-26T21:12:54.840803+0000","last_unstale":"2024-04-26T21:12:57.017598+0000","last_undegraded":"2024-04-26T21:12:57.017598+0000","last_fullsized":"2024-04-26T21:12:57.017598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:45.922188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018748+0000","last_change":"2024-04-26T21:12:55.842589+0000","last_active":"2024-04-26T21:12:57.018748+0000","last_peered":"2024-04-26T21:12:57.018748+0000","last_clean":"2024-04-26T21:12:57.018748+0000","last_became_active":"2024-04-26T21:12:55.841589+0000","last_became_peered":"2024-04-26T21:12:55.841589+0000","last_unstale":"2024-04-26T21:12:57.018748+0000","last_undegraded":"2024-04-26T21:12:57.018748+0000","last_fullsized":"2024-04-26T21:12:57.018748+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:57.131512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829652+0000","last_change":"2024-04-26T21:12:54.833964+0000","last_active":"2024-04-26T21:12:56.829652+0000","last_peered":"2024-04-26T21:12:56.829652+0000","last_clean":"2024-04-26T21:12:56.829652+0000","last_became_active":"2024-04-26T21:12:54.833662+0000","last_became_peered":"2024-04-26T21:12:54.833662+0000","last_unstale":"2024-04-26T21:12:56.829652+0000","last_undegraded":"2024-04-26T21:12:56.829652+0000","last_fullsized":"2024-04-26T21:12:56.829652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:11.073082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018468+0000","last_change":"2024-04-26T21:12:55.839174+0000","last_active":"2024-04-26T21:12:57.018468+0000","last_peered":"2024-04-26T21:12:57.018468+0000","last_clean":"2024-04-26T21:12:57.018468+0000","last_became_active":"2024-04-26T21:12:55.838902+0000","last_became_peered":"2024-04-26T21:12:55.838902+0000","last_unstale":"2024-04-26T21:12:57.018468+0000","last_undegraded":"2024-04-26T21:12:57.018468+0000","last_fullsized":"2024-04-26T21:12:57.018468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018348+0000","last_change":"2024-04-26T21:12:54.838674+0000","last_active":"2024-04-26T21:12:57.018348+0000","last_peered":"2024-04-26T21:12:57.018348+0000","last_clean":"2024-04-26T21:12:57.018348+0000","last_became_active":"2024-04-26T21:12:54.838357+0000","last_became_peered":"2024-04-26T21:12:54.838357+0000","last_unstale":"2024-04-26T21:12:57.018348+0000","last_undegraded":"2024-04-26T21:12:57.018348+0000","last_fullsized":"2024-04-26T21:12:57.018348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835420+0000","last_change":"2024-04-26T21:12:55.833557+0000","last_active":"2024-04-26T21:12:56.835420+0000","last_peered":"2024-04-26T21:12:56.835420+0000","last_clean":"2024-04-26T21:12:56.835420+0000","last_became_active":"2024-04-26T21:12:55.833295+0000","last_became_peered":"2024-04-26T21:12:55.833295+0000","last_unstale":"2024-04-26T21:12:56.835420+0000","last_undegraded":"2024-04-26T21:12:56.835420+0000","last_fullsized":"2024-04-26T21:12:56.835420+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:13.173859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018262+0000","last_change":"2024-04-26T21:12:54.837240+0000","last_active":"2024-04-26T21:12:57.018262+0000","last_peered":"2024-04-26T21:12:57.018262+0000","last_clean":"2024-04-26T21:12:57.018262+0000","last_became_active":"2024-04-26T21:12:54.837022+0000","last_became_peered":"2024-04-26T21:12:54.837022+0000","last_unstale":"2024-04-26T21:12:57.018262+0000","last_undegraded":"2024-04-26T21:12:57.018262+0000","last_fullsized":"2024-04-26T21:12:57.018262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:20.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017917+0000","last_change":"2024-04-26T21:12:55.828118+0000","last_active":"2024-04-26T21:12:57.017917+0000","last_peered":"2024-04-26T21:12:57.017917+0000","last_clean":"2024-04-26T21:12:57.017917+0000","last_became_active":"2024-04-26T21:12:55.827805+0000","last_became_peered":"2024-04-26T21:12:55.827805+0000","last_unstale":"2024-04-26T21:12:57.017917+0000","last_undegraded":"2024-04-26T21:12:57.017917+0000","last_fullsized":"2024-04-26T21:12:57.017917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017693+0000","last_change":"2024-04-26T21:12:54.835535+0000","last_active":"2024-04-26T21:12:57.017693+0000","last_peered":"2024-04-26T21:12:57.017693+0000","last_clean":"2024-04-26T21:12:57.017693+0000","last_became_active":"2024-04-26T21:12:54.835258+0000","last_became_peered":"2024-04-26T21:12:54.835258+0000","last_unstale":"2024-04-26T21:12:57.017693+0000","last_undegraded":"2024-04-26T21:12:57.017693+0000","last_fullsized":"2024-04-26T21:12:57.017693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017840+0000","last_change":"2024-04-26T21:12:55.840788+0000","last_active":"2024-04-26T21:12:57.017840+0000","last_peered":"2024-04-26T21:12:57.017840+0000","last_clean":"2024-04-26T21:12:57.017840+0000","last_became_active":"2024-04-26T21:12:55.840492+0000","last_became_peered":"2024-04-26T21:12:55.840492+0000","last_unstale":"2024-04-26T21:12:57.017840+0000","last_undegraded":"2024-04-26T21:12:57.017840+0000","last_fullsized":"2024-04-26T21:12:57.017840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:46.344935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017760+0000","last_change":"2024-04-26T21:12:54.833665+0000","last_active":"2024-04-26T21:12:57.017760+0000","last_peered":"2024-04-26T21:12:57.017760+0000","last_clean":"2024-04-26T21:12:57.017760+0000","last_became_active":"2024-04-26T21:12:54.833394+0000","last_became_peered":"2024-04-26T21:12:54.833394+0000","last_unstale":"2024-04-26T21:12:57.017760+0000","last_undegraded":"2024-04-26T21:12:57.017760+0000","last_fullsized":"2024-04-26T21:12:57.017760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:18.903622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.836233+0000","last_change":"2024-04-26T21:12:55.844791+0000","last_active":"2024-04-26T21:12:56.836233+0000","last_peered":"2024-04-26T21:12:56.836233+0000","last_clean":"2024-04-26T21:12:56.836233+0000","last_became_active":"2024-04-26T21:12:55.844567+0000","last_became_peered":"2024-04-26T21:12:55.844567+0000","last_unstale":"2024-04-26T21:12:56.836233+0000","last_undegraded":"2024-04-26T21:12:56.836233+0000","last_fullsized":"2024-04-26T21:12:56.836233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:31:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017815+0000","last_change":"2024-04-26T21:12:54.842287+0000","last_active":"2024-04-26T21:12:57.017815+0000","last_peered":"2024-04-26T21:12:57.017815+0000","last_clean":"2024-04-26T21:12:57.017815+0000","last_became_active":"2024-04-26T21:12:54.841969+0000","last_became_peered":"2024-04-26T21:12:54.841969+0000","last_unstale":"2024-04-26T21:12:57.017815+0000","last_undegraded":"2024-04-26T21:12:57.017815+0000","last_fullsized":"2024-04-26T21:12:57.017815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:06.740507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017709+0000","last_change":"2024-04-26T21:12:55.852140+0000","last_active":"2024-04-26T21:12:57.017709+0000","last_peered":"2024-04-26T21:12:57.017709+0000","last_clean":"2024-04-26T21:12:57.017709+0000","last_became_active":"2024-04-26T21:12:55.851753+0000","last_became_peered":"2024-04-26T21:12:55.851753+0000","last_unstale":"2024-04-26T21:12:57.017709+0000","last_undegraded":"2024-04-26T21:12:57.017709+0000","last_fullsized":"2024-04-26T21:12:57.017709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:54.832871+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:54.832587+0000","last_became_peered":"2024-04-26T21:12:54.832587+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017990+0000","last_change":"2024-04-26T21:12:55.851660+0000","last_active":"2024-04-26T21:12:57.017990+0000","last_peered":"2024-04-26T21:12:57.017990+0000","last_clean":"2024-04-26T21:12:57.017990+0000","last_became_active":"2024-04-26T21:12:55.850979+0000","last_became_peered":"2024-04-26T21:12:55.850979+0000","last_unstale":"2024-04-26T21:12:57.017990+0000","last_undegraded":"2024-04-26T21:12:57.017990+0000","last_fullsized":"2024-04-26T21:12:57.017990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835911+0000","last_change":"2024-04-26T21:12:54.830694+0000","last_active":"2024-04-26T21:12:56.835911+0000","last_peered":"2024-04-26T21:12:56.835911+0000","last_clean":"2024-04-26T21:12:56.835911+0000","last_became_active":"2024-04-26T21:12:54.830373+0000","last_became_peered":"2024-04-26T21:12:54.830373+0000","last_unstale":"2024-04-26T21:12:56.835911+0000","last_undegraded":"2024-04-26T21:12:56.835911+0000","last_fullsized":"2024-04-26T21:12:56.835911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835683+0000","last_change":"2024-04-26T21:12:55.846932+0000","last_active":"2024-04-26T21:12:56.835683+0000","last_peered":"2024-04-26T21:12:56.835683+0000","last_clean":"2024-04-26T21:12:56.835683+0000","last_became_active":"2024-04-26T21:12:55.846616+0000","last_became_peered":"2024-04-26T21:12:55.846616+0000","last_unstale":"2024-04-26T21:12:56.835683+0000","last_undegraded":"2024-04-26T21:12:56.835683+0000","last_fullsized":"2024-04-26T21:12:56.835683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017971+0000","last_change":"2024-04-26T21:12:54.840841+0000","last_active":"2024-04-26T21:12:57.017971+0000","last_peered":"2024-04-26T21:12:57.017971+0000","last_clean":"2024-04-26T21:12:57.017971+0000","last_became_active":"2024-04-26T21:12:54.840336+0000","last_became_peered":"2024-04-26T21:12:54.840336+0000","last_unstale":"2024-04-26T21:12:57.017971+0000","last_undegraded":"2024-04-26T21:12:57.017971+0000","last_fullsized":"2024-04-26T21:12:57.017971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:17.058129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018332+0000","last_change":"2024-04-26T21:12:55.843140+0000","last_active":"2024-04-26T21:12:57.018332+0000","last_peered":"2024-04-26T21:12:57.018332+0000","last_clean":"2024-04-26T21:12:57.018332+0000","last_became_active":"2024-04-26T21:12:55.842833+0000","last_became_peered":"2024-04-26T21:12:55.842833+0000","last_unstale":"2024-04-26T21:12:57.018332+0000","last_undegraded":"2024-04-26T21:12:57.018332+0000","last_fullsized":"2024-04-26T21:12:57.018332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:46.288609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018411+0000","last_change":"2024-04-26T21:12:54.833969+0000","last_active":"2024-04-26T21:12:57.018411+0000","last_peered":"2024-04-26T21:12:57.018411+0000","last_clean":"2024-04-26T21:12:57.018411+0000","last_became_active":"2024-04-26T21:12:54.833628+0000","last_became_peered":"2024-04-26T21:12:54.833628+0000","last_unstale":"2024-04-26T21:12:57.018411+0000","last_undegraded":"2024-04-26T21:12:57.018411+0000","last_fullsized":"2024-04-26T21:12:57.018411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017791+0000","last_change":"2024-04-26T21:12:55.844863+0000","last_active":"2024-04-26T21:12:57.017791+0000","last_peered":"2024-04-26T21:12:57.017791+0000","last_clean":"2024-04-26T21:12:57.017791+0000","last_became_active":"2024-04-26T21:12:55.844486+0000","last_became_peered":"2024-04-26T21:12:55.844486+0000","last_unstale":"2024-04-26T21:12:57.017791+0000","last_undegraded":"2024-04-26T21:12:57.017791+0000","last_fullsized":"2024-04-26T21:12:57.017791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:23.328333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018263+0000","last_change":"2024-04-26T21:12:54.831096+0000","last_active":"2024-04-26T21:12:57.018263+0000","last_peered":"2024-04-26T21:12:57.018263+0000","last_clean":"2024-04-26T21:12:57.018263+0000","last_became_active":"2024-04-26T21:12:54.830785+0000","last_became_peered":"2024-04-26T21:12:54.830785+0000","last_unstale":"2024-04-26T21:12:57.018263+0000","last_undegraded":"2024-04-26T21:12:57.018263+0000","last_fullsized":"2024-04-26T21:12:57.018263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:45.252155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018645+0000","last_change":"2024-04-26T21:12:55.834991+0000","last_active":"2024-04-26T21:12:57.018645+0000","last_peered":"2024-04-26T21:12:57.018645+0000","last_clean":"2024-04-26T21:12:57.018645+0000","last_became_active":"2024-04-26T21:12:55.834663+0000","last_became_peered":"2024-04-26T21:12:55.834663+0000","last_unstale":"2024-04-26T21:12:57.018645+0000","last_undegraded":"2024-04-26T21:12:57.018645+0000","last_fullsized":"2024-04-26T21:12:57.018645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:03.521134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017474+0000","last_change":"2024-04-26T21:12:54.842208+0000","last_active":"2024-04-26T21:12:57.017474+0000","last_peered":"2024-04-26T21:12:57.017474+0000","last_clean":"2024-04-26T21:12:57.017474+0000","last_became_active":"2024-04-26T21:12:54.841932+0000","last_became_peered":"2024-04-26T21:12:54.841932+0000","last_unstale":"2024-04-26T21:12:57.017474+0000","last_undegraded":"2024-04-26T21:12:57.017474+0000","last_fullsized":"2024-04-26T21:12:57.017474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:29.534664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835302+0000","last_change":"2024-04-26T21:12:55.830389+0000","last_active":"2024-04-26T21:12:56.835302+0000","last_peered":"2024-04-26T21:12:56.835302+0000","last_clean":"2024-04-26T21:12:56.835302+0000","last_became_active":"2024-04-26T21:12:55.830080+0000","last_became_peered":"2024-04-26T21:12:55.830080+0000","last_unstale":"2024-04-26T21:12:56.835302+0000","last_undegraded":"2024-04-26T21:12:56.835302+0000","last_fullsized":"2024-04-26T21:12:56.835302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:18.064459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017777+0000","last_change":"2024-04-26T21:12:54.833724+0000","last_active":"2024-04-26T21:12:57.017777+0000","last_peered":"2024-04-26T21:12:57.017777+0000","last_clean":"2024-04-26T21:12:57.017777+0000","last_became_active":"2024-04-26T21:12:54.833458+0000","last_became_peered":"2024-04-26T21:12:54.833458+0000","last_unstale":"2024-04-26T21:12:57.017777+0000","last_undegraded":"2024-04-26T21:12:57.017777+0000","last_fullsized":"2024-04-26T21:12:57.017777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:26.631042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017998+0000","last_change":"2024-04-26T21:12:55.835158+0000","last_active":"2024-04-26T21:12:57.017998+0000","last_peered":"2024-04-26T21:12:57.017998+0000","last_clean":"2024-04-26T21:12:57.017998+0000","last_became_active":"2024-04-26T21:12:55.834796+0000","last_became_peered":"2024-04-26T21:12:55.834796+0000","last_unstale":"2024-04-26T21:12:57.017998+0000","last_undegraded":"2024-04-26T21:12:57.017998+0000","last_fullsized":"2024-04-26T21:12:57.017998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:33.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018703+0000","last_change":"2024-04-26T21:12:54.837631+0000","last_active":"2024-04-26T21:12:57.018703+0000","last_peered":"2024-04-26T21:12:57.018703+0000","last_clean":"2024-04-26T21:12:57.018703+0000","last_became_active":"2024-04-26T21:12:54.837117+0000","last_became_peered":"2024-04-26T21:12:54.837117+0000","last_unstale":"2024-04-26T21:12:57.018703+0000","last_undegraded":"2024-04-26T21:12:57.018703+0000","last_fullsized":"2024-04-26T21:12:57.018703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:45.775778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835652+0000","last_change":"2024-04-26T21:12:55.837208+0000","last_active":"2024-04-26T21:12:56.835652+0000","last_peered":"2024-04-26T21:12:56.835652+0000","last_clean":"2024-04-26T21:12:56.835652+0000","last_became_active":"2024-04-26T21:12:55.836934+0000","last_became_peered":"2024-04-26T21:12:55.836934+0000","last_unstale":"2024-04-26T21:12:56.835652+0000","last_undegraded":"2024-04-26T21:12:56.835652+0000","last_fullsized":"2024-04-26T21:12:56.835652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018106+0000","last_change":"2024-04-26T21:12:54.833728+0000","last_active":"2024-04-26T21:12:57.018106+0000","last_peered":"2024-04-26T21:12:57.018106+0000","last_clean":"2024-04-26T21:12:57.018106+0000","last_became_active":"2024-04-26T21:12:54.833388+0000","last_became_peered":"2024-04-26T21:12:54.833388+0000","last_unstale":"2024-04-26T21:12:57.018106+0000","last_undegraded":"2024-04-26T21:12:57.018106+0000","last_fullsized":"2024-04-26T21:12:57.018106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019012+0000","last_change":"2024-04-26T21:12:55.838065+0000","last_active":"2024-04-26T21:12:57.019012+0000","last_peered":"2024-04-26T21:12:57.019012+0000","last_clean":"2024-04-26T21:12:57.019012+0000","last_became_active":"2024-04-26T21:12:55.837784+0000","last_became_peered":"2024-04-26T21:12:55.837784+0000","last_unstale":"2024-04-26T21:12:57.019012+0000","last_undegraded":"2024-04-26T21:12:57.019012+0000","last_fullsized":"2024-04-26T21:12:57.019012+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:13.088171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018083+0000","last_change":"2024-04-26T21:12:54.832422+0000","last_active":"2024-04-26T21:12:57.018083+0000","last_peered":"2024-04-26T21:12:57.018083+0000","last_clean":"2024-04-26T21:12:57.018083+0000","last_became_active":"2024-04-26T21:12:54.832171+0000","last_became_peered":"2024-04-26T21:12:54.832171+0000","last_unstale":"2024-04-26T21:12:57.018083+0000","last_undegraded":"2024-04-26T21:12:57.018083+0000","last_fullsized":"2024-04-26T21:12:57.018083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:33.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829689+0000","last_change":"2024-04-26T21:12:55.840206+0000","last_active":"2024-04-26T21:12:56.829689+0000","last_peered":"2024-04-26T21:12:56.829689+0000","last_clean":"2024-04-26T21:12:56.829689+0000","last_became_active":"2024-04-26T21:12:55.839915+0000","last_became_peered":"2024-04-26T21:12:55.839915+0000","last_unstale":"2024-04-26T21:12:56.829689+0000","last_undegraded":"2024-04-26T21:12:56.829689+0000","last_fullsized":"2024-04-26T21:12:56.829689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:47.934501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018739+0000","last_change":"2024-04-26T21:12:54.838655+0000","last_active":"2024-04-26T21:12:57.018739+0000","last_peered":"2024-04-26T21:12:57.018739+0000","last_clean":"2024-04-26T21:12:57.018739+0000","last_became_active":"2024-04-26T21:12:54.838244+0000","last_became_peered":"2024-04-26T21:12:54.838244+0000","last_unstale":"2024-04-26T21:12:57.018739+0000","last_undegraded":"2024-04-26T21:12:57.018739+0000","last_fullsized":"2024-04-26T21:12:57.018739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:26.055217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.834792+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.834492+0000","last_became_peered":"2024-04-26T21:12:55.834492+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:12.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018066+0000","last_change":"2024-04-26T21:12:54.834125+0000","last_active":"2024-04-26T21:12:57.018066+0000","last_peered":"2024-04-26T21:12:57.018066+0000","last_clean":"2024-04-26T21:12:57.018066+0000","last_became_active":"2024-04-26T21:12:54.833830+0000","last_became_peered":"2024-04-26T21:12:54.833830+0000","last_unstale":"2024-04-26T21:12:57.018066+0000","last_undegraded":"2024-04-26T21:12:57.018066+0000","last_fullsized":"2024-04-26T21:12:57.018066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:17.724781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829576+0000","last_change":"2024-04-26T21:12:55.829614+0000","last_active":"2024-04-26T21:12:56.829576+0000","last_peered":"2024-04-26T21:12:56.829576+0000","last_clean":"2024-04-26T21:12:56.829576+0000","last_became_active":"2024-04-26T21:12:55.829102+0000","last_became_peered":"2024-04-26T21:12:55.829102+0000","last_unstale":"2024-04-26T21:12:56.829576+0000","last_undegraded":"2024-04-26T21:12:56.829576+0000","last_fullsized":"2024-04-26T21:12:56.829576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:14.539573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835517+0000","last_change":"2024-04-26T21:12:54.833584+0000","last_active":"2024-04-26T21:12:56.835517+0000","last_peered":"2024-04-26T21:12:56.835517+0000","last_clean":"2024-04-26T21:12:56.835517+0000","last_became_active":"2024-04-26T21:12:54.833302+0000","last_became_peered":"2024-04-26T21:12:54.833302+0000","last_unstale":"2024-04-26T21:12:56.835517+0000","last_undegraded":"2024-04-26T21:12:56.835517+0000","last_fullsized":"2024-04-26T21:12:56.835517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:26.240049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829386+0000","last_change":"2024-04-26T21:12:55.829378+0000","last_active":"2024-04-26T21:12:56.829386+0000","last_peered":"2024-04-26T21:12:56.829386+0000","last_clean":"2024-04-26T21:12:56.829386+0000","last_became_active":"2024-04-26T21:12:55.829065+0000","last_became_peered":"2024-04-26T21:12:55.829065+0000","last_unstale":"2024-04-26T21:12:56.829386+0000","last_undegraded":"2024-04-26T21:12:56.829386+0000","last_fullsized":"2024-04-26T21:12:56.829386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:01.592713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017725+0000","last_change":"2024-04-26T21:12:54.834382+0000","last_active":"2024-04-26T21:12:57.017725+0000","last_peered":"2024-04-26T21:12:57.017725+0000","last_clean":"2024-04-26T21:12:57.017725+0000","last_became_active":"2024-04-26T21:12:54.833899+0000","last_became_peered":"2024-04-26T21:12:54.833899+0000","last_unstale":"2024-04-26T21:12:57.017725+0000","last_undegraded":"2024-04-26T21:12:57.017725+0000","last_fullsized":"2024-04-26T21:12:57.017725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:11.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018255+0000","last_change":"2024-04-26T21:12:55.831832+0000","last_active":"2024-04-26T21:12:57.018255+0000","last_peered":"2024-04-26T21:12:57.018255+0000","last_clean":"2024-04-26T21:12:57.018255+0000","last_became_active":"2024-04-26T21:12:55.831550+0000","last_became_peered":"2024-04-26T21:12:55.831550+0000","last_unstale":"2024-04-26T21:12:57.018255+0000","last_undegraded":"2024-04-26T21:12:57.018255+0000","last_fullsized":"2024-04-26T21:12:57.018255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:39.343507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835186+0000","last_change":"2024-04-26T21:12:54.830666+0000","last_active":"2024-04-26T21:12:56.835186+0000","last_peered":"2024-04-26T21:12:56.835186+0000","last_clean":"2024-04-26T21:12:56.835186+0000","last_became_active":"2024-04-26T21:12:54.830338+0000","last_became_peered":"2024-04-26T21:12:54.830338+0000","last_unstale":"2024-04-26T21:12:56.835186+0000","last_undegraded":"2024-04-26T21:12:56.835186+0000","last_fullsized":"2024-04-26T21:12:56.835186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:32.465827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.829680+0000","last_change":"2024-04-26T21:12:55.834082+0000","last_active":"2024-04-26T21:12:56.829680+0000","last_peered":"2024-04-26T21:12:56.829680+0000","last_clean":"2024-04-26T21:12:56.829680+0000","last_became_active":"2024-04-26T21:12:55.833778+0000","last_became_peered":"2024-04-26T21:12:55.833778+0000","last_unstale":"2024-04-26T21:12:56.829680+0000","last_undegraded":"2024-04-26T21:12:56.829680+0000","last_fullsized":"2024-04-26T21:12:56.829680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:57.707945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018803+0000","last_change":"2024-04-26T21:12:54.837054+0000","last_active":"2024-04-26T21:12:57.018803+0000","last_peered":"2024-04-26T21:12:57.018803+0000","last_clean":"2024-04-26T21:12:57.018803+0000","last_became_active":"2024-04-26T21:12:54.836690+0000","last_became_peered":"2024-04-26T21:12:54.836690+0000","last_unstale":"2024-04-26T21:12:57.018803+0000","last_undegraded":"2024-04-26T21:12:57.018803+0000","last_fullsized":"2024-04-26T21:12:57.018803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:18.816955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835819+0000","last_change":"2024-04-26T21:12:55.840516+0000","last_active":"2024-04-26T21:12:56.835819+0000","last_peered":"2024-04-26T21:12:56.835819+0000","last_clean":"2024-04-26T21:12:56.835819+0000","last_became_active":"2024-04-26T21:12:55.840212+0000","last_became_peered":"2024-04-26T21:12:55.840212+0000","last_unstale":"2024-04-26T21:12:56.835819+0000","last_undegraded":"2024-04-26T21:12:56.835819+0000","last_fullsized":"2024-04-26T21:12:56.835819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018109+0000","last_change":"2024-04-26T21:12:54.834650+0000","last_active":"2024-04-26T21:12:57.018109+0000","last_peered":"2024-04-26T21:12:57.018109+0000","last_clean":"2024-04-26T21:12:57.018109+0000","last_became_active":"2024-04-26T21:12:54.834365+0000","last_became_peered":"2024-04-26T21:12:54.834365+0000","last_unstale":"2024-04-26T21:12:57.018109+0000","last_undegraded":"2024-04-26T21:12:57.018109+0000","last_fullsized":"2024-04-26T21:12:57.018109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:13.533243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017612+0000","last_change":"2024-04-26T21:12:55.846844+0000","last_active":"2024-04-26T21:12:57.017612+0000","last_peered":"2024-04-26T21:12:57.017612+0000","last_clean":"2024-04-26T21:12:57.017612+0000","last_became_active":"2024-04-26T21:12:55.846587+0000","last_became_peered":"2024-04-26T21:12:55.846587+0000","last_unstale":"2024-04-26T21:12:57.017612+0000","last_undegraded":"2024-04-26T21:12:57.017612+0000","last_fullsized":"2024-04-26T21:12:57.017612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:07.746837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.019113+0000","last_change":"2024-04-26T21:12:54.833945+0000","last_active":"2024-04-26T21:12:57.019113+0000","last_peered":"2024-04-26T21:12:57.019113+0000","last_clean":"2024-04-26T21:12:57.019113+0000","last_became_active":"2024-04-26T21:12:54.833186+0000","last_became_peered":"2024-04-26T21:12:54.833186+0000","last_unstale":"2024-04-26T21:12:57.019113+0000","last_undegraded":"2024-04-26T21:12:57.019113+0000","last_fullsized":"2024-04-26T21:12:57.019113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:10.115123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.835471+0000","last_change":"2024-04-26T21:12:55.831340+0000","last_active":"2024-04-26T21:12:56.835471+0000","last_peered":"2024-04-26T21:12:56.835471+0000","last_clean":"2024-04-26T21:12:56.835471+0000","last_became_active":"2024-04-26T21:12:55.831068+0000","last_became_peered":"2024-04-26T21:12:55.831068+0000","last_unstale":"2024-04-26T21:12:56.835471+0000","last_undegraded":"2024-04-26T21:12:56.835471+0000","last_fullsized":"2024-04-26T21:12:56.835471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018242+0000","last_change":"2024-04-26T21:12:54.825357+0000","last_active":"2024-04-26T21:12:57.018242+0000","last_peered":"2024-04-26T21:12:57.018242+0000","last_clean":"2024-04-26T21:12:57.018242+0000","last_became_active":"2024-04-26T21:12:54.824948+0000","last_became_peered":"2024-04-26T21:12:54.824948+0000","last_unstale":"2024-04-26T21:12:57.018242+0000","last_undegraded":"2024-04-26T21:12:57.018242+0000","last_fullsized":"2024-04-26T21:12:57.018242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:56.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017961+0000","last_change":"2024-04-26T21:12:55.832196+0000","last_active":"2024-04-26T21:12:57.017961+0000","last_peered":"2024-04-26T21:12:57.017961+0000","last_clean":"2024-04-26T21:12:57.017961+0000","last_became_active":"2024-04-26T21:12:55.831921+0000","last_became_peered":"2024-04-26T21:12:55.831921+0000","last_unstale":"2024-04-26T21:12:57.017961+0000","last_undegraded":"2024-04-26T21:12:57.017961+0000","last_fullsized":"2024-04-26T21:12:57.017961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:51.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018382+0000","last_change":"2024-04-26T21:12:54.833203+0000","last_active":"2024-04-26T21:12:57.018382+0000","last_peered":"2024-04-26T21:12:57.018382+0000","last_clean":"2024-04-26T21:12:57.018382+0000","last_became_active":"2024-04-26T21:12:54.832884+0000","last_became_peered":"2024-04-26T21:12:54.832884+0000","last_unstale":"2024-04-26T21:12:57.018382+0000","last_undegraded":"2024-04-26T21:12:57.018382+0000","last_fullsized":"2024-04-26T21:12:57.018382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:36.900099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017397+0000","last_change":"2024-04-26T21:12:55.848208+0000","last_active":"2024-04-26T21:12:57.017397+0000","last_peered":"2024-04-26T21:12:57.017397+0000","last_clean":"2024-04-26T21:12:57.017397+0000","last_became_active":"2024-04-26T21:12:55.847911+0000","last_became_peered":"2024-04-26T21:12:55.847911+0000","last_unstale":"2024-04-26T21:12:57.017397+0000","last_undegraded":"2024-04-26T21:12:57.017397+0000","last_fullsized":"2024-04-26T21:12:57.017397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:58.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018179+0000","last_change":"2024-04-26T21:12:54.829043+0000","last_active":"2024-04-26T21:12:57.018179+0000","last_peered":"2024-04-26T21:12:57.018179+0000","last_clean":"2024-04-26T21:12:57.018179+0000","last_became_active":"2024-04-26T21:12:54.828752+0000","last_became_peered":"2024-04-26T21:12:54.828752+0000","last_unstale":"2024-04-26T21:12:57.018179+0000","last_undegraded":"2024-04-26T21:12:57.018179+0000","last_fullsized":"2024-04-26T21:12:57.018179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:00.586383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017744+0000","last_change":"2024-04-26T21:12:55.846026+0000","last_active":"2024-04-26T21:12:57.017744+0000","last_peered":"2024-04-26T21:12:57.017744+0000","last_clean":"2024-04-26T21:12:57.017744+0000","last_became_active":"2024-04-26T21:12:55.845739+0000","last_became_peered":"2024-04-26T21:12:55.845739+0000","last_unstale":"2024-04-26T21:12:57.017744+0000","last_undegraded":"2024-04-26T21:12:57.017744+0000","last_fullsized":"2024-04-26T21:12:57.017744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:10:46.928518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.017377+0000","last_change":"2024-04-26T21:12:54.842196+0000","last_active":"2024-04-26T21:12:57.017377+0000","last_peered":"2024-04-26T21:12:57.017377+0000","last_clean":"2024-04-26T21:12:57.017377+0000","last_became_active":"2024-04-26T21:12:54.841890+0000","last_became_peered":"2024-04-26T21:12:54.841890+0000","last_unstale":"2024-04-26T21:12:57.017377+0000","last_undegraded":"2024-04-26T21:12:57.017377+0000","last_fullsized":"2024-04-26T21:12:57.017377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:46.928170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018560+0000","last_change":"2024-04-26T21:12:55.829002+0000","last_active":"2024-04-26T21:12:57.018560+0000","last_peered":"2024-04-26T21:12:57.018560+0000","last_clean":"2024-04-26T21:12:57.018560+0000","last_became_active":"2024-04-26T21:12:55.828722+0000","last_became_peered":"2024-04-26T21:12:55.828722+0000","last_unstale":"2024-04-26T21:12:57.018560+0000","last_undegraded":"2024-04-26T21:12:57.018560+0000","last_fullsized":"2024-04-26T21:12:57.018560+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:54.809395+0000","last_clean_scrub_stamp":"2024-04-26T21:12:54.809395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.651336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:56.834952+0000","last_change":"2024-04-26T21:12:54.833670+0000","last_active":"2024-04-26T21:12:56.834952+0000","last_peered":"2024-04-26T21:12:56.834952+0000","last_clean":"2024-04-26T21:12:56.834952+0000","last_became_active":"2024-04-26T21:12:54.833382+0000","last_became_peered":"2024-04-26T21:12:54.833382+0000","last_unstale":"2024-04-26T21:12:56.834952+0000","last_undegraded":"2024-04-26T21:12:56.834952+0000","last_fullsized":"2024-04-26T21:12:56.834952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:12:53.803064+0000","last_clean_scrub_stamp":"2024-04-26T21:12:53.803064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:19.842655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T21:12:57.018043+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:12:57.018043+0000","last_peered":"2024-04-26T21:12:57.018043+0000","last_clean":"2024-04-26T21:12:57.018043+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:12:57.018043+0000","last_undegraded":"2024-04-26T21:12:57.018043+0000","last_fullsized":"2024-04-26T21:12:57.018043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":747537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":157183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":157149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":158713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:13:04.681 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:13:04.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:13:04.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 Processor -- start 2024-04-26T21:13:04.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 -- start start 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa184162af0 0x7fa1840cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1840b57e0 0x7fa1840d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 0x7fa1840d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa18406d5c0 con 0x7fa1840b57e0 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa18406daf0 con 0x7fa1840b4280 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.824+0000 7fa18a4d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa18406d810 con 0x7fa184162af0 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1840b57e0 0x7fa1840d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1840b57e0 0x7fa1840d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37680/0 (socket says 172.21.15.40:37680) 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 -- 172.21.15.40:0/4014218760 learned_addr learned my addr 172.21.15.40:0/4014218760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:04.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa183fff640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa184162af0 0x7fa1840cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa188a76640 1 --2- 172.21.15.40:0/4014218760 >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 0x7fa1840d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 -- 172.21.15.40:0/4014218760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa184162af0 msgr2=0x7fa1840cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa184162af0 0x7fa1840cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 -- 172.21.15.40:0/4014218760 >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 msgr2=0x7fa1840d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 --2- 172.21.15.40:0/4014218760 >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 0x7fa1840d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa1837fe640 1 -- 172.21.15.40:0/4014218760 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18406de60 con 0x7fa1840b57e0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.825+0000 7fa188a76640 1 --2- 172.21.15.40:0/4014218760 >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 0x7fa1840d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa1837fe640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840b57e0 0x7fa1840d1d30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa184073ac0 tx=0x7fa17802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e9ae389fb6b0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa182ffd640 1 -- 172.21.15.40:0/4014218760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178037070 con 0x7fa1840b57e0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa182ffd640 1 -- 172.21.15.40:0/4014218760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa178032440 con 0x7fa1840b57e0 2024-04-26T21:13:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa182ffd640 1 -- 172.21.15.40:0/4014218760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178032810 con 0x7fa1840b57e0 2024-04-26T21:13:04.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 -- 172.21.15.40:0/4014218760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840b57e0 msgr2=0x7fa1840d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840b57e0 0x7fa1840d1d30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa184073ac0 tx=0x7fa17802a140 comp rx=0 tx=0).stop 2024-04-26T21:13:04.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 -- 172.21.15.40:0/4014218760 shutdown_connections 2024-04-26T21:13:04.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/4014218760 >> v2:172.21.15.184:3300/0 conn(0x7fa1840b4280 0x7fa1840d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840b57e0 0x7fa1840d1d30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/4014218760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa184162af0 0x7fa1840cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 -- 172.21.15.40:0/4014218760 >> 172.21.15.40:0/4014218760 conn(0x7fa1840b4ab0 msgr2=0x7fa1840b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 -- 172.21.15.40:0/4014218760 shutdown_connections 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.826+0000 7fa18a4d9640 1 -- 172.21.15.40:0/4014218760 wait complete. 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 Processor -- start 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 -- start start 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1840b57e0 0x7fa18415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1840b75e0 0x7fa1840b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840cdcb0 con 0x7fa184162af0 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840cdb30 con 0x7fa1840b57e0 2024-04-26T21:13:04.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.828+0000 7fa18a4d9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa184162ed0 con 0x7fa1840b75e0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37686/0 (socket says 172.21.15.40:37686) 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 -- 172.21.15.40:0/380248342 learned_addr learned my addr 172.21.15.40:0/380248342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa183fff640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1840b57e0 0x7fa18415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa188a76640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1840b75e0 0x7fa1840b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 -- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1840b75e0 msgr2=0x7fa1840b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1840b75e0 0x7fa1840b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 -- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1840b57e0 msgr2=0x7fa18415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1840b57e0 0x7fa18415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 -- 172.21.15.40:0/380248342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18415c5a0 con 0x7fa184162af0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa1837fe640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa178000c00 tx=0x7fa1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178041020 con 0x7fa184162af0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1780306c0 con 0x7fa184162af0 2024-04-26T21:13:04.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1840b7b30 con 0x7fa184162af0 2024-04-26T21:13:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178037040 con 0x7fa184162af0 2024-04-26T21:13:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.829+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1840b80f0 con 0x7fa184162af0 2024-04-26T21:13:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.831+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1840693d0 con 0x7fa184162af0 2024-04-26T21:13:04.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.831+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa178044020 con 0x7fa184162af0 2024-04-26T21:13:04.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.831+0000 7fa180ff9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:04.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.831+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa1780ac230 con 0x7fa184162af0 2024-04-26T21:13:04.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.832+0000 7fa183fff640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:04.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.832+0000 7fa183fff640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 0x7fa17006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1740079a0 tx=0x7fa174008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:04.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:04.836+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa17804b030 con 0x7fa184162af0 2024-04-26T21:13:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.091+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa1840b8bf0 con 0x7fa184162af0 2024-04-26T21:13:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.093+0000 7fa180ff9640 1 -- 172.21.15.40:0/380248342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fa18415c5a0 con 0x7fa184162af0 2024-04-26T21:13:05.094 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:13:05.095 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:13:05.095 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:13:05.095 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:13:05.095 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:13:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:13:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 msgr2=0x7fa17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 0x7fa17006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1740079a0 tx=0x7fa174008040 comp rx=0 tx=0).stop 2024-04-26T21:13:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 msgr2=0x7fa1840b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa178000c00 tx=0x7fa1780046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 shutdown_connections 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1840b75e0 0x7fa1840b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa184162af0 0x7fa1840b5bc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 --2- 172.21.15.40:0/380248342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1840b57e0 0x7fa18415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 >> 172.21.15.40:0/380248342 conn(0x7fa1840b4ab0 msgr2=0x7fa18415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 shutdown_connections 2024-04-26T21:13:05.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.096+0000 7fa18a4d9640 1 -- 172.21.15.40:0/380248342 wait complete. 2024-04-26T21:13:05.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.208+0000 7f3854166640 1 Processor -- start 2024-04-26T21:13:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.208+0000 7f3854166640 1 -- start start 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 0x7f384c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c0d11b0 0x7f384c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 0x7f384c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f384c06e750 con 0x7f384c0d11b0 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f384c0d1740 con 0x7f384c06e2c0 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3854166640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f384c0d1a20 con 0x7f384c0cf2e0 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c0d11b0 0x7f384c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c0d11b0 0x7f384c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37690/0 (socket says 172.21.15.40:37690) 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 -- 172.21.15.40:0/3474070331 learned_addr learned my addr 172.21.15.40:0/3474070331 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851f02640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 0x7f384c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3852703640 1 --2- 172.21.15.40:0/3474070331 >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 0x7f384c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 -- 172.21.15.40:0/3474070331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 msgr2=0x7f384c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 0x7f384c06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 -- 172.21.15.40:0/3474070331 >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 msgr2=0x7f384c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 --2- 172.21.15.40:0/3474070331 >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 0x7f384c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851701640 1 -- 172.21.15.40:0/3474070331 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c0d1d00 con 0x7f384c0d11b0 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3851f02640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 0x7f384c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.209+0000 7f3852703640 1 --2- 172.21.15.40:0/3474070331 >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 0x7f384c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3851701640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0d11b0 0x7f384c06dd80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f384c163ff0 tx=0x7f383c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb7020e06f56b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3850f00640 1 -- 172.21.15.40:0/3474070331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c037070 con 0x7f384c0d11b0 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3850f00640 1 -- 172.21.15.40:0/3474070331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c032440 con 0x7f384c0d11b0 2024-04-26T21:13:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3850f00640 1 -- 172.21.15.40:0/3474070331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c032810 con 0x7f384c0d11b0 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 -- 172.21.15.40:0/3474070331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0d11b0 msgr2=0x7f384c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0d11b0 0x7f384c06dd80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f384c163ff0 tx=0x7f383c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 -- 172.21.15.40:0/3474070331 shutdown_connections 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 --2- 172.21.15.40:0/3474070331 >> v2:172.21.15.184:3300/0 conn(0x7f384c06e2c0 0x7f384c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0d11b0 0x7f384c06dd80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 --2- 172.21.15.40:0/3474070331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0cf2e0 0x7f384c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.210+0000 7f3854166640 1 -- 172.21.15.40:0/3474070331 >> 172.21.15.40:0/3474070331 conn(0x7f384c0b4a60 msgr2=0x7f384c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.211+0000 7f3854166640 1 -- 172.21.15.40:0/3474070331 shutdown_connections 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.211+0000 7f3854166640 1 -- 172.21.15.40:0/3474070331 wait complete. 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 Processor -- start 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 -- start start 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c0cf2e0 0x7f384c09ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0d11b0 0x7f384c09f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f384c06da10 con 0x7f384c06e2c0 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0d22c0 con 0x7f384c0cf2e0 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.212+0000 7f3854166640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f384c0cf6c0 con 0x7f384c0d11b0 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37696/0 (socket says 172.21.15.40:37696) 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 -- 172.21.15.40:0/1634319768 learned_addr learned my addr 172.21.15.40:0/1634319768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3852703640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0d11b0 0x7f384c09f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851701640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c0cf2e0 0x7f384c09ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 -- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0d11b0 msgr2=0x7f384c09f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0d11b0 0x7f384c09f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 -- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c0cf2e0 msgr2=0x7f384c09ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c0cf2e0 0x7f384c09ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 -- 172.21.15.40:0/1634319768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c0cd1f0 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3851f02640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f384800c7f0 tx=0x7f384800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848015070 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848007a30 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c0c8680 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848007e00 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.213+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c0c8c40 con 0x7f384c06e2c0 2024-04-26T21:13:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.214+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c06a290 con 0x7f384c06e2c0 2024-04-26T21:13:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.215+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3848018070 con 0x7f384c06e2c0 2024-04-26T21:13:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.215+0000 7f38427fc640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.215+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f384808bb00 con 0x7f384c06e2c0 2024-04-26T21:13:05.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.216+0000 7f3851701640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.216+0000 7f3851701640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 0x7f383006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.220+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3848059f20 con 0x7f384c06e2c0 2024-04-26T21:13:05.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.445+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f384c0b8ef0 con 0x7f384c06e2c0 2024-04-26T21:13:05.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.446+0000 7f38427fc640 1 -- 172.21.15.40:0/1634319768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f3848059820 con 0x7f384c06e2c0 2024-04-26T21:13:05.447 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:05.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:05.447 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 msgr2=0x7f383006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 0x7f383006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c02e000 comp rx=0 tx=0).stop 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 msgr2=0x7f384c09ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f384800c7f0 tx=0x7f384800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 shutdown_connections 2024-04-26T21:13:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c0d11b0 0x7f384c09f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c0cf2e0 0x7f384c09ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 --2- 172.21.15.40:0/1634319768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c06e2c0 0x7f384c09ea00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 >> 172.21.15.40:0/1634319768 conn(0x7f384c0b4a60 msgr2=0x7f384c070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 shutdown_connections 2024-04-26T21:13:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.448+0000 7f3854166640 1 -- 172.21.15.40:0/1634319768 wait complete. 2024-04-26T21:13:05.862 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:12:55.189839+0000', 'modified': '2024-04-26T21:12:57.948896+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5145}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5145': {'gid': 5145, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6834/1813208926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1813208926}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 1813208926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5145, 'qdb_cluster': [5145]} 2024-04-26T21:13:05.862 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:13:05.862 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:13:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.967+0000 7fd25b007640 1 Processor -- start 2024-04-26T21:13:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.967+0000 7fd25b007640 1 -- start start 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd25b007640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd25415f340 0x7fd2540b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd25b007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd2540d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd25b007640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2540b2510 con 0x7fd25415f720 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd25b007640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd2540b27f0 con 0x7fd25415f340 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd25b007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2540b2ad0 con 0x7fd2540d5610 2024-04-26T21:13:05.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd253fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd25415f720 0x7fd2540d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52406/0 (socket says 172.21.15.40:52406) 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd253fff640 1 -- 172.21.15.40:0/3853617380 learned_addr learned my addr 172.21.15.40:0/3853617380 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd258da3640 1 --2- 172.21.15.40:0/3853617380 >> v2:172.21.15.184:3300/0 conn(0x7fd25415f340 0x7fd2540b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2595a4640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd2540d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633319729 0 0) 0x7fd2540b2510 con 0x7fd25415f720 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd23c003520 con 0x7fd25415f720 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3413323108 0 0) 0x7fd23c003520 con 0x7fd25415f720 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 msgr2=0x7fd2540d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd2540d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 >> v2:172.21.15.184:3300/0 conn(0x7fd25415f340 msgr2=0x7fd2540b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 --2- 172.21.15.40:0/3853617380 >> v2:172.21.15.184:3300/0 conn(0x7fd25415f340 0x7fd2540b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.968+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2540b2db0 con 0x7fd25415f720 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd248001670 con 0x7fd25415f720 2024-04-26T21:13:05.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 >> v1:172.21.15.40:6789/0 conn(0x7fd25415f720 legacy=0x7fd2540d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0039b0 0x7fd23c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c004800 0x7fd23c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd23c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540b27f0 con 0x7fd23c0039b0 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540b2ad0 con 0x7fd23c004800 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd23c00b9c0 con 0x7fd2540d5610 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0039b0 0x7fd23c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd253fff640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c004800 0x7fd23c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2595a4640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd23c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 -- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 msgr2=0x7fd23c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd23c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 -- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c004800 msgr2=0x7fd23c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c004800 0x7fd23c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd258da3640 1 -- 172.21.15.40:0/3853617380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c003520 con 0x7fd23c0039b0 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd2595a4640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2540d5610 0x7fd23c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.969+0000 7fd253fff640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c004800 0x7fd23c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd258da3640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0039b0 0x7fd23c003dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd244009af0 tx=0x7fd23c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e6844c3455cd2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244036070 con 0x7fd23c0039b0 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd244031400 con 0x7fd23c0039b0 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd2537fe640 1 -- 172.21.15.40:0/3853617380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244009690 con 0x7fd23c0039b0 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd25b007640 1 -- 172.21.15.40:0/3853617380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0039b0 msgr2=0x7fd23c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd25b007640 1 --2- 172.21.15.40:0/3853617380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0039b0 0x7fd23c003dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd244009af0 tx=0x7fd23c005830 comp rx=0 tx=0).stop 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd2595a4640 1 -- 172.21.15.40:0/3853617380 reap_dead start 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd25b007640 1 -- 172.21.15.40:0/3853617380 shutdown_connections 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd25b007640 1 -- 172.21.15.40:0/3853617380 >> 172.21.15.40:0/3853617380 conn(0x7fd25408ec70 msgr2=0x7fd25409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.970+0000 7fd25b007640 1 -- 172.21.15.40:0/3853617380 shutdown_connections 2024-04-26T21:13:05.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.971+0000 7fd25b007640 1 -- 172.21.15.40:0/3853617380 wait complete. 2024-04-26T21:13:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.972+0000 7fd25b007640 1 Processor -- start 2024-04-26T21:13:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.972+0000 7fd25b007640 1 -- start start 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c0039b0 0x7fd254154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25415fbc0 0x7fd2540b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540d40d0 con 0x7fd254160040 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540d3f50 con 0x7fd23c0039b0 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2540b35b0 con 0x7fd25415fbc0 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37720/0 (socket says 172.21.15.40:37720) 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 -- 172.21.15.40:0/2561764789 learned_addr learned my addr 172.21.15.40:0/2561764789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd253fff640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25415fbc0 0x7fd2540b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd258da3640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c0039b0 0x7fd254154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 -- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25415fbc0 msgr2=0x7fd2540b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25415fbc0 0x7fd2540b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 -- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c0039b0 msgr2=0x7fd254154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c0039b0 0x7fd254154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 -- 172.21.15.40:0/2561764789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd254160610 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2540c8d50 tx=0x7fd24c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2517fa640 1 -- 172.21.15.40:0/2561764789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c011e60 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2517fa640 1 -- 172.21.15.40:0/2561764789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24c01c070 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd2517fa640 1 -- 172.21.15.40:0/2561764789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0166b0 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2541608f0 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.973+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd254160eb0 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.975+0000 7fd2517fa640 1 -- 172.21.15.40:0/2561764789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd24c005d10 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.975+0000 7fd2517fa640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.975+0000 7fd2517fa640 1 -- 172.21.15.40:0/2561764789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd24c08c3c0 con 0x7fd254160040 2024-04-26T21:13:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.976+0000 7fd258da3640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.976+0000 7fd25ddf7740 1 Processor -- start 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.976+0000 7fd25ddf7740 1 -- start start 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.976+0000 7fd258da3640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 0x7fd24006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd244033000 tx=0x7fd244008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b230b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 0x560b230d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b22fef740 con 0x560b2308b120 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b22f8b3a0 con 0x560b230b5440 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560b22feb700 con 0x560b230d2ce0 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37724/0 (socket says 172.21.15.40:37724) 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 -- 172.21.15.40:0/2268523557 learned_addr learned my addr 172.21.15.40:0/2268523557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd258da3640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b230b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd253fff640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 0x560b230d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 -- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 msgr2=0x560b230d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 0x560b230d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 -- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 msgr2=0x560b230b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b230b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd2595a4640 1 -- 172.21.15.40:0/2268523557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22fa7750 con 0x560b2308b120 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.977+0000 7fd253fff640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 0x560b230d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd2595a4640 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd24c05a090 tx=0x7fd24c05a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8b43592fc7a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd23a7fc640 1 -- 172.21.15.40:0/2268523557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c058400 con 0x560b2308b120 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd23a7fc640 1 -- 172.21.15.40:0/2268523557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24c05e0d0 con 0x560b2308b120 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd23a7fc640 1 -- 172.21.15.40:0/2268523557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c056770 con 0x560b2308b120 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 -- 172.21.15.40:0/2268523557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 msgr2=0x560b230d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd24c05a090 tx=0x7fd24c05a280 comp rx=0 tx=0).stop 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 -- 172.21.15.40:0/2268523557 shutdown_connections 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b2308b120 0x560b230d13a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230d2ce0 0x560b230d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/2268523557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b230b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 -- 172.21.15.40:0/2268523557 >> 172.21.15.40:0/2268523557 conn(0x560b23087a50 msgr2=0x560b230d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 -- 172.21.15.40:0/2268523557 shutdown_connections 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.978+0000 7fd25ddf7740 1 -- 172.21.15.40:0/2268523557 wait complete. 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.979+0000 7fd25ddf7740 1 -- start start 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.979+0000 7fd25ddf7740 1 Processor -- start 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b7cc0 0x560b230d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b2308bec0 con 0x560b230d2ce0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b22f76fe0 con 0x560b230b7cc0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560b23085980 con 0x560b230b5440 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd25ddf7740 10 client.0 _ll_register_callbacks cb 0x560b22f8d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37728/0 (socket says 172.21.15.40:37728) 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 -- 172.21.15.40:0/3469331387 learned_addr learned my addr 172.21.15.40:0/3469331387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd2595a4640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b7cc0 0x560b230d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd258da3640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 -- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 msgr2=0x560b230e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 -- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b7cc0 msgr2=0x560b230d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b7cc0 0x560b230d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.980+0000 7fd253fff640 1 -- 172.21.15.40:0/3469331387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b22d012d0 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd253fff640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2480135b0 tx=0x7fd248013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534354e285e7ccfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24801a070 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd248015740 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd24c05e0d0 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fd248015940 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 10 client.5313 fetch_fsmap learned FSMap version 14 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x560b22dcb6b0 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24c05e0d0 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7fd248028080 con 0x560b230d2ce0 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 10 client.5313 fetch_fsmap finished waiting for FSMap version 14 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 10 client.5313 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:13:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 20 client.5313 populate_metadata read hostname 'smithi040' 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c7890 0x560b230c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 4 client.5313 mds_command: new command op to 5145 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 --> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b22ffd350 con 0x560b230c7890 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.981+0000 7fd258da3640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c7890 0x560b230c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd258da3640 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c7890 0x560b230c7c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50c9769472187e6 server_cookie=6ddc065458d8cadc in_seq=0 out_seq=0 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd2337fe640 10 client.5313 ms_handle_connect on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd2337fe640 1 -- 172.21.15.40:0/3469331387 <== mds.0 v2:172.21.15.40:6832/1813208926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b22ffd350 con 0x560b230c7890 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd2337fe640 10 client.5313 handle_command_reply: tid=0 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd25ddf7740 1 client.5313 shutdown 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 msgr2=0x560b230d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2480135b0 tx=0x7fd248013a70 comp rx=0 tx=0).stop 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.982+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 shutdown_connections 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c7890 0x560b230c7c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b7cc0 0x560b230d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230d0d00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd2337fe640 0 client.5313 ms_handle_reset on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/3469331387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 >> 172.21.15.40:0/3469331387 conn(0x560b2308b120 msgr2=0x560b230d4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 shutdown_connections 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 1 -- 172.21.15.40:0/3469331387 wait complete. 2024-04-26T21:13:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:05.983+0000 7fd25ddf7740 20 client.5313 trim_cache size 0 max 16384 2024-04-26T21:13:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.001+0000 7fd25ddf7740 1 Processor -- start 2024-04-26T21:13:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.001+0000 7fd25ddf7740 1 -- start start 2024-04-26T21:13:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230d2ce0 0x560b230c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd254201000 con 0x560b230b7cc0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540cda30 con 0x560b230d2ce0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd254200a50 con 0x560b230b5440 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37738/0 (socket says 172.21.15.40:37738) 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 -- 172.21.15.40:0/303647054 learned_addr learned my addr 172.21.15.40:0/303647054 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd2595a4640 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230d2ce0 0x560b230c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 -- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 msgr2=0x560b230c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 -- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230d2ce0 msgr2=0x560b230c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230d2ce0 0x560b230c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 -- 172.21.15.40:0/303647054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b23042fc0 con 0x560b230b7cc0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd253fff640 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd248015b30 tx=0x7fd24800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307b178d80a2ec6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd239ffb640 1 -- 172.21.15.40:0/303647054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24801a070 con 0x560b230b7cc0 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd239ffb640 1 -- 172.21.15.40:0/303647054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248015740 con 0x560b230b7cc0 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.002+0000 7fd239ffb640 1 -- 172.21.15.40:0/303647054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248016450 con 0x560b230b7cc0 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- 172.21.15.40:0/303647054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 msgr2=0x560b231417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd248015b30 tx=0x7fd24800c000 comp rx=0 tx=0).stop 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- 172.21.15.40:0/303647054 shutdown_connections 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230d2ce0 0x560b230c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230b7cc0 0x560b231417a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/303647054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b5440 0x560b230c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- 172.21.15.40:0/303647054 >> 172.21.15.40:0/303647054 conn(0x560b2308b120 msgr2=0x560b2308b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- 172.21.15.40:0/303647054 shutdown_connections 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- 172.21.15.40:0/303647054 wait complete. 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.003+0000 7fd25ddf7740 1 -- start start 2024-04-26T21:13:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 Processor -- start 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b2313ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b7cc0 0x560b230e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b230cf420 con 0x560b230d2ce0 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560b230b6e40 con 0x560b230b5440 2024-04-26T21:13:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.004+0000 7fd25ddf7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560b230e4420 con 0x560b230b7cc0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd25ddf7740 10 client.0 _ll_register_callbacks cb 0x7fd2480057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37742/0 (socket says 172.21.15.40:37742) 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 -- 172.21.15.40:0/677826008 learned_addr learned my addr 172.21.15.40:0/677826008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd253fff640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b7cc0 0x560b230e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd258da3640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b2313ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 -- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b7cc0 msgr2=0x560b230e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b7cc0 0x560b230e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 -- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 msgr2=0x560b2313ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b2313ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 -- 172.21.15.40:0/677826008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b230b4130 con 0x560b230d2ce0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2595a4640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd24c05a090 tx=0x7fd24c023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c6c5c01aa82673 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c059550 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd24c0596f0 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248015740 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fd24c055e50 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd25ddf7740 10 client.5319 fetch_fsmap learned FSMap version 14 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x560b230f3230 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.005+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248015740 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7fd24c055ff0 con 0x560b230d2ce0 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 10 client.5319 fetch_fsmap finished waiting for FSMap version 14 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 10 client.5319 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 20 client.5319 populate_metadata read hostname 'smithi040' 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c8d00 0x560b230c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 4 client.5319 mds_command: new command op to 5145 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 --> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] -- command(tid 0: {"prefix": "status"}) -- 0x560b22ff71c0 con 0x560b230c8d00 2024-04-26T21:13:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd258da3640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c8d00 0x560b230c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd2337fe640 10 client.5319 ms_handle_connect on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd258da3640 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c8d00 0x560b230c90e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98feffb2b690eb39 server_cookie=6ef77d68b8b873c6 in_seq=0 out_seq=0 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd2337fe640 1 -- 172.21.15.40:0/677826008 <== mds.0 v2:172.21.15.40:6832/1813208926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x560b22ff71c0 con 0x560b230c8d00 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd2337fe640 10 client.5319 handle_command_reply: tid=0 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 1 client.5319 shutdown 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 msgr2=0x560b230e5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.006+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd24c05a090 tx=0x7fd24c023040 comp rx=0 tx=0).stop 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 shutdown_connections 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x560b230c8d00 0x560b230c90e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b230d2ce0 0x560b230e5130 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560b230b7cc0 0x560b230e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd2337fe640 0 client.5319 ms_handle_reset on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 --2- 172.21.15.40:0/677826008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560b230b5440 0x560b2313ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 >> 172.21.15.40:0/677826008 conn(0x560b2308b120 msgr2=0x560b23089ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 shutdown_connections 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 1 -- 172.21.15.40:0/677826008 wait complete. 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25ddf7740 20 client.5319 trim_cache size 0 max 16384 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5145, 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:13:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.190321501, 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 14, 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 36, 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 36, 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.230378322 2024-04-26T21:13:06.015 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 msgr2=0x7fd24006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 0x7fd24006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd244033000 tx=0x7fd244008030 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 msgr2=0x7fd2540b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2540c8d50 tx=0x7fd24c009f90 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 shutdown_connections 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd254160040 0x7fd2540b9ef0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd25415fbc0 0x7fd2540b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 --2- 172.21.15.40:0/2561764789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd23c0039b0 0x7fd254154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 >> 172.21.15.40:0/2561764789 conn(0x7fd25408ec70 msgr2=0x7fd25406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:06.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 shutdown_connections 2024-04-26T21:13:06.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.007+0000 7fd25b007640 1 -- 172.21.15.40:0/2561764789 wait complete. 2024-04-26T21:13:06.019 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:13:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.119+0000 7f385f938640 1 Processor -- start 2024-04-26T21:13:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.119+0000 7f385f938640 1 -- start start 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385f938640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 0x7f38580b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385f938640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38580d5610 0x7f38580d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385f938640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38580b2510 con 0x7f385815f720 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385f938640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f38580b27f0 con 0x7f38580d5610 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385f938640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f38580b2ad0 con 0x7f385815f340 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38580d5610 0x7f38580d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38580d5610 0x7f38580d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37746/0 (socket says 172.21.15.40:37746) 2024-04-26T21:13:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 -- 172.21.15.40:0/1310786933 learned_addr learned my addr 172.21.15.40:0/1310786933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385d6d4640 1 --2- 172.21.15.40:0/1310786933 >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 0x7f38580b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 -- 172.21.15.40:0/1310786933 >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 msgr2=0x7f38580b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 --2- 172.21.15.40:0/1310786933 >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 0x7f38580b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 -- 172.21.15.40:0/1310786933 >> v1:172.21.15.184:6789/0 conn(0x7f385815f720 legacy=0x7f38580d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385ded5640 1 -- 172.21.15.40:0/1310786933 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38580b2db0 con 0x7f38580d5610 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.120+0000 7f385d6d4640 1 --2- 172.21.15.40:0/1310786933 >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 0x7f38580b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.121+0000 7f385ded5640 1 --2- 172.21.15.40:0/1310786933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38580d5610 0x7f38580d3d40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f38580d6000 tx=0x7f385002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2fd50e0bdd3057 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.121+0000 7f384ffff640 1 -- 172.21.15.40:0/1310786933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850036070 con 0x7f38580d5610 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.121+0000 7f384ffff640 1 -- 172.21.15.40:0/1310786933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3850031600 con 0x7f38580d5610 2024-04-26T21:13:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.121+0000 7f384ffff640 1 -- 172.21.15.40:0/1310786933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38500319d0 con 0x7f38580d5610 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 -- 172.21.15.40:0/1310786933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38580d5610 msgr2=0x7f38580d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 --2- 172.21.15.40:0/1310786933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38580d5610 0x7f38580d3d40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f38580d6000 tx=0x7f385002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 -- 172.21.15.40:0/1310786933 shutdown_connections 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 --2- 172.21.15.40:0/1310786933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38580d5610 0x7f38580d3d40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 --2- 172.21.15.40:0/1310786933 >> v2:172.21.15.184:3300/0 conn(0x7f385815f340 0x7f38580b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 -- 172.21.15.40:0/1310786933 >> 172.21.15.40:0/1310786933 conn(0x7f385808ec70 msgr2=0x7f385809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 -- 172.21.15.40:0/1310786933 shutdown_connections 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.122+0000 7f385f938640 1 -- 172.21.15.40:0/1310786933 wait complete. 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.123+0000 7f385f938640 1 Processor -- start 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.123+0000 7f385f938640 1 -- start start 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38580d5610 0x7f38581555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f385815f720 0x7f38580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38580b3940 con 0x7f385815f340 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38580b37c0 con 0x7f38580d5610 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38580b3ac0 con 0x7f385815f720 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37748/0 (socket says 172.21.15.40:37748) 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 -- 172.21.15.40:0/2850341714 learned_addr learned my addr 172.21.15.40:0/2850341714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385d6d4640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38580d5610 0x7f38581555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ded5640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f385815f720 0x7f38580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 -- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f385815f720 msgr2=0x7f38580b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f385815f720 0x7f38580b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 -- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38580d5610 msgr2=0x7f38581555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38580d5610 0x7f38581555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 -- 172.21.15.40:0/2850341714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38580a22f0 con 0x7f385815f340 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385ced3640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f384800c440 tx=0x7f384800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480052d0 con 0x7f385815f340 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848005470 con 0x7f385815f340 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38580a25d0 con 0x7f385815f340 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.124+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848007260 con 0x7f385815f340 2024-04-26T21:13:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.125+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38580b8540 con 0x7f385815f340 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.126+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385806a290 con 0x7f385815f340 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.126+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38480074c0 con 0x7f385815f340 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.126+0000 7f384dffb640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 0x7f383c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.126+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f384808b070 con 0x7f385815f340 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.127+0000 7f385d6d4640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 0x7f383c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.127+0000 7f385d6d4640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 0x7f383c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3854006fd0 tx=0x7f3854008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.131+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3848059400 con 0x7f385815f340 2024-04-26T21:13:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.356+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f38580d3020 con 0x7f385815f340 2024-04-26T21:13:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.356+0000 7f384dffb640 1 -- 172.21.15.40:0/2850341714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f3848058d00 con 0x7f385815f340 2024-04-26T21:13:06.358 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.358+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 msgr2=0x7f383c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.358+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 0x7f383c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3854006fd0 tx=0x7f3854008040 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 msgr2=0x7f38580a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f384800c440 tx=0x7f384800c900 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 shutdown_connections 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383c06c8f0 0x7f383c06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f385815f720 0x7f38580b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f385815f340 0x7f38580a18a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 --2- 172.21.15.40:0/2850341714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38580d5610 0x7f38581555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 >> 172.21.15.40:0/2850341714 conn(0x7f385808ec70 msgr2=0x7f38580b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 shutdown_connections 2024-04-26T21:13:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06.359+0000 7f385f938640 1 -- 172.21.15.40:0/2850341714 wait complete. 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:13:06.371 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:13:06.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.372 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:06.388 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42683sec preferred_lft 42683sec 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:13:06.389 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:13:06.390 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:06.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.391 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:13:06.451 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:13:06.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.452 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:13:06.509 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:13:06.509 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:13:06.509 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:13:06.509 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:13:06.509 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:13:06.509 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:13:06.510 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:13:06.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:13:06.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:13:06.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:13:06.510 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:13:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:13:06.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:06.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.636 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:13:06.652 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:13:06.652 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:13:06.652 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:13:06.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:13:06.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:13:06.717 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:13:06.772 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:13:06.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:13:06.772 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:13:06.827 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:13:06.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:13:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:13:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:13:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:13:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:13:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:13:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:13:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:13:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:13:06.924 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:13:06.925 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:13:06.927 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:13:06.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:06.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:13:06.993 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:13:07.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:13:07.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:07.098 INFO:teuthology.orchestra.run.smithi040.stdout:client4835 2024-04-26T21:13:07.099 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:07.099 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:07.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:13:07.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.254+0000 7fda67722640 1 Processor -- start 2024-04-26T21:13:07.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.254+0000 7fda67722640 1 -- start start 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda67722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda6006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda67722640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda600d11b0 0x7fda6006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda67722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda600d1740 con 0x7fda6006e2c0 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda67722640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fda6006e750 con 0x7fda600d11b0 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda67722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda600d1a20 con 0x7fda600cf2e0 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda65cbf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda6006e2c0 0x7fda6015ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52410/0 (socket says 172.21.15.40:52410) 2024-04-26T21:13:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda65cbf640 1 -- 172.21.15.40:0/3842385529 learned_addr learned my addr 172.21.15.40:0/3842385529 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda6006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda64cbd640 1 --2- 172.21.15.40:0/3842385529 >> v2:172.21.15.184:3300/0 conn(0x7fda600d11b0 0x7fda6006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2428693822 0 0) 0x7fda600d1740 con 0x7fda6006e2c0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.255+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda4c003520 con 0x7fda6006e2c0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3549086531 0 0) 0x7fda4c003520 con 0x7fda6006e2c0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 msgr2=0x7fda6006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda6006d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 >> v2:172.21.15.184:3300/0 conn(0x7fda600d11b0 msgr2=0x7fda6006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 --2- 172.21.15.40:0/3842385529 >> v2:172.21.15.184:3300/0 conn(0x7fda600d11b0 0x7fda6006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda600d1d00 con 0x7fda6006e2c0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fda580019f0 con 0x7fda6006e2c0 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda64cbd640 1 --2- 172.21.15.40:0/3842385529 >> v2:172.21.15.184:3300/0 conn(0x7fda600d11b0 0x7fda6006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda6006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:13:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 >> v1:172.21.15.40:6789/0 conn(0x7fda6006e2c0 legacy=0x7fda6015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda4c004d40 0x7fda4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda6006e750 con 0x7fda4c0039b0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda600d1a20 con 0x7fda4c004d40 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c00b9c0 con 0x7fda600cf2e0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda65cbf640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda4c004d40 0x7fda4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda64cbd640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 -- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 msgr2=0x7fda4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda600cf2e0 0x7fda4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 -- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda4c004d40 msgr2=0x7fda4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda4c004d40 0x7fda4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda654be640 1 -- 172.21.15.40:0/3842385529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c003520 con 0x7fda4c0039b0 2024-04-26T21:13:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.256+0000 7fda65cbf640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda4c004d40 0x7fda4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda654be640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda4c003dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009c90 tx=0x7fda4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a745a870ff692fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c036070 con 0x7fda4c0039b0 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c031400 con 0x7fda4c0039b0 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda57fff640 1 -- 172.21.15.40:0/3842385529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c009860 con 0x7fda4c0039b0 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda67722640 1 -- 172.21.15.40:0/3842385529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 msgr2=0x7fda4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda67722640 1 --2- 172.21.15.40:0/3842385529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda4c003dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009c90 tx=0x7fda4c005830 comp rx=0 tx=0).stop 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda65cbf640 1 -- 172.21.15.40:0/3842385529 reap_dead start 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda67722640 1 -- 172.21.15.40:0/3842385529 shutdown_connections 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.257+0000 7fda67722640 1 -- 172.21.15.40:0/3842385529 >> 172.21.15.40:0/3842385529 conn(0x7fda600b4a60 msgr2=0x7fda600b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.258+0000 7fda67722640 1 -- 172.21.15.40:0/3842385529 shutdown_connections 2024-04-26T21:13:07.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.258+0000 7fda67722640 1 -- 172.21.15.40:0/3842385529 wait complete. 2024-04-26T21:13:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 Processor -- start 2024-04-26T21:13:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 -- start start 2024-04-26T21:13:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda601f82f0 0x7fda60161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda600c7450 0x7fda600c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda600cf6c0 con 0x7fda4c0039b0 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda600d22c0 con 0x7fda600c7450 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.259+0000 7fda67722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda600cf840 con 0x7fda601f82f0 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37772/0 (socket says 172.21.15.40:37772) 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 -- 172.21.15.40:0/2352973186 learned_addr learned my addr 172.21.15.40:0/2352973186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda64cbd640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda601f82f0 0x7fda60161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda65cbf640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda600c7450 0x7fda600c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 -- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda601f82f0 msgr2=0x7fda60161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda601f82f0 0x7fda60161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 -- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda600c7450 msgr2=0x7fda600c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda600c7450 0x7fda600c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 -- 172.21.15.40:0/2352973186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60161870 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda654be640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fda5c000c00 tx=0x7fda5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c040070 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c0093e0 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda601f87d0 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.260+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c036040 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.261+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601f8d90 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.261+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6006a290 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.262+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda5c043070 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.263+0000 7fda55ffb640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 0x7fda4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.263+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fda5c0ac1d0 con 0x7fda4c0039b0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.263+0000 7fda64cbd640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 0x7fda4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:07.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.263+0000 7fda64cbd640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 0x7fda4406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda601f8250 tx=0x7fda50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:07.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.267+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda5c079f10 con 0x7fda4c0039b0 2024-04-26T21:13:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.491+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda600b8ef0 con 0x7fda4c0039b0 2024-04-26T21:13:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.491+0000 7fda55ffb640 1 -- 172.21.15.40:0/2352973186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fda5c07e210 con 0x7fda4c0039b0 2024-04-26T21:13:07.492 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:13:07.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 msgr2=0x7fda4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 0x7fda4406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda601f8250 tx=0x7fda50008040 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 msgr2=0x7fda6009ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fda5c000c00 tx=0x7fda5c004730 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 shutdown_connections 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda4406c920 0x7fda4406ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda600c7450 0x7fda600c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda601f82f0 0x7fda60161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 --2- 172.21.15.40:0/2352973186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4c0039b0 0x7fda6009ff30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 >> 172.21.15.40:0/2352973186 conn(0x7fda600b4a60 msgr2=0x7fda600c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 shutdown_connections 2024-04-26T21:13:07.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:07.493+0000 7fda67722640 1 -- 172.21.15.40:0/2352973186 wait complete. 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:13:07.505 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:13:07.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.506 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:07.526 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42686sec preferred_lft 42686sec 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:13:07.527 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:13:07.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.529 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:13:07.595 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:13:07.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.597 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:13:07.656 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:13:07.656 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:13:07.656 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:13:07.656 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:13:07.656 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:13:07.657 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:13:07.750 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:13:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:13:07.782 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:13:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:07.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.789 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:13:07.804 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:13:07.804 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:13:07.804 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:13:07.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.805 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:13:07.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:13:07.869 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:13:07.925 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:13:07.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:13:07.925 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:13:07.979 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:13:07.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:07.979 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:13:08.061 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:13:08.070 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:13:08.077 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:13:08.078 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:13:08.079 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:13:08.080 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:13:08.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:08.080 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:13:08.145 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:13:08.234 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:13:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:13:08.236 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:13:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:08.241 INFO:teuthology.orchestra.run.smithi184.stdout:client4844 2024-04-26T21:13:08.242 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:13:08.242 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:08.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2024-04-26T21:13:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.372+0000 7efeec371640 1 Processor -- start 2024-04-26T21:13:08.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.372+0000 7efeec371640 1 -- start start 2024-04-26T21:13:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeec371640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efee40b2000 0x7efee406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeec371640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee409c790 con 0x7efee40d1170 2024-04-26T21:13:08.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeec371640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee4163ec0 con 0x7efee41631f0 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeec371640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efee40d1600 con 0x7efee40b2000 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeea90e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efee40d1170 0x7efee40b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52412/0 (socket says 172.21.15.40:52412) 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efeea90e640 1 -- 172.21.15.40:0/2843319066 learned_addr learned my addr 172.21.15.40:0/2843319066 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee990c640 1 --2- 172.21.15.40:0/2843319066 >> v2:172.21.15.184:3300/0 conn(0x7efee40b2000 0x7efee406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007485189 0 0) 0x7efee409c790 con 0x7efee40d1170 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed0003520 con 0x7efee40d1170 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee990c640 1 -- 172.21.15.40:0/2843319066 >> v1:172.21.15.184:6789/0 conn(0x7efee41631f0 legacy=0x7efee41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348841939 0 0) 0x7efee4163ec0 con 0x7efee41631f0 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.373+0000 7efee990c640 1 -- 172.21.15.40:0/2843319066 >> v1:172.21.15.40:6789/0 conn(0x7efee40d1170 legacy=0x7efee40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.374+0000 7efee990c640 1 -- 172.21.15.40:0/2843319066 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40d1a40 con 0x7efee40b2000 2024-04-26T21:13:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.374+0000 7efee990c640 1 --2- 172.21.15.40:0/2843319066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee40b2000 0x7efee406ea30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efee00097a0 tx=0x7efee002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd898e6bada73ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.374+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0037070 con 0x7efee40b2000 2024-04-26T21:13:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.374+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee0032440 con 0x7efee40b2000 2024-04-26T21:13:08.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.374+0000 7efee910b640 1 -- 172.21.15.40:0/2843319066 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0032810 con 0x7efee40b2000 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 -- 172.21.15.40:0/2843319066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee40b2000 msgr2=0x7efee406ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 --2- 172.21.15.40:0/2843319066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee40b2000 0x7efee406ea30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efee00097a0 tx=0x7efee002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 -- 172.21.15.40:0/2843319066 shutdown_connections 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 --2- 172.21.15.40:0/2843319066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee40b2000 0x7efee406ea30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 -- 172.21.15.40:0/2843319066 >> 172.21.15.40:0/2843319066 conn(0x7efee409ba40 msgr2=0x7efee409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 -- 172.21.15.40:0/2843319066 shutdown_connections 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.375+0000 7efeec371640 1 -- 172.21.15.40:0/2843319066 wait complete. 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.376+0000 7efeec371640 1 Processor -- start 2024-04-26T21:13:08.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.376+0000 7efeec371640 1 -- start start 2024-04-26T21:13:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 0x7efee40b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 0x7efee409fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efee40d0230 con 0x7efee40d1170 2024-04-26T21:13:08.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efee40d2000 con 0x7efee41631f0 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeec371640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efee40b8be0 con 0x7efee40b2000 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37782/0 (socket says 172.21.15.40:37782) 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 -- 172.21.15.40:0/1506234257 learned_addr learned my addr 172.21.15.40:0/1506234257 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeea10d640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 0x7efee40b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeea90e640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 0x7efee409fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 -- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 msgr2=0x7efee40b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 0x7efee40b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 -- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 msgr2=0x7efee409fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 0x7efee409fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efee990c640 1 -- 172.21.15.40:0/1506234257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee4161a50 con 0x7efee40d1170 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeea10d640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 0x7efee40b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.377+0000 7efeea90e640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 0x7efee409fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efee990c640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efee0035040 tx=0x7efee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0041070 con 0x7efee40d1170 2024-04-26T21:13:08.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee0004240 con 0x7efee40d1170 2024-04-26T21:13:08.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee4161770 con 0x7efee40d1170 2024-04-26T21:13:08.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0037040 con 0x7efee40d1170 2024-04-26T21:13:08.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.378+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee414b4d0 con 0x7efee40d1170 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.379+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee406a290 con 0x7efee40d1170 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.379+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efee003a070 con 0x7efee40d1170 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.380+0000 7efed6ffd640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 0x7efec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.380+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7efee00ac0e0 con 0x7efee40d1170 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.380+0000 7efeea10d640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 0x7efec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.380+0000 7efeea10d640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 0x7efec806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efed8000f60 tx=0x7efed8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.384+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efee007a5a0 con 0x7efee40d1170 2024-04-26T21:13:08.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.570+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7efee40b8f10 con 0x7efee40d1170 2024-04-26T21:13:08.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.573+0000 7efed6ffd640 1 -- 172.21.15.40:0/1506234257 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7efee0079ea0 con 0x7efee40d1170 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.575+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 msgr2=0x7efec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.575+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 0x7efec806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efed8000f60 tx=0x7efed8009040 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.575+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 msgr2=0x7efee409f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.575+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efee0035040 tx=0x7efee00046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 shutdown_connections 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efec806c8f0 0x7efec806edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efee41631f0 0x7efee409fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efee40d1170 0x7efee409f690 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 --2- 172.21.15.40:0/1506234257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efee40b2000 0x7efee40b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 >> 172.21.15.40:0/1506234257 conn(0x7efee409ba40 msgr2=0x7efee4070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:08.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 shutdown_connections 2024-04-26T21:13:08.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.576+0000 7efeec371640 1 -- 172.21.15.40:0/1506234257 wait complete. 2024-04-26T21:13:08.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2024-04-26T21:13:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.690+0000 7f1c91c69640 1 Processor -- start 2024-04-26T21:13:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.690+0000 7f1c91c69640 1 -- start start 2024-04-26T21:13:08.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c91c69640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c91c69640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0ceff0 0x7f1c8c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c91c69640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c8c15dca0 con 0x7f1c8c143840 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c91c69640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1c8c143c20 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c91c69640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c15df80 con 0x7f1c8c0ceff0 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37794/0 (socket says 172.21.15.40:37794) 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 -- 172.21.15.40:0/2908754360 learned_addr learned my addr 172.21.15.40:0/2908754360 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8bfff640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0ceff0 0x7f1c8c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 -- 172.21.15.40:0/2908754360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0ceff0 msgr2=0x7f1c8c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0ceff0 0x7f1c8c0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 -- 172.21.15.40:0/2908754360 >> v1:172.21.15.184:6789/0 conn(0x7f1c8c143840 legacy=0x7f1c8c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.691+0000 7f1c8affd640 1 -- 172.21.15.40:0/2908754360 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c15e260 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.692+0000 7f1c8affd640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009870 tx=0x7f1c8002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b404dacd85612dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.692+0000 7f1c8a7fc640 1 -- 172.21.15.40:0/2908754360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80037070 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.692+0000 7f1c8a7fc640 1 -- 172.21.15.40:0/2908754360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c80032440 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.692+0000 7f1c8a7fc640 1 -- 172.21.15.40:0/2908754360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80032810 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 -- 172.21.15.40:0/2908754360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c0b4eb0 msgr2=0x7f1c8c0b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009870 tx=0x7f1c8002a210 comp rx=0 tx=0).stop 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 -- 172.21.15.40:0/2908754360 shutdown_connections 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0ceff0 0x7f1c8c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2908754360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c0b4eb0 0x7f1c8c0b5330 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 -- 172.21.15.40:0/2908754360 >> 172.21.15.40:0/2908754360 conn(0x7f1c8c0cd8a0 msgr2=0x7f1c8c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 -- 172.21.15.40:0/2908754360 shutdown_connections 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.693+0000 7f1c91c69640 1 -- 172.21.15.40:0/2908754360 wait complete. 2024-04-26T21:13:08.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.694+0000 7f1c91c69640 1 Processor -- start 2024-04-26T21:13:08.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.694+0000 7f1c91c69640 1 -- start start 2024-04-26T21:13:08.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 0x7f1c8c132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c8c0ceff0 0x7f1c8c0abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c15d8c0 con 0x7f1c8c143840 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c0b46f0 con 0x7f1c8c0ceff0 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c91c69640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c15da40 con 0x7f1c8c0b4eb0 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37808/0 (socket says 172.21.15.40:37808) 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 -- 172.21.15.40:0/2329526733 learned_addr learned my addr 172.21.15.40:0/2329526733 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8affd640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c8c0ceff0 0x7f1c8c0abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8b7fe640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 0x7f1c8c132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 -- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 msgr2=0x7f1c8c132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 0x7f1c8c132ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 -- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c8c0ceff0 msgr2=0x7f1c8c0abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c8c0ceff0 0x7f1c8c0abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8bfff640 1 -- 172.21.15.40:0/2329526733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c145730 con 0x7f1c8c143840 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.695+0000 7f1c8b7fe640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 0x7f1c8c132ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c8bfff640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c84007e40 tx=0x7f1c8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84014620 con 0x7f1c8c143840 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84019070 con 0x7f1c8c143840 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c1447b0 con 0x7f1c8c143840 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84014a20 con 0x7f1c8c143840 2024-04-26T21:13:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.696+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1459d0 con 0x7f1c8c143840 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.697+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8c06a290 con 0x7f1c8c143840 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.697+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c84014bc0 con 0x7f1c8c143840 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.698+0000 7f1c73fff640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 0x7f1c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.698+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1c8408c3f0 con 0x7f1c8c143840 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.698+0000 7f1c8b7fe640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 0x7f1c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.699+0000 7f1c8b7fe640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 0x7f1c7806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c006fd0 tx=0x7f1c7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:08.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.702+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c8405a790 con 0x7f1c8c143840 2024-04-26T21:13:08.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.888+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) -- 0x7f1c8c0ce420 con 0x7f1c8c143840 2024-04-26T21:13:08.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.891+0000 7f1c73fff640 1 -- 172.21.15.40:0/2329526733 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) ==== 135+0+0 (secure 0 0 0) 0x7f1c8405a090 con 0x7f1c8c143840 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 msgr2=0x7f1c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 0x7f1c7806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c006fd0 tx=0x7f1c7c008040 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 msgr2=0x7f1c8c143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c84007e40 tx=0x7f1c8400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 shutdown_connections 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c7806c8f0 0x7f1c7806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c143840 0x7f1c8c143da0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c8c0ceff0 0x7f1c8c0abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 --2- 172.21.15.40:0/2329526733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c8c0b4eb0 0x7f1c8c132ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:08.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 >> 172.21.15.40:0/2329526733 conn(0x7f1c8c0cd8a0 msgr2=0x7f1c8c15ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 shutdown_connections 2024-04-26T21:13:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:08.894+0000 7f1c91c69640 1 -- 172.21.15.40:0/2329526733 wait complete. 2024-04-26T21:13:08.906 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:08.906 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> n = 25 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-04-26T21:13:08.922 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-04-26T21:13:08.923 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-04-26T21:13:08.924 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:13:09.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:09.169 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:09.169 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> n = 25 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2024-04-26T21:13:09.225 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-04-26T21:13:09.226 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-04-26T21:13:09.227 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:13:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:09.461 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:09.461 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> n = 25 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-04-26T21:13:09.517 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-04-26T21:13:09.518 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-04-26T21:13:09.519 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:13:09.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:09.751 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:09.751 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> n = 25 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-04-26T21:13:09.807 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.808 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-04-26T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:13:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:10.051 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:10.051 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:10.107 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:13:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:13:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:10.204 INFO:teuthology.orchestra.run.smithi040.stdout:client4835 2024-04-26T21:13:10.204 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'find . | wc') 2024-04-26T21:13:10.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:13:31.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:13:31.092 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:13:49.275 INFO:teuthology.orchestra.run.smithi040.stdout: 105 105 1490 2024-04-26T21:13:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:13:49.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:49.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.395+0000 7f989db6a640 1 Processor -- start 2024-04-26T21:13:49.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.395+0000 7f989db6a640 1 -- start start 2024-04-26T21:13:49.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f989db6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f989db6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f989806d2b0 0x7f989806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f989db6a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98980b4a10 con 0x7f989815f1e0 2024-04-26T21:13:49.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f989db6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f989806dc60 con 0x7f989806d2b0 2024-04-26T21:13:49.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f989db6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f989806eeb0 con 0x7f98980b44e0 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f9897fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f989806d2b0 0x7f989806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:60978/0 (socket says 172.21.15.40:60978) 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f98977fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f989815f1e0 0x7f98980ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37428/0 (socket says 172.21.15.40:37428) 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.396+0000 7f9896ffd640 1 -- 172.21.15.40:0/1421505222 learned_addr learned my addr 172.21.15.40:0/1421505222 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.397+0000 7f9896ffd640 1 -- 172.21.15.40:0/1421505222 >> v2:172.21.15.184:3300/0 conn(0x7f989806d2b0 msgr2=0x7f989806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.397+0000 7f9896ffd640 1 --2- 172.21.15.40:0/1421505222 >> v2:172.21.15.184:3300/0 conn(0x7f989806d2b0 0x7f989806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.397+0000 7f9896ffd640 1 -- 172.21.15.40:0/1421505222 >> v1:172.21.15.184:6789/0 conn(0x7f989815f1e0 legacy=0x7f98980ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.397+0000 7f9896ffd640 1 -- 172.21.15.40:0/1421505222 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98980b4cf0 con 0x7f98980b44e0 2024-04-26T21:13:49.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.397+0000 7f9896ffd640 1 --2- 172.21.15.40:0/1421505222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f988c009870 tx=0x7f988c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1177c1c454c5e3df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f98967fc640 1 -- 172.21.15.40:0/1421505222 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c037070 con 0x7f98980b44e0 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f98967fc640 1 -- 172.21.15.40:0/1421505222 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988c032440 con 0x7f98980b44e0 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f98967fc640 1 -- 172.21.15.40:0/1421505222 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c032810 con 0x7f98980b44e0 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 -- 172.21.15.40:0/1421505222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 msgr2=0x7f989806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 --2- 172.21.15.40:0/1421505222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f988c009870 tx=0x7f988c02a270 comp rx=0 tx=0).stop 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 -- 172.21.15.40:0/1421505222 shutdown_connections 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 --2- 172.21.15.40:0/1421505222 >> v2:172.21.15.184:3300/0 conn(0x7f989806d2b0 0x7f989806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 --2- 172.21.15.40:0/1421505222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98980b44e0 0x7f989806e970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 -- 172.21.15.40:0/1421505222 >> 172.21.15.40:0/1421505222 conn(0x7f98980728a0 msgr2=0x7f9898072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 -- 172.21.15.40:0/1421505222 shutdown_connections 2024-04-26T21:13:49.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.398+0000 7f989db6a640 1 -- 172.21.15.40:0/1421505222 wait complete. 2024-04-26T21:13:49.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.399+0000 7f989db6a640 1 Processor -- start 2024-04-26T21:13:49.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 -- start start 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 0x7f98981437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 0x7f98981621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98980b5590 con 0x7f98980b44e0 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98980b52b0 con 0x7f989806d2b0 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.400+0000 7f989db6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f989815f5c0 con 0x7f989815f1e0 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50470/0 (socket says 172.21.15.40:50470) 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 -- 172.21.15.40:0/3731468248 learned_addr learned my addr 172.21.15.40:0/3731468248 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9897fff640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 0x7f98981621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f98977fe640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 0x7f98981437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 -- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 msgr2=0x7f98981621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 0x7f98981621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 -- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 msgr2=0x7f98981437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 0x7f98981437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 -- 172.21.15.40:0/3731468248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989815c5c0 con 0x7f98980b44e0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9897fff640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 0x7f98981621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f98977fe640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 0x7f98981437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f9896ffd640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f988c02a2b0 tx=0x7f988c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c037040 con 0x7f98980b44e0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989815c8a0 con 0x7f98980b44e0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988c042070 con 0x7f98980b44e0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c030880 con 0x7f98980b44e0 2024-04-26T21:13:49.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.401+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989815ce60 con 0x7f98980b44e0 2024-04-26T21:13:49.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.402+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989806a290 con 0x7f98980b44e0 2024-04-26T21:13:49.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.403+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f988c030a20 con 0x7f98980b44e0 2024-04-26T21:13:49.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.403+0000 7f987ffff640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 0x7f988406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.403+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f988c0ac490 con 0x7f98980b44e0 2024-04-26T21:13:49.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.404+0000 7f98977fe640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 0x7f988406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.404+0000 7f98977fe640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 0x7f988406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.408+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f988c045360 con 0x7f98980b44e0 2024-04-26T21:13:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.633+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9898163640 con 0x7f98980b44e0 2024-04-26T21:13:49.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.634+0000 7f987ffff640 1 -- 172.21.15.40:0/3731468248 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f988c07a840 con 0x7f98980b44e0 2024-04-26T21:13:49.635 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:49.635 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:49.636 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.636+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 msgr2=0x7f988406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.636+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 0x7f988406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.636+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 msgr2=0x7f9898154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.636+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f988c02a2b0 tx=0x7f988c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 shutdown_connections 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f988406c8f0 0x7f988406edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f989815f1e0 0x7f98981621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98980b44e0 0x7f9898154860 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 --2- 172.21.15.40:0/3731468248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989806d2b0 0x7f98981437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 >> 172.21.15.40:0/3731468248 conn(0x7f98980728a0 msgr2=0x7f98980af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 shutdown_connections 2024-04-26T21:13:49.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.637+0000 7f989db6a640 1 -- 172.21.15.40:0/3731468248 wait complete. 2024-04-26T21:13:49.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-04-26T21:13:49.766 INFO:teuthology.orchestra.run.smithi040.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":135,"msgr_send_messages":137,"msgr_recv_bytes":33989,"msgr_send_bytes":499226,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.014600966,"msgr_running_send_time":0.005262389,"msgr_running_recv_time":0.004471990,"msgr_running_fast_dispatch_time":0.002857100,"msgr_send_messages_queue_lat":{"avgcount":135,"sum":0.075119050,"avgtime":0.000556437},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":471,"msgr_send_messages":536,"msgr_recv_bytes":222477,"msgr_send_bytes":281451,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.038845871,"msgr_running_send_time":0.018201611,"msgr_running_recv_time":0.025173436,"msgr_running_fast_dispatch_time":0.002648600,"msgr_send_messages_queue_lat":{"avgcount":536,"sum":0.046580802,"avgtime":0.000086904},"msgr_handle_ack_lat":{"avgcount":467,"sum":0.000095359,"avgtime":0.000000204},"msgr_recv_encrypted_bytes":118816,"msgr_send_encrypted_bytes":18560},"AsyncMessenger::Worker-2":{"msgr_recv_messages":64,"msgr_send_messages":41,"msgr_recv_bytes":22195,"msgr_send_bytes":124151,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.010988884,"msgr_running_send_time":0.002317078,"msgr_running_recv_time":0.005571433,"msgr_running_fast_dispatch_time":0.002727598,"msgr_send_messages_queue_lat":{"avgcount":40,"sum":0.114388820,"avgtime":0.002859720},"msgr_handle_ack_lat":{"avgcount":15,"sum":0.000005437,"avgtime":0.000000362},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":108400},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":263,"sum":0.096263060,"avgtime":0.000366019}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000018314,"avgtime":0.000009157}},"mds":{"request":148,"reply":148,"reply_latency":{"avgcount":148,"sum":39.094707386,"avgtime":0.264153428},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":105,"caps":106,"subtrees":2,"traverse":188,"traverse_hit":174,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":4,"ceph_cap_op_grant":112,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":104,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":14},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":219,"evex":0,"evtrm":0,"ev":219,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4639770,"rdpos":0,"jlat":{"avgcount":109,"sum":0.173814692,"avgtime":0.001594630},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":115,"dn-":1,"cap":101,"cap+":106,"cap-":5,"rss":50964,"heap":231788},"mds_server":{"dispatch_client_request":230,"dispatch_server_request":0,"handle_client_request":148,"handle_client_session":106,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.063563830,"avgtime":0.000635638},"req_getattr_latency":{"avgcount":32,"sum":0.007222377,"avgtime":0.000225699},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":4,"sum":0.000400633,"avgtime":0.000100158},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.005185441,"avgtime":0.001296360},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000281211,"avgtime":0.000140605},"req_readdir_latency":{"avgcount":5,"sum":39.017630666,"avgtime":7.803526133},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000423228,"avgtime":0.000423228},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":93,"average_load":46,"avg_session_uptime":41,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":238452,"buffer_anon_items":213,"buffer_meta_bytes":1584,"buffer_meta_items":18,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":420136,"mds_co_items":4385,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":151,"op_send_bytes":463245,"op_resend":37,"op_reply":151,"op_latency":{"avgcount":151,"sum":1.068987133,"avgtime":0.007079384},"op_inflight":0,"oplen_avg":{"avgcount":151,"sum":178},"op":151,"op_r":0,"op_w":151,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":109,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":36,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":20,"omap_rd":0,"omap_del":2},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":2},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":666,"get_sum":220946,"get_or_fail_fail":0,"get_or_fail_success":666,"take":0,"take_sum":0,"put":666,"put_sum":220946,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":151,"take_sum":463245,"put":151,"put_sum":463245,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":151,"take_sum":151,"put":151,"put_sum":151,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55c6b1852500":{"val":0,"max":3758096384,"get_started":0,"get":219,"get_sum":445466,"get_or_fail_fail":0,"get_or_fail_success":219,"take":0,"take_sum":0,"put":109,"put_sum":445466,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-26T21:13:49.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 33989, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 135, "msgr_running_fast_dispatch_time": 0.0028571, "msgr_running_recv_time": 0.00447199, "msgr_running_send_time": 0.005262389, "msgr_running_total_time": 0.014600966, "msgr_send_bytes": 499226, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 137, "msgr_send_messages_queue_lat": { "avgcount": 135, "avgtime": 0.000556437, "sum": 0.07511905 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 467, "avgtime": 2.04e-07, "sum": 9.5359e-05 }, "msgr_recv_bytes": 222477, "msgr_recv_encrypted_bytes": 118816, "msgr_recv_messages": 471, "msgr_running_fast_dispatch_time": 0.0026486, "msgr_running_recv_time": 0.025173436, "msgr_running_send_time": 0.018201611, "msgr_running_total_time": 0.038845871, "msgr_send_bytes": 281451, "msgr_send_encrypted_bytes": 18560, "msgr_send_messages": 536, "msgr_send_messages_queue_lat": { "avgcount": 536, "avgtime": 8.6904e-05, "sum": 0.046580802 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 15, "avgtime": 3.62e-07, "sum": 5.437e-06 }, "msgr_recv_bytes": 22195, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 64, "msgr_running_fast_dispatch_time": 0.002727598, "msgr_running_recv_time": 0.005571433, "msgr_running_send_time": 0.002317078, "msgr_running_total_time": 0.010988884, "msgr_send_bytes": 124151, "msgr_send_encrypted_bytes": 108400, "msgr_send_messages": 41, "msgr_send_messages_queue_lat": { "avgcount": 40, "avgtime": 0.00285972, "sum": 0.11438882 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 263, "avgtime": 0.000366019, "sum": 0.09626306 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.157e-06, "sum": 1.8314e-05 }, "queue_len": 0 }, "mds": { "caps": 106, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 112, "ceph_cap_op_revoke": 4, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 104, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 105, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 14, "q": 0, "reply": 148, "reply_latency": { "avgcount": 148, "avgtime": 0.264153428, "sum": 39.094707386 }, "request": 148, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 188, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 174, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 219, "evadd": 219, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 109, "avgtime": 0.00159463, "sum": 0.173814692 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4639770 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 115, "dn-": 1, "heap": 231788, "ino": 117, "ino+": 117, "ino-": 0, "rss": 50964 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 230, "dispatch_server_request": 0, "handle_client_request": 148, "handle_client_session": 106, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000635638, "sum": 0.06356383 }, "req_getattr_latency": { "avgcount": 32, "avgtime": 0.000225699, "sum": 0.007222377 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 4, "avgtime": 0.000100158, "sum": 0.000400633 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.00129636, "sum": 0.005185441 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000140605, "sum": 0.000281211 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.803526133, "sum": 39.017630666 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000423228, "sum": 0.000423228 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 46, "avg_session_uptime": 41, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 93 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 238452, "buffer_anon_items": 213, "buffer_meta_bytes": 1584, "buffer_meta_items": 18, "mds_co_bytes": 420136, "mds_co_items": 4385, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 36, "map_full": 0, "map_inc": 1, "omap_del": 2, "omap_rd": 0, "omap_wr": 20, "op": 151, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 151, "avgtime": 0.007079384, "sum": 1.068987133 }, "op_pg": 0, "op_r": 0, "op_reply": 151, "op_resend": 37, "op_rmw": 0, "op_send": 151, "op_send_bytes": 463245, "op_w": 151, "oplen_avg": { "avgcount": 151, "sum": 178 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 109, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 666, "get_or_fail_fail": 0, "get_or_fail_success": 666, "get_started": 0, "get_sum": 220946, "max": 104857600, "put": 666, "put_sum": 220946, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 151, "put_sum": 463245, "take": 151, "take_sum": 463245, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 151, "put_sum": 151, "take": 151, "take_sum": 151, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55c6b1852500": { "get": 219, "get_or_fail_fail": 0, "get_or_fail_success": 219, "get_started": 0, "get_sum": 445466, "max": 3758096384, "put": 109, "put_sum": 445466, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-26T21:13:49.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:49.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 Processor -- start 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 -- start start 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b980d5610 0x7f4b980b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b980b2510 con 0x7f4b9815f720 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b980b27f0 con 0x7f4b9815f340 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.869+0000 7f4b9f589640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4b980b2ad0 con 0x7f4b980d5610 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b980d5610 0x7f4b980b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b980d5610 0x7f4b980b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50480/0 (socket says 172.21.15.40:50480) 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 -- 172.21.15.40:0/4270509149 learned_addr learned my addr 172.21.15.40:0/4270509149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9cb24640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4b9815f720 0x7f4b980b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45790/0 (socket says 172.21.15.40:45790) 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 -- 172.21.15.40:0/4270509149 >> v1:172.21.15.184:6789/0 conn(0x7f4b9815f340 legacy=0x7f4b980d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 -- 172.21.15.40:0/4270509149 >> v1:172.21.15.40:6789/0 conn(0x7f4b9815f720 legacy=0x7f4b980b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b8ffff640 1 -- 172.21.15.40:0/4270509149 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703725333 0 0) 0x7f4b980b2510 con 0x7f4b9815f720 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.870+0000 7f4b9db26640 1 -- 172.21.15.40:0/4270509149 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b980b2db0 con 0x7f4b980d5610 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9db26640 1 --2- 172.21.15.40:0/4270509149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b980b1e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009e50 tx=0x7f4b9002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d80b013814d921c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b8ffff640 1 -- 172.21.15.40:0/4270509149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90036070 con 0x7f4b980d5610 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b8ffff640 1 -- 172.21.15.40:0/4270509149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b90031600 con 0x7f4b980d5610 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b8ffff640 1 -- 172.21.15.40:0/4270509149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b900319d0 con 0x7f4b980d5610 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 -- 172.21.15.40:0/4270509149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 msgr2=0x7f4b980b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 --2- 172.21.15.40:0/4270509149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b980b1e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009e50 tx=0x7f4b9002a6d0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 -- 172.21.15.40:0/4270509149 shutdown_connections 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 --2- 172.21.15.40:0/4270509149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b980b1e80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 -- 172.21.15.40:0/4270509149 >> 172.21.15.40:0/4270509149 conn(0x7f4b9808ec70 msgr2=0x7f4b9809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.871+0000 7f4b9f589640 1 -- 172.21.15.40:0/4270509149 shutdown_connections 2024-04-26T21:13:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.872+0000 7f4b9f589640 1 -- 172.21.15.40:0/4270509149 wait complete. 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 Processor -- start 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 -- start start 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 0x7f4b980cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 0x7f4b980d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4b980d40d0 con 0x7f4b980d5610 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b980d3f50 con 0x7f4b9815f340 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9f589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4b980b3cc0 con 0x7f4b9815f720 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9d325640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9d325640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50484/0 (socket says 172.21.15.40:50484) 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.873+0000 7f4b9d325640 1 -- 172.21.15.40:0/954687205 learned_addr learned my addr 172.21.15.40:0/954687205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9cb24640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 0x7f4b980cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9db26640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 0x7f4b980d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 -- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 msgr2=0x7f4b980d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 0x7f4b980d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 -- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 msgr2=0x7f4b980cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 0x7f4b980cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 -- 172.21.15.40:0/954687205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b980b8430 con 0x7f4b980d5610 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9cb24640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 0x7f4b980cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9db26640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 0x7f4b980d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9d325640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400c460 tx=0x7f4b9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94016020 con 0x7f4b980d5610 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b94012870 con 0x7f4b980d5610 2024-04-26T21:13:49.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94012c60 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b980b8710 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.874+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b980b8cd0 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.875+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9806a290 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.875+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b94019020 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.876+0000 7f4b8dffb640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 0x7f4b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.876+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f4b9408bad0 con 0x7f4b980d5610 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.876+0000 7f4b9cb24640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 0x7f4b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:49.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.877+0000 7f4b9cb24640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 0x7f4b7c06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000f60 tx=0x7f4b88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:49.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:49.880+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b94020020 con 0x7f4b980d5610 2024-04-26T21:13:50.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.103+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b980b49f0 con 0x7f4b980d5610 2024-04-26T21:13:50.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.104+0000 7f4b8dffb640 1 -- 172.21.15.40:0/954687205 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f4b940182c0 con 0x7f4b980d5610 2024-04-26T21:13:50.105 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:50.105 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:50.105 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 msgr2=0x7f4b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 0x7f4b7c06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000f60 tx=0x7f4b88009040 comp rx=0 tx=0).stop 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 msgr2=0x7f4b981590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400c460 tx=0x7f4b9400c920 comp rx=0 tx=0).stop 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 shutdown_connections 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4b7c06c8f0 0x7f4b7c06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b9815f720 0x7f4b980d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9815f340 0x7f4b980cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 --2- 172.21.15.40:0/954687205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b980d5610 0x7f4b981590b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 >> 172.21.15.40:0/954687205 conn(0x7f4b9808ec70 msgr2=0x7f4b980b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 shutdown_connections 2024-04-26T21:13:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.106+0000 7f4b9f589640 1 -- 172.21.15.40:0/954687205 wait complete. 2024-04-26T21:13:50.118 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-26T21:13:50.237 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":4844,"entity":{"name":{"type":"client","num":4844},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1961955384}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":42.156492749999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":8.1883888684198336,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.4844 v1:192.168.0.1:0/1961955384","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":4835,"entity":{"name":{"type":"client","num":4835},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1587806991}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":90,"uptime":43.3074789,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":12.909432656485032,"halflife":60},"release_caps":{"value":8.2999063389260908,"halflife":60},"recall_caps_throttle":{"value":4.1928498037319821,"halflife":1.5},"recall_caps_throttle2o":{"value":2.9484102170721838,"halflife":0.5},"session_cache_liveness":{"value":199.77939831413175,"halflife":300},"cap_acquisition":{"value":60.437054967257808,"halflife":30},"last_trim_completed_requests_tid":144,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.4835 v1:172.21.15.40:0/1587806991","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:13:50.238 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1961955384, "type": "v1" }, "name": { "num": 4844, "type": "client" } }, "id": 4844, "inst": "client.4844 v1:192.168.0.1:0/1961955384", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 8.188388868419834 }, "state": "open", "uptime": 42.15649275 }, { "cap_acquisition": { "halflife": 30, "value": 60.43705496725781 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1587806991, "type": "v1" }, "name": { "num": 4835, "type": "client" } }, "id": 4835, "inst": "client.4835 v1:172.21.15.40:0/1587806991", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 144, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 12.909432656485032 }, "recall_caps_throttle": { "halflife": 1.5, "value": 4.192849803731982 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 2.9484102170721838 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 8.29990633892609 }, "request_load_avg": 90, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 199.77939831413175 }, "state": "open", "uptime": 43.3074789 } ] 2024-04-26T21:13:50.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2024-04-26T21:13:50.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.339+0000 7f08b5504640 1 Processor -- start 2024-04-26T21:13:50.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.339+0000 7f08b5504640 1 -- start start 2024-04-26T21:13:50.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08b5504640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08b013ff70 0x7f08b0146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08b5504640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 0x7f08b01465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08b5504640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00d05b0 con 0x7f08b0140350 2024-04-26T21:13:50.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08b5504640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f08b00d0890 con 0x7f08b013ff70 2024-04-26T21:13:50.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08b5504640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f08b006e450 con 0x7f08b00cff20 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08b013ff70 0x7f08b0146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08b013ff70 0x7f08b0146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50500/0 (socket says 172.21.15.40:50500) 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 -- 172.21.15.40:0/3714933422 learned_addr learned my addr 172.21.15.40:0/3714933422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08af7fe640 1 --2- 172.21.15.40:0/3714933422 >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 0x7f08b01465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 -- 172.21.15.40:0/3714933422 >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 msgr2=0x7f08b01465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 --2- 172.21.15.40:0/3714933422 >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 0x7f08b01465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 -- 172.21.15.40:0/3714933422 >> v1:172.21.15.184:6789/0 conn(0x7f08b0140350 legacy=0x7f08b00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08aeffd640 1 -- 172.21.15.40:0/3714933422 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b006e730 con 0x7f08b013ff70 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.340+0000 7f08af7fe640 1 --2- 172.21.15.40:0/3714933422 >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 0x7f08b01465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.341+0000 7f08aeffd640 1 --2- 172.21.15.40:0/3714933422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b013ff70 0x7f08b0146080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f08b00cec30 tx=0x7f08a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb3ee80dd0fa885 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.341+0000 7f08adffb640 1 -- 172.21.15.40:0/3714933422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8037070 con 0x7f08b013ff70 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.341+0000 7f08adffb640 1 -- 172.21.15.40:0/3714933422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a8032440 con 0x7f08b013ff70 2024-04-26T21:13:50.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.341+0000 7f08adffb640 1 -- 172.21.15.40:0/3714933422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8032810 con 0x7f08b013ff70 2024-04-26T21:13:50.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 -- 172.21.15.40:0/3714933422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b013ff70 msgr2=0x7f08b0146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 --2- 172.21.15.40:0/3714933422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b013ff70 0x7f08b0146080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f08b00cec30 tx=0x7f08a802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 -- 172.21.15.40:0/3714933422 shutdown_connections 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 --2- 172.21.15.40:0/3714933422 >> v2:172.21.15.184:3300/0 conn(0x7f08b00cff20 0x7f08b01465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 --2- 172.21.15.40:0/3714933422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b013ff70 0x7f08b0146080 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 -- 172.21.15.40:0/3714933422 >> 172.21.15.40:0/3714933422 conn(0x7f08b00d0d30 msgr2=0x7f08b00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 -- 172.21.15.40:0/3714933422 shutdown_connections 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.342+0000 7f08b5504640 1 -- 172.21.15.40:0/3714933422 wait complete. 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.343+0000 7f08b5504640 1 Processor -- start 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.343+0000 7f08b5504640 1 -- start start 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08b013ff70 0x7f08b015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08b0140350 0x7f08b00738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08b006efe0 con 0x7f08b00cff20 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08b006ee60 con 0x7f08b013ff70 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08b5504640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f08b006f160 con 0x7f08b0140350 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50510/0 (socket says 172.21.15.40:50510) 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 -- 172.21.15.40:0/34079235 learned_addr learned my addr 172.21.15.40:0/34079235 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 -- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08b0140350 msgr2=0x7f08b00738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08af7fe640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08b0140350 0x7f08b00738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08b0140350 0x7f08b00738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 -- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08b013ff70 msgr2=0x7f08b015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08b013ff70 0x7f08b015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08b013ff70 0x7f08b015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.344+0000 7f08aeffd640 1 -- 172.21.15.40:0/34079235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01f57f0 con 0x7f08b00cff20 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08aeffd640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08a77fe640 1 -- 172.21.15.40:0/34079235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8037040 con 0x7f08b00cff20 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08a77fe640 1 -- 172.21.15.40:0/34079235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a8042070 con 0x7f08b00cff20 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01f5ad0 con 0x7f08b00cff20 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08a77fe640 1 -- 172.21.15.40:0/34079235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8030bb0 con 0x7f08b00cff20 2024-04-26T21:13:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.345+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01f6090 con 0x7f08b00cff20 2024-04-26T21:13:50.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.346+0000 7f08a77fe640 1 -- 172.21.15.40:0/34079235 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08a8047460 con 0x7f08b00cff20 2024-04-26T21:13:50.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.347+0000 7f08a77fe640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 0x7f089006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.347+0000 7f08a77fe640 1 -- 172.21.15.40:0/34079235 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f08a80ac7a0 con 0x7f08b00cff20 2024-04-26T21:13:50.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.347+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 0x7f089006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.347+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 0x7f089006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08b0160490 tx=0x7f0898008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 Processor -- start 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 -- start start 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce83a20 0x56082ce58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56082cdbd740 con 0x56082ce84160 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56082cd59670 con 0x56082ce83a20 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.348+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56082cdb9700 con 0x56082cea1040 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50516/0 (socket says 172.21.15.40:50516) 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1979285134 learned_addr learned my addr 172.21.15.40:0/1979285134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08af7fe640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce83a20 0x56082ce58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 msgr2=0x56082ce53c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce53c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce83a20 msgr2=0x56082ce58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce83a20 0x56082ce58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1979285134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082cacf3f0 con 0x56082ce84160 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce53c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f089800c930 tx=0x7f089800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4455d3efad9e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08a4ff9640 1 -- 172.21.15.40:0/1979285134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898019040 con 0x56082ce84160 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08a4ff9640 1 -- 172.21.15.40:0/1979285134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898022070 con 0x56082ce84160 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.349+0000 7f08a4ff9640 1 -- 172.21.15.40:0/1979285134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898013dc0 con 0x56082ce84160 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- 172.21.15.40:0/1979285134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 msgr2=0x56082ce834e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f089800c930 tx=0x7f089800f000 comp rx=0 tx=0).stop 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- 172.21.15.40:0/1979285134 shutdown_connections 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce83a20 0x56082ce58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce834e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 --2- 172.21.15.40:0/1979285134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce53c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- 172.21.15.40:0/1979285134 >> 172.21.15.40:0/1979285134 conn(0x56082ce55ed0 msgr2=0x56082ce9e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- 172.21.15.40:0/1979285134 shutdown_connections 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- 172.21.15.40:0/1979285134 wait complete. 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.350+0000 7f08b8219740 1 -- start start 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 Processor -- start 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 0x56082ce83130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce71740 0x56082ce708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56082ce57df0 con 0x56082cea1040 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56082cd57310 con 0x56082ce84160 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56082ce571e0 con 0x56082ce71740 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08b8219740 10 client.0 _ll_register_callbacks cb 0x56082cd442b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:13:50.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50532/0 (socket says 172.21.15.40:50532) 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.351+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1843137078 learned_addr learned my addr 172.21.15.40:0/1843137078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08af7fe640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce71740 0x56082ce708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 0x56082ce83130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce71740 msgr2=0x56082ce708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce71740 0x56082ce708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 msgr2=0x56082ce83130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 0x56082ce83130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 -- 172.21.15.40:0/1843137078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082cb97760 con 0x56082cea1040 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 0x56082ce83130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08980137d0 tx=0x7f0898014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aa7e0c41c98f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898013e50 con 0x56082cea1040 2024-04-26T21:13:50.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f0898004020 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0898022070 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f0898038050 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 10 client.5367 fetch_fsmap learned FSMap version 14 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x56082ce854d0 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898022070 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f089803b020 con 0x56082cea1040 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 10 client.5367 fetch_fsmap finished waiting for FSMap version 14 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 10 client.5367 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.352+0000 7f08b8219740 20 client.5367 populate_metadata read hostname 'smithi040' 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce85cd0 0x56082ce860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08b8219740 4 client.5367 mds_command: new command op to 5145 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 --> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082cdcb350 con 0x56082ce85cd0 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce85cd0 0x56082ce860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce85cd0 0x56082ce860b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9576dd3307c37e server_cookie=20e8187847a0b6e in_seq=0 out_seq=0 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f088dffb640 10 client.5367 ms_handle_connect on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f088dffb640 1 -- 172.21.15.40:0/1843137078 <== mds.0 v2:172.21.15.40:6832/1813208926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56082cdcb350 con 0x56082ce85cd0 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f088dffb640 10 client.5367 handle_command_reply: tid=0 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.353+0000 7f08b8219740 1 client.5367 shutdown 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 msgr2=0x56082ce9e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08980137d0 tx=0x7f0898014000 comp rx=0 tx=0).stop 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 shutdown_connections 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce85cd0 0x56082ce860b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce71740 0x56082ce708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f088dffb640 0 client.5367 ms_handle_reset on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082cea1040 0x56082ce9e8e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 --2- 172.21.15.40:0/1843137078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce84160 0x56082ce83130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 >> 172.21.15.40:0/1843137078 conn(0x56082ce83a20 msgr2=0x56082ce9f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 shutdown_connections 2024-04-26T21:13:50.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 1 -- 172.21.15.40:0/1843137078 wait complete. 2024-04-26T21:13:50.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.354+0000 7f08b8219740 20 client.5367 trim_cache size 0 max 16384 2024-04-26T21:13:50.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.368+0000 7f08b8219740 1 Processor -- start 2024-04-26T21:13:50.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.368+0000 7f08b8219740 1 -- start start 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce71740 0x56082ce82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce7e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01450b0 con 0x56082ce84160 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08b013e800 con 0x56082ce71740 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f08b01f97a0 con 0x56082cea1040 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50542/0 (socket says 172.21.15.40:50542) 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 -- 172.21.15.40:0/2540326446 learned_addr learned my addr 172.21.15.40:0/2540326446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 -- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 msgr2=0x56082ce7e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce7e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 -- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce71740 msgr2=0x56082ce82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce71740 0x56082ce82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.369+0000 7f08ae7fc640 1 -- 172.21.15.40:0/2540326446 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082ce95b10 con 0x56082ce84160 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f089800a480 tx=0x7f0898016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad199ec725aa7945 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f088ffff640 1 -- 172.21.15.40:0/2540326446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800c560 con 0x56082ce84160 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f088ffff640 1 -- 172.21.15.40:0/2540326446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898041070 con 0x56082ce84160 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f088ffff640 1 -- 172.21.15.40:0/2540326446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898013e50 con 0x56082ce84160 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 -- 172.21.15.40:0/2540326446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 msgr2=0x56082ce52af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f089800a480 tx=0x7f0898016000 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 -- 172.21.15.40:0/2540326446 shutdown_connections 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082cea1040 0x56082ce7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce84160 0x56082ce52af0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 --2- 172.21.15.40:0/2540326446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082ce71740 0x56082ce82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 -- 172.21.15.40:0/2540326446 >> 172.21.15.40:0/2540326446 conn(0x56082ce83a20 msgr2=0x56082ce9fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 -- 172.21.15.40:0/2540326446 shutdown_connections 2024-04-26T21:13:50.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.370+0000 7f08b8219740 1 -- 172.21.15.40:0/2540326446 wait complete. 2024-04-26T21:13:50.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.371+0000 7f08b8219740 1 -- start start 2024-04-26T21:13:50.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.371+0000 7f08b8219740 1 Processor -- start 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce84160 0x56082ce591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082cea1040 0x56082ce59730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08b013e1b0 con 0x56082ce71740 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56082ce82560 con 0x56082cea1040 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56082ce9d640 con 0x56082ce84160 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08b8219740 10 client.0 _ll_register_callbacks cb 0x7f08a00086d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50552/0 (socket says 172.21.15.40:50552) 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 -- 172.21.15.40:0/1225022745 learned_addr learned my addr 172.21.15.40:0/1225022745 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce84160 0x56082ce591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08af7fe640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082cea1040 0x56082ce59730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 -- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce84160 msgr2=0x56082ce591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce84160 0x56082ce591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 -- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082cea1040 msgr2=0x56082ce59730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082cea1040 0x56082ce59730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 -- 172.21.15.40:0/1225022745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082ce80070 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f08aeffd640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f08a802a1e0 tx=0x7f08a807ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25ff5f62a9cbf44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f088dffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.372+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80830b0 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f08a8030d50 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0898041070 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f08a8076b40 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 10 client.5373 fetch_fsmap learned FSMap version 14 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x56082ce870c0 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898041070 con 0x56082ce71740 2024-04-26T21:13:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f08a803a030 con 0x56082ce71740 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 10 client.5373 fetch_fsmap finished waiting for FSMap version 14 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 10 client.5373 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 20 client.5373 populate_metadata read hostname 'smithi040' 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce9e0d0 0x56082cea1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 4 client.5373 mds_command: new command op to 5145 tid=0 multi_id=0 [{"prefix": "dump_historic_ops"}] 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 --> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] -- command(tid 0: {"prefix": "dump_historic_ops"}) -- 0x56082cdc51c0 con 0x56082ce9e0d0 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce9e0d0 0x56082cea1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f08ae7fc640 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce9e0d0 0x56082cea1900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe5cc615005683e server_cookie=171900f178b633a7 in_seq=0 out_seq=0 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.373+0000 7f088dffb640 10 client.5373 ms_handle_connect on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f088dffb640 1 -- 172.21.15.40:0/1225022745 <== mds.0 v2:172.21.15.40:6832/1813208926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+71422 (crc 0 0 0) 0x56082cdc51c0 con 0x56082ce9e0d0 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f088dffb640 10 client.5373 handle_command_reply: tid=0 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 client.5373 shutdown 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 msgr2=0x56082ce9c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f08a802a1e0 tx=0x7f08a807ac90 comp rx=0 tx=0).stop 2024-04-26T21:13:50.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 shutdown_connections 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:6832/1813208926,v1:172.21.15.40:6834/1813208926] conn(0x56082ce9e0d0 0x56082cea1900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56082cea1040 0x56082ce59730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56082ce84160 0x56082ce591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 --2- 172.21.15.40:0/1225022745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56082ce71740 0x56082ce9c920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 >> 172.21.15.40:0/1225022745 conn(0x56082ce83a20 msgr2=0x56082ceb0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f088dffb640 0 client.5373 ms_handle_reset on v2:172.21.15.40:6832/1813208926 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 shutdown_connections 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 1 -- 172.21.15.40:0/1225022745 wait complete. 2024-04-26T21:13:50.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.375+0000 7f08b8219740 20 client.5373 trim_cache size 0 max 16384 2024-04-26T21:13:50.382 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:13:50.382 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-04-26T21:13:50.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-04-26T21:13:50.382 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:2 setattr mode=041777 #0x1 2024-04-26T21:13:06.990024+0000 caller_uid=0, caller_gid=0{0,})", 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:06.991327+0000", 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.383463534999997, 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0827849270000001, 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 2 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991327+0000", 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991327+0000", 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991329+0000", 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991342+0000", 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991393+0000", 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991652+0000", 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991756+0000", 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:06.991757+0000", 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:08.073967+0000", 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:08.074008+0000", 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:08.074092+0000", 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:08.074111+0000", 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:08.074112+0000", 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:27 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_0 2024-04-26T21:13:09.037000+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.037945+0000", 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.336845478000001, 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061171100000000003, 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 27 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.037945+0000", 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.037945+0000", 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.037946+0000", 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.037952+0000", 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.037969+0000", 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.038148+0000", 2024-04-26T21:13:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.038420+0000", 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.038420+0000", 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043506+0000", 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043731+0000", 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043887+0000", 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044060+0000", 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044062+0000", 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:28 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_1 2024-04-26T21:13:09.043000+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.043987+0000", 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.330803305000003, 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056566749999999999, 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 28 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043987+0000", 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043987+0000", 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043988+0000", 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.043993+0000", 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044011+0000", 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044283+0000", 2024-04-26T21:13:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044601+0000", 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.044601+0000", 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049116+0000", 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049320+0000", 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049477+0000", 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049642+0000", 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049644+0000", 2024-04-26T21:13:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:29 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_2 2024-04-26T21:13:09.048000+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.049567+0000", 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.325223080000001, 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.007846011, 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 29 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049567+0000", 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049567+0000", 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049568+0000", 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049572+0000", 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049591+0000", 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.049870+0000", 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.050184+0000", 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.050185+0000", 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.056886+0000", 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.057099+0000", 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.057244+0000", 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.057412+0000", 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.057413+0000", 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:31 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_4 2024-04-26T21:13:09.060999+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.062020+0000", 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.312770997999998, 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054195249999999997, 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 31 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062020+0000", 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062020+0000", 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062020+0000", 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062025+0000", 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062050+0000", 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062329+0000", 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062639+0000", 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.062639+0000", 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.066911+0000", 2024-04-26T21:13:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.067120+0000", 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.067271+0000", 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.067437+0000", 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.067439+0000", 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:34 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_7 2024-04-26T21:13:09.074999+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.076433+0000", 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.298357129999999, 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0052011590000000003, 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 34 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076433+0000", 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076433+0000", 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076434+0000", 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076439+0000", 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076457+0000", 2024-04-26T21:13:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.076726+0000", 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.077046+0000", 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.077046+0000", 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.081106+0000", 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.081317+0000", 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.081466+0000", 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.081633+0000", 2024-04-26T21:13:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.081635+0000", 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:38 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_11 2024-04-26T21:13:09.092999+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.094614+0000", 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.280176275000002, 2024-04-26T21:13:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069265669999999998, 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 38 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094614+0000", 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094614+0000", 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094615+0000", 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094621+0000", 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094641+0000", 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.094883+0000", 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.095181+0000", 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.095181+0000", 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.101252+0000", 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.101382+0000", 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.101462+0000", 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.101540+0000", 2024-04-26T21:13:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.101541+0000", 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:40 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_13 2024-04-26T21:13:09.103999+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.105669+0000", 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.269121237, 2024-04-26T21:13:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067425840000000003, 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 40 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105669+0000", 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105669+0000", 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105670+0000", 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105681+0000", 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105697+0000", 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.105942+0000", 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.106259+0000", 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.106260+0000", 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.111880+0000", 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.112098+0000", 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.112246+0000", 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.112410+0000", 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.112412+0000", 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:54 mkdir owner_uid=1000, owner_gid=1316 #0x1/dir2 2024-04-26T21:13:09.336996+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.338250+0000", 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 41.036540883999997, 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.005127688, 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 54 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338250+0000", 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338250+0000", 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338250+0000", 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338254+0000", 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338266+0000", 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338472+0000", 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "failed to wrlock, waiting" 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.338879+0000", 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.339148+0000", 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.339149+0000", 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.343015+0000", 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.343180+0000", 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.343319+0000", 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.343377+0000", 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.343377+0000", 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:78 create owner_uid=1000, owner_gid=1316 #0x1000000020e/file_23 2024-04-26T21:13:09.436995+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.438823+0000", 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.935967617999999, 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073111690000000002, 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 78 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.438823+0000", 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.438823+0000", 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.438824+0000", 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.438828+0000", 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.438851+0000", 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.439142+0000", 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.439465+0000", 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.439466+0000", 2024-04-26T21:13:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.445837+0000", 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.445973+0000", 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.446054+0000", 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.446134+0000", 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.446134+0000", 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:110 mkdir owner_uid=1000, owner_gid=1316 #0x1/dir4 2024-04-26T21:13:09.918989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.920244+0000", 2024-04-26T21:13:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.454546272000002, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054793460000000004, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 110 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920244+0000", 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920244+0000", 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920245+0000", 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920249+0000", 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920264+0000", 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920467+0000", 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "failed to wrlock, waiting" 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.920903+0000", 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.921165+0000", 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.921166+0000", 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.925473+0000", 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.925585+0000", 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.925663+0000", 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.925723+0000", 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.925724+0000", 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:112 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_1 2024-04-26T21:13:09.924989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.926086+0000", 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.448704919000001, 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051807540000000001, 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 112 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926086+0000", 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926086+0000", 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926086+0000", 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926092+0000", 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926111+0000", 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.926695+0000", 2024-04-26T21:13:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.927220+0000", 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.927221+0000", 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.930893+0000", 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.931031+0000", 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.931131+0000", 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.931265+0000", 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.931266+0000", 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:115 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_4 2024-04-26T21:13:09.937989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.939892+0000", 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.434898048999997, 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071567260000000004, 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 115 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.939892+0000", 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.939892+0000", 2024-04-26T21:13:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.939893+0000", 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.939899+0000", 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.939917+0000", 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.940274+0000", 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.940596+0000", 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.940597+0000", 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.946663+0000", 2024-04-26T21:13:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.946810+0000", 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.946932+0000", 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.947048+0000", 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.947049+0000", 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:117 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_6 2024-04-26T21:13:09.949989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.951040+0000", 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.423750693000002, 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063785789999999997, 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 117 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951040+0000", 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951040+0000", 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951041+0000", 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951047+0000", 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951066+0000", 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951419+0000", 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951739+0000", 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.951740+0000", 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.957065+0000", 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.957205+0000", 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.957305+0000", 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.957418+0000", 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.957418+0000", 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:119 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_8 2024-04-26T21:13:09.959989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.961876+0000", 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.412914405000002, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051507849999999997, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 119 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.961876+0000", 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.961876+0000", 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.961877+0000", 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.961884+0000", 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.961903+0000", 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.962260+0000", 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.962576+0000", 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.962577+0000", 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.966656+0000", 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.966794+0000", 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.966905+0000", 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.967026+0000", 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.967027+0000", 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:121 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_10 2024-04-26T21:13:09.969989+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.971545+0000", 2024-04-26T21:13:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.403245560999999, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066075209999999999, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 121 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971545+0000", 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971545+0000", 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971546+0000", 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971552+0000", 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971570+0000", 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.971909+0000", 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.972218+0000", 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.972218+0000", 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.977763+0000", 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.977917+0000", 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978018+0000", 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978151+0000", 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978152+0000", 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:122 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_11 2024-04-26T21:13:09.976988+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.978122+0000", 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.396668445000003, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0072095670000000001, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 122 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978122+0000", 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978122+0000", 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978123+0000", 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978129+0000", 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978162+0000", 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978510+0000", 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978821+0000", 2024-04-26T21:13:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.978821+0000", 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.984957+0000", 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.985100+0000", 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.985205+0000", 2024-04-26T21:13:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.985327+0000", 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.985332+0000", 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:126 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_15 2024-04-26T21:13:09.996988+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:09.998177+0000", 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.376613691000003, 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063253490000000001, 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 126 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998177+0000", 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998177+0000", 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998178+0000", 2024-04-26T21:13:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998184+0000", 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998202+0000", 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998558+0000", 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998875+0000", 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "early_replied" 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:09.998875+0000", 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "submit entry: journal_and_reply" 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.004134+0000", 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "journal_committed: " 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.004279+0000", 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.004389+0000", 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.004501+0000", 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.004502+0000", 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:143 readdir #0x10000000242 2024-04-26T21:13:10.256985+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:10.257998+0000", 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 40.116792789000002, 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 16.507331709999999, 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 143 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.257998+0000", 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.257998+0000", 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.257998+0000", 2024-04-26T21:13:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.258002+0000", 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.258021+0000", 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.258100+0000", 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:10.758196+0000", 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:11.258346+0000", 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:11.758477+0000", 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:12.258679+0000", 2024-04-26T21:13:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:12.758854+0000", 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:13.259069+0000", 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:13.759256+0000", 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:14.259422+0000", 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:14.759602+0000", 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:15.259791+0000", 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:15.759954+0000", 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:16.260140+0000", 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:16.760347+0000", 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:17.260551+0000", 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:17.760734+0000", 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:18.260910+0000", 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:18.761046+0000", 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:19.261173+0000", 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:19.761371+0000", 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:20.261577+0000", 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:20.761785+0000", 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:21.261967+0000", 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:21.762086+0000", 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:22.262271+0000", 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:22.762438+0000", 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:23.262613+0000", 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:23.762815+0000", 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:24.263013+0000", 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:24.763197+0000", 2024-04-26T21:13:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:25.263383+0000", 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:25.763568+0000", 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.263769+0000", 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.764183+0000", 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765074+0000", 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765140+0000", 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765327+0000", 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765329+0000", 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "client_request(client.4835:144 readdir #0x10000000228 2024-04-26T21:13:26.764786+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-04-26T21:13:26.765472+0000", 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.609318226999999, 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 22.509417683999999, 2024-04-26T21:13:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "continuous": false, 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "result": 0, 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "done", 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "reqid": { 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": { 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4835 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 144 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "op_type": "client_request", 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765472+0000", 2024-04-26T21:13:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated" 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765472+0000", 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read" 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765473+0000", 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled" 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765477+0000", 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read" 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765534+0000", 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched" 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:26.765637+0000", 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:27.265843+0000", 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:27.766024+0000", 2024-04-26T21:13:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:28.266236+0000", 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:28.766409+0000", 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:29.266612+0000", 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:29.766772+0000", 2024-04-26T21:13:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:30.266935+0000", 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:30.767081+0000", 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:31.267309+0000", 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:31.767466+0000", 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:32.267651+0000", 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:32.767839+0000", 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:33.268026+0000", 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:33.768205+0000", 2024-04-26T21:13:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:34.268400+0000", 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:34.768587+0000", 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:35.268765+0000", 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:35.768957+0000", 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:36.269150+0000", 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:36.769348+0000", 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:37.269502+0000", 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:37.769664+0000", 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:38.269752+0000", 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:38.769947+0000", 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:39.270055+0000", 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:39.770221+0000", 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:40.270383+0000", 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:40.770503+0000", 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:41.270681+0000", 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:41.770885+0000", 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:42.271066+0000", 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:42.771226+0000", 2024-04-26T21:13:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:43.271426+0000", 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:43.771622+0000", 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:44.271744+0000", 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:44.771906+0000", 2024-04-26T21:13:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:45.272090+0000", 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:45.772276+0000", 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:46.272400+0000", 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:46.772575+0000", 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:47.272768+0000", 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:47.772937+0000", 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:48.273077+0000", 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:48.773259+0000", 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cap_acquisition_throttle" 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:49.273794+0000", 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "acquired locks" 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:49.274609+0000", 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "replying" 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:49.274681+0000", 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "finishing request" 2024-04-26T21:13:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:49.274887+0000", 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "cleaned up request" 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-04-26T21:13:49.274890+0000", 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done" 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "locks": null 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:13:50.504 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 msgr2=0x7f089006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 0x7f089006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08b0160490 tx=0x7f0898008040 comp rx=0 tx=0).stop 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 msgr2=0x7f08b00ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a80046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 shutdown_connections 2024-04-26T21:13:50.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f089006c8f0 0x7f089006edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08b0140350 0x7f08b00738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08b013ff70 0x7f08b015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 --2- 172.21.15.40:0/34079235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08b00cff20 0x7f08b00ae8b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 >> 172.21.15.40:0/34079235 conn(0x7f08b00d0d30 msgr2=0x7f08b00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 shutdown_connections 2024-04-26T21:13:50.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.376+0000 7f08b5504640 1 -- 172.21.15.40:0/34079235 wait complete. 2024-04-26T21:13:50.508 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:13:50.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:13:50.549 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:13:50.593 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:13:50.594 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:13:50.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:50.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:50.610 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:50.610 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:50.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:13:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.808+0000 7f8ba1608640 1 Processor -- start 2024-04-26T21:13:50.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.808+0000 7f8ba1608640 1 -- start start 2024-04-26T21:13:50.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8ba1608640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b9c15f340 0x7f8b9c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8ba1608640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8ba1608640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0b2510 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8ba1608640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8b9c0b27f0 con 0x7f8b9c15f340 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8ba1608640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9c0b2ad0 con 0x7f8b9c15f720 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b9c15f340 0x7f8b9c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b9c0d4f00 0x7f8b9c0d5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52844/0 (socket says 172.21.15.40:52844) 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b9c15f340 0x7f8b9c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36194/0 (socket says 172.21.15.40:36194) 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9b7fe640 1 -- 172.21.15.40:0/3895772571 learned_addr learned my addr 172.21.15.40:0/3895772571 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9a7fc640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 -- 172.21.15.40:0/3895772571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 msgr2=0x7f8b9c0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 -- 172.21.15.40:0/3895772571 >> v1:172.21.15.40:6789/0 conn(0x7f8b9c0d4f00 legacy=0x7f8b9c0d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b9affd640 1 -- 172.21.15.40:0/3895772571 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c0b2db0 con 0x7f8b9c15f340 2024-04-26T21:13:50.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.809+0000 7f8b99ffb640 1 -- 172.21.15.40:0/3895772571 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743273555 0 0) 0x7f8b9c0b2510 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.810+0000 7f8b9affd640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0b1e80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8b940097a0 tx=0x7f8b9402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354ec02b351a890e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.810+0000 7f8b99ffb640 1 -- 172.21.15.40:0/3895772571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94037070 con 0x7f8b9c15f340 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.810+0000 7f8b99ffb640 1 -- 172.21.15.40:0/3895772571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9402ae50 con 0x7f8b9c15f340 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.810+0000 7f8b99ffb640 1 -- 172.21.15.40:0/3895772571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94032660 con 0x7f8b9c15f340 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 -- 172.21.15.40:0/3895772571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c15f340 msgr2=0x7f8b9c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0b1e80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8b940097a0 tx=0x7f8b9402a100 comp rx=0 tx=0).stop 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 -- 172.21.15.40:0/3895772571 shutdown_connections 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3895772571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0b1e80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 -- 172.21.15.40:0/3895772571 >> 172.21.15.40:0/3895772571 conn(0x7f8b9c08ec70 msgr2=0x7f8b9c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 -- 172.21.15.40:0/3895772571 shutdown_connections 2024-04-26T21:13:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.811+0000 7f8ba1608640 1 -- 172.21.15.40:0/3895772571 wait complete. 2024-04-26T21:13:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.812+0000 7f8ba1608640 1 Processor -- start 2024-04-26T21:13:50.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.812+0000 7f8ba1608640 1 -- start start 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c0b3980 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c0b3800 con 0x7f8b9c15f340 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8ba1608640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9c0b3b00 con 0x7f8b9c15f720 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36204/0 (socket says 172.21.15.40:36204) 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 -- 172.21.15.40:0/3477512192 learned_addr learned my addr 172.21.15.40:0/3477512192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9b7fe640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9a7fc640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 -- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 msgr2=0x7f8b9c0b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 -- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b9c15f340 msgr2=0x7f8b9c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 -- 172.21.15.40:0/3477512192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c0ca780 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.813+0000 7f8b9affd640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098c0 tx=0x7f8b940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.814+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94041030 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.814+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9402ac50 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.814+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94037040 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.814+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c0caa60 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.814+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c0cb020 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.815+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c069ae0 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.815+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b94044070 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.815+0000 7f8b937fe640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.816+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8b9402ae00 con 0x7f8b9c0d4f00 2024-04-26T21:13:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.816+0000 7f8b9a7fc640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:50.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.816+0000 7f8b9a7fc640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006fd0 tx=0x7f8b84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:50.820+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b9c069ae0 con 0x7f8b9c0d4f00 2024-04-26T21:13:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.008+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b9c06a290 con 0x7f8b9c0d4f00 2024-04-26T21:13:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.009+0000 7f8b937fe640 1 -- 172.21.15.40:0/3477512192 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7f8b940ad090 con 0x7f8b9c0d4f00 2024-04-26T21:13:51.010 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:51.011 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:13:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 msgr2=0x7f8b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006fd0 tx=0x7f8b84008040 comp rx=0 tx=0).stop 2024-04-26T21:13:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 msgr2=0x7f8b9c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098c0 tx=0x7f8b940046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 shutdown_connections 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b9c15f720 0x7f8b9c0b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b9c15f340 0x7f8b9c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 --2- 172.21.15.40:0/3477512192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9c0d4f00 0x7f8b9c0cb820 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 >> 172.21.15.40:0/3477512192 conn(0x7f8b9c08ec70 msgr2=0x7f8b9c0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 shutdown_connections 2024-04-26T21:13:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.011+0000 7f8ba1608640 1 -- 172.21.15.40:0/3477512192 wait complete. 2024-04-26T21:13:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:12:53.618832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:54.135627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:13:51.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:13:51.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.126+0000 7f2577d39640 1 Processor -- start 2024-04-26T21:13:51.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 -- start start 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f257015f260 0x7f257006d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2570070560 0x7f25700b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257015f640 con 0x7f257006ebe0 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f257006c2a0 con 0x7f257015f260 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.127+0000 7f2577d39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f257006c580 con 0x7f2570070560 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f257015f260 0x7f257006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f257015f260 0x7f257006d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36214/0 (socket says 172.21.15.40:36214) 2024-04-26T21:13:51.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 -- 172.21.15.40:0/704684765 learned_addr learned my addr 172.21.15.40:0/704684765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f25752d4640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2570070560 0x7f25700b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 -- 172.21.15.40:0/704684765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2570070560 msgr2=0x7f25700b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2570070560 0x7f25700b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 -- 172.21.15.40:0/704684765 >> v1:172.21.15.184:6789/0 conn(0x7f257006ebe0 legacy=0x7f25700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.128+0000 7f2575ad5640 1 -- 172.21.15.40:0/704684765 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257006c860 con 0x7f257015f260 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.129+0000 7f2575ad5640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257015f260 0x7f257006d620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc9de71692ec184 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.129+0000 7f2574ad3640 1 -- 172.21.15.40:0/704684765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c037070 con 0x7f257015f260 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.129+0000 7f2574ad3640 1 -- 172.21.15.40:0/704684765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c032440 con 0x7f257015f260 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.129+0000 7f2574ad3640 1 -- 172.21.15.40:0/704684765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c032810 con 0x7f257015f260 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 -- 172.21.15.40:0/704684765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257015f260 msgr2=0x7f257006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257015f260 0x7f257006d620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a250 comp rx=0 tx=0).stop 2024-04-26T21:13:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 -- 172.21.15.40:0/704684765 shutdown_connections 2024-04-26T21:13:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2570070560 0x7f25700b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 --2- 172.21.15.40:0/704684765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257015f260 0x7f257006d620 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 -- 172.21.15.40:0/704684765 >> 172.21.15.40:0/704684765 conn(0x7f25700d1660 msgr2=0x7f25700d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 -- 172.21.15.40:0/704684765 shutdown_connections 2024-04-26T21:13:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.130+0000 7f2577d39640 1 -- 172.21.15.40:0/704684765 wait complete. 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.131+0000 7f2577d39640 1 Processor -- start 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.131+0000 7f2577d39640 1 -- start start 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 0x7f25701544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 0x7f2570151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f257006d100 con 0x7f257006ebe0 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f257006ce20 con 0x7f2570070560 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f257006db60 con 0x7f257015f260 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36222/0 (socket says 172.21.15.40:36222) 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 -- 172.21.15.40:0/1367818908 learned_addr learned my addr 172.21.15.40:0/1367818908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25762d6640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 0x7f2570151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25752d4640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 0x7f25701544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 -- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 msgr2=0x7f2570151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 0x7f2570151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 -- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 msgr2=0x7f25701544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 0x7f25701544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 -- 172.21.15.40:0/1367818908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25701549e0 con 0x7f257006ebe0 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25762d6640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 0x7f2570151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:51.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25752d4640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 0x7f25701544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2575ad5640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f256c000c00 tx=0x7f256c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c041020 con 0x7f257006ebe0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c0306c0 con 0x7f257006ebe0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25700ce2b0 con 0x7f257006ebe0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c037040 con 0x7f257006ebe0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.132+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25700ce870 con 0x7f257006ebe0 2024-04-26T21:13:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.133+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257006a290 con 0x7f257006ebe0 2024-04-26T21:13:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.134+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f256c044020 con 0x7f257006ebe0 2024-04-26T21:13:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.134+0000 7f25667fc640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.134+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f256c0ac020 con 0x7f257006ebe0 2024-04-26T21:13:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.135+0000 7f25752d4640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.135+0000 7f25752d4640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 0x7f255406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.138+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f256c04b070 con 0x7f257006ebe0 2024-04-26T21:13:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.322+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25700682b0 con 0x7f257006ebe0 2024-04-26T21:13:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.323+0000 7f25667fc640 1 -- 172.21.15.40:0/1367818908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f256c07a440 con 0x7f257006ebe0 2024-04-26T21:13:51.324 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.324+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 msgr2=0x7f255406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.324+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 0x7f255406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560008040 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.324+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 msgr2=0x7f2570153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.324+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f256c000c00 tx=0x7f256c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 shutdown_connections 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f255406c8f0 0x7f255406edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257015f260 0x7f2570151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2570070560 0x7f25701544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 --2- 172.21.15.40:0/1367818908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f257006ebe0 0x7f2570153200 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 >> 172.21.15.40:0/1367818908 conn(0x7f25700d1660 msgr2=0x7f25700d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 shutdown_connections 2024-04-26T21:13:51.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.325+0000 7f2577d39640 1 -- 172.21.15.40:0/1367818908 wait complete. 2024-04-26T21:13:51.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:13:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.439+0000 7fd21d0d3640 1 Processor -- start 2024-04-26T21:13:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.439+0000 7fd21d0d3640 1 -- start start 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21d0d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd2180d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21d0d3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2180d0d10 0x7fd2180d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21d0d3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2180b63e0 con 0x7fd2180d2230 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21d0d3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd2180d5ed0 con 0x7fd2180d0d10 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21d0d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180d3120 con 0x7fd21815f950 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2180d0d10 0x7fd2180d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd2180d2230 0x7fd2180c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52846/0 (socket says 172.21.15.40:52846) 2024-04-26T21:13:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.440+0000 7fd21659c640 1 -- 172.21.15.40:0/2588862523 learned_addr learned my addr 172.21.15.40:0/2588862523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd216d9d640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd2180d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd21759e640 1 -- 172.21.15.40:0/2588862523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 msgr2=0x7fd2180d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd21759e640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd2180d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd21759e640 1 -- 172.21.15.40:0/2588862523 >> v1:172.21.15.40:6789/0 conn(0x7fd2180d2230 legacy=0x7fd2180c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd21759e640 1 -- 172.21.15.40:0/2588862523 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2180b69c0 con 0x7fd2180d0d10 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd215d9b640 1 -- 172.21.15.40:0/2588862523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532305894 0 0) 0x7fd2180b63e0 con 0x7fd2180d2230 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd216d9d640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd2180d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd21759e640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd2180d3c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd208009dc0 tx=0x7fd20802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d983fda6eaf736e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd215d9b640 1 -- 172.21.15.40:0/2588862523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208007740 con 0x7fd2180d0d10 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.441+0000 7fd215d9b640 1 -- 172.21.15.40:0/2588862523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2080078e0 con 0x7fd2180d0d10 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd215d9b640 1 -- 172.21.15.40:0/2588862523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208007cb0 con 0x7fd2180d0d10 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 -- 172.21.15.40:0/2588862523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 msgr2=0x7fd2180d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd2180d3c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd208009dc0 tx=0x7fd20802a5f0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 -- 172.21.15.40:0/2588862523 shutdown_connections 2024-04-26T21:13:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd2180d3c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/2588862523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd2180d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 -- 172.21.15.40:0/2588862523 >> 172.21.15.40:0/2588862523 conn(0x7fd2180d5160 msgr2=0x7fd2180d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 -- 172.21.15.40:0/2588862523 shutdown_connections 2024-04-26T21:13:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.442+0000 7fd21d0d3640 1 -- 172.21.15.40:0/2588862523 wait complete. 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.443+0000 7fd21d0d3640 1 Processor -- start 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 -- start start 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2180d2230 0x7fd2181618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd218161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180c9890 con 0x7fd2180d0d10 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180c9710 con 0x7fd2180d2230 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd21d0d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180c9a10 con 0x7fd21815f950 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd216d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd216d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36238/0 (socket says 172.21.15.40:36238) 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.444+0000 7fd216d9d640 1 -- 172.21.15.40:0/3314094803 learned_addr learned my addr 172.21.15.40:0/3314094803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd21759e640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd218161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd21659c640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2180d2230 0x7fd2181618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 -- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 msgr2=0x7fd218161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd218161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 -- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2180d2230 msgr2=0x7fd2181618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2180d2230 0x7fd2181618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 -- 172.21.15.40:0/3314094803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd218162430 con 0x7fd2180d0d10 2024-04-26T21:13:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd21759e640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd218161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd216d9d640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c7f0 tx=0x7fd21000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210015070 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2100077c0 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd218162710 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210007b70 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.445+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2180ccd50 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.446+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2180693d0 con 0x7fd2180d0d10 2024-04-26T21:13:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.447+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd210018070 con 0x7fd2180d0d10 2024-04-26T21:13:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.447+0000 7fd20f7fe640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 0x7fd1f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.447+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd21008bcc0 con 0x7fd2180d0d10 2024-04-26T21:13:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.447+0000 7fd21659c640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 0x7fd1f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.448+0000 7fd21659c640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 0x7fd1f806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2180d1ed0 tx=0x7fd200007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.451+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd21005a060 con 0x7fd2180d0d10 2024-04-26T21:13:51.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.636+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd2180d3520 con 0x7fd2180d0d10 2024-04-26T21:13:51.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.637+0000 7fd20f7fe640 1 -- 172.21.15.40:0/3314094803 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7fd210059960 con 0x7fd2180d0d10 2024-04-26T21:13:51.639 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 msgr2=0x7fd1f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 0x7fd1f806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2180d1ed0 tx=0x7fd200007b70 comp rx=0 tx=0).stop 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 msgr2=0x7fd21815a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c7f0 tx=0x7fd21000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 shutdown_connections 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.639+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1f806c8f0 0x7fd1f806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd21815f950 0x7fd218161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2180d2230 0x7fd2181618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 --2- 172.21.15.40:0/3314094803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2180d0d10 0x7fd21815a530 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 >> 172.21.15.40:0/3314094803 conn(0x7fd2180d5160 msgr2=0x7fd21806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 shutdown_connections 2024-04-26T21:13:51.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.640+0000 7fd21d0d3640 1 -- 172.21.15.40:0/3314094803 wait complete. 2024-04-26T21:13:51.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:13:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 Processor -- start 2024-04-26T21:13:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 -- start start 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaf0143840 0x7faaf00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf00b55c0 0x7faaf00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf015dca0 con 0x7faaf00b4520 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faaf0143c20 con 0x7faaf0143840 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.754+0000 7faaf511c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faaf015df80 con 0x7faaf00b55c0 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaf0143840 0x7faaf00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faaf0143840 0x7faaf00b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:51752/0 (socket says 172.21.15.40:51752) 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 -- 172.21.15.40:0/3187058873 learned_addr learned my addr 172.21.15.40:0/3187058873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaef59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf00b55c0 0x7faaf00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaee59c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7faaf00b4520 0x7faaf00b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:56476/0 (socket says 172.21.15.40:56476) 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 -- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf00b55c0 msgr2=0x7faaf00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 --2- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf00b55c0 0x7faaf00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 -- 172.21.15.40:0/3187058873 >> v1:172.21.15.184:6789/0 conn(0x7faaf00b4520 legacy=0x7faaf00b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.755+0000 7faaeed9d640 1 -- 172.21.15.40:0/3187058873 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf015e260 con 0x7faaf0143840 2024-04-26T21:13:51.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.756+0000 7faaeed9d640 1 --2- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf0143840 0x7faaf00b3fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faae0009870 tx=0x7faae002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cda993c339cad6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.756+0000 7faaedd9b640 1 -- 172.21.15.40:0/3187058873 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0037070 con 0x7faaf0143840 2024-04-26T21:13:51.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.756+0000 7faaedd9b640 1 -- 172.21.15.40:0/3187058873 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae0032440 con 0x7faaf0143840 2024-04-26T21:13:51.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.756+0000 7faaedd9b640 1 -- 172.21.15.40:0/3187058873 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0032810 con 0x7faaf0143840 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 -- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf0143840 msgr2=0x7faaf00b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 --2- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf0143840 0x7faaf00b3fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faae0009870 tx=0x7faae002a270 comp rx=0 tx=0).stop 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 -- 172.21.15.40:0/3187058873 shutdown_connections 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 --2- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf00b55c0 0x7faaf00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 --2- 172.21.15.40:0/3187058873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf0143840 0x7faaf00b3fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 -- 172.21.15.40:0/3187058873 >> 172.21.15.40:0/3187058873 conn(0x7faaf00cd8a0 msgr2=0x7faaf00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 -- 172.21.15.40:0/3187058873 shutdown_connections 2024-04-26T21:13:51.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.757+0000 7faaf511c640 1 -- 172.21.15.40:0/3187058873 wait complete. 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.758+0000 7faaf511c640 1 Processor -- start 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.758+0000 7faaf511c640 1 -- start start 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf00b55c0 0x7faaf01311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf0143840 0x7faaf0134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faaf015da40 con 0x7faaf00b4520 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faaf015d8c0 con 0x7faaf00b55c0 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaf511c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faaf00cf530 con 0x7faaf0143840 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36244/0 (socket says 172.21.15.40:36244) 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 -- 172.21.15.40:0/544654803 learned_addr learned my addr 172.21.15.40:0/544654803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaef59e640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf0143840 0x7faaf0134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaee59c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf00b55c0 0x7faaf01311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 -- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf0143840 msgr2=0x7faaf0134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf0143840 0x7faaf0134480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 -- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf00b55c0 msgr2=0x7faaf01311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf00b55c0 0x7faaf01311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.759+0000 7faaeed9d640 1 -- 172.21.15.40:0/544654803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf0131710 con 0x7faaf00b4520 2024-04-26T21:13:51.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faaeed9d640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faae0000c00 tx=0x7faae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0037040 con 0x7faaf00b4520 2024-04-26T21:13:51.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae0042070 con 0x7faaf00b4520 2024-04-26T21:13:51.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf01349c0 con 0x7faaf00b4520 2024-04-26T21:13:51.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0030860 con 0x7faaf00b4520 2024-04-26T21:13:51.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.760+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf0145ad0 con 0x7faaf00b4520 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.761+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf006a290 con 0x7faaf00b4520 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.761+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faae0030a00 con 0x7faaf00b4520 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.762+0000 7faad77fe640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 0x7faadc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.762+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7faae00ac230 con 0x7faaf00b4520 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.762+0000 7faaee59c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 0x7faadc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:51.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.763+0000 7faaee59c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 0x7faadc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaf006fd70 tx=0x7faae4000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:51.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.766+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faae007a0a0 con 0x7faaf00b4520 2024-04-26T21:13:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.958+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faaf00ce420 con 0x7faaf00b4520 2024-04-26T21:13:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.958+0000 7faad77fe640 1 -- 172.21.15.40:0/544654803 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7faae007e3a0 con 0x7faaf00b4520 2024-04-26T21:13:51.959 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 msgr2=0x7faadc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 0x7faadc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaf006fd70 tx=0x7faae4000ed0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 msgr2=0x7faaf0146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faae0000c00 tx=0x7faae00046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 shutdown_connections 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faadc06c8f0 0x7faadc06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faaf0143840 0x7faaf0134480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faaf00b55c0 0x7faaf01311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 --2- 172.21.15.40:0/544654803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00b4520 0x7faaf0146890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.960+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 >> 172.21.15.40:0/544654803 conn(0x7faaf00cd8a0 msgr2=0x7faaf00b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.961+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 shutdown_connections 2024-04-26T21:13:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:51.961+0000 7faaf511c640 1 -- 172.21.15.40:0/544654803 wait complete. 2024-04-26T21:13:51.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:13:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.073+0000 7f0db635f640 1 Processor -- start 2024-04-26T21:13:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.073+0000 7f0db635f640 1 -- start start 2024-04-26T21:13:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db635f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0db00d11b0 0x7f0db006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db635f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0db006e2c0 0x7f0db015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db635f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db00d1740 con 0x7f0db00cf2e0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db635f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0db006e710 con 0x7f0db00d11b0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db635f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0db00d1a20 con 0x7f0db006e2c0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0db00d11b0 0x7f0db006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0db00cf2e0 0x7f0db006d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52852/0 (socket says 172.21.15.40:52852) 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daffff640 1 -- 172.21.15.40:0/579086748 learned_addr learned my addr 172.21.15.40:0/579086748 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0db00d11b0 0x7f0db006dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36252/0 (socket says 172.21.15.40:36252) 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 -- 172.21.15.40:0/579086748 >> v2:172.21.15.184:3300/0 conn(0x7f0db006e2c0 msgr2=0x7f0db015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0db48fc640 1 --2- 172.21.15.40:0/579086748 >> v2:172.21.15.184:3300/0 conn(0x7f0db006e2c0 0x7f0db015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 --2- 172.21.15.40:0/579086748 >> v2:172.21.15.184:3300/0 conn(0x7f0db006e2c0 0x7f0db015ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daeffd640 1 -- 172.21.15.40:0/579086748 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307842755 0 0) 0x7f0db00d1740 con 0x7f0db00cf2e0 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 -- 172.21.15.40:0/579086748 >> v1:172.21.15.40:6789/0 conn(0x7f0db00cf2e0 legacy=0x7f0db006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.074+0000 7f0daf7fe640 1 -- 172.21.15.40:0/579086748 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db00d1d00 con 0x7f0db00d11b0 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.075+0000 7f0daf7fe640 1 --2- 172.21.15.40:0/579086748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db006dd80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ba6d36d9dcb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.075+0000 7f0daeffd640 1 -- 172.21.15.40:0/579086748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0037070 con 0x7f0db00d11b0 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.075+0000 7f0daeffd640 1 -- 172.21.15.40:0/579086748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0032440 con 0x7f0db00d11b0 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.075+0000 7f0daeffd640 1 -- 172.21.15.40:0/579086748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0032810 con 0x7f0db00d11b0 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 -- 172.21.15.40:0/579086748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 msgr2=0x7f0db006dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 --2- 172.21.15.40:0/579086748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db006dd80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 -- 172.21.15.40:0/579086748 shutdown_connections 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 --2- 172.21.15.40:0/579086748 >> v2:172.21.15.184:3300/0 conn(0x7f0db006e2c0 0x7f0db015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 --2- 172.21.15.40:0/579086748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db006dd80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 -- 172.21.15.40:0/579086748 >> 172.21.15.40:0/579086748 conn(0x7f0db00b4a60 msgr2=0x7f0db00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 -- 172.21.15.40:0/579086748 shutdown_connections 2024-04-26T21:13:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.076+0000 7f0db635f640 1 -- 172.21.15.40:0/579086748 wait complete. 2024-04-26T21:13:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.077+0000 7f0db635f640 1 Processor -- start 2024-04-26T21:13:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.077+0000 7f0db635f640 1 -- start start 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db006e2c0 0x7f0db0152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0db00cf2e0 0x7f0db00b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00cf6c0 con 0x7f0db00d11b0 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00d22c0 con 0x7f0db006e2c0 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db635f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0db00cf840 con 0x7f0db00cf2e0 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36256/0 (socket says 172.21.15.40:36256) 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 -- 172.21.15.40:0/2245857076 learned_addr learned my addr 172.21.15.40:0/2245857076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 -- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0db00cf2e0 msgr2=0x7f0db00b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0db00cf2e0 0x7f0db00b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 -- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db006e2c0 msgr2=0x7f0db0152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db006e2c0 0x7f0db0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 -- 172.21.15.40:0/2245857076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db00b3c40 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.078+0000 7f0db48fc640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0da800cc30 tx=0x7f0da800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.079+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800ce40 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.079+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8003990 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.079+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db00b33a0 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.079+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003d40 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.079+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db00b3960 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.080+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db006a290 con 0x7f0db00d11b0 2024-04-26T21:13:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.080+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0da8003fa0 con 0x7f0db00d11b0 2024-04-26T21:13:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.080+0000 7f0dacff9640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 0x7f0d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.080+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f0da808c0f0 con 0x7f0db00d11b0 2024-04-26T21:13:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.081+0000 7f0daffff640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 0x7f0d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.081+0000 7f0daffff640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 0x7f0d9006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d98007fd0 tx=0x7f0d98007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.085+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0da8014030 con 0x7f0db00d11b0 2024-04-26T21:13:52.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.285+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0db00b3080 con 0x7f0db00d11b0 2024-04-26T21:13:52.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.285+0000 7f0dacff9640 1 -- 172.21.15.40:0/2245857076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f0da805a410 con 0x7f0db00d11b0 2024-04-26T21:13:52.287 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 msgr2=0x7f0d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 0x7f0d9006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d98007fd0 tx=0x7f0d98007960 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 msgr2=0x7f0db00b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0da800cc30 tx=0x7f0da800a5c0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 shutdown_connections 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d9006c8f0 0x7f0d9006edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00d11b0 0x7f0db00b2b40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0db00cf2e0 0x7f0db00b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 --2- 172.21.15.40:0/2245857076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0db006e2c0 0x7f0db0152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 >> 172.21.15.40:0/2245857076 conn(0x7f0db00b4a60 msgr2=0x7f0db00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.287+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 shutdown_connections 2024-04-26T21:13:52.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.288+0000 7f0db635f640 1 -- 172.21.15.40:0/2245857076 wait complete. 2024-04-26T21:13:52.299 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:13:52.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:52.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:13:52.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:13:52.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:52.387 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:13:52.402 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:13:52.402 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:13:52.402 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:52.419 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:13:52.420 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:52.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:13:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.576+0000 7f8ec4272640 1 Processor -- start 2024-04-26T21:13:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.576+0000 7f8ec4272640 1 -- start start 2024-04-26T21:13:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec4272640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8ebc0cf2e0 0x7f8ebc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec4272640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc0d1740 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec4272640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc0d1a20 con 0x7f8ebc06e490 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec4272640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8ebc0d1d00 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec180d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8ebc0d11b0 0x7f8ebc06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52864/0 (socket says 172.21.15.40:52864) 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec180d640 1 -- 172.21.15.40:0/366469079 learned_addr learned my addr 172.21.15.40:0/366469079 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec200e640 1 --2- 172.21.15.40:0/366469079 >> v2:172.21.15.184:3300/0 conn(0x7f8ebc0cf2e0 0x7f8ebc06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032486008 0 0) 0x7f8ebc0d1740 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.577+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea8003520 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.578+0000 7f8ec200e640 1 -- 172.21.15.40:0/366469079 >> v1:172.21.15.184:6789/0 conn(0x7f8ebc06e490 legacy=0x7f8ebc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.578+0000 7f8ec200e640 1 -- 172.21.15.40:0/366469079 >> v1:172.21.15.40:6789/0 conn(0x7f8ebc0d11b0 legacy=0x7f8ebc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.578+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1857798975 0 0) 0x7f8ea8003520 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.578+0000 7f8ec200e640 1 -- 172.21.15.40:0/366469079 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc0d1fe0 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.578+0000 7f8ec200e640 1 --2- 172.21.15.40:0/366469079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc06d4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80097d0 tx=0x7f8eb802a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50eea60683f2aafe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8037070 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb802ac90 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec100c640 1 -- 172.21.15.40:0/366469079 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac001670 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 -- 172.21.15.40:0/366469079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 msgr2=0x7f8ebc06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 --2- 172.21.15.40:0/366469079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc06d4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80097d0 tx=0x7f8eb802a200 comp rx=0 tx=0).stop 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 -- 172.21.15.40:0/366469079 shutdown_connections 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 --2- 172.21.15.40:0/366469079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc06d4d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 -- 172.21.15.40:0/366469079 >> 172.21.15.40:0/366469079 conn(0x7f8ebc0b4a60 msgr2=0x7f8ebc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 -- 172.21.15.40:0/366469079 shutdown_connections 2024-04-26T21:13:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.579+0000 7f8ec4272640 1 -- 172.21.15.40:0/366469079 wait complete. 2024-04-26T21:13:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.580+0000 7f8ec4272640 1 Processor -- start 2024-04-26T21:13:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.580+0000 7f8ec4272640 1 -- start start 2024-04-26T21:13:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 0x7f8ebc0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0cf6c0 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc06da10 con 0x7f8ebc0cf2e0 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec4272640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc0cf840 con 0x7f8ebc06e490 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec280f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec280f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36272/0 (socket says 172.21.15.40:36272) 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.581+0000 7f8ec280f640 1 -- 172.21.15.40:0/3477223943 learned_addr learned my addr 172.21.15.40:0/3477223943 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec180d640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec200e640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 0x7f8ebc0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 -- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 msgr2=0x7f8ebc0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 0x7f8ebc0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 -- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 msgr2=0x7f8ebc159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 -- 172.21.15.40:0/3477223943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc0b66e0 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec200e640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 0x7f8ebc0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec280f640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c730 tx=0x7f8eb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4003b70 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4003d10 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4005310 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc0b6960 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.582+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc0b6f20 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.583+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc0b8ce0 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.584+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eb40056c0 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.584+0000 7f8eb2ffd640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.584+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8eb408bb90 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.584+0000 7f8ec200e640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.585+0000 7f8ec200e640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 0x7f8ea006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.589+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ebc0b8ce0 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.775+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ebc075b80 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.776+0000 7f8eb2ffd640 1 -- 172.21.15.40:0/3477223943 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7f8ebc075b80 con 0x7f8ebc0d11b0 2024-04-26T21:13:52.778 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:52.778 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:13:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 msgr2=0x7f8ea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 0x7f8ea006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb802e000 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 msgr2=0x7f8ebc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c730 tx=0x7f8eb400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 shutdown_connections 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ebc0d11b0 0x7f8ebc0b61a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ebc0cf2e0 0x7f8ebc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 --2- 172.21.15.40:0/3477223943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ebc06e490 0x7f8ebc0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 >> 172.21.15.40:0/3477223943 conn(0x7f8ebc0b4a60 msgr2=0x7f8ebc0c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 shutdown_connections 2024-04-26T21:13:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.778+0000 7f8ec4272640 1 -- 172.21.15.40:0/3477223943 wait complete. 2024-04-26T21:13:52.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:12:53.618832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:54.135627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:13:52.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:13:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.893+0000 7f3c8d9f0640 1 Processor -- start 2024-04-26T21:13:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.893+0000 7f3c8d9f0640 1 -- start start 2024-04-26T21:13:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c8d9f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c880d0300 0x7f3c880b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c8d9f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 0x7f3c880720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c8d9f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c880b46a0 con 0x7f3c8806f340 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c8d9f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c880b5db0 con 0x7f3c880d0300 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c8d9f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3c880b6090 con 0x7f3c880b3e50 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c880d0300 0x7f3c880b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c880d0300 0x7f3c880b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36286/0 (socket says 172.21.15.40:36286) 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 -- 172.21.15.40:0/4089180094 learned_addr learned my addr 172.21.15.40:0/4089180094 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c877fe640 1 --2- 172.21.15.40:0/4089180094 >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 0x7f3c880720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 -- 172.21.15.40:0/4089180094 >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 msgr2=0x7f3c880720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 --2- 172.21.15.40:0/4089180094 >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 0x7f3c880720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 -- 172.21.15.40:0/4089180094 >> v1:172.21.15.184:6789/0 conn(0x7f3c8806f340 legacy=0x7f3c8806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c867fc640 1 -- 172.21.15.40:0/4089180094 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c880b6370 con 0x7f3c880d0300 2024-04-26T21:13:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.894+0000 7f3c877fe640 1 --2- 172.21.15.40:0/4089180094 >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 0x7f3c880720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.895+0000 7f3c867fc640 1 --2- 172.21.15.40:0/4089180094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880d0300 0x7f3c880b6b70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c88070020 tx=0x7f3c7002c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c523a9ba3c4deed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.895+0000 7f3c85ffb640 1 -- 172.21.15.40:0/4089180094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70032710 con 0x7f3c880d0300 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.895+0000 7f3c85ffb640 1 -- 172.21.15.40:0/4089180094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c700328b0 con 0x7f3c880d0300 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.895+0000 7f3c85ffb640 1 -- 172.21.15.40:0/4089180094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70032ca0 con 0x7f3c880d0300 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/4089180094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880d0300 msgr2=0x7f3c880b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/4089180094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880d0300 0x7f3c880b6b70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c88070020 tx=0x7f3c7002c810 comp rx=0 tx=0).stop 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/4089180094 shutdown_connections 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/4089180094 >> v2:172.21.15.184:3300/0 conn(0x7f3c880b3e50 0x7f3c880720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/4089180094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880d0300 0x7f3c880b6b70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/4089180094 >> 172.21.15.40:0/4089180094 conn(0x7f3c880cd8a0 msgr2=0x7f3c880cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/4089180094 shutdown_connections 2024-04-26T21:13:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.896+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/4089180094 wait complete. 2024-04-26T21:13:52.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.897+0000 7f3c8d9f0640 1 Processor -- start 2024-04-26T21:13:52.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.897+0000 7f3c8d9f0640 1 -- start start 2024-04-26T21:13:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 0x7f3c8815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c880d0300 0x7f3c88153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c880b4460 con 0x7f3c880b3e50 2024-04-26T21:13:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c880b42e0 con 0x7f3c8806f340 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c8d9f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3c88142a30 con 0x7f3c880d0300 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c867fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c867fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36290/0 (socket says 172.21.15.40:36290) 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c867fc640 1 -- 172.21.15.40:0/3367270690 learned_addr learned my addr 172.21.15.40:0/3367270690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c86ffd640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 0x7f3c8815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.898+0000 7f3c877fe640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c880d0300 0x7f3c88153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 -- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c880d0300 msgr2=0x7f3c88153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c880d0300 0x7f3c88153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 -- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 msgr2=0x7f3c8815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 0x7f3c8815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 -- 172.21.15.40:0/3367270690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8815dc30 con 0x7f3c880b3e50 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c86ffd640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 0x7f3c8815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c867fc640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c7002c850 tx=0x7f3c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70044070 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c70003830 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70003c20 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8815df10 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.899+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8815e4d0 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.900+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8806a290 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.901+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c70047070 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.901+0000 7f3c7ffff640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.901+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3c700aed90 con 0x7f3c880b3e50 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.901+0000 7f3c86ffd640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.902+0000 7f3c86ffd640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 0x7f3c6806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c80001700 tx=0x7f3c80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:52.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:52.906+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c7007d2f0 con 0x7f3c880b3e50 2024-04-26T21:13:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.090+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c881438d0 con 0x7f3c880b3e50 2024-04-26T21:13:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.090+0000 7f3c7ffff640 1 -- 172.21.15.40:0/3367270690 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f3c7007cbf0 con 0x7f3c880b3e50 2024-04-26T21:13:53.091 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 msgr2=0x7f3c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 0x7f3c6806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c80001700 tx=0x7f3c80009040 comp rx=0 tx=0).stop 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 msgr2=0x7f3c88152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c7002c850 tx=0x7f3c700046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 shutdown_connections 2024-04-26T21:13:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c880d0300 0x7f3c88153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c880b3e50 0x7f3c88152860 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 --2- 172.21.15.40:0/3367270690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c8806f340 0x7f3c8815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 >> 172.21.15.40:0/3367270690 conn(0x7f3c880cd8a0 msgr2=0x7f3c881431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 shutdown_connections 2024-04-26T21:13:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.092+0000 7f3c8d9f0640 1 -- 172.21.15.40:0/3367270690 wait complete. 2024-04-26T21:13:53.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:13:53.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.206+0000 7fe2170f8640 1 Processor -- start 2024-04-26T21:13:53.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.206+0000 7fe2170f8640 1 -- start start 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe2170f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2100d4f20 0x7fe2100d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe2170f8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100d3c90 con 0x7fe2100d3820 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe2170f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100d58c0 con 0x7fe21015f680 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe2170f8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe21015fa60 con 0x7fe2100d4f20 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe20ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe2100d3820 0x7fe2100b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52870/0 (socket says 172.21.15.40:52870) 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.207+0000 7fe20ffff640 1 -- 172.21.15.40:0/779245639 learned_addr learned my addr 172.21.15.40:0/779245639 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe215695640 1 --2- 172.21.15.40:0/779245639 >> v2:172.21.15.184:3300/0 conn(0x7fe2100d4f20 0x7fe2100d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453074568 0 0) 0x7fe2100d3c90 con 0x7fe2100d3820 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1f8003520 con 0x7fe2100d3820 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3152443438 0 0) 0x7fe1f8003520 con 0x7fe2100d3820 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 >> v2:172.21.15.184:3300/0 conn(0x7fe2100d4f20 msgr2=0x7fe2100d5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 --2- 172.21.15.40:0/779245639 >> v2:172.21.15.184:3300/0 conn(0x7fe2100d4f20 0x7fe2100d5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 >> v1:172.21.15.184:6789/0 conn(0x7fe21015f680 legacy=0x7fe210163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2100d3f50 con 0x7fe2100d3820 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe204001670 con 0x7fe2100d3820 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 >> v1:172.21.15.40:6789/0 conn(0x7fe2100d3820 legacy=0x7fe2100b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe1f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f8004800 0x7fe1f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1f80051c0 0x7fe1f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe21015fa60 con 0x7fe1f80039b0 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f800c060 con 0x7fe1f8004800 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f800c260 con 0x7fe1f80051c0 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe1f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe20ffff640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f8004800 0x7fe1f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe215695640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1f80051c0 0x7fe1f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 -- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1f80051c0 msgr2=0x7fe1f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1f80051c0 0x7fe1f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 -- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f8004800 msgr2=0x7fe1f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f8004800 0x7fe1f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe214e94640 1 -- 172.21.15.40:0/779245639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f8003520 con 0x7fe1f80039b0 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.208+0000 7fe215695640 1 -- 172.21.15.40:0/779245639 reap_dead start 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.209+0000 7fe214e94640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe1f8003dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe208001e50 tx=0x7fe1f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b1ca788000eab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.209+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208001a70 con 0x7fe1f80039b0 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.209+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe208003970 con 0x7fe1f80039b0 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.209+0000 7fe20f7fe640 1 -- 172.21.15.40:0/779245639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208003d60 con 0x7fe1f80039b0 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 -- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 msgr2=0x7fe1f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe1f8003dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe208001e50 tx=0x7fe1f8005830 comp rx=0 tx=0).stop 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 -- 172.21.15.40:0/779245639 shutdown_connections 2024-04-26T21:13:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 --2- 172.21.15.40:0/779245639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe1f8003dd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 -- 172.21.15.40:0/779245639 >> 172.21.15.40:0/779245639 conn(0x7fe21008eca0 msgr2=0x7fe21009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 -- 172.21.15.40:0/779245639 shutdown_connections 2024-04-26T21:13:53.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.210+0000 7fe2170f8640 1 -- 172.21.15.40:0/779245639 wait complete. 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.211+0000 7fe2170f8640 1 Processor -- start 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.211+0000 7fe2170f8640 1 -- start start 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe210162250 0x7fe2101536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 0x7fe2100b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100d5380 con 0x7fe1f80039b0 2024-04-26T21:13:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2100b5370 con 0x7fe210162250 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe2170f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2100d5500 con 0x7fe2101f51a0 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36310/0 (socket says 172.21.15.40:36310) 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 -- 172.21.15.40:0/1766827736 learned_addr learned my addr 172.21.15.40:0/1766827736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe20ffff640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe210162250 0x7fe2101536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe215695640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 0x7fe2100b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 -- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 msgr2=0x7fe2100b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 0x7fe2100b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 -- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe210162250 msgr2=0x7fe2101536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe210162250 0x7fe2101536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 -- 172.21.15.40:0/1766827736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2101f96c0 con 0x7fe1f80039b0 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe215695640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 0x7fe2100b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.212+0000 7fe214e94640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe208000f80 tx=0x7fe2080014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.213+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20803f030 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.213+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2080050d0 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.213+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2101f5bd0 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.213+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208037040 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.213+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2101f8980 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.214+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2101f60f0 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.214+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe208042070 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.215+0000 7fe20d7fa640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.215+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe2080ab7a0 con 0x7fe1f80039b0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.215+0000 7fe20ffff640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.215+0000 7fe20ffff640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe210162670 tx=0x7fe204009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.219+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2101f60f0 con 0x7fe1f80039b0 2024-04-26T21:13:53.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.403+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe21006a290 con 0x7fe1f80039b0 2024-04-26T21:13:53.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.403+0000 7fe20d7fa640 1 -- 172.21.15.40:0/1766827736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7fe2101f96c0 con 0x7fe1f80039b0 2024-04-26T21:13:53.404 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 msgr2=0x7fe1fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe210162670 tx=0x7fe204009040 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 msgr2=0x7fe2101f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe208000f80 tx=0x7fe2080014a0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 shutdown_connections 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2101f51a0 0x7fe2100b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe210162250 0x7fe2101536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 --2- 172.21.15.40:0/1766827736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1f80039b0 0x7fe2101f9180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 >> 172.21.15.40:0/1766827736 conn(0x7fe21008eca0 msgr2=0x7fe21015e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.405+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 shutdown_connections 2024-04-26T21:13:53.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.406+0000 7fe2170f8640 1 -- 172.21.15.40:0/1766827736 wait complete. 2024-04-26T21:13:53.417 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:13:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.519+0000 7f046f97f640 1 Processor -- start 2024-04-26T21:13:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.519+0000 7f046f97f640 1 -- start start 2024-04-26T21:13:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046f97f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04680b4e00 0x7f04680ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046f97f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f046815f640 con 0x7f0468070560 2024-04-26T21:13:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046f97f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f046806c2a0 con 0x7f046815f260 2024-04-26T21:13:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046f97f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f046806c580 con 0x7f04680b4e00 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046df1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04680b4e00 0x7f04680ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046cf1a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0468070560 0x7f04680b46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52886/0 (socket says 172.21.15.40:52886) 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046cf1a640 1 -- 172.21.15.40:0/3677604462 learned_addr learned my addr 172.21.15.40:0/3677604462 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.520+0000 7f046df1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04680b4e00 0x7f04680ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36312/0 (socket says 172.21.15.40:36312) 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f046df1c640 1 -- 172.21.15.40:0/3677604462 >> v1:172.21.15.184:6789/0 conn(0x7f046815f260 legacy=0x7f046806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f046df1c640 1 -- 172.21.15.40:0/3677604462 >> v1:172.21.15.40:6789/0 conn(0x7f0468070560 legacy=0x7f04680b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f0467fff640 1 -- 172.21.15.40:0/3677604462 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1979619091 0 0) 0x7f046815f640 con 0x7f0468070560 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f046df1c640 1 -- 172.21.15.40:0/3677604462 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046806c860 con 0x7f04680b4e00 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f046df1c640 1 --2- 172.21.15.40:0/3677604462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04680b4e00 0x7f04680ae8b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04600097a0 tx=0x7f046002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3bdebe604b45d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f0467fff640 1 -- 172.21.15.40:0/3677604462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460037070 con 0x7f04680b4e00 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f0467fff640 1 -- 172.21.15.40:0/3677604462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460032440 con 0x7f04680b4e00 2024-04-26T21:13:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.521+0000 7f0467fff640 1 -- 172.21.15.40:0/3677604462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460032810 con 0x7f04680b4e00 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 -- 172.21.15.40:0/3677604462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04680b4e00 msgr2=0x7f04680ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 --2- 172.21.15.40:0/3677604462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04680b4e00 0x7f04680ae8b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04600097a0 tx=0x7f046002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 -- 172.21.15.40:0/3677604462 shutdown_connections 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 --2- 172.21.15.40:0/3677604462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04680b4e00 0x7f04680ae8b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 -- 172.21.15.40:0/3677604462 >> 172.21.15.40:0/3677604462 conn(0x7f04680d1660 msgr2=0x7f04680d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 -- 172.21.15.40:0/3677604462 shutdown_connections 2024-04-26T21:13:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.522+0000 7f046f97f640 1 -- 172.21.15.40:0/3677604462 wait complete. 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.523+0000 7f046f97f640 1 Processor -- start 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.523+0000 7f046f97f640 1 -- start start 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0468070560 0x7f04681ea780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04680b4e00 0x7f046815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f046806d100 con 0x7f046815f260 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f046806ce20 con 0x7f04680b4e00 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046f97f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f046806ebe0 con 0x7f0468070560 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36326/0 (socket says 172.21.15.40:36326) 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 -- 172.21.15.40:0/2466569363 learned_addr learned my addr 172.21.15.40:0/2466569363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046cf1a640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04680b4e00 0x7f046815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046d71b640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0468070560 0x7f04681ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 -- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0468070560 msgr2=0x7f04681ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0468070560 0x7f04681ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 -- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04680b4e00 msgr2=0x7f046815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04680b4e00 0x7f046815f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.524+0000 7f046df1c640 1 -- 172.21.15.40:0/2466569363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046815fb80 con 0x7f046815f260 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f046df1c640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0460035040 tx=0x7f04600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460037040 con 0x7f046815f260 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460004230 con 0x7f046815f260 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0468162520 con 0x7f046815f260 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460032e20 con 0x7f046815f260 2024-04-26T21:13:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.525+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681ea260 con 0x7f046815f260 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.526+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046806a290 con 0x7f046815f260 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.527+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0460030770 con 0x7f046815f260 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.527+0000 7f0465ffb640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 0x7f045406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.527+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f04600acd40 con 0x7f046815f260 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.527+0000 7f046d71b640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 0x7f045406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.528+0000 7f046d71b640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 0x7f045406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0458009f20 tx=0x7f04580094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.531+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f046007b200 con 0x7f046815f260 2024-04-26T21:13:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.716+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04681e9ac0 con 0x7f046815f260 2024-04-26T21:13:53.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.716+0000 7f0465ffb640 1 -- 172.21.15.40:0/2466569363 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f046007ab00 con 0x7f046815f260 2024-04-26T21:13:53.718 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:13:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 msgr2=0x7f045406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 0x7f045406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0458009f20 tx=0x7f04580094d0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 msgr2=0x7f0468159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0460035040 tx=0x7f04600046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 shutdown_connections 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f045406c8f0 0x7f045406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815f260 0x7f0468159aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04680b4e00 0x7f046815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 --2- 172.21.15.40:0/2466569363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0468070560 0x7f04681ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 >> 172.21.15.40:0/2466569363 conn(0x7f04680d1660 msgr2=0x7f04681633f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 shutdown_connections 2024-04-26T21:13:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.718+0000 7f046f97f640 1 -- 172.21.15.40:0/2466569363 wait complete. 2024-04-26T21:13:53.730 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:13:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.830+0000 7f56880da640 1 Processor -- start 2024-04-26T21:13:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.830+0000 7f56880da640 1 -- start start 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f56880da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 0x7f56800b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f56880da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56800d5610 0x7f56800d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f56880da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56800b2510 con 0x7f568015f720 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f56880da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f56800b27f0 con 0x7f56800d5610 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f56880da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f56800b2ad0 con 0x7f568015f340 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5686677640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56800d5610 0x7f56800d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5685675640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f568015f720 0x7f56800d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52898/0 (socket says 172.21.15.40:52898) 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5685675640 1 -- 172.21.15.40:0/3746271158 learned_addr learned my addr 172.21.15.40:0/3746271158 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5685e76640 1 --2- 172.21.15.40:0/3746271158 >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 0x7f56800b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5686677640 1 -- 172.21.15.40:0/3746271158 >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 msgr2=0x7f56800b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5686677640 1 --2- 172.21.15.40:0/3746271158 >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 0x7f56800b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5686677640 1 -- 172.21.15.40:0/3746271158 >> v1:172.21.15.40:6789/0 conn(0x7f568015f720 legacy=0x7f56800d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.831+0000 7f5686677640 1 -- 172.21.15.40:0/3746271158 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56800b2db0 con 0x7f56800d5610 2024-04-26T21:13:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.832+0000 7f5685e76640 1 --2- 172.21.15.40:0/3746271158 >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 0x7f56800b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.832+0000 7f5686677640 1 --2- 172.21.15.40:0/3746271158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800d5610 0x7f56800d3d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f56800d6000 tx=0x7f567802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0cf5771377be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.832+0000 7f5684e74640 1 -- 172.21.15.40:0/3746271158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678007740 con 0x7f56800d5610 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.832+0000 7f5684e74640 1 -- 172.21.15.40:0/3746271158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56780078e0 con 0x7f56800d5610 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.832+0000 7f5684e74640 1 -- 172.21.15.40:0/3746271158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678007cb0 con 0x7f56800d5610 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 -- 172.21.15.40:0/3746271158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800d5610 msgr2=0x7f56800d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 --2- 172.21.15.40:0/3746271158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800d5610 0x7f56800d3d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f56800d6000 tx=0x7f567802a5f0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 -- 172.21.15.40:0/3746271158 shutdown_connections 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 --2- 172.21.15.40:0/3746271158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800d5610 0x7f56800d3d40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 --2- 172.21.15.40:0/3746271158 >> v2:172.21.15.184:3300/0 conn(0x7f568015f340 0x7f56800b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 -- 172.21.15.40:0/3746271158 >> 172.21.15.40:0/3746271158 conn(0x7f568008ec70 msgr2=0x7f568009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 -- 172.21.15.40:0/3746271158 shutdown_connections 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.833+0000 7f56880da640 1 -- 172.21.15.40:0/3746271158 wait complete. 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.834+0000 7f56880da640 1 Processor -- start 2024-04-26T21:13:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.834+0000 7f56880da640 1 -- start start 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 0x7f5680158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 0x7f568009f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56800b3940 con 0x7f568015f720 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56800b37c0 con 0x7f568015f340 2024-04-26T21:13:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f56880da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56800b3ac0 con 0x7f56800d5610 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36338/0 (socket says 172.21.15.40:36338) 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 -- 172.21.15.40:0/110254531 learned_addr learned my addr 172.21.15.40:0/110254531 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5685675640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 0x7f568009f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 -- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 msgr2=0x7f5680158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5685e76640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 0x7f5680158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 0x7f5680158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 -- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 msgr2=0x7f568009f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 0x7f568009f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5686677640 1 -- 172.21.15.40:0/110254531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56800a2990 con 0x7f568015f720 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5685e76640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 0x7f5680158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.835+0000 7f5685675640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 0x7f568009f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f5686677640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f567802abe0 tx=0x7f5678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678040070 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5678031bc0 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56800a23d0 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678045460 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.836+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56800b7ee0 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.837+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568006a290 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.837+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5678039040 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.838+0000 7f56767fc640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 0x7f566406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.838+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f56780ad4c0 con 0x7f568015f720 2024-04-26T21:13:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.838+0000 7f5685e76640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 0x7f566406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.839+0000 7f5685e76640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 0x7f566406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f567c006fd0 tx=0x7f567c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:53.842+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f567807b860 con 0x7f568015f720 2024-04-26T21:13:54.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.037+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f56800b4bd0 con 0x7f568015f720 2024-04-26T21:13:54.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.037+0000 7f56767fc640 1 -- 172.21.15.40:0/110254531 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f567807b160 con 0x7f568015f720 2024-04-26T21:13:54.039 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 msgr2=0x7f566406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 0x7f566406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f567c006fd0 tx=0x7f567c008040 comp rx=0 tx=0).stop 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 msgr2=0x7f56800cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f567802abe0 tx=0x7f5678004a40 comp rx=0 tx=0).stop 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 shutdown_connections 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566406c8f0 0x7f566406edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568015f720 0x7f56800cd880 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568015f340 0x7f568009f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 --2- 172.21.15.40:0/110254531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56800d5610 0x7f5680158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 >> 172.21.15.40:0/110254531 conn(0x7f568008ec70 msgr2=0x7f56800b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 shutdown_connections 2024-04-26T21:13:54.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.039+0000 7f56880da640 1 -- 172.21.15.40:0/110254531 wait complete. 2024-04-26T21:13:54.051 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:13:54.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:54.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:13:54.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:13:54.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:13:54.084 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:13:54.142 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:13:54.142 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:13:54.143 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:54.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:13:54.159 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:13:54.159 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:13:54.159 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:13:54.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:13:54.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.299+0000 7feea3d89640 1 Processor -- start 2024-04-26T21:13:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.299+0000 7feea3d89640 1 -- start start 2024-04-26T21:13:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea3d89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee9c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea3d89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c0d1740 con 0x7fee9c06e2c0 2024-04-26T21:13:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea3d89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c0d1a20 con 0x7fee9c0cf2e0 2024-04-26T21:13:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea3d89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c06e750 con 0x7fee9c0d11b0 2024-04-26T21:13:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea2326640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fee9c06e2c0 0x7fee9c15ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52908/0 (socket says 172.21.15.40:52908) 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea2326640 1 -- 172.21.15.40:0/1655594347 learned_addr learned my addr 172.21.15.40:0/1655594347 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea1324640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee9c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12955726 0 0) 0x7fee9c0d1740 con 0x7fee9c06e2c0 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.300+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee84003520 con 0x7fee9c06e2c0 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 622780583 0 0) 0x7fee84003520 con 0x7fee9c06e2c0 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 msgr2=0x7fee9c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee9c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 >> v1:172.21.15.184:6789/0 conn(0x7fee9c0cf2e0 legacy=0x7fee9c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c0d1d00 con 0x7fee9c06e2c0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fee8c0019f0 con 0x7fee9c06e2c0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 >> v1:172.21.15.40:6789/0 conn(0x7fee9c06e2c0 legacy=0x7fee9c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee84003f20 0x7fee84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee84004d40 0x7fee840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c06e750 con 0x7fee84003f20 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee8400ba70 con 0x7fee84004d40 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee8400bc70 con 0x7fee9c0d11b0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee84003f20 0x7fee84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1b25640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea2326640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee84004d40 0x7fee840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 -- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 msgr2=0x7fee840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee9c0d11b0 0x7fee840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 -- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee84004d40 msgr2=0x7fee840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee84004d40 0x7fee840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.301+0000 7feea1324640 1 -- 172.21.15.40:0/1655594347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee84003520 con 0x7fee84003f20 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.302+0000 7feea1324640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee84003f20 0x7fee84004800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee90009ac0 tx=0x7fee84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35a4f336ea4b27d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.302+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90036070 con 0x7fee84003f20 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.302+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee90031400 con 0x7fee84003f20 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.302+0000 7feea0b23640 1 -- 172.21.15.40:0/1655594347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90009690 con 0x7fee84003f20 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 -- 172.21.15.40:0/1655594347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee84003f20 msgr2=0x7fee84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 --2- 172.21.15.40:0/1655594347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee84003f20 0x7fee84004800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee90009ac0 tx=0x7fee84005830 comp rx=0 tx=0).stop 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea2326640 1 -- 172.21.15.40:0/1655594347 reap_dead start 2024-04-26T21:13:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 -- 172.21.15.40:0/1655594347 shutdown_connections 2024-04-26T21:13:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 -- 172.21.15.40:0/1655594347 >> 172.21.15.40:0/1655594347 conn(0x7fee9c0b4a60 msgr2=0x7fee9c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 -- 172.21.15.40:0/1655594347 shutdown_connections 2024-04-26T21:13:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.303+0000 7feea3d89640 1 -- 172.21.15.40:0/1655594347 wait complete. 2024-04-26T21:13:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.304+0000 7feea3d89640 1 Processor -- start 2024-04-26T21:13:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.304+0000 7feea3d89640 1 -- start start 2024-04-26T21:13:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee84003f20 0x7fee9c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee9c0c94e0 0x7fee9c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0cf6c0 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0d22c0 con 0x7fee9c0c94e0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0cf840 con 0x7fee84003f20 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36352/0 (socket says 172.21.15.40:36352) 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 -- 172.21.15.40:0/3698814408 learned_addr learned my addr 172.21.15.40:0/3698814408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1b25640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee84003f20 0x7fee9c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea2326640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee9c0c94e0 0x7fee9c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 -- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee84003f20 msgr2=0x7fee9c0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee84003f20 0x7fee9c0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 -- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee9c0c94e0 msgr2=0x7fee9c0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee9c0c94e0 0x7fee9c0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 -- 172.21.15.40:0/3698814408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c0c7280 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea1324640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fee90034040 tx=0x7fee90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90040020 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee90003bc0 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c0c7560 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.305+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90036040 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.306+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c0c7b20 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.307+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c06a290 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.307+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee90043020 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.307+0000 7fee9a7fc640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 0x7fee8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.308+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fee900abb70 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.308+0000 7feea1b25640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 0x7fee8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.308+0000 7feea1b25640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 0x7fee8806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee94001910 tx=0x7fee94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.312+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee9004a070 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.532+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9c071270 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.533+0000 7fee9a7fc640 1 -- 172.21.15.40:0/3698814408 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7fee900044e0 con 0x7fee9c0c6aa0 2024-04-26T21:13:54.534 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:54.534 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:54.534 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 msgr2=0x7fee8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 0x7fee8806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee94001910 tx=0x7fee94009040 comp rx=0 tx=0).stop 2024-04-26T21:13:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 msgr2=0x7fee9c15e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fee90034040 tx=0x7fee90004730 comp rx=0 tx=0).stop 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 shutdown_connections 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8806c920 0x7fee8806ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee9c0c94e0 0x7fee9c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee9c0c6aa0 0x7fee9c15e550 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 --2- 172.21.15.40:0/3698814408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee84003f20 0x7fee9c0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 >> 172.21.15.40:0/3698814408 conn(0x7fee9c0b4a60 msgr2=0x7fee9c070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 shutdown_connections 2024-04-26T21:13:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.535+0000 7feea3d89640 1 -- 172.21.15.40:0/3698814408 wait complete. 2024-04-26T21:13:54.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:13:54.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.647+0000 7fbca77f7640 1 Processor -- start 2024-04-26T21:13:54.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.647+0000 7fbca77f7640 1 -- start start 2024-04-26T21:13:54.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca77f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbca00d1510 0x7fbca00d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca77f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca00ceef0 con 0x7fbca00d0b20 2024-04-26T21:13:54.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca77f7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca00cda60 con 0x7fbca015d360 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca77f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbca015d740 con 0x7fbca00d1510 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca4d92640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbca00d0b20 0x7fbca00ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52924/0 (socket says 172.21.15.40:52924) 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca4d92640 1 -- 172.21.15.40:0/155019459 learned_addr learned my addr 172.21.15.40:0/155019459 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca5d94640 1 --2- 172.21.15.40:0/155019459 >> v2:172.21.15.184:3300/0 conn(0x7fbca00d1510 0x7fbca00d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca5d94640 1 -- 172.21.15.40:0/155019459 >> v1:172.21.15.184:6789/0 conn(0x7fbca015d360 legacy=0x7fbca009c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca5d94640 1 -- 172.21.15.40:0/155019459 >> v1:172.21.15.40:6789/0 conn(0x7fbca00d0b20 legacy=0x7fbca00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.648+0000 7fbca5d94640 1 -- 172.21.15.40:0/155019459 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca00d1100 con 0x7fbca00d1510 2024-04-26T21:13:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.649+0000 7fbca5d94640 1 --2- 172.21.15.40:0/155019459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca00d1510 0x7fbca00d19a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009d90 tx=0x7fbc9802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43c4037ee49fe41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbc97fff640 1 -- 172.21.15.40:0/155019459 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98007740 con 0x7fbca00d1510 2024-04-26T21:13:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbc97fff640 1 -- 172.21.15.40:0/155019459 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc980078e0 con 0x7fbca00d1510 2024-04-26T21:13:54.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbc97fff640 1 -- 172.21.15.40:0/155019459 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98007cb0 con 0x7fbca00d1510 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbca77f7640 1 -- 172.21.15.40:0/155019459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca00d1510 msgr2=0x7fbca00d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbca77f7640 1 --2- 172.21.15.40:0/155019459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca00d1510 0x7fbca00d19a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009d90 tx=0x7fbc9802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbca77f7640 1 -- 172.21.15.40:0/155019459 shutdown_connections 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbca77f7640 1 --2- 172.21.15.40:0/155019459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca00d1510 0x7fbca00d19a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.650+0000 7fbca77f7640 1 -- 172.21.15.40:0/155019459 >> 172.21.15.40:0/155019459 conn(0x7fbca00b4a40 msgr2=0x7fbca00b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.651+0000 7fbca77f7640 1 -- 172.21.15.40:0/155019459 shutdown_connections 2024-04-26T21:13:54.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.651+0000 7fbca77f7640 1 -- 172.21.15.40:0/155019459 wait complete. 2024-04-26T21:13:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.652+0000 7fbca77f7640 1 Processor -- start 2024-04-26T21:13:54.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.652+0000 7fbca77f7640 1 -- start start 2024-04-26T21:13:54.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 0x7fbca01548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca015d360 0x7fbca015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0070310 con 0x7fbca00d1510 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbca00cdfe0 con 0x7fbca015d360 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbca0070490 con 0x7fbca00d0b20 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36360/0 (socket says 172.21.15.40:36360) 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 -- 172.21.15.40:0/1092187476 learned_addr learned my addr 172.21.15.40:0/1092187476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca5593640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 0x7fbca01548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca5d94640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca015d360 0x7fbca015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 -- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 msgr2=0x7fbca01548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 0x7fbca01548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 -- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca015d360 msgr2=0x7fbca015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca015d360 0x7fbca015e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 -- 172.21.15.40:0/1092187476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca015f0e0 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca5593640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 0x7fbca01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca4d92640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c4d0 tx=0x7fbc9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90017070 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc90005ce0 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.653+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca015eb20 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.654+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90007270 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.654+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca006d9d0 con 0x7fbca00d1510 2024-04-26T21:13:54.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.655+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca00693d0 con 0x7fbca00d1510 2024-04-26T21:13:54.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.655+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc9001a070 con 0x7fbca00d1510 2024-04-26T21:13:54.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.656+0000 7fbc95ffb640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 0x7fbc8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.656+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fbc9008adb0 con 0x7fbca00d1510 2024-04-26T21:13:54.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.656+0000 7fbca5593640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 0x7fbc8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.656+0000 7fbca5593640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 0x7fbc8406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c001660 tx=0x7fbc9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.660+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc9008f070 con 0x7fbca00d1510 2024-04-26T21:13:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.845+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbca0074880 con 0x7fbca00d1510 2024-04-26T21:13:54.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.846+0000 7fbc95ffb640 1 -- 172.21.15.40:0/1092187476 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7fbc90058b80 con 0x7fbca00d1510 2024-04-26T21:13:54.847 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:54.847 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 msgr2=0x7fbc8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 0x7fbc8406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c001660 tx=0x7fbc9c009040 comp rx=0 tx=0).stop 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 msgr2=0x7fbca0151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c4d0 tx=0x7fbc9000c990 comp rx=0 tx=0).stop 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 shutdown_connections 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc8406c8f0 0x7fbc8406edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbca015d360 0x7fbca015e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00d1510 0x7fbca0151990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 --2- 172.21.15.40:0/1092187476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbca00d0b20 0x7fbca01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 >> 172.21.15.40:0/1092187476 conn(0x7fbca00b4a40 msgr2=0x7fbca015b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 shutdown_connections 2024-04-26T21:13:54.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.848+0000 7fbca77f7640 1 -- 172.21.15.40:0/1092187476 wait complete. 2024-04-26T21:13:54.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:12:53.618832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:12:54.135627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:13:54.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:13:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 Processor -- start 2024-04-26T21:13:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 -- start start 2024-04-26T21:13:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f455c0cd940 0x7f455c070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c0cfec0 con 0x7f455c0737a0 2024-04-26T21:13:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c0d01a0 con 0x7f455c0d14d0 2024-04-26T21:13:54.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.963+0000 7f4561914640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0d07f0 con 0x7f455c0cd940 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f455c0737a0 0x7f455c06f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52926/0 (socket says 172.21.15.40:52926) 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455b7fe640 1 -- 172.21.15.40:0/2377072658 learned_addr learned my addr 172.21.15.40:0/2377072658 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325515366 0 0) 0x7f455c0cfec0 con 0x7f455c0737a0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4544003520 con 0x7f455c0737a0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455affd640 1 --2- 172.21.15.40:0/2377072658 >> v2:172.21.15.184:3300/0 conn(0x7f455c0cd940 0x7f455c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1898236948 0 0) 0x7f4544003520 con 0x7f455c0737a0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 >> v2:172.21.15.184:3300/0 conn(0x7f455c0cd940 msgr2=0x7f455c070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 --2- 172.21.15.40:0/2377072658 >> v2:172.21.15.184:3300/0 conn(0x7f455c0cd940 0x7f455c070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 >> v1:172.21.15.184:6789/0 conn(0x7f455c0d14d0 legacy=0x7f455c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0d0480 con 0x7f455c0737a0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f4554001a10 con 0x7f455c0737a0 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 >> v1:172.21.15.40:6789/0 conn(0x7f455c0737a0 legacy=0x7f455c06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f4544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4544004800 0x7f4544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45440051c0 0x7f454400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f454400c030 con 0x7f45440039b0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f454400c1e0 con 0x7f45440051c0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f454400c3e0 con 0x7f4544004800 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455affd640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f4544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455b7fe640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45440051c0 0x7f454400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.964+0000 7f455a7fc640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4544004800 0x7f4544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 -- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4544004800 msgr2=0x7f4544004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4544004800 0x7f4544004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 -- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45440051c0 msgr2=0x7f454400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45440051c0 0x7f454400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455b7fe640 1 -- 172.21.15.40:0/2377072658 reap_dead start 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 -- 172.21.15.40:0/2377072658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4544003520 con 0x7f45440039b0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f455affd640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f4544003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f454c009ac0 tx=0x7f4544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260dc76e8d09dbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c036070 con 0x7f45440039b0 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c031400 con 0x7f45440039b0 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.965+0000 7f4559ffb640 1 -- 172.21.15.40:0/2377072658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c009690 con 0x7f45440039b0 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 -- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 msgr2=0x7f4544003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f4544003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f454c009ac0 tx=0x7f4544005830 comp rx=0 tx=0).stop 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 -- 172.21.15.40:0/2377072658 shutdown_connections 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 --2- 172.21.15.40:0/2377072658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f4544003dd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 -- 172.21.15.40:0/2377072658 >> 172.21.15.40:0/2377072658 conn(0x7f455c0d1e40 msgr2=0x7f455c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 -- 172.21.15.40:0/2377072658 shutdown_connections 2024-04-26T21:13:54.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.966+0000 7f4561914640 1 -- 172.21.15.40:0/2377072658 wait complete. 2024-04-26T21:13:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.967+0000 7f4561914640 1 Processor -- start 2024-04-26T21:13:54.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.967+0000 7f4561914640 1 -- start start 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 0x7f455c14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 0x7f455c163270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f455c0d0c80 con 0x7f45440039b0 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f455c0d0e00 con 0x7f455c162df0 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f4561914640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f455c0d0f80 con 0x7f455c0c5950 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f455affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f455affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36374/0 (socket says 172.21.15.40:36374) 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f455affd640 1 -- 172.21.15.40:0/2829171160 learned_addr learned my addr 172.21.15.40:0/2829171160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f455b7fe640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 0x7f455c163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.968+0000 7f455a7fc640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 0x7f455c14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 -- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 msgr2=0x7f455c14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 0x7f455c14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 -- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 msgr2=0x7f455c163270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 0x7f455c163270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 -- 172.21.15.40:0/2829171160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0ca450 con 0x7f45440039b0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455a7fc640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 0x7f455c14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455b7fe640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 0x7f455c163270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f455affd640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f454c034040 tx=0x7f454c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c03f020 con 0x7f45440039b0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c037040 con 0x7f45440039b0 2024-04-26T21:13:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c0c60c0 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c036070 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.969+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c0c6680 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.970+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c0693d0 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.970+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f454c042020 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.971+0000 7f4543fff640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 0x7f454806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.971+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f454c0ab890 con 0x7f45440039b0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.971+0000 7f455a7fc640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 0x7f454806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.972+0000 7f455a7fc640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 0x7f454806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4550000c00 tx=0x7f4550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:54.975+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f454c049070 con 0x7f45440039b0 2024-04-26T21:13:55.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.157+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f455c0b7dc0 con 0x7f45440039b0 2024-04-26T21:13:55.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.157+0000 7f4543fff640 1 -- 172.21.15.40:0/2829171160 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f454c079cc0 con 0x7f45440039b0 2024-04-26T21:13:55.158 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 msgr2=0x7f454806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 0x7f454806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4550000c00 tx=0x7f4550009040 comp rx=0 tx=0).stop 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 msgr2=0x7f455c0cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f454c034040 tx=0x7f454c004730 comp rx=0 tx=0).stop 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 shutdown_connections 2024-04-26T21:13:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f454806c8f0 0x7f454806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f455c162df0 0x7f455c163270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f455c0c5950 0x7f455c14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 --2- 172.21.15.40:0/2829171160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45440039b0 0x7f455c0cb910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 >> 172.21.15.40:0/2829171160 conn(0x7f455c0d1e40 msgr2=0x7f455c15fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 shutdown_connections 2024-04-26T21:13:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.159+0000 7f4561914640 1 -- 172.21.15.40:0/2829171160 wait complete. 2024-04-26T21:13:55.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:13:55.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.273+0000 7f46c5af2640 1 Processor -- start 2024-04-26T21:13:55.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.273+0000 7f46c5af2640 1 -- start start 2024-04-26T21:13:55.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46c5af2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46c006f340 0x7f46c00b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46c5af2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c00b46a0 con 0x7f46c00b3e50 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46c5af2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c00b5db0 con 0x7f46c00d0300 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46c5af2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f46c00b6090 con 0x7f46c006f340 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46c006f340 0x7f46c00b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f46c00b3e50 0x7f46c00720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52932/0 (socket says 172.21.15.40:52932) 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46c006f340 0x7f46c00b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36386/0 (socket says 172.21.15.40:36386) 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bffff640 1 -- 172.21.15.40:0/1857952087 learned_addr learned my addr 172.21.15.40:0/1857952087 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bf7fe640 1 -- 172.21.15.40:0/1857952087 >> v1:172.21.15.184:6789/0 conn(0x7f46c00d0300 legacy=0x7f46c006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bf7fe640 1 -- 172.21.15.40:0/1857952087 >> v1:172.21.15.40:6789/0 conn(0x7f46c00b3e50 legacy=0x7f46c00720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46bf7fe640 1 -- 172.21.15.40:0/1857952087 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c00b6370 con 0x7f46c006f340 2024-04-26T21:13:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.274+0000 7f46be7fc640 1 -- 172.21.15.40:0/1857952087 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501734034 0 0) 0x7f46c00b46a0 con 0x7f46c00b3e50 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.275+0000 7f46bf7fe640 1 --2- 172.21.15.40:0/1857952087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c006f340 0x7f46c00b6b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91493174737949bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.275+0000 7f46be7fc640 1 -- 172.21.15.40:0/1857952087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0037070 con 0x7f46c006f340 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.275+0000 7f46be7fc640 1 -- 172.21.15.40:0/1857952087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b0032440 con 0x7f46c006f340 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.275+0000 7f46be7fc640 1 -- 172.21.15.40:0/1857952087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b002ae60 con 0x7f46c006f340 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 -- 172.21.15.40:0/1857952087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c006f340 msgr2=0x7f46c00b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 --2- 172.21.15.40:0/1857952087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c006f340 0x7f46c00b6b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b002a110 comp rx=0 tx=0).stop 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 -- 172.21.15.40:0/1857952087 shutdown_connections 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 --2- 172.21.15.40:0/1857952087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c006f340 0x7f46c00b6b70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 -- 172.21.15.40:0/1857952087 >> 172.21.15.40:0/1857952087 conn(0x7f46c00cd8a0 msgr2=0x7f46c00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 -- 172.21.15.40:0/1857952087 shutdown_connections 2024-04-26T21:13:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.276+0000 7f46c5af2640 1 -- 172.21.15.40:0/1857952087 wait complete. 2024-04-26T21:13:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.277+0000 7f46c5af2640 1 Processor -- start 2024-04-26T21:13:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.277+0000 7f46c5af2640 1 -- start start 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c006f340 0x7f46c014faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c00d0300 0x7f46c00ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00b4460 con 0x7f46c00b3e50 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00b42e0 con 0x7f46c006f340 2024-04-26T21:13:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46c5af2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f46c0142a30 con 0x7f46c00d0300 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36398/0 (socket says 172.21.15.40:36398) 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46beffd640 1 -- 172.21.15.40:0/3670458008 learned_addr learned my addr 172.21.15.40:0/3670458008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46bffff640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c00d0300 0x7f46c00ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.278+0000 7f46bf7fe640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c006f340 0x7f46c014faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 -- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c00d0300 msgr2=0x7f46c00ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c00d0300 0x7f46c00ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 -- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c006f340 msgr2=0x7f46c014faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c006f340 0x7f46c014faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 -- 172.21.15.40:0/3670458008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c013c100 con 0x7f46c00b3e50 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46beffd640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c450 tx=0x7f46b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052d0 con 0x7f46c00b3e50 2024-04-26T21:13:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b4005470 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c013c3e0 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4012870 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.279+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c013c9a0 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.280+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c006a290 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.280+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46b4012a10 con 0x7f46c00b3e50 2024-04-26T21:13:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.281+0000 7f46a7fff640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 0x7f46ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.281+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f46b408b730 con 0x7f46c00b3e50 2024-04-26T21:13:55.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.281+0000 7f46bf7fe640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 0x7f46ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.282+0000 7f46bf7fe640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 0x7f46ac06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f46b0034000 tx=0x7f46b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.285+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46b4059400 con 0x7f46c00b3e50 2024-04-26T21:13:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.467+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f46c00b46a0 con 0x7f46c00b3e50 2024-04-26T21:13:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.468+0000 7f46a7fff640 1 -- 172.21.15.40:0/3670458008 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f46b405d700 con 0x7f46c00b3e50 2024-04-26T21:13:55.469 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.469+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 msgr2=0x7f46ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.469+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 0x7f46ac06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f46b0034000 tx=0x7f46b002e000 comp rx=0 tx=0).stop 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.469+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 msgr2=0x7f46c013b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.469+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c450 tx=0x7f46b400c910 comp rx=0 tx=0).stop 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 shutdown_connections 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f46ac06c8f0 0x7f46ac06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c00d0300 0x7f46c00ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c00b3e50 0x7f46c013b770 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 --2- 172.21.15.40:0/3670458008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c006f340 0x7f46c014faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 >> 172.21.15.40:0/3670458008 conn(0x7f46c00cd8a0 msgr2=0x7f46c0142d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 shutdown_connections 2024-04-26T21:13:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.470+0000 7f46c5af2640 1 -- 172.21.15.40:0/3670458008 wait complete. 2024-04-26T21:13:55.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:13:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.583+0000 7f55387e4640 1 Processor -- start 2024-04-26T21:13:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.583+0000 7f55387e4640 1 -- start start 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f55387e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f55300d0300 0x7f55300b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f55387e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f55300720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f55387e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55300b46a0 con 0x7f553006f340 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f55387e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f55300b5db0 con 0x7f55300d0300 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f55387e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55300b6090 con 0x7f55300b3e50 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f5536580640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f553006f340 0x7f553006f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52938/0 (socket says 172.21.15.40:52938) 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f5536580640 1 -- 172.21.15.40:0/1817489169 learned_addr learned my addr 172.21.15.40:0/1817489169 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898622980 0 0) 0x7f55300b46a0 con 0x7f553006f340 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.584+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f551c003520 con 0x7f553006f340 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5536d81640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f55300720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> v2:172.21.15.184:3300/0 conn(0x7f55300d0300 0x7f55300b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2147364091 0 0) 0x7f551c003520 con 0x7f553006f340 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 msgr2=0x7f55300720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f55300720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 >> v2:172.21.15.184:3300/0 conn(0x7f55300d0300 msgr2=0x7f55300b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 --2- 172.21.15.40:0/1817489169 >> v2:172.21.15.184:3300/0 conn(0x7f55300d0300 0x7f55300b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55300b6370 con 0x7f553006f340 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5528001670 con 0x7f553006f340 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> v2:172.21.15.184:3300/0 conn(0x7f55300d0300 0x7f55300b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 >> v1:172.21.15.40:6789/0 conn(0x7f553006f340 legacy=0x7f553006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f551c0039b0 0x7f551c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f551c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f551c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55300b5db0 con 0x7f551c004800 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55300b6090 con 0x7f551c0039b0 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f551c00bb20 con 0x7f55300b3e50 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f551c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5536580640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f551c0039b0 0x7f551c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5536d81640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f551c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 -- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 msgr2=0x7f551c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55300b3e50 0x7f551c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 -- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f551c0039b0 msgr2=0x7f551c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f551c0039b0 0x7f551c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.585+0000 7f5535d7f640 1 -- 172.21.15.40:0/1817489169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c003520 con 0x7f551c004800 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.586+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f551c004c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f55200098a0 tx=0x7f551c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf05e112b9e299a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.586+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552000f040 con 0x7f551c004800 2024-04-26T21:13:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.586+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552002ad20 con 0x7f551c004800 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.586+0000 7f553557e640 1 -- 172.21.15.40:0/1817489169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520004250 con 0x7f551c004800 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 -- 172.21.15.40:0/1817489169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 msgr2=0x7f551c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 --2- 172.21.15.40:0/1817489169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f551c004c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f55200098a0 tx=0x7f551c005830 comp rx=0 tx=0).stop 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f5536d81640 1 -- 172.21.15.40:0/1817489169 reap_dead start 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 -- 172.21.15.40:0/1817489169 shutdown_connections 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 -- 172.21.15.40:0/1817489169 >> 172.21.15.40:0/1817489169 conn(0x7f55300cd8a0 msgr2=0x7f55300cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 -- 172.21.15.40:0/1817489169 shutdown_connections 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.587+0000 7f55387e4640 1 -- 172.21.15.40:0/1817489169 wait complete. 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.588+0000 7f55387e4640 1 Processor -- start 2024-04-26T21:13:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.588+0000 7f55387e4640 1 -- start start 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553015e150 0x7f553015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55301f9000 0x7f55300cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55300b4460 con 0x7f551c004800 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55300b42e0 con 0x7f55301f9000 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5530142a30 con 0x7f553015e150 2024-04-26T21:13:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36428/0 (socket says 172.21.15.40:36428) 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 -- 172.21.15.40:0/1228736414 learned_addr learned my addr 172.21.15.40:0/1228736414 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553015e150 0x7f553015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536d81640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55301f9000 0x7f55300cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 -- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553015e150 msgr2=0x7f553015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553015e150 0x7f553015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 -- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55301f9000 msgr2=0x7f55300cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55301f9000 0x7f55300cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 -- 172.21.15.40:0/1228736414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553015e660 con 0x7f551c004800 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5536580640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5528009390 tx=0x7f5528009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552800fae0 con 0x7f551c004800 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552800fc80 con 0x7f551c004800 2024-04-26T21:13:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.589+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553015e940 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.590+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528016430 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.590+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553015ef00 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.590+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553006a290 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.591+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f552801d020 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.591+0000 7f5526ffd640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.591+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f552808bb30 con 0x7f551c004800 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.592+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.592+0000 7f5535d7f640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 0x7f551406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5520033000 tx=0x7f552002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.595+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5528059ff0 con 0x7f551c004800 2024-04-26T21:13:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.784+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f55300b46a0 con 0x7f551c004800 2024-04-26T21:13:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.784+0000 7f5526ffd640 1 -- 172.21.15.40:0/1228736414 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f55280598f0 con 0x7f551c004800 2024-04-26T21:13:55.785 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.786+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 msgr2=0x7f551406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.786+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 0x7f551406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5520033000 tx=0x7f552002e000 comp rx=0 tx=0).stop 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.786+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 msgr2=0x7f5530131db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5528009390 tx=0x7f5528009850 comp rx=0 tx=0).stop 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 shutdown_connections 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55301f9000 0x7f55300cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553015e150 0x7f553015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 --2- 172.21.15.40:0/1228736414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c004800 0x7f5530131db0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 >> 172.21.15.40:0/1228736414 conn(0x7f55300cd8a0 msgr2=0x7f55300737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 shutdown_connections 2024-04-26T21:13:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.787+0000 7f55387e4640 1 -- 172.21.15.40:0/1228736414 wait complete. 2024-04-26T21:13:55.798 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.898+0000 7f5240751640 1 Processor -- start 2024-04-26T21:13:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.898+0000 7f5240751640 1 -- start start 2024-04-26T21:13:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f5240751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f523815f340 0x7f52380b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f5240751640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f523815f720 0x7f52380d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f5240751640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52380b2510 con 0x7f52380d4f00 2024-04-26T21:13:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f5240751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f52380b27f0 con 0x7f523815f340 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f5240751640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f52380b2ad0 con 0x7f523815f720 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f523815f340 0x7f52380b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f523815f340 0x7f52380b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36432/0 (socket says 172.21.15.40:36432) 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 -- 172.21.15.40:0/718857739 learned_addr learned my addr 172.21.15.40:0/718857739 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523dcec640 1 --2- 172.21.15.40:0/718857739 >> v2:172.21.15.184:3300/0 conn(0x7f523815f720 0x7f52380d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 -- 172.21.15.40:0/718857739 >> v2:172.21.15.184:3300/0 conn(0x7f523815f720 msgr2=0x7f52380d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 --2- 172.21.15.40:0/718857739 >> v2:172.21.15.184:3300/0 conn(0x7f523815f720 0x7f52380d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 -- 172.21.15.40:0/718857739 >> v1:172.21.15.184:6789/0 conn(0x7f52380d4f00 legacy=0x7f52380d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.899+0000 7f523e4ed640 1 -- 172.21.15.40:0/718857739 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52380b2db0 con 0x7f523815f340 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.900+0000 7f523e4ed640 1 --2- 172.21.15.40:0/718857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523815f340 0x7f52380b1e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f52380cf8a0 tx=0x7f523002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb22512707762d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.900+0000 7f523d4eb640 1 -- 172.21.15.40:0/718857739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230037070 con 0x7f523815f340 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.900+0000 7f523d4eb640 1 -- 172.21.15.40:0/718857739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5230032440 con 0x7f523815f340 2024-04-26T21:13:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.900+0000 7f523d4eb640 1 -- 172.21.15.40:0/718857739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230032810 con 0x7f523815f340 2024-04-26T21:13:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 -- 172.21.15.40:0/718857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523815f340 msgr2=0x7f52380b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 --2- 172.21.15.40:0/718857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523815f340 0x7f52380b1e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f52380cf8a0 tx=0x7f523002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 -- 172.21.15.40:0/718857739 shutdown_connections 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 --2- 172.21.15.40:0/718857739 >> v2:172.21.15.184:3300/0 conn(0x7f523815f720 0x7f52380d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 --2- 172.21.15.40:0/718857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f523815f340 0x7f52380b1e80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 -- 172.21.15.40:0/718857739 >> 172.21.15.40:0/718857739 conn(0x7f523808ec70 msgr2=0x7f523809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 -- 172.21.15.40:0/718857739 shutdown_connections 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.901+0000 7f5240751640 1 -- 172.21.15.40:0/718857739 wait complete. 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.902+0000 7f5240751640 1 Processor -- start 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.902+0000 7f5240751640 1 -- start start 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523815f340 0x7f523814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 0x7f523814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52380b3940 con 0x7f52380d4f00 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52380b37c0 con 0x7f523815f720 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f5240751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52380b3ac0 con 0x7f523815f340 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36440/0 (socket says 172.21.15.40:36440) 2024-04-26T21:13:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 -- 172.21.15.40:0/2052685507 learned_addr learned my addr 172.21.15.40:0/2052685507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523dcec640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523815f340 0x7f523814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523ecee640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 0x7f523814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 -- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523815f340 msgr2=0x7f523814b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523815f340 0x7f523814b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 -- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 msgr2=0x7f523814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 0x7f523814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523e4ed640 1 -- 172.21.15.40:0/2052685507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523814c4c0 con 0x7f52380d4f00 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.903+0000 7f523ecee640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 0x7f523814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f523e4ed640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f52300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230041070 con 0x7f52380d4f00 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523003c870 con 0x7f52380d4f00 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523814c7a0 con 0x7f52380d4f00 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230037040 con 0x7f52380d4f00 2024-04-26T21:13:55.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.904+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523814cd60 con 0x7f52380d4f00 2024-04-26T21:13:55.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.905+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523806a290 con 0x7f52380d4f00 2024-04-26T21:13:55.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.905+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5230044070 con 0x7f52380d4f00 2024-04-26T21:13:55.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.906+0000 7f5236ffd640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 0x7f522406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:55.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.906+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f52300ac590 con 0x7f52380d4f00 2024-04-26T21:13:55.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.906+0000 7f523dcec640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 0x7f522406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:55.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.906+0000 7f523dcec640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 0x7f522406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f522c006fd0 tx=0x7f522c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:55.910+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f523007a9b0 con 0x7f52380d4f00 2024-04-26T21:13:56.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.132+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f523815edf0 con 0x7f52380d4f00 2024-04-26T21:13:56.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.132+0000 7f5236ffd640 1 -- 172.21.15.40:0/2052685507 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f523007a2b0 con 0x7f52380d4f00 2024-04-26T21:13:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:56.133 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:56.134 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:56.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 msgr2=0x7f522406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 0x7f522406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f522c006fd0 tx=0x7f522c008040 comp rx=0 tx=0).stop 2024-04-26T21:13:56.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 msgr2=0x7f523814d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f52300046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 shutdown_connections 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f522406c8f0 0x7f522406edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f523815f720 0x7f523814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f523815f340 0x7f523814b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 --2- 172.21.15.40:0/2052685507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52380d4f00 0x7f523814d560 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 >> 172.21.15.40:0/2052685507 conn(0x7f523808ec70 msgr2=0x7f52380b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 shutdown_connections 2024-04-26T21:13:56.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.134+0000 7f5240751640 1 -- 172.21.15.40:0/2052685507 wait complete. 2024-04-26T21:13:56.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:13:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.245+0000 7f0e79bac640 1 Processor -- start 2024-04-26T21:13:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.245+0000 7f0e79bac640 1 -- start start 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e79bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e740cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e79bac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e740d3ee0 0x7f0e740b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e79bac640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e740d03a0 con 0x7f0e7415f750 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e79bac640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0e740b4d10 con 0x7f0e740d3ee0 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e79bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e740d0680 con 0x7f0e7415f370 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e73fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e740d3ee0 0x7f0e740b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e73fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e740d3ee0 0x7f0e740b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36452/0 (socket says 172.21.15.40:36452) 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e72ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e7415f750 0x7f0e740d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52950/0 (socket says 172.21.15.40:52950) 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.246+0000 7f0e73fff640 1 -- 172.21.15.40:0/835510106 learned_addr learned my addr 172.21.15.40:0/835510106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e737fe640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e740cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e73fff640 1 -- 172.21.15.40:0/835510106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 msgr2=0x7f0e740cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e73fff640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e740cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e73fff640 1 -- 172.21.15.40:0/835510106 >> v1:172.21.15.40:6789/0 conn(0x7f0e7415f750 legacy=0x7f0e740d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e727fc640 1 -- 172.21.15.40:0/835510106 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953084421 0 0) 0x7f0e740d03a0 con 0x7f0e7415f750 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e73fff640 1 -- 172.21.15.40:0/835510106 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e740d0960 con 0x7f0e740d3ee0 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.247+0000 7f0e73fff640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e740d3ee0 0x7f0e740b47d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009e50 tx=0x7f0e6c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5ec4232fc55c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e727fc640 1 -- 172.21.15.40:0/835510106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c036070 con 0x7f0e740d3ee0 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e727fc640 1 -- 172.21.15.40:0/835510106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6c031600 con 0x7f0e740d3ee0 2024-04-26T21:13:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e727fc640 1 -- 172.21.15.40:0/835510106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0319d0 con 0x7f0e740d3ee0 2024-04-26T21:13:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 -- 172.21.15.40:0/835510106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e740d3ee0 msgr2=0x7f0e740b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e740d3ee0 0x7f0e740b47d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009e50 tx=0x7f0e6c02a680 comp rx=0 tx=0).stop 2024-04-26T21:13:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 -- 172.21.15.40:0/835510106 shutdown_connections 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e740d3ee0 0x7f0e740b47d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 --2- 172.21.15.40:0/835510106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e740cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 -- 172.21.15.40:0/835510106 >> 172.21.15.40:0/835510106 conn(0x7f0e7408ec10 msgr2=0x7f0e7409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 -- 172.21.15.40:0/835510106 shutdown_connections 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.248+0000 7f0e79bac640 1 -- 172.21.15.40:0/835510106 wait complete. 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.249+0000 7f0e79bac640 1 Processor -- start 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 -- start start 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e740d3ee0 0x7f0e740ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e741574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e740b51c0 con 0x7f0e7415f750 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e740d0f20 con 0x7f0e740d3ee0 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e79bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e740b5340 con 0x7f0e7415f370 2024-04-26T21:13:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36462/0 (socket says 172.21.15.40:36462) 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.250+0000 7f0e73fff640 1 -- 172.21.15.40:0/2808273895 learned_addr learned my addr 172.21.15.40:0/2808273895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e72ffd640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e741574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e737fe640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e740d3ee0 0x7f0e740ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 -- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 msgr2=0x7f0e741574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e741574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 -- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e740d3ee0 msgr2=0x7f0e740ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e740d3ee0 0x7f0e740ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 -- 172.21.15.40:0/2808273895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e740ce960 con 0x7f0e7415f750 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e73fff640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c034040 tx=0x7f0e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c041020 con 0x7f0e7415f750 2024-04-26T21:13:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6c037040 con 0x7f0e7415f750 2024-04-26T21:13:56.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e740cc510 con 0x7f0e7415f750 2024-04-26T21:13:56.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c036050 con 0x7f0e7415f750 2024-04-26T21:13:56.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.251+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e74157a30 con 0x7f0e7415f750 2024-04-26T21:13:56.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.252+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7406a290 con 0x7f0e7415f750 2024-04-26T21:13:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.252+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e6c007c30 con 0x7f0e7415f750 2024-04-26T21:13:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.253+0000 7f0e5bfff640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.253+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f0e6c0ac290 con 0x7f0e7415f750 2024-04-26T21:13:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.253+0000 7f0e737fe640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.254+0000 7f0e737fe640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0e64006fd0 tx=0x7f0e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.257+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e6c04b060 con 0x7f0e7415f750 2024-04-26T21:13:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.442+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e740b3ae0 con 0x7f0e7415f750 2024-04-26T21:13:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.443+0000 7f0e5bfff640 1 -- 172.21.15.40:0/2808273895 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7f0e6c0a9130 con 0x7f0e7415f750 2024-04-26T21:13:56.444 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:56.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 msgr2=0x7f0e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0e64006fd0 tx=0x7f0e64008040 comp rx=0 tx=0).stop 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 msgr2=0x7f0e740ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c034040 tx=0x7f0e6c004a40 comp rx=0 tx=0).stop 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 shutdown_connections 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7415f750 0x7f0e740ce010 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e7415f370 0x7f0e741574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 --2- 172.21.15.40:0/2808273895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e740d3ee0 0x7f0e740ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 >> 172.21.15.40:0/2808273895 conn(0x7f0e7408ec10 msgr2=0x7f0e740d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 shutdown_connections 2024-04-26T21:13:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.445+0000 7f0e79bac640 1 -- 172.21.15.40:0/2808273895 wait complete. 2024-04-26T21:13:56.457 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:13:56.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.556+0000 7f0aa7f89640 1 Processor -- start 2024-04-26T21:13:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.556+0000 7f0aa7f89640 1 -- start start 2024-04-26T21:13:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa7f89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0aa015f720 0x7f0aa00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa7f89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0aa00d5610 0x7f0aa00d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa7f89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa00b2510 con 0x7f0aa015f340 2024-04-26T21:13:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa7f89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00b27f0 con 0x7f0aa00d5610 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa7f89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00b2ad0 con 0x7f0aa015f720 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa6526640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0aa00d5610 0x7f0aa00d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa6526640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0aa00d5610 0x7f0aa00d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36470/0 (socket says 172.21.15.40:36470) 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa6526640 1 -- 172.21.15.40:0/805753233 learned_addr learned my addr 172.21.15.40:0/805753233 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa5524640 1 --2- 172.21.15.40:0/805753233 >> v2:172.21.15.184:3300/0 conn(0x7f0aa015f720 0x7f0aa00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa6526640 1 -- 172.21.15.40:0/805753233 >> v2:172.21.15.184:3300/0 conn(0x7f0aa015f720 msgr2=0x7f0aa00b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.557+0000 7f0aa6526640 1 --2- 172.21.15.40:0/805753233 >> v2:172.21.15.184:3300/0 conn(0x7f0aa015f720 0x7f0aa00b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa6526640 1 -- 172.21.15.40:0/805753233 >> v1:172.21.15.184:6789/0 conn(0x7f0aa015f340 legacy=0x7f0aa00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa6526640 1 -- 172.21.15.40:0/805753233 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00b2db0 con 0x7f0aa00d5610 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa6526640 1 --2- 172.21.15.40:0/805753233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa00d5610 0x7f0aa00d3d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00cf8a0 tx=0x7f0a9002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312918eabda22aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa4d23640 1 -- 172.21.15.40:0/805753233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90036070 con 0x7f0aa00d5610 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa4d23640 1 -- 172.21.15.40:0/805753233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90031600 con 0x7f0aa00d5610 2024-04-26T21:13:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.558+0000 7f0aa4d23640 1 -- 172.21.15.40:0/805753233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900319d0 con 0x7f0aa00d5610 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 -- 172.21.15.40:0/805753233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa00d5610 msgr2=0x7f0aa00d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/805753233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa00d5610 0x7f0aa00d3d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00cf8a0 tx=0x7f0a9002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 -- 172.21.15.40:0/805753233 shutdown_connections 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/805753233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa00d5610 0x7f0aa00d3d40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/805753233 >> v2:172.21.15.184:3300/0 conn(0x7f0aa015f720 0x7f0aa00b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 -- 172.21.15.40:0/805753233 >> 172.21.15.40:0/805753233 conn(0x7f0aa008ec70 msgr2=0x7f0aa009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 -- 172.21.15.40:0/805753233 shutdown_connections 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.559+0000 7f0aa7f89640 1 -- 172.21.15.40:0/805753233 wait complete. 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.560+0000 7f0aa7f89640 1 Processor -- start 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.560+0000 7f0aa7f89640 1 -- start start 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00d5610 0x7f0aa00ba250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa015f340 0x7f0aa014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b3940 con 0x7f0aa015f720 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b37c0 con 0x7f0aa015f340 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa7f89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00b3ac0 con 0x7f0aa00d5610 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36484/0 (socket says 172.21.15.40:36484) 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 -- 172.21.15.40:0/1966664299 learned_addr learned my addr 172.21.15.40:0/1966664299 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa5d25640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00d5610 0x7f0aa00ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa5524640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa015f340 0x7f0aa014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 -- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00d5610 msgr2=0x7f0aa00ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00d5610 0x7f0aa00ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 -- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa015f340 msgr2=0x7f0aa014e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa015f340 0x7f0aa014e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.561+0000 7f0aa6526640 1 -- 172.21.15.40:0/1966664299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa0159c00 con 0x7f0aa015f720 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0aa6526640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002ac90 tx=0x7f0a90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9003f020 con 0x7f0aa015f720 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90037040 con 0x7f0aa015f720 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00b8a50 con 0x7f0aa015f720 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90036070 con 0x7f0aa015f720 2024-04-26T21:13:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.562+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa00b9a50 con 0x7f0aa015f720 2024-04-26T21:13:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.563+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa006a290 con 0x7f0aa015f720 2024-04-26T21:13:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.563+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a90042020 con 0x7f0aa015f720 2024-04-26T21:13:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.564+0000 7f0a9e7fc640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.564+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f0a900abfa0 con 0x7f0aa015f720 2024-04-26T21:13:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.564+0000 7f0aa5d25640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.564+0000 7f0aa5d25640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001640 tx=0x7f0a98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.568+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a90049030 con 0x7f0aa015f720 2024-04-26T21:13:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.793+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa00d3020 con 0x7f0aa015f720 2024-04-26T21:13:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.793+0000 7f0a9e7fc640 1 -- 172.21.15.40:0/1966664299 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7f0aa0159c00 con 0x7f0aa015f720 2024-04-26T21:13:56.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:56.794 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 msgr2=0x7f0a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001640 tx=0x7f0a98009040 comp rx=0 tx=0).stop 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 msgr2=0x7f0aa00ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002ac90 tx=0x7f0a90004a40 comp rx=0 tx=0).stop 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 shutdown_connections 2024-04-26T21:13:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa015f720 0x7f0aa00ca7f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa015f340 0x7f0aa014e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 --2- 172.21.15.40:0/1966664299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00d5610 0x7f0aa00ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 >> 172.21.15.40:0/1966664299 conn(0x7f0aa008ec70 msgr2=0x7f0aa00b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 shutdown_connections 2024-04-26T21:13:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.795+0000 7f0aa7f89640 1 -- 172.21.15.40:0/1966664299 wait complete. 2024-04-26T21:13:56.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:13:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.907+0000 7ff7829a4640 1 Processor -- start 2024-04-26T21:13:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.907+0000 7ff7829a4640 1 -- start start 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff7829a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff77c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff7829a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff77c0b3de0 0x7ff77c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff7829a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77c0b4260 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff7829a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0b6440 con 0x7ff77c0b3de0 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff7829a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0b5140 con 0x7ff77c145980 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff780f41640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff77c0b5fd0 0x7ff77c15c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52958/0 (socket says 172.21.15.40:52958) 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff780f41640 1 -- 172.21.15.40:0/386252531 learned_addr learned my addr 172.21.15.40:0/386252531 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77bfff640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff77c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77b7fe640 1 --2- 172.21.15.40:0/386252531 >> v2:172.21.15.184:3300/0 conn(0x7ff77c0b3de0 0x7ff77c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798708746 0 0) 0x7ff77c0b4260 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff764005730 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 196577400 0 0) 0x7ff764005730 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 msgr2=0x7ff77c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff77c160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 >> v2:172.21.15.184:3300/0 conn(0x7ff77c0b3de0 msgr2=0x7ff77c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 --2- 172.21.15.40:0/386252531 >> v2:172.21.15.184:3300/0 conn(0x7ff77c0b3de0 0x7ff77c0b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c0b4530 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff774001a10 con 0x7ff77c0b5fd0 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 >> v1:172.21.15.40:6789/0 conn(0x7ff77c0b5fd0 legacy=0x7ff77c15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.908+0000 7ff77affd640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff764005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77affd640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff764006140 0x7ff764006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77affd640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff764006f50 0x7ff7640073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b6440 con 0x7ff764006f50 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b5140 con 0x7ff764006140 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff76400dc20 con 0x7ff77c145980 2024-04-26T21:13:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff764006f50 0x7ff7640073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77bfff640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff764005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff77b7fe640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff764006140 0x7ff764006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 -- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 msgr2=0x7ff764005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c145980 0x7ff764005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 -- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff764006140 msgr2=0x7ff764006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff764006140 0x7ff764006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.909+0000 7ff780f41640 1 -- 172.21.15.40:0/386252531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff764005730 con 0x7ff764006f50 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff780f41640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff764006f50 0x7ff7640073d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff77400a8b0 tx=0x7ff764007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd10409297f282e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77400a4d0 con 0x7ff764006f50 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77402f070 con 0x7ff764006f50 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff77affd640 1 -- 172.21.15.40:0/386252531 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77402bca0 con 0x7ff764006f50 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 -- 172.21.15.40:0/386252531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff764006f50 msgr2=0x7ff7640073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 --2- 172.21.15.40:0/386252531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff764006f50 0x7ff7640073d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff77400a8b0 tx=0x7ff764007a40 comp rx=0 tx=0).stop 2024-04-26T21:13:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff780f41640 1 -- 172.21.15.40:0/386252531 reap_dead start 2024-04-26T21:13:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 -- 172.21.15.40:0/386252531 shutdown_connections 2024-04-26T21:13:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 -- 172.21.15.40:0/386252531 >> 172.21.15.40:0/386252531 conn(0x7ff77c0d0d30 msgr2=0x7ff77c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 -- 172.21.15.40:0/386252531 shutdown_connections 2024-04-26T21:13:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.910+0000 7ff7829a4640 1 -- 172.21.15.40:0/386252531 wait complete. 2024-04-26T21:13:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 Processor -- start 2024-04-26T21:13:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 -- start start 2024-04-26T21:13:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c15b350 0x7ff77c1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff77c15b730 0x7ff77c138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b4dd0 con 0x7ff77c143b30 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b4af0 con 0x7ff77c15b730 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.912+0000 7ff7829a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0730e0 con 0x7ff77c15b350 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36512/0 (socket says 172.21.15.40:36512) 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 -- 172.21.15.40:0/1719752639 learned_addr learned my addr 172.21.15.40:0/1719752639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff77bfff640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c15b350 0x7ff77c1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff77b7fe640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff77c15b730 0x7ff77c138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 -- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c15b350 msgr2=0x7ff77c1f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c15b350 0x7ff77c1f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 -- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff77c15b730 msgr2=0x7ff77c138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff77c15b730 0x7ff77c138c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 -- 172.21.15.40:0/1719752639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c1384b0 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff780f41640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff774001d10 tx=0x7ff77c0b50b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77400a4d0 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77402bd20 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c138790 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774002020 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.913+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1f1380 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.914+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c06a290 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.915+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff774034470 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.915+0000 7ff778ff9640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 0x7ff76806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.915+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7ff7740abae0 con 0x7ff77c143b30 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.915+0000 7ff77bfff640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 0x7ff76806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.916+0000 7ff77bfff640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 0x7ff76806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009bc0 tx=0x7ff76c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:56.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:56.920+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff774079fd0 con 0x7ff77c143b30 2024-04-26T21:13:57.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.146+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77c073cc0 con 0x7ff77c143b30 2024-04-26T21:13:57.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.147+0000 7ff778ff9640 1 -- 172.21.15.40:0/1719752639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5186 (secure 0 0 0) 0x7ff7740798d0 con 0x7ff77c143b30 2024-04-26T21:13:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:57.148 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:12:55.189839+0000","modified":"2024-04-26T21:12:57.948896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/1813208926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1813208926},{"type":"v1","addr":"172.21.15.40:6834","nonce":1813208926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5145,"qdb_cluster":[5145]},"id":2}]} 2024-04-26T21:13:57.149 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-26T21:13:57.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 msgr2=0x7ff76806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 0x7ff76806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009bc0 tx=0x7ff76c009190 comp rx=0 tx=0).stop 2024-04-26T21:13:57.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 msgr2=0x7ff77c143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff774001d10 tx=0x7ff77c0b50b0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 shutdown_connections 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff76806c8f0 0x7ff76806edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff77c143b30 0x7ff77c143fb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff77c15b730 0x7ff77c138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 --2- 172.21.15.40:0/1719752639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff77c15b350 0x7ff77c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 >> 172.21.15.40:0/1719752639 conn(0x7ff77c0d0d30 msgr2=0x7ff77c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 shutdown_connections 2024-04-26T21:13:57.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.149+0000 7ff7829a4640 1 -- 172.21.15.40:0/1719752639 wait complete. 2024-04-26T21:13:57.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:13:57.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.261+0000 7fa9407f0640 1 Processor -- start 2024-04-26T21:13:57.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.261+0000 7fa9407f0640 1 -- start start 2024-04-26T21:13:57.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.261+0000 7fa9407f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa93815f340 0x7fa9380b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa9407f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa9407f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9380b2510 con 0x7fa93815f720 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa9407f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa9380b27f0 con 0x7fa93815f340 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa9407f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380b2ad0 con 0x7fa9380d5610 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93dd8b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa93815f720 0x7fa9380d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52964/0 (socket says 172.21.15.40:52964) 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93dd8b640 1 -- 172.21.15.40:0/3326226202 learned_addr learned my addr 172.21.15.40:0/3326226202 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93e58c640 1 --2- 172.21.15.40:0/3326226202 >> v2:172.21.15.184:3300/0 conn(0x7fa93815f340 0x7fa9380b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93ed8d640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93d58a640 1 -- 172.21.15.40:0/3326226202 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852725814 0 0) 0x7fa9380b2510 con 0x7fa93815f720 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93d58a640 1 -- 172.21.15.40:0/3326226202 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa924003520 con 0x7fa93815f720 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93e58c640 1 -- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 msgr2=0x7fa9380d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93e58c640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93e58c640 1 -- 172.21.15.40:0/3326226202 >> v1:172.21.15.40:6789/0 conn(0x7fa93815f720 legacy=0x7fa9380d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93e58c640 1 -- 172.21.15.40:0/3326226202 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380b2db0 con 0x7fa93815f340 2024-04-26T21:13:57.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.262+0000 7fa93ed8d640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.263+0000 7fa93e58c640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f340 0x7fa9380b1e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9340097a0 tx=0x7fa93402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0ec168d73cbb012 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.263+0000 7fa93d58a640 1 -- 172.21.15.40:0/3326226202 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934037070 con 0x7fa93815f340 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.263+0000 7fa93d58a640 1 -- 172.21.15.40:0/3326226202 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93402ae50 con 0x7fa93815f340 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.263+0000 7fa93d58a640 1 -- 172.21.15.40:0/3326226202 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934032640 con 0x7fa93815f340 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 -- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f340 msgr2=0x7fa9380b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f340 0x7fa9380b1e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9340097a0 tx=0x7fa93402a100 comp rx=0 tx=0).stop 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 -- 172.21.15.40:0/3326226202 shutdown_connections 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 --2- 172.21.15.40:0/3326226202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f340 0x7fa9380b1e80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 -- 172.21.15.40:0/3326226202 >> 172.21.15.40:0/3326226202 conn(0x7fa93808ec70 msgr2=0x7fa93809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 -- 172.21.15.40:0/3326226202 shutdown_connections 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.264+0000 7fa9407f0640 1 -- 172.21.15.40:0/3326226202 wait complete. 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.265+0000 7fa9407f0640 1 Processor -- start 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.265+0000 7fa9407f0640 1 -- start start 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f720 0x7fa9380ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380b3940 con 0x7fa93815f340 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380b37c0 con 0x7fa93815f720 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380b3ac0 con 0x7fa9380d5610 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36520/0 (socket says 172.21.15.40:36520) 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 -- 172.21.15.40:0/2219103062 learned_addr learned my addr 172.21.15.40:0/2219103062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:57.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93ed8d640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f720 0x7fa9380ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93e58c640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 -- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 msgr2=0x7fa9380c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 -- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f720 msgr2=0x7fa9380ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f720 0x7fa9380ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 -- 172.21.15.40:0/2219103062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380ca490 con 0x7fa93815f340 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa93dd8b640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c390 tx=0x7fa92800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928017070 con 0x7fa93815f340 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa928005d00 con 0x7fa93815f340 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9380ca770 con 0x7fa93815f340 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.266+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928007310 con 0x7fa93815f340 2024-04-26T21:13:57.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.267+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380cad30 con 0x7fa93815f340 2024-04-26T21:13:57.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.268+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93806a290 con 0x7fa93815f340 2024-04-26T21:13:57.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.268+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa92801a070 con 0x7fa93815f340 2024-04-26T21:13:57.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.268+0000 7fa92effd640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.269+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa92808af00 con 0x7fa93815f340 2024-04-26T21:13:57.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.269+0000 7fa93e58c640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.269+0000 7fa93e58c640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 0x7fa91c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa934034000 tx=0x7fa93402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.273+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa928058c40 con 0x7fa93815f340 2024-04-26T21:13:57.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.457+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9380d3020 con 0x7fa93815f340 2024-04-26T21:13:57.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.458+0000 7fa92effd640 1 -- 172.21.15.40:0/2219103062 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16669 (secure 0 0 0) 0x7fa92805cf40 con 0x7fa93815f340 2024-04-26T21:13:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:13:57.460 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:12:56.826384+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:12:53.618832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:12:54.135627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 msgr2=0x7fa91c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 0x7fa91c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa934034000 tx=0x7fa93402e000 comp rx=0 tx=0).stop 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 msgr2=0x7fa9381586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c390 tx=0x7fa92800c850 comp rx=0 tx=0).stop 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 shutdown_connections 2024-04-26T21:13:57.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa93815f720 0x7fa9380ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa93815f340 0x7fa9381586e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 --2- 172.21.15.40:0/2219103062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9380d5610 0x7fa9380c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.460+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 >> 172.21.15.40:0/2219103062 conn(0x7fa93808ec70 msgr2=0x7fa9380b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.461+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 shutdown_connections 2024-04-26T21:13:57.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.461+0000 7fa9407f0640 1 -- 172.21.15.40:0/2219103062 wait complete. 2024-04-26T21:13:57.472 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:13:57.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.572+0000 7fe472efc640 1 Processor -- start 2024-04-26T21:13:57.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.572+0000 7fe472efc640 1 -- start start 2024-04-26T21:13:57.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 0x7fe46c146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe46c140350 0x7fe46c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe46c0d05b0 con 0x7fe46c140350 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe46c0d0890 con 0x7fe46c13ff70 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe472efc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe46c06e450 con 0x7fe46c0cf810 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe46bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe46c140350 0x7fe46c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.573+0000 7fe46bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe46c140350 0x7fe46c1465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36524/0 (socket says 172.21.15.40:36524) 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 -- 172.21.15.40:0/1612725108 learned_addr learned my addr 172.21.15.40:0/1612725108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe470c98640 1 --2- 172.21.15.40:0/1612725108 >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 0x7fe46c146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe471499640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 -- 172.21.15.40:0/1612725108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 msgr2=0x7fe46c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 -- 172.21.15.40:0/1612725108 >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 msgr2=0x7fe46c146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 --2- 172.21.15.40:0/1612725108 >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 0x7fe46c146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 -- 172.21.15.40:0/1612725108 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c06e730 con 0x7fe46c140350 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe470c98640 1 --2- 172.21.15.40:0/1612725108 >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 0x7fe46c146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.574+0000 7fe46bfff640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c140350 0x7fe46c1465c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe46c074200 tx=0x7fe46002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae4119b5b1ee3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe46b7fe640 1 -- 172.21.15.40:0/1612725108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460037070 con 0x7fe46c140350 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe46b7fe640 1 -- 172.21.15.40:0/1612725108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460032440 con 0x7fe46c140350 2024-04-26T21:13:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe46b7fe640 1 -- 172.21.15.40:0/1612725108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460032810 con 0x7fe46c140350 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 -- 172.21.15.40:0/1612725108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c140350 msgr2=0x7fe46c1465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c140350 0x7fe46c1465c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe46c074200 tx=0x7fe46002a140 comp rx=0 tx=0).stop 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 -- 172.21.15.40:0/1612725108 shutdown_connections 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 --2- 172.21.15.40:0/1612725108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c140350 0x7fe46c1465c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 --2- 172.21.15.40:0/1612725108 >> v2:172.21.15.184:3300/0 conn(0x7fe46c13ff70 0x7fe46c146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 -- 172.21.15.40:0/1612725108 >> 172.21.15.40:0/1612725108 conn(0x7fe46c0d0d30 msgr2=0x7fe46c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 -- 172.21.15.40:0/1612725108 shutdown_connections 2024-04-26T21:13:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.575+0000 7fe472efc640 1 -- 172.21.15.40:0/1612725108 wait complete. 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.576+0000 7fe472efc640 1 Processor -- start 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.576+0000 7fe472efc640 1 -- start start 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 0x7fe46c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c142a40 con 0x7fe46c13ff70 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1428c0 con 0x7fe46c140350 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe472efc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe46c142bc0 con 0x7fe46c0cf810 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36526/0 (socket says 172.21.15.40:36526) 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.577+0000 7fe46bfff640 1 -- 172.21.15.40:0/3790376473 learned_addr learned my addr 172.21.15.40:0/3790376473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe470c98640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 -- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 msgr2=0x7fe46c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe471499640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 0x7fe46c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 -- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 msgr2=0x7fe46c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 0x7fe46c158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 -- 172.21.15.40:0/3790376473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c0b0390 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe471499640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 0x7fe46c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe46bfff640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe4600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460037040 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460042070 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c0af7c0 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460030bb0 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.578+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c0af200 con 0x7fe46c13ff70 2024-04-26T21:13:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.579+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c06a290 con 0x7fe46c13ff70 2024-04-26T21:13:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.580+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe460047460 con 0x7fe46c13ff70 2024-04-26T21:13:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.580+0000 7fe4697fa640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 0x7fe45806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.580+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe4600ac7d0 con 0x7fe46c13ff70 2024-04-26T21:13:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.580+0000 7fe470c98640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 0x7fe45806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.581+0000 7fe470c98640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 0x7fe45806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006fd0 tx=0x7fe45c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.584+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe46007abf0 con 0x7fe46c13ff70 2024-04-26T21:13:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.814+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe46c0af5a0 con 0x7fe46c13ff70 2024-04-26T21:13:57.815 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:13:57.814+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:13:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.816+0000 7fe4697fa640 1 -- 172.21.15.40:0/3790376473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7fe46007a4f0 con 0x7fe46c13ff70 2024-04-26T21:13:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 msgr2=0x7fe45806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 0x7fe45806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006fd0 tx=0x7fe45c008040 comp rx=0 tx=0).stop 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 msgr2=0x7fe46c141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe4600046c0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 shutdown_connections 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe45806c8f0 0x7fe45806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe46c140350 0x7fe46c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe46c13ff70 0x7fe46c141700 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 --2- 172.21.15.40:0/3790376473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe46c0cf810 0x7fe46c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 >> 172.21.15.40:0/3790376473 conn(0x7fe46c0d0d30 msgr2=0x7fe46c06eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 shutdown_connections 2024-04-26T21:13:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.819+0000 7fe472efc640 1 -- 172.21.15.40:0/3790376473 wait complete. 2024-04-26T21:13:57.835 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:13:57.862 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:13:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.939+0000 7f88b0f5f640 1 Processor -- start 2024-04-26T21:13:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.939+0000 7f88b0f5f640 1 -- start start 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.939+0000 7f88b0f5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88ac06f340 0x7f88ac0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88b0f5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88b0f5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ac0b46a0 con 0x7f88ac0d0300 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88b0f5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f88ac0b5db0 con 0x7f88ac06f340 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88b0f5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88ac0b6090 con 0x7f88ac0b3e50 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aa59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88ac06f340 0x7f88ac0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88a1d9b640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f88ac0d0300 0x7f88ac06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:56574/0 (socket says 172.21.15.40:56574) 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88a1d9b640 1 -- 172.21.15.40:0/1273471281 learned_addr learned my addr 172.21.15.40:0/1273471281 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aa59c640 1 -- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 msgr2=0x7f88ac0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aa59c640 1 --2- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aa59c640 1 -- 172.21.15.40:0/1273471281 >> v1:172.21.15.184:6789/0 conn(0x7f88ac0d0300 legacy=0x7f88ac06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88a9d9b640 1 -- 172.21.15.40:0/1273471281 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1874476013 0 0) 0x7f88ac0b46a0 con 0x7f88ac0d0300 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.940+0000 7f88aa59c640 1 -- 172.21.15.40:0/1273471281 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac0b6370 con 0x7f88ac06f340 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.941+0000 7f88aa59c640 1 --2- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac06f340 0x7f88ac0b6b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f889c009d90 tx=0x7f889c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82cebda19b07b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.941+0000 7f88a9d9b640 1 -- 172.21.15.40:0/1273471281 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c007740 con 0x7f88ac06f340 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.941+0000 7f88a9d9b640 1 -- 172.21.15.40:0/1273471281 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c0078e0 con 0x7f88ac06f340 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.941+0000 7f88a9d9b640 1 -- 172.21.15.40:0/1273471281 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c007cb0 con 0x7f88ac06f340 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 -- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac06f340 msgr2=0x7f88ac0b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac06f340 0x7f88ac0b6b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f889c009d90 tx=0x7f889c02a530 comp rx=0 tx=0).stop 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 -- 172.21.15.40:0/1273471281 shutdown_connections 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/1273471281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac06f340 0x7f88ac0b6b70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 -- 172.21.15.40:0/1273471281 >> 172.21.15.40:0/1273471281 conn(0x7f88ac0cd8a0 msgr2=0x7f88ac0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 -- 172.21.15.40:0/1273471281 shutdown_connections 2024-04-26T21:13:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.942+0000 7f88b0f5f640 1 -- 172.21.15.40:0/1273471281 wait complete. 2024-04-26T21:13:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.943+0000 7f88b0f5f640 1 Processor -- start 2024-04-26T21:13:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 -- start start 2024-04-26T21:13:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac0d0300 0x7f88ac13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac142bb0 con 0x7f88ac06f340 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac142a30 con 0x7f88ac0d0300 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88b0f5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f88ac0b4260 con 0x7f88ac0b3e50 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88aa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88aa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36550/0 (socket says 172.21.15.40:36550) 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.944+0000 7f88aa59c640 1 -- 172.21.15.40:0/64582294 learned_addr learned my addr 172.21.15.40:0/64582294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88a1d9b640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aad9d640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac0d0300 0x7f88ac13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 -- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 msgr2=0x7f88ac1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac1f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 -- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac0d0300 msgr2=0x7f88ac13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac0d0300 0x7f88ac13f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 -- 172.21.15.40:0/64582294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac15f5a0 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88aa59c640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f889c009eb0 tx=0x7f889c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c040070 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c031c40 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac0736c0 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c009450 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.945+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac073c80 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.946+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ac1f8240 con 0x7f88ac06f340 2024-04-26T21:13:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.947+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f889c039040 con 0x7f88ac06f340 2024-04-26T21:13:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.947+0000 7f88a37fe640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 0x7f888c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.947+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f889c0acf50 con 0x7f88ac06f340 2024-04-26T21:13:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.947+0000 7f88a1d9b640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 0x7f888c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.948+0000 7f88a1d9b640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 0x7f888c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0ccf70 tx=0x7f8894007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:57.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:57.951+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f889c07acd0 con 0x7f88ac06f340 2024-04-26T21:13:58.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.174+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f88ac06a290 con 0x7f88ac06f340 2024-04-26T21:13:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.817+0000 7f88a37fe640 1 -- 172.21.15.40:0/64582294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f889c07efd0 con 0x7f88ac06f340 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 msgr2=0x7f888c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 0x7f888c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0ccf70 tx=0x7f8894007de0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 msgr2=0x7f88ac1414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f889c009eb0 tx=0x7f889c004a40 comp rx=0 tx=0).stop 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 shutdown_connections 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f888c06c8f0 0x7f888c06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88ac0d0300 0x7f88ac13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88ac0b3e50 0x7f88ac1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 --2- 172.21.15.40:0/64582294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88ac06f340 0x7f88ac1414a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 >> 172.21.15.40:0/64582294 conn(0x7f88ac0cd8a0 msgr2=0x7f88ac142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 shutdown_connections 2024-04-26T21:13:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.820+0000 7f88b0f5f640 1 -- 172.21.15.40:0/64582294 wait complete. 2024-04-26T21:13:58.835 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:13:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.984+0000 7f572c511640 1 Processor -- start 2024-04-26T21:13:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.984+0000 7f572c511640 1 -- start start 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572c511640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5724070940 0x7f572406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572c511640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f57240731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572c511640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57240736e0 con 0x7f57240d21f0 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572c511640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5724163e20 con 0x7f5724070940 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572c511640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57241632c0 con 0x7f5724148d40 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572aaae640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f57240d21f0 0x7f57240d1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52976/0 (socket says 172.21.15.40:52976) 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572aaae640 1 -- 172.21.15.40:0/3763010165 learned_addr learned my addr 172.21.15.40:0/3763010165 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811772924 0 0) 0x7f57240736e0 con 0x7f57240d21f0 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f5729aac640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f57240731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f572a2ad640 1 --2- 172.21.15.40:0/3763010165 >> v2:172.21.15.184:3300/0 conn(0x7f5724070940 0x7f572406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.985+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f570c003520 con 0x7f57240d21f0 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 848256411 0 0) 0x7f570c003520 con 0x7f57240d21f0 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 msgr2=0x7f57240731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f57240731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 >> v2:172.21.15.184:3300/0 conn(0x7f5724070940 msgr2=0x7f572406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 --2- 172.21.15.40:0/3763010165 >> v2:172.21.15.184:3300/0 conn(0x7f5724070940 0x7f572406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5724163580 con 0x7f57240d21f0 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f57140019f0 con 0x7f57240d21f0 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 >> v1:172.21.15.40:6789/0 conn(0x7f57240d21f0 legacy=0x7f57240d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f570c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f570c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5724163e20 con 0x7f570c0039b0 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57241632c0 con 0x7f570c004800 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f570c00b9c0 con 0x7f5724148d40 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f570c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f5729aac640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572aaae640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f570c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 -- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 msgr2=0x7f570c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5724148d40 0x7f570c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 -- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f570c004800 msgr2=0x7f570c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.986+0000 7f572a2ad640 1 -- 172.21.15.40:0/3763010165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c003520 con 0x7f570c0039b0 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.987+0000 7f572a2ad640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f570c003dd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f571c009ac0 tx=0x7f570c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af27dcf9079c0bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.987+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c036070 con 0x7f570c0039b0 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.987+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c031400 con 0x7f570c0039b0 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.987+0000 7f57292ab640 1 -- 172.21.15.40:0/3763010165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c009690 con 0x7f570c0039b0 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 -- 172.21.15.40:0/3763010165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 msgr2=0x7f570c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 --2- 172.21.15.40:0/3763010165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f570c003dd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f571c009ac0 tx=0x7f570c005830 comp rx=0 tx=0).stop 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572aaae640 1 -- 172.21.15.40:0/3763010165 reap_dead start 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 -- 172.21.15.40:0/3763010165 shutdown_connections 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 -- 172.21.15.40:0/3763010165 >> 172.21.15.40:0/3763010165 conn(0x7f572406d590 msgr2=0x7f572406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 -- 172.21.15.40:0/3763010165 shutdown_connections 2024-04-26T21:13:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.988+0000 7f572c511640 1 -- 172.21.15.40:0/3763010165 wait complete. 2024-04-26T21:13:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.989+0000 7f572c511640 1 Processor -- start 2024-04-26T21:13:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.989+0000 7f572c511640 1 -- start start 2024-04-26T21:13:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5724140450 0x7f572413d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f572413ddd0 0x7f57241f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5724162fb0 con 0x7f570c0039b0 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5724163b40 con 0x7f5724140450 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57240731a0 con 0x7f572413ddd0 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36562/0 (socket says 172.21.15.40:36562) 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 -- 172.21.15.40:0/125251966 learned_addr learned my addr 172.21.15.40:0/125251966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572aaae640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f572413ddd0 0x7f57241f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f5729aac640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5724140450 0x7f572413d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 -- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f572413ddd0 msgr2=0x7f57241f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f572413ddd0 0x7f57241f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 -- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5724140450 msgr2=0x7f572413d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5724140450 0x7f572413d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 -- 172.21.15.40:0/125251966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57240b12c0 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572a2ad640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f571c009be0 tx=0x7f571c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c040020 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c003950 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572413e580 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c036040 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.990+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572413eb40 con 0x7f570c0039b0 2024-04-26T21:13:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.991+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572406a290 con 0x7f570c0039b0 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.992+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f571c043020 con 0x7f570c0039b0 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.992+0000 7f5722ffd640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.992+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f571c0ac080 con 0x7f570c0039b0 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.992+0000 7f5729aac640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.993+0000 7f5729aac640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 0x7f571006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5724162b70 tx=0x7f5718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:58.997+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f571c04a030 con 0x7f570c0039b0 2024-04-26T21:13:59.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.186+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5724162ba0 con 0x7f570c0039b0 2024-04-26T21:13:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.822+0000 7f5722ffd640 1 -- 172.21.15.40:0/125251966 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f571c07a3b0 con 0x7f570c0039b0 2024-04-26T21:13:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:13:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 msgr2=0x7f571006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 0x7f571006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5724162b70 tx=0x7f5718008040 comp rx=0 tx=0).stop 2024-04-26T21:13:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 msgr2=0x7f572413ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f571c009be0 tx=0x7f571c004730 comp rx=0 tx=0).stop 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 shutdown_connections 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f572413ddd0 0x7f57241f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5724140450 0x7f572413d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 --2- 172.21.15.40:0/125251966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f570c0039b0 0x7f572413ff10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 >> 172.21.15.40:0/125251966 conn(0x7f572406d590 msgr2=0x7f5724146ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 shutdown_connections 2024-04-26T21:13:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.826+0000 7f572c511640 1 -- 172.21.15.40:0/125251966 wait complete. 2024-04-26T21:13:59.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:13:59.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.952+0000 7fdee0e93640 1 Processor -- start 2024-04-26T21:13:59.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.952+0000 7fdee0e93640 1 -- start start 2024-04-26T21:13:59.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdee0e93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdedc162a10 0x7fdedc0d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:59.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdee0e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdedc0d33a0 con 0x7fdedc0d2140 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdee0e93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdedc0d3680 con 0x7fdedc0b6020 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdee0e93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdedc0b4970 con 0x7fdedc162a10 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdeda59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdedc162a10 0x7fdedc0d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdedad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdedc0d2140 0x7fdedc0cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52982/0 (socket says 172.21.15.40:52982) 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdeda59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdedc162a10 0x7fdedc0d2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36578/0 (socket says 172.21.15.40:36578) 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdedad9d640 1 -- 172.21.15.40:0/3265411840 learned_addr learned my addr 172.21.15.40:0/3265411840 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdeda59c640 1 -- 172.21.15.40:0/3265411840 >> v1:172.21.15.184:6789/0 conn(0x7fdedc0b6020 legacy=0x7fdedc0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdeda59c640 1 -- 172.21.15.40:0/3265411840 >> v1:172.21.15.40:6789/0 conn(0x7fdedc0d2140 legacy=0x7fdedc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fdeda59c640 1 -- 172.21.15.40:0/3265411840 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc0d3960 con 0x7fdedc162a10 2024-04-26T21:13:59.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.953+0000 7fded959a640 1 -- 172.21.15.40:0/3265411840 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485374228 0 0) 0x7fdedc0d33a0 con 0x7fdedc0d2140 2024-04-26T21:13:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.954+0000 7fdeda59c640 1 --2- 172.21.15.40:0/3265411840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0d2dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fded40097a0 tx=0x7fded402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43222e42658e7089 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.954+0000 7fded959a640 1 -- 172.21.15.40:0/3265411840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037070 con 0x7fdedc162a10 2024-04-26T21:13:59.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.954+0000 7fded959a640 1 -- 172.21.15.40:0/3265411840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4032440 con 0x7fdedc162a10 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.954+0000 7fded959a640 1 -- 172.21.15.40:0/3265411840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4032810 con 0x7fdedc162a10 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 -- 172.21.15.40:0/3265411840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 msgr2=0x7fdedc0d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 --2- 172.21.15.40:0/3265411840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0d2dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fded40097a0 tx=0x7fded402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 -- 172.21.15.40:0/3265411840 shutdown_connections 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 --2- 172.21.15.40:0/3265411840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0d2dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 -- 172.21.15.40:0/3265411840 >> 172.21.15.40:0/3265411840 conn(0x7fdedc09e530 msgr2=0x7fdedc0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 -- 172.21.15.40:0/3265411840 shutdown_connections 2024-04-26T21:13:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.955+0000 7fdee0e93640 1 -- 172.21.15.40:0/3265411840 wait complete. 2024-04-26T21:13:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.956+0000 7fdee0e93640 1 Processor -- start 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.956+0000 7fdee0e93640 1 -- start start 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdedc0b6020 0x7fdedc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 0x7fdedc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc162df0 con 0x7fdedc162a10 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc0d3f20 con 0x7fdedc0b6020 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdee0e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdedc0b6600 con 0x7fdedc0d2140 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36584/0 (socket says 172.21.15.40:36584) 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 -- 172.21.15.40:0/324308153 learned_addr learned my addr 172.21.15.40:0/324308153 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdeda59c640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdedc0b6020 0x7fdedc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fded9d9b640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 0x7fdedc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 -- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 msgr2=0x7fdedc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 0x7fdedc15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 -- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdedc0b6020 msgr2=0x7fdedc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdedc0b6020 0x7fdedc15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fdedad9d640 1 -- 172.21.15.40:0/324308153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc1f9070 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.957+0000 7fded9d9b640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 0x7fdedc15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fdedad9d640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c730 tx=0x7fdecc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc018070 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc00ce40 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00a880 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1f9310 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.958+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc1f98d0 con 0x7fdedc162a10 2024-04-26T21:13:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.959+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdedc068460 con 0x7fdedc162a10 2024-04-26T21:13:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.959+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdecc00aa20 con 0x7fdedc162a10 2024-04-26T21:13:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.960+0000 7fded2ffd640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 0x7fdebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:13:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.960+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5357+0+0 (secure 0 0 0) 0x7fdecc08b130 con 0x7fdedc162a10 2024-04-26T21:13:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.960+0000 7fdeda59c640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 0x7fdebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:13:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.961+0000 7fdeda59c640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 0x7fdebc06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fded4034000 tx=0x7fded402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:13:59.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:13:59.964+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdedc068460 con 0x7fdedc162a10 2024-04-26T21:14:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.157+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdedc0b6ac0 con 0x7fdedc162a10 2024-04-26T21:14:00.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.824+0000 7fded2ffd640 1 -- 172.21.15.40:0/324308153 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7fdecc0595a0 con 0x7fdedc162a10 2024-04-26T21:14:00.825 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:14:00.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.827+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 msgr2=0x7fdebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.827+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 0x7fdebc06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fded4034000 tx=0x7fded402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:00.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.827+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 msgr2=0x7fdedc0b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.827+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c730 tx=0x7fdecc00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 shutdown_connections 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdebc06c8f0 0x7fdebc06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdedc162a10 0x7fdedc0b2400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdedc0d2140 0x7fdedc15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 --2- 172.21.15.40:0/324308153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdedc0b6020 0x7fdedc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.828+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 >> 172.21.15.40:0/324308153 conn(0x7fdedc09e530 msgr2=0x7fdedc0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:00.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.830+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 shutdown_connections 2024-04-26T21:14:00.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.831+0000 7fdee0e93640 1 -- 172.21.15.40:0/324308153 wait complete. 2024-04-26T21:14:00.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2024-04-26T21:14:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.957+0000 7ff5fd86b640 1 Processor -- start 2024-04-26T21:14:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.957+0000 7ff5fd86b640 1 -- start start 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5fd86b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5f8160950 0x7ff5f80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5fd86b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5f80cf430 0x7ff5f80cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5fd86b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f80d07e0 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5fd86b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f80cf040 con 0x7ff5f80cf430 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5fd86b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f80d2090 con 0x7ff5f8160950 2024-04-26T21:14:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5f80cf430 0x7ff5f80cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5f80cf430 0x7ff5f80cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41316/0 (socket says 172.21.15.40:41316) 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 -- 172.21.15.40:0/2400084360 learned_addr learned my addr 172.21.15.40:0/2400084360 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f6ffd640 1 --2- 172.21.15.40:0/2400084360 >> v2:172.21.15.184:3300/0 conn(0x7ff5f8160950 0x7ff5f80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 -- 172.21.15.40:0/2400084360 >> v2:172.21.15.184:3300/0 conn(0x7ff5f8160950 msgr2=0x7ff5f80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 --2- 172.21.15.40:0/2400084360 >> v2:172.21.15.184:3300/0 conn(0x7ff5f8160950 0x7ff5f80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 -- 172.21.15.40:0/2400084360 >> v1:172.21.15.40:6789/0 conn(0x7ff5f80b1ef0 legacy=0x7ff5f8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.958+0000 7ff5f77fe640 1 -- 172.21.15.40:0/2400084360 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f809c700 con 0x7ff5f80cf430 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.959+0000 7ff5f77fe640 1 --2- 172.21.15.40:0/2400084360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80cf430 0x7ff5f80cfec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80d2ef0 tx=0x7ff5f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90511e69ba052692 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.959+0000 7ff5f67fc640 1 -- 172.21.15.40:0/2400084360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0036070 con 0x7ff5f80cf430 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.959+0000 7ff5f67fc640 1 -- 172.21.15.40:0/2400084360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f0031600 con 0x7ff5f80cf430 2024-04-26T21:14:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.959+0000 7ff5f67fc640 1 -- 172.21.15.40:0/2400084360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00319d0 con 0x7ff5f80cf430 2024-04-26T21:14:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 -- 172.21.15.40:0/2400084360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80cf430 msgr2=0x7ff5f80cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/2400084360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80cf430 0x7ff5f80cfec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80d2ef0 tx=0x7ff5f002a730 comp rx=0 tx=0).stop 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 -- 172.21.15.40:0/2400084360 shutdown_connections 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/2400084360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80cf430 0x7ff5f80cfec0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/2400084360 >> v2:172.21.15.184:3300/0 conn(0x7ff5f8160950 0x7ff5f80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 -- 172.21.15.40:0/2400084360 >> 172.21.15.40:0/2400084360 conn(0x7ff5f809ba40 msgr2=0x7ff5f809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 -- 172.21.15.40:0/2400084360 shutdown_connections 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.960+0000 7ff5fd86b640 1 -- 172.21.15.40:0/2400084360 wait complete. 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.961+0000 7ff5fd86b640 1 Processor -- start 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.961+0000 7ff5fd86b640 1 -- start start 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f80cf430 0x7ff5f80b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5f8160950 0x7ff5f814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8160d30 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80d1d20 con 0x7ff5f80cf430 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5fd86b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f80b28f0 con 0x7ff5f8160950 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41328/0 (socket says 172.21.15.40:41328) 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 -- 172.21.15.40:0/3863280760 learned_addr learned my addr 172.21.15.40:0/3863280760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f77fe640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5f8160950 0x7ff5f814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5effff640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f80cf430 0x7ff5f80b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 -- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5f8160950 msgr2=0x7ff5f814d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5f8160950 0x7ff5f814d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 -- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f80cf430 msgr2=0x7ff5f80b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f80cf430 0x7ff5f80b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 -- 172.21.15.40:0/3863280760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f814f090 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.962+0000 7ff5f6ffd640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d830 tx=0x7ff5e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.963+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800b6c0 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.963+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e8019070 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.963+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f814c630 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.963+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800ba80 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.963+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f814ced0 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.964+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f806a290 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.964+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5e801e460 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.965+0000 7ff5f4ff9640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 0x7ff5d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.965+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7ff5e808bf30 con 0x7ff5f80b1ef0 2024-04-26T21:14:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.965+0000 7ff5effff640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 0x7ff5d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.965+0000 7ff5effff640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 0x7ff5d806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007fd0 tx=0x7ff5e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:00.969+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5f814f090 con 0x7ff5f80b1ef0 2024-04-26T21:14:01.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:14:01.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:14:01.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.160+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) -- 0x7ff5f80d13f0 con 0x7ff5f80b1ef0 2024-04-26T21:14:01.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.163+0000 7ff5f4ff9640 1 -- 172.21.15.40:0/3863280760 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7ff5e805a650 con 0x7ff5f80b1ef0 2024-04-26T21:14:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 msgr2=0x7ff5d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 0x7ff5d806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007fd0 tx=0x7ff5e0009040 comp rx=0 tx=0).stop 2024-04-26T21:14:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 msgr2=0x7ff5f814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d830 tx=0x7ff5e800dcf0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 shutdown_connections 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff5d806c8f0 0x7ff5d806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5f8160950 0x7ff5f814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f80cf430 0x7ff5f80b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 --2- 172.21.15.40:0/3863280760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f814eb20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 >> 172.21.15.40:0/3863280760 conn(0x7ff5f809ba40 msgr2=0x7ff5f8070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 shutdown_connections 2024-04-26T21:14:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.165+0000 7ff5fd86b640 1 -- 172.21.15.40:0/3863280760 wait complete. 2024-04-26T21:14:01.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:14:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 Processor -- start 2024-04-26T21:14:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 -- start start 2024-04-26T21:14:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f735406f340 0x7f73540b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73540b46a0 con 0x7f73540d0300 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73540b5db0 con 0x7f73540b3e50 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.349+0000 7f73591d4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f73540b6090 con 0x7f735406f340 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f735259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f73540d0300 0x7f735406f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32978/0 (socket says 172.21.15.40:32978) 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f735259c640 1 -- 172.21.15.40:0/3829019292 learned_addr learned my addr 172.21.15.40:0/3829019292 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3829019292 >> v2:172.21.15.184:3300/0 conn(0x7f735406f340 0x7f73540b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197278941 0 0) 0x7f73540b46a0 con 0x7f73540d0300 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7340003520 con 0x7f73540d0300 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 622103957 0 0) 0x7f7340003520 con 0x7f73540d0300 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 >> v2:172.21.15.184:3300/0 conn(0x7f735406f340 msgr2=0x7f73540b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 --2- 172.21.15.40:0/3829019292 >> v2:172.21.15.184:3300/0 conn(0x7f735406f340 0x7f73540b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 >> v1:172.21.15.184:6789/0 conn(0x7f73540b3e50 legacy=0x7f73540720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73540b6370 con 0x7f73540d0300 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f733c001670 con 0x7f73540d0300 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 >> v1:172.21.15.40:6789/0 conn(0x7f73540d0300 legacy=0x7f735406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73400039b0 0x7f7340003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 0x7f7340004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73400052e0 0x7f734000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73540b6090 con 0x7f7340004800 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f734000c1b0 con 0x7f73400039b0 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f734000c3b0 con 0x7f73400052e0 2024-04-26T21:14:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f735259c640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 0x7f7340004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73400039b0 0x7f7340003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.350+0000 7f735359e640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73400052e0 0x7f734000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 -- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73400052e0 msgr2=0x7f734000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73400052e0 0x7f734000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 -- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73400039b0 msgr2=0x7f7340003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73400039b0 0x7f7340003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 -- 172.21.15.40:0/3829019292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7340003520 con 0x7f7340004800 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735359e640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73400052e0 0x7f734000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735359e640 1 -- 172.21.15.40:0/3829019292 reap_dead start 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.351+0000 7f735259c640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 0x7f7340004c80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f733c001eb0 tx=0x7f7340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d1ef6a243372e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c007d30 con 0x7f7340004800 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c005af0 con 0x7f7340004800 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f7351d9b640 1 -- 172.21.15.40:0/3829019292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c007070 con 0x7f7340004800 2024-04-26T21:14:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 -- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 msgr2=0x7f7340004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 0x7f7340004c80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f733c001eb0 tx=0x7f7340005830 comp rx=0 tx=0).stop 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 -- 172.21.15.40:0/3829019292 shutdown_connections 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 --2- 172.21.15.40:0/3829019292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7340004800 0x7f7340004c80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 -- 172.21.15.40:0/3829019292 >> 172.21.15.40:0/3829019292 conn(0x7f73540cd8a0 msgr2=0x7f73540cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 -- 172.21.15.40:0/3829019292 shutdown_connections 2024-04-26T21:14:01.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.352+0000 7f73591d4640 1 -- 172.21.15.40:0/3829019292 wait complete. 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 Processor -- start 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 -- start start 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 0x7f735413d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73541e8d70 0x7f73541e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73540b4460 con 0x7f735413b190 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73540b42e0 con 0x7f7340004800 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f73591d4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7354142a30 con 0x7f73541e8d70 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f735259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f735259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41334/0 (socket says 172.21.15.40:41334) 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.354+0000 7f735259c640 1 -- 172.21.15.40:0/3769680232 learned_addr learned my addr 172.21.15.40:0/3769680232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 0x7f735413d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735359e640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73541e8d70 0x7f73541e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 -- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73541e8d70 msgr2=0x7f73541e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73541e8d70 0x7f73541e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 -- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 msgr2=0x7f735413d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 0x7f735413d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 -- 172.21.15.40:0/3769680232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735413c7b0 con 0x7f735413b190 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 0x7f735413d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f735259c640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c300 tx=0x7f733c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c037040 con 0x7f735413b190 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c041070 con 0x7f735413b190 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c005e20 con 0x7f735413b190 2024-04-26T21:14:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735413d9f0 con 0x7f735413b190 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.355+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735413bf10 con 0x7f735413b190 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.356+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735406a290 con 0x7f735413b190 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.357+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f733c0047e0 con 0x7f735413b190 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.357+0000 7f734b7fe640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 0x7f733406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.357+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f733c0ab4a0 con 0x7f735413b190 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.357+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 0x7f733406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.358+0000 7f7352d9d640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 0x7f733406ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f734c007440 tx=0x7f734c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.361+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f733c005040 con 0x7f735413b190 2024-04-26T21:14:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.551+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7354073e90 con 0x7f735413b190 2024-04-26T21:14:01.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.554+0000 7f734b7fe640 1 -- 172.21.15.40:0/3769680232 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f733c079b40 con 0x7f735413b190 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 msgr2=0x7f733406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 0x7f733406ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f734c007440 tx=0x7f734c00b040 comp rx=0 tx=0).stop 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 msgr2=0x7f735413b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c300 tx=0x7f733c001a40 comp rx=0 tx=0).stop 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 shutdown_connections 2024-04-26T21:14:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f733406c920 0x7f733406ede0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73541e8d70 0x7f73541e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735413b190 0x7f735413b610 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 --2- 172.21.15.40:0/3769680232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7340004800 0x7f735413d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 >> 172.21.15.40:0/3769680232 conn(0x7f73540cd8a0 msgr2=0x7f73540ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 shutdown_connections 2024-04-26T21:14:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.557+0000 7f73591d4640 1 -- 172.21.15.40:0/3769680232 wait complete. 2024-04-26T21:14:01.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-04-26T21:14:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.670+0000 7f300a0b8640 1 Processor -- start 2024-04-26T21:14:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.670+0000 7f300a0b8640 1 -- start start 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f300a0b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 0x7f30040ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f300a0b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 0x7f30040d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f300a0b8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30040cdd40 con 0x7f30040d1d90 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f300a0b8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f30040d2280 con 0x7f300415e7e0 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f300a0b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f30040cdfe0 con 0x7f3004161cb0 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f30037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 0x7f30040ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f30037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 0x7f30040ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41336/0 (socket says 172.21.15.40:41336) 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f30037fe640 1 -- 172.21.15.40:0/2338134800 learned_addr learned my addr 172.21.15.40:0/2338134800 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.671+0000 7f3002ffd640 1 --2- 172.21.15.40:0/2338134800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 0x7f30040d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f30027fc640 1 -- 172.21.15.40:0/2338134800 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740714122 0 0) 0x7f30040cdd40 con 0x7f30040d1d90 2024-04-26T21:14:01.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f30027fc640 1 -- 172.21.15.40:0/2338134800 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fec003520 con 0x7f30040d1d90 2024-04-26T21:14:01.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f3002ffd640 1 -- 172.21.15.40:0/2338134800 >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 msgr2=0x7f30040ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f3002ffd640 1 --2- 172.21.15.40:0/2338134800 >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 0x7f30040ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f3002ffd640 1 -- 172.21.15.40:0/2338134800 >> v1:172.21.15.184:6789/0 conn(0x7f30040d1d90 legacy=0x7f3004161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.672+0000 7f3002ffd640 1 -- 172.21.15.40:0/2338134800 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30040ce2c0 con 0x7f3004161cb0 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.673+0000 7f3002ffd640 1 --2- 172.21.15.40:0/2338134800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 0x7f30040d1850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80097a0 tx=0x7f2ff802e590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9262354091a5c7d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.673+0000 7f30027fc640 1 -- 172.21.15.40:0/2338134800 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff803a070 con 0x7f3004161cb0 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.673+0000 7f30027fc640 1 -- 172.21.15.40:0/2338134800 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8033710 con 0x7f3004161cb0 2024-04-26T21:14:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.673+0000 7f30027fc640 1 -- 172.21.15.40:0/2338134800 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8033ae0 con 0x7f3004161cb0 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 -- 172.21.15.40:0/2338134800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 msgr2=0x7f30040d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 --2- 172.21.15.40:0/2338134800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 0x7f30040d1850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80097a0 tx=0x7f2ff802e590 comp rx=0 tx=0).stop 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 -- 172.21.15.40:0/2338134800 shutdown_connections 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 --2- 172.21.15.40:0/2338134800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3004161cb0 0x7f30040d1850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 --2- 172.21.15.40:0/2338134800 >> v2:172.21.15.40:3300/0 conn(0x7f300415e7e0 0x7f30040ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 -- 172.21.15.40:0/2338134800 >> 172.21.15.40:0/2338134800 conn(0x7f300409ba40 msgr2=0x7f300409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 -- 172.21.15.40:0/2338134800 shutdown_connections 2024-04-26T21:14:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.674+0000 7f300a0b8640 1 -- 172.21.15.40:0/2338134800 wait complete. 2024-04-26T21:14:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.676+0000 7f300a0b8640 1 Processor -- start 2024-04-26T21:14:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.676+0000 7f300a0b8640 1 -- start start 2024-04-26T21:14:01.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30040d1d90 0x7f30040b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f300415e7e0 0x7f30041564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f30040cf2e0 con 0x7f3004161cb0 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30040ce880 con 0x7f300415e7e0 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f300a0b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3004161790 con 0x7f30040d1d90 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f3003fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f3003fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41342/0 (socket says 172.21.15.40:41342) 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.677+0000 7f3003fff640 1 -- 172.21.15.40:0/812197505 learned_addr learned my addr 172.21.15.40:0/812197505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f30037fe640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30040d1d90 0x7f30040b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3002ffd640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f300415e7e0 0x7f30041564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 -- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30040d1d90 msgr2=0x7f30040b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30040d1d90 0x7f30040b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 -- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f300415e7e0 msgr2=0x7f30041564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f300415e7e0 0x7f30041564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 -- 172.21.15.40:0/812197505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30041569f0 con 0x7f3004161cb0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f3003fff640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c710 tx=0x7f2ffc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003b70 con 0x7f3004161cb0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc003d10 con 0x7f3004161cb0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00aa50 con 0x7f3004161cb0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30040b08d0 con 0x7f3004161cb0 2024-04-26T21:14:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.678+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30040b0e90 con 0x7f3004161cb0 2024-04-26T21:14:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.679+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300406a290 con 0x7f3004161cb0 2024-04-26T21:14:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.680+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ffc00abf0 con 0x7f3004161cb0 2024-04-26T21:14:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.680+0000 7f2febfff640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 0x7f2ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.680+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f2ffc08b5c0 con 0x7f3004161cb0 2024-04-26T21:14:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.680+0000 7f30037fe640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 0x7f2ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.681+0000 7f30037fe640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 0x7f2ff006edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009bd0 tx=0x7f2ff4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.684+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ffc00ce20 con 0x7f3004161cb0 2024-04-26T21:14:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.872+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7f3004161570 con 0x7f3004161cb0 2024-04-26T21:14:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.875+0000 7f2febfff640 1 -- 172.21.15.40:0/812197505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v15) ==== 106+0+0 (secure 0 0 0) 0x7f2ffc0596a0 con 0x7f3004161cb0 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 msgr2=0x7f2ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 0x7f2ff006edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009bd0 tx=0x7f2ff4009240 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 msgr2=0x7f3004160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c710 tx=0x7f2ffc00cbd0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 shutdown_connections 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ff006c8f0 0x7f2ff006edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004161cb0 0x7f3004160240 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f300415e7e0 0x7f30041564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 --2- 172.21.15.40:0/812197505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30040d1d90 0x7f30040b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 >> 172.21.15.40:0/812197505 conn(0x7f300409ba40 msgr2=0x7f3004156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 shutdown_connections 2024-04-26T21:14:01.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.877+0000 7f300a0b8640 1 -- 172.21.15.40:0/812197505 wait complete. 2024-04-26T21:14:01.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-04-26T21:14:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.989+0000 7f7dad12c640 1 Processor -- start 2024-04-26T21:14:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.989+0000 7f7dad12c640 1 -- start start 2024-04-26T21:14:01.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7dad12c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7da815f370 0x7f7da80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7dad12c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da80d03a0 con 0x7f7da80d3ee0 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7dad12c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da80d0680 con 0x7f7da815f750 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7dad12c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7da80d0960 con 0x7f7da815f370 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7da759e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7da80d3ee0 0x7f7da80b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32988/0 (socket says 172.21.15.40:32988) 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7da759e640 1 -- 172.21.15.40:0/2168936525 learned_addr learned my addr 172.21.15.40:0/2168936525 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.990+0000 7f7da6d9d640 1 --2- 172.21.15.40:0/2168936525 >> v2:172.21.15.184:3300/0 conn(0x7f7da815f370 0x7f7da80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295594851 0 0) 0x7f7da80d03a0 con 0x7f7da80d3ee0 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d94003520 con 0x7f7da80d3ee0 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534318893 0 0) 0x7f7da80d0680 con 0x7f7da815f750 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da80d03a0 con 0x7f7da815f750 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da6d9d640 1 -- 172.21.15.40:0/2168936525 >> v1:172.21.15.184:6789/0 conn(0x7f7da815f750 legacy=0x7f7da80d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da6d9d640 1 -- 172.21.15.40:0/2168936525 >> v1:172.21.15.40:6789/0 conn(0x7f7da80d3ee0 legacy=0x7f7da80b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.991+0000 7f7da6d9d640 1 -- 172.21.15.40:0/2168936525 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da80d0c40 con 0x7f7da815f370 2024-04-26T21:14:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7da6d9d640 1 --2- 172.21.15.40:0/2168936525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80cfd70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009d90 tx=0x7f7d9802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996c2bcb33b0343e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007740 con 0x7f7da815f370 2024-04-26T21:14:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d980078e0 con 0x7f7da815f370 2024-04-26T21:14:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7da5d9b640 1 -- 172.21.15.40:0/2168936525 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007c90 con 0x7f7da815f370 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 -- 172.21.15.40:0/2168936525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 msgr2=0x7f7da80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 --2- 172.21.15.40:0/2168936525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80cfd70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009d90 tx=0x7f7d9802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 -- 172.21.15.40:0/2168936525 shutdown_connections 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 --2- 172.21.15.40:0/2168936525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80cfd70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 -- 172.21.15.40:0/2168936525 >> 172.21.15.40:0/2168936525 conn(0x7f7da808ec10 msgr2=0x7f7da809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 -- 172.21.15.40:0/2168936525 shutdown_connections 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.992+0000 7f7dad12c640 1 -- 172.21.15.40:0/2168936525 wait complete. 2024-04-26T21:14:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 Processor -- start 2024-04-26T21:14:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 -- start start 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7da80d3ee0 0x7f7da80b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7da80b4950 con 0x7f7da815f750 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7da80b47d0 con 0x7f7da815f370 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.994+0000 7f7dad12c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7da80b4ad0 con 0x7f7da80d3ee0 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41346/0 (socket says 172.21.15.40:41346) 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da759e640 1 -- 172.21.15.40:0/461121075 learned_addr learned my addr 172.21.15.40:0/461121075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da6d9d640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7da80d3ee0 0x7f7da80b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 -- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7da80d3ee0 msgr2=0x7f7da80b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7da80d3ee0 0x7f7da80b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 -- 172.21.15.40:0/461121075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 msgr2=0x7f7da80c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 -- 172.21.15.40:0/461121075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da80ca8f0 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da759e640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7da659c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80b8760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c3c0 tx=0x7f7d9000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90017070 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d900052f0 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da80cabd0 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90003830 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.995+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da80cb190 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.996+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da806a290 con 0x7f7da815f370 2024-04-26T21:14:01.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.997+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d9001a070 con 0x7f7da815f370 2024-04-26T21:14:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.997+0000 7f7d9f7fe640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 0x7f7d8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.997+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f7d9008ab60 con 0x7f7da815f370 2024-04-26T21:14:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.997+0000 7f7da6d9d640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 0x7f7d8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:01.998+0000 7f7da6d9d640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 0x7f7d8806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d98034000 tx=0x7f7d9802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:02.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.002+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d90058c50 con 0x7f7da815f370 2024-04-26T21:14:02.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.259+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f7da81e80c0 con 0x7f7da815f370 2024-04-26T21:14:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.837+0000 7f7d9f7fe640 1 -- 172.21.15.40:0/461121075 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v119) ==== 165+0+0 (secure 0 0 0) 0x7f7d9005cf50 con 0x7f7da815f370 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 msgr2=0x7f7d8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 0x7f7d8806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d98034000 tx=0x7f7d9802f000 comp rx=0 tx=0).stop 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 msgr2=0x7f7da80b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80b8760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c3c0 tx=0x7f7d9000c880 comp rx=0 tx=0).stop 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 shutdown_connections 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7d8806c8f0 0x7f7d8806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7da815f750 0x7f7da80c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7da815f370 0x7f7da80b8760 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 --2- 172.21.15.40:0/461121075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7da80d3ee0 0x7f7da80b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 >> 172.21.15.40:0/461121075 conn(0x7f7da808ec10 msgr2=0x7f7da80d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 shutdown_connections 2024-04-26T21:14:02.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.840+0000 7f7dad12c640 1 -- 172.21.15.40:0/461121075 wait complete. 2024-04-26T21:14:02.854 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2024-04-26T21:14:02.854 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2024-04-26T21:14:02.855 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:14:02.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.961+0000 7fc4bebec640 1 Processor -- start 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.961+0000 7fc4bebec640 1 -- start start 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bebec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b815f370 0x7fc4b80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bebec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f750 0x7fc4b80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bebec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b80d03a0 con 0x7fc4b80d37d0 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bebec640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80b4d10 con 0x7fc4b815f370 2024-04-26T21:14:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bebec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80d0680 con 0x7fc4b815f750 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b815f370 0x7fc4b80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4b815f370 0x7fc4b80cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41360/0 (socket says 172.21.15.40:41360) 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 -- 172.21.15.40:0/3196645965 learned_addr learned my addr 172.21.15.40:0/3196645965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4affff640 1 --2- 172.21.15.40:0/3196645965 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f750 0x7fc4b80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 -- 172.21.15.40:0/3196645965 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f750 msgr2=0x7fc4b80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 --2- 172.21.15.40:0/3196645965 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f750 0x7fc4b80b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 -- 172.21.15.40:0/3196645965 >> v1:172.21.15.184:6789/0 conn(0x7fc4b80d37d0 legacy=0x7fc4b80d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.962+0000 7fc4bc988640 1 -- 172.21.15.40:0/3196645965 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b80d0960 con 0x7fc4b815f370 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.963+0000 7fc4bc988640 1 --2- 172.21.15.40:0/3196645965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b80cfd70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80d02b0 tx=0x7fc4b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556b0b64f7c75869 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.963+0000 7fc4af7fe640 1 -- 172.21.15.40:0/3196645965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4037070 con 0x7fc4b815f370 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.963+0000 7fc4af7fe640 1 -- 172.21.15.40:0/3196645965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b4032440 con 0x7fc4b815f370 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.963+0000 7fc4af7fe640 1 -- 172.21.15.40:0/3196645965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4032810 con 0x7fc4b815f370 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 -- 172.21.15.40:0/3196645965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 msgr2=0x7fc4b80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 --2- 172.21.15.40:0/3196645965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b80cfd70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80d02b0 tx=0x7fc4b402a270 comp rx=0 tx=0).stop 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 -- 172.21.15.40:0/3196645965 shutdown_connections 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 --2- 172.21.15.40:0/3196645965 >> v2:172.21.15.184:3300/0 conn(0x7fc4b815f750 0x7fc4b80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 --2- 172.21.15.40:0/3196645965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b80cfd70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 -- 172.21.15.40:0/3196645965 >> 172.21.15.40:0/3196645965 conn(0x7fc4b808ec10 msgr2=0x7fc4b809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 -- 172.21.15.40:0/3196645965 shutdown_connections 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.964+0000 7fc4bebec640 1 -- 172.21.15.40:0/3196645965 wait complete. 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.965+0000 7fc4bebec640 1 Processor -- start 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.965+0000 7fc4bebec640 1 -- start start 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d37d0 0x7fc4b81f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f750 0x7fc4b81f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80b61d0 con 0x7fc4b815f370 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80d0f20 con 0x7fc4b815f750 2024-04-26T21:14:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bebec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80b6350 con 0x7fc4b80d37d0 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41368/0 (socket says 172.21.15.40:41368) 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 -- 172.21.15.40:0/2229175342 learned_addr learned my addr 172.21.15.40:0/2229175342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bd189640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f750 0x7fc4b81f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4bc988640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d37d0 0x7fc4b81f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 -- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d37d0 msgr2=0x7fc4b81f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d37d0 0x7fc4b81f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 -- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f750 msgr2=0x7fc4b81f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f750 0x7fc4b81f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.966+0000 7fc4affff640 1 -- 172.21.15.40:0/2229175342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b814b9b0 con 0x7fc4b815f370 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4affff640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800d800 tx=0x7fc4a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800b6c0 con 0x7fc4b815f370 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a8019070 con 0x7fc4b815f370 2024-04-26T21:14:02.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b80cdc80 con 0x7fc4b815f370 2024-04-26T21:14:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800ba80 con 0x7fc4b815f370 2024-04-26T21:14:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.967+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b80ce240 con 0x7fc4b815f370 2024-04-26T21:14:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.968+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b806a290 con 0x7fc4b815f370 2024-04-26T21:14:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.968+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4a800bc20 con 0x7fc4b815f370 2024-04-26T21:14:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.969+0000 7fc4ad7fa640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 0x7fc4a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:02.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.969+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fc4a808bb10 con 0x7fc4b815f370 2024-04-26T21:14:02.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.969+0000 7fc4bc988640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 0x7fc4a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:02.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.969+0000 7fc4bc988640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 0x7fc4a406edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034000 tx=0x7fc4b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:02.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:02.973+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4a8059c80 con 0x7fc4b815f370 2024-04-26T21:14:03.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.162+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4b806d8d0 con 0x7fc4b815f370 2024-04-26T21:14:03.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.162+0000 7fc4ad7fa640 1 -- 172.21.15.40:0/2229175342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13020 (secure 0 0 0) 0x7fc4a805df80 con 0x7fc4b815f370 2024-04-26T21:14:03.163 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:00.823643+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/1813208926":"2024-04-27T21:13:57.815384+0000","172.21.15.40:6832/1813208926":"2024-04-27T21:13:57.815384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 msgr2=0x7fc4a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 0x7fc4a406edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034000 tx=0x7fc4b402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 msgr2=0x7fc4b814b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800d800 tx=0x7fc4a800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 shutdown_connections 2024-04-26T21:14:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4a406c8f0 0x7fc4a406edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b815f750 0x7fc4b81f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4b815f370 0x7fc4b814b240 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 --2- 172.21.15.40:0/2229175342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b80d37d0 0x7fc4b81f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 >> 172.21.15.40:0/2229175342 conn(0x7fc4b808ec10 msgr2=0x7fc4b806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 shutdown_connections 2024-04-26T21:14:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.165+0000 7fc4bebec640 1 -- 172.21.15.40:0/2229175342 wait complete. 2024-04-26T21:14:03.177 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:03.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.279+0000 7f97a6389640 1 Processor -- start 2024-04-26T21:14:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 -- start start 2024-04-26T21:14:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 0x7f97a00d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a00d58f0 con 0x7f97a0163aa0 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f97a015fa60 con 0x7f97a015f680 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.280+0000 7f97a6389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97a00d3f50 con 0x7f97a00d4f20 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f979ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 0x7f97a00d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46674/0 (socket says 172.21.15.40:46674) 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f979ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 0x7f97a00d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:39626/0 (socket says 172.21.15.40:39626) 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 -- 172.21.15.40:0/3366889125 learned_addr learned my addr 172.21.15.40:0/3366889125 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f979f7fe640 1 --1- 172.21.15.40:0/3366889125 >> v1:172.21.15.184:6789/0 conn(0x7f97a0163aa0 0x7f97a00b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48512/0 (socket says 172.21.15.40:48512) 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 -- 172.21.15.40:0/3366889125 >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 msgr2=0x7f97a00d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 --2- 172.21.15.40:0/3366889125 >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 0x7f97a00d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 -- 172.21.15.40:0/3366889125 >> v1:172.21.15.184:6789/0 conn(0x7f97a0163aa0 legacy=0x7f97a00b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.281+0000 7f97a4926640 1 -- 172.21.15.40:0/3366889125 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a00b5090 con 0x7f97a00d4f20 2024-04-26T21:14:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a4926640 1 --2- 172.21.15.40:0/3366889125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9798009e50 tx=0x7f979802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56860d759be12a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f979effd640 1 -- 172.21.15.40:0/3366889125 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798036070 con 0x7f97a00d4f20 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f979effd640 1 -- 172.21.15.40:0/3366889125 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798031600 con 0x7f97a00d4f20 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f979effd640 1 -- 172.21.15.40:0/3366889125 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980319d0 con 0x7f97a00d4f20 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 -- 172.21.15.40:0/3366889125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 msgr2=0x7f97a00d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 --2- 172.21.15.40:0/3366889125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9798009e50 tx=0x7f979802a710 comp rx=0 tx=0).stop 2024-04-26T21:14:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 -- 172.21.15.40:0/3366889125 shutdown_connections 2024-04-26T21:14:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 --2- 172.21.15.40:0/3366889125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00d53b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 --2- 172.21.15.40:0/3366889125 >> v2:172.21.15.184:3300/0 conn(0x7f97a015f680 0x7f97a00d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 -- 172.21.15.40:0/3366889125 >> 172.21.15.40:0/3366889125 conn(0x7f97a008eca0 msgr2=0x7f97a009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 -- 172.21.15.40:0/3366889125 shutdown_connections 2024-04-26T21:14:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.282+0000 7f97a6389640 1 -- 172.21.15.40:0/3366889125 wait complete. 2024-04-26T21:14:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 Processor -- start 2024-04-26T21:14:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 -- start start 2024-04-26T21:14:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97a015f680 0x7f97a014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0163aa0 0x7f97a015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0163f30 con 0x7f97a0163aa0 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97a00b6280 con 0x7f97a015f680 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.284+0000 7f97a6389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97a00cfd80 con 0x7f97a00d4f20 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f97a4926640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0163aa0 0x7f97a015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97a015f680 0x7f97a014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:46680/0 (socket says 172.21.15.40:46680) 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 -- 172.21.15.40:0/3916500228 learned_addr learned my addr 172.21.15.40:0/3916500228 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 -- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97a015f680 msgr2=0x7f97a014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97a015f680 0x7f97a014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 -- 172.21.15.40:0/3916500228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0163aa0 msgr2=0x7f97a015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0163aa0 0x7f97a015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 -- 172.21.15.40:0/3916500228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a014f4e0 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979ffff640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9b0 tx=0x7f979000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790017070 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979000c9a0 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000cd90 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a00c8da0 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.285+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a0160bd0 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.286+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a006a290 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.287+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f979001a070 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.287+0000 7f979cff9640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 0x7f978c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.288+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f979008bc00 con 0x7f97a00d4f20 2024-04-26T21:14:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.288+0000 7f979f7fe640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 0x7f978c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.288+0000 7f979f7fe640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 0x7f978c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97a015c6b0 tx=0x7f9794007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.292+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9790059cc0 con 0x7f97a00d4f20 2024-04-26T21:14:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.479+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97a00b4cd0 con 0x7f97a00d4f20 2024-04-26T21:14:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.479+0000 7f979cff9640 1 -- 172.21.15.40:0/3916500228 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f979005dfc0 con 0x7f97a00d4f20 2024-04-26T21:14:03.480 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 msgr2=0x7f978c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 0x7f978c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97a015c6b0 tx=0x7f9794007e80 comp rx=0 tx=0).stop 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 msgr2=0x7f97a00ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9b0 tx=0x7f979000ee70 comp rx=0 tx=0).stop 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 shutdown_connections 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f978c06c8f0 0x7f978c06edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0163aa0 0x7f97a015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97a015f680 0x7f97a014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 --2- 172.21.15.40:0/3916500228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97a00d4f20 0x7f97a00ced30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 >> 172.21.15.40:0/3916500228 conn(0x7f97a008eca0 msgr2=0x7f97a00d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:03.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 shutdown_connections 2024-04-26T21:14:03.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.482+0000 7f97a6389640 1 -- 172.21.15.40:0/3916500228 wait complete. 2024-04-26T21:14:03.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-04-26T21:14:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 Processor -- start 2024-04-26T21:14:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 -- start start 2024-04-26T21:14:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde5815f720 0x7fde580b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde580d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde580b2510 con 0x7fde5815f340 2024-04-26T21:14:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fde580b27f0 con 0x7fde5815f720 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.594+0000 7fde5d093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde580b2ad0 con 0x7fde580d5610 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde5815f720 0x7fde580b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde5815f720 0x7fde580b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41398/0 (socket says 172.21.15.40:41398) 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 -- 172.21.15.40:0/407481600 learned_addr learned my addr 172.21.15.40:0/407481600 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde5759e640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde580d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 -- 172.21.15.40:0/407481600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 msgr2=0x7fde580d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde580d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 -- 172.21.15.40:0/407481600 >> v1:172.21.15.184:6789/0 conn(0x7fde5815f340 legacy=0x7fde580d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.595+0000 7fde4ffff640 1 -- 172.21.15.40:0/407481600 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde580b2db0 con 0x7fde5815f720 2024-04-26T21:14:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde4ffff640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f720 0x7fde580b1e80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde40009870 tx=0x7fde4002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c9681c127ec311 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5659c640 1 -- 172.21.15.40:0/407481600 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40037070 con 0x7fde5815f720 2024-04-26T21:14:03.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5659c640 1 -- 172.21.15.40:0/407481600 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde40032440 con 0x7fde5815f720 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5659c640 1 -- 172.21.15.40:0/407481600 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40032810 con 0x7fde5815f720 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 -- 172.21.15.40:0/407481600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f720 msgr2=0x7fde580b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f720 0x7fde580b1e80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde40009870 tx=0x7fde4002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 -- 172.21.15.40:0/407481600 shutdown_connections 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde580d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 --2- 172.21.15.40:0/407481600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f720 0x7fde580b1e80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 -- 172.21.15.40:0/407481600 >> 172.21.15.40:0/407481600 conn(0x7fde5808ec70 msgr2=0x7fde5809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 -- 172.21.15.40:0/407481600 shutdown_connections 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.596+0000 7fde5d093640 1 -- 172.21.15.40:0/407481600 wait complete. 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 Processor -- start 2024-04-26T21:14:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 -- start start 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde58151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde5815f720 0x7fde58152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fde580b3940 con 0x7fde5815f340 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fde580b37c0 con 0x7fde5815f720 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.598+0000 7fde5d093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fde580b3ac0 con 0x7fde580d5610 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41406/0 (socket says 172.21.15.40:41406) 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 -- 172.21.15.40:0/2459592495 learned_addr learned my addr 172.21.15.40:0/2459592495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde5759e640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde5815f720 0x7fde58152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde56d9d640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde58151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 -- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 msgr2=0x7fde58151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde58151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 -- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde5815f720 msgr2=0x7fde58152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde5815f720 0x7fde58152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 -- 172.21.15.40:0/2459592495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde58152080 con 0x7fde5815f340 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4ffff640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40037040 con 0x7fde5815f340 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde40042070 con 0x7fde5815f340 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40030880 con 0x7fde5815f340 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde580cdb90 con 0x7fde5815f340 2024-04-26T21:14:03.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.599+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde580ce150 con 0x7fde5815f340 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.600+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5806a290 con 0x7fde5815f340 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.601+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde4002ad10 con 0x7fde5815f340 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.601+0000 7fde4f7fe640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 0x7fde3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.601+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fde400abdc0 con 0x7fde5815f340 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.601+0000 7fde56d9d640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 0x7fde3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.602+0000 7fde56d9d640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 0x7fde3806ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde480015a0 tx=0x7fde48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:03.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.605+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde4007a4f0 con 0x7fde5815f340 2024-04-26T21:14:03.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:03.862+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7fde580d3020 con 0x7fde5815f340 2024-04-26T21:14:04.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.842+0000 7fde4f7fe640 1 -- 172.21.15.40:0/2459592495 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v121) ==== 165+0+0 (secure 0 0 0) 0x7fde40079df0 con 0x7fde5815f340 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 msgr2=0x7fde3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 0x7fde3806ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde480015a0 tx=0x7fde48009040 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 msgr2=0x7fde580cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde400046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 shutdown_connections 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fde3806c920 0x7fde3806ede0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde5815f720 0x7fde58152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815f340 0x7fde580cd1b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 --2- 172.21.15.40:0/2459592495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde580d5610 0x7fde58151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 >> 172.21.15.40:0/2459592495 conn(0x7fde5808ec70 msgr2=0x7fde580cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 shutdown_connections 2024-04-26T21:14:04.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.845+0000 7fde5d093640 1 -- 172.21.15.40:0/2459592495 wait complete. 2024-04-26T21:14:04.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:14:04.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 Processor -- start 2024-04-26T21:14:04.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 -- start start 2024-04-26T21:14:04.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f864815f680 0x7f86480d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f86480d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86480d58f0 con 0x7f8648163aa0 2024-04-26T21:14:04.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f864815fa60 con 0x7f864815f680 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.970+0000 7f864e7b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86480d3f50 con 0x7f86480d4f20 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f864815f680 0x7f86480d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f864815f680 0x7f86480d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41412/0 (socket says 172.21.15.40:41412) 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 -- 172.21.15.40:0/62540673 learned_addr learned my addr 172.21.15.40:0/62540673 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f864cd54640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f86480d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 -- 172.21.15.40:0/62540673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 msgr2=0x7f86480d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f86480d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 -- 172.21.15.40:0/62540673 >> v1:172.21.15.184:6789/0 conn(0x7f8648163aa0 legacy=0x7f86480b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:04.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.971+0000 7f8647fff640 1 -- 172.21.15.40:0/62540673 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86480b5090 con 0x7f864815f680 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f8647fff640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f864815f680 0x7f86480d3820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8638009870 tx=0x7f863802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd4642c64ab1cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f8646ffd640 1 -- 172.21.15.40:0/62540673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638037070 con 0x7f864815f680 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f8646ffd640 1 -- 172.21.15.40:0/62540673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8638032440 con 0x7f864815f680 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f8646ffd640 1 -- 172.21.15.40:0/62540673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638032810 con 0x7f864815f680 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 -- 172.21.15.40:0/62540673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f864815f680 msgr2=0x7f86480d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f864815f680 0x7f86480d3820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8638009870 tx=0x7f863802a210 comp rx=0 tx=0).stop 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 -- 172.21.15.40:0/62540673 shutdown_connections 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f86480d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 --2- 172.21.15.40:0/62540673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f864815f680 0x7f86480d3820 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 -- 172.21.15.40:0/62540673 >> 172.21.15.40:0/62540673 conn(0x7f864808eca0 msgr2=0x7f864809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 -- 172.21.15.40:0/62540673 shutdown_connections 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.972+0000 7f864e7b7640 1 -- 172.21.15.40:0/62540673 wait complete. 2024-04-26T21:14:04.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 Processor -- start 2024-04-26T21:14:04.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 -- start start 2024-04-26T21:14:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f864814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 0x7f86481604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8648163f30 con 0x7f8648163aa0 2024-04-26T21:14:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86480b6280 con 0x7f864815f680 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.974+0000 7f864e7b7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86480cfd80 con 0x7f86480d4f20 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41418/0 (socket says 172.21.15.40:41418) 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 -- 172.21.15.40:0/1276534304 learned_addr learned my addr 172.21.15.40:0/1276534304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f8647fff640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f864814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f86477fe640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 0x7f86481604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 -- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 msgr2=0x7f864814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f864814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 -- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 msgr2=0x7f86481604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 0x7f86481604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 -- 172.21.15.40:0/1276534304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8648162360 con 0x7f8648163aa0 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f86477fe640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 0x7f86481604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864cd54640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8640007e40 tx=0x7f864000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640014620 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640019070 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8648162640 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640014a20 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.975+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86480b2f50 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.976+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864806a290 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.977+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8640014bc0 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.977+0000 7f8644ff9640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 0x7f863406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.977+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f864008c070 con 0x7f8648163aa0 2024-04-26T21:14:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.978+0000 7f8647fff640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 0x7f863406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:04.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.978+0000 7f8647fff640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 0x7f863406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f863802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:04.981+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f864005a7a0 con 0x7f8648163aa0 2024-04-26T21:14:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.167+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f86480b4cd0 con 0x7f8648163aa0 2024-04-26T21:14:05.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.172+0000 7f8644ff9640 1 -- 172.21.15.40:0/1276534304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f864005a0a0 con 0x7f8648163aa0 2024-04-26T21:14:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.174+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 msgr2=0x7f863406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.174+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 0x7f863406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f863802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.174+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 msgr2=0x7f86480b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.174+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8640007e40 tx=0x7f864000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 shutdown_connections 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f863406c8f0 0x7f863406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8648163aa0 0x7f86480b3750 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864815f680 0x7f86481604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 --2- 172.21.15.40:0/1276534304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86480d4f20 0x7f864814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 >> 172.21.15.40:0/1276534304 conn(0x7f864808eca0 msgr2=0x7f86480d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 shutdown_connections 2024-04-26T21:14:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.175+0000 7f864e7b7640 1 -- 172.21.15.40:0/1276534304 wait complete. 2024-04-26T21:14:05.187 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:14:05.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:05.226 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:05.274 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:14:05.274 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:05.274 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:05.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:05.289 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:05.289 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:05.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:05.306 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:05.307 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:05.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:05.345 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:05.346 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:05.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:05.363 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:14:05.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:05.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.484+0000 7fafc774e640 1 Processor -- start 2024-04-26T21:14:05.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.484+0000 7fafc774e640 1 -- start start 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc774e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafc00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc774e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc00d05b0 con 0x7fafc0140350 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc774e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc00d0890 con 0x7fafc013ff70 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc774e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafc006e450 con 0x7fafc0146790 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc4ce9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fafc0140350 0x7fafc00cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32996/0 (socket says 172.21.15.40:32996) 2024-04-26T21:14:05.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc4ce9640 1 -- 172.21.15.40:0/2364757355 learned_addr learned my addr 172.21.15.40:0/2364757355 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafc5ceb640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafc00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509082027 0 0) 0x7fafc00d05b0 con 0x7fafc0140350 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafac003520 con 0x7fafc0140350 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3129874831 0 0) 0x7fafac003520 con 0x7fafc0140350 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 msgr2=0x7fafc00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafc00cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 >> v1:172.21.15.184:6789/0 conn(0x7fafc013ff70 legacy=0x7fafc0146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc006e730 con 0x7fafc0140350 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fafb0001670 con 0x7fafc0140350 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 >> v1:172.21.15.40:6789/0 conn(0x7fafc0140350 legacy=0x7fafc00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafac0039e0 0x7fafac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafc006e450 con 0x7fafac004d40 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00baf0 con 0x7fafac0039e0 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.485+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafac00bcf0 con 0x7fafc0146790 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc54ea640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafac0039e0 0x7fafac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc4ce9640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 -- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 msgr2=0x7fafac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0146790 0x7fafac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 -- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafac0039e0 msgr2=0x7fafac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafac0039e0 0x7fafac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.486+0000 7fafc5ceb640 1 -- 172.21.15.40:0/2364757355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac003520 con 0x7fafac004d40 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc5ceb640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafac0051c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fafc00cf230 tx=0x7fafac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12f5edc553ef3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8037070 con 0x7fafac004d40 2024-04-26T21:14:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb802e050 con 0x7fafac004d40 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafb7fff640 1 -- 172.21.15.40:0/2364757355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8009c10 con 0x7fafac004d40 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 -- 172.21.15.40:0/2364757355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 msgr2=0x7fafac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 --2- 172.21.15.40:0/2364757355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafac0051c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fafc00cf230 tx=0x7fafac005830 comp rx=0 tx=0).stop 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc5ceb640 1 -- 172.21.15.40:0/2364757355 reap_dead start 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 -- 172.21.15.40:0/2364757355 shutdown_connections 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 -- 172.21.15.40:0/2364757355 >> 172.21.15.40:0/2364757355 conn(0x7fafc00d0d30 msgr2=0x7fafc00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 -- 172.21.15.40:0/2364757355 shutdown_connections 2024-04-26T21:14:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.487+0000 7fafc774e640 1 -- 172.21.15.40:0/2364757355 wait complete. 2024-04-26T21:14:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.488+0000 7fafc774e640 1 Processor -- start 2024-04-26T21:14:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 -- start start 2024-04-26T21:14:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc0142fa0 0x7fafc00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0072d00 0x7fafc00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d00a0 con 0x7fafac004d40 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00cff20 con 0x7fafc0142fa0 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc774e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fafc00d0220 con 0x7fafc0072d00 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc54ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc54ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41442/0 (socket says 172.21.15.40:41442) 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.489+0000 7fafc54ea640 1 -- 172.21.15.40:0/3921715368 learned_addr learned my addr 172.21.15.40:0/3921715368 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 -- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0072d00 msgr2=0x7fafc00731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0072d00 0x7fafc00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 -- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc0142fa0 msgr2=0x7fafc00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc0142fa0 0x7fafc00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 -- 172.21.15.40:0/3921715368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc0143430 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc54ea640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00f850 tx=0x7fafbc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00fe40 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc018070 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc0143710 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00d6b0 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.490+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc0143cd0 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.491+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc006a290 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.491+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafbc00d850 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.492+0000 7fafb5ffb640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 0x7fafa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.492+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fafbc08b730 con 0x7fafac004d40 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.492+0000 7fafc4ce9640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 0x7fafa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.492+0000 7fafc4ce9640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 0x7fafa406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000f60 tx=0x7fafb0000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.496+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafbc059e00 con 0x7fafac004d40 2024-04-26T21:14:05.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.719+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc00b45e0 con 0x7fafac004d40 2024-04-26T21:14:05.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.719+0000 7fafb5ffb640 1 -- 172.21.15.40:0/3921715368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3874 (secure 0 0 0) 0x7fafbc059700 con 0x7fafac004d40 2024-04-26T21:14:05.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:05.721 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3711110670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3711110670},{"type":"v1","addr":"172.21.15.184:6836","nonce":3711110670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":4691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3928825009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3928825009},{"type":"v1","addr":"172.21.15.40:6835","nonce":3928825009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1289329083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1289329083},{"type":"v1","addr":"172.21.15.184:6837","nonce":1289329083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2437940921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2437940921},{"type":"v1","addr":"172.21.15.40:6834","nonce":2437940921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-04-26T21:14:05.721 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-04-26T21:14:05.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 msgr2=0x7fafa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 0x7fafa406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000f60 tx=0x7fafb0000ed0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 msgr2=0x7fafc013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00f850 tx=0x7fafbc00fd10 comp rx=0 tx=0).stop 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 shutdown_connections 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fafa406c8f0 0x7fafa406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fafc0072d00 0x7fafc00731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc0142fa0 0x7fafc00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 --2- 172.21.15.40:0/3921715368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafac004d40 0x7fafc013b630 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 >> 172.21.15.40:0/3921715368 conn(0x7fafc00d0d30 msgr2=0x7fafc0160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 shutdown_connections 2024-04-26T21:14:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.721+0000 7fafc774e640 1 -- 172.21.15.40:0/3921715368 wait complete. 2024-04-26T21:14:05.733 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:14:05.733 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:14:05.734 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:14:05.734 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:05.734 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:14:05.734 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:14:05.734 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:14:05.734 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:05.734 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:14:05.735 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:14:05.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:14:05.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:05.735 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:14:05.735 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:14:05.735 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:14:05.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:05.743 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:14:05.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:14:05.744 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:14:05.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:14:05.747 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:14:05.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:14:05.748 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:14:05.749 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:14:05.786 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:14:05.786 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:05.787 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:14:05.787 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:05.790 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:14:05.790 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:05.791 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:14:05.791 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:05.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:14:05.862 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:14:05.863 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:14:05.875 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:14:05.875 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:14:05.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.897+0000 7f3f5646d640 1 Processor -- start 2024-04-26T21:14:05.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.897+0000 7f3f5646d640 1 -- start start 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f5646d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f500d3ee0 0x7f3f500cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f5646d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f5015f750 0x7f3f500b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f5646d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f500d03a0 con 0x7f3f5015f370 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f5646d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f500b4d10 con 0x7f3f500d3ee0 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f5646d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500d0680 con 0x7f3f5015f750 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f500d3ee0 0x7f3f500cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f500d3ee0 0x7f3f500cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41482/0 (socket says 172.21.15.40:41482) 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/637768393 learned_addr learned my addr 172.21.15.40:0/637768393 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f54a0a640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f5015f750 0x7f3f500b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/637768393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f5015f750 msgr2=0x7f3f500b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f5015f750 0x7f3f500b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/637768393 >> v1:172.21.15.40:6789/0 conn(0x7f3f5015f370 legacy=0x7f3f500d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.898+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/637768393 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500d0960 con 0x7f3f500d3ee0 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.899+0000 7f3f4f7fe640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f500d3ee0 0x7f3f500cfd70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e553b1029dc92f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.899+0000 7f3f4effd640 1 -- 172.21.15.40:0/637768393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38037070 con 0x7f3f500d3ee0 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.899+0000 7f3f4effd640 1 -- 172.21.15.40:0/637768393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38032440 con 0x7f3f500d3ee0 2024-04-26T21:14:05.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.899+0000 7f3f4effd640 1 -- 172.21.15.40:0/637768393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38032810 con 0x7f3f500d3ee0 2024-04-26T21:14:05.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 -- 172.21.15.40:0/637768393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f500d3ee0 msgr2=0x7f3f500cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f500d3ee0 0x7f3f500cfd70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f3802a270 comp rx=0 tx=0).stop 2024-04-26T21:14:05.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 -- 172.21.15.40:0/637768393 shutdown_connections 2024-04-26T21:14:05.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f5015f750 0x7f3f500b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 --2- 172.21.15.40:0/637768393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f500d3ee0 0x7f3f500cfd70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 -- 172.21.15.40:0/637768393 >> 172.21.15.40:0/637768393 conn(0x7f3f5008ec10 msgr2=0x7f3f5009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:05.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 -- 172.21.15.40:0/637768393 shutdown_connections 2024-04-26T21:14:05.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.900+0000 7f3f5646d640 1 -- 172.21.15.40:0/637768393 wait complete. 2024-04-26T21:14:05.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.901+0000 7f3f5646d640 1 Processor -- start 2024-04-26T21:14:05.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.901+0000 7f3f5646d640 1 -- start start 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500d3ee0 0x7f3f501f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f5015f750 0x7f3f500b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500b51c0 con 0x7f3f5015f370 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500d0f20 con 0x7f3f5015f750 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f5646d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500b5340 con 0x7f3f500d3ee0 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41496/0 (socket says 172.21.15.40:41496) 2024-04-26T21:14:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/3269587574 learned_addr learned my addr 172.21.15.40:0/3269587574 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4ffff640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500d3ee0 0x7f3f501f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f54a0a640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f5015f750 0x7f3f500b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500d3ee0 msgr2=0x7f3f501f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500d3ee0 0x7f3f501f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f5015f750 msgr2=0x7f3f500b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f5015f750 0x7f3f500b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.902+0000 7f3f4f7fe640 1 -- 172.21.15.40:0/3269587574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f501f8900 con 0x7f3f5015f370 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f4f7fe640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3f38000c00 tx=0x7f3f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38041020 con 0x7f3f5015f370 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f380306c0 con 0x7f3f5015f370 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38037040 con 0x7f3f5015f370 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501f8be0 con 0x7f3f5015f370 2024-04-26T21:14:05.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.903+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501f91a0 con 0x7f3f5015f370 2024-04-26T21:14:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.904+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5006a290 con 0x7f3f5015f370 2024-04-26T21:14:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.904+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f38044020 con 0x7f3f5015f370 2024-04-26T21:14:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.904+0000 7f3f4cff9640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.905+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f3f380abf00 con 0x7f3f5015f370 2024-04-26T21:14:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.905+0000 7f3f4ffff640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:05.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.905+0000 7f3f4ffff640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 0x7f3f3006edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001780 tx=0x7f3f40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:05.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:05.909+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f3804b030 con 0x7f3f5015f370 2024-04-26T21:14:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.108+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3f500c9e40 con 0x7f3f5015f370 2024-04-26T21:14:06.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.868+0000 7f3f4cff9640 1 -- 172.21.15.40:0/3269587574 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7f3f38079fe0 con 0x7f3f5015f370 2024-04-26T21:14:06.869 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 msgr2=0x7f3f3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 0x7f3f3006edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001780 tx=0x7f3f40009040 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 msgr2=0x7f3f501f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3f38000c00 tx=0x7f3f380046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 shutdown_connections 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.871+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f5015f750 0x7f3f500b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f5015f370 0x7f3f501f22f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 --2- 172.21.15.40:0/3269587574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f500d3ee0 0x7f3f501f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 >> 172.21.15.40:0/3269587574 conn(0x7f3f5008ec10 msgr2=0x7f3f500d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 shutdown_connections 2024-04-26T21:14:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.872+0000 7f3f5646d640 1 -- 172.21.15.40:0/3269587574 wait complete. 2024-04-26T21:14:06.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:14:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.993+0000 7f681a67b640 1 Processor -- start 2024-04-26T21:14:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.993+0000 7f681a67b640 1 -- start start 2024-04-26T21:14:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f681a67b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68140b61a0 0x7f6814160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f681a67b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68140b3de0 0x7f68140720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f681a67b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68140b4270 con 0x7f6814145980 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f681a67b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f68140b5140 con 0x7f68140b61a0 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f681a67b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f68140b4530 con 0x7f68140b3de0 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68140b61a0 0x7f6814160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68140b61a0 0x7f6814160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41512/0 (socket says 172.21.15.40:41512) 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 -- 172.21.15.40:0/550289339 learned_addr learned my addr 172.21.15.40:0/550289339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f6818c18640 1 --2- 172.21.15.40:0/550289339 >> v2:172.21.15.184:3300/0 conn(0x7f68140b3de0 0x7f68140720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 -- 172.21.15.40:0/550289339 >> v2:172.21.15.184:3300/0 conn(0x7f68140b3de0 msgr2=0x7f68140720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 --2- 172.21.15.40:0/550289339 >> v2:172.21.15.184:3300/0 conn(0x7f68140b3de0 0x7f68140720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 -- 172.21.15.40:0/550289339 >> v1:172.21.15.40:6789/0 conn(0x7f6814145980 legacy=0x7f68140b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.994+0000 7f68137fe640 1 -- 172.21.15.40:0/550289339 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68140b4810 con 0x7f68140b61a0 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.995+0000 7f68137fe640 1 --2- 172.21.15.40:0/550289339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b61a0 0x7f6814160630 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f68140b52c0 tx=0x7f680402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebada5e7fc1dbfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.995+0000 7f6812ffd640 1 -- 172.21.15.40:0/550289339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804032790 con 0x7f68140b61a0 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.995+0000 7f6812ffd640 1 -- 172.21.15.40:0/550289339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6804038070 con 0x7f68140b61a0 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.995+0000 7f6812ffd640 1 -- 172.21.15.40:0/550289339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804032b50 con 0x7f68140b61a0 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 -- 172.21.15.40:0/550289339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b61a0 msgr2=0x7f6814160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 --2- 172.21.15.40:0/550289339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b61a0 0x7f6814160630 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f68140b52c0 tx=0x7f680402c890 comp rx=0 tx=0).stop 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 -- 172.21.15.40:0/550289339 shutdown_connections 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 --2- 172.21.15.40:0/550289339 >> v2:172.21.15.184:3300/0 conn(0x7f68140b3de0 0x7f68140720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 --2- 172.21.15.40:0/550289339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b61a0 0x7f6814160630 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 -- 172.21.15.40:0/550289339 >> 172.21.15.40:0/550289339 conn(0x7f68140d0d30 msgr2=0x7f68140d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 -- 172.21.15.40:0/550289339 shutdown_connections 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.996+0000 7f681a67b640 1 -- 172.21.15.40:0/550289339 wait complete. 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.997+0000 7f681a67b640 1 Processor -- start 2024-04-26T21:14:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.997+0000 7f681a67b640 1 -- start start 2024-04-26T21:14:07.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68140b61a0 0x7f681413cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6814145980 0x7f681413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6814072610 con 0x7f68140b3de0 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68140b4dd0 con 0x7f68140b61a0 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f681a67b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6814072790 con 0x7f6814145980 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41518/0 (socket says 172.21.15.40:41518) 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 -- 172.21.15.40:0/4260453626 learned_addr learned my addr 172.21.15.40:0/4260453626 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6818c18640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6814145980 0x7f681413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f68137fe640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68140b61a0 0x7f681413cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 -- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6814145980 msgr2=0x7f681413e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6814145980 0x7f681413e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 -- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68140b61a0 msgr2=0x7f681413cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68140b61a0 0x7f681413cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 -- 172.21.15.40:0/4260453626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681413ea40 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.998+0000 7f6813fff640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f680800c3c0 tx=0x7f680800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.999+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68080052d0 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.999+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6808005490 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.999+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681413ed20 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.999+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808007260 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:06.999+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681413f2e0 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.000+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681406a290 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.000+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6808007400 con 0x7f68140b3de0 2024-04-26T21:14:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.000+0000 7f6810ff9640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 0x7f680006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.001+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f680808acc0 con 0x7f68140b3de0 2024-04-26T21:14:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.001+0000 7f68137fe640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 0x7f680006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.001+0000 7f68137fe640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 0x7f680006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6804037000 tx=0x7f6804032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.005+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6808059440 con 0x7f68140b3de0 2024-04-26T21:14:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.268+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6814072390 con 0x7f68140b3de0 2024-04-26T21:14:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.269+0000 7f6810ff9640 1 -- 172.21.15.40:0/4260453626 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f6808058d40 con 0x7f68140b3de0 2024-04-26T21:14:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:14:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:14:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:14:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:14:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:14:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:14:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:14:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:14:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:14:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:14:07.286 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:14:07.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 msgr2=0x7f680006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 0x7f680006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6804037000 tx=0x7f6804032000 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 msgr2=0x7f68140b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f680800c3c0 tx=0x7f680800c880 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 shutdown_connections 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f680006c8f0 0x7f680006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6814145980 0x7f681413e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68140b61a0 0x7f681413cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 --2- 172.21.15.40:0/4260453626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68140b3de0 0x7f68140b00e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 >> 172.21.15.40:0/4260453626 conn(0x7f68140d0d30 msgr2=0x7f681415d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 shutdown_connections 2024-04-26T21:14:07.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.271+0000 7f681a67b640 1 -- 172.21.15.40:0/4260453626 wait complete. 2024-04-26T21:14:07.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:07.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.388+0000 7f609ff14640 1 Processor -- start 2024-04-26T21:14:07.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.388+0000 7f609ff14640 1 -- start start 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609ff14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f609815f340 0x7f60980b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609ff14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f609815f720 0x7f60980d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609ff14640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60980b2510 con 0x7f60980d4f00 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609ff14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f60980b27f0 con 0x7f609815f340 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609ff14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60980b2ad0 con 0x7f609815f720 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609dcb0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f609815f340 0x7f60980b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609dcb0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f609815f340 0x7f60980b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41524/0 (socket says 172.21.15.40:41524) 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609dcb0640 1 -- 172.21.15.40:0/3740246066 learned_addr learned my addr 172.21.15.40:0/3740246066 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.389+0000 7f609d4af640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f609815f720 0x7f60980d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.390+0000 7f609dcb0640 1 -- 172.21.15.40:0/3740246066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f609815f720 msgr2=0x7f60980d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.390+0000 7f609dcb0640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f609815f720 0x7f60980d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.390+0000 7f609dcb0640 1 -- 172.21.15.40:0/3740246066 >> v1:172.21.15.40:6789/0 conn(0x7f60980d4f00 legacy=0x7f60980d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.390+0000 7f609dcb0640 1 -- 172.21.15.40:0/3740246066 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60980b2db0 con 0x7f609815f340 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.390+0000 7f609dcb0640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f340 0x7f60980b1e80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6090009870 tx=0x7f609002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e4c8fbf2fec8be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ccae640 1 -- 172.21.15.40:0/3740246066 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090037070 con 0x7f609815f340 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ccae640 1 -- 172.21.15.40:0/3740246066 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6090032440 con 0x7f609815f340 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ccae640 1 -- 172.21.15.40:0/3740246066 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090032810 con 0x7f609815f340 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 -- 172.21.15.40:0/3740246066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f340 msgr2=0x7f60980b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f340 0x7f60980b1e80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6090009870 tx=0x7f609002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 -- 172.21.15.40:0/3740246066 shutdown_connections 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f609815f720 0x7f60980d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 --2- 172.21.15.40:0/3740246066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f340 0x7f60980b1e80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 -- 172.21.15.40:0/3740246066 >> 172.21.15.40:0/3740246066 conn(0x7f609808ec70 msgr2=0x7f609809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 -- 172.21.15.40:0/3740246066 shutdown_connections 2024-04-26T21:14:07.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.391+0000 7f609ff14640 1 -- 172.21.15.40:0/3740246066 wait complete. 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.392+0000 7f609ff14640 1 Processor -- start 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 -- start start 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980d4f00 0x7f6098159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f609815f340 0x7f60980ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60980b3980 con 0x7f609815f720 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60980b3800 con 0x7f609815f340 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609ff14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60980b3b00 con 0x7f60980d4f00 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609e4b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609e4b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41534/0 (socket says 172.21.15.40:41534) 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609e4b1640 1 -- 172.21.15.40:0/2646950323 learned_addr learned my addr 172.21.15.40:0/2646950323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609d4af640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f609815f340 0x7f60980ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.393+0000 7f609dcb0640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980d4f00 0x7f6098159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 -- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980d4f00 msgr2=0x7f6098159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980d4f00 0x7f6098159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 -- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f609815f340 msgr2=0x7f60980ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f609815f340 0x7f60980ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 -- 172.21.15.40:0/2646950323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6098160a80 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609e4b1640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f608800c710 tx=0x7f608800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088016020 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6088006d50 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6098160d00 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088003bb0 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.394+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60981612c0 con 0x7f609815f720 2024-04-26T21:14:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.395+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6098069ae0 con 0x7f609815f720 2024-04-26T21:14:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.396+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f608800a850 con 0x7f609815f720 2024-04-26T21:14:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.396+0000 7f60967fc640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 0x7f608406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.396+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f608808b2d0 con 0x7f609815f720 2024-04-26T21:14:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.396+0000 7f609dcb0640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 0x7f608406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.397+0000 7f609dcb0640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 0x7f608406ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6090034000 tx=0x7f609002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.400+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6098069ae0 con 0x7f609815f720 2024-04-26T21:14:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.586+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60980c86f0 con 0x7f609815f720 2024-04-26T21:14:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.587+0000 7f60967fc640 1 -- 172.21.15.40:0/2646950323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+12891 (secure 0 0 0) 0x7f60980c86f0 con 0x7f609815f720 2024-04-26T21:14:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:07.588 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":43,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:06.880844+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:07.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 msgr2=0x7f608406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 0x7f608406ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6090034000 tx=0x7f609002e000 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 msgr2=0x7f6098160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f608800c710 tx=0x7f608800cbd0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 shutdown_connections 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f608406c920 0x7f608406ede0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f609815f720 0x7f6098160540 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f609815f340 0x7f60980ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 --2- 172.21.15.40:0/2646950323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980d4f00 0x7f6098159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 >> 172.21.15.40:0/2646950323 conn(0x7f609808ec70 msgr2=0x7f60980b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 shutdown_connections 2024-04-26T21:14:07.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.589+0000 7f609ff14640 1 -- 172.21.15.40:0/2646950323 wait complete. 2024-04-26T21:14:07.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:07.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.704+0000 7f5178b89640 1 Processor -- start 2024-04-26T21:14:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.704+0000 7f5178b89640 1 -- start start 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.705+0000 7f5178b89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f517415f990 0x7f51740d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.705+0000 7f5178b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51740d0f40 0x7f51740d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.705+0000 7f5178b89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51740b6aa0 con 0x7f517415f5b0 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.705+0000 7f5178b89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f51740d3eb0 con 0x7f517415f990 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.705+0000 7f5178b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51740d4190 con 0x7f51740d0f40 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f517415f990 0x7f51740d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5172d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51740d0f40 0x7f51740d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f517259c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f517415f5b0 0x7f51740b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48584/0 (socket says 172.21.15.40:48584) 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f517415f990 0x7f51740d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:39772/0 (socket says 172.21.15.40:39772) 2024-04-26T21:14:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f517259c640 1 -- 172.21.15.40:0/2276416876 learned_addr learned my addr 172.21.15.40:0/2276416876 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 -- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51740d0f40 msgr2=0x7f51740d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 --2- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51740d0f40 0x7f51740d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 -- 172.21.15.40:0/2276416876 >> v1:172.21.15.184:6789/0 conn(0x7f517415f5b0 legacy=0x7f51740b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.706+0000 7f5171d9b640 1 -- 172.21.15.40:0/2276416876 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51740d0200 con 0x7f517415f990 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.707+0000 7f5171d9b640 1 --2- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517415f990 0x7f51740d0a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c110fdc182a267 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.707+0000 7f517159a640 1 -- 172.21.15.40:0/2276416876 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c037070 con 0x7f517415f990 2024-04-26T21:14:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.707+0000 7f517159a640 1 -- 172.21.15.40:0/2276416876 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c02ac90 con 0x7f517415f990 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.707+0000 7f517159a640 1 -- 172.21.15.40:0/2276416876 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c032660 con 0x7f517415f990 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 -- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517415f990 msgr2=0x7f51740d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 --2- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517415f990 0x7f51740d0a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f515c0097a0 tx=0x7f515c02a000 comp rx=0 tx=0).stop 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 -- 172.21.15.40:0/2276416876 shutdown_connections 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 --2- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51740d0f40 0x7f51740d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 --2- 172.21.15.40:0/2276416876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517415f990 0x7f51740d0a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 -- 172.21.15.40:0/2276416876 >> 172.21.15.40:0/2276416876 conn(0x7f517408ec40 msgr2=0x7f517409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 -- 172.21.15.40:0/2276416876 shutdown_connections 2024-04-26T21:14:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.708+0000 7f5178b89640 1 -- 172.21.15.40:0/2276416876 wait complete. 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.709+0000 7f5178b89640 1 Processor -- start 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.709+0000 7f5178b89640 1 -- start start 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 0x7f517415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f517415f990 0x7f51740cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b5900 con 0x7f517415f5b0 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b5780 con 0x7f51740d0f40 2024-04-26T21:14:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5178b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51740d3880 con 0x7f517415f990 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41546/0 (socket says 172.21.15.40:41546) 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 -- 172.21.15.40:0/3760144544 learned_addr learned my addr 172.21.15.40:0/3760144544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f517259c640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 0x7f517415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 -- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f517415f990 msgr2=0x7f51740cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f517415f990 0x7f51740cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 -- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 msgr2=0x7f517415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 0x7f517415fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 -- 172.21.15.40:0/3760144544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51741f5670 con 0x7f517415f5b0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f517259c640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 0x7f517415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.710+0000 7f5171d9b640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f515c02a040 tx=0x7f515c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.711+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c037040 con 0x7f517415f5b0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.711+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c004030 con 0x7f517415f5b0 2024-04-26T21:14:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.711+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c006e10 con 0x7f517415f5b0 2024-04-26T21:14:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.711+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51741f5950 con 0x7f517415f5b0 2024-04-26T21:14:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.711+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51741f5f10 con 0x7f517415f5b0 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.712+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517406a290 con 0x7f517415f5b0 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.712+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f515c003610 con 0x7f517415f5b0 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.712+0000 7f516affd640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 0x7f515406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.712+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f515c0ac0d0 con 0x7f517415f5b0 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.713+0000 7f517259c640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 0x7f515406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.713+0000 7f517259c640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 0x7f515406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f516c000f60 tx=0x7f516c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.717+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f515c03a350 con 0x7f517415f5b0 2024-04-26T21:14:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.902+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51740d4600 con 0x7f517415f5b0 2024-04-26T21:14:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.902+0000 7f516affd640 1 -- 172.21.15.40:0/3760144544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f515c07a8e0 con 0x7f517415f5b0 2024-04-26T21:14:07.903 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 msgr2=0x7f515406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 0x7f515406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f516c000f60 tx=0x7f516c009040 comp rx=0 tx=0).stop 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 msgr2=0x7f51741615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f515c02a040 tx=0x7f515c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 shutdown_connections 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515406c8f0 0x7f515406edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f517415f990 0x7f51740cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f517415f5b0 0x7f51741615a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 --2- 172.21.15.40:0/3760144544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51740d0f40 0x7f517415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 >> 172.21.15.40:0/3760144544 conn(0x7f517408ec40 msgr2=0x7f51740b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 shutdown_connections 2024-04-26T21:14:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:07.904+0000 7f5178b89640 1 -- 172.21.15.40:0/3760144544 wait complete. 2024-04-26T21:14:07.916 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:14:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.015+0000 7f5501da4640 1 Processor -- start 2024-04-26T21:14:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.015+0000 7f5501da4640 1 -- start start 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f5501da4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54fc15f370 0x7f54fc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f5501da4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc0d03a0 con 0x7f54fc0d3ee0 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f5501da4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc0d0680 con 0x7f54fc15f750 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f5501da4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54fc0d0960 con 0x7f54fc15f370 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f54fb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54fc15f370 0x7f54fc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f54fb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54fc15f370 0x7f54fc0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41552/0 (socket says 172.21.15.40:41552) 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.016+0000 7f54fb7fe640 1 -- 172.21.15.40:0/1128982718 learned_addr learned my addr 172.21.15.40:0/1128982718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.017+0000 7f54fb7fe640 1 -- 172.21.15.40:0/1128982718 >> v1:172.21.15.184:6789/0 conn(0x7f54fc15f750 legacy=0x7f54fc0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.017+0000 7f54fb7fe640 1 -- 172.21.15.40:0/1128982718 >> v1:172.21.15.40:6789/0 conn(0x7f54fc0d3ee0 legacy=0x7f54fc0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.017+0000 7f54fb7fe640 1 -- 172.21.15.40:0/1128982718 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc0d0c40 con 0x7f54fc15f370 2024-04-26T21:14:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.017+0000 7f54fb7fe640 1 --2- 172.21.15.40:0/1128982718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc0cfd70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009870 tx=0x7f54ec02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baa61ae4f3dfcfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f54fa7fc640 1 -- 172.21.15.40:0/1128982718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec037070 con 0x7f54fc15f370 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f54fa7fc640 1 -- 172.21.15.40:0/1128982718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec032440 con 0x7f54fc15f370 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f54fa7fc640 1 -- 172.21.15.40:0/1128982718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec032810 con 0x7f54fc15f370 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 -- 172.21.15.40:0/1128982718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 msgr2=0x7f54fc0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 --2- 172.21.15.40:0/1128982718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc0cfd70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009870 tx=0x7f54ec02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 -- 172.21.15.40:0/1128982718 shutdown_connections 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 --2- 172.21.15.40:0/1128982718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc0cfd70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 -- 172.21.15.40:0/1128982718 >> 172.21.15.40:0/1128982718 conn(0x7f54fc08ec10 msgr2=0x7f54fc09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 -- 172.21.15.40:0/1128982718 shutdown_connections 2024-04-26T21:14:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.018+0000 7f5501da4640 1 -- 172.21.15.40:0/1128982718 wait complete. 2024-04-26T21:14:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.019+0000 7f5501da4640 1 Processor -- start 2024-04-26T21:14:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 -- start start 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 0x7f54fc157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 0x7f54fc1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc0b4950 con 0x7f54fc15f370 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc0b47d0 con 0x7f54fc0d3ee0 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f5501da4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc0b4ad0 con 0x7f54fc15f750 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f54faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f54faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41566/0 (socket says 172.21.15.40:41566) 2024-04-26T21:14:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.020+0000 7f54faffd640 1 -- 172.21.15.40:0/1784475425 learned_addr learned my addr 172.21.15.40:0/1784475425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54fbfff640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 0x7f54fc1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54fb7fe640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 0x7f54fc157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 -- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 msgr2=0x7f54fc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 0x7f54fc1621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 -- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 msgr2=0x7f54fc157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 0x7f54fc157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 -- 172.21.15.40:0/1784475425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc160f90 con 0x7f54fc15f370 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54fbfff640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 0x7f54fc1621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54fb7fe640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 0x7f54fc157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54faffd640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c430 tx=0x7f54f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0011790 con 0x7f54fc15f370 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0017070 con 0x7f54fc15f370 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0011b70 con 0x7f54fc15f370 2024-04-26T21:14:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc161270 con 0x7f54fc15f370 2024-04-26T21:14:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.021+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc161830 con 0x7f54fc15f370 2024-04-26T21:14:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.022+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54fc06a290 con 0x7f54fc15f370 2024-04-26T21:14:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.023+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54f0011d10 con 0x7f54fc15f370 2024-04-26T21:14:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.023+0000 7f54e3fff640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.023+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f54f008b630 con 0x7f54fc15f370 2024-04-26T21:14:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.023+0000 7f54fb7fe640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.024+0000 7f54fb7fe640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 0x7f54e806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f54ec034000 tx=0x7f54ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.028+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54f0059e40 con 0x7f54fc15f370 2024-04-26T21:14:08.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.214+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f54fc0bae20 con 0x7f54fc15f370 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.215+0000 7f54e3fff640 1 -- 172.21.15.40:0/1784475425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7f54f0059740 con 0x7f54fc15f370 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:08.216 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 msgr2=0x7f54e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 0x7f54e806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f54ec034000 tx=0x7f54ec02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 msgr2=0x7f54fc1605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c430 tx=0x7f54f000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 shutdown_connections 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f54e806c8f0 0x7f54e806edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54fc15f750 0x7f54fc1621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc15f370 0x7f54fc1605d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 --2- 172.21.15.40:0/1784475425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54fc0d3ee0 0x7f54fc157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.216+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 >> 172.21.15.40:0/1784475425 conn(0x7f54fc08ec10 msgr2=0x7f54fc0d5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.217+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 shutdown_connections 2024-04-26T21:14:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.217+0000 7f5501da4640 1 -- 172.21.15.40:0/1784475425 wait complete. 2024-04-26T21:14:08.228 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:14:08.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:14:08.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.332+0000 7f7f5c104640 1 Processor -- start 2024-04-26T21:14:08.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.332+0000 7f7f5c104640 1 -- start start 2024-04-26T21:14:08.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5c104640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7f540d2140 0x7f7f540d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5c104640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f540b6020 0x7f7f540cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5c104640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f540d33a0 con 0x7f7f54162a10 2024-04-26T21:14:08.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5c104640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7f540b4970 con 0x7f7f540b6020 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5c104640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7f540d3680 con 0x7f7f540d2140 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f540b6020 0x7f7f540cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f540b6020 0x7f7f540cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41574/0 (socket says 172.21.15.40:41574) 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 -- 172.21.15.40:0/3962952610 learned_addr learned my addr 172.21.15.40:0/3962952610 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5969f640 1 --2- 172.21.15.40:0/3962952610 >> v2:172.21.15.184:3300/0 conn(0x7f7f540d2140 0x7f7f540d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 -- 172.21.15.40:0/3962952610 >> v2:172.21.15.184:3300/0 conn(0x7f7f540d2140 msgr2=0x7f7f540d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 --2- 172.21.15.40:0/3962952610 >> v2:172.21.15.184:3300/0 conn(0x7f7f540d2140 0x7f7f540d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 -- 172.21.15.40:0/3962952610 >> v1:172.21.15.184:6789/0 conn(0x7f7f54162a10 legacy=0x7f7f540d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:08.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.333+0000 7f7f5a6a1640 1 -- 172.21.15.40:0/3962952610 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f540d3960 con 0x7f7f540b6020 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.334+0000 7f7f5a6a1640 1 --2- 172.21.15.40:0/3962952610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540b6020 0x7f7f540cffb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7f5408f0b0 tx=0x7f7f4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b831e43ec7b70d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.334+0000 7f7f58e9e640 1 -- 172.21.15.40:0/3962952610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44036070 con 0x7f7f540b6020 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.334+0000 7f7f58e9e640 1 -- 172.21.15.40:0/3962952610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f44031600 con 0x7f7f540b6020 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.334+0000 7f7f58e9e640 1 -- 172.21.15.40:0/3962952610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440319d0 con 0x7f7f540b6020 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 -- 172.21.15.40:0/3962952610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540b6020 msgr2=0x7f7f540cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 --2- 172.21.15.40:0/3962952610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540b6020 0x7f7f540cffb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7f5408f0b0 tx=0x7f7f4402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 -- 172.21.15.40:0/3962952610 shutdown_connections 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 --2- 172.21.15.40:0/3962952610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540b6020 0x7f7f540cffb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 --2- 172.21.15.40:0/3962952610 >> v2:172.21.15.184:3300/0 conn(0x7f7f540d2140 0x7f7f540d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 -- 172.21.15.40:0/3962952610 >> 172.21.15.40:0/3962952610 conn(0x7f7f5409e530 msgr2=0x7f7f540d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:08.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 -- 172.21.15.40:0/3962952610 shutdown_connections 2024-04-26T21:14:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.335+0000 7f7f5c104640 1 -- 172.21.15.40:0/3962952610 wait complete. 2024-04-26T21:14:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.336+0000 7f7f5c104640 1 Processor -- start 2024-04-26T21:14:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.336+0000 7f7f5c104640 1 -- start start 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f540b6020 0x7f7f540d1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f54162a10 0x7f7f5415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54162df0 con 0x7f7f540d2140 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f540d3f20 con 0x7f7f540b6020 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5c104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7f540b6600 con 0x7f7f54162a10 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41584/0 (socket says 172.21.15.40:41584) 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 -- 172.21.15.40:0/1503832480 learned_addr learned my addr 172.21.15.40:0/1503832480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5a6a1640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f54162a10 0x7f7f5415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f59ea0640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f540b6020 0x7f7f540d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 -- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f54162a10 msgr2=0x7f7f5415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f54162a10 0x7f7f5415c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 -- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f540b6020 msgr2=0x7f7f540d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f540b6020 0x7f7f540d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 -- 172.21.15.40:0/1503832480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f540cd4a0 con 0x7f7f540d2140 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.337+0000 7f7f5969f640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d800 tx=0x7f7f4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.338+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800b660 con 0x7f7f540d2140 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.338+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f48019070 con 0x7f7f540d2140 2024-04-26T21:14:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.338+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f540cd780 con 0x7f7f540d2140 2024-04-26T21:14:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.338+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800ba20 con 0x7f7f540d2140 2024-04-26T21:14:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.338+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f540cdd40 con 0x7f7f540d2140 2024-04-26T21:14:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.339+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5406a290 con 0x7f7f540d2140 2024-04-26T21:14:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.339+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f4800bbc0 con 0x7f7f540d2140 2024-04-26T21:14:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.340+0000 7f7f527fc640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 0x7f7f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.340+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f7f4808bae0 con 0x7f7f540d2140 2024-04-26T21:14:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.340+0000 7f7f59ea0640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 0x7f7f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:08.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.340+0000 7f7f59ea0640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 0x7f7f4006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006fd0 tx=0x7f7f4c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.344+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f48059c10 con 0x7f7f540d2140 2024-04-26T21:14:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.526+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7f540b6ac0 con 0x7f7f540d2140 2024-04-26T21:14:08.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.885+0000 7f7f527fc640 1 -- 172.21.15.40:0/1503832480 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f7f4805df10 con 0x7f7f540d2140 2024-04-26T21:14:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.888+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 msgr2=0x7f7f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.888+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 0x7f7f4006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006fd0 tx=0x7f7f4c009060 comp rx=0 tx=0).stop 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 msgr2=0x7f7f540ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d800 tx=0x7f7f4800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 shutdown_connections 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7f4006c8f0 0x7f7f4006edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f54162a10 0x7f7f5415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f540d2140 0x7f7f540ce540 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 --2- 172.21.15.40:0/1503832480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f540b6020 0x7f7f540d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:08.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 >> 172.21.15.40:0/1503832480 conn(0x7f7f5409e530 msgr2=0x7f7f5406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:08.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 shutdown_connections 2024-04-26T21:14:08.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:08.889+0000 7f7f5c104640 1 -- 172.21.15.40:0/1503832480 wait complete. 2024-04-26T21:14:08.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:14:09.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 Processor -- start 2024-04-26T21:14:09.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 -- start start 2024-04-26T21:14:09.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01580b1ea0 0x7f01580d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01580b2580 con 0x7f01580d56a0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01580b2860 con 0x7f015815f6e0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.012+0000 7f015caf1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f01580b2330 con 0x7f01580b1ea0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0156d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f01580d56a0 0x7f01580b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33060/0 (socket says 172.21.15.40:33060) 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0156d9d640 1 -- 172.21.15.40:0/745765813 learned_addr learned my addr 172.21.15.40:0/745765813 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f014ffff640 1 --2- 172.21.15.40:0/745765813 >> v2:172.21.15.184:3300/0 conn(0x7f01580b1ea0 0x7f01580d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855405302 0 0) 0x7f01580b2580 con 0x7f01580d56a0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0144003520 con 0x7f01580d56a0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374257344 0 0) 0x7f0144003520 con 0x7f01580d56a0 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 >> v2:172.21.15.184:3300/0 conn(0x7f01580b1ea0 msgr2=0x7f01580d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 --2- 172.21.15.40:0/745765813 >> v2:172.21.15.184:3300/0 conn(0x7f01580b1ea0 0x7f01580d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 >> v1:172.21.15.184:6789/0 conn(0x7f015815f6e0 legacy=0x7f01580d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01580b2b40 con 0x7f01580d56a0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f01500019f0 con 0x7f01580d56a0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 >> v1:172.21.15.40:6789/0 conn(0x7f01580d56a0 legacy=0x7f01580b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01440039b0 0x7f0144003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01440051c0 0x7f014400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b2330 con 0x7f0144004800 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f014400c060 con 0x7f01440039b0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f014400c260 con 0x7f01440051c0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f014ffff640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.013+0000 7f015659c640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01440039b0 0x7f0144003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f0156d9d640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01440051c0 0x7f014400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 -- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01440051c0 msgr2=0x7f014400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01440051c0 0x7f014400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 -- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01440039b0 msgr2=0x7f0144003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01440039b0 0x7f0144003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 -- 172.21.15.40:0/745765813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0144003520 con 0x7f0144004800 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f0156d9d640 1 -- 172.21.15.40:0/745765813 reap_dead start 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f014ffff640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0144004c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0140009ac0 tx=0x7f0144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2c7c96b49f6047 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140036070 con 0x7f0144004800 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140031400 con 0x7f0144004800 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.014+0000 7f0155d9b640 1 -- 172.21.15.40:0/745765813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140009690 con 0x7f0144004800 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 -- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 msgr2=0x7f0144004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0144004c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0140009ac0 tx=0x7f0144005830 comp rx=0 tx=0).stop 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 -- 172.21.15.40:0/745765813 shutdown_connections 2024-04-26T21:14:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 --2- 172.21.15.40:0/745765813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0144004c80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 -- 172.21.15.40:0/745765813 >> 172.21.15.40:0/745765813 conn(0x7f015808ec20 msgr2=0x7f015809e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:09.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 -- 172.21.15.40:0/745765813 shutdown_connections 2024-04-26T21:14:09.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.015+0000 7f015caf1640 1 -- 172.21.15.40:0/745765813 wait complete. 2024-04-26T21:14:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.016+0000 7f015caf1640 1 Processor -- start 2024-04-26T21:14:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.016+0000 7f015caf1640 1 -- start start 2024-04-26T21:14:09.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 0x7f015815fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01581599d0 0x7f01581577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f01580d5160 con 0x7f0144004800 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01580d5ad0 con 0x7f0158159550 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015caf1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f01580d52e0 con 0x7f01581599d0 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41600/0 (socket says 172.21.15.40:41600) 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.017+0000 7f015659c640 1 -- 172.21.15.40:0/1794075256 learned_addr learned my addr 172.21.15.40:0/1794075256 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 -- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01581599d0 msgr2=0x7f01581577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01581599d0 0x7f01581577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 -- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 msgr2=0x7f015815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f014ffff640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 0x7f015815fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 0x7f015815fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 -- 172.21.15.40:0/1794075256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015815aef0 con 0x7f0144004800 2024-04-26T21:14:09.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f014ffff640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 0x7f015815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015659c640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f014800f7a0 tx=0x7f014800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480050d0 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0148018070 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0158159e50 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014800d730 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.018+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015815a410 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.019+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015806a290 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.020+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f014800d8d0 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.020+0000 7f014effd640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 0x7f013806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.020+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7f014808b7e0 con 0x7f0144004800 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.020+0000 7f014ffff640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 0x7f013806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:09.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.021+0000 7f014ffff640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 0x7f013806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0140033000 tx=0x7f0140008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:09.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.025+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0148059d70 con 0x7f0144004800 2024-04-26T21:14:09.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.209+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f015815c5d0 con 0x7f0144004800 2024-04-26T21:14:09.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.888+0000 7f014effd640 1 -- 172.21.15.40:0/1794075256 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7f0148059670 con 0x7f0144004800 2024-04-26T21:14:09.889 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 msgr2=0x7f013806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 0x7f013806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0140033000 tx=0x7f0140008030 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 msgr2=0x7f0158162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f014800f7a0 tx=0x7f014800fc60 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 shutdown_connections 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f013806c920 0x7f013806ede0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0140033000 tx=0x7f0140008030 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01581599d0 0x7f01581577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.897+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0158159550 0x7f015815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.898+0000 7f015caf1640 1 --2- 172.21.15.40:0/1794075256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0144004800 0x7f0158162670 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f014800f7a0 tx=0x7f014800fc60 comp rx=0 tx=0).stop 2024-04-26T21:14:09.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.898+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 >> 172.21.15.40:0/1794075256 conn(0x7f015808ec20 msgr2=0x7f01580cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:09.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.901+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 shutdown_connections 2024-04-26T21:14:09.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:09.903+0000 7f015caf1640 1 -- 172.21.15.40:0/1794075256 wait complete. 2024-04-26T21:14:09.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:14:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.026+0000 7f4991672640 1 Processor -- start 2024-04-26T21:14:10.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.026+0000 7f4991672640 1 -- start start 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f4991672640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f498c1631f0 0x7f498c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f4991672640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f498c06ea30 0x7f498c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f4991672640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498c09c790 con 0x7f498c0d1170 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f4991672640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f498c06eec0 con 0x7f498c06ea30 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f4991672640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f498c0d15f0 con 0x7f498c1631f0 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f498c06ea30 0x7f498c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f498c06ea30 0x7f498c0b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41614/0 (socket says 172.21.15.40:41614) 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 -- 172.21.15.40:0/3886477492 learned_addr learned my addr 172.21.15.40:0/3886477492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 -- 172.21.15.40:0/3886477492 >> v2:172.21.15.184:3300/0 conn(0x7f498c1631f0 msgr2=0x7f498c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498affd640 1 --2- 172.21.15.40:0/3886477492 >> v2:172.21.15.184:3300/0 conn(0x7f498c1631f0 0x7f498c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 --2- 172.21.15.40:0/3886477492 >> v2:172.21.15.184:3300/0 conn(0x7f498c1631f0 0x7f498c0b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 -- 172.21.15.40:0/3886477492 >> v1:172.21.15.184:6789/0 conn(0x7f498c0d1170 legacy=0x7f498c1635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.027+0000 7f498b7fe640 1 -- 172.21.15.40:0/3886477492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c163ec0 con 0x7f498c06ea30 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.028+0000 7f498b7fe640 1 --2- 172.21.15.40:0/3886477492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c0b3300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f498c0d2ef0 tx=0x7f498402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11761a0c7cd68d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.028+0000 7f4989ffb640 1 -- 172.21.15.40:0/3886477492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984036070 con 0x7f498c06ea30 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.028+0000 7f4989ffb640 1 -- 172.21.15.40:0/3886477492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984031600 con 0x7f498c06ea30 2024-04-26T21:14:10.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.028+0000 7f4989ffb640 1 -- 172.21.15.40:0/3886477492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840319d0 con 0x7f498c06ea30 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 -- 172.21.15.40:0/3886477492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 msgr2=0x7f498c0b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 --2- 172.21.15.40:0/3886477492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c0b3300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f498c0d2ef0 tx=0x7f498402a660 comp rx=0 tx=0).stop 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 -- 172.21.15.40:0/3886477492 shutdown_connections 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 --2- 172.21.15.40:0/3886477492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c0b3300 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 --2- 172.21.15.40:0/3886477492 >> v2:172.21.15.184:3300/0 conn(0x7f498c1631f0 0x7f498c0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 -- 172.21.15.40:0/3886477492 >> 172.21.15.40:0/3886477492 conn(0x7f498c09ba40 msgr2=0x7f498c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 -- 172.21.15.40:0/3886477492 shutdown_connections 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.029+0000 7f4991672640 1 -- 172.21.15.40:0/3886477492 wait complete. 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.030+0000 7f4991672640 1 Processor -- start 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.030+0000 7f4991672640 1 -- start start 2024-04-26T21:14:10.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 0x7f498c1ea990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c1631f0 0x7f498c15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0d2000 con 0x7f498c06ea30 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0d1d20 con 0x7f498c0d1170 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f4991672640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f498c0d0230 con 0x7f498c1631f0 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41616/0 (socket says 172.21.15.40:41616) 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 -- 172.21.15.40:0/1314005578 learned_addr learned my addr 172.21.15.40:0/1314005578 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498b7fe640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c1631f0 0x7f498c15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498a7fc640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 0x7f498c1ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 -- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c1631f0 msgr2=0x7f498c15c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c1631f0 0x7f498c15c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 -- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 msgr2=0x7f498c1ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:10.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 0x7f498c1ea990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 -- 172.21.15.40:0/1314005578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c15b4d0 con 0x7f498c06ea30 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498a7fc640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 0x7f498c1ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f498affd640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c820 tx=0x7f497c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.031+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c015070 con 0x7f498c06ea30 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.032+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c004240 con 0x7f498c06ea30 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.032+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c149440 con 0x7f498c06ea30 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.032+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c00ce10 con 0x7f498c06ea30 2024-04-26T21:14:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.032+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c149a00 con 0x7f498c06ea30 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.033+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c06a290 con 0x7f498c06ea30 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.033+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f497c007830 con 0x7f498c06ea30 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.034+0000 7f49737fe640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.034+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7f497c08b970 con 0x7f498c06ea30 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.034+0000 7f498a7fc640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.035+0000 7f498a7fc640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 0x7f497806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4980007fd0 tx=0x7f4980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:10.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.039+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f497c059e90 con 0x7f498c06ea30 2024-04-26T21:14:10.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.267+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f498c0b8f10 con 0x7f498c06ea30 2024-04-26T21:14:10.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.907+0000 7f49737fe640 1 -- 172.21.15.40:0/1314005578 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) ==== 172+0+0 (secure 0 0 0) 0x7f497c059790 con 0x7f498c06ea30 2024-04-26T21:14:10.908 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 6 and data pool 7 2024-04-26T21:14:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 msgr2=0x7f497806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 0x7f497806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4980007fd0 tx=0x7f4980009040 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 msgr2=0x7f498c15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c820 tx=0x7f497c00cce0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 shutdown_connections 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f498c1631f0 0x7f498c15c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f498c0d1170 0x7f498c1ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 --2- 172.21.15.40:0/1314005578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c06ea30 0x7f498c15af90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.910+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 >> 172.21.15.40:0/1314005578 conn(0x7f498c09ba40 msgr2=0x7f498c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.911+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 shutdown_connections 2024-04-26T21:14:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:10.911+0000 7f4991672640 1 -- 172.21.15.40:0/1314005578 wait complete. 2024-04-26T21:14:10.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.047+0000 7fbf22197640 1 Processor -- start 2024-04-26T21:14:11.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.047+0000 7fbf22197640 1 -- start start 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf22197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf22197640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbf1c15f750 0x7fbf1c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf22197640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1c0d03a0 con 0x7fbf1c15f370 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf22197640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1c0b4d10 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf22197640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1c0d0680 con 0x7fbf1c15f750 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf1affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf1affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59886/0 (socket says 172.21.15.40:59886) 2024-04-26T21:14:11.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.048+0000 7fbf1affd640 1 -- 172.21.15.40:0/2519734144 learned_addr learned my addr 172.21.15.40:0/2519734144 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1bfff640 1 --2- 172.21.15.40:0/2519734144 >> v2:172.21.15.184:3300/0 conn(0x7fbf1c15f750 0x7fbf1c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1affd640 1 -- 172.21.15.40:0/2519734144 >> v2:172.21.15.184:3300/0 conn(0x7fbf1c15f750 msgr2=0x7fbf1c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:11.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1affd640 1 --2- 172.21.15.40:0/2519734144 >> v2:172.21.15.184:3300/0 conn(0x7fbf1c15f750 0x7fbf1c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1affd640 1 -- 172.21.15.40:0/2519734144 >> v1:172.21.15.40:6789/0 conn(0x7fbf1c15f370 legacy=0x7fbf1c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1affd640 1 -- 172.21.15.40:0/2519734144 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c0d0960 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.049+0000 7fbf1affd640 1 --2- 172.21.15.40:0/2519734144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009870 tx=0x7fbf1002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8ee085f231d9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf1a7fc640 1 -- 172.21.15.40:0/2519734144 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10037070 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf1a7fc640 1 -- 172.21.15.40:0/2519734144 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf10032440 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf1a7fc640 1 -- 172.21.15.40:0/2519734144 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10032810 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 -- 172.21.15.40:0/2519734144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c0d3ee0 msgr2=0x7fbf1c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 --2- 172.21.15.40:0/2519734144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009870 tx=0x7fbf1002a2a0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 -- 172.21.15.40:0/2519734144 shutdown_connections 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 --2- 172.21.15.40:0/2519734144 >> v2:172.21.15.184:3300/0 conn(0x7fbf1c15f750 0x7fbf1c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 --2- 172.21.15.40:0/2519734144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c0cfd70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 -- 172.21.15.40:0/2519734144 >> 172.21.15.40:0/2519734144 conn(0x7fbf1c08ec10 msgr2=0x7fbf1c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 -- 172.21.15.40:0/2519734144 shutdown_connections 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.050+0000 7fbf22197640 1 -- 172.21.15.40:0/2519734144 wait complete. 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.051+0000 7fbf22197640 1 Processor -- start 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 -- start start 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c14bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c15f750 0x7fbf1c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c0b51c0 con 0x7fbf1c15f370 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c0d0f20 con 0x7fbf1c0d3ee0 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.052+0000 7fbf22197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c0b5340 con 0x7fbf1c15f750 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59898/0 (socket says 172.21.15.40:59898) 2024-04-26T21:14:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 -- 172.21.15.40:0/3127015982 learned_addr learned my addr 172.21.15.40:0/3127015982 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 -- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c15f750 msgr2=0x7fbf1c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c15f750 0x7fbf1c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 -- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c0d3ee0 msgr2=0x7fbf1c14bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c14bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 -- 172.21.15.40:0/3127015982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c1f93b0 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf1affd640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000c00 tx=0x7fbf100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10041020 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf100306c0 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c14e7e0 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10037040 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.053+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c14eda0 con 0x7fbf1c15f370 2024-04-26T21:14:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.054+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c06a290 con 0x7fbf1c15f370 2024-04-26T21:14:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.055+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf10044020 con 0x7fbf1c15f370 2024-04-26T21:14:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.055+0000 7fbf03fff640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.055+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbf100ac3b0 con 0x7fbf1c15f370 2024-04-26T21:14:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.055+0000 7fbf1b7fe640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.056+0000 7fbf1b7fe640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 0x7fbf0806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000f60 tx=0x7fbf0c0079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.059+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf1004b030 con 0x7fbf1c15f370 2024-04-26T21:14:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.248+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf1c0d14e0 con 0x7fbf1c15f370 2024-04-26T21:14:11.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.249+0000 7fbf03fff640 1 -- 172.21.15.40:0/3127015982 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16626 (secure 0 0 0) 0x7fbf1007a7e0 con 0x7fbf1c15f370 2024-04-26T21:14:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:11.250 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:10.268848+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 msgr2=0x7fbf0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 0x7fbf0806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000f60 tx=0x7fbf0c0079d0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 msgr2=0x7fbf1c0b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000c00 tx=0x7fbf100046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 shutdown_connections 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c15f750 0x7fbf1c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf1c15f370 0x7fbf1c0b2830 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 --2- 172.21.15.40:0/3127015982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c0d3ee0 0x7fbf1c14bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 >> 172.21.15.40:0/3127015982 conn(0x7fbf1c08ec10 msgr2=0x7fbf1c06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 shutdown_connections 2024-04-26T21:14:11.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.251+0000 7fbf22197640 1 -- 172.21.15.40:0/3127015982 wait complete. 2024-04-26T21:14:11.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.362+0000 7fe25bf97640 1 Processor -- start 2024-04-26T21:14:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.362+0000 7fe25bf97640 1 -- start start 2024-04-26T21:14:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe25bf97640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe25415f370 0x7fe2540cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe25bf97640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 0x7fe2540b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe25bf97640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2540d03a0 con 0x7fe2540d37d0 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe25bf97640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe2540b4d10 con 0x7fe25415f370 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe25bf97640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe2540d0680 con 0x7fe25415f750 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe259d33640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe25415f370 0x7fe2540cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe259d33640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe25415f370 0x7fe2540cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59906/0 (socket says 172.21.15.40:59906) 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.363+0000 7fe259d33640 1 -- 172.21.15.40:0/3839535713 learned_addr learned my addr 172.21.15.40:0/3839535713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259532640 1 --2- 172.21.15.40:0/3839535713 >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 0x7fe2540b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259d33640 1 -- 172.21.15.40:0/3839535713 >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 msgr2=0x7fe2540b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259d33640 1 --2- 172.21.15.40:0/3839535713 >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 0x7fe2540b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259d33640 1 -- 172.21.15.40:0/3839535713 >> v1:172.21.15.40:6789/0 conn(0x7fe2540d37d0 legacy=0x7fe2540d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259d33640 1 -- 172.21.15.40:0/3839535713 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2540d0960 con 0x7fe25415f370 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259532640 1 --2- 172.21.15.40:0/3839535713 >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 0x7fe2540b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.364+0000 7fe259d33640 1 --2- 172.21.15.40:0/3839535713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f370 0x7fe2540cfd70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe2540d02b0 tx=0x7fe25002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7c77cc17495b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe258d31640 1 -- 172.21.15.40:0/3839535713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250037070 con 0x7fe25415f370 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe258d31640 1 -- 172.21.15.40:0/3839535713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe250032440 con 0x7fe25415f370 2024-04-26T21:14:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe258d31640 1 -- 172.21.15.40:0/3839535713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250032810 con 0x7fe25415f370 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 -- 172.21.15.40:0/3839535713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f370 msgr2=0x7fe2540cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 --2- 172.21.15.40:0/3839535713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f370 0x7fe2540cfd70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe2540d02b0 tx=0x7fe25002a270 comp rx=0 tx=0).stop 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 -- 172.21.15.40:0/3839535713 shutdown_connections 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 --2- 172.21.15.40:0/3839535713 >> v2:172.21.15.184:3300/0 conn(0x7fe25415f750 0x7fe2540b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 --2- 172.21.15.40:0/3839535713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f370 0x7fe2540cfd70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 -- 172.21.15.40:0/3839535713 >> 172.21.15.40:0/3839535713 conn(0x7fe25408ec10 msgr2=0x7fe25409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 -- 172.21.15.40:0/3839535713 shutdown_connections 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.365+0000 7fe25bf97640 1 -- 172.21.15.40:0/3839535713 wait complete. 2024-04-26T21:14:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.366+0000 7fe25bf97640 1 Processor -- start 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.366+0000 7fe25bf97640 1 -- start start 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2540d37d0 0x7fe2540b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe25415f370 0x7fe254161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe2540b61d0 con 0x7fe25415f750 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2540d0f20 con 0x7fe2540d37d0 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25bf97640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe2540b6350 con 0x7fe25415f370 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25a534640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25a534640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59918/0 (socket says 172.21.15.40:59918) 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.367+0000 7fe25a534640 1 -- 172.21.15.40:0/502265633 learned_addr learned my addr 172.21.15.40:0/502265633 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 -- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe25415f370 msgr2=0x7fe254161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe25415f370 0x7fe254161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 -- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2540d37d0 msgr2=0x7fe2540b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2540d37d0 0x7fe2540b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 -- 172.21.15.40:0/502265633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2540c9090 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25a534640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c6b0 tx=0x7fe24c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c003b70 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c003d30 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2540c9370 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c005310 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.368+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2540c9930 con 0x7fe25415f750 2024-04-26T21:14:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.369+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25406a290 con 0x7fe25415f750 2024-04-26T21:14:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.369+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe24c0054b0 con 0x7fe25415f750 2024-04-26T21:14:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.370+0000 7fe24a7fc640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 0x7fe23806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.370+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe24c08bab0 con 0x7fe25415f750 2024-04-26T21:14:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.370+0000 7fe259d33640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 0x7fe23806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.371+0000 7fe259d33640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 0x7fe23806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe250034000 tx=0x7fe25002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.374+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe24c059e40 con 0x7fe25415f750 2024-04-26T21:14:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.562+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2540c8ad0 con 0x7fe25415f750 2024-04-26T21:14:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.563+0000 7fe24a7fc640 1 -- 172.21.15.40:0/502265633 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16626 (secure 0 0 0) 0x7fe24c059740 con 0x7fe25415f750 2024-04-26T21:14:11.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:11.564 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:10.268848+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 msgr2=0x7fe23806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 0x7fe23806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe250034000 tx=0x7fe25002c000 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 msgr2=0x7fe2541622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c6b0 tx=0x7fe24c00cb70 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 shutdown_connections 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe23806c8f0 0x7fe23806edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe25415f750 0x7fe2541622f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe25415f370 0x7fe254161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 --2- 172.21.15.40:0/502265633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2540d37d0 0x7fe2540b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 >> 172.21.15.40:0/502265633 conn(0x7fe25408ec10 msgr2=0x7fe2540d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 shutdown_connections 2024-04-26T21:14:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.565+0000 7fe25bf97640 1 -- 172.21.15.40:0/502265633 wait complete. 2024-04-26T21:14:11.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:14:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.680+0000 7f8f3ecb0640 1 Processor -- start 2024-04-26T21:14:11.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.680+0000 7f8f3ecb0640 1 -- start start 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ecb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815f2b0 0x7f8f3806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ecb0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f380706a0 0x7f8f3806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ecb0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f38070b30 con 0x7f8f3806c7e0 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ecb0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8f380af610 con 0x7f8f380706a0 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ecb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f380700b0 con 0x7f8f3815f2b0 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f37fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f380706a0 0x7f8f3806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f37fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f380706a0 0x7f8f3806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:55524/0 (socket says 172.21.15.40:55524) 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f3ca4c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815f2b0 0x7f8f3806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.681+0000 7f8f37fff640 1 -- 172.21.15.40:0/2589551441 learned_addr learned my addr 172.21.15.40:0/2589551441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.682+0000 7f8f37fff640 1 -- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815f2b0 msgr2=0x7f8f3806fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.682+0000 7f8f37fff640 1 --2- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815f2b0 0x7f8f3806fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.682+0000 7f8f37fff640 1 -- 172.21.15.40:0/2589551441 >> v1:172.21.15.184:6789/0 conn(0x7f8f3806c7e0 legacy=0x7f8f38162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.682+0000 7f8f37fff640 1 -- 172.21.15.40:0/2589551441 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f38070e10 con 0x7f8f380706a0 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.682+0000 7f8f37fff640 1 --2- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f380706a0 0x7f8f3806c2a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009870 tx=0x7f8f2002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6999f7b55044ede2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f377fe640 1 -- 172.21.15.40:0/2589551441 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20037070 con 0x7f8f380706a0 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f377fe640 1 -- 172.21.15.40:0/2589551441 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20032440 con 0x7f8f380706a0 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f377fe640 1 -- 172.21.15.40:0/2589551441 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20032810 con 0x7f8f380706a0 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f380706a0 msgr2=0x7f8f3806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f380706a0 0x7f8f3806c2a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009870 tx=0x7f8f2002a270 comp rx=0 tx=0).stop 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/2589551441 shutdown_connections 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f380706a0 0x7f8f3806c2a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/2589551441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815f2b0 0x7f8f3806fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/2589551441 >> 172.21.15.40:0/2589551441 conn(0x7f8f3806f050 msgr2=0x7f8f3806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/2589551441 shutdown_connections 2024-04-26T21:14:11.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.683+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/2589551441 wait complete. 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.684+0000 7f8f3ecb0640 1 Processor -- start 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.684+0000 7f8f3ecb0640 1 -- start start 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3806c7e0 0x7f8f3814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815c4d0 0x7f8f380b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380713d0 con 0x7f8f3815f2b0 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3806cc30 con 0x7f8f3806c7e0 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f3ecb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f380b4f40 con 0x7f8f3815c4d0 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.685+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59934/0 (socket says 172.21.15.40:59934) 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 -- 172.21.15.40:0/1873840542 learned_addr learned my addr 172.21.15.40:0/1873840542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:11.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f3ca4c640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3806c7e0 0x7f8f3814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f3d24d640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815c4d0 0x7f8f380b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 -- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815c4d0 msgr2=0x7f8f380b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815c4d0 0x7f8f380b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 -- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3806c7e0 msgr2=0x7f8f3814a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3806c7e0 0x7f8f3814a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 -- 172.21.15.40:0/1873840542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3815d6a0 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f37fff640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f20000c00 tx=0x7f8f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20037040 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20042070 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3815d980 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20030880 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.686+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3814ad40 con 0x7f8f3815f2b0 2024-04-26T21:14:11.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.687+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3806a290 con 0x7f8f3815f2b0 2024-04-26T21:14:11.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.688+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f20030a20 con 0x7f8f3815f2b0 2024-04-26T21:14:11.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.688+0000 7f8f357fa640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 0x7f8f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:11.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.688+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8f200ac220 con 0x7f8f3815f2b0 2024-04-26T21:14:11.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.689+0000 7f8f3ca4c640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 0x7f8f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:11.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.689+0000 7f8f3ca4c640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 0x7f8f1806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f30011340 tx=0x7f8f3000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:11.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.693+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f2007a000 con 0x7f8f3815f2b0 2024-04-26T21:14:11.870 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:14:11.869+0000 7f38c63b4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:14:11.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:11.917+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8f381570f0 con 0x7f8f3815f2b0 2024-04-26T21:14:12.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.910+0000 7f8f357fa640 1 -- 172.21.15.40:0/1873840542 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7f8f2007e300 con 0x7f8f3815f2b0 2024-04-26T21:14:12.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 msgr2=0x7f8f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:12.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 0x7f8f1806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f30011340 tx=0x7f8f3000b040 comp rx=0 tx=0).stop 2024-04-26T21:14:12.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 msgr2=0x7f8f3815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f20000c00 tx=0x7f8f200046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 shutdown_connections 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1806c8f0 0x7f8f1806edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f3815c4d0 0x7f8f380b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3815f2b0 0x7f8f3815cde0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 --2- 172.21.15.40:0/1873840542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3806c7e0 0x7f8f3814a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 >> 172.21.15.40:0/1873840542 conn(0x7f8f3806f050 msgr2=0x7f8f380b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 shutdown_connections 2024-04-26T21:14:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:12.913+0000 7f8f3ecb0640 1 -- 172.21.15.40:0/1873840542 wait complete. 2024-04-26T21:14:12.927 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:14:12.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:14:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.037+0000 7fc0fcfc6640 1 Processor -- start 2024-04-26T21:14:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.037+0000 7fc0fcfc6640 1 -- start start 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0fcfc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc0f815f5b0 0x7fc0f80b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0fcfc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0f80b55b0 0x7fc0f80d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0fcfc6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f80b6aa0 con 0x7fc0f815f990 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0fcfc6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f80d3eb0 con 0x7fc0f80b55b0 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0fcfc6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f80d4190 con 0x7fc0f815f5b0 2024-04-26T21:14:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0f80b55b0 0x7fc0f80d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0f80b55b0 0x7fc0f80d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59946/0 (socket says 172.21.15.40:59946) 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 -- 172.21.15.40:0/255304273 learned_addr learned my addr 172.21.15.40:0/255304273 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f659c640 1 --2- 172.21.15.40:0/255304273 >> v2:172.21.15.184:3300/0 conn(0x7fc0f815f5b0 0x7fc0f80b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 -- 172.21.15.40:0/255304273 >> v2:172.21.15.184:3300/0 conn(0x7fc0f815f5b0 msgr2=0x7fc0f80b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 --2- 172.21.15.40:0/255304273 >> v2:172.21.15.184:3300/0 conn(0x7fc0f815f5b0 0x7fc0f80b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 -- 172.21.15.40:0/255304273 >> v1:172.21.15.184:6789/0 conn(0x7fc0f815f990 legacy=0x7fc0f80d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.038+0000 7fc0f6d9d640 1 -- 172.21.15.40:0/255304273 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f80d0200 con 0x7fc0f80b55b0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.039+0000 7fc0f6d9d640 1 --2- 172.21.15.40:0/255304273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f80b55b0 0x7fc0f80d3340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0f808f0f0 tx=0x7fc0f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83a82b0d1cd7664 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.039+0000 7fc0f559a640 1 -- 172.21.15.40:0/255304273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0036070 con 0x7fc0f80b55b0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.039+0000 7fc0f559a640 1 -- 172.21.15.40:0/255304273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f0031600 con 0x7fc0f80b55b0 2024-04-26T21:14:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.039+0000 7fc0f559a640 1 -- 172.21.15.40:0/255304273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00319d0 con 0x7fc0f80b55b0 2024-04-26T21:14:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/255304273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f80b55b0 msgr2=0x7fc0f80d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/255304273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f80b55b0 0x7fc0f80d3340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0f808f0f0 tx=0x7fc0f002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/255304273 shutdown_connections 2024-04-26T21:14:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/255304273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f80b55b0 0x7fc0f80d3340 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/255304273 >> v2:172.21.15.184:3300/0 conn(0x7fc0f815f5b0 0x7fc0f80b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/255304273 >> 172.21.15.40:0/255304273 conn(0x7fc0f808ec40 msgr2=0x7fc0f809e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/255304273 shutdown_connections 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.040+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/255304273 wait complete. 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.041+0000 7fc0fcfc6640 1 Processor -- start 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.041+0000 7fc0fcfc6640 1 -- start start 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80b55b0 0x7fc0f8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f815f990 0x7fc0f815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f80d1290 con 0x7fc0f815f5b0 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f80d1110 con 0x7fc0f80b55b0 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0fcfc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f80d3880 con 0x7fc0f815f990 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59948/0 (socket says 172.21.15.40:59948) 2024-04-26T21:14:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 -- 172.21.15.40:0/1508238768 learned_addr learned my addr 172.21.15.40:0/1508238768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f659c640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80b55b0 0x7fc0f8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f6d9d640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f815f990 0x7fc0f815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 -- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f815f990 msgr2=0x7fc0f815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f815f990 0x7fc0f815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 -- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80b55b0 msgr2=0x7fc0f8156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80b55b0 0x7fc0f8156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.042+0000 7fc0f5d9b640 1 -- 172.21.15.40:0/1508238768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f8160b00 con 0x7fc0f815f5b0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0f5d9b640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c3c0 tx=0x7fc0ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0052d0 con 0x7fc0f815f5b0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ec005490 con 0x7fc0f815f5b0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f8160de0 con 0x7fc0f815f5b0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec007260 con 0x7fc0f815f5b0 2024-04-26T21:14:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.043+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f80cafc0 con 0x7fc0f815f5b0 2024-04-26T21:14:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.044+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f806a290 con 0x7fc0f815f5b0 2024-04-26T21:14:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.044+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0ec007400 con 0x7fc0f815f5b0 2024-04-26T21:14:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.045+0000 7fc0e6ffd640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 0x7fc0d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.045+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc0ec08af70 con 0x7fc0f815f5b0 2024-04-26T21:14:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.045+0000 7fc0f659c640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 0x7fc0d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.046+0000 7fc0f659c640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 0x7fc0d806ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80d2400 tx=0x7fc0e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.049+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0ec059430 con 0x7fc0f815f5b0 2024-04-26T21:14:13.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.275+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc0f80d1e00 con 0x7fc0f815f5b0 2024-04-26T21:14:13.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.928+0000 7fc0e6ffd640 1 -- 172.21.15.40:0/1508238768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fc0ec058d30 con 0x7fc0f815f5b0 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 msgr2=0x7fc0d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 0x7fc0d806ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80d2400 tx=0x7fc0e8008040 comp rx=0 tx=0).stop 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 msgr2=0x7fc0f815d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c3c0 tx=0x7fc0ec00c880 comp rx=0 tx=0).stop 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 shutdown_connections 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0d806c920 0x7fc0d806ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f815f990 0x7fc0f815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f815f5b0 0x7fc0f815d580 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 --2- 172.21.15.40:0/1508238768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80b55b0 0x7fc0f8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 >> 172.21.15.40:0/1508238768 conn(0x7fc0f808ec40 msgr2=0x7fc0f80b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 shutdown_connections 2024-04-26T21:14:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:13.931+0000 7fc0fcfc6640 1 -- 172.21.15.40:0/1508238768 wait complete. 2024-04-26T21:14:13.945 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:14:13.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:14.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.051+0000 7f58490c8640 1 Processor -- start 2024-04-26T21:14:14.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.051+0000 7f58490c8640 1 -- start start 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 0x7f584406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58440d11b0 0x7f584406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 0x7f584415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f584406e750 con 0x7f58440d11b0 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f58440d1740 con 0x7f584406e2c0 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f58490c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58440d1a20 con 0x7f58440cf2e0 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58440d11b0 0x7f584406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58440d11b0 0x7f584406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59964/0 (socket says 172.21.15.40:59964) 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 -- 172.21.15.40:0/2132692202 learned_addr learned my addr 172.21.15.40:0/2132692202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f5842d9d640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 0x7f584406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f584359e640 1 --2- 172.21.15.40:0/2132692202 >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 0x7f584415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 -- 172.21.15.40:0/2132692202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 msgr2=0x7f584406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 0x7f584406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 -- 172.21.15.40:0/2132692202 >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 msgr2=0x7f584415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 --2- 172.21.15.40:0/2132692202 >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 0x7f584415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f583bfff640 1 -- 172.21.15.40:0/2132692202 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440d1d00 con 0x7f58440d11b0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f5842d9d640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 0x7f584406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.052+0000 7f584359e640 1 --2- 172.21.15.40:0/2132692202 >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 0x7f584415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.053+0000 7f583bfff640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440d11b0 0x7f584406dd80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5844163ff0 tx=0x7f582c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499abebe7657f0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.053+0000 7f584259c640 1 -- 172.21.15.40:0/2132692202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c037070 con 0x7f58440d11b0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.053+0000 7f584259c640 1 -- 172.21.15.40:0/2132692202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c032440 con 0x7f58440d11b0 2024-04-26T21:14:14.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.053+0000 7f584259c640 1 -- 172.21.15.40:0/2132692202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c032810 con 0x7f58440d11b0 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 -- 172.21.15.40:0/2132692202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440d11b0 msgr2=0x7f584406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440d11b0 0x7f584406dd80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5844163ff0 tx=0x7f582c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 -- 172.21.15.40:0/2132692202 shutdown_connections 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 --2- 172.21.15.40:0/2132692202 >> v2:172.21.15.184:3300/0 conn(0x7f584406e2c0 0x7f584415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440d11b0 0x7f584406dd80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 --2- 172.21.15.40:0/2132692202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440cf2e0 0x7f584406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 -- 172.21.15.40:0/2132692202 >> 172.21.15.40:0/2132692202 conn(0x7f58440b4a60 msgr2=0x7f58440b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 -- 172.21.15.40:0/2132692202 shutdown_connections 2024-04-26T21:14:14.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.054+0000 7f58490c8640 1 -- 172.21.15.40:0/2132692202 wait complete. 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.055+0000 7f58490c8640 1 Processor -- start 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.055+0000 7f58490c8640 1 -- start start 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f584406e2c0 0x7f58441602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58440d11b0 0x7f5844156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f584406da10 con 0x7f58440cf2e0 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58440d22c0 con 0x7f58440d11b0 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f58490c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58440cf6c0 con 0x7f584406e2c0 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59970/0 (socket says 172.21.15.40:59970) 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 -- 172.21.15.40:0/4083253232 learned_addr learned my addr 172.21.15.40:0/4083253232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f5842d9d640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f584406e2c0 0x7f58441602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f584359e640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58440d11b0 0x7f5844156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 -- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f584406e2c0 msgr2=0x7f58441602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f584406e2c0 0x7f58441602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 -- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58440d11b0 msgr2=0x7f5844156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58440d11b0 0x7f5844156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 -- 172.21.15.40:0/4083253232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584409f650 con 0x7f58440cf2e0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.056+0000 7f583bfff640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f582c0090b0 tx=0x7f582c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.057+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c041070 con 0x7f58440cf2e0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.057+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c03c890 con 0x7f58440cf2e0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.057+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584409f930 con 0x7f58440cf2e0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.057+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c037040 con 0x7f58440cf2e0 2024-04-26T21:14:14.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.057+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584409fef0 con 0x7f58440cf2e0 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.058+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584406a290 con 0x7f58440cf2e0 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.058+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f582c044070 con 0x7f58440cf2e0 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.059+0000 7f583b7fe640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 0x7f582406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.059+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f582c0ac870 con 0x7f58440cf2e0 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.059+0000 7f5842d9d640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 0x7f582406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.060+0000 7f5842d9d640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 0x7f582406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5834006e80 tx=0x7f5834008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.063+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f582c07a6d0 con 0x7f58440cf2e0 2024-04-26T21:14:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.284+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58441625d0 con 0x7f58440cf2e0 2024-04-26T21:14:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.285+0000 7f583b7fe640 1 -- 172.21.15.40:0/4083253232 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7f582c07e9d0 con 0x7f58440cf2e0 2024-04-26T21:14:14.286 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:14.286 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 msgr2=0x7f582406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 0x7f582406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5834006e80 tx=0x7f5834008040 comp rx=0 tx=0).stop 2024-04-26T21:14:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 msgr2=0x7f584409ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f582c0090b0 tx=0x7f582c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 shutdown_connections 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f582406c8f0 0x7f582406edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58440d11b0 0x7f5844156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58440cf2e0 0x7f584409ed70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 --2- 172.21.15.40:0/4083253232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f584406e2c0 0x7f58441602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 >> 172.21.15.40:0/4083253232 conn(0x7f58440b4a60 msgr2=0x7f58441611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 shutdown_connections 2024-04-26T21:14:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.287+0000 7f58490c8640 1 -- 172.21.15.40:0/4083253232 wait complete. 2024-04-26T21:14:14.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:14.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.398+0000 7f5810514640 1 Processor -- start 2024-04-26T21:14:14.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.398+0000 7f5810514640 1 -- start start 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f5810514640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f580815f340 0x7f58080b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f5810514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58080d5610 0x7f58080d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f5810514640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080b2510 con 0x7f580815f720 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f5810514640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58080b27f0 con 0x7f580815f340 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f5810514640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58080b2ad0 con 0x7f58080d5610 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f580e2b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f580815f340 0x7f58080b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f580e2b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f580815f340 0x7f58080b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59986/0 (socket says 172.21.15.40:59986) 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.399+0000 7f580e2b0640 1 -- 172.21.15.40:0/3851989104 learned_addr learned my addr 172.21.15.40:0/3851989104 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580eab1640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58080d5610 0x7f58080d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580e2b0640 1 -- 172.21.15.40:0/3851989104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58080d5610 msgr2=0x7f58080d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580e2b0640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58080d5610 0x7f58080d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580e2b0640 1 -- 172.21.15.40:0/3851989104 >> v1:172.21.15.40:6789/0 conn(0x7f580815f720 legacy=0x7f58080d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580e2b0640 1 -- 172.21.15.40:0/3851989104 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080b2db0 con 0x7f580815f340 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580e2b0640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080b1e80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24af44504ad42bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.400+0000 7f580d2ae640 1 -- 172.21.15.40:0/3851989104 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804037070 con 0x7f580815f340 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f580d2ae640 1 -- 172.21.15.40:0/3851989104 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804032440 con 0x7f580815f340 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f580d2ae640 1 -- 172.21.15.40:0/3851989104 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804032810 con 0x7f580815f340 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 -- 172.21.15.40:0/3851989104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 msgr2=0x7f58080b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080b1e80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 -- 172.21.15.40:0/3851989104 shutdown_connections 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58080d5610 0x7f58080d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 --2- 172.21.15.40:0/3851989104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080b1e80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 -- 172.21.15.40:0/3851989104 >> 172.21.15.40:0/3851989104 conn(0x7f580808ec70 msgr2=0x7f580809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 -- 172.21.15.40:0/3851989104 shutdown_connections 2024-04-26T21:14:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.401+0000 7f5810514640 1 -- 172.21.15.40:0/3851989104 wait complete. 2024-04-26T21:14:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.402+0000 7f5810514640 1 Processor -- start 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.402+0000 7f5810514640 1 -- start start 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 0x7f58081fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f580815f720 0x7f5808159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58080b3940 con 0x7f580815f340 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58080b37c0 con 0x7f58080d5610 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f5810514640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58080b3ac0 con 0x7f580815f720 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580daaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580daaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59996/0 (socket says 172.21.15.40:59996) 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580daaf640 1 -- 172.21.15.40:0/2093441747 learned_addr learned my addr 172.21.15.40:0/2093441747 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580e2b0640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 0x7f58081fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580eab1640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f580815f720 0x7f5808159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580daaf640 1 -- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f580815f720 msgr2=0x7f5808159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.403+0000 7f580daaf640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f580815f720 0x7f5808159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f580daaf640 1 -- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 msgr2=0x7f58081fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f580daaf640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 0x7f58081fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f580daaf640 1 -- 172.21.15.40:0/2093441747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080c7e50 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f580e2b0640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 0x7f58081fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f580daaf640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c3c0 tx=0x7f57f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8017070 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f80052f0 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003830 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58080c8130 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.404+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580806d0b0 con 0x7f580815f340 2024-04-26T21:14:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.405+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580806a290 con 0x7f580815f340 2024-04-26T21:14:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.405+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57f801a070 con 0x7f580815f340 2024-04-26T21:14:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.406+0000 7f57feffd640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.406+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f57f808ae40 con 0x7f580815f340 2024-04-26T21:14:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.406+0000 7f580e2b0640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.407+0000 7f580e2b0640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5804034000 tx=0x7f580402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.410+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57f8059280 con 0x7f580815f340 2024-04-26T21:14:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.594+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58080d3020 con 0x7f580815f340 2024-04-26T21:14:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.595+0000 7f57feffd640 1 -- 172.21.15.40:0/2093441747 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7f57f8058b80 con 0x7f580815f340 2024-04-26T21:14:14.596 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:14.597 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 msgr2=0x7f57ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 0x7f57ec06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5804034000 tx=0x7f580402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 msgr2=0x7f58080ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c3c0 tx=0x7f57f800c880 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 shutdown_connections 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57ec06c8f0 0x7f57ec06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f580815f720 0x7f5808159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f580815f340 0x7f58080ccc80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 --2- 172.21.15.40:0/2093441747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58080d5610 0x7f58081fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 >> 172.21.15.40:0/2093441747 conn(0x7f580808ec70 msgr2=0x7f58080b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 shutdown_connections 2024-04-26T21:14:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.597+0000 7f5810514640 1 -- 172.21.15.40:0/2093441747 wait complete. 2024-04-26T21:14:14.609 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:14:14.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:14.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.709+0000 7ff8cb672640 1 Processor -- start 2024-04-26T21:14:14.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.709+0000 7ff8cb672640 1 -- start start 2024-04-26T21:14:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8cb672640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff8c415b4e0 0x7ff8c4071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8cb672640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c40733e0 con 0x7ff8c406f2c0 2024-04-26T21:14:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8cb672640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c4072f00 con 0x7ff8c415b8c0 2024-04-26T21:14:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8cb672640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c4072660 con 0x7ff8c415b4e0 2024-04-26T21:14:14.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c9c0f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff8c406f2c0 0x7ff8c40b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34902/0 (socket says 172.21.15.40:34902) 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c9c0f640 1 -- 172.21.15.40:0/2161439562 learned_addr learned my addr 172.21.15.40:0/2161439562 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c940e640 1 --2- 172.21.15.40:0/2161439562 >> v2:172.21.15.184:3300/0 conn(0x7ff8c415b4e0 0x7ff8c4071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2722073587 0 0) 0x7ff8c40733e0 con 0x7ff8c406f2c0 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8b0003520 con 0x7ff8c406f2c0 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c940e640 1 -- 172.21.15.40:0/2161439562 >> v1:172.21.15.184:6789/0 conn(0x7ff8c415b8c0 legacy=0x7ff8c406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913841595 0 0) 0x7ff8c4072f00 con 0x7ff8c415b8c0 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c940e640 1 -- 172.21.15.40:0/2161439562 >> v1:172.21.15.40:6789/0 conn(0x7ff8c406f2c0 legacy=0x7ff8c40b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.710+0000 7ff8c940e640 1 -- 172.21.15.40:0/2161439562 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c4072940 con 0x7ff8c415b4e0 2024-04-26T21:14:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.711+0000 7ff8c940e640 1 --2- 172.21.15.40:0/2161439562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c415b4e0 0x7ff8c4071fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00a940 tx=0x7ff8bc02b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbd270317a48422 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.711+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0001970 con 0x7ff8c415b4e0 2024-04-26T21:14:14.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.711+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc030510 con 0x7ff8c415b4e0 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.711+0000 7ff8bbfff640 1 -- 172.21.15.40:0/2161439562 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00b040 con 0x7ff8c415b4e0 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 -- 172.21.15.40:0/2161439562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c415b4e0 msgr2=0x7ff8c4071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2161439562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c415b4e0 0x7ff8c4071fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00a940 tx=0x7ff8bc02b4d0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 -- 172.21.15.40:0/2161439562 shutdown_connections 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2161439562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c415b4e0 0x7ff8c4071fd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 -- 172.21.15.40:0/2161439562 >> 172.21.15.40:0/2161439562 conn(0x7ff8c40d0d30 msgr2=0x7ff8c40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 -- 172.21.15.40:0/2161439562 shutdown_connections 2024-04-26T21:14:14.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.712+0000 7ff8cb672640 1 -- 172.21.15.40:0/2161439562 wait complete. 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.713+0000 7ff8cb672640 1 Processor -- start 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.713+0000 7ff8cb672640 1 -- start start 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c406f2c0 0x7ff8c414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c415b4e0 0x7ff8c40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4073820 con 0x7ff8c415b8c0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40b3e80 con 0x7ff8c406f2c0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8cb672640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c40739a0 con 0x7ff8c415b4e0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59998/0 (socket says 172.21.15.40:59998) 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 -- 172.21.15.40:0/2301301435 learned_addr learned my addr 172.21.15.40:0/2301301435 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c940e640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c406f2c0 0x7ff8c414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c8c0d640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c415b4e0 0x7ff8c40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 -- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c415b4e0 msgr2=0x7ff8c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c415b4e0 0x7ff8c40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 -- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c406f2c0 msgr2=0x7ff8c414daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c406f2c0 0x7ff8c414daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.714+0000 7ff8c9c0f640 1 -- 172.21.15.40:0/2301301435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c4158450 con 0x7ff8c415b8c0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8c9c0f640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000cac0 tx=0x7ff8c000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0018070 con 0x7ff8c415b8c0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c000a700 con 0x7ff8c415b8c0 2024-04-26T21:14:14.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c40ce7a0 con 0x7ff8c415b8c0 2024-04-26T21:14:14.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000aab0 con 0x7ff8c415b8c0 2024-04-26T21:14:14.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.715+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c4158730 con 0x7ff8c415b8c0 2024-04-26T21:14:14.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.716+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c406a290 con 0x7ff8c415b8c0 2024-04-26T21:14:14.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.716+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8c001b070 con 0x7ff8c415b8c0 2024-04-26T21:14:14.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.717+0000 7ff8b9ffb640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 0x7ff8a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.717+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff8c008b430 con 0x7ff8c415b8c0 2024-04-26T21:14:14.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.717+0000 7ff8c940e640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 0x7ff8a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.717+0000 7ff8c940e640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 0x7ff8a806edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc035000 tx=0x7ff8bc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.721+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8c0059210 con 0x7ff8c415b8c0 2024-04-26T21:14:14.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.903+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8c40b2d10 con 0x7ff8a806c8f0 2024-04-26T21:14:14.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.910+0000 7ff8b9ffb640 1 -- 172.21.15.40:0/2301301435 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249871 (secure 0 0 0) 0x7ff8c40b2d10 con 0x7ff8a806c8f0 2024-04-26T21:14:14.914 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:14.915 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 msgr2=0x7ff8a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 0x7ff8a806edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc035000 tx=0x7ff8bc030000 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 msgr2=0x7ff8c4143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000cac0 tx=0x7ff8c000cf80 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 shutdown_connections 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff8a806c8f0 0x7ff8a806edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8c415b8c0 0x7ff8c4143840 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c415b4e0 0x7ff8c40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 --2- 172.21.15.40:0/2301301435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c406f2c0 0x7ff8c414daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.913+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 >> 172.21.15.40:0/2301301435 conn(0x7ff8c40d0d30 msgr2=0x7ff8c40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.914+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 shutdown_connections 2024-04-26T21:14:14.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:14.914+0000 7ff8cb672640 1 -- 172.21.15.40:0/2301301435 wait complete. 2024-04-26T21:14:14.925 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-04-26T21:14:12.949021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234032,"kb_used_data":5372,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740688,"statfs":{"total":773094113280,"available":772854464512,"internally_reserved":0,"allocated":5500928,"data_stored":2896181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.061331"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913490+0000","last_change":"2024-04-26T21:14:10.939188+0000","last_active":"2024-04-26T21:14:11.913490+0000","last_peered":"2024-04-26T21:14:11.913490+0000","last_clean":"2024-04-26T21:14:11.913490+0000","last_became_active":"2024-04-26T21:14:10.938935+0000","last_became_peered":"2024-04-26T21:14:10.938935+0000","last_unstale":"2024-04-26T21:14:11.913490+0000","last_undegraded":"2024-04-26T21:14:11.913490+0000","last_fullsized":"2024-04-26T21:14:11.913490+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914266+0000","last_change":"2024-04-26T21:14:10.944548+0000","last_active":"2024-04-26T21:14:11.914266+0000","last_peered":"2024-04-26T21:14:11.914266+0000","last_clean":"2024-04-26T21:14:11.914266+0000","last_became_active":"2024-04-26T21:14:10.944248+0000","last_became_peered":"2024-04-26T21:14:10.944248+0000","last_unstale":"2024-04-26T21:14:11.914266+0000","last_undegraded":"2024-04-26T21:14:11.914266+0000","last_fullsized":"2024-04-26T21:14:11.914266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:46.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915895+0000","last_change":"2024-04-26T21:14:10.917770+0000","last_active":"2024-04-26T21:14:11.915895+0000","last_peered":"2024-04-26T21:14:11.915895+0000","last_clean":"2024-04-26T21:14:11.915895+0000","last_became_active":"2024-04-26T21:14:10.917473+0000","last_became_peered":"2024-04-26T21:14:10.917473+0000","last_unstale":"2024-04-26T21:14:11.915895+0000","last_undegraded":"2024-04-26T21:14:11.915895+0000","last_fullsized":"2024-04-26T21:14:11.915895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:22.505944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684737+0000","last_change":"2024-04-26T21:14:10.935906+0000","last_active":"2024-04-26T21:14:12.684737+0000","last_peered":"2024-04-26T21:14:12.684737+0000","last_clean":"2024-04-26T21:14:12.684737+0000","last_became_active":"2024-04-26T21:14:10.935600+0000","last_became_peered":"2024-04-26T21:14:10.935600+0000","last_unstale":"2024-04-26T21:14:12.684737+0000","last_undegraded":"2024-04-26T21:14:12.684737+0000","last_fullsized":"2024-04-26T21:14:12.684737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:48.056421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913179+0000","last_change":"2024-04-26T21:14:10.943004+0000","last_active":"2024-04-26T21:14:11.913179+0000","last_peered":"2024-04-26T21:14:11.913179+0000","last_clean":"2024-04-26T21:14:11.913179+0000","last_became_active":"2024-04-26T21:14:10.942668+0000","last_became_peered":"2024-04-26T21:14:10.942668+0000","last_unstale":"2024-04-26T21:14:11.913179+0000","last_undegraded":"2024-04-26T21:14:11.913179+0000","last_fullsized":"2024-04-26T21:14:11.913179+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:46.122041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912722+0000","last_change":"2024-04-26T21:14:10.934097+0000","last_active":"2024-04-26T21:14:11.912722+0000","last_peered":"2024-04-26T21:14:11.912722+0000","last_clean":"2024-04-26T21:14:11.912722+0000","last_became_active":"2024-04-26T21:14:10.933814+0000","last_became_peered":"2024-04-26T21:14:10.933814+0000","last_unstale":"2024-04-26T21:14:11.912722+0000","last_undegraded":"2024-04-26T21:14:11.912722+0000","last_fullsized":"2024-04-26T21:14:11.912722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:53.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913641+0000","last_change":"2024-04-26T21:14:10.946950+0000","last_active":"2024-04-26T21:14:11.913641+0000","last_peered":"2024-04-26T21:14:11.913641+0000","last_clean":"2024-04-26T21:14:11.913641+0000","last_became_active":"2024-04-26T21:14:10.946628+0000","last_became_peered":"2024-04-26T21:14:10.946628+0000","last_unstale":"2024-04-26T21:14:11.913641+0000","last_undegraded":"2024-04-26T21:14:11.913641+0000","last_fullsized":"2024-04-26T21:14:11.913641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684224+0000","last_change":"2024-04-26T21:14:10.943471+0000","last_active":"2024-04-26T21:14:12.684224+0000","last_peered":"2024-04-26T21:14:12.684224+0000","last_clean":"2024-04-26T21:14:12.684224+0000","last_became_active":"2024-04-26T21:14:10.942879+0000","last_became_peered":"2024-04-26T21:14:10.942879+0000","last_unstale":"2024-04-26T21:14:12.684224+0000","last_undegraded":"2024-04-26T21:14:12.684224+0000","last_fullsized":"2024-04-26T21:14:12.684224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:47.754614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685292+0000","last_change":"2024-04-26T21:14:10.941189+0000","last_active":"2024-04-26T21:14:12.685292+0000","last_peered":"2024-04-26T21:14:12.685292+0000","last_clean":"2024-04-26T21:14:12.685292+0000","last_became_active":"2024-04-26T21:14:10.940477+0000","last_became_peered":"2024-04-26T21:14:10.940477+0000","last_unstale":"2024-04-26T21:14:12.685292+0000","last_undegraded":"2024-04-26T21:14:12.685292+0000","last_fullsized":"2024-04-26T21:14:12.685292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:46.122041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685400+0000","last_change":"2024-04-26T21:14:10.933740+0000","last_active":"2024-04-26T21:14:12.685400+0000","last_peered":"2024-04-26T21:14:12.685400+0000","last_clean":"2024-04-26T21:14:12.685400+0000","last_became_active":"2024-04-26T21:14:10.933419+0000","last_became_peered":"2024-04-26T21:14:10.933419+0000","last_unstale":"2024-04-26T21:14:12.685400+0000","last_undegraded":"2024-04-26T21:14:12.685400+0000","last_fullsized":"2024-04-26T21:14:12.685400+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685154+0000","last_change":"2024-04-26T21:14:10.942468+0000","last_active":"2024-04-26T21:14:12.685154+0000","last_peered":"2024-04-26T21:14:12.685154+0000","last_clean":"2024-04-26T21:14:12.685154+0000","last_became_active":"2024-04-26T21:14:10.942182+0000","last_became_peered":"2024-04-26T21:14:10.942182+0000","last_unstale":"2024-04-26T21:14:12.685154+0000","last_undegraded":"2024-04-26T21:14:12.685154+0000","last_fullsized":"2024-04-26T21:14:12.685154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.680017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913413+0000","last_change":"2024-04-26T21:14:10.921134+0000","last_active":"2024-04-26T21:14:11.913413+0000","last_peered":"2024-04-26T21:14:11.913413+0000","last_clean":"2024-04-26T21:14:11.913413+0000","last_became_active":"2024-04-26T21:14:10.920642+0000","last_became_peered":"2024-04-26T21:14:10.920642+0000","last_unstale":"2024-04-26T21:14:11.913413+0000","last_undegraded":"2024-04-26T21:14:11.913413+0000","last_fullsized":"2024-04-26T21:14:11.913413+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:39.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911996+0000","last_change":"2024-04-26T21:14:10.930737+0000","last_active":"2024-04-26T21:14:11.911996+0000","last_peered":"2024-04-26T21:14:11.911996+0000","last_clean":"2024-04-26T21:14:11.911996+0000","last_became_active":"2024-04-26T21:14:10.930435+0000","last_became_peered":"2024-04-26T21:14:10.930435+0000","last_unstale":"2024-04-26T21:14:11.911996+0000","last_undegraded":"2024-04-26T21:14:11.911996+0000","last_fullsized":"2024-04-26T21:14:11.911996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:17.093360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684439+0000","last_change":"2024-04-26T21:14:10.947299+0000","last_active":"2024-04-26T21:14:12.684439+0000","last_peered":"2024-04-26T21:14:12.684439+0000","last_clean":"2024-04-26T21:14:12.684439+0000","last_became_active":"2024-04-26T21:14:10.946866+0000","last_became_peered":"2024-04-26T21:14:10.946866+0000","last_unstale":"2024-04-26T21:14:12.684439+0000","last_undegraded":"2024-04-26T21:14:12.684439+0000","last_fullsized":"2024-04-26T21:14:12.684439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:04.756658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912621+0000","last_change":"2024-04-26T21:14:10.939154+0000","last_active":"2024-04-26T21:14:11.912621+0000","last_peered":"2024-04-26T21:14:11.912621+0000","last_clean":"2024-04-26T21:14:11.912621+0000","last_became_active":"2024-04-26T21:14:10.938864+0000","last_became_peered":"2024-04-26T21:14:10.938864+0000","last_unstale":"2024-04-26T21:14:11.912621+0000","last_undegraded":"2024-04-26T21:14:11.912621+0000","last_fullsized":"2024-04-26T21:14:11.912621+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:40.789418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916102+0000","last_change":"2024-04-26T21:14:10.919596+0000","last_active":"2024-04-26T21:14:11.916102+0000","last_peered":"2024-04-26T21:14:11.916102+0000","last_clean":"2024-04-26T21:14:11.916102+0000","last_became_active":"2024-04-26T21:14:10.919235+0000","last_became_peered":"2024-04-26T21:14:10.919235+0000","last_unstale":"2024-04-26T21:14:11.916102+0000","last_undegraded":"2024-04-26T21:14:11.916102+0000","last_fullsized":"2024-04-26T21:14:11.916102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.304369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915952+0000","last_change":"2024-04-26T21:14:10.945274+0000","last_active":"2024-04-26T21:14:11.915952+0000","last_peered":"2024-04-26T21:14:11.915952+0000","last_clean":"2024-04-26T21:14:11.915952+0000","last_became_active":"2024-04-26T21:14:10.944969+0000","last_became_peered":"2024-04-26T21:14:10.944969+0000","last_unstale":"2024-04-26T21:14:11.915952+0000","last_undegraded":"2024-04-26T21:14:11.915952+0000","last_fullsized":"2024-04-26T21:14:11.915952+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:54.218317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911727+0000","last_change":"2024-04-26T21:14:10.942738+0000","last_active":"2024-04-26T21:14:11.911727+0000","last_peered":"2024-04-26T21:14:11.911727+0000","last_clean":"2024-04-26T21:14:11.911727+0000","last_became_active":"2024-04-26T21:14:10.942258+0000","last_became_peered":"2024-04-26T21:14:10.942258+0000","last_unstale":"2024-04-26T21:14:11.911727+0000","last_undegraded":"2024-04-26T21:14:11.911727+0000","last_fullsized":"2024-04-26T21:14:11.911727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:36.641758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912818+0000","last_change":"2024-04-26T21:14:10.940649+0000","last_active":"2024-04-26T21:14:11.912818+0000","last_peered":"2024-04-26T21:14:11.912818+0000","last_clean":"2024-04-26T21:14:11.912818+0000","last_became_active":"2024-04-26T21:14:10.939744+0000","last_became_peered":"2024-04-26T21:14:10.939744+0000","last_unstale":"2024-04-26T21:14:11.912818+0000","last_undegraded":"2024-04-26T21:14:11.912818+0000","last_fullsized":"2024-04-26T21:14:11.912818+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:08.899304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684462+0000","last_change":"2024-04-26T21:14:10.938159+0000","last_active":"2024-04-26T21:14:12.684462+0000","last_peered":"2024-04-26T21:14:12.684462+0000","last_clean":"2024-04-26T21:14:12.684462+0000","last_became_active":"2024-04-26T21:14:10.937658+0000","last_became_peered":"2024-04-26T21:14:10.937658+0000","last_unstale":"2024-04-26T21:14:12.684462+0000","last_undegraded":"2024-04-26T21:14:12.684462+0000","last_fullsized":"2024-04-26T21:14:12.684462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:22.505944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911618+0000","last_change":"2024-04-26T21:14:10.936935+0000","last_active":"2024-04-26T21:14:11.911618+0000","last_peered":"2024-04-26T21:14:11.911618+0000","last_clean":"2024-04-26T21:14:11.911618+0000","last_became_active":"2024-04-26T21:14:10.936588+0000","last_became_peered":"2024-04-26T21:14:10.936588+0000","last_unstale":"2024-04-26T21:14:11.911618+0000","last_undegraded":"2024-04-26T21:14:11.911618+0000","last_fullsized":"2024-04-26T21:14:11.911618+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:24.974256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916006+0000","last_change":"2024-04-26T21:14:10.918939+0000","last_active":"2024-04-26T21:14:11.916006+0000","last_peered":"2024-04-26T21:14:11.916006+0000","last_clean":"2024-04-26T21:14:11.916006+0000","last_became_active":"2024-04-26T21:14:10.918632+0000","last_became_peered":"2024-04-26T21:14:10.918632+0000","last_unstale":"2024-04-26T21:14:11.916006+0000","last_undegraded":"2024-04-26T21:14:11.916006+0000","last_fullsized":"2024-04-26T21:14:11.916006+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:48.056421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913499+0000","last_change":"2024-04-26T21:14:10.920909+0000","last_active":"2024-04-26T21:14:11.913499+0000","last_peered":"2024-04-26T21:14:11.913499+0000","last_clean":"2024-04-26T21:14:11.913499+0000","last_became_active":"2024-04-26T21:14:10.920294+0000","last_became_peered":"2024-04-26T21:14:10.920294+0000","last_unstale":"2024-04-26T21:14:11.913499+0000","last_undegraded":"2024-04-26T21:14:11.913499+0000","last_fullsized":"2024-04-26T21:14:11.913499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915771+0000","last_change":"2024-04-26T21:14:10.930302+0000","last_active":"2024-04-26T21:14:11.915771+0000","last_peered":"2024-04-26T21:14:11.915771+0000","last_clean":"2024-04-26T21:14:11.915771+0000","last_became_active":"2024-04-26T21:14:10.930023+0000","last_became_peered":"2024-04-26T21:14:10.930023+0000","last_unstale":"2024-04-26T21:14:11.915771+0000","last_undegraded":"2024-04-26T21:14:11.915771+0000","last_fullsized":"2024-04-26T21:14:11.915771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:19.030796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912704+0000","last_change":"2024-04-26T21:14:10.929723+0000","last_active":"2024-04-26T21:14:11.912704+0000","last_peered":"2024-04-26T21:14:11.912704+0000","last_clean":"2024-04-26T21:14:11.912704+0000","last_became_active":"2024-04-26T21:14:10.929453+0000","last_became_peered":"2024-04-26T21:14:10.929453+0000","last_unstale":"2024-04-26T21:14:11.912704+0000","last_undegraded":"2024-04-26T21:14:11.912704+0000","last_fullsized":"2024-04-26T21:14:11.912704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:05.712074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911573+0000","last_change":"2024-04-26T21:14:10.938091+0000","last_active":"2024-04-26T21:14:11.911573+0000","last_peered":"2024-04-26T21:14:11.911573+0000","last_clean":"2024-04-26T21:14:11.911573+0000","last_became_active":"2024-04-26T21:14:10.937760+0000","last_became_peered":"2024-04-26T21:14:10.937760+0000","last_unstale":"2024-04-26T21:14:11.911573+0000","last_undegraded":"2024-04-26T21:14:11.911573+0000","last_fullsized":"2024-04-26T21:14:11.911573+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:23.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-26T21:14:09.892722+0000","last_change":"2024-04-26T21:14:09.892722+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.892722+0000","last_undegraded":"2024-04-26T21:14:09.892722+0000","last_fullsized":"2024-04-26T21:14:09.892722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913068+0000","last_change":"2024-04-26T21:14:09.909494+0000","last_active":"2024-04-26T21:14:11.913068+0000","last_peered":"2024-04-26T21:14:11.913068+0000","last_clean":"2024-04-26T21:14:11.913068+0000","last_became_active":"2024-04-26T21:14:09.909163+0000","last_became_peered":"2024-04-26T21:14:09.909163+0000","last_unstale":"2024-04-26T21:14:11.913068+0000","last_undegraded":"2024-04-26T21:14:11.913068+0000","last_fullsized":"2024-04-26T21:14:11.913068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916149+0000","last_change":"2024-04-26T21:14:09.907581+0000","last_active":"2024-04-26T21:14:11.916149+0000","last_peered":"2024-04-26T21:14:11.916149+0000","last_clean":"2024-04-26T21:14:11.916149+0000","last_became_active":"2024-04-26T21:14:09.907319+0000","last_became_peered":"2024-04-26T21:14:09.907319+0000","last_unstale":"2024-04-26T21:14:11.916149+0000","last_undegraded":"2024-04-26T21:14:11.916149+0000","last_fullsized":"2024-04-26T21:14:11.916149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:06.104049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913576+0000","last_change":"2024-04-26T21:14:10.931029+0000","last_active":"2024-04-26T21:14:11.913576+0000","last_peered":"2024-04-26T21:14:11.913576+0000","last_clean":"2024-04-26T21:14:11.913576+0000","last_became_active":"2024-04-26T21:14:10.930755+0000","last_became_peered":"2024-04-26T21:14:10.930755+0000","last_unstale":"2024-04-26T21:14:11.913576+0000","last_undegraded":"2024-04-26T21:14:11.913576+0000","last_fullsized":"2024-04-26T21:14:11.913576+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:14:09.909187+0000","last_change":"2024-04-26T21:14:09.908272+0000","last_active":"2024-04-26T21:14:09.909187+0000","last_peered":"2024-04-26T21:14:09.909187+0000","last_clean":"2024-04-26T21:14:09.909187+0000","last_became_active":"2024-04-26T21:14:09.907945+0000","last_became_peered":"2024-04-26T21:14:09.907945+0000","last_unstale":"2024-04-26T21:14:09.909187+0000","last_undegraded":"2024-04-26T21:14:09.909187+0000","last_fullsized":"2024-04-26T21:14:09.909187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:52.561315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-26T21:14:09.893089+0000","last_change":"2024-04-26T21:14:09.893089+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.893089+0000","last_undegraded":"2024-04-26T21:14:09.893089+0000","last_fullsized":"2024-04-26T21:14:09.893089+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912924+0000","last_change":"2024-04-26T21:14:09.913677+0000","last_active":"2024-04-26T21:14:11.912924+0000","last_peered":"2024-04-26T21:14:11.912924+0000","last_clean":"2024-04-26T21:14:11.912924+0000","last_became_active":"2024-04-26T21:14:09.913263+0000","last_became_peered":"2024-04-26T21:14:09.913263+0000","last_unstale":"2024-04-26T21:14:11.912924+0000","last_undegraded":"2024-04-26T21:14:11.912924+0000","last_fullsized":"2024-04-26T21:14:11.912924+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:21.504417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915820+0000","last_change":"2024-04-26T21:14:10.925252+0000","last_active":"2024-04-26T21:14:11.915820+0000","last_peered":"2024-04-26T21:14:11.915820+0000","last_clean":"2024-04-26T21:14:11.915820+0000","last_became_active":"2024-04-26T21:14:10.924967+0000","last_became_peered":"2024-04-26T21:14:10.924967+0000","last_unstale":"2024-04-26T21:14:11.915820+0000","last_undegraded":"2024-04-26T21:14:11.915820+0000","last_fullsized":"2024-04-26T21:14:11.915820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:04.756658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912981+0000","last_change":"2024-04-26T21:14:09.906360+0000","last_active":"2024-04-26T21:14:11.912981+0000","last_peered":"2024-04-26T21:14:11.912981+0000","last_clean":"2024-04-26T21:14:11.912981+0000","last_became_active":"2024-04-26T21:14:09.906084+0000","last_became_peered":"2024-04-26T21:14:09.906084+0000","last_unstale":"2024-04-26T21:14:11.912981+0000","last_undegraded":"2024-04-26T21:14:11.912981+0000","last_fullsized":"2024-04-26T21:14:11.912981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:31.302842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684713+0000","last_change":"2024-04-26T21:14:10.940653+0000","last_active":"2024-04-26T21:14:12.684713+0000","last_peered":"2024-04-26T21:14:12.684713+0000","last_clean":"2024-04-26T21:14:12.684713+0000","last_became_active":"2024-04-26T21:14:10.940376+0000","last_became_peered":"2024-04-26T21:14:10.940376+0000","last_unstale":"2024-04-26T21:14:12.684713+0000","last_undegraded":"2024-04-26T21:14:12.684713+0000","last_fullsized":"2024-04-26T21:14:12.684713+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:54.218317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915935+0000","last_change":"2024-04-26T21:14:09.907047+0000","last_active":"2024-04-26T21:14:11.915935+0000","last_peered":"2024-04-26T21:14:11.915935+0000","last_clean":"2024-04-26T21:14:11.915935+0000","last_became_active":"2024-04-26T21:14:09.906676+0000","last_became_peered":"2024-04-26T21:14:09.906676+0000","last_unstale":"2024-04-26T21:14:11.915935+0000","last_undegraded":"2024-04-26T21:14:11.915935+0000","last_fullsized":"2024-04-26T21:14:11.915935+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:31.505524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684416+0000","last_change":"2024-04-26T21:14:10.943138+0000","last_active":"2024-04-26T21:14:12.684416+0000","last_peered":"2024-04-26T21:14:12.684416+0000","last_clean":"2024-04-26T21:14:12.684416+0000","last_became_active":"2024-04-26T21:14:10.941274+0000","last_became_peered":"2024-04-26T21:14:10.941274+0000","last_unstale":"2024-04-26T21:14:12.684416+0000","last_undegraded":"2024-04-26T21:14:12.684416+0000","last_fullsized":"2024-04-26T21:14:12.684416+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:19.030796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912994+0000","last_change":"2024-04-26T21:14:09.906600+0000","last_active":"2024-04-26T21:14:11.912994+0000","last_peered":"2024-04-26T21:14:11.912994+0000","last_clean":"2024-04-26T21:14:11.912994+0000","last_became_active":"2024-04-26T21:14:09.906314+0000","last_became_peered":"2024-04-26T21:14:09.906314+0000","last_unstale":"2024-04-26T21:14:11.912994+0000","last_undegraded":"2024-04-26T21:14:11.912994+0000","last_fullsized":"2024-04-26T21:14:11.912994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:15.278713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685099+0000","last_change":"2024-04-26T21:14:09.906801+0000","last_active":"2024-04-26T21:14:12.685099+0000","last_peered":"2024-04-26T21:14:12.685099+0000","last_clean":"2024-04-26T21:14:12.685099+0000","last_became_active":"2024-04-26T21:14:09.906512+0000","last_became_peered":"2024-04-26T21:14:09.906512+0000","last_unstale":"2024-04-26T21:14:12.685099+0000","last_undegraded":"2024-04-26T21:14:12.685099+0000","last_fullsized":"2024-04-26T21:14:12.685099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:16.091833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916003+0000","last_change":"2024-04-26T21:14:10.945343+0000","last_active":"2024-04-26T21:14:11.916003+0000","last_peered":"2024-04-26T21:14:11.916003+0000","last_clean":"2024-04-26T21:14:11.916003+0000","last_became_active":"2024-04-26T21:14:10.945061+0000","last_became_peered":"2024-04-26T21:14:10.945061+0000","last_unstale":"2024-04-26T21:14:11.916003+0000","last_undegraded":"2024-04-26T21:14:11.916003+0000","last_fullsized":"2024-04-26T21:14:11.916003+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:21.179012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916194+0000","last_change":"2024-04-26T21:14:09.902491+0000","last_active":"2024-04-26T21:14:11.916194+0000","last_peered":"2024-04-26T21:14:11.916194+0000","last_clean":"2024-04-26T21:14:11.916194+0000","last_became_active":"2024-04-26T21:14:09.902237+0000","last_became_peered":"2024-04-26T21:14:09.902237+0000","last_unstale":"2024-04-26T21:14:11.916194+0000","last_undegraded":"2024-04-26T21:14:11.916194+0000","last_fullsized":"2024-04-26T21:14:11.916194+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:21.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913893+0000","last_change":"2024-04-26T21:14:10.930600+0000","last_active":"2024-04-26T21:14:11.913893+0000","last_peered":"2024-04-26T21:14:11.913893+0000","last_clean":"2024-04-26T21:14:11.913893+0000","last_became_active":"2024-04-26T21:14:10.930334+0000","last_became_peered":"2024-04-26T21:14:10.930334+0000","last_unstale":"2024-04-26T21:14:11.913893+0000","last_undegraded":"2024-04-26T21:14:11.913893+0000","last_fullsized":"2024-04-26T21:14:11.913893+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:39.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685466+0000","last_change":"2024-04-26T21:14:09.908596+0000","last_active":"2024-04-26T21:14:12.685466+0000","last_peered":"2024-04-26T21:14:12.685466+0000","last_clean":"2024-04-26T21:14:12.685466+0000","last_became_active":"2024-04-26T21:14:09.908271+0000","last_became_peered":"2024-04-26T21:14:09.908271+0000","last_unstale":"2024-04-26T21:14:12.685466+0000","last_undegraded":"2024-04-26T21:14:12.685466+0000","last_fullsized":"2024-04-26T21:14:12.685466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:03.641208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913020+0000","last_change":"2024-04-26T21:14:10.935980+0000","last_active":"2024-04-26T21:14:11.913020+0000","last_peered":"2024-04-26T21:14:11.913020+0000","last_clean":"2024-04-26T21:14:11.913020+0000","last_became_active":"2024-04-26T21:14:10.935666+0000","last_became_peered":"2024-04-26T21:14:10.935666+0000","last_unstale":"2024-04-26T21:14:11.913020+0000","last_undegraded":"2024-04-26T21:14:11.913020+0000","last_fullsized":"2024-04-26T21:14:11.913020+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:30.450958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914202+0000","last_change":"2024-04-26T21:14:09.909632+0000","last_active":"2024-04-26T21:14:11.914202+0000","last_peered":"2024-04-26T21:14:11.914202+0000","last_clean":"2024-04-26T21:14:11.914202+0000","last_became_active":"2024-04-26T21:14:09.909362+0000","last_became_peered":"2024-04-26T21:14:09.909362+0000","last_unstale":"2024-04-26T21:14:11.914202+0000","last_undegraded":"2024-04-26T21:14:11.914202+0000","last_fullsized":"2024-04-26T21:14:11.914202+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:53.216790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914037+0000","last_change":"2024-04-26T21:14:10.948347+0000","last_active":"2024-04-26T21:14:11.914037+0000","last_peered":"2024-04-26T21:14:11.914037+0000","last_clean":"2024-04-26T21:14:11.914037+0000","last_became_active":"2024-04-26T21:14:10.948088+0000","last_became_peered":"2024-04-26T21:14:10.948088+0000","last_unstale":"2024-04-26T21:14:11.914037+0000","last_undegraded":"2024-04-26T21:14:11.914037+0000","last_fullsized":"2024-04-26T21:14:11.914037+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:18.169012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911736+0000","last_change":"2024-04-26T21:14:09.900158+0000","last_active":"2024-04-26T21:14:11.911736+0000","last_peered":"2024-04-26T21:14:11.911736+0000","last_clean":"2024-04-26T21:14:11.911736+0000","last_became_active":"2024-04-26T21:14:09.899864+0000","last_became_peered":"2024-04-26T21:14:09.899864+0000","last_unstale":"2024-04-26T21:14:11.911736+0000","last_undegraded":"2024-04-26T21:14:11.911736+0000","last_fullsized":"2024-04-26T21:14:11.911736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:33:11.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913089+0000","last_change":"2024-04-26T21:14:09.900449+0000","last_active":"2024-04-26T21:14:11.913089+0000","last_peered":"2024-04-26T21:14:11.913089+0000","last_clean":"2024-04-26T21:14:11.913089+0000","last_became_active":"2024-04-26T21:14:09.900117+0000","last_became_peered":"2024-04-26T21:14:09.900117+0000","last_unstale":"2024-04-26T21:14:11.913089+0000","last_undegraded":"2024-04-26T21:14:11.913089+0000","last_fullsized":"2024-04-26T21:14:11.913089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:53.216790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684897+0000","last_change":"2024-04-26T21:14:10.935939+0000","last_active":"2024-04-26T21:14:12.684897+0000","last_peered":"2024-04-26T21:14:12.684897+0000","last_clean":"2024-04-26T21:14:12.684897+0000","last_became_active":"2024-04-26T21:14:10.935593+0000","last_became_peered":"2024-04-26T21:14:10.935593+0000","last_unstale":"2024-04-26T21:14:12.684897+0000","last_undegraded":"2024-04-26T21:14:12.684897+0000","last_fullsized":"2024-04-26T21:14:12.684897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.304369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913122+0000","last_change":"2024-04-26T21:14:09.905800+0000","last_active":"2024-04-26T21:14:11.913122+0000","last_peered":"2024-04-26T21:14:11.913122+0000","last_clean":"2024-04-26T21:14:11.913122+0000","last_became_active":"2024-04-26T21:14:09.905504+0000","last_became_peered":"2024-04-26T21:14:09.905504+0000","last_unstale":"2024-04-26T21:14:11.913122+0000","last_undegraded":"2024-04-26T21:14:11.913122+0000","last_fullsized":"2024-04-26T21:14:11.913122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:46.753087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912613+0000","last_change":"2024-04-26T21:14:10.933158+0000","last_active":"2024-04-26T21:14:11.912613+0000","last_peered":"2024-04-26T21:14:11.912613+0000","last_clean":"2024-04-26T21:14:11.912613+0000","last_became_active":"2024-04-26T21:14:10.932865+0000","last_became_peered":"2024-04-26T21:14:10.932865+0000","last_unstale":"2024-04-26T21:14:11.912613+0000","last_undegraded":"2024-04-26T21:14:11.912613+0000","last_fullsized":"2024-04-26T21:14:11.912613+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:51.619536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912718+0000","last_change":"2024-04-26T21:14:09.898468+0000","last_active":"2024-04-26T21:14:11.912718+0000","last_peered":"2024-04-26T21:14:11.912718+0000","last_clean":"2024-04-26T21:14:11.912718+0000","last_became_active":"2024-04-26T21:14:09.898160+0000","last_became_peered":"2024-04-26T21:14:09.898160+0000","last_unstale":"2024-04-26T21:14:11.912718+0000","last_undegraded":"2024-04-26T21:14:11.912718+0000","last_fullsized":"2024-04-26T21:14:11.912718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:21.504417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913830+0000","last_change":"2024-04-26T21:14:10.928754+0000","last_active":"2024-04-26T21:14:11.913830+0000","last_peered":"2024-04-26T21:14:11.913830+0000","last_clean":"2024-04-26T21:14:11.913830+0000","last_became_active":"2024-04-26T21:14:10.928383+0000","last_became_peered":"2024-04-26T21:14:10.928383+0000","last_unstale":"2024-04-26T21:14:11.913830+0000","last_undegraded":"2024-04-26T21:14:11.913830+0000","last_fullsized":"2024-04-26T21:14:11.913830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:46.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685090+0000","last_change":"2024-04-26T21:14:09.913657+0000","last_active":"2024-04-26T21:14:12.685090+0000","last_peered":"2024-04-26T21:14:12.685090+0000","last_clean":"2024-04-26T21:14:12.685090+0000","last_became_active":"2024-04-26T21:14:09.913317+0000","last_became_peered":"2024-04-26T21:14:09.913317+0000","last_unstale":"2024-04-26T21:14:12.685090+0000","last_undegraded":"2024-04-26T21:14:12.685090+0000","last_fullsized":"2024-04-26T21:14:12.685090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:00.688207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911700+0000","last_change":"2024-04-26T21:14:10.919213+0000","last_active":"2024-04-26T21:14:11.911700+0000","last_peered":"2024-04-26T21:14:11.911700+0000","last_clean":"2024-04-26T21:14:11.911700+0000","last_became_active":"2024-04-26T21:14:10.918921+0000","last_became_peered":"2024-04-26T21:14:10.918921+0000","last_unstale":"2024-04-26T21:14:11.911700+0000","last_undegraded":"2024-04-26T21:14:11.911700+0000","last_fullsized":"2024-04-26T21:14:11.911700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:04.642735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911878+0000","last_change":"2024-04-26T21:14:09.903462+0000","last_active":"2024-04-26T21:14:11.911878+0000","last_peered":"2024-04-26T21:14:11.911878+0000","last_clean":"2024-04-26T21:14:11.911878+0000","last_became_active":"2024-04-26T21:14:09.902981+0000","last_became_peered":"2024-04-26T21:14:09.902981+0000","last_unstale":"2024-04-26T21:14:11.911878+0000","last_undegraded":"2024-04-26T21:14:11.911878+0000","last_fullsized":"2024-04-26T21:14:11.911878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:32.454682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685357+0000","last_change":"2024-04-26T21:14:10.935101+0000","last_active":"2024-04-26T21:14:12.685357+0000","last_peered":"2024-04-26T21:14:12.685357+0000","last_clean":"2024-04-26T21:14:12.685357+0000","last_became_active":"2024-04-26T21:14:10.934820+0000","last_became_peered":"2024-04-26T21:14:10.934820+0000","last_unstale":"2024-04-26T21:14:12.685357+0000","last_undegraded":"2024-04-26T21:14:12.685357+0000","last_fullsized":"2024-04-26T21:14:12.685357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:39.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916109+0000","last_change":"2024-04-26T21:14:09.899047+0000","last_active":"2024-04-26T21:14:11.916109+0000","last_peered":"2024-04-26T21:14:11.916109+0000","last_clean":"2024-04-26T21:14:11.916109+0000","last_became_active":"2024-04-26T21:14:09.898714+0000","last_became_peered":"2024-04-26T21:14:09.898714+0000","last_unstale":"2024-04-26T21:14:11.916109+0000","last_undegraded":"2024-04-26T21:14:11.916109+0000","last_fullsized":"2024-04-26T21:14:11.916109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:10.827803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913108+0000","last_change":"2024-04-26T21:14:09.906753+0000","last_active":"2024-04-26T21:14:11.913108+0000","last_peered":"2024-04-26T21:14:11.913108+0000","last_clean":"2024-04-26T21:14:11.913108+0000","last_became_active":"2024-04-26T21:14:09.906441+0000","last_became_peered":"2024-04-26T21:14:09.906441+0000","last_unstale":"2024-04-26T21:14:11.913108+0000","last_undegraded":"2024-04-26T21:14:11.913108+0000","last_fullsized":"2024-04-26T21:14:11.913108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:46.753087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913127+0000","last_change":"2024-04-26T21:14:09.904804+0000","last_active":"2024-04-26T21:14:11.913127+0000","last_peered":"2024-04-26T21:14:11.913127+0000","last_clean":"2024-04-26T21:14:11.913127+0000","last_became_active":"2024-04-26T21:14:09.904380+0000","last_became_peered":"2024-04-26T21:14:09.904380+0000","last_unstale":"2024-04-26T21:14:11.913127+0000","last_undegraded":"2024-04-26T21:14:11.913127+0000","last_fullsized":"2024-04-26T21:14:11.913127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:23.479723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912760+0000","last_change":"2024-04-26T21:14:10.934581+0000","last_active":"2024-04-26T21:14:11.912760+0000","last_peered":"2024-04-26T21:14:11.912760+0000","last_clean":"2024-04-26T21:14:11.912760+0000","last_became_active":"2024-04-26T21:14:10.933414+0000","last_became_peered":"2024-04-26T21:14:10.933414+0000","last_unstale":"2024-04-26T21:14:11.912760+0000","last_undegraded":"2024-04-26T21:14:11.912760+0000","last_fullsized":"2024-04-26T21:14:11.912760+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:38.008369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913167+0000","last_change":"2024-04-26T21:14:09.904770+0000","last_active":"2024-04-26T21:14:11.913167+0000","last_peered":"2024-04-26T21:14:11.913167+0000","last_clean":"2024-04-26T21:14:11.913167+0000","last_became_active":"2024-04-26T21:14:09.904491+0000","last_became_peered":"2024-04-26T21:14:09.904491+0000","last_unstale":"2024-04-26T21:14:11.913167+0000","last_undegraded":"2024-04-26T21:14:11.913167+0000","last_fullsized":"2024-04-26T21:14:11.913167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:47.054894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-26T21:14:09.891650+0000","last_change":"2024-04-26T21:14:09.891650+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.891650+0000","last_undegraded":"2024-04-26T21:14:09.891650+0000","last_fullsized":"2024-04-26T21:14:09.891650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912896+0000","last_change":"2024-04-26T21:14:09.907449+0000","last_active":"2024-04-26T21:14:11.912896+0000","last_peered":"2024-04-26T21:14:11.912896+0000","last_clean":"2024-04-26T21:14:11.912896+0000","last_became_active":"2024-04-26T21:14:09.907159+0000","last_became_peered":"2024-04-26T21:14:09.907159+0000","last_unstale":"2024-04-26T21:14:11.912896+0000","last_undegraded":"2024-04-26T21:14:11.912896+0000","last_fullsized":"2024-04-26T21:14:11.912896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.185568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911483+0000","last_change":"2024-04-26T21:14:10.929174+0000","last_active":"2024-04-26T21:14:11.911483+0000","last_peered":"2024-04-26T21:14:11.911483+0000","last_clean":"2024-04-26T21:14:11.911483+0000","last_became_active":"2024-04-26T21:14:10.928814+0000","last_became_peered":"2024-04-26T21:14:10.928814+0000","last_unstale":"2024-04-26T21:14:11.911483+0000","last_undegraded":"2024-04-26T21:14:11.911483+0000","last_fullsized":"2024-04-26T21:14:11.911483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:41.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685067+0000","last_change":"2024-04-26T21:14:09.907834+0000","last_active":"2024-04-26T21:14:12.685067+0000","last_peered":"2024-04-26T21:14:12.685067+0000","last_clean":"2024-04-26T21:14:12.685067+0000","last_became_active":"2024-04-26T21:14:09.907570+0000","last_became_peered":"2024-04-26T21:14:09.907570+0000","last_unstale":"2024-04-26T21:14:12.685067+0000","last_undegraded":"2024-04-26T21:14:12.685067+0000","last_fullsized":"2024-04-26T21:14:12.685067+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:35.640232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684819+0000","last_change":"2024-04-26T21:14:10.934459+0000","last_active":"2024-04-26T21:14:12.684819+0000","last_peered":"2024-04-26T21:14:12.684819+0000","last_clean":"2024-04-26T21:14:12.684819+0000","last_became_active":"2024-04-26T21:14:10.934076+0000","last_became_peered":"2024-04-26T21:14:10.934076+0000","last_unstale":"2024-04-26T21:14:12.684819+0000","last_undegraded":"2024-04-26T21:14:12.684819+0000","last_fullsized":"2024-04-26T21:14:12.684819+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:21.179012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914162+0000","last_change":"2024-04-26T21:14:09.905848+0000","last_active":"2024-04-26T21:14:11.914162+0000","last_peered":"2024-04-26T21:14:11.914162+0000","last_clean":"2024-04-26T21:14:11.914162+0000","last_became_active":"2024-04-26T21:14:09.905519+0000","last_became_peered":"2024-04-26T21:14:09.905519+0000","last_unstale":"2024-04-26T21:14:11.914162+0000","last_undegraded":"2024-04-26T21:14:11.914162+0000","last_fullsized":"2024-04-26T21:14:11.914162+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:18.029269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:09.886645+0000","last_change":"2024-04-26T21:14:09.886645+0000","last_active":"2024-04-26T21:14:09.886645+0000","last_peered":"2024-04-26T21:14:09.886645+0000","last_clean":"2024-04-26T21:14:09.886645+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:09.886645+0000","last_undegraded":"2024-04-26T21:14:09.886645+0000","last_fullsized":"2024-04-26T21:14:09.886645+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914737+0000","last_change":"2024-04-26T21:14:09.901751+0000","last_active":"2024-04-26T21:14:11.914737+0000","last_peered":"2024-04-26T21:14:11.914737+0000","last_clean":"2024-04-26T21:14:11.914737+0000","last_became_active":"2024-04-26T21:14:09.901432+0000","last_became_peered":"2024-04-26T21:14:09.901432+0000","last_unstale":"2024-04-26T21:14:11.914737+0000","last_undegraded":"2024-04-26T21:14:11.914737+0000","last_fullsized":"2024-04-26T21:14:11.914737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:20.177485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685036+0000","last_change":"2024-04-26T21:14:10.930260+0000","last_active":"2024-04-26T21:14:12.685036+0000","last_peered":"2024-04-26T21:14:12.685036+0000","last_clean":"2024-04-26T21:14:12.685036+0000","last_became_active":"2024-04-26T21:14:10.929969+0000","last_became_peered":"2024-04-26T21:14:10.929969+0000","last_unstale":"2024-04-26T21:14:12.685036+0000","last_undegraded":"2024-04-26T21:14:12.685036+0000","last_fullsized":"2024-04-26T21:14:12.685036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.481250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914293+0000","last_change":"2024-04-26T21:14:09.909717+0000","last_active":"2024-04-26T21:14:11.914293+0000","last_peered":"2024-04-26T21:14:11.914293+0000","last_clean":"2024-04-26T21:14:11.914293+0000","last_became_active":"2024-04-26T21:14:09.909423+0000","last_became_peered":"2024-04-26T21:14:09.909423+0000","last_unstale":"2024-04-26T21:14:11.914293+0000","last_undegraded":"2024-04-26T21:14:11.914293+0000","last_fullsized":"2024-04-26T21:14:11.914293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:47.054894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914200+0000","last_change":"2024-04-26T21:14:10.931405+0000","last_active":"2024-04-26T21:14:11.914200+0000","last_peered":"2024-04-26T21:14:11.914200+0000","last_clean":"2024-04-26T21:14:11.914200+0000","last_became_active":"2024-04-26T21:14:10.931128+0000","last_became_peered":"2024-04-26T21:14:10.931128+0000","last_unstale":"2024-04-26T21:14:11.914200+0000","last_undegraded":"2024-04-26T21:14:11.914200+0000","last_fullsized":"2024-04-26T21:14:11.914200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:39.495997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:14:09.909133+0000","last_change":"2024-04-26T21:14:09.907803+0000","last_active":"2024-04-26T21:14:09.909133+0000","last_peered":"2024-04-26T21:14:09.909133+0000","last_clean":"2024-04-26T21:14:09.909133+0000","last_became_active":"2024-04-26T21:14:09.907512+0000","last_became_peered":"2024-04-26T21:14:09.907512+0000","last_unstale":"2024-04-26T21:14:09.909133+0000","last_undegraded":"2024-04-26T21:14:09.909133+0000","last_fullsized":"2024-04-26T21:14:09.909133+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:26.512170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912884+0000","last_change":"2024-04-26T21:14:10.947295+0000","last_active":"2024-04-26T21:14:11.912884+0000","last_peered":"2024-04-26T21:14:11.912884+0000","last_clean":"2024-04-26T21:14:11.912884+0000","last_became_active":"2024-04-26T21:14:10.946873+0000","last_became_peered":"2024-04-26T21:14:10.946873+0000","last_unstale":"2024-04-26T21:14:11.912884+0000","last_undegraded":"2024-04-26T21:14:11.912884+0000","last_fullsized":"2024-04-26T21:14:11.912884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:23.542292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913850+0000","last_change":"2024-04-26T21:14:09.909705+0000","last_active":"2024-04-26T21:14:11.913850+0000","last_peered":"2024-04-26T21:14:11.913850+0000","last_clean":"2024-04-26T21:14:11.913850+0000","last_became_active":"2024-04-26T21:14:09.909401+0000","last_became_peered":"2024-04-26T21:14:09.909401+0000","last_unstale":"2024-04-26T21:14:11.913850+0000","last_undegraded":"2024-04-26T21:14:11.913850+0000","last_fullsized":"2024-04-26T21:14:11.913850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912986+0000","last_change":"2024-04-26T21:14:10.940118+0000","last_active":"2024-04-26T21:14:11.912986+0000","last_peered":"2024-04-26T21:14:11.912986+0000","last_clean":"2024-04-26T21:14:11.912986+0000","last_became_active":"2024-04-26T21:14:10.935696+0000","last_became_peered":"2024-04-26T21:14:10.935696+0000","last_unstale":"2024-04-26T21:14:11.912986+0000","last_undegraded":"2024-04-26T21:14:11.912986+0000","last_fullsized":"2024-04-26T21:14:11.912986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:38.645661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915947+0000","last_change":"2024-04-26T21:14:09.900715+0000","last_active":"2024-04-26T21:14:11.915947+0000","last_peered":"2024-04-26T21:14:11.915947+0000","last_clean":"2024-04-26T21:14:11.915947+0000","last_became_active":"2024-04-26T21:14:09.900427+0000","last_became_peered":"2024-04-26T21:14:09.900427+0000","last_unstale":"2024-04-26T21:14:11.915947+0000","last_undegraded":"2024-04-26T21:14:11.915947+0000","last_fullsized":"2024-04-26T21:14:11.915947+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:35.101215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912939+0000","last_change":"2024-04-26T21:14:10.928335+0000","last_active":"2024-04-26T21:14:11.912939+0000","last_peered":"2024-04-26T21:14:11.912939+0000","last_clean":"2024-04-26T21:14:11.912939+0000","last_became_active":"2024-04-26T21:14:10.928030+0000","last_became_peered":"2024-04-26T21:14:10.928030+0000","last_unstale":"2024-04-26T21:14:11.912939+0000","last_undegraded":"2024-04-26T21:14:11.912939+0000","last_fullsized":"2024-04-26T21:14:11.912939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.680017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T21:14:09.906972+0000","last_change":"2024-04-26T21:14:09.905590+0000","last_active":"2024-04-26T21:14:09.906972+0000","last_peered":"2024-04-26T21:14:09.906972+0000","last_clean":"2024-04-26T21:14:09.906972+0000","last_became_active":"2024-04-26T21:14:09.905255+0000","last_became_peered":"2024-04-26T21:14:09.905255+0000","last_unstale":"2024-04-26T21:14:09.906972+0000","last_undegraded":"2024-04-26T21:14:09.906972+0000","last_fullsized":"2024-04-26T21:14:09.906972+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:05.901603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915762+0000","last_change":"2024-04-26T21:14:10.929650+0000","last_active":"2024-04-26T21:14:11.915762+0000","last_peered":"2024-04-26T21:14:11.915762+0000","last_clean":"2024-04-26T21:14:11.915762+0000","last_became_active":"2024-04-26T21:14:10.929013+0000","last_became_peered":"2024-04-26T21:14:10.929013+0000","last_unstale":"2024-04-26T21:14:11.915762+0000","last_undegraded":"2024-04-26T21:14:11.915762+0000","last_fullsized":"2024-04-26T21:14:11.915762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:47.754614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685276+0000","last_change":"2024-04-26T21:14:09.902635+0000","last_active":"2024-04-26T21:14:12.685276+0000","last_peered":"2024-04-26T21:14:12.685276+0000","last_clean":"2024-04-26T21:14:12.685276+0000","last_became_active":"2024-04-26T21:14:09.902350+0000","last_became_peered":"2024-04-26T21:14:09.902350+0000","last_unstale":"2024-04-26T21:14:12.685276+0000","last_undegraded":"2024-04-26T21:14:12.685276+0000","last_fullsized":"2024-04-26T21:14:12.685276+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:14.176996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685478+0000","last_change":"2024-04-26T21:14:10.932196+0000","last_active":"2024-04-26T21:14:12.685478+0000","last_peered":"2024-04-26T21:14:12.685478+0000","last_clean":"2024-04-26T21:14:12.685478+0000","last_became_active":"2024-04-26T21:14:10.931213+0000","last_became_peered":"2024-04-26T21:14:10.931213+0000","last_unstale":"2024-04-26T21:14:12.685478+0000","last_undegraded":"2024-04-26T21:14:12.685478+0000","last_fullsized":"2024-04-26T21:14:12.685478+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913310+0000","last_change":"2024-04-26T21:14:09.906889+0000","last_active":"2024-04-26T21:14:11.913310+0000","last_peered":"2024-04-26T21:14:11.913310+0000","last_clean":"2024-04-26T21:14:11.913310+0000","last_became_active":"2024-04-26T21:14:09.906180+0000","last_became_peered":"2024-04-26T21:14:09.906180+0000","last_unstale":"2024-04-26T21:14:11.913310+0000","last_undegraded":"2024-04-26T21:14:11.913310+0000","last_fullsized":"2024-04-26T21:14:11.913310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:18.029269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":213,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913982+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:11.913982+0000","last_peered":"2024-04-26T21:14:11.913982+0000","last_clean":"2024-04-26T21:14:11.913982+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:11.913982+0000","last_undegraded":"2024-04-26T21:14:11.913982+0000","last_fullsized":"2024-04-26T21:14:11.913982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":810729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":195112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":220341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":221883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":557056,"data_stored":220909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":221905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":804373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":200929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:14.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.066+0000 7fdc55e93640 1 Processor -- start 2024-04-26T21:14:15.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.066+0000 7fdc55e93640 1 -- start start 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.066+0000 7fdc55e93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc500cf2e0 0x7fdc5006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.066+0000 7fdc55e93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc500d11b0 0x7fdc5006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc55e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc500d1740 con 0x7fdc5006e2c0 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc55e93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5006e710 con 0x7fdc500cf2e0 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc55e93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc500d1a20 con 0x7fdc500d11b0 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc500cf2e0 0x7fdc5006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc500cf2e0 0x7fdc5006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60006/0 (socket says 172.21.15.40:60006) 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc5006e2c0 0x7fdc5015ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34916/0 (socket says 172.21.15.40:34916) 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/4115267298 learned_addr learned my addr 172.21.15.40:0/4115267298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4effd640 1 --2- 172.21.15.40:0/4115267298 >> v2:172.21.15.184:3300/0 conn(0x7fdc500d11b0 0x7fdc5006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/4115267298 >> v2:172.21.15.184:3300/0 conn(0x7fdc500d11b0 msgr2=0x7fdc5006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 --2- 172.21.15.40:0/4115267298 >> v2:172.21.15.184:3300/0 conn(0x7fdc500d11b0 0x7fdc5006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/4115267298 >> v1:172.21.15.40:6789/0 conn(0x7fdc5006e2c0 legacy=0x7fdc5015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.067+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/4115267298 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500d1d00 con 0x7fdc500cf2e0 2024-04-26T21:14:15.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc4f7fe640 1 --2- 172.21.15.40:0/4115267298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc500cf2e0 0x7fdc5006d4d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009870 tx=0x7fdc4402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5206909d12279c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc4e7fc640 1 -- 172.21.15.40:0/4115267298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44037070 con 0x7fdc500cf2e0 2024-04-26T21:14:15.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc4e7fc640 1 -- 172.21.15.40:0/4115267298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44032440 con 0x7fdc500cf2e0 2024-04-26T21:14:15.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc4e7fc640 1 -- 172.21.15.40:0/4115267298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44032810 con 0x7fdc500cf2e0 2024-04-26T21:14:15.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 -- 172.21.15.40:0/4115267298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc500cf2e0 msgr2=0x7fdc5006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 --2- 172.21.15.40:0/4115267298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc500cf2e0 0x7fdc5006d4d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009870 tx=0x7fdc4402a2a0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 -- 172.21.15.40:0/4115267298 shutdown_connections 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 --2- 172.21.15.40:0/4115267298 >> v2:172.21.15.184:3300/0 conn(0x7fdc500d11b0 0x7fdc5006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 --2- 172.21.15.40:0/4115267298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc500cf2e0 0x7fdc5006d4d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 -- 172.21.15.40:0/4115267298 >> 172.21.15.40:0/4115267298 conn(0x7fdc500b4a60 msgr2=0x7fdc500b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 -- 172.21.15.40:0/4115267298 shutdown_connections 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.068+0000 7fdc55e93640 1 -- 172.21.15.40:0/4115267298 wait complete. 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 Processor -- start 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 -- start start 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc500cf2e0 0x7fdc500b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 0x7fdc5014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.070+0000 7fdc55e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5006da10 con 0x7fdc5006e2c0 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc55e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500d22c0 con 0x7fdc500d11b0 2024-04-26T21:14:15.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc55e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc500cf6c0 con 0x7fdc500cf2e0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60016/0 (socket says 172.21.15.40:60016) 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/2818175596 learned_addr learned my addr 172.21.15.40:0/2818175596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4effd640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc500cf2e0 0x7fdc500b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc500cf2e0 msgr2=0x7fdc500b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4ffff640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 0x7fdc5014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc500cf2e0 0x7fdc500b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 msgr2=0x7fdc5014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 0x7fdc5014ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 -- 172.21.15.40:0/2818175596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500b6560 con 0x7fdc5006e2c0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4ffff640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 0x7fdc5014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc4f7fe640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44041020 con 0x7fdc5006e2c0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc440306c0 con 0x7fdc5006e2c0 2024-04-26T21:14:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc500b6840 con 0x7fdc5006e2c0 2024-04-26T21:14:15.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44037040 con 0x7fdc5006e2c0 2024-04-26T21:14:15.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.071+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc500b6e00 con 0x7fdc5006e2c0 2024-04-26T21:14:15.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.072+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5015e3e0 con 0x7fdc5006e2c0 2024-04-26T21:14:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.073+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc44044020 con 0x7fdc5006e2c0 2024-04-26T21:14:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.073+0000 7fdc37fff640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.073+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fdc440ac260 con 0x7fdc5006e2c0 2024-04-26T21:14:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.074+0000 7fdc4effd640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.074+0000 7fdc4effd640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdc40006fd0 tx=0x7fdc40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.078+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc4404b030 con 0x7fdc5006e2c0 2024-04-26T21:14:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.262+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc500b8ef0 con 0x7fdc3c06c8f0 2024-04-26T21:14:15.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.269+0000 7fdc37fff640 1 -- 172.21.15.40:0/2818175596 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251958 (secure 0 0 0) 0x7fdc500b8ef0 con 0x7fdc3c06c8f0 2024-04-26T21:14:15.272 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:15.273 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 msgr2=0x7fdc3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdc40006fd0 tx=0x7fdc40008040 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 msgr2=0x7fdc500cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc440046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 shutdown_connections 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc500d11b0 0x7fdc5014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc500cf2e0 0x7fdc500b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 --2- 172.21.15.40:0/2818175596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5006e2c0 0x7fdc500cfbd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 >> 172.21.15.40:0/2818175596 conn(0x7fdc500b4a60 msgr2=0x7fdc5009c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 shutdown_connections 2024-04-26T21:14:15.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.272+0000 7fdc55e93640 1 -- 172.21.15.40:0/2818175596 wait complete. 2024-04-26T21:14:15.284 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-26T21:14:14.949595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234132,"kb_used_data":5468,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740588,"statfs":{"total":773094113280,"available":772854362112,"internally_reserved":0,"allocated":5599232,"data_stored":2941424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001300"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913490+0000","last_change":"2024-04-26T21:14:10.939188+0000","last_active":"2024-04-26T21:14:11.913490+0000","last_peered":"2024-04-26T21:14:11.913490+0000","last_clean":"2024-04-26T21:14:11.913490+0000","last_became_active":"2024-04-26T21:14:10.938935+0000","last_became_peered":"2024-04-26T21:14:10.938935+0000","last_unstale":"2024-04-26T21:14:11.913490+0000","last_undegraded":"2024-04-26T21:14:11.913490+0000","last_fullsized":"2024-04-26T21:14:11.913490+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914266+0000","last_change":"2024-04-26T21:14:10.944548+0000","last_active":"2024-04-26T21:14:11.914266+0000","last_peered":"2024-04-26T21:14:11.914266+0000","last_clean":"2024-04-26T21:14:11.914266+0000","last_became_active":"2024-04-26T21:14:10.944248+0000","last_became_peered":"2024-04-26T21:14:10.944248+0000","last_unstale":"2024-04-26T21:14:11.914266+0000","last_undegraded":"2024-04-26T21:14:11.914266+0000","last_fullsized":"2024-04-26T21:14:11.914266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:46.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915895+0000","last_change":"2024-04-26T21:14:10.917770+0000","last_active":"2024-04-26T21:14:11.915895+0000","last_peered":"2024-04-26T21:14:11.915895+0000","last_clean":"2024-04-26T21:14:11.915895+0000","last_became_active":"2024-04-26T21:14:10.917473+0000","last_became_peered":"2024-04-26T21:14:10.917473+0000","last_unstale":"2024-04-26T21:14:11.915895+0000","last_undegraded":"2024-04-26T21:14:11.915895+0000","last_fullsized":"2024-04-26T21:14:11.915895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:22.505944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684737+0000","last_change":"2024-04-26T21:14:10.935906+0000","last_active":"2024-04-26T21:14:12.684737+0000","last_peered":"2024-04-26T21:14:12.684737+0000","last_clean":"2024-04-26T21:14:12.684737+0000","last_became_active":"2024-04-26T21:14:10.935600+0000","last_became_peered":"2024-04-26T21:14:10.935600+0000","last_unstale":"2024-04-26T21:14:12.684737+0000","last_undegraded":"2024-04-26T21:14:12.684737+0000","last_fullsized":"2024-04-26T21:14:12.684737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:48.056421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913179+0000","last_change":"2024-04-26T21:14:10.943004+0000","last_active":"2024-04-26T21:14:11.913179+0000","last_peered":"2024-04-26T21:14:11.913179+0000","last_clean":"2024-04-26T21:14:11.913179+0000","last_became_active":"2024-04-26T21:14:10.942668+0000","last_became_peered":"2024-04-26T21:14:10.942668+0000","last_unstale":"2024-04-26T21:14:11.913179+0000","last_undegraded":"2024-04-26T21:14:11.913179+0000","last_fullsized":"2024-04-26T21:14:11.913179+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:46.122041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912722+0000","last_change":"2024-04-26T21:14:10.934097+0000","last_active":"2024-04-26T21:14:11.912722+0000","last_peered":"2024-04-26T21:14:11.912722+0000","last_clean":"2024-04-26T21:14:11.912722+0000","last_became_active":"2024-04-26T21:14:10.933814+0000","last_became_peered":"2024-04-26T21:14:10.933814+0000","last_unstale":"2024-04-26T21:14:11.912722+0000","last_undegraded":"2024-04-26T21:14:11.912722+0000","last_fullsized":"2024-04-26T21:14:11.912722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:53.562201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913641+0000","last_change":"2024-04-26T21:14:10.946950+0000","last_active":"2024-04-26T21:14:11.913641+0000","last_peered":"2024-04-26T21:14:11.913641+0000","last_clean":"2024-04-26T21:14:11.913641+0000","last_became_active":"2024-04-26T21:14:10.946628+0000","last_became_peered":"2024-04-26T21:14:10.946628+0000","last_unstale":"2024-04-26T21:14:11.913641+0000","last_undegraded":"2024-04-26T21:14:11.913641+0000","last_fullsized":"2024-04-26T21:14:11.913641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684224+0000","last_change":"2024-04-26T21:14:10.943471+0000","last_active":"2024-04-26T21:14:12.684224+0000","last_peered":"2024-04-26T21:14:12.684224+0000","last_clean":"2024-04-26T21:14:12.684224+0000","last_became_active":"2024-04-26T21:14:10.942879+0000","last_became_peered":"2024-04-26T21:14:10.942879+0000","last_unstale":"2024-04-26T21:14:12.684224+0000","last_undegraded":"2024-04-26T21:14:12.684224+0000","last_fullsized":"2024-04-26T21:14:12.684224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:47.754614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685292+0000","last_change":"2024-04-26T21:14:10.941189+0000","last_active":"2024-04-26T21:14:12.685292+0000","last_peered":"2024-04-26T21:14:12.685292+0000","last_clean":"2024-04-26T21:14:12.685292+0000","last_became_active":"2024-04-26T21:14:10.940477+0000","last_became_peered":"2024-04-26T21:14:10.940477+0000","last_unstale":"2024-04-26T21:14:12.685292+0000","last_undegraded":"2024-04-26T21:14:12.685292+0000","last_fullsized":"2024-04-26T21:14:12.685292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:52:46.122041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685400+0000","last_change":"2024-04-26T21:14:10.933740+0000","last_active":"2024-04-26T21:14:12.685400+0000","last_peered":"2024-04-26T21:14:12.685400+0000","last_clean":"2024-04-26T21:14:12.685400+0000","last_became_active":"2024-04-26T21:14:10.933419+0000","last_became_peered":"2024-04-26T21:14:10.933419+0000","last_unstale":"2024-04-26T21:14:12.685400+0000","last_undegraded":"2024-04-26T21:14:12.685400+0000","last_fullsized":"2024-04-26T21:14:12.685400+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685154+0000","last_change":"2024-04-26T21:14:10.942468+0000","last_active":"2024-04-26T21:14:12.685154+0000","last_peered":"2024-04-26T21:14:12.685154+0000","last_clean":"2024-04-26T21:14:12.685154+0000","last_became_active":"2024-04-26T21:14:10.942182+0000","last_became_peered":"2024-04-26T21:14:10.942182+0000","last_unstale":"2024-04-26T21:14:12.685154+0000","last_undegraded":"2024-04-26T21:14:12.685154+0000","last_fullsized":"2024-04-26T21:14:12.685154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.680017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913413+0000","last_change":"2024-04-26T21:14:10.921134+0000","last_active":"2024-04-26T21:14:11.913413+0000","last_peered":"2024-04-26T21:14:11.913413+0000","last_clean":"2024-04-26T21:14:11.913413+0000","last_became_active":"2024-04-26T21:14:10.920642+0000","last_became_peered":"2024-04-26T21:14:10.920642+0000","last_unstale":"2024-04-26T21:14:11.913413+0000","last_undegraded":"2024-04-26T21:14:11.913413+0000","last_fullsized":"2024-04-26T21:14:11.913413+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:39.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911996+0000","last_change":"2024-04-26T21:14:10.930737+0000","last_active":"2024-04-26T21:14:11.911996+0000","last_peered":"2024-04-26T21:14:11.911996+0000","last_clean":"2024-04-26T21:14:11.911996+0000","last_became_active":"2024-04-26T21:14:10.930435+0000","last_became_peered":"2024-04-26T21:14:10.930435+0000","last_unstale":"2024-04-26T21:14:11.911996+0000","last_undegraded":"2024-04-26T21:14:11.911996+0000","last_fullsized":"2024-04-26T21:14:11.911996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:17.093360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684439+0000","last_change":"2024-04-26T21:14:10.947299+0000","last_active":"2024-04-26T21:14:12.684439+0000","last_peered":"2024-04-26T21:14:12.684439+0000","last_clean":"2024-04-26T21:14:12.684439+0000","last_became_active":"2024-04-26T21:14:10.946866+0000","last_became_peered":"2024-04-26T21:14:10.946866+0000","last_unstale":"2024-04-26T21:14:12.684439+0000","last_undegraded":"2024-04-26T21:14:12.684439+0000","last_fullsized":"2024-04-26T21:14:12.684439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:04.756658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912621+0000","last_change":"2024-04-26T21:14:10.939154+0000","last_active":"2024-04-26T21:14:11.912621+0000","last_peered":"2024-04-26T21:14:11.912621+0000","last_clean":"2024-04-26T21:14:11.912621+0000","last_became_active":"2024-04-26T21:14:10.938864+0000","last_became_peered":"2024-04-26T21:14:10.938864+0000","last_unstale":"2024-04-26T21:14:11.912621+0000","last_undegraded":"2024-04-26T21:14:11.912621+0000","last_fullsized":"2024-04-26T21:14:11.912621+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:40.789418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684928+0000","last_change":"2024-04-26T21:14:10.944600+0000","last_active":"2024-04-26T21:14:12.684928+0000","last_peered":"2024-04-26T21:14:12.684928+0000","last_clean":"2024-04-26T21:14:12.684928+0000","last_became_active":"2024-04-26T21:14:10.944301+0000","last_became_peered":"2024-04-26T21:14:10.944301+0000","last_unstale":"2024-04-26T21:14:12.684928+0000","last_undegraded":"2024-04-26T21:14:12.684928+0000","last_fullsized":"2024-04-26T21:14:12.684928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:10.523936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916102+0000","last_change":"2024-04-26T21:14:10.919596+0000","last_active":"2024-04-26T21:14:11.916102+0000","last_peered":"2024-04-26T21:14:11.916102+0000","last_clean":"2024-04-26T21:14:11.916102+0000","last_became_active":"2024-04-26T21:14:10.919235+0000","last_became_peered":"2024-04-26T21:14:10.919235+0000","last_unstale":"2024-04-26T21:14:11.916102+0000","last_undegraded":"2024-04-26T21:14:11.916102+0000","last_fullsized":"2024-04-26T21:14:11.916102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.304369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915952+0000","last_change":"2024-04-26T21:14:10.945274+0000","last_active":"2024-04-26T21:14:11.915952+0000","last_peered":"2024-04-26T21:14:11.915952+0000","last_clean":"2024-04-26T21:14:11.915952+0000","last_became_active":"2024-04-26T21:14:10.944969+0000","last_became_peered":"2024-04-26T21:14:10.944969+0000","last_unstale":"2024-04-26T21:14:11.915952+0000","last_undegraded":"2024-04-26T21:14:11.915952+0000","last_fullsized":"2024-04-26T21:14:11.915952+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:54.218317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685115+0000","last_change":"2024-04-26T21:14:10.943025+0000","last_active":"2024-04-26T21:14:12.685115+0000","last_peered":"2024-04-26T21:14:12.685115+0000","last_clean":"2024-04-26T21:14:12.685115+0000","last_became_active":"2024-04-26T21:14:10.942635+0000","last_became_peered":"2024-04-26T21:14:10.942635+0000","last_unstale":"2024-04-26T21:14:12.685115+0000","last_undegraded":"2024-04-26T21:14:12.685115+0000","last_fullsized":"2024-04-26T21:14:12.685115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:16.280240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911727+0000","last_change":"2024-04-26T21:14:10.942738+0000","last_active":"2024-04-26T21:14:11.911727+0000","last_peered":"2024-04-26T21:14:11.911727+0000","last_clean":"2024-04-26T21:14:11.911727+0000","last_became_active":"2024-04-26T21:14:10.942258+0000","last_became_peered":"2024-04-26T21:14:10.942258+0000","last_unstale":"2024-04-26T21:14:11.911727+0000","last_undegraded":"2024-04-26T21:14:11.911727+0000","last_fullsized":"2024-04-26T21:14:11.911727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:36.641758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912818+0000","last_change":"2024-04-26T21:14:10.940649+0000","last_active":"2024-04-26T21:14:11.912818+0000","last_peered":"2024-04-26T21:14:11.912818+0000","last_clean":"2024-04-26T21:14:11.912818+0000","last_became_active":"2024-04-26T21:14:10.939744+0000","last_became_peered":"2024-04-26T21:14:10.939744+0000","last_unstale":"2024-04-26T21:14:11.912818+0000","last_undegraded":"2024-04-26T21:14:11.912818+0000","last_fullsized":"2024-04-26T21:14:11.912818+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:08.899304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684462+0000","last_change":"2024-04-26T21:14:10.938159+0000","last_active":"2024-04-26T21:14:12.684462+0000","last_peered":"2024-04-26T21:14:12.684462+0000","last_clean":"2024-04-26T21:14:12.684462+0000","last_became_active":"2024-04-26T21:14:10.937658+0000","last_became_peered":"2024-04-26T21:14:10.937658+0000","last_unstale":"2024-04-26T21:14:12.684462+0000","last_undegraded":"2024-04-26T21:14:12.684462+0000","last_fullsized":"2024-04-26T21:14:12.684462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:22.505944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684547+0000","last_change":"2024-04-26T21:14:10.929235+0000","last_active":"2024-04-26T21:14:12.684547+0000","last_peered":"2024-04-26T21:14:12.684547+0000","last_clean":"2024-04-26T21:14:12.684547+0000","last_became_active":"2024-04-26T21:14:10.928857+0000","last_became_peered":"2024-04-26T21:14:10.928857+0000","last_unstale":"2024-04-26T21:14:12.684547+0000","last_undegraded":"2024-04-26T21:14:12.684547+0000","last_fullsized":"2024-04-26T21:14:12.684547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:32.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684970+0000","last_change":"2024-04-26T21:14:10.926481+0000","last_active":"2024-04-26T21:14:12.684970+0000","last_peered":"2024-04-26T21:14:12.684970+0000","last_clean":"2024-04-26T21:14:12.684970+0000","last_became_active":"2024-04-26T21:14:10.926179+0000","last_became_peered":"2024-04-26T21:14:10.926179+0000","last_unstale":"2024-04-26T21:14:12.684970+0000","last_undegraded":"2024-04-26T21:14:12.684970+0000","last_fullsized":"2024-04-26T21:14:12.684970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:24.400849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684754+0000","last_change":"2024-04-26T21:14:10.938049+0000","last_active":"2024-04-26T21:14:12.684754+0000","last_peered":"2024-04-26T21:14:12.684754+0000","last_clean":"2024-04-26T21:14:12.684754+0000","last_became_active":"2024-04-26T21:14:10.937695+0000","last_became_peered":"2024-04-26T21:14:10.937695+0000","last_unstale":"2024-04-26T21:14:12.684754+0000","last_undegraded":"2024-04-26T21:14:12.684754+0000","last_fullsized":"2024-04-26T21:14:12.684754+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:07.105576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911618+0000","last_change":"2024-04-26T21:14:10.936935+0000","last_active":"2024-04-26T21:14:11.911618+0000","last_peered":"2024-04-26T21:14:11.911618+0000","last_clean":"2024-04-26T21:14:11.911618+0000","last_became_active":"2024-04-26T21:14:10.936588+0000","last_became_peered":"2024-04-26T21:14:10.936588+0000","last_unstale":"2024-04-26T21:14:11.911618+0000","last_undegraded":"2024-04-26T21:14:11.911618+0000","last_fullsized":"2024-04-26T21:14:11.911618+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:24.974256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916006+0000","last_change":"2024-04-26T21:14:10.918939+0000","last_active":"2024-04-26T21:14:11.916006+0000","last_peered":"2024-04-26T21:14:11.916006+0000","last_clean":"2024-04-26T21:14:11.916006+0000","last_became_active":"2024-04-26T21:14:10.918632+0000","last_became_peered":"2024-04-26T21:14:10.918632+0000","last_unstale":"2024-04-26T21:14:11.916006+0000","last_undegraded":"2024-04-26T21:14:11.916006+0000","last_fullsized":"2024-04-26T21:14:11.916006+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:48.056421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913499+0000","last_change":"2024-04-26T21:14:10.920909+0000","last_active":"2024-04-26T21:14:11.913499+0000","last_peered":"2024-04-26T21:14:11.913499+0000","last_clean":"2024-04-26T21:14:11.913499+0000","last_became_active":"2024-04-26T21:14:10.920294+0000","last_became_peered":"2024-04-26T21:14:10.920294+0000","last_unstale":"2024-04-26T21:14:11.913499+0000","last_undegraded":"2024-04-26T21:14:11.913499+0000","last_fullsized":"2024-04-26T21:14:11.913499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:47.436087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684636+0000","last_change":"2024-04-26T21:14:10.937907+0000","last_active":"2024-04-26T21:14:12.684636+0000","last_peered":"2024-04-26T21:14:12.684636+0000","last_clean":"2024-04-26T21:14:12.684636+0000","last_became_active":"2024-04-26T21:14:10.937659+0000","last_became_peered":"2024-04-26T21:14:10.937659+0000","last_unstale":"2024-04-26T21:14:12.684636+0000","last_undegraded":"2024-04-26T21:14:12.684636+0000","last_fullsized":"2024-04-26T21:14:12.684636+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:11.829329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915771+0000","last_change":"2024-04-26T21:14:10.930302+0000","last_active":"2024-04-26T21:14:11.915771+0000","last_peered":"2024-04-26T21:14:11.915771+0000","last_clean":"2024-04-26T21:14:11.915771+0000","last_became_active":"2024-04-26T21:14:10.930023+0000","last_became_peered":"2024-04-26T21:14:10.930023+0000","last_unstale":"2024-04-26T21:14:11.915771+0000","last_undegraded":"2024-04-26T21:14:11.915771+0000","last_fullsized":"2024-04-26T21:14:11.915771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:19.030796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912704+0000","last_change":"2024-04-26T21:14:10.929723+0000","last_active":"2024-04-26T21:14:11.912704+0000","last_peered":"2024-04-26T21:14:11.912704+0000","last_clean":"2024-04-26T21:14:11.912704+0000","last_became_active":"2024-04-26T21:14:10.929453+0000","last_became_peered":"2024-04-26T21:14:10.929453+0000","last_unstale":"2024-04-26T21:14:11.912704+0000","last_undegraded":"2024-04-26T21:14:11.912704+0000","last_fullsized":"2024-04-26T21:14:11.912704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:05.712074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911573+0000","last_change":"2024-04-26T21:14:10.938091+0000","last_active":"2024-04-26T21:14:11.911573+0000","last_peered":"2024-04-26T21:14:11.911573+0000","last_clean":"2024-04-26T21:14:11.911573+0000","last_became_active":"2024-04-26T21:14:10.937760+0000","last_became_peered":"2024-04-26T21:14:10.937760+0000","last_unstale":"2024-04-26T21:14:11.911573+0000","last_undegraded":"2024-04-26T21:14:11.911573+0000","last_fullsized":"2024-04-26T21:14:11.911573+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:05:23.704316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684734+0000","last_change":"2024-04-26T21:14:10.936390+0000","last_active":"2024-04-26T21:14:12.684734+0000","last_peered":"2024-04-26T21:14:12.684734+0000","last_clean":"2024-04-26T21:14:12.684734+0000","last_became_active":"2024-04-26T21:14:10.936106+0000","last_became_peered":"2024-04-26T21:14:10.936106+0000","last_unstale":"2024-04-26T21:14:12.684734+0000","last_undegraded":"2024-04-26T21:14:12.684734+0000","last_fullsized":"2024-04-26T21:14:12.684734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:09.919120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913068+0000","last_change":"2024-04-26T21:14:09.909494+0000","last_active":"2024-04-26T21:14:11.913068+0000","last_peered":"2024-04-26T21:14:11.913068+0000","last_clean":"2024-04-26T21:14:11.913068+0000","last_became_active":"2024-04-26T21:14:09.909163+0000","last_became_peered":"2024-04-26T21:14:09.909163+0000","last_unstale":"2024-04-26T21:14:11.913068+0000","last_undegraded":"2024-04-26T21:14:11.913068+0000","last_fullsized":"2024-04-26T21:14:11.913068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684804+0000","last_change":"2024-04-26T21:14:10.926641+0000","last_active":"2024-04-26T21:14:12.684804+0000","last_peered":"2024-04-26T21:14:12.684804+0000","last_clean":"2024-04-26T21:14:12.684804+0000","last_became_active":"2024-04-26T21:14:10.926205+0000","last_became_peered":"2024-04-26T21:14:10.926205+0000","last_unstale":"2024-04-26T21:14:12.684804+0000","last_undegraded":"2024-04-26T21:14:12.684804+0000","last_fullsized":"2024-04-26T21:14:12.684804+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:39.866736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916149+0000","last_change":"2024-04-26T21:14:09.907581+0000","last_active":"2024-04-26T21:14:11.916149+0000","last_peered":"2024-04-26T21:14:11.916149+0000","last_clean":"2024-04-26T21:14:11.916149+0000","last_became_active":"2024-04-26T21:14:09.907319+0000","last_became_peered":"2024-04-26T21:14:09.907319+0000","last_unstale":"2024-04-26T21:14:11.916149+0000","last_undegraded":"2024-04-26T21:14:11.916149+0000","last_fullsized":"2024-04-26T21:14:11.916149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:06.104049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913576+0000","last_change":"2024-04-26T21:14:10.931029+0000","last_active":"2024-04-26T21:14:11.913576+0000","last_peered":"2024-04-26T21:14:11.913576+0000","last_clean":"2024-04-26T21:14:11.913576+0000","last_became_active":"2024-04-26T21:14:10.930755+0000","last_became_peered":"2024-04-26T21:14:10.930755+0000","last_unstale":"2024-04-26T21:14:11.913576+0000","last_undegraded":"2024-04-26T21:14:11.913576+0000","last_fullsized":"2024-04-26T21:14:11.913576+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684932+0000","last_change":"2024-04-26T21:14:09.908272+0000","last_active":"2024-04-26T21:14:12.684932+0000","last_peered":"2024-04-26T21:14:12.684932+0000","last_clean":"2024-04-26T21:14:12.684932+0000","last_became_active":"2024-04-26T21:14:09.907945+0000","last_became_peered":"2024-04-26T21:14:09.907945+0000","last_unstale":"2024-04-26T21:14:12.684932+0000","last_undegraded":"2024-04-26T21:14:12.684932+0000","last_fullsized":"2024-04-26T21:14:12.684932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:54.883413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684815+0000","last_change":"2024-04-26T21:14:10.941033+0000","last_active":"2024-04-26T21:14:12.684815+0000","last_peered":"2024-04-26T21:14:12.684815+0000","last_clean":"2024-04-26T21:14:12.684815+0000","last_became_active":"2024-04-26T21:14:10.937348+0000","last_became_peered":"2024-04-26T21:14:10.937348+0000","last_unstale":"2024-04-26T21:14:12.684815+0000","last_undegraded":"2024-04-26T21:14:12.684815+0000","last_fullsized":"2024-04-26T21:14:12.684815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:15.996445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912924+0000","last_change":"2024-04-26T21:14:09.913677+0000","last_active":"2024-04-26T21:14:11.912924+0000","last_peered":"2024-04-26T21:14:11.912924+0000","last_clean":"2024-04-26T21:14:11.912924+0000","last_became_active":"2024-04-26T21:14:09.913263+0000","last_became_peered":"2024-04-26T21:14:09.913263+0000","last_unstale":"2024-04-26T21:14:11.912924+0000","last_undegraded":"2024-04-26T21:14:11.912924+0000","last_fullsized":"2024-04-26T21:14:11.912924+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:21.504417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915820+0000","last_change":"2024-04-26T21:14:10.925252+0000","last_active":"2024-04-26T21:14:11.915820+0000","last_peered":"2024-04-26T21:14:11.915820+0000","last_clean":"2024-04-26T21:14:11.915820+0000","last_became_active":"2024-04-26T21:14:10.924967+0000","last_became_peered":"2024-04-26T21:14:10.924967+0000","last_unstale":"2024-04-26T21:14:11.915820+0000","last_undegraded":"2024-04-26T21:14:11.915820+0000","last_fullsized":"2024-04-26T21:14:11.915820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:04.756658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912981+0000","last_change":"2024-04-26T21:14:09.906360+0000","last_active":"2024-04-26T21:14:11.912981+0000","last_peered":"2024-04-26T21:14:11.912981+0000","last_clean":"2024-04-26T21:14:11.912981+0000","last_became_active":"2024-04-26T21:14:09.906084+0000","last_became_peered":"2024-04-26T21:14:09.906084+0000","last_unstale":"2024-04-26T21:14:11.912981+0000","last_undegraded":"2024-04-26T21:14:11.912981+0000","last_fullsized":"2024-04-26T21:14:11.912981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:31.302842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684713+0000","last_change":"2024-04-26T21:14:10.940653+0000","last_active":"2024-04-26T21:14:12.684713+0000","last_peered":"2024-04-26T21:14:12.684713+0000","last_clean":"2024-04-26T21:14:12.684713+0000","last_became_active":"2024-04-26T21:14:10.940376+0000","last_became_peered":"2024-04-26T21:14:10.940376+0000","last_unstale":"2024-04-26T21:14:12.684713+0000","last_undegraded":"2024-04-26T21:14:12.684713+0000","last_fullsized":"2024-04-26T21:14:12.684713+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:54.218317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915935+0000","last_change":"2024-04-26T21:14:09.907047+0000","last_active":"2024-04-26T21:14:11.915935+0000","last_peered":"2024-04-26T21:14:11.915935+0000","last_clean":"2024-04-26T21:14:11.915935+0000","last_became_active":"2024-04-26T21:14:09.906676+0000","last_became_peered":"2024-04-26T21:14:09.906676+0000","last_unstale":"2024-04-26T21:14:11.915935+0000","last_undegraded":"2024-04-26T21:14:11.915935+0000","last_fullsized":"2024-04-26T21:14:11.915935+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:31.505524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684416+0000","last_change":"2024-04-26T21:14:10.943138+0000","last_active":"2024-04-26T21:14:12.684416+0000","last_peered":"2024-04-26T21:14:12.684416+0000","last_clean":"2024-04-26T21:14:12.684416+0000","last_became_active":"2024-04-26T21:14:10.941274+0000","last_became_peered":"2024-04-26T21:14:10.941274+0000","last_unstale":"2024-04-26T21:14:12.684416+0000","last_undegraded":"2024-04-26T21:14:12.684416+0000","last_fullsized":"2024-04-26T21:14:12.684416+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:19.030796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912994+0000","last_change":"2024-04-26T21:14:09.906600+0000","last_active":"2024-04-26T21:14:11.912994+0000","last_peered":"2024-04-26T21:14:11.912994+0000","last_clean":"2024-04-26T21:14:11.912994+0000","last_became_active":"2024-04-26T21:14:09.906314+0000","last_became_peered":"2024-04-26T21:14:09.906314+0000","last_unstale":"2024-04-26T21:14:11.912994+0000","last_undegraded":"2024-04-26T21:14:11.912994+0000","last_fullsized":"2024-04-26T21:14:11.912994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:15.278713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684473+0000","last_change":"2024-04-26T21:14:10.930100+0000","last_active":"2024-04-26T21:14:12.684473+0000","last_peered":"2024-04-26T21:14:12.684473+0000","last_clean":"2024-04-26T21:14:12.684473+0000","last_became_active":"2024-04-26T21:14:10.929816+0000","last_became_peered":"2024-04-26T21:14:10.929816+0000","last_unstale":"2024-04-26T21:14:12.684473+0000","last_undegraded":"2024-04-26T21:14:12.684473+0000","last_fullsized":"2024-04-26T21:14:12.684473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:22.267005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685099+0000","last_change":"2024-04-26T21:14:09.906801+0000","last_active":"2024-04-26T21:14:12.685099+0000","last_peered":"2024-04-26T21:14:12.685099+0000","last_clean":"2024-04-26T21:14:12.685099+0000","last_became_active":"2024-04-26T21:14:09.906512+0000","last_became_peered":"2024-04-26T21:14:09.906512+0000","last_unstale":"2024-04-26T21:14:12.685099+0000","last_undegraded":"2024-04-26T21:14:12.685099+0000","last_fullsized":"2024-04-26T21:14:12.685099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:16.091833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916003+0000","last_change":"2024-04-26T21:14:10.945343+0000","last_active":"2024-04-26T21:14:11.916003+0000","last_peered":"2024-04-26T21:14:11.916003+0000","last_clean":"2024-04-26T21:14:11.916003+0000","last_became_active":"2024-04-26T21:14:10.945061+0000","last_became_peered":"2024-04-26T21:14:10.945061+0000","last_unstale":"2024-04-26T21:14:11.916003+0000","last_undegraded":"2024-04-26T21:14:11.916003+0000","last_fullsized":"2024-04-26T21:14:11.916003+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:21.179012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916194+0000","last_change":"2024-04-26T21:14:09.902491+0000","last_active":"2024-04-26T21:14:11.916194+0000","last_peered":"2024-04-26T21:14:11.916194+0000","last_clean":"2024-04-26T21:14:11.916194+0000","last_became_active":"2024-04-26T21:14:09.902237+0000","last_became_peered":"2024-04-26T21:14:09.902237+0000","last_unstale":"2024-04-26T21:14:11.916194+0000","last_undegraded":"2024-04-26T21:14:11.916194+0000","last_fullsized":"2024-04-26T21:14:11.916194+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:21.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913893+0000","last_change":"2024-04-26T21:14:10.930600+0000","last_active":"2024-04-26T21:14:11.913893+0000","last_peered":"2024-04-26T21:14:11.913893+0000","last_clean":"2024-04-26T21:14:11.913893+0000","last_became_active":"2024-04-26T21:14:10.930334+0000","last_became_peered":"2024-04-26T21:14:10.930334+0000","last_unstale":"2024-04-26T21:14:11.913893+0000","last_undegraded":"2024-04-26T21:14:11.913893+0000","last_fullsized":"2024-04-26T21:14:11.913893+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:39.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685466+0000","last_change":"2024-04-26T21:14:09.908596+0000","last_active":"2024-04-26T21:14:12.685466+0000","last_peered":"2024-04-26T21:14:12.685466+0000","last_clean":"2024-04-26T21:14:12.685466+0000","last_became_active":"2024-04-26T21:14:09.908271+0000","last_became_peered":"2024-04-26T21:14:09.908271+0000","last_unstale":"2024-04-26T21:14:12.685466+0000","last_undegraded":"2024-04-26T21:14:12.685466+0000","last_fullsized":"2024-04-26T21:14:12.685466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:03.641208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913020+0000","last_change":"2024-04-26T21:14:10.935980+0000","last_active":"2024-04-26T21:14:11.913020+0000","last_peered":"2024-04-26T21:14:11.913020+0000","last_clean":"2024-04-26T21:14:11.913020+0000","last_became_active":"2024-04-26T21:14:10.935666+0000","last_became_peered":"2024-04-26T21:14:10.935666+0000","last_unstale":"2024-04-26T21:14:11.913020+0000","last_undegraded":"2024-04-26T21:14:11.913020+0000","last_fullsized":"2024-04-26T21:14:11.913020+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:30.450958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914202+0000","last_change":"2024-04-26T21:14:09.909632+0000","last_active":"2024-04-26T21:14:11.914202+0000","last_peered":"2024-04-26T21:14:11.914202+0000","last_clean":"2024-04-26T21:14:11.914202+0000","last_became_active":"2024-04-26T21:14:09.909362+0000","last_became_peered":"2024-04-26T21:14:09.909362+0000","last_unstale":"2024-04-26T21:14:11.914202+0000","last_undegraded":"2024-04-26T21:14:11.914202+0000","last_fullsized":"2024-04-26T21:14:11.914202+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:53.216790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914037+0000","last_change":"2024-04-26T21:14:10.948347+0000","last_active":"2024-04-26T21:14:11.914037+0000","last_peered":"2024-04-26T21:14:11.914037+0000","last_clean":"2024-04-26T21:14:11.914037+0000","last_became_active":"2024-04-26T21:14:10.948088+0000","last_became_peered":"2024-04-26T21:14:10.948088+0000","last_unstale":"2024-04-26T21:14:11.914037+0000","last_undegraded":"2024-04-26T21:14:11.914037+0000","last_fullsized":"2024-04-26T21:14:11.914037+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:18.169012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911736+0000","last_change":"2024-04-26T21:14:09.900158+0000","last_active":"2024-04-26T21:14:11.911736+0000","last_peered":"2024-04-26T21:14:11.911736+0000","last_clean":"2024-04-26T21:14:11.911736+0000","last_became_active":"2024-04-26T21:14:09.899864+0000","last_became_peered":"2024-04-26T21:14:09.899864+0000","last_unstale":"2024-04-26T21:14:11.911736+0000","last_undegraded":"2024-04-26T21:14:11.911736+0000","last_fullsized":"2024-04-26T21:14:11.911736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:33:11.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684742+0000","last_change":"2024-04-26T21:14:10.943316+0000","last_active":"2024-04-26T21:14:12.684742+0000","last_peered":"2024-04-26T21:14:12.684742+0000","last_clean":"2024-04-26T21:14:12.684742+0000","last_became_active":"2024-04-26T21:14:10.943003+0000","last_became_peered":"2024-04-26T21:14:10.943003+0000","last_unstale":"2024-04-26T21:14:12.684742+0000","last_undegraded":"2024-04-26T21:14:12.684742+0000","last_fullsized":"2024-04-26T21:14:12.684742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:36.102742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913089+0000","last_change":"2024-04-26T21:14:09.900449+0000","last_active":"2024-04-26T21:14:11.913089+0000","last_peered":"2024-04-26T21:14:11.913089+0000","last_clean":"2024-04-26T21:14:11.913089+0000","last_became_active":"2024-04-26T21:14:09.900117+0000","last_became_peered":"2024-04-26T21:14:09.900117+0000","last_unstale":"2024-04-26T21:14:11.913089+0000","last_undegraded":"2024-04-26T21:14:11.913089+0000","last_fullsized":"2024-04-26T21:14:11.913089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:53.216790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684897+0000","last_change":"2024-04-26T21:14:10.935939+0000","last_active":"2024-04-26T21:14:12.684897+0000","last_peered":"2024-04-26T21:14:12.684897+0000","last_clean":"2024-04-26T21:14:12.684897+0000","last_became_active":"2024-04-26T21:14:10.935593+0000","last_became_peered":"2024-04-26T21:14:10.935593+0000","last_unstale":"2024-04-26T21:14:12.684897+0000","last_undegraded":"2024-04-26T21:14:12.684897+0000","last_fullsized":"2024-04-26T21:14:12.684897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.304369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913122+0000","last_change":"2024-04-26T21:14:09.905800+0000","last_active":"2024-04-26T21:14:11.913122+0000","last_peered":"2024-04-26T21:14:11.913122+0000","last_clean":"2024-04-26T21:14:11.913122+0000","last_became_active":"2024-04-26T21:14:09.905504+0000","last_became_peered":"2024-04-26T21:14:09.905504+0000","last_unstale":"2024-04-26T21:14:11.913122+0000","last_undegraded":"2024-04-26T21:14:11.913122+0000","last_fullsized":"2024-04-26T21:14:11.913122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:46.753087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912613+0000","last_change":"2024-04-26T21:14:10.933158+0000","last_active":"2024-04-26T21:14:11.912613+0000","last_peered":"2024-04-26T21:14:11.912613+0000","last_clean":"2024-04-26T21:14:11.912613+0000","last_became_active":"2024-04-26T21:14:10.932865+0000","last_became_peered":"2024-04-26T21:14:10.932865+0000","last_unstale":"2024-04-26T21:14:11.912613+0000","last_undegraded":"2024-04-26T21:14:11.912613+0000","last_fullsized":"2024-04-26T21:14:11.912613+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:51.619536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912718+0000","last_change":"2024-04-26T21:14:09.898468+0000","last_active":"2024-04-26T21:14:11.912718+0000","last_peered":"2024-04-26T21:14:11.912718+0000","last_clean":"2024-04-26T21:14:11.912718+0000","last_became_active":"2024-04-26T21:14:09.898160+0000","last_became_peered":"2024-04-26T21:14:09.898160+0000","last_unstale":"2024-04-26T21:14:11.912718+0000","last_undegraded":"2024-04-26T21:14:11.912718+0000","last_fullsized":"2024-04-26T21:14:11.912718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:21.504417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913830+0000","last_change":"2024-04-26T21:14:10.928754+0000","last_active":"2024-04-26T21:14:11.913830+0000","last_peered":"2024-04-26T21:14:11.913830+0000","last_clean":"2024-04-26T21:14:11.913830+0000","last_became_active":"2024-04-26T21:14:10.928383+0000","last_became_peered":"2024-04-26T21:14:10.928383+0000","last_unstale":"2024-04-26T21:14:11.913830+0000","last_undegraded":"2024-04-26T21:14:11.913830+0000","last_fullsized":"2024-04-26T21:14:11.913830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:46.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685090+0000","last_change":"2024-04-26T21:14:09.913657+0000","last_active":"2024-04-26T21:14:12.685090+0000","last_peered":"2024-04-26T21:14:12.685090+0000","last_clean":"2024-04-26T21:14:12.685090+0000","last_became_active":"2024-04-26T21:14:09.913317+0000","last_became_peered":"2024-04-26T21:14:09.913317+0000","last_unstale":"2024-04-26T21:14:12.685090+0000","last_undegraded":"2024-04-26T21:14:12.685090+0000","last_fullsized":"2024-04-26T21:14:12.685090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:00.688207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911700+0000","last_change":"2024-04-26T21:14:10.919213+0000","last_active":"2024-04-26T21:14:11.911700+0000","last_peered":"2024-04-26T21:14:11.911700+0000","last_clean":"2024-04-26T21:14:11.911700+0000","last_became_active":"2024-04-26T21:14:10.918921+0000","last_became_peered":"2024-04-26T21:14:10.918921+0000","last_unstale":"2024-04-26T21:14:11.911700+0000","last_undegraded":"2024-04-26T21:14:11.911700+0000","last_fullsized":"2024-04-26T21:14:11.911700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:04.642735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911878+0000","last_change":"2024-04-26T21:14:09.903462+0000","last_active":"2024-04-26T21:14:11.911878+0000","last_peered":"2024-04-26T21:14:11.911878+0000","last_clean":"2024-04-26T21:14:11.911878+0000","last_became_active":"2024-04-26T21:14:09.902981+0000","last_became_peered":"2024-04-26T21:14:09.902981+0000","last_unstale":"2024-04-26T21:14:11.911878+0000","last_undegraded":"2024-04-26T21:14:11.911878+0000","last_fullsized":"2024-04-26T21:14:11.911878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:32.454682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685357+0000","last_change":"2024-04-26T21:14:10.935101+0000","last_active":"2024-04-26T21:14:12.685357+0000","last_peered":"2024-04-26T21:14:12.685357+0000","last_clean":"2024-04-26T21:14:12.685357+0000","last_became_active":"2024-04-26T21:14:10.934820+0000","last_became_peered":"2024-04-26T21:14:10.934820+0000","last_unstale":"2024-04-26T21:14:12.685357+0000","last_undegraded":"2024-04-26T21:14:12.685357+0000","last_fullsized":"2024-04-26T21:14:12.685357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:39.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.916109+0000","last_change":"2024-04-26T21:14:09.899047+0000","last_active":"2024-04-26T21:14:11.916109+0000","last_peered":"2024-04-26T21:14:11.916109+0000","last_clean":"2024-04-26T21:14:11.916109+0000","last_became_active":"2024-04-26T21:14:09.898714+0000","last_became_peered":"2024-04-26T21:14:09.898714+0000","last_unstale":"2024-04-26T21:14:11.916109+0000","last_undegraded":"2024-04-26T21:14:11.916109+0000","last_fullsized":"2024-04-26T21:14:11.916109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:10.827803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684794+0000","last_change":"2024-04-26T21:14:10.932136+0000","last_active":"2024-04-26T21:14:12.684794+0000","last_peered":"2024-04-26T21:14:12.684794+0000","last_clean":"2024-04-26T21:14:12.684794+0000","last_became_active":"2024-04-26T21:14:10.931867+0000","last_became_peered":"2024-04-26T21:14:10.931867+0000","last_unstale":"2024-04-26T21:14:12.684794+0000","last_undegraded":"2024-04-26T21:14:12.684794+0000","last_fullsized":"2024-04-26T21:14:12.684794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:13.073855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913108+0000","last_change":"2024-04-26T21:14:09.906753+0000","last_active":"2024-04-26T21:14:11.913108+0000","last_peered":"2024-04-26T21:14:11.913108+0000","last_clean":"2024-04-26T21:14:11.913108+0000","last_became_active":"2024-04-26T21:14:09.906441+0000","last_became_peered":"2024-04-26T21:14:09.906441+0000","last_unstale":"2024-04-26T21:14:11.913108+0000","last_undegraded":"2024-04-26T21:14:11.913108+0000","last_fullsized":"2024-04-26T21:14:11.913108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:46.753087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684971+0000","last_change":"2024-04-26T21:14:10.941533+0000","last_active":"2024-04-26T21:14:12.684971+0000","last_peered":"2024-04-26T21:14:12.684971+0000","last_clean":"2024-04-26T21:14:12.684971+0000","last_became_active":"2024-04-26T21:14:10.941304+0000","last_became_peered":"2024-04-26T21:14:10.941304+0000","last_unstale":"2024-04-26T21:14:12.684971+0000","last_undegraded":"2024-04-26T21:14:12.684971+0000","last_fullsized":"2024-04-26T21:14:12.684971+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:39.042895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913127+0000","last_change":"2024-04-26T21:14:09.904804+0000","last_active":"2024-04-26T21:14:11.913127+0000","last_peered":"2024-04-26T21:14:11.913127+0000","last_clean":"2024-04-26T21:14:11.913127+0000","last_became_active":"2024-04-26T21:14:09.904380+0000","last_became_peered":"2024-04-26T21:14:09.904380+0000","last_unstale":"2024-04-26T21:14:11.913127+0000","last_undegraded":"2024-04-26T21:14:11.913127+0000","last_fullsized":"2024-04-26T21:14:11.913127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:23.479723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912760+0000","last_change":"2024-04-26T21:14:10.934581+0000","last_active":"2024-04-26T21:14:11.912760+0000","last_peered":"2024-04-26T21:14:11.912760+0000","last_clean":"2024-04-26T21:14:11.912760+0000","last_became_active":"2024-04-26T21:14:10.933414+0000","last_became_peered":"2024-04-26T21:14:10.933414+0000","last_unstale":"2024-04-26T21:14:11.912760+0000","last_undegraded":"2024-04-26T21:14:11.912760+0000","last_fullsized":"2024-04-26T21:14:11.912760+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:38.008369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913167+0000","last_change":"2024-04-26T21:14:09.904770+0000","last_active":"2024-04-26T21:14:11.913167+0000","last_peered":"2024-04-26T21:14:11.913167+0000","last_clean":"2024-04-26T21:14:11.913167+0000","last_became_active":"2024-04-26T21:14:09.904491+0000","last_became_peered":"2024-04-26T21:14:09.904491+0000","last_unstale":"2024-04-26T21:14:11.913167+0000","last_undegraded":"2024-04-26T21:14:11.913167+0000","last_fullsized":"2024-04-26T21:14:11.913167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:47.054894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684544+0000","last_change":"2024-04-26T21:14:10.928557+0000","last_active":"2024-04-26T21:14:12.684544+0000","last_peered":"2024-04-26T21:14:12.684544+0000","last_clean":"2024-04-26T21:14:12.684544+0000","last_became_active":"2024-04-26T21:14:10.927055+0000","last_became_peered":"2024-04-26T21:14:10.927055+0000","last_unstale":"2024-04-26T21:14:12.684544+0000","last_undegraded":"2024-04-26T21:14:12.684544+0000","last_fullsized":"2024-04-26T21:14:12.684544+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:58.092739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912896+0000","last_change":"2024-04-26T21:14:09.907449+0000","last_active":"2024-04-26T21:14:11.912896+0000","last_peered":"2024-04-26T21:14:11.912896+0000","last_clean":"2024-04-26T21:14:11.912896+0000","last_became_active":"2024-04-26T21:14:09.907159+0000","last_became_peered":"2024-04-26T21:14:09.907159+0000","last_unstale":"2024-04-26T21:14:11.912896+0000","last_undegraded":"2024-04-26T21:14:11.912896+0000","last_fullsized":"2024-04-26T21:14:11.912896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:40.185568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.911483+0000","last_change":"2024-04-26T21:14:10.929174+0000","last_active":"2024-04-26T21:14:11.911483+0000","last_peered":"2024-04-26T21:14:11.911483+0000","last_clean":"2024-04-26T21:14:11.911483+0000","last_became_active":"2024-04-26T21:14:10.928814+0000","last_became_peered":"2024-04-26T21:14:10.928814+0000","last_unstale":"2024-04-26T21:14:11.911483+0000","last_undegraded":"2024-04-26T21:14:11.911483+0000","last_fullsized":"2024-04-26T21:14:11.911483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:41.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685067+0000","last_change":"2024-04-26T21:14:09.907834+0000","last_active":"2024-04-26T21:14:12.685067+0000","last_peered":"2024-04-26T21:14:12.685067+0000","last_clean":"2024-04-26T21:14:12.685067+0000","last_became_active":"2024-04-26T21:14:09.907570+0000","last_became_peered":"2024-04-26T21:14:09.907570+0000","last_unstale":"2024-04-26T21:14:12.685067+0000","last_undegraded":"2024-04-26T21:14:12.685067+0000","last_fullsized":"2024-04-26T21:14:12.685067+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:35.640232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684819+0000","last_change":"2024-04-26T21:14:10.934459+0000","last_active":"2024-04-26T21:14:12.684819+0000","last_peered":"2024-04-26T21:14:12.684819+0000","last_clean":"2024-04-26T21:14:12.684819+0000","last_became_active":"2024-04-26T21:14:10.934076+0000","last_became_peered":"2024-04-26T21:14:10.934076+0000","last_unstale":"2024-04-26T21:14:12.684819+0000","last_undegraded":"2024-04-26T21:14:12.684819+0000","last_fullsized":"2024-04-26T21:14:12.684819+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:21.179012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914162+0000","last_change":"2024-04-26T21:14:09.905848+0000","last_active":"2024-04-26T21:14:11.914162+0000","last_peered":"2024-04-26T21:14:11.914162+0000","last_clean":"2024-04-26T21:14:11.914162+0000","last_became_active":"2024-04-26T21:14:09.905519+0000","last_became_peered":"2024-04-26T21:14:09.905519+0000","last_unstale":"2024-04-26T21:14:11.914162+0000","last_undegraded":"2024-04-26T21:14:11.914162+0000","last_fullsized":"2024-04-26T21:14:11.914162+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:18.029269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684929+0000","last_change":"2024-04-26T21:14:10.944555+0000","last_active":"2024-04-26T21:14:12.684929+0000","last_peered":"2024-04-26T21:14:12.684929+0000","last_clean":"2024-04-26T21:14:12.684929+0000","last_became_active":"2024-04-26T21:14:10.944192+0000","last_became_peered":"2024-04-26T21:14:10.944192+0000","last_unstale":"2024-04-26T21:14:12.684929+0000","last_undegraded":"2024-04-26T21:14:12.684929+0000","last_fullsized":"2024-04-26T21:14:12.684929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:31.438319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914737+0000","last_change":"2024-04-26T21:14:09.901751+0000","last_active":"2024-04-26T21:14:11.914737+0000","last_peered":"2024-04-26T21:14:11.914737+0000","last_clean":"2024-04-26T21:14:11.914737+0000","last_became_active":"2024-04-26T21:14:09.901432+0000","last_became_peered":"2024-04-26T21:14:09.901432+0000","last_unstale":"2024-04-26T21:14:11.914737+0000","last_undegraded":"2024-04-26T21:14:11.914737+0000","last_fullsized":"2024-04-26T21:14:11.914737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:20.177485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685036+0000","last_change":"2024-04-26T21:14:10.930260+0000","last_active":"2024-04-26T21:14:12.685036+0000","last_peered":"2024-04-26T21:14:12.685036+0000","last_clean":"2024-04-26T21:14:12.685036+0000","last_became_active":"2024-04-26T21:14:10.929969+0000","last_became_peered":"2024-04-26T21:14:10.929969+0000","last_unstale":"2024-04-26T21:14:12.685036+0000","last_undegraded":"2024-04-26T21:14:12.685036+0000","last_fullsized":"2024-04-26T21:14:12.685036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.481250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914293+0000","last_change":"2024-04-26T21:14:09.909717+0000","last_active":"2024-04-26T21:14:11.914293+0000","last_peered":"2024-04-26T21:14:11.914293+0000","last_clean":"2024-04-26T21:14:11.914293+0000","last_became_active":"2024-04-26T21:14:09.909423+0000","last_became_peered":"2024-04-26T21:14:09.909423+0000","last_unstale":"2024-04-26T21:14:11.914293+0000","last_undegraded":"2024-04-26T21:14:11.914293+0000","last_fullsized":"2024-04-26T21:14:11.914293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:47.054894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.914200+0000","last_change":"2024-04-26T21:14:10.931405+0000","last_active":"2024-04-26T21:14:11.914200+0000","last_peered":"2024-04-26T21:14:11.914200+0000","last_clean":"2024-04-26T21:14:11.914200+0000","last_became_active":"2024-04-26T21:14:10.931128+0000","last_became_peered":"2024-04-26T21:14:10.931128+0000","last_unstale":"2024-04-26T21:14:11.914200+0000","last_undegraded":"2024-04-26T21:14:11.914200+0000","last_fullsized":"2024-04-26T21:14:11.914200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:39.495997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684379+0000","last_change":"2024-04-26T21:14:09.907803+0000","last_active":"2024-04-26T21:14:12.684379+0000","last_peered":"2024-04-26T21:14:12.684379+0000","last_clean":"2024-04-26T21:14:12.684379+0000","last_became_active":"2024-04-26T21:14:09.907512+0000","last_became_peered":"2024-04-26T21:14:09.907512+0000","last_unstale":"2024-04-26T21:14:12.684379+0000","last_undegraded":"2024-04-26T21:14:12.684379+0000","last_fullsized":"2024-04-26T21:14:12.684379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:23.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912884+0000","last_change":"2024-04-26T21:14:10.947295+0000","last_active":"2024-04-26T21:14:11.912884+0000","last_peered":"2024-04-26T21:14:11.912884+0000","last_clean":"2024-04-26T21:14:11.912884+0000","last_became_active":"2024-04-26T21:14:10.946873+0000","last_became_peered":"2024-04-26T21:14:10.946873+0000","last_unstale":"2024-04-26T21:14:11.912884+0000","last_undegraded":"2024-04-26T21:14:11.912884+0000","last_fullsized":"2024-04-26T21:14:11.912884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:23.542292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913850+0000","last_change":"2024-04-26T21:14:09.909705+0000","last_active":"2024-04-26T21:14:11.913850+0000","last_peered":"2024-04-26T21:14:11.913850+0000","last_clean":"2024-04-26T21:14:11.913850+0000","last_became_active":"2024-04-26T21:14:09.909401+0000","last_became_peered":"2024-04-26T21:14:09.909401+0000","last_unstale":"2024-04-26T21:14:11.913850+0000","last_undegraded":"2024-04-26T21:14:11.913850+0000","last_fullsized":"2024-04-26T21:14:11.913850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912986+0000","last_change":"2024-04-26T21:14:10.940118+0000","last_active":"2024-04-26T21:14:11.912986+0000","last_peered":"2024-04-26T21:14:11.912986+0000","last_clean":"2024-04-26T21:14:11.912986+0000","last_became_active":"2024-04-26T21:14:10.935696+0000","last_became_peered":"2024-04-26T21:14:10.935696+0000","last_unstale":"2024-04-26T21:14:11.912986+0000","last_undegraded":"2024-04-26T21:14:11.912986+0000","last_fullsized":"2024-04-26T21:14:11.912986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:38.645661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915947+0000","last_change":"2024-04-26T21:14:09.900715+0000","last_active":"2024-04-26T21:14:11.915947+0000","last_peered":"2024-04-26T21:14:11.915947+0000","last_clean":"2024-04-26T21:14:11.915947+0000","last_became_active":"2024-04-26T21:14:09.900427+0000","last_became_peered":"2024-04-26T21:14:09.900427+0000","last_unstale":"2024-04-26T21:14:11.915947+0000","last_undegraded":"2024-04-26T21:14:11.915947+0000","last_fullsized":"2024-04-26T21:14:11.915947+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:35.101215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.912939+0000","last_change":"2024-04-26T21:14:10.928335+0000","last_active":"2024-04-26T21:14:11.912939+0000","last_peered":"2024-04-26T21:14:11.912939+0000","last_clean":"2024-04-26T21:14:11.912939+0000","last_became_active":"2024-04-26T21:14:10.928030+0000","last_became_peered":"2024-04-26T21:14:10.928030+0000","last_unstale":"2024-04-26T21:14:11.912939+0000","last_undegraded":"2024-04-26T21:14:11.912939+0000","last_fullsized":"2024-04-26T21:14:11.912939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:52.680017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.684553+0000","last_change":"2024-04-26T21:14:09.905590+0000","last_active":"2024-04-26T21:14:12.684553+0000","last_peered":"2024-04-26T21:14:12.684553+0000","last_clean":"2024-04-26T21:14:12.684553+0000","last_became_active":"2024-04-26T21:14:09.905255+0000","last_became_peered":"2024-04-26T21:14:09.905255+0000","last_unstale":"2024-04-26T21:14:12.684553+0000","last_undegraded":"2024-04-26T21:14:12.684553+0000","last_fullsized":"2024-04-26T21:14:12.684553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:08.088781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.915762+0000","last_change":"2024-04-26T21:14:10.929650+0000","last_active":"2024-04-26T21:14:11.915762+0000","last_peered":"2024-04-26T21:14:11.915762+0000","last_clean":"2024-04-26T21:14:11.915762+0000","last_became_active":"2024-04-26T21:14:10.929013+0000","last_became_peered":"2024-04-26T21:14:10.929013+0000","last_unstale":"2024-04-26T21:14:11.915762+0000","last_undegraded":"2024-04-26T21:14:11.915762+0000","last_fullsized":"2024-04-26T21:14:11.915762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:47.754614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685276+0000","last_change":"2024-04-26T21:14:09.902635+0000","last_active":"2024-04-26T21:14:12.685276+0000","last_peered":"2024-04-26T21:14:12.685276+0000","last_clean":"2024-04-26T21:14:12.685276+0000","last_became_active":"2024-04-26T21:14:09.902350+0000","last_became_peered":"2024-04-26T21:14:09.902350+0000","last_unstale":"2024-04-26T21:14:12.685276+0000","last_undegraded":"2024-04-26T21:14:12.685276+0000","last_fullsized":"2024-04-26T21:14:12.685276+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:14.176996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:12.685478+0000","last_change":"2024-04-26T21:14:10.932196+0000","last_active":"2024-04-26T21:14:12.685478+0000","last_peered":"2024-04-26T21:14:12.685478+0000","last_clean":"2024-04-26T21:14:12.685478+0000","last_became_active":"2024-04-26T21:14:10.931213+0000","last_became_peered":"2024-04-26T21:14:10.931213+0000","last_unstale":"2024-04-26T21:14:12.685478+0000","last_undegraded":"2024-04-26T21:14:12.685478+0000","last_fullsized":"2024-04-26T21:14:12.685478+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:09.886645+0000","last_clean_scrub_stamp":"2024-04-26T21:14:09.886645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:12.877154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913310+0000","last_change":"2024-04-26T21:14:09.906889+0000","last_active":"2024-04-26T21:14:11.913310+0000","last_peered":"2024-04-26T21:14:11.913310+0000","last_clean":"2024-04-26T21:14:11.913310+0000","last_became_active":"2024-04-26T21:14:09.906180+0000","last_became_peered":"2024-04-26T21:14:09.906180+0000","last_unstale":"2024-04-26T21:14:11.913310+0000","last_undegraded":"2024-04-26T21:14:11.913310+0000","last_fullsized":"2024-04-26T21:14:11.913310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:08.885118+0000","last_clean_scrub_stamp":"2024-04-26T21:14:08.885118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:18.029269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":213,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:14:11.913982+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:11.913982+0000","last_peered":"2024-04-26T21:14:11.913982+0000","last_clean":"2024-04-26T21:14:11.913982+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:11.913982+0000","last_undegraded":"2024-04-26T21:14:11.913982+0000","last_fullsized":"2024-04-26T21:14:11.913982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1142784,"data_stored":810729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":220943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":220341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":221883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":557056,"data_stored":220909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":221905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":804373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":220341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:14:15.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:14:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.430+0000 7fb227a45640 1 Processor -- start 2024-04-26T21:14:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 -- start start 2024-04-26T21:14:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2200b1ea0 0x7fb2200d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200b2580 con 0x7fb22015f6e0 2024-04-26T21:14:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb2200d5b10 con 0x7fb2200b1ea0 2024-04-26T21:14:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.431+0000 7fb227a45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200b2330 con 0x7fb2200d56a0 2024-04-26T21:14:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2200b1ea0 0x7fb2200d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb225fe2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb2257e1640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb22015f6e0 0x7fb2200d3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:36944/0 (socket says 172.21.15.40:36944) 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb2257e1640 1 -- 172.21.15.40:0/1504618121 learned_addr learned my addr 172.21.15.40:0/1504618121 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2200b1ea0 0x7fb2200d5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:55618/0 (socket says 172.21.15.40:55618) 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb225fe2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:37232/0 (socket says 172.21.15.40:37232) 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 -- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 msgr2=0x7fb2200b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 -- 172.21.15.40:0/1504618121 >> v1:172.21.15.184:6789/0 conn(0x7fb22015f6e0 legacy=0x7fb2200d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb224fe0640 1 -- 172.21.15.40:0/1504618121 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200b2860 con 0x7fb2200b1ea0 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb217fff640 1 -- 172.21.15.40:0/1504618121 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807303927 0 0) 0x7fb2200b2580 con 0x7fb22015f6e0 2024-04-26T21:14:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.432+0000 7fb225fe2640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.433+0000 7fb224fe0640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2200b1ea0 0x7fb2200d5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097a0 tx=0x7fb21002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d98c7b7bc728c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.433+0000 7fb217fff640 1 -- 172.21.15.40:0/1504618121 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210037070 con 0x7fb2200b1ea0 2024-04-26T21:14:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.433+0000 7fb217fff640 1 -- 172.21.15.40:0/1504618121 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210032440 con 0x7fb2200b1ea0 2024-04-26T21:14:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.433+0000 7fb217fff640 1 -- 172.21.15.40:0/1504618121 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210032810 con 0x7fb2200b1ea0 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 -- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2200b1ea0 msgr2=0x7fb2200d5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2200b1ea0 0x7fb2200d5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097a0 tx=0x7fb21002a180 comp rx=0 tx=0).stop 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 -- 172.21.15.40:0/1504618121 shutdown_connections 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200d56a0 0x7fb2200b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 --2- 172.21.15.40:0/1504618121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2200b1ea0 0x7fb2200d5160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 -- 172.21.15.40:0/1504618121 >> 172.21.15.40:0/1504618121 conn(0x7fb22008ec20 msgr2=0x7fb22009e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 -- 172.21.15.40:0/1504618121 shutdown_connections 2024-04-26T21:14:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.434+0000 7fb227a45640 1 -- 172.21.15.40:0/1504618121 wait complete. 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.435+0000 7fb227a45640 1 Processor -- start 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.435+0000 7fb227a45640 1 -- start start 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200b1ea0 0x7fb2200cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb22015f6e0 0x7fb220161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200b3b60 con 0x7fb2200d56a0 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200b2e20 con 0x7fb22015f6e0 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb227a45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200b3ce0 con 0x7fb2200b1ea0 2024-04-26T21:14:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60030/0 (socket says 172.21.15.40:60030) 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 -- 172.21.15.40:0/2836104090 learned_addr learned my addr 172.21.15.40:0/2836104090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb225fe2640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb22015f6e0 0x7fb220161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb2257e1640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200b1ea0 0x7fb2200cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 -- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200b1ea0 msgr2=0x7fb2200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200b1ea0 0x7fb2200cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 -- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb22015f6e0 msgr2=0x7fb220161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb22015f6e0 0x7fb220161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.436+0000 7fb224fe0640 1 -- 172.21.15.40:0/2836104090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200cb570 con 0x7fb2200d56a0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb224fe0640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097a0 tx=0x7fb2100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210037050 con 0x7fb2200d56a0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210038040 con 0x7fb2200d56a0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210004490 con 0x7fb2200d56a0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200ca4d0 con 0x7fb2200d56a0 2024-04-26T21:14:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.437+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200caa90 con 0x7fb2200d56a0 2024-04-26T21:14:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.438+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22006a290 con 0x7fb2200d56a0 2024-04-26T21:14:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.439+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb210003610 con 0x7fb2200d56a0 2024-04-26T21:14:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.439+0000 7fb215ffb640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.439+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fb2100ac640 con 0x7fb2200d56a0 2024-04-26T21:14:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.439+0000 7fb2257e1640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.440+0000 7fb2257e1640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 0x7fb20406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007fd0 tx=0x7fb21c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.443+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb21007a8c0 con 0x7fb2200d56a0 2024-04-26T21:14:15.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.697+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb22006e410 con 0x7fb2200d56a0 2024-04-26T21:14:15.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.699+0000 7fb215ffb640 1 -- 172.21.15.40:0/2836104090 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fb21007a1c0 con 0x7fb2200d56a0 2024-04-26T21:14:15.700 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:14:15.700 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:14:15.700 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:14:15.701 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:14:15.701 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:14:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 msgr2=0x7fb20406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 0x7fb20406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007fd0 tx=0x7fb21c007de0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 msgr2=0x7fb220154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097a0 tx=0x7fb2100046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 shutdown_connections 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb22015f6e0 0x7fb220161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2200d56a0 0x7fb220154250 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 --2- 172.21.15.40:0/2836104090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2200b1ea0 0x7fb2200cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 >> 172.21.15.40:0/2836104090 conn(0x7fb22008ec20 msgr2=0x7fb2200b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 shutdown_connections 2024-04-26T21:14:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.701+0000 7fb227a45640 1 -- 172.21.15.40:0/2836104090 wait complete. 2024-04-26T21:14:15.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.812+0000 7f5d33893640 1 Processor -- start 2024-04-26T21:14:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 -- start start 2024-04-26T21:14:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5d2c15d0c0 0x7f5d2c15fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d2c160200 0x7f5d2c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2c160780 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2c06f800 con 0x7f5d2c15d0c0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.813+0000 7f5d33893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c160a60 con 0x7f5d2c160200 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d31e30640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5d2c06f3c0 0x7f5d2c073100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34920/0 (socket says 172.21.15.40:34920) 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d31e30640 1 -- 172.21.15.40:0/2845822283 learned_addr learned my addr 172.21.15.40:0/2845822283 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814434401 0 0) 0x7f5d2c160780 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d18003520 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d2c160200 0x7f5d2c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> v2:172.21.15.184:3300/0 conn(0x7f5d2c15d0c0 0x7f5d2c15fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3496561925 0 0) 0x7f5d18003520 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d2c160200 msgr2=0x7f5d2c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d2c160200 0x7f5d2c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 >> v2:172.21.15.184:3300/0 conn(0x7f5d2c15d0c0 msgr2=0x7f5d2c15fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 --2- 172.21.15.40:0/2845822283 >> v2:172.21.15.184:3300/0 conn(0x7f5d2c15d0c0 0x7f5d2c15fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c0cff90 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5d240019f0 con 0x7f5d2c06f3c0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> v2:172.21.15.184:3300/0 conn(0x7f5d2c15d0c0 0x7f5d2c15fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 >> v1:172.21.15.40:6789/0 conn(0x7f5d2c06f3c0 legacy=0x7f5d2c073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d180039b0 0x7f5d18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d18004800 0x7f5d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180051c0 0x7f5d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c06f800 con 0x7f5d180039b0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c160a60 con 0x7f5d18004800 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1800c000 con 0x7f5d180051c0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d2c160200 0x7f5d2c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d180039b0 0x7f5d18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d31e30640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180051c0 0x7f5d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d18004800 0x7f5d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 -- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180051c0 msgr2=0x7f5d1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180051c0 0x7f5d1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 -- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d18004800 msgr2=0x7f5d18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.814+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d18004800 0x7f5d18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.815+0000 7f5d3162f640 1 -- 172.21.15.40:0/2845822283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d18003520 con 0x7f5d180039b0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.815+0000 7f5d3162f640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d180039b0 0x7f5d18003dd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098d0 tx=0x7f5d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6506614577cc7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.815+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800f040 con 0x7f5d180039b0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.815+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2802ad50 con 0x7f5d180039b0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.815+0000 7f5d23fff640 1 -- 172.21.15.40:0/2845822283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004250 con 0x7f5d180039b0 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 -- 172.21.15.40:0/2845822283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d180039b0 msgr2=0x7f5d18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 --2- 172.21.15.40:0/2845822283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d180039b0 0x7f5d18003dd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098d0 tx=0x7f5d18005830 comp rx=0 tx=0).stop 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d31e30640 1 -- 172.21.15.40:0/2845822283 reap_dead start 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 -- 172.21.15.40:0/2845822283 shutdown_connections 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 -- 172.21.15.40:0/2845822283 >> 172.21.15.40:0/2845822283 conn(0x7f5d2c0b3b10 msgr2=0x7f5d2c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 -- 172.21.15.40:0/2845822283 shutdown_connections 2024-04-26T21:14:15.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.816+0000 7f5d33893640 1 -- 172.21.15.40:0/2845822283 wait complete. 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.817+0000 7f5d33893640 1 Processor -- start 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.817+0000 7f5d33893640 1 -- start start 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 0x7f5d2c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d2c0b0540 0x7f5d2c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c15d4a0 con 0x7f5d2c0b6680 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c15c9b0 con 0x7f5d2c0b0540 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d33893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c15d620 con 0x7f5d180039b0 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60062/0 (socket says 172.21.15.40:60062) 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 -- 172.21.15.40:0/1550354778 learned_addr learned my addr 172.21.15.40:0/1550354778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d3162f640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 0x7f5d2c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d31e30640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d2c0b0540 0x7f5d2c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 -- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 msgr2=0x7f5d2c15b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 0x7f5d2c15b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 -- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d2c0b0540 msgr2=0x7f5d2c0b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d2c0b0540 0x7f5d2c0b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d30e2e640 1 -- 172.21.15.40:0/1550354778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c15bad0 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.818+0000 7f5d3162f640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 0x7f5d2c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d30e2e640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c010810 tx=0x7f5d1c010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00c930 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f040 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c09f980 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.819+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c0b5ba0 con 0x7f5d2c0b6680 2024-04-26T21:14:15.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.820+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2c15cc10 con 0x7f5d2c0b6680 2024-04-26T21:14:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.820+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d1c01d070 con 0x7f5d2c0b6680 2024-04-26T21:14:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.821+0000 7f5d21ffb640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.821+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f5d1c08bfe0 con 0x7f5d2c0b6680 2024-04-26T21:14:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.821+0000 7f5d3162f640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:15.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.821+0000 7f5d3162f640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 0x7f5d1006ede0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d28033000 tx=0x7f5d2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:15.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:15.825+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d2c15cc10 con 0x7f5d2c0b6680 2024-04-26T21:14:16.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.049+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d2c0693d0 con 0x7f5d2c0b6680 2024-04-26T21:14:16.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.050+0000 7f5d21ffb640 1 -- 172.21.15.40:0/1550354778 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7f5d1c05a4e0 con 0x7f5d2c0b6680 2024-04-26T21:14:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:16.051 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:16.051 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.051+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 msgr2=0x7f5d1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.051+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 0x7f5d1006ede0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d28033000 tx=0x7f5d2802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 msgr2=0x7f5d2c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c010810 tx=0x7f5d1c010cd0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 shutdown_connections 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d2c0b0540 0x7f5d2c0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d2c0b6680 0x7f5d2c0b6b00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 --2- 172.21.15.40:0/1550354778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d180039b0 0x7f5d2c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 >> 172.21.15.40:0/1550354778 conn(0x7f5d2c0b3b10 msgr2=0x7f5d2c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 shutdown_connections 2024-04-26T21:14:16.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.052+0000 7f5d33893640 1 -- 172.21.15.40:0/1550354778 wait complete. 2024-04-26T21:14:16.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:14:10.268813+0000', 'modified': '2024-04-26T21:14:13.276446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5607': {'gid': 5607, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/363892172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 363892172}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 363892172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5607, 'qdb_cluster': [5607]} 2024-04-26T21:14:16.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:14:16.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:14:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.571+0000 7fef70283640 1 Processor -- start 2024-04-26T21:14:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.571+0000 7fef70283640 1 -- start start 2024-04-26T21:14:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef70283640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef680d5610 0x7fef680b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef70283640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef680b2510 con 0x7fef6815f720 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef70283640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef680b27f0 con 0x7fef6815f340 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef70283640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef680b2ad0 con 0x7fef680d5610 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef680d5610 0x7fef680b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef680d5610 0x7fef680b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60070/0 (socket says 172.21.15.40:60070) 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 -- 172.21.15.40:0/2647149718 learned_addr learned my addr 172.21.15.40:0/2647149718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6d81e640 1 --1- 172.21.15.40:0/2647149718 >> v1:172.21.15.40:6789/0 conn(0x7fef6815f720 0x7fef680b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34932/0 (socket says 172.21.15.40:34932) 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 -- 172.21.15.40:0/2647149718 >> v1:172.21.15.184:6789/0 conn(0x7fef6815f340 legacy=0x7fef680d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 -- 172.21.15.40:0/2647149718 >> v1:172.21.15.40:6789/0 conn(0x7fef6815f720 legacy=0x7fef680b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.572+0000 7fef6e820640 1 -- 172.21.15.40:0/2647149718 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef680b2db0 con 0x7fef680d5610 2024-04-26T21:14:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.573+0000 7fef6e820640 1 --2- 172.21.15.40:0/2647149718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680b1e80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fef60009870 tx=0x7fef6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7742a1bb92077546 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.573+0000 7fef6d01d640 1 -- 172.21.15.40:0/2647149718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60037070 con 0x7fef680d5610 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.573+0000 7fef6d01d640 1 -- 172.21.15.40:0/2647149718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60032440 con 0x7fef680d5610 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.573+0000 7fef6d01d640 1 -- 172.21.15.40:0/2647149718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60032810 con 0x7fef680d5610 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 -- 172.21.15.40:0/2647149718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 msgr2=0x7fef680b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 --2- 172.21.15.40:0/2647149718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680b1e80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fef60009870 tx=0x7fef6002a210 comp rx=0 tx=0).stop 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 -- 172.21.15.40:0/2647149718 shutdown_connections 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 --2- 172.21.15.40:0/2647149718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680b1e80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 -- 172.21.15.40:0/2647149718 >> 172.21.15.40:0/2647149718 conn(0x7fef6808ec70 msgr2=0x7fef6809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 -- 172.21.15.40:0/2647149718 shutdown_connections 2024-04-26T21:14:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.574+0000 7fef70283640 1 -- 172.21.15.40:0/2647149718 wait complete. 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.575+0000 7fef70283640 1 Processor -- start 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.575+0000 7fef70283640 1 -- start start 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 0x7fef680b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 0x7fef680b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef680d40d0 con 0x7fef680d5610 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef680d3f50 con 0x7fef6815f720 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef70283640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef680b3cc0 con 0x7fef6815f340 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60074/0 (socket says 172.21.15.40:60074) 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e01f640 1 -- 172.21.15.40:0/2794016637 learned_addr learned my addr 172.21.15.40:0/2794016637 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e01f640 1 -- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 msgr2=0x7fef680b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 0x7fef680b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e820640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 0x7fef680b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.576+0000 7fef6e01f640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 0x7fef680b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6e01f640 1 -- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 msgr2=0x7fef680b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6e01f640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 0x7fef680b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6e01f640 1 -- 172.21.15.40:0/2794016637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef681f60f0 con 0x7fef680d5610 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 0x7fef680b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6e820640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 0x7fef680b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef6e01f640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c7c0 tx=0x7fef5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef66ffd640 1 -- 172.21.15.40:0/2794016637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58016020 con 0x7fef680d5610 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef66ffd640 1 -- 172.21.15.40:0/2794016637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5800aa50 con 0x7fef680d5610 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef66ffd640 1 -- 172.21.15.40:0/2794016637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58012420 con 0x7fef680d5610 2024-04-26T21:14:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef680cac00 con 0x7fef680d5610 2024-04-26T21:14:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.577+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef680c8eb0 con 0x7fef680d5610 2024-04-26T21:14:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.578+0000 7fef66ffd640 1 -- 172.21.15.40:0/2794016637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef58019020 con 0x7fef680d5610 2024-04-26T21:14:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.579+0000 7fef66ffd640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.579+0000 7fef66ffd640 1 -- 172.21.15.40:0/2794016637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fef5808b8d0 con 0x7fef680d5610 2024-04-26T21:14:16.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.579+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.580+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 0x7fef5406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef680cb6d0 tx=0x7fef5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.580+0000 7fef72fa1740 1 Processor -- start 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.580+0000 7fef72fa1740 1 -- start start 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb398820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb36e120 0x55acdb3b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55acdb2d2740 con 0x55acdb3b5ce0 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdb26e3a0 con 0x55acdb36e120 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55acdb2ce700 con 0x55acdb398440 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60084/0 (socket says 172.21.15.40:60084) 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 -- 172.21.15.40:0/2717599594 learned_addr learned my addr 172.21.15.40:0/2717599594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6e01f640 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb398820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6e820640 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb36e120 0x55acdb3b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 -- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 msgr2=0x55acdb398820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb398820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 -- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb36e120 msgr2=0x55acdb3b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.581+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb36e120 0x55acdb3b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.582+0000 7fef6d81e640 1 -- 172.21.15.40:0/2717599594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdb28a750 con 0x55acdb3b5ce0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.582+0000 7fef6d81e640 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fef5c007400 tx=0x7fef5c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c074bd056036a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.582+0000 7fef4ffff640 1 -- 172.21.15.40:0/2717599594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00d8b0 con 0x55acdb3b5ce0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.582+0000 7fef4ffff640 1 -- 172.21.15.40:0/2717599594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c00da50 con 0x55acdb3b5ce0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.582+0000 7fef4ffff640 1 -- 172.21.15.40:0/2717599594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00de00 con 0x55acdb3b5ce0 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- 172.21.15.40:0/2717599594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 msgr2=0x55acdb3b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fef5c007400 tx=0x7fef5c010000 comp rx=0 tx=0).stop 2024-04-26T21:14:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- 172.21.15.40:0/2717599594 shutdown_connections 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb36e120 0x55acdb3b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb3b5ce0 0x55acdb3b6230 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 --2- 172.21.15.40:0/2717599594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb398820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- 172.21.15.40:0/2717599594 >> 172.21.15.40:0/2717599594 conn(0x55acdb36aa50 msgr2=0x55acdb3b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- 172.21.15.40:0/2717599594 shutdown_connections 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- 172.21.15.40:0/2717599594 wait complete. 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 -- start start 2024-04-26T21:14:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.583+0000 7fef72fa1740 1 Processor -- start 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb39bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb3ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55acdb36eec0 con 0x55acdb398440 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdb259fe0 con 0x55acdb399e00 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55acdb368980 con 0x55acdb3b5ce0 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef72fa1740 10 client.0 _ll_register_callbacks cb 0x55acdb270150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60098/0 (socket says 172.21.15.40:60098) 2024-04-26T21:14:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.584+0000 7fef6e01f640 1 -- 172.21.15.40:0/1142942844 learned_addr learned my addr 172.21.15.40:0/1142942844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6d81e640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb39bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e820640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb3ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 -- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 msgr2=0x55acdb39bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb39bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 -- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 msgr2=0x55acdb3ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb3ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 -- 172.21.15.40:0/1142942844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdafe42d0 con 0x55acdb398440 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6d81e640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb39bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef6e01f640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fef58089df0 tx=0x7fef5805da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfc4c93b2204a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58088060 con 0x55acdb398440 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fef58061f40 con 0x55acdb398440 2024-04-26T21:14:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5c00da50 con 0x55acdb398440 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fef58071070 con 0x55acdb398440 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 10 client.5748 fetch_fsmap learned FSMap version 22 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55acdb0ae6b0 con 0x55acdb398440 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c00da50 con 0x55acdb398440 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fef58006b50 con 0x55acdb398440 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 10 client.5748 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 10 client.5748 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 20 client.5748 populate_metadata read hostname 'smithi040' 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb3aa570 0x55acdb3aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 4 client.5748 mds_command: new command op to 5607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.585+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 --> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acdb2e0350 con 0x55acdb3aa570 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.586+0000 7fef6d81e640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb3aa570 0x55acdb3aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.586+0000 7fef6d81e640 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb3aa570 0x55acdb3aec80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c71700985210723 server_cookie=1f28589aa7039e5 in_seq=0 out_seq=0 2024-04-26T21:14:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.586+0000 7fef4d7fa640 10 client.5748 ms_handle_connect on v2:172.21.15.40:6833/363892172 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.586+0000 7fef4d7fa640 1 -- 172.21.15.40:0/1142942844 <== mds.0 v2:172.21.15.40:6833/363892172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acdb2e0350 con 0x55acdb3aa570 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.586+0000 7fef4d7fa640 10 client.5748 handle_command_reply: tid=0 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 client.5748 shutdown 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 msgr2=0x55acdb394e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fef58089df0 tx=0x7fef5805da60 comp rx=0 tx=0).stop 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 shutdown_connections 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb3aa570 0x55acdb3aec80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb3ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb39bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef4d7fa640 0 client.5748 ms_handle_reset on v2:172.21.15.40:6833/363892172 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 --2- 172.21.15.40:0/1142942844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb394e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 >> 172.21.15.40:0/1142942844 conn(0x55acdb36e120 msgr2=0x55acdb3b7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 shutdown_connections 2024-04-26T21:14:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 1 -- 172.21.15.40:0/1142942844 wait complete. 2024-04-26T21:14:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.587+0000 7fef72fa1740 20 client.5748 trim_cache size 0 max 16384 2024-04-26T21:14:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.605+0000 7fef72fa1740 1 Processor -- start 2024-04-26T21:14:16.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.605+0000 7fef72fa1740 1 -- start start 2024-04-26T21:14:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb36ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb396c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef68200ad0 con 0x55acdb398440 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef681f8970 con 0x55acdb399e00 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef681f8fc0 con 0x55acdb3b5ce0 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60114/0 (socket says 172.21.15.40:60114) 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 -- 172.21.15.40:0/719566755 learned_addr learned my addr 172.21.15.40:0/719566755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 -- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 msgr2=0x55acdb396c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb396c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 -- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 msgr2=0x55acdb36ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb36ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 -- 172.21.15.40:0/719566755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdb364700 con 0x55acdb398440 2024-04-26T21:14:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.606+0000 7fef6e01f640 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fef58059580 tx=0x7fef5805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3609648ced2bf27c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef4f7fe640 1 -- 172.21.15.40:0/719566755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5807e070 con 0x55acdb398440 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef4f7fe640 1 -- 172.21.15.40:0/719566755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef58061d40 con 0x55acdb398440 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef4f7fe640 1 -- 172.21.15.40:0/719566755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58088040 con 0x55acdb398440 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 -- 172.21.15.40:0/719566755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 msgr2=0x55acdb36e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fef58059580 tx=0x7fef5805c000 comp rx=0 tx=0).stop 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 -- 172.21.15.40:0/719566755 shutdown_connections 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb3b5ce0 0x55acdb396c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb399e00 0x55acdb36ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 --2- 172.21.15.40:0/719566755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb398440 0x55acdb36e500 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 -- 172.21.15.40:0/719566755 >> 172.21.15.40:0/719566755 conn(0x55acdb36e120 msgr2=0x55acdb3ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 -- 172.21.15.40:0/719566755 shutdown_connections 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.607+0000 7fef72fa1740 1 -- 172.21.15.40:0/719566755 wait complete. 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.608+0000 7fef72fa1740 1 -- start start 2024-04-26T21:14:16.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.608+0000 7fef72fa1740 1 Processor -- start 2024-04-26T21:14:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb430b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb3b5ce0 0x55acdb3ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55acdb3aa3f0 con 0x55acdb399e00 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdb3af9a0 con 0x55acdb3b5ce0 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55acdb3aa980 con 0x55acdb398440 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 10 client.0 _ll_register_callbacks cb 0x7fef58058f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60120/0 (socket says 172.21.15.40:60120) 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 -- 172.21.15.40:0/4192235199 learned_addr learned my addr 172.21.15.40:0/4192235199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6e01f640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb430b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6e820640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb3b5ce0 0x55acdb3ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 -- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 msgr2=0x55acdb430b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb430b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 -- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb3b5ce0 msgr2=0x55acdb3ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb3b5ce0 0x55acdb3ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 -- 172.21.15.40:0/4192235199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdb397720 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6e01f640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb430b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef6d81e640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fef5c013000 tx=0x7fef5c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd913be0dda2e5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00da30 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fef5c00dbd0 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.609+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef58061d40 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fef5c03b050 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 10 client.5754 fetch_fsmap learned FSMap version 22 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55acdb3af6f0 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef58061d40 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fef5c03e020 con 0x55acdb399e00 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 10 client.5754 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 10 client.5754 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:14:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 20 client.5754 populate_metadata read hostname 'smithi040' 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb393600 0x55acdb3cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 4 client.5754 mds_command: new command op to 5607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 --> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] -- command(tid 0: {"prefix": "status"}) -- 0x55acdb3adf90 con 0x55acdb393600 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef6e01f640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb393600 0x55acdb3cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef6e01f640 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb393600 0x55acdb3cc770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9e8c90ddc6a926 server_cookie=f6a997d015ba699 in_seq=0 out_seq=0 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.610+0000 7fef4d7fa640 10 client.5754 ms_handle_connect on v2:172.21.15.40:6833/363892172 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef4d7fa640 1 -- 172.21.15.40:0/4192235199 <== mds.0 v2:172.21.15.40:6833/363892172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55acdb3adf90 con 0x55acdb393600 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef4d7fa640 10 client.5754 handle_command_reply: tid=0 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 client.5754 shutdown 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 msgr2=0x55acdb38e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fef5c013000 tx=0x7fef5c000ed0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 shutdown_connections 2024-04-26T21:14:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:6833/363892172,v1:172.21.15.40:6835/363892172] conn(0x55acdb393600 0x55acdb3cc770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdb3b5ce0 0x55acdb3ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55acdb399e00 0x55acdb38e8d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 --2- 172.21.15.40:0/4192235199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55acdb398440 0x55acdb430b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef4d7fa640 0 client.5754 ms_handle_reset on v2:172.21.15.40:6833/363892172 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 >> 172.21.15.40:0/4192235199 conn(0x55acdb36e120 msgr2=0x55acdb36e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 shutdown_connections 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 1 -- 172.21.15.40:0/4192235199 wait complete. 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef72fa1740 20 client.5754 trim_cache size 0 max 16384 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:14:16.618 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5607, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 5.7029137199999997, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 22, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 47, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 47, 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.747455413999999 2024-04-26T21:14:16.619 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 msgr2=0x7fef5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 0x7fef5406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef680cb6d0 tx=0x7fef5c009040 comp rx=0 tx=0).stop 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 msgr2=0x7fef680c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c7c0 tx=0x7fef5800cc80 comp rx=0 tx=0).stop 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.611+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 shutdown_connections 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef5406c8f0 0x7fef5406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef6815f720 0x7fef680b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef6815f340 0x7fef680b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 --2- 172.21.15.40:0/2794016637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef680d5610 0x7fef680c9990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 >> 172.21.15.40:0/2794016637 conn(0x7fef6808ec70 msgr2=0x7fef680b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 shutdown_connections 2024-04-26T21:14:16.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.612+0000 7fef70283640 1 -- 172.21.15.40:0/2794016637 wait complete. 2024-04-26T21:14:16.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.723+0000 7f6dcdd78640 1 Processor -- start 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.723+0000 7f6dcdd78640 1 -- start start 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 0x7f6dc8146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dc8140350 0x7f6dc81465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc80cf810 0x7f6dc80cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80d05b0 con 0x7f6dc8140350 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80d0890 con 0x7f6dc813ff70 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dcdd78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc806e450 con 0x7f6dc80cf810 2024-04-26T21:14:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dc8140350 0x7f6dc81465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dc8140350 0x7f6dc81465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60134/0 (socket says 172.21.15.40:60134) 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2269191721 learned_addr learned my addr 172.21.15.40:0/2269191721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc7fff640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc80cf810 0x7f6dc80cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc77fe640 1 --2- 172.21.15.40:0/2269191721 >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 0x7f6dc8146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2269191721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc80cf810 msgr2=0x7f6dc80cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc80cf810 0x7f6dc80cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2269191721 >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 msgr2=0x7f6dc8146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2269191721 >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 0x7f6dc8146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2269191721 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc806e730 con 0x7f6dc8140350 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.724+0000 7f6dc77fe640 1 --2- 172.21.15.40:0/2269191721 >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 0x7f6dc8146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc8140350 0x7f6dc81465c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8074200 tx=0x7f6dbc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8739bc05a20ff7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dc67fc640 1 -- 172.21.15.40:0/2269191721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc037070 con 0x7f6dc8140350 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dc67fc640 1 -- 172.21.15.40:0/2269191721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc032440 con 0x7f6dc8140350 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dc67fc640 1 -- 172.21.15.40:0/2269191721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc032810 con 0x7f6dc8140350 2024-04-26T21:14:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2269191721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc8140350 msgr2=0x7f6dc81465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.725+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc8140350 0x7f6dc81465c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8074200 tx=0x7f6dbc02a140 comp rx=0 tx=0).stop 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2269191721 shutdown_connections 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc80cf810 0x7f6dc80cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2269191721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc8140350 0x7f6dc81465c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2269191721 >> v2:172.21.15.184:3300/0 conn(0x7f6dc813ff70 0x7f6dc8146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2269191721 >> 172.21.15.40:0/2269191721 conn(0x7f6dc80d0d30 msgr2=0x7f6dc80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2269191721 shutdown_connections 2024-04-26T21:14:16.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.726+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2269191721 wait complete. 2024-04-26T21:14:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.727+0000 7f6dcdd78640 1 Processor -- start 2024-04-26T21:14:16.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.727+0000 7f6dcdd78640 1 -- start start 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dc80cf810 0x7f6dc813bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc8140350 0x7f6dc813dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8142a40 con 0x7f6dc813ff70 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81428c0 con 0x7f6dc80cf810 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc8142bc0 con 0x7f6dc8140350 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60138/0 (socket says 172.21.15.40:60138) 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2532462388 learned_addr learned my addr 172.21.15.40:0/2532462388 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc77fe640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dc80cf810 0x7f6dc813bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc7fff640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc8140350 0x7f6dc813dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc8140350 msgr2=0x7f6dc813dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc8140350 0x7f6dc813dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dc80cf810 msgr2=0x7f6dc813bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dc80cf810 0x7f6dc813bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 -- 172.21.15.40:0/2532462388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc813a110 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dc6ffd640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc041020 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0306c0 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc813a3f0 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.728+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc037040 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.729+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc813a9b0 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.730+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc806a290 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.730+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dbc044020 con 0x7f6dc813ff70 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.730+0000 7f6daffff640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 0x7f6db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:16.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.730+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f6dbc0ac220 con 0x7f6dc813ff70 2024-04-26T21:14:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.731+0000 7f6dc77fe640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 0x7f6db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:16.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.731+0000 7f6dc77fe640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 0x7f6db406edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6dc813e5d0 tx=0x7f6db8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:16.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.735+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dbc04b030 con 0x7f6dc813ff70 2024-04-26T21:14:16.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.961+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6dc813f4b0 con 0x7f6dc813ff70 2024-04-26T21:14:16.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.961+0000 7f6daffff640 1 -- 172.21.15.40:0/2532462388 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f6dbc07a090 con 0x7f6dc813ff70 2024-04-26T21:14:16.962 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 msgr2=0x7f6db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 0x7f6db406edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6dc813e5d0 tx=0x7f6db8008040 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 msgr2=0x7f6dc813d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 shutdown_connections 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6db406c8f0 0x7f6db406edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dc8140350 0x7f6dc813dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dc813ff70 0x7f6dc813d840 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 --2- 172.21.15.40:0/2532462388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dc80cf810 0x7f6dc813bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 >> 172.21.15.40:0/2532462388 conn(0x7f6dc80d0d30 msgr2=0x7f6dc80b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 shutdown_connections 2024-04-26T21:14:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:16.963+0000 7f6dcdd78640 1 -- 172.21.15.40:0/2532462388 wait complete. 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:14:16.975 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:16.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:16.976 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:14:16.991 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:14:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42613sec preferred_lft 42613sec 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:14:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:16.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:16.994 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:14:17.054 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:14:17.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:17.056 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:14:17.112 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:14:17.113 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:14:17.113 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:14:17.113 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:14:17.113 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:14:17.113 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:14:17.113 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:14:17.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:14:17.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:14:17.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:14:17.114 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:14:17.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:14:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:17.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:17.239 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:14:17.254 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:14:17.254 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:14:17.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:17.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:17.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:14:17.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:17.319 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:17.375 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:17.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:17.375 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:17.429 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:14:17.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:17.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:17.512 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:17.513 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:17.513 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:17.521 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:17.521 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:17.521 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:17.521 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:17.521 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:17.526 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:17.526 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:14:17.527 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:14:17.528 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:14:17.528 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:14:17.529 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:14:17.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:17.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:17.595 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:14:17.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:14:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:17.695 INFO:teuthology.orchestra.run.smithi040.stdout:client5069 2024-04-26T21:14:17.695 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:17.695 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:17.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:14:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.853+0000 7f6af899d640 1 Processor -- start 2024-04-26T21:14:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.853+0000 7f6af899d640 1 -- start start 2024-04-26T21:14:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af899d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6af4145980 0x7f6af4160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af899d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6af40720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af899d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af40b4260 con 0x7f6af40b3de0 2024-04-26T21:14:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af899d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6af40b5140 con 0x7f6af4145980 2024-04-26T21:14:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af899d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40b4530 con 0x7f6af40b61a0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6aebfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6af40b3de0 0x7f6af40b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34944/0 (socket says 172.21.15.40:34944) 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6aebfff640 1 -- 172.21.15.40:0/2104510150 learned_addr learned my addr 172.21.15.40:0/2104510150 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af259c640 1 --2- 172.21.15.40:0/2104510150 >> v2:172.21.15.184:3300/0 conn(0x7f6af4145980 0x7f6af4160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6af40720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983895153 0 0) 0x7f6af40b4260 con 0x7f6af40b3de0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ae0005730 con 0x7f6af40b3de0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3266573537 0 0) 0x7f6ae0005730 con 0x7f6af40b3de0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 msgr2=0x7f6af40720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6af40720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 >> v2:172.21.15.184:3300/0 conn(0x7f6af4145980 msgr2=0x7f6af4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 --2- 172.21.15.40:0/2104510150 >> v2:172.21.15.184:3300/0 conn(0x7f6af4145980 0x7f6af4160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af40b4810 con 0x7f6af40b3de0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f6adc001670 con 0x7f6af40b3de0 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af259c640 1 --2- 172.21.15.40:0/2104510150 >> v2:172.21.15.184:3300/0 conn(0x7f6af4145980 0x7f6af4160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.854+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6af40720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 >> v1:172.21.15.40:6789/0 conn(0x7f6af40b3de0 legacy=0x7f6af40b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ae0005bc0 0x7f6ae0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6ae0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae0006f50 0x7f6ae00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40b5140 con 0x7f6ae0006f50 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40b4530 con 0x7f6ae0005bc0 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae000dbd0 con 0x7f6af40b61a0 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae0006f50 0x7f6ae00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6aebfff640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6ae0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af259c640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ae0005bc0 0x7f6ae0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 -- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 msgr2=0x7f6ae0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6af40b61a0 0x7f6ae0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 -- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ae0005bc0 msgr2=0x7f6ae0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ae0005bc0 0x7f6ae0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.855+0000 7f6af2d9d640 1 -- 172.21.15.40:0/2104510150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae0005730 con 0x7f6ae0006f50 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae0006f50 0x7f6ae00073d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e350 tx=0x7f6ae0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bbef53dbcd72c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec033070 con 0x7f6ae0006f50 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec02f660 con 0x7f6ae0006f50 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af1d9b640 1 -- 172.21.15.40:0/2104510150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec009de0 con 0x7f6ae0006f50 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af899d640 1 -- 172.21.15.40:0/2104510150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae0006f50 msgr2=0x7f6ae00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af899d640 1 --2- 172.21.15.40:0/2104510150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ae0006f50 0x7f6ae00073d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e350 tx=0x7f6ae0007a40 comp rx=0 tx=0).stop 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af2d9d640 1 -- 172.21.15.40:0/2104510150 reap_dead start 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af899d640 1 -- 172.21.15.40:0/2104510150 shutdown_connections 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.856+0000 7f6af899d640 1 -- 172.21.15.40:0/2104510150 >> 172.21.15.40:0/2104510150 conn(0x7f6af40d0d30 msgr2=0x7f6af40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:17.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.857+0000 7f6af899d640 1 -- 172.21.15.40:0/2104510150 shutdown_connections 2024-04-26T21:14:17.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.857+0000 7f6af899d640 1 -- 172.21.15.40:0/2104510150 wait complete. 2024-04-26T21:14:17.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.858+0000 7f6af899d640 1 Processor -- start 2024-04-26T21:14:17.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.858+0000 7f6af899d640 1 -- start start 2024-04-26T21:14:17.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ae0006f50 0x7f6af413c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6af415b120 0x7f6af40af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6af406ebb0 con 0x7f6af40cf8f0 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40b4dd0 con 0x7f6af415b120 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6af406ed30 con 0x7f6ae0006f50 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60146/0 (socket says 172.21.15.40:60146) 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 -- 172.21.15.40:0/949739814 learned_addr learned my addr 172.21.15.40:0/949739814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af259c640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ae0006f50 0x7f6af413c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6aebfff640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6af415b120 0x7f6af40af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 -- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ae0006f50 msgr2=0x7f6af413c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ae0006f50 0x7f6af413c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 -- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6af415b120 msgr2=0x7f6af40af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6af415b120 0x7f6af40af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 -- 172.21.15.40:0/949739814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af40af7d0 con 0x7f6af40cf8f0 2024-04-26T21:14:17.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af2d9d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e470 tx=0x7f6aec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec033040 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec009a50 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af40afaa0 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec009de0 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.859+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40b0060 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.860+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af413cc50 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.861+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6aec036070 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.861+0000 7f6aeaffd640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.862+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f6aec0ac5e0 con 0x7f6af40cf8f0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.862+0000 7f6af259c640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.862+0000 7f6af259c640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 0x7f6ad406ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40093d0 tx=0x7f6ae4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:17.866+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6aec047060 con 0x7f6af40cf8f0 2024-04-26T21:14:18.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.092+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6af406a290 con 0x7f6af40cf8f0 2024-04-26T21:14:18.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.092+0000 7f6aeaffd640 1 -- 172.21.15.40:0/949739814 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f6aec07a3c0 con 0x7f6af40cf8f0 2024-04-26T21:14:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 msgr2=0x7f6ad406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 0x7f6ad406ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40093d0 tx=0x7f6ae4009190 comp rx=0 tx=0).stop 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 msgr2=0x7f6af40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e470 tx=0x7f6aec009750 comp rx=0 tx=0).stop 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 shutdown_connections 2024-04-26T21:14:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40cf8f0 0x7f6af40cfd70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6af415b120 0x7f6af40af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 --2- 172.21.15.40:0/949739814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ae0006f50 0x7f6af413c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 >> 172.21.15.40:0/949739814 conn(0x7f6af40d0d30 msgr2=0x7f6af40b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 shutdown_connections 2024-04-26T21:14:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:18.094+0000 7f6af899d640 1 -- 172.21.15.40:0/949739814 wait complete. 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:18.106 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:14:18.107 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:14:18.107 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:14:18.107 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.107 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:14:18.126 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:14:18.127 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:18.128 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:18.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.129 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:14:18.193 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:14:18.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.195 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:14:18.252 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:14:18.252 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:14:18.252 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:14:18.252 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:14:18.252 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:14:18.253 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:14:18.344 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:14:18.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:18.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.381 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:14:18.396 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:14:18.396 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:14:18.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:18.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:14:18.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:18.461 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:18.517 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:18.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:18.517 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:18.571 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:14:18.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.571 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:18.651 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:18.651 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:18.652 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:18.660 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:18.660 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:18.660 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:18.660 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:18.660 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:14:18.665 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:14:18.666 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:14:18.669 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:14:18.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:18.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:18.733 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:14:18.821 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:14:18.823 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:18.828 INFO:teuthology.orchestra.run.smithi184.stdout:client5769 2024-04-26T21:14:18.829 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:18.829 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:18.844 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:14:18.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:18.885 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:18.920 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:14:18.921 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:18.921 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:18.921 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:18.947 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:18.947 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:19.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.150+0000 7f8a54bc1640 1 Processor -- start 2024-04-26T21:14:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.150+0000 7f8a54bc1640 1 -- start start 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a54bc1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a500d11b0 0x7f8a5006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a54bc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5006e2c0 0x7f8a5015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a54bc1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a500d1740 con 0x7f8a500cf2e0 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a54bc1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5006e750 con 0x7f8a500d11b0 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a54bc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a500d1a20 con 0x7f8a5006e2c0 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a500d11b0 0x7f8a5006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a500cf2e0 0x7f8a5006d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34954/0 (socket says 172.21.15.40:34954) 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a500d11b0 0x7f8a5006dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60156/0 (socket says 172.21.15.40:60156) 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4e59c640 1 -- 172.21.15.40:0/2329081660 learned_addr learned my addr 172.21.15.40:0/2329081660 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4dd9b640 1 -- 172.21.15.40:0/2329081660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5006e2c0 msgr2=0x7f8a5015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.151+0000 7f8a4ed9d640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5006e2c0 0x7f8a5015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4dd9b640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5006e2c0 0x7f8a5015ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4dd9b640 1 -- 172.21.15.40:0/2329081660 >> v1:172.21.15.40:6789/0 conn(0x7f8a500cf2e0 legacy=0x7f8a5006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4d59a640 1 -- 172.21.15.40:0/2329081660 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735421910 0 0) 0x7f8a500d1740 con 0x7f8a500cf2e0 2024-04-26T21:14:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4dd9b640 1 -- 172.21.15.40:0/2329081660 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a500d1d00 con 0x7f8a500d11b0 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4dd9b640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a500d11b0 0x7f8a5006dd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a440097d0 tx=0x7f8a4402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaf31a5050d306a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4d59a640 1 -- 172.21.15.40:0/2329081660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44037070 con 0x7f8a500d11b0 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4d59a640 1 -- 172.21.15.40:0/2329081660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a44032440 con 0x7f8a500d11b0 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.152+0000 7f8a4d59a640 1 -- 172.21.15.40:0/2329081660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44032810 con 0x7f8a500d11b0 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2329081660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a500d11b0 msgr2=0x7f8a5006dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a500d11b0 0x7f8a5006dd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a440097d0 tx=0x7f8a4402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2329081660 shutdown_connections 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5006e2c0 0x7f8a5015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2329081660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a500d11b0 0x7f8a5006dd80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2329081660 >> 172.21.15.40:0/2329081660 conn(0x7f8a500b4a60 msgr2=0x7f8a500b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2329081660 shutdown_connections 2024-04-26T21:14:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.153+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2329081660 wait complete. 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.154+0000 7f8a54bc1640 1 Processor -- start 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 -- start start 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a500cf2e0 0x7f8a501eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 0x7f8a501494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500cf6c0 con 0x7f8a5006e2c0 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500d22c0 con 0x7f8a500d11b0 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a54bc1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a500cf840 con 0x7f8a500cf2e0 2024-04-26T21:14:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a4e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a4e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60172/0 (socket says 172.21.15.40:60172) 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.155+0000 7f8a4e59c640 1 -- 172.21.15.40:0/2096327441 learned_addr learned my addr 172.21.15.40:0/2096327441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4dd9b640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a500cf2e0 0x7f8a501eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4ed9d640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 0x7f8a501494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 -- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a500cf2e0 msgr2=0x7f8a501eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a500cf2e0 0x7f8a501eae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 -- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 msgr2=0x7f8a501494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 0x7f8a501494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 -- 172.21.15.40:0/2096327441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a500b8540 con 0x7f8a5006e2c0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4ed9d640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 0x7f8a501494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a4e59c640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c430 tx=0x7f8a4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400074e0 con 0x7f8a5006e2c0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a400047e0 con 0x7f8a5006e2c0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a500b8820 con 0x7f8a5006e2c0 2024-04-26T21:14:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40004bd0 con 0x7f8a5006e2c0 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.156+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501ea930 con 0x7f8a5006e2c0 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.157+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50075b50 con 0x7f8a5006e2c0 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.157+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a40011070 con 0x7f8a5006e2c0 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.158+0000 7f8a3effd640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 0x7f8a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.158+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f8a4008b810 con 0x7f8a5006e2c0 2024-04-26T21:14:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.158+0000 7f8a4dd9b640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 0x7f8a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.159+0000 7f8a4dd9b640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 0x7f8a3006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034000 tx=0x7f8a4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.162+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a50075b50 con 0x7f8a5006e2c0 2024-04-26T21:14:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.346+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a5006a290 con 0x7f8a5006e2c0 2024-04-26T21:14:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.347+0000 7f8a3effd640 1 -- 172.21.15.40:0/2096327441 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7f8a40059c40 con 0x7f8a5006e2c0 2024-04-26T21:14:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:19.349 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 msgr2=0x7f8a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 0x7f8a3006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034000 tx=0x7f8a4402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 msgr2=0x7f8a500b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c430 tx=0x7f8a4000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 shutdown_connections 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a3006c8f0 0x7f8a3006edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a500d11b0 0x7f8a501494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a500cf2e0 0x7f8a501eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 --2- 172.21.15.40:0/2096327441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5006e2c0 0x7f8a500b5790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 >> 172.21.15.40:0/2096327441 conn(0x7f8a500b4a60 msgr2=0x7f8a50070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 shutdown_connections 2024-04-26T21:14:19.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.349+0000 7f8a54bc1640 1 -- 172.21.15.40:0/2096327441 wait complete. 2024-04-26T21:14:19.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:08.531061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:09.214001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:19.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.463+0000 7f2c078d1640 1 Processor -- start 2024-04-26T21:14:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 -- start start 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c000d0300 0x7f2c000b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 0x7f2c000720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c000b46a0 con 0x7f2c0006f340 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2c000b5db0 con 0x7f2c000d0300 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.464+0000 7f2c078d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c000b6090 con 0x7f2c000b3e50 2024-04-26T21:14:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c000d0300 0x7f2c000b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c000d0300 0x7f2c000b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60176/0 (socket says 172.21.15.40:60176) 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 -- 172.21.15.40:0/3085424403 learned_addr learned my addr 172.21.15.40:0/3085424403 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c05e6e640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 0x7f2c000720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 -- 172.21.15.40:0/3085424403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 msgr2=0x7f2c000720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 0x7f2c000720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 -- 172.21.15.40:0/3085424403 >> v1:172.21.15.184:6789/0 conn(0x7f2c0006f340 legacy=0x7f2c0006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c04e6c640 1 -- 172.21.15.40:0/3085424403 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c000b6370 con 0x7f2c000d0300 2024-04-26T21:14:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.465+0000 7f2c05e6e640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 0x7f2c000720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:19.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c000d0300 0x7f2c000b6b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf402c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cedb2417eef77f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2bfffff640 1 -- 172.21.15.40:0/3085424403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4039070 con 0x7f2c000d0300 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2bfffff640 1 -- 172.21.15.40:0/3085424403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4034640 con 0x7f2c000d0300 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2bfffff640 1 -- 172.21.15.40:0/3085424403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4034a10 con 0x7f2c000d0300 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 -- 172.21.15.40:0/3085424403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c000d0300 msgr2=0x7f2c000b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c000d0300 0x7f2c000b6b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf402c340 comp rx=0 tx=0).stop 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 -- 172.21.15.40:0/3085424403 shutdown_connections 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000b3e50 0x7f2c000720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 --2- 172.21.15.40:0/3085424403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c000d0300 0x7f2c000b6b70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 -- 172.21.15.40:0/3085424403 >> 172.21.15.40:0/3085424403 conn(0x7f2c000cd8a0 msgr2=0x7f2c000cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 -- 172.21.15.40:0/3085424403 shutdown_connections 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.466+0000 7f2c078d1640 1 -- 172.21.15.40:0/3085424403 wait complete. 2024-04-26T21:14:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 Processor -- start 2024-04-26T21:14:19.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 -- start start 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 0x7f2c001f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 0x7f2c001f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000b4460 con 0x7f2c0006f340 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000b42e0 con 0x7f2c000b3e50 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c078d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c00142a30 con 0x7f2c000d0300 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c0566d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c0566d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60192/0 (socket says 172.21.15.40:60192) 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.468+0000 7f2c0566d640 1 -- 172.21.15.40:0/1869206448 learned_addr learned my addr 172.21.15.40:0/1869206448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 0x7f2c001f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c05e6e640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 0x7f2c001f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 -- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 msgr2=0x7f2c001f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 0x7f2c001f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 -- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 msgr2=0x7f2c001f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 0x7f2c001f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 -- 172.21.15.40:0/1869206448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0015f410 con 0x7f2c0006f340 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 0x7f2c001f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c05e6e640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 0x7f2c001f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c0566d640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c730 tx=0x7f2bf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0016020 con 0x7f2c0006f340 2024-04-26T21:14:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf0006d50 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c00151e40 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0003bb0 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.469+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c00152400 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.470+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0006a290 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.471+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf000a850 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.471+0000 7f2bfdffb640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.471+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2bf008b7a0 con 0x7f2c0006f340 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.471+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.472+0000 7f2c04e6c640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 0x7f2bec06edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4036000 tx=0x7f2bf4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.475+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bf0020020 con 0x7f2c0006f340 2024-04-26T21:14:19.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.660+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c00151c10 con 0x7f2c0006f340 2024-04-26T21:14:19.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.661+0000 7f2bfdffb640 1 -- 172.21.15.40:0/1869206448 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f2bf0019310 con 0x7f2c0006f340 2024-04-26T21:14:19.662 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 msgr2=0x7f2bec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 0x7f2bec06edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4036000 tx=0x7f2bf4030000 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 msgr2=0x7f2c000ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c730 tx=0x7f2bf000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 shutdown_connections 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c000d0300 0x7f2c001f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c000b3e50 0x7f2c001f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 --2- 172.21.15.40:0/1869206448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c0006f340 0x7f2c000ca340 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 >> 172.21.15.40:0/1869206448 conn(0x7f2c000cd8a0 msgr2=0x7f2c00143210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 shutdown_connections 2024-04-26T21:14:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.663+0000 7f2c078d1640 1 -- 172.21.15.40:0/1869206448 wait complete. 2024-04-26T21:14:19.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:14:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.777+0000 7f2bc235f640 1 Processor -- start 2024-04-26T21:14:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.777+0000 7f2bc235f640 1 -- start start 2024-04-26T21:14:19.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc235f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc235f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2bbc0ceff0 0x7f2bbc0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc235f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bbc15dca0 con 0x7f2bbc143840 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc235f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc143c20 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc235f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc15df80 con 0x7f2bbc0b4eb0 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc08fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2bbc0ceff0 0x7f2bbc0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bbbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2bbc143840 0x7f2bbc0b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34970/0 (socket says 172.21.15.40:34970) 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bbbfff640 1 -- 172.21.15.40:0/1626737316 learned_addr learned my addr 172.21.15.40:0/1626737316 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bbb7fe640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc08fc640 1 -- 172.21.15.40:0/1626737316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 msgr2=0x7f2bbc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc08fc640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc08fc640 1 -- 172.21.15.40:0/1626737316 >> v1:172.21.15.40:6789/0 conn(0x7f2bbc143840 legacy=0x7f2bbc0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bc08fc640 1 -- 172.21.15.40:0/1626737316 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc15e260 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bbaffd640 1 -- 172.21.15.40:0/1626737316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268094865 0 0) 0x7f2bbc15dca0 con 0x7f2bbc143840 2024-04-26T21:14:19.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.778+0000 7f2bbb7fe640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.779+0000 7f2bc08fc640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0cf450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009d90 tx=0x7f2bb402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1009e733342c3fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.779+0000 7f2bbaffd640 1 -- 172.21.15.40:0/1626737316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4007740 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.779+0000 7f2bbaffd640 1 -- 172.21.15.40:0/1626737316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb40078e0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.779+0000 7f2bbaffd640 1 -- 172.21.15.40:0/1626737316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4007cb0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 -- 172.21.15.40:0/1626737316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 msgr2=0x7f2bbc0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0cf450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009d90 tx=0x7f2bb402a660 comp rx=0 tx=0).stop 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 -- 172.21.15.40:0/1626737316 shutdown_connections 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0cf450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 --2- 172.21.15.40:0/1626737316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 -- 172.21.15.40:0/1626737316 >> 172.21.15.40:0/1626737316 conn(0x7f2bbc0cd8a0 msgr2=0x7f2bbc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 -- 172.21.15.40:0/1626737316 shutdown_connections 2024-04-26T21:14:19.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.780+0000 7f2bc235f640 1 -- 172.21.15.40:0/1626737316 wait complete. 2024-04-26T21:14:19.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.781+0000 7f2bc235f640 1 Processor -- start 2024-04-26T21:14:19.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.781+0000 7f2bc235f640 1 -- start start 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc143840 0x7f2bbc135130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc15d8c0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc0b46f0 con 0x7f2bbc143840 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc235f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc15da40 con 0x7f2bbc0b4eb0 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60214/0 (socket says 172.21.15.40:60214) 2024-04-26T21:14:19.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 -- 172.21.15.40:0/3207538119 learned_addr learned my addr 172.21.15.40:0/3207538119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbbfff640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bc08fc640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc143840 0x7f2bbc135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 -- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 msgr2=0x7f2bbc15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc15a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 -- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc143840 msgr2=0x7f2bbc135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc143840 0x7f2bbc135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 -- 172.21.15.40:0/3207538119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc0aeba0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.782+0000 7f2bbb7fe640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e7e0 tx=0x7f2bac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.783+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f040 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.783+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bac004030 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.783+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc0aee80 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.783+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac004400 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.783+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc0af440 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.784+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc06a290 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.784+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bac00c6d0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.785+0000 7f2bb8ff9640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 0x7f2ba806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:19.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.785+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2bac08b860 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.785+0000 7f2bbbfff640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 0x7f2ba806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:19.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.785+0000 7f2bbbfff640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 0x7f2ba806ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0007fd0 tx=0x7f2bb0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:19.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.789+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bac017020 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.976+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2bbc0ce420 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.976+0000 7f2bb8ff9640 1 -- 172.21.15.40:0/3207538119 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f2bac08b0a0 con 0x7f2bbc0ceff0 2024-04-26T21:14:19.977 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:14:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 msgr2=0x7f2ba806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 0x7f2ba806ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0007fd0 tx=0x7f2bb0007de0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 msgr2=0x7f2bbc0ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e7e0 tx=0x7f2bac00eca0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 shutdown_connections 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2ba806c920 0x7f2ba806ede0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc143840 0x7f2bbc135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bbc0ceff0 0x7f2bbc0ae150 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 --2- 172.21.15.40:0/3207538119 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc0b4eb0 0x7f2bbc15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 >> 172.21.15.40:0/3207538119 conn(0x7f2bbc0cd8a0 msgr2=0x7f2bbc0b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 shutdown_connections 2024-04-26T21:14:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:19.978+0000 7f2bc235f640 1 -- 172.21.15.40:0/3207538119 wait complete. 2024-04-26T21:14:19.990 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:14:20.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.092+0000 7f3b686b0640 1 Processor -- start 2024-04-26T21:14:20.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.092+0000 7f3b686b0640 1 -- start start 2024-04-26T21:14:20.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b686b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3b6006ea30 0x7f3b600d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b686b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b6009c790 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b686b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b60163ec0 con 0x7f3b601631f0 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b686b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3b6006eec0 con 0x7f3b6006ea30 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b65c4b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b600b2000 0x7f3b600b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34982/0 (socket says 172.21.15.40:34982) 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b65c4b640 1 -- 172.21.15.40:0/3355117587 learned_addr learned my addr 172.21.15.40:0/3355117587 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/3355117587 >> v2:172.21.15.184:3300/0 conn(0x7f3b6006ea30 0x7f3b600d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270703247 0 0) 0x7f3b6009c790 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b4c003520 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2529723991 0 0) 0x7f3b4c003520 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 >> v2:172.21.15.184:3300/0 conn(0x7f3b6006ea30 msgr2=0x7f3b600d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 --2- 172.21.15.40:0/3355117587 >> v2:172.21.15.184:3300/0 conn(0x7f3b6006ea30 0x7f3b600d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 >> v1:172.21.15.184:6789/0 conn(0x7f3b601631f0 legacy=0x7f3b601635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600d1a40 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3b50001670 con 0x7f3b600b2000 2024-04-26T21:14:20.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.093+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 >> v1:172.21.15.40:6789/0 conn(0x7f3b600b2000 legacy=0x7f3b600b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0039b0 0x7f3b4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b4c004800 0x7f3b4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c00c130 con 0x7f3b4c0051c0 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c00c2e0 con 0x7f3b4c004800 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c00c4e0 con 0x7f3b4c0039b0 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b65c4b640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b4c004800 0x7f3b4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b6644c640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0039b0 0x7f3b4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 -- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0039b0 msgr2=0x7f3b4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0039b0 0x7f3b4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 -- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b4c004800 msgr2=0x7f3b4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b4c004800 0x7f3b4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.094+0000 7f3b66c4d640 1 -- 172.21.15.40:0/3355117587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c003520 con 0x7f3b4c0051c0 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b66c4d640 1 -- 172.21.15.40:0/3355117587 reap_dead start 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00b9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009fc0 tx=0x7f3b4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3307fa02553e43de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58035020 con 0x7f3b4c0051c0 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b58003bd0 con 0x7f3b4c0051c0 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b6544a640 1 -- 172.21.15.40:0/3355117587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58009c10 con 0x7f3b4c0051c0 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b686b0640 1 -- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 msgr2=0x7f3b4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.095+0000 7f3b686b0640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00b9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009fc0 tx=0x7f3b4c005830 comp rx=0 tx=0).stop 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.096+0000 7f3b686b0640 1 -- 172.21.15.40:0/3355117587 shutdown_connections 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.096+0000 7f3b686b0640 1 --2- 172.21.15.40:0/3355117587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00b9c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.096+0000 7f3b686b0640 1 -- 172.21.15.40:0/3355117587 >> 172.21.15.40:0/3355117587 conn(0x7f3b6009ba40 msgr2=0x7f3b6009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.096+0000 7f3b686b0640 1 -- 172.21.15.40:0/3355117587 shutdown_connections 2024-04-26T21:14:20.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.096+0000 7f3b686b0640 1 -- 172.21.15.40:0/3355117587 wait complete. 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 Processor -- start 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 -- start start 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0051c0 0x7f3b6015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b60154540 0x7f3b601f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600d0230 con 0x7f3b60158db0 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600d2000 con 0x7f3b60154540 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.097+0000 7f3b686b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b60159b70 con 0x7f3b4c0051c0 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60242/0 (socket says 172.21.15.40:60242) 2024-04-26T21:14:20.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 -- 172.21.15.40:0/4280531036 learned_addr learned my addr 172.21.15.40:0/4280531036 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b65c4b640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b60154540 0x7f3b601f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b6644c640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0051c0 0x7f3b6015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 -- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0051c0 msgr2=0x7f3b6015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0051c0 0x7f3b6015afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 -- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b60154540 msgr2=0x7f3b601f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b60154540 0x7f3b601f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 -- 172.21.15.40:0/4280531036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b601549c0 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b66c4d640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e230 tx=0x7f3b58009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58035070 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b58041070 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b601f60e0 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5802d070 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.098+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b601f6660 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.099+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6006a290 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.100+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b58005d10 con 0x7f3b60158db0 2024-04-26T21:14:20.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.100+0000 7f3b56ffd640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 0x7f3b4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.101+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3b580ac450 con 0x7f3b60158db0 2024-04-26T21:14:20.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.101+0000 7f3b6644c640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 0x7f3b4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.101+0000 7f3b6644c640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 0x7f3b4406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c000c00 tx=0x7f3b5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.105+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b601f60e0 con 0x7f3b60158db0 2024-04-26T21:14:20.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.290+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b600b8f10 con 0x7f3b60158db0 2024-04-26T21:14:20.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.291+0000 7f3b56ffd640 1 -- 172.21.15.40:0/4280531036 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f3b5807a190 con 0x7f3b60158db0 2024-04-26T21:14:20.292 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 msgr2=0x7f3b4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 0x7f3b4406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c000c00 tx=0x7f3b5c009040 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 msgr2=0x7f3b60159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e230 tx=0x7f3b58009750 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 shutdown_connections 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3b4406c920 0x7f3b4406ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b60158db0 0x7f3b60159260 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b60154540 0x7f3b601f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 --2- 172.21.15.40:0/4280531036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b4c0051c0 0x7f3b6015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 >> 172.21.15.40:0/4280531036 conn(0x7f3b6009ba40 msgr2=0x7f3b600b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:20.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 shutdown_connections 2024-04-26T21:14:20.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.293+0000 7f3b686b0640 1 -- 172.21.15.40:0/4280531036 wait complete. 2024-04-26T21:14:20.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:14:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.406+0000 7f2863209640 1 Processor -- start 2024-04-26T21:14:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.406+0000 7f2863209640 1 -- start start 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f2863209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f2863209640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f285c0b46f0 0x7f285c0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f2863209640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c0d0190 con 0x7f285c15d050 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f2863209640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f285c068530 con 0x7f285c0b46f0 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f2863209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0d0430 con 0x7f285c068090 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f285c0b46f0 0x7f285c0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f285c0b46f0 0x7f285c0b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60252/0 (socket says 172.21.15.40:60252) 2024-04-26T21:14:20.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 -- 172.21.15.40:0/1303182902 learned_addr learned my addr 172.21.15.40:0/1303182902 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f285bfff640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 -- 172.21.15.40:0/1303182902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 msgr2=0x7f285c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 -- 172.21.15.40:0/1303182902 >> v1:172.21.15.184:6789/0 conn(0x7f285c15d050 legacy=0x7f285c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f285b7fe640 1 -- 172.21.15.40:0/1303182902 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894169844 0 0) 0x7f285c0d0190 con 0x7f285c15d050 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.407+0000 7f28617a6640 1 -- 172.21.15.40:0/1303182902 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0d0710 con 0x7f285c0b46f0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.408+0000 7f28617a6640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c0b4ba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2854009e50 tx=0x7f285402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146342e62feda0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.408+0000 7f285b7fe640 1 -- 172.21.15.40:0/1303182902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854036070 con 0x7f285c0b46f0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.408+0000 7f285b7fe640 1 -- 172.21.15.40:0/1303182902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854031600 con 0x7f285c0b46f0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.408+0000 7f285b7fe640 1 -- 172.21.15.40:0/1303182902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540319d0 con 0x7f285c0b46f0 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.408+0000 7f2863209640 1 -- 172.21.15.40:0/1303182902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 msgr2=0x7f285c0b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c0b4ba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2854009e50 tx=0x7f285402a710 comp rx=0 tx=0).stop 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 -- 172.21.15.40:0/1303182902 shutdown_connections 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c0b4ba0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 --2- 172.21.15.40:0/1303182902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 -- 172.21.15.40:0/1303182902 >> 172.21.15.40:0/1303182902 conn(0x7f285c072900 msgr2=0x7f285c072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 -- 172.21.15.40:0/1303182902 shutdown_connections 2024-04-26T21:14:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.409+0000 7f2863209640 1 -- 172.21.15.40:0/1303182902 wait complete. 2024-04-26T21:14:20.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.410+0000 7f2863209640 1 Processor -- start 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.410+0000 7f2863209640 1 -- start start 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f285c15d050 0x7f285c1e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f285c15d430 con 0x7f285c0b46f0 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f285c073af0 con 0x7f285c15d050 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0d1340 con 0x7f285c068090 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60262/0 (socket says 172.21.15.40:60262) 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 -- 172.21.15.40:0/548748983 learned_addr learned my addr 172.21.15.40:0/548748983 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2860fa5640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f28617a6640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f285c15d050 0x7f285c1e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 -- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 msgr2=0x7f285c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 -- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f285c15d050 msgr2=0x7f285c1e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f285c15d050 0x7f285c1e8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 -- 172.21.15.40:0/548748983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c1487a0 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f285bfff640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9e0 tx=0x7f284400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844017070 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284400c9d0 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.411+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c06f750 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.412+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400cdc0 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.412+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c06fd10 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.412+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c0693d0 con 0x7f285c0b46f0 2024-04-26T21:14:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.413+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f284401a070 con 0x7f285c0b46f0 2024-04-26T21:14:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.413+0000 7f28597fa640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.413+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f284408bf80 con 0x7f285c0b46f0 2024-04-26T21:14:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.414+0000 7f2860fa5640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.414+0000 7f2860fa5640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 0x7f283c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f284c000f60 tx=0x7f284c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.418+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f284405a330 con 0x7f285c0b46f0 2024-04-26T21:14:20.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.615+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f285c06d700 con 0x7f285c0b46f0 2024-04-26T21:14:20.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.615+0000 7f28597fa640 1 -- 172.21.15.40:0/548748983 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f2844059c30 con 0x7f285c0b46f0 2024-04-26T21:14:20.616 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:14:20.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 msgr2=0x7f283c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 0x7f283c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f284c000f60 tx=0x7f284c009040 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 msgr2=0x7f285c1585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9e0 tx=0x7f284400eea0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 shutdown_connections 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f285c15d050 0x7f285c1e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f285c0b46f0 0x7f285c1585b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 --2- 172.21.15.40:0/548748983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f285c068090 0x7f285c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 >> 172.21.15.40:0/548748983 conn(0x7f285c072900 msgr2=0x7f285c15c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 shutdown_connections 2024-04-26T21:14:20.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.617+0000 7f2863209640 1 -- 172.21.15.40:0/548748983 wait complete. 2024-04-26T21:14:20.629 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:14:20.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:20.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:20.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:14:20.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:20.657 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:14:20.713 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:20.713 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:20.713 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:20.732 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:20.732 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:20.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.891+0000 7f4e26e93640 1 Processor -- start 2024-04-26T21:14:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.891+0000 7f4e26e93640 1 -- start start 2024-04-26T21:14:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e26e93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e2015f370 0x7f4e200cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e26e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e200d03a0 con 0x7f4e200d3ee0 2024-04-26T21:14:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e26e93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e200d0680 con 0x7f4e2015f750 2024-04-26T21:14:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e26e93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4e200d0960 con 0x7f4e2015f370 2024-04-26T21:14:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e25430640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4e200d3ee0 0x7f4e200b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34984/0 (socket says 172.21.15.40:34984) 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e25430640 1 -- 172.21.15.40:0/2235170814 learned_addr learned my addr 172.21.15.40:0/2235170814 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.892+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/2235170814 >> v2:172.21.15.184:3300/0 conn(0x7f4e2015f370 0x7f4e200cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278243623 0 0) 0x7f4e200d03a0 con 0x7f4e200d3ee0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e08003520 con 0x7f4e200d3ee0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2741126529 0 0) 0x7f4e08003520 con 0x7f4e200d3ee0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 >> v2:172.21.15.184:3300/0 conn(0x7f4e2015f370 msgr2=0x7f4e200cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 --2- 172.21.15.40:0/2235170814 >> v2:172.21.15.184:3300/0 conn(0x7f4e2015f370 0x7f4e200cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 >> v1:172.21.15.184:6789/0 conn(0x7f4e2015f750 legacy=0x7f4e200d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e200d0c40 con 0x7f4e200d3ee0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f4e180019f0 con 0x7f4e200d3ee0 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 >> v1:172.21.15.40:6789/0 conn(0x7f4e200d3ee0 legacy=0x7f4e200b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e08004800 0x7f4e08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e080051c0 0x7f4e0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e200d0960 con 0x7f4e080039b0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0800c060 con 0x7f4e080051c0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0800c260 con 0x7f4e08004800 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e1ffff640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e08004800 0x7f4e08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e25430640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e080051c0 0x7f4e0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 -- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e08004800 msgr2=0x7f4e08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e08004800 0x7f4e08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 -- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e080051c0 msgr2=0x7f4e0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e080051c0 0x7f4e0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e24c2f640 1 -- 172.21.15.40:0/2235170814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e08003520 con 0x7f4e080039b0 2024-04-26T21:14:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.893+0000 7f4e25430640 1 -- 172.21.15.40:0/2235170814 reap_dead start 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.894+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e08003dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009ac0 tx=0x7f4e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db966c85730720d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.894+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10036070 con 0x7f4e080039b0 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.894+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10031400 con 0x7f4e080039b0 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.894+0000 7f4e1f7fe640 1 -- 172.21.15.40:0/2235170814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10009690 con 0x7f4e080039b0 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 -- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 msgr2=0x7f4e08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e08003dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009ac0 tx=0x7f4e08005830 comp rx=0 tx=0).stop 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 -- 172.21.15.40:0/2235170814 shutdown_connections 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 --2- 172.21.15.40:0/2235170814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e08003dd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 -- 172.21.15.40:0/2235170814 >> 172.21.15.40:0/2235170814 conn(0x7f4e2008ec10 msgr2=0x7f4e2009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 -- 172.21.15.40:0/2235170814 shutdown_connections 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.895+0000 7f4e26e93640 1 -- 172.21.15.40:0/2235170814 wait complete. 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.896+0000 7f4e26e93640 1 Processor -- start 2024-04-26T21:14:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.896+0000 7f4e26e93640 1 -- start start 2024-04-26T21:14:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e200b9af0 0x7f4e200b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e200b9f70 0x7f4e200b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e200cfef0 con 0x7f4e080039b0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e200cfd70 con 0x7f4e200b9f70 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e26e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4e200d0070 con 0x7f4e200b9af0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59846/0 (socket says 172.21.15.40:59846) 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 -- 172.21.15.40:0/3783321032 learned_addr learned my addr 172.21.15.40:0/3783321032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e1ffff640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e200b9af0 0x7f4e200b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e25430640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e200b9f70 0x7f4e200b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 -- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e200b9af0 msgr2=0x7f4e200b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e200b9af0 0x7f4e200b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 -- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e200b9f70 msgr2=0x7f4e200b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e200b9f70 0x7f4e200b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 -- 172.21.15.40:0/3783321032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e200ba4d0 con 0x7f4e080039b0 2024-04-26T21:14:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.897+0000 7f4e24c2f640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009be0 tx=0x7f4e10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.898+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1003f020 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.898+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10037040 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.898+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e200ba7b0 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.898+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10036070 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.898+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2015c640 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.899+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e200ba9d0 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.899+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e10042020 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.900+0000 7f4e1d7fa640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 0x7f4e0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.900+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f4e100ab880 con 0x7f4e080039b0 2024-04-26T21:14:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.900+0000 7f4e1ffff640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 0x7f4e0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.901+0000 7f4e1ffff640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 0x7f4e0c06ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e14007fd0 tx=0x7f4e14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:20.904+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e10049070 con 0x7f4e080039b0 2024-04-26T21:14:21.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.093+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e2006a290 con 0x7f4e080039b0 2024-04-26T21:14:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.095+0000 7f4e1d7fa640 1 -- 172.21.15.40:0/3783321032 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7f4e10079cb0 con 0x7f4e080039b0 2024-04-26T21:14:21.096 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:21.096 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 msgr2=0x7f4e0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 0x7f4e0c06ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e14007fd0 tx=0x7f4e14009040 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 msgr2=0x7f4e2015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009be0 tx=0x7f4e10004730 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 shutdown_connections 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4e0c06c920 0x7f4e0c06ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e200b9f70 0x7f4e200b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e200b9af0 0x7f4e200b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 --2- 172.21.15.40:0/3783321032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e080039b0 0x7f4e2015bdd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 >> 172.21.15.40:0/3783321032 conn(0x7f4e2008ec10 msgr2=0x7f4e2006de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 shutdown_connections 2024-04-26T21:14:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.097+0000 7f4e26e93640 1 -- 172.21.15.40:0/3783321032 wait complete. 2024-04-26T21:14:21.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:08.531061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:09.214001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:21.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:21.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.212+0000 7f9e65f5f640 1 Processor -- start 2024-04-26T21:14:21.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.212+0000 7f9e65f5f640 1 -- start start 2024-04-26T21:14:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e65f5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e6006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e65f5f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6009c700 con 0x7f9e600b3c20 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e65f5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e600d1c50 con 0x7f9e600b33e0 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e65f5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e600b4480 con 0x7f9e600d0010 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e600b3c20 0x7f9e600b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34996/0 (socket says 172.21.15.40:34996) 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5effd640 1 -- 172.21.15.40:0/2656168839 learned_addr learned my addr 172.21.15.40:0/2656168839 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e6006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130931093 0 0) 0x7f9e6009c700 con 0x7f9e600b3c20 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e48003520 con 0x7f9e600b3c20 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216745405 0 0) 0x7f9e48003520 con 0x7f9e600b3c20 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 msgr2=0x7f9e6006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e6006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 >> v1:172.21.15.184:6789/0 conn(0x7f9e600b33e0 legacy=0x7f9e60163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600d1f30 con 0x7f9e600b3c20 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9e50001670 con 0x7f9e600b3c20 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 >> v1:172.21.15.40:6789/0 conn(0x7f9e600b3c20 legacy=0x7f9e600b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e480039e0 0x7f9e48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e600b4480 con 0x7f9e48004800 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800ba70 con 0x7f9e480039e0 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4800bc70 con 0x7f9e600d0010 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.213+0000 7f9e5effd640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e480039e0 0x7f9e48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5ffff640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 -- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 msgr2=0x7f9e480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600d0010 0x7f9e480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 -- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e480039e0 msgr2=0x7f9e48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e480039e0 0x7f9e48003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 -- 172.21.15.40:0/2656168839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e48003520 con 0x7f9e48004800 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.214+0000 7f9e5effd640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001eb0 tx=0x7f9e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f0bf3e497604ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50007d30 con 0x7f9e48004800 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50005af0 con 0x7f9e48004800 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e5e7fc640 1 -- 172.21.15.40:0/2656168839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50007070 con 0x7f9e48004800 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2656168839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e48004800 msgr2=0x7f9e48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2656168839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001eb0 tx=0x7f9e48005830 comp rx=0 tx=0).stop 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e5ffff640 1 -- 172.21.15.40:0/2656168839 reap_dead start 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2656168839 shutdown_connections 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2656168839 >> 172.21.15.40:0/2656168839 conn(0x7f9e6009ba40 msgr2=0x7f9e6009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2656168839 shutdown_connections 2024-04-26T21:14:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.215+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2656168839 wait complete. 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.216+0000 7f9e65f5f640 1 Processor -- start 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 -- start start 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 0x7f9e600b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600a01e0 0x7f9e6015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006dc90 con 0x7f9e6015abd0 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006db10 con 0x7f9e48004800 2024-04-26T21:14:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e65f5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6006de10 con 0x7f9e600a01e0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59862/0 (socket says 172.21.15.40:59862) 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.217+0000 7f9e5ffff640 1 -- 172.21.15.40:0/2700923383 learned_addr learned my addr 172.21.15.40:0/2700923383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 0x7f9e600b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5effd640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600a01e0 0x7f9e6015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 -- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600a01e0 msgr2=0x7f9e6015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600a01e0 0x7f9e6015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 -- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 msgr2=0x7f9e600b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 0x7f9e600b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 -- 172.21.15.40:0/2700923383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6009f420 con 0x7f9e6015abd0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 0x7f9e600b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e5ffff640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9e60073530 tx=0x7f9e5800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003970 con 0x7f9e6015abd0 2024-04-26T21:14:21.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e58018070 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6009f700 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003d50 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.218+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6015b030 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.219+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6006a290 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.220+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e580050d0 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.220+0000 7f9e47fff640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 0x7f9e4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.220+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5808b7d0 con 0x7f9e6015abd0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.220+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 0x7f9e4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.221+0000 7f9e5f7fe640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 0x7f9e4c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009ab0 tx=0x7f9e54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.224+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e58059c90 con 0x7f9e6015abd0 2024-04-26T21:14:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.406+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e60068e00 con 0x7f9e6015abd0 2024-04-26T21:14:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.406+0000 7f9e47fff640 1 -- 172.21.15.40:0/2700923383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f9e58059590 con 0x7f9e6015abd0 2024-04-26T21:14:21.407 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 msgr2=0x7f9e4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 0x7f9e4c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009ab0 tx=0x7f9e54008040 comp rx=0 tx=0).stop 2024-04-26T21:14:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 msgr2=0x7f9e60156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9e60073530 tx=0x7f9e5800ec10 comp rx=0 tx=0).stop 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 shutdown_connections 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9e4c06c8f0 0x7f9e4c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e6015abd0 0x7f9e60156fd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e600a01e0 0x7f9e6015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 --2- 172.21.15.40:0/2700923383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e48004800 0x7f9e600b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 >> 172.21.15.40:0/2700923383 conn(0x7f9e6009ba40 msgr2=0x7f9e6015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 shutdown_connections 2024-04-26T21:14:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.408+0000 7f9e65f5f640 1 -- 172.21.15.40:0/2700923383 wait complete. 2024-04-26T21:14:21.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:14:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.523+0000 7fa045852640 1 Processor -- start 2024-04-26T21:14:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.523+0000 7fa045852640 1 -- start start 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.523+0000 7fa045852640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa04015d340 0x7fa04009c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.523+0000 7fa045852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b2420 0x7fa0400d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.523+0000 7fa045852640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0400cee60 con 0x7fa0400d1770 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa045852640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa0400d1be0 con 0x7fa04015d340 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa045852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0400ce9d0 con 0x7fa0400b2420 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa04015d340 0x7fa04009c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa0400d1770 0x7fa0400cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35008/0 (socket says 172.21.15.40:35008) 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03f7fe640 1 -- 172.21.15.40:0/2755586585 learned_addr learned my addr 172.21.15.40:0/2755586585 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03e7fc640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b2420 0x7fa0400d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03effd640 1 -- 172.21.15.40:0/2755586585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b2420 msgr2=0x7fa0400d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03effd640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b2420 0x7fa0400d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03effd640 1 -- 172.21.15.40:0/2755586585 >> v1:172.21.15.40:6789/0 conn(0x7fa0400d1770 legacy=0x7fa0400cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03dffb640 1 -- 172.21.15.40:0/2755586585 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48417851 0 0) 0x7fa0400cee60 con 0x7fa0400d1770 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.524+0000 7fa03effd640 1 -- 172.21.15.40:0/2755586585 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400d0ff0 con 0x7fa04015d340 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa03effd640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa04015d340 0x7fa04009c560 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097a0 tx=0x7fa03002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53aeb92e997cd1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa03dffb640 1 -- 172.21.15.40:0/2755586585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030037070 con 0x7fa04015d340 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa03dffb640 1 -- 172.21.15.40:0/2755586585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa030032440 con 0x7fa04015d340 2024-04-26T21:14:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa03dffb640 1 -- 172.21.15.40:0/2755586585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030032810 con 0x7fa04015d340 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 -- 172.21.15.40:0/2755586585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa04015d340 msgr2=0x7fa04009c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa04015d340 0x7fa04009c560 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097a0 tx=0x7fa03002a180 comp rx=0 tx=0).stop 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 -- 172.21.15.40:0/2755586585 shutdown_connections 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b2420 0x7fa0400d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 --2- 172.21.15.40:0/2755586585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa04015d340 0x7fa04009c560 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 -- 172.21.15.40:0/2755586585 >> 172.21.15.40:0/2755586585 conn(0x7fa04009ba40 msgr2=0x7fa04009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 -- 172.21.15.40:0/2755586585 shutdown_connections 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.525+0000 7fa045852640 1 -- 172.21.15.40:0/2755586585 wait complete. 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 Processor -- start 2024-04-26T21:14:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 -- start start 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d340 0x7fa0400a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 0x7fa0400b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400712b0 con 0x7fa0400d1770 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400ce660 con 0x7fa04015d340 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.527+0000 7fa045852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa04015d720 con 0x7fa0400b6cc0 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59888/0 (socket says 172.21.15.40:59888) 2024-04-26T21:14:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 -- 172.21.15.40:0/2326965549 learned_addr learned my addr 172.21.15.40:0/2326965549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03f7fe640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 0x7fa0400b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03e7fc640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d340 0x7fa0400a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 -- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 msgr2=0x7fa0400b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 0x7fa0400b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 -- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d340 msgr2=0x7fa0400a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d340 0x7fa0400a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 -- 172.21.15.40:0/2326965549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa040152000 con 0x7fa0400d1770 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03f7fe640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 0x7fa0400b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa03effd640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa030035040 tx=0x7fa0300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030037040 con 0x7fa0400d1770 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa030030770 con 0x7fa0400d1770 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa040151660 con 0x7fa0400d1770 2024-04-26T21:14:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030030b80 con 0x7fa0400d1770 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.528+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04006d9e0 con 0x7fa0400d1770 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.529+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04006d7d0 con 0x7fa0400d1770 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.530+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa030044020 con 0x7fa0400d1770 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.530+0000 7fa027fff640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 0x7fa02c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.530+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa0300ac000 con 0x7fa0400d1770 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.530+0000 7fa03e7fc640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 0x7fa02c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.531+0000 7fa03e7fc640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 0x7fa02c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0400b2080 tx=0x7fa034008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.534+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa030032e20 con 0x7fa0400d1770 2024-04-26T21:14:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.718+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0400693d0 con 0x7fa0400d1770 2024-04-26T21:14:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.718+0000 7fa027fff640 1 -- 172.21.15.40:0/2326965549 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7fa03007a430 con 0x7fa0400d1770 2024-04-26T21:14:21.720 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 msgr2=0x7fa02c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 0x7fa02c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0400b2080 tx=0x7fa034008040 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 msgr2=0x7fa04009fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa030035040 tx=0x7fa0300046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 shutdown_connections 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02c06c8f0 0x7fa02c06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b6cc0 0x7fa0400b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d340 0x7fa0400a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 --2- 172.21.15.40:0/2326965549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400d1770 0x7fa04009fe90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.720+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 >> 172.21.15.40:0/2326965549 conn(0x7fa04009ba40 msgr2=0x7fa0400cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.721+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 shutdown_connections 2024-04-26T21:14:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.721+0000 7fa045852640 1 -- 172.21.15.40:0/2326965549 wait complete. 2024-04-26T21:14:21.732 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:14:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.835+0000 7fd1003bf640 1 Processor -- start 2024-04-26T21:14:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.835+0000 7fd1003bf640 1 -- start start 2024-04-26T21:14:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd1003bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0f81631f0 0x7fd0f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd1003bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f809c790 con 0x7fd0f806ea30 2024-04-26T21:14:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd1003bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f8163ec0 con 0x7fd0f80d1170 2024-04-26T21:14:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd1003bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f806eec0 con 0x7fd0f81631f0 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe15b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0f81631f0 0x7fd0f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe95c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd0f806ea30 0x7fd0f80b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35018/0 (socket says 172.21.15.40:35018) 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe15b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0f81631f0 0x7fd0f80b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59904/0 (socket says 172.21.15.40:59904) 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe95c640 1 -- 172.21.15.40:0/3106854621 learned_addr learned my addr 172.21.15.40:0/3106854621 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3106854621 >> v1:172.21.15.184:6789/0 conn(0x7fd0f80d1170 legacy=0x7fd0f81635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3106854621 >> v1:172.21.15.40:6789/0 conn(0x7fd0f806ea30 legacy=0x7fd0f80b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3106854621 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f80d1a40 con 0x7fd0f81631f0 2024-04-26T21:14:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.836+0000 7fd0fd159640 1 -- 172.21.15.40:0/3106854621 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969430550 0 0) 0x7fd0f809c790 con 0x7fd0f806ea30 2024-04-26T21:14:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.837+0000 7fd0fe15b640 1 --2- 172.21.15.40:0/3106854621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f81631f0 0x7fd0f80b2000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40097a0 tx=0x7fd0f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3abf9d650cced6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.837+0000 7fd0fd159640 1 -- 172.21.15.40:0/3106854621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4037070 con 0x7fd0f81631f0 2024-04-26T21:14:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.837+0000 7fd0fd159640 1 -- 172.21.15.40:0/3106854621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f4032440 con 0x7fd0f81631f0 2024-04-26T21:14:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.837+0000 7fd0fd159640 1 -- 172.21.15.40:0/3106854621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4032810 con 0x7fd0f81631f0 2024-04-26T21:14:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 -- 172.21.15.40:0/3106854621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f81631f0 msgr2=0x7fd0f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3106854621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f81631f0 0x7fd0f80b2000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40097a0 tx=0x7fd0f402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 -- 172.21.15.40:0/3106854621 shutdown_connections 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3106854621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f81631f0 0x7fd0f80b2000 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 -- 172.21.15.40:0/3106854621 >> 172.21.15.40:0/3106854621 conn(0x7fd0f809ba40 msgr2=0x7fd0f809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 -- 172.21.15.40:0/3106854621 shutdown_connections 2024-04-26T21:14:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.838+0000 7fd1003bf640 1 -- 172.21.15.40:0/3106854621 wait complete. 2024-04-26T21:14:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.839+0000 7fd1003bf640 1 Processor -- start 2024-04-26T21:14:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.839+0000 7fd1003bf640 1 -- start start 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 0x7fd0f815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 0x7fd0f8153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f80d2000 con 0x7fd0f806ea30 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f80d15f0 con 0x7fd0f81631f0 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f80d0230 con 0x7fd0f80d1170 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59912/0 (socket says 172.21.15.40:59912) 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3392493767 learned_addr learned my addr 172.21.15.40:0/3392493767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fd95a640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 0x7fd0f815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe95c640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 0x7fd0f8153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 msgr2=0x7fd0f815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 0x7fd0f815f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 msgr2=0x7fd0f8153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 0x7fd0f8153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 -- 172.21.15.40:0/3392493767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f8160490 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fd95a640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 0x7fd0f815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe95c640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 0x7fd0f8153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0fe15b640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40097a0 tx=0x7fd0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4041070 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f403c870 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f8160770 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.840+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4037040 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.841+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f8160d30 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.841+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f806a290 con 0x7fd0f806ea30 2024-04-26T21:14:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.842+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0f4044070 con 0x7fd0f806ea30 2024-04-26T21:14:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.842+0000 7fd0eeffd640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 0x7fd0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.842+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fd0f40ac7a0 con 0x7fd0f806ea30 2024-04-26T21:14:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.843+0000 7fd0fd95a640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 0x7fd0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.843+0000 7fd0fd95a640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 0x7fd0dc06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007fd0 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:21.847+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0f407ab30 con 0x7fd0f806ea30 2024-04-26T21:14:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.035+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd0f80b8f10 con 0x7fd0f806ea30 2024-04-26T21:14:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.035+0000 7fd0eeffd640 1 -- 172.21.15.40:0/3392493767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fd0f407a430 con 0x7fd0f806ea30 2024-04-26T21:14:22.036 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:14:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 msgr2=0x7fd0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 0x7fd0dc06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007fd0 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 msgr2=0x7fd0f8161c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40097a0 tx=0x7fd0f40046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 shutdown_connections 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0dc06c8f0 0x7fd0dc06edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f81631f0 0x7fd0f8153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80d1170 0x7fd0f815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 --2- 172.21.15.40:0/3392493767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0f806ea30 0x7fd0f8161c50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 >> 172.21.15.40:0/3392493767 conn(0x7fd0f809ba40 msgr2=0x7fd0f81628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 shutdown_connections 2024-04-26T21:14:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.037+0000 7fd1003bf640 1 -- 172.21.15.40:0/3392493767 wait complete. 2024-04-26T21:14:22.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:14:22.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.149+0000 7f94dee93640 1 Processor -- start 2024-04-26T21:14:22.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94dee93640 1 -- start start 2024-04-26T21:14:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94dee93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94d806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94dee93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d80b4a10 con 0x7f94d80b44e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94dee93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d80b4cf0 con 0x7f94d815f1e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94dee93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94d806eeb0 con 0x7f94d806d9c0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.150+0000 7f94d7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94d80b44e0 0x7f94d806d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35032/0 (socket says 172.21.15.40:35032) 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d7fff640 1 -- 172.21.15.40:0/655905882 learned_addr learned my addr 172.21.15.40:0/655905882 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94dd430640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94d806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255698804 0 0) 0x7f94d80b4a10 con 0x7f94d80b44e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94c8003520 con 0x7f94d80b44e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619652098 0 0) 0x7f94d80b4cf0 con 0x7f94d815f1e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d80b4a10 con 0x7f94d815f1e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 88301348 0 0) 0x7f94c8003520 con 0x7f94d80b44e0 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 msgr2=0x7f94d806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94d806e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 >> v1:172.21.15.184:6789/0 conn(0x7f94d815f1e0 legacy=0x7f94d80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d80b4fd0 con 0x7f94d80b44e0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f94c4001670 con 0x7f94d80b44e0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 >> v1:172.21.15.40:6789/0 conn(0x7f94d80b44e0 legacy=0x7f94d806d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c8003b70 0x7f94c8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c8004800 0x7f94c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94d806eeb0 con 0x7f94c8003b70 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800ba70 con 0x7f94c8004800 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94c800bc70 con 0x7f94d806d9c0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c8003b70 0x7f94c8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94d7fff640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c8004800 0x7f94c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.151+0000 7f94dd430640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 -- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 msgr2=0x7f94c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 -- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c8004800 msgr2=0x7f94c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c8004800 0x7f94c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 -- 172.21.15.40:0/655905882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c8003520 con 0x7f94c8003b70 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dd430640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94d806d9c0 0x7f94c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c8003b70 0x7f94c8003f70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0018a0 tx=0x7f94c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc92d844272aa1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0014f0 con 0x7f94c8003b70 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0052d0 con 0x7f94c8003b70 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.152+0000 7f94d77fe640 1 -- 172.21.15.40:0/655905882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc02ac40 con 0x7f94c8003b70 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 -- 172.21.15.40:0/655905882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c8003b70 msgr2=0x7f94c8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 --2- 172.21.15.40:0/655905882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c8003b70 0x7f94c8003f70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0018a0 tx=0x7f94c8005830 comp rx=0 tx=0).stop 2024-04-26T21:14:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dd430640 1 -- 172.21.15.40:0/655905882 reap_dead start 2024-04-26T21:14:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 -- 172.21.15.40:0/655905882 shutdown_connections 2024-04-26T21:14:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 -- 172.21.15.40:0/655905882 >> 172.21.15.40:0/655905882 conn(0x7f94d80728a0 msgr2=0x7f94d8072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 -- 172.21.15.40:0/655905882 shutdown_connections 2024-04-26T21:14:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.153+0000 7f94dee93640 1 -- 172.21.15.40:0/655905882 wait complete. 2024-04-26T21:14:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.154+0000 7f94dee93640 1 Processor -- start 2024-04-26T21:14:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.154+0000 7f94dee93640 1 -- start start 2024-04-26T21:14:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c8003b70 0x7f94d8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94d815d7b0 0x7f94d815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94d815f5c0 con 0x7f94d8160920 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94d80b5590 con 0x7f94d815d7b0 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94dee93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94d8161370 con 0x7f94c8003b70 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59940/0 (socket says 172.21.15.40:59940) 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 -- 172.21.15.40:0/1640728393 learned_addr learned my addr 172.21.15.40:0/1640728393 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 -- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c8003b70 msgr2=0x7f94d8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c8003b70 0x7f94d8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 -- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94d815d7b0 msgr2=0x7f94d815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94d815d7b0 0x7f94d815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.155+0000 7f94d7fff640 1 -- 172.21.15.40:0/1640728393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d8161650 con 0x7f94d8160920 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94d7fff640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94c4007340 tx=0x7f94c4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400f7e0 con 0x7f94d8160920 2024-04-26T21:14:22.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4018070 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d8161930 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400fbe0 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.156+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d8161ef0 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.157+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d806a290 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.157+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94c401d460 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.158+0000 7f94d57fa640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 0x7f94bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.158+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f94c408bbd0 con 0x7f94d8160920 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.158+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 0x7f94bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.159+0000 7f94dcc2f640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 0x7f94bc06ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0018a0 tx=0x7f94cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.162+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94c4059ed0 con 0x7f94d8160920 2024-04-26T21:14:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.361+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94d8153d30 con 0x7f94d8160920 2024-04-26T21:14:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.362+0000 7f94d57fa640 1 -- 172.21.15.40:0/1640728393 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f94c40597d0 con 0x7f94d8160920 2024-04-26T21:14:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 msgr2=0x7f94bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 0x7f94bc06ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0018a0 tx=0x7f94cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 msgr2=0x7f94d8160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94c4007340 tx=0x7f94c4007800 comp rx=0 tx=0).stop 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 shutdown_connections 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94bc06c920 0x7f94bc06ede0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94d815d7b0 0x7f94d815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94d8160920 0x7f94d8160d80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 --2- 172.21.15.40:0/1640728393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c8003b70 0x7f94d8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 >> 172.21.15.40:0/1640728393 conn(0x7f94d80728a0 msgr2=0x7f94d806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 shutdown_connections 2024-04-26T21:14:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.364+0000 7f94dee93640 1 -- 172.21.15.40:0/1640728393 wait complete. 2024-04-26T21:14:22.376 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:14:22.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:22.376 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:22.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:14:22.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:22.409 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:14:22.467 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:22.468 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:22.468 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:22.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:22.484 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:22.484 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:22.484 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:22.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:22.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:22.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.624+0000 7f741e837640 1 Processor -- start 2024-04-26T21:14:22.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.624+0000 7f741e837640 1 -- start start 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741e837640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7418148a30 0x7f7418073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741e837640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7418160640 0x7f74181636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741e837640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74180cfa80 con 0x7f7418163be0 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741e837640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f74180cfd60 con 0x7f7418148a30 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741e837640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f74180d0040 con 0x7f7418160640 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7418148a30 0x7f7418073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7418148a30 0x7f7418073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59956/0 (socket says 172.21.15.40:59956) 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 -- 172.21.15.40:0/813393168 learned_addr learned my addr 172.21.15.40:0/813393168 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f741cdd4640 1 --1- 172.21.15.40:0/813393168 >> v1:172.21.15.40:6789/0 conn(0x7f7418163be0 0x7f74180b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35042/0 (socket says 172.21.15.40:35042) 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f74177fe640 1 --2- 172.21.15.40:0/813393168 >> v2:172.21.15.184:3300/0 conn(0x7f7418160640 0x7f74181636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 -- 172.21.15.40:0/813393168 >> v2:172.21.15.184:3300/0 conn(0x7f7418160640 msgr2=0x7f74181636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 --2- 172.21.15.40:0/813393168 >> v2:172.21.15.184:3300/0 conn(0x7f7418160640 0x7f74181636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 -- 172.21.15.40:0/813393168 >> v1:172.21.15.40:6789/0 conn(0x7f7418163be0 legacy=0x7f74180b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.625+0000 7f7417fff640 1 -- 172.21.15.40:0/813393168 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7418162ea0 con 0x7f7418148a30 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.626+0000 7f7417fff640 1 --2- 172.21.15.40:0/813393168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418148a30 0x7f7418073ed0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f74180b1aa0 tx=0x7f740802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0ff0119075ac20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.626+0000 7f7416ffd640 1 -- 172.21.15.40:0/813393168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408037070 con 0x7f7418148a30 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.626+0000 7f7416ffd640 1 -- 172.21.15.40:0/813393168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7408032440 con 0x7f7418148a30 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.626+0000 7f7416ffd640 1 -- 172.21.15.40:0/813393168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408032810 con 0x7f7418148a30 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 -- 172.21.15.40:0/813393168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418148a30 msgr2=0x7f7418073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 --2- 172.21.15.40:0/813393168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418148a30 0x7f7418073ed0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f74180b1aa0 tx=0x7f740802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 -- 172.21.15.40:0/813393168 shutdown_connections 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 --2- 172.21.15.40:0/813393168 >> v2:172.21.15.184:3300/0 conn(0x7f7418160640 0x7f74181636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 --2- 172.21.15.40:0/813393168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418148a30 0x7f7418073ed0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 -- 172.21.15.40:0/813393168 >> 172.21.15.40:0/813393168 conn(0x7f74180ce8b0 msgr2=0x7f74180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 -- 172.21.15.40:0/813393168 shutdown_connections 2024-04-26T21:14:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.627+0000 7f741e837640 1 -- 172.21.15.40:0/813393168 wait complete. 2024-04-26T21:14:22.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.628+0000 7f741e837640 1 Processor -- start 2024-04-26T21:14:22.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.628+0000 7f741e837640 1 -- start start 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 0x7f7418136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 0x7f74180b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7418074590 con 0x7f7418163be0 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7418074410 con 0x7f7418148a30 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741e837640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7418148e10 con 0x7f7418160640 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59962/0 (socket says 172.21.15.40:59962) 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 -- 172.21.15.40:0/524516146 learned_addr learned my addr 172.21.15.40:0/524516146 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f7417fff640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 0x7f7418136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f74177fe640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 0x7f74180b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 -- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 msgr2=0x7f74180b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 0x7f74180b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 -- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 msgr2=0x7f7418136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 0x7f7418136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 -- 172.21.15.40:0/524516146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181f9200 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f7417fff640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 0x7f7418136900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f74177fe640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 0x7f74180b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.629+0000 7f741cdd4640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f741000c780 tx=0x7f741000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.630+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410017070 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.630+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741000a640 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.630+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74180b4e60 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.630+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741000aa30 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.630+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74180b5420 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.631+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74180b4c70 con 0x7f7418163be0 2024-04-26T21:14:22.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.631+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f741001a070 con 0x7f7418163be0 2024-04-26T21:14:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.632+0000 7f7414ff9640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.632+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f741008afe0 con 0x7f7418163be0 2024-04-26T21:14:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.632+0000 7f7417fff640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.632+0000 7f7417fff640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 0x7f740406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7408034000 tx=0x7f740802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.636+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7410058e50 con 0x7f7418163be0 2024-04-26T21:14:22.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.861+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f741806a290 con 0x7f7418163be0 2024-04-26T21:14:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.862+0000 7f7414ff9640 1 -- 172.21.15.40:0/524516146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7f741005d150 con 0x7f7418163be0 2024-04-26T21:14:22.863 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:22.863 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 msgr2=0x7f740406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 0x7f740406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7408034000 tx=0x7f740802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 msgr2=0x7f7418161290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f741000c780 tx=0x7f741000cc40 comp rx=0 tx=0).stop 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 shutdown_connections 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7418163be0 0x7f7418161290 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7418160640 0x7f74180b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 --2- 172.21.15.40:0/524516146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7418148a30 0x7f7418136900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 >> 172.21.15.40:0/524516146 conn(0x7f74180ce8b0 msgr2=0x7f74180d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 shutdown_connections 2024-04-26T21:14:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.864+0000 7f741e837640 1 -- 172.21.15.40:0/524516146 wait complete. 2024-04-26T21:14:22.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.977+0000 7f2a4967e640 1 Processor -- start 2024-04-26T21:14:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.977+0000 7f2a4967e640 1 -- start start 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a4967e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a4415f720 0x7f2a440b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a4967e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440b2510 con 0x7f2a4415f340 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a4967e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440b27f0 con 0x7f2a440d5610 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a4967e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2a440b2ad0 con 0x7f2a4415f720 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a427fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a4415f720 0x7f2a440b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a427fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a4415f720 0x7f2a440b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59974/0 (socket says 172.21.15.40:59974) 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.978+0000 7f2a427fc640 1 -- 172.21.15.40:0/1846249984 learned_addr learned my addr 172.21.15.40:0/1846249984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.979+0000 7f2a427fc640 1 -- 172.21.15.40:0/1846249984 >> v1:172.21.15.184:6789/0 conn(0x7f2a440d5610 legacy=0x7f2a440b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.979+0000 7f2a427fc640 1 -- 172.21.15.40:0/1846249984 >> v1:172.21.15.40:6789/0 conn(0x7f2a4415f340 legacy=0x7f2a440d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.979+0000 7f2a427fc640 1 -- 172.21.15.40:0/1846249984 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440b2db0 con 0x7f2a4415f720 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.979+0000 7f2a427fc640 1 --2- 172.21.15.40:0/1846249984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a4415f720 0x7f2a440b1e80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009870 tx=0x7f2a3802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e48400df9aa713b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.979+0000 7f2a41ffb640 1 -- 172.21.15.40:0/1846249984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38037070 con 0x7f2a4415f720 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a41ffb640 1 -- 172.21.15.40:0/1846249984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a38032440 con 0x7f2a4415f720 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a41ffb640 1 -- 172.21.15.40:0/1846249984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38032810 con 0x7f2a4415f720 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 -- 172.21.15.40:0/1846249984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a4415f720 msgr2=0x7f2a440b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 --2- 172.21.15.40:0/1846249984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a4415f720 0x7f2a440b1e80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009870 tx=0x7f2a3802a210 comp rx=0 tx=0).stop 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 -- 172.21.15.40:0/1846249984 shutdown_connections 2024-04-26T21:14:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 --2- 172.21.15.40:0/1846249984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a4415f720 0x7f2a440b1e80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 -- 172.21.15.40:0/1846249984 >> 172.21.15.40:0/1846249984 conn(0x7f2a4408ec70 msgr2=0x7f2a4409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:22.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 -- 172.21.15.40:0/1846249984 shutdown_connections 2024-04-26T21:14:22.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.980+0000 7f2a4967e640 1 -- 172.21.15.40:0/1846249984 wait complete. 2024-04-26T21:14:22.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.981+0000 7f2a4967e640 1 Processor -- start 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.981+0000 7f2a4967e640 1 -- start start 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4415f340 0x7f2a440b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 0x7f2a440c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2a440d40d0 con 0x7f2a440d5610 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a440d3f50 con 0x7f2a4415f340 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a4967e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2a440b3cc0 con 0x7f2a4415f720 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a42ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a42ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59982/0 (socket says 172.21.15.40:59982) 2024-04-26T21:14:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a42ffd640 1 -- 172.21.15.40:0/754493050 learned_addr learned my addr 172.21.15.40:0/754493050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a427fc640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4415f340 0x7f2a440b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.982+0000 7f2a437fe640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 0x7f2a440c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 -- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 msgr2=0x7f2a440c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 0x7f2a440c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 -- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4415f340 msgr2=0x7f2a440b8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4415f340 0x7f2a440b8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 -- 172.21.15.40:0/754493050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440c8e20 con 0x7f2a440d5610 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a437fe640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 0x7f2a440c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a42ffd640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c400 tx=0x7f2a3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0052d0 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3c005490 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c007260 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a440c9100 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.983+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a440c96c0 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.984+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a440b1e80 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.984+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a3c007400 con 0x7f2a440d5610 2024-04-26T21:14:22.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.985+0000 7f2a2b7fe640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:22.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.985+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2a3c08b0c0 con 0x7f2a440d5610 2024-04-26T21:14:22.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.985+0000 7f2a427fc640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:22.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.986+0000 7f2a427fc640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a38034000 tx=0x7f2a3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:22.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:22.989+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a440b1e80 con 0x7f2a440d5610 2024-04-26T21:14:23.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.174+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a440d3020 con 0x7f2a440d5610 2024-04-26T21:14:23.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.175+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/754493050 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7f2a3c058e10 con 0x7f2a440d5610 2024-04-26T21:14:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:23.176 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 msgr2=0x7f2a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a38034000 tx=0x7f2a3802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 msgr2=0x7f2a44153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c400 tx=0x7f2a3c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 shutdown_connections 2024-04-26T21:14:23.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4415f720 0x7f2a440c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4415f340 0x7f2a440b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 --2- 172.21.15.40:0/754493050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a440d5610 0x7f2a44153b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 >> 172.21.15.40:0/754493050 conn(0x7f2a4408ec70 msgr2=0x7f2a4406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 shutdown_connections 2024-04-26T21:14:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.177+0000 7f2a4967e640 1 -- 172.21.15.40:0/754493050 wait complete. 2024-04-26T21:14:23.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:08.531061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:09.214001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:23.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:23.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.292+0000 7ff0db114640 1 Processor -- start 2024-04-26T21:14:23.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.292+0000 7ff0db114640 1 -- start start 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0db114640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0d40b2000 0x7ff0d406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0db114640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d409c790 con 0x7ff0d41631f0 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0db114640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d4163ec0 con 0x7ff0d40d1170 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0db114640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d40d1600 con 0x7ff0d40b2000 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0d8eb0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff0d41631f0 0x7ff0d41635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35064/0 (socket says 172.21.15.40:35064) 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/1534986288 learned_addr learned my addr 172.21.15.40:0/1534986288 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/1534986288 >> v2:172.21.15.184:3300/0 conn(0x7ff0d40b2000 0x7ff0d406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140380019 0 0) 0x7ff0d409c790 con 0x7ff0d41631f0 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.293+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0bc003520 con 0x7ff0d41631f0 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107291353 0 0) 0x7ff0bc003520 con 0x7ff0d41631f0 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 >> v2:172.21.15.184:3300/0 conn(0x7ff0d40b2000 msgr2=0x7ff0d406ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 --2- 172.21.15.40:0/1534986288 >> v2:172.21.15.184:3300/0 conn(0x7ff0d40b2000 0x7ff0d406ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 >> v1:172.21.15.184:6789/0 conn(0x7ff0d40d1170 legacy=0x7ff0d40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d40d1a40 con 0x7ff0d41631f0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff0c4001670 con 0x7ff0d41631f0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 >> v1:172.21.15.40:6789/0 conn(0x7ff0d41631f0 legacy=0x7ff0d41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0bc004800 0x7ff0bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0bc0051c0 0x7ff0bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc00c130 con 0x7ff0bc0039b0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc00c2e0 con 0x7ff0bc004800 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0bc00c4e0 con 0x7ff0bc0051c0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0bc004800 0x7ff0bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d96b1640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0bc0051c0 0x7ff0bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0bc0051c0 msgr2=0x7ff0bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0bc0051c0 0x7ff0bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0bc004800 msgr2=0x7ff0bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0bc004800 0x7ff0bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/1534986288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc003520 con 0x7ff0bc0039b0 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d96b1640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0bc0051c0 0x7ff0bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.294+0000 7ff0d96b1640 1 -- 172.21.15.40:0/1534986288 reap_dead start 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0bc003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c210 tx=0x7ff0bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e942543dde4bd553 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007d30 con 0x7ff0bc0039b0 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c4005ad0 con 0x7ff0bc0039b0 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0d37fe640 1 -- 172.21.15.40:0/1534986288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007070 con 0x7ff0bc0039b0 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0db114640 1 -- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 msgr2=0x7ff0bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.295+0000 7ff0db114640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0bc003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c210 tx=0x7ff0bc005830 comp rx=0 tx=0).stop 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.296+0000 7ff0db114640 1 -- 172.21.15.40:0/1534986288 shutdown_connections 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.296+0000 7ff0db114640 1 --2- 172.21.15.40:0/1534986288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0bc003dd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.296+0000 7ff0db114640 1 -- 172.21.15.40:0/1534986288 >> 172.21.15.40:0/1534986288 conn(0x7ff0d409ba40 msgr2=0x7ff0d409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.296+0000 7ff0db114640 1 -- 172.21.15.40:0/1534986288 shutdown_connections 2024-04-26T21:14:23.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.296+0000 7ff0db114640 1 -- 172.21.15.40:0/1534986288 wait complete. 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.297+0000 7ff0db114640 1 Processor -- start 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.297+0000 7ff0db114640 1 -- start start 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0d4153d40 0x7ff0d41f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0d41f5a00 0x7ff0d4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40d0230 con 0x7ff0bc0039b0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40d2000 con 0x7ff0d41f5a00 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d4155d70 con 0x7ff0d4153d40 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59992/0 (socket says 172.21.15.40:59992) 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/805560272 learned_addr learned my addr 172.21.15.40:0/805560272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0d4153d40 0x7ff0d41f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d96b1640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0d41f5a00 0x7ff0d4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0d4153d40 msgr2=0x7ff0d41f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0d4153d40 0x7ff0d41f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0d41f5a00 msgr2=0x7ff0d4156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0d41f5a00 0x7ff0d4156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 -- 172.21.15.40:0/805560272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d4155160 con 0x7ff0bc0039b0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d8eb0640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034040 tx=0x7ff0c4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4041070 con 0x7ff0bc0039b0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c4000ed0 con 0x7ff0bc0039b0 2024-04-26T21:14:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d4155440 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007d30 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.298+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d4155a00 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.299+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d406f0f0 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.300+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0c4044070 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.300+0000 7ff0d17fa640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 0x7ff0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.300+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff0c40abe40 con 0x7ff0bc0039b0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.300+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 0x7ff0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.301+0000 7ff0d3fff640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 0x7ff0c006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4158320 tx=0x7ff0c8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.305+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0d406f0f0 con 0x7ff0bc0039b0 2024-04-26T21:14:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.490+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0d406a290 con 0x7ff0bc0039b0 2024-04-26T21:14:23.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.491+0000 7ff0d17fa640 1 -- 172.21.15.40:0/805560272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7ff0c404b070 con 0x7ff0bc0039b0 2024-04-26T21:14:23.492 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 msgr2=0x7ff0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 0x7ff0c006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4158320 tx=0x7ff0c8009190 comp rx=0 tx=0).stop 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 msgr2=0x7ff0d4153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034040 tx=0x7ff0c4001a40 comp rx=0 tx=0).stop 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 shutdown_connections 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff0c006c8f0 0x7ff0c006edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0d41f5a00 0x7ff0d4156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0d4153d40 0x7ff0d41f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 --2- 172.21.15.40:0/805560272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc0039b0 0x7ff0d4153740 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 >> 172.21.15.40:0/805560272 conn(0x7ff0d409ba40 msgr2=0x7ff0d415fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 shutdown_connections 2024-04-26T21:14:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.493+0000 7ff0db114640 1 -- 172.21.15.40:0/805560272 wait complete. 2024-04-26T21:14:23.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:14:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.607+0000 7f6562b9d640 1 Processor -- start 2024-04-26T21:14:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f6562b9d640 1 -- start start 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f6562b9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c0d4f20 0x7f655c0d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f6562b9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c0d3c90 con 0x7f655c0d3820 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f6562b9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c0d58c0 con 0x7f655c15f680 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f6562b9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f655c15fa60 con 0x7f655c0d4f20 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.608+0000 7f656113a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c0d4f20 0x7f655c0d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c0d4f20 0x7f655c0d5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60002/0 (socket says 172.21.15.40:60002) 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 -- 172.21.15.40:0/269011292 learned_addr learned my addr 172.21.15.40:0/269011292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 -- 172.21.15.40:0/269011292 >> v1:172.21.15.184:6789/0 conn(0x7f655c15f680 legacy=0x7f655c163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 -- 172.21.15.40:0/269011292 >> v1:172.21.15.40:6789/0 conn(0x7f655c0d3820 legacy=0x7f655c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 -- 172.21.15.40:0/269011292 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0d3f50 con 0x7f655c0d4f20 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.609+0000 7f656113a640 1 --2- 172.21.15.40:0/269011292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c0d5380 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6554009e50 tx=0x7f655402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117b18560b095e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f655b7fe640 1 -- 172.21.15.40:0/269011292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554036070 con 0x7f655c0d4f20 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f655b7fe640 1 -- 172.21.15.40:0/269011292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554031600 con 0x7f655c0d4f20 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f655b7fe640 1 -- 172.21.15.40:0/269011292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540319f0 con 0x7f655c0d4f20 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 -- 172.21.15.40:0/269011292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 msgr2=0x7f655c0d5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 --2- 172.21.15.40:0/269011292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c0d5380 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6554009e50 tx=0x7f655402a7c0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 -- 172.21.15.40:0/269011292 shutdown_connections 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 --2- 172.21.15.40:0/269011292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c0d5380 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 -- 172.21.15.40:0/269011292 >> 172.21.15.40:0/269011292 conn(0x7f655c08eca0 msgr2=0x7f655c09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 -- 172.21.15.40:0/269011292 shutdown_connections 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.610+0000 7f6562b9d640 1 -- 172.21.15.40:0/269011292 wait complete. 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.611+0000 7f6562b9d640 1 Processor -- start 2024-04-26T21:14:23.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.611+0000 7f6562b9d640 1 -- start start 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 0x7f655c154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c15f680 0x7f655c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0b6280 con 0x7f655c0d4f20 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0b5370 con 0x7f655c0d3820 2024-04-26T21:14:23.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f6562b9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0cfd80 con 0x7f655c15f680 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f655bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f655bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60018/0 (socket says 172.21.15.40:60018) 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.612+0000 7f655bfff640 1 -- 172.21.15.40:0/2502493342 learned_addr learned my addr 172.21.15.40:0/2502493342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f6560939640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 0x7f655c154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f656113a640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c15f680 0x7f655c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 -- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c15f680 msgr2=0x7f655c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c15f680 0x7f655c0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 -- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 msgr2=0x7f655c154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 0x7f655c154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 -- 172.21.15.40:0/2502493342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c159cd0 con 0x7f655c0d4f20 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f6560939640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 0x7f655c154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:23.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f655bfff640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f654800c450 tx=0x7f654800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65480052d0 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6548005470 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c159fb0 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548012870 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.613+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c15a570 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.614+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c06a290 con 0x7f655c0d4f20 2024-04-26T21:14:23.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.615+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6548012a10 con 0x7f655c0d4f20 2024-04-26T21:14:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.615+0000 7f65597fa640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 0x7f654006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.615+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f654808b6c0 con 0x7f655c0d4f20 2024-04-26T21:14:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.615+0000 7f6560939640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 0x7f654006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.616+0000 7f6560939640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 0x7f654006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f65500015e0 tx=0x7f6550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.619+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6548059b80 con 0x7f655c0d4f20 2024-04-26T21:14:23.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.806+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f655c0b4cd0 con 0x7f655c0d4f20 2024-04-26T21:14:23.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.807+0000 7f65597fa640 1 -- 172.21.15.40:0/2502493342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f6548059480 con 0x7f655c0d4f20 2024-04-26T21:14:23.808 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 msgr2=0x7f654006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 0x7f654006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f65500015e0 tx=0x7f6550009040 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 msgr2=0x7f655c15c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f654800c450 tx=0x7f654800c910 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 shutdown_connections 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f654006c8f0 0x7f654006edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f655c15f680 0x7f655c0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0d4f20 0x7f655c15c300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 --2- 172.21.15.40:0/2502493342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f655c0d3820 0x7f655c154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 >> 172.21.15.40:0/2502493342 conn(0x7f655c08eca0 msgr2=0x7f655c0d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 shutdown_connections 2024-04-26T21:14:23.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.809+0000 7f6562b9d640 1 -- 172.21.15.40:0/2502493342 wait complete. 2024-04-26T21:14:23.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:14:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.923+0000 7f877db90640 1 Processor -- start 2024-04-26T21:14:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.923+0000 7f877db90640 1 -- start start 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f877db90640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f877815c4c0 0x7f877806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f877db90640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 0x7f877815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f877db90640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f877806f730 con 0x7f877815c0e0 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f877db90640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8778160aa0 con 0x7f877815c4c0 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f877db90640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f877806dd20 con 0x7f877806d8f0 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f8776ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f877815c4c0 0x7f877806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f8776ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f877815c4c0 0x7f877806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60032/0 (socket says 172.21.15.40:60032) 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.924+0000 7f8776ffd640 1 -- 172.21.15.40:0/2054795966 learned_addr learned my addr 172.21.15.40:0/2054795966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8777fff640 1 --2- 172.21.15.40:0/2054795966 >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 0x7f877815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8776ffd640 1 -- 172.21.15.40:0/2054795966 >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 msgr2=0x7f877815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8776ffd640 1 --2- 172.21.15.40:0/2054795966 >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 0x7f877815cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8776ffd640 1 -- 172.21.15.40:0/2054795966 >> v1:172.21.15.40:6789/0 conn(0x7f877815c0e0 legacy=0x7f877806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8776ffd640 1 -- 172.21.15.40:0/2054795966 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87780cff00 con 0x7f877815c4c0 2024-04-26T21:14:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8777fff640 1 --2- 172.21.15.40:0/2054795966 >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 0x7f877815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.925+0000 7f8776ffd640 1 --2- 172.21.15.40:0/2054795966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c4c0 0x7f877806d3b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8778073020 tx=0x7f876c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3116908ebe1f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f87767fc640 1 -- 172.21.15.40:0/2054795966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c037070 con 0x7f877815c4c0 2024-04-26T21:14:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f87767fc640 1 -- 172.21.15.40:0/2054795966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876c032440 con 0x7f877815c4c0 2024-04-26T21:14:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f87767fc640 1 -- 172.21.15.40:0/2054795966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c032810 con 0x7f877815c4c0 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 -- 172.21.15.40:0/2054795966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c4c0 msgr2=0x7f877806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 --2- 172.21.15.40:0/2054795966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c4c0 0x7f877806d3b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8778073020 tx=0x7f876c02a270 comp rx=0 tx=0).stop 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 -- 172.21.15.40:0/2054795966 shutdown_connections 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 --2- 172.21.15.40:0/2054795966 >> v2:172.21.15.184:3300/0 conn(0x7f877806d8f0 0x7f877815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 --2- 172.21.15.40:0/2054795966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c4c0 0x7f877806d3b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 -- 172.21.15.40:0/2054795966 >> 172.21.15.40:0/2054795966 conn(0x7f87780d1e40 msgr2=0x7f87780d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 -- 172.21.15.40:0/2054795966 shutdown_connections 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.926+0000 7f877db90640 1 -- 172.21.15.40:0/2054795966 wait complete. 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.927+0000 7f877db90640 1 Processor -- start 2024-04-26T21:14:23.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 -- start start 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 0x7f87780b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877815c4c0 0x7f87780afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8778160450 con 0x7f877815c0e0 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8778160170 con 0x7f877806d8f0 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f877db90640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8778160730 con 0x7f877815c4c0 2024-04-26T21:14:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f8776ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f8776ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60040/0 (socket says 172.21.15.40:60040) 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.928+0000 7f8776ffd640 1 -- 172.21.15.40:0/1337508644 learned_addr learned my addr 172.21.15.40:0/1337508644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f87777fe640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 0x7f87780b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 -- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877815c4c0 msgr2=0x7f87780afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877815c4c0 0x7f87780afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 -- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 msgr2=0x7f87780b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 0x7f87780b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 -- 172.21.15.40:0/1337508644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87780cbd20 con 0x7f877815c0e0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f87777fe640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 0x7f87780b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f8776ffd640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f876c02a2b0 tx=0x7f876c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c037040 con 0x7f877815c0e0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876c042070 con 0x7f877815c0e0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877814ef40 con 0x7f877815c0e0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c030bb0 con 0x7f877815c0e0 2024-04-26T21:14:23.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.929+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877814f500 con 0x7f877815c0e0 2024-04-26T21:14:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.930+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877814ebc0 con 0x7f877815c0e0 2024-04-26T21:14:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.931+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f876c047460 con 0x7f877815c0e0 2024-04-26T21:14:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.931+0000 7f875ffff640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 0x7f876406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:23.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.931+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f876c0ac7a0 con 0x7f877815c0e0 2024-04-26T21:14:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.931+0000 7f87777fe640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 0x7f876406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:23.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.932+0000 7f87777fe640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 0x7f876406edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8768001680 tx=0x7f8768009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:23.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:23.935+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f877814ebc0 con 0x7f877815c0e0 2024-04-26T21:14:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.119+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87780693d0 con 0x7f877815c0e0 2024-04-26T21:14:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.120+0000 7f875ffff640 1 -- 172.21.15.40:0/1337508644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f876c045360 con 0x7f877815c0e0 2024-04-26T21:14:24.121 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:14:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 msgr2=0x7f876406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 0x7f876406edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8768001680 tx=0x7f8768009040 comp rx=0 tx=0).stop 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 msgr2=0x7f87780cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f876c02a2b0 tx=0x7f876c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 shutdown_connections 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f876406c8f0 0x7f876406edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877815c4c0 0x7f87780afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877815c0e0 0x7f87780cb410 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 --2- 172.21.15.40:0/1337508644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f877806d8f0 0x7f87780b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 >> 172.21.15.40:0/1337508644 conn(0x7f87780d1e40 msgr2=0x7f87780cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 shutdown_connections 2024-04-26T21:14:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.122+0000 7f877db90640 1 -- 172.21.15.40:0/1337508644 wait complete. 2024-04-26T21:14:24.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.234+0000 7f7745662640 1 Processor -- start 2024-04-26T21:14:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.234+0000 7f7745662640 1 -- start start 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f7745662640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7740070d20 0x7f7740071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f7745662640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7740070520 0x7f7740163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f7745662640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77400d14c0 con 0x7f774009c400 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f7745662640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f77400d1290 con 0x7f7740070d20 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f7745662640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7740071640 con 0x7f7740070520 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f773effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7740070d20 0x7f7740071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f773effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7740070d20 0x7f7740071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60048/0 (socket says 172.21.15.40:60048) 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.235+0000 7f773effd640 1 -- 172.21.15.40:0/2942139092 learned_addr learned my addr 172.21.15.40:0/2942139092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773f7fe640 1 --2- 172.21.15.40:0/2942139092 >> v2:172.21.15.184:3300/0 conn(0x7f7740070520 0x7f7740163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773effd640 1 -- 172.21.15.40:0/2942139092 >> v2:172.21.15.184:3300/0 conn(0x7f7740070520 msgr2=0x7f7740163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773effd640 1 --2- 172.21.15.40:0/2942139092 >> v2:172.21.15.184:3300/0 conn(0x7f7740070520 0x7f7740163640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773effd640 1 -- 172.21.15.40:0/2942139092 >> v1:172.21.15.184:6789/0 conn(0x7f774009c400 legacy=0x7f7740162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773effd640 1 -- 172.21.15.40:0/2942139092 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77400ce5b0 con 0x7f7740070d20 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.236+0000 7f773effd640 1 --2- 172.21.15.40:0/2942139092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7740070d20 0x7f7740071100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f77400cf980 tx=0x7f773802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd285066cecec1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f773dffb640 1 -- 172.21.15.40:0/2942139092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738037070 con 0x7f7740070d20 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f773dffb640 1 -- 172.21.15.40:0/2942139092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7738032440 con 0x7f7740070d20 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f773dffb640 1 -- 172.21.15.40:0/2942139092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738032810 con 0x7f7740070d20 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 -- 172.21.15.40:0/2942139092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7740070d20 msgr2=0x7f7740071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 --2- 172.21.15.40:0/2942139092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7740070d20 0x7f7740071100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f77400cf980 tx=0x7f773802a230 comp rx=0 tx=0).stop 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 -- 172.21.15.40:0/2942139092 shutdown_connections 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 --2- 172.21.15.40:0/2942139092 >> v2:172.21.15.184:3300/0 conn(0x7f7740070520 0x7f7740163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 --2- 172.21.15.40:0/2942139092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7740070d20 0x7f7740071100 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 -- 172.21.15.40:0/2942139092 >> 172.21.15.40:0/2942139092 conn(0x7f77400b4a40 msgr2=0x7f77400b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 -- 172.21.15.40:0/2942139092 shutdown_connections 2024-04-26T21:14:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.237+0000 7f7745662640 1 -- 172.21.15.40:0/2942139092 wait complete. 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 Processor -- start 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 -- start start 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7740070520 0x7f7740151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7740070d20 0x7f77400b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77400d1d20 con 0x7f774009c400 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77400d1a40 con 0x7f7740070520 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f7745662640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77400d2000 con 0x7f7740070d20 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60056/0 (socket says 172.21.15.40:60056) 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.239+0000 7f773f7fe640 1 -- 172.21.15.40:0/470691219 learned_addr learned my addr 172.21.15.40:0/470691219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773effd640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7740070520 0x7f7740151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773e7fc640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7740070d20 0x7f77400b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 -- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7740070d20 msgr2=0x7f77400b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7740070d20 0x7f77400b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 -- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7740070520 msgr2=0x7f7740151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7740070520 0x7f7740151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 -- 172.21.15.40:0/470691219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7740153d00 con 0x7f774009c400 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f773f7fe640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f773000cc30 tx=0x7f773000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773000ce40 con 0x7f774009c400 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7730003990 con 0x7f774009c400 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7740153fe0 con 0x7f774009c400 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730003d40 con 0x7f774009c400 2024-04-26T21:14:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.240+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77401f8c90 con 0x7f774009c400 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.241+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77400693d0 con 0x7f774009c400 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.242+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7730003fa0 con 0x7f774009c400 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.242+0000 7f77377fe640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 0x7f772006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.242+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f773008c040 con 0x7f774009c400 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.242+0000 7f773effd640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 0x7f772006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.243+0000 7f773effd640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 0x7f772006ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7738034000 tx=0x7f773802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.246+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7730014030 con 0x7f774009c400 2024-04-26T21:14:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.469+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77401549c0 con 0x7f774009c400 2024-04-26T21:14:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.470+0000 7f77377fe640 1 -- 172.21.15.40:0/470691219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7f773005a3f0 con 0x7f774009c400 2024-04-26T21:14:24.471 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:24.471 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 msgr2=0x7f772006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 0x7f772006ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7738034000 tx=0x7f773802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 msgr2=0x7f77400b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f773000cc30 tx=0x7f773000a5c0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 shutdown_connections 2024-04-26T21:14:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f772006c920 0x7f772006ede0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f774009c400 0x7f77400b76a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7740070d20 0x7f77400b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 --2- 172.21.15.40:0/470691219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7740070520 0x7f7740151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 >> 172.21.15.40:0/470691219 conn(0x7f77400b4a40 msgr2=0x7f77400b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 shutdown_connections 2024-04-26T21:14:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.472+0000 7f7745662640 1 -- 172.21.15.40:0/470691219 wait complete. 2024-04-26T21:14:24.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 Processor -- start 2024-04-26T21:14:24.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 -- start start 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f064c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c0d03a0 con 0x7f064c0d3ee0 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c0d0680 con 0x7f064c15f750 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.585+0000 7f0652893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0d0960 con 0x7f064c15f370 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f0650e30640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f064c0d3ee0 0x7f064c0b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35090/0 (socket says 172.21.15.40:35090) 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f0650e30640 1 -- 172.21.15.40:0/3624805930 learned_addr learned my addr 172.21.15.40:0/3624805930 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f064c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637924899 0 0) 0x7f064c0d03a0 con 0x7f064c0d3ee0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0634003520 con 0x7f064c0d3ee0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245227112 0 0) 0x7f064c0d0680 con 0x7f064c15f750 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f064c0d03a0 con 0x7f064c15f750 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2743693220 0 0) 0x7f0634003520 con 0x7f064c0d3ee0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 msgr2=0x7f064c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f064c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 >> v1:172.21.15.184:6789/0 conn(0x7f064c15f750 legacy=0x7f064c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0d0c40 con 0x7f064c0d3ee0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f06440019f0 con 0x7f064c0d3ee0 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f064c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 >> v1:172.21.15.40:6789/0 conn(0x7f064c0d3ee0 legacy=0x7f064c0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0634003b70 0x7f0634003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634004800 0x7f0634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f06340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0d0960 con 0x7f0634003b70 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f063400baa0 con 0x7f0634004800 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f063400bca0 con 0x7f064c15f370 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0634003b70 0x7f0634003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.586+0000 7f064b7fe640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634004800 0x7f0634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f0650e30640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f06340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 -- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 msgr2=0x7f06340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f06340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 -- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634004800 msgr2=0x7f0634004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634004800 0x7f0634004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 -- 172.21.15.40:0/3624805930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0634003520 con 0x7f0634003b70 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f0650e30640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c15f370 0x7f06340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.587+0000 7f064bfff640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0634003b70 0x7f0634003f70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f063c009c60 tx=0x7f0634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a887f8ea6beeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c036070 con 0x7f0634003b70 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c031400 con 0x7f0634003b70 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f064affd640 1 -- 172.21.15.40:0/3624805930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c009860 con 0x7f0634003b70 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 -- 172.21.15.40:0/3624805930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0634003b70 msgr2=0x7f0634003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 --2- 172.21.15.40:0/3624805930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0634003b70 0x7f0634003f70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f063c009c60 tx=0x7f0634005830 comp rx=0 tx=0).stop 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0650e30640 1 -- 172.21.15.40:0/3624805930 reap_dead start 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 -- 172.21.15.40:0/3624805930 shutdown_connections 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 -- 172.21.15.40:0/3624805930 >> 172.21.15.40:0/3624805930 conn(0x7f064c08ec10 msgr2=0x7f064c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 -- 172.21.15.40:0/3624805930 shutdown_connections 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.588+0000 7f0652893640 1 -- 172.21.15.40:0/3624805930 wait complete. 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.589+0000 7f0652893640 1 Processor -- start 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 -- start start 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 0x7f064c1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c0c96f0 0x7f064c0b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0cfef0 con 0x7f064c0ba4b0 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0cfd70 con 0x7f0634003b70 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0652893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0d0070 con 0x7f064c0c96f0 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0650e30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0650e30640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60080/0 (socket says 172.21.15.40:60080) 2024-04-26T21:14:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.590+0000 7f0650e30640 1 -- 172.21.15.40:0/211495206 learned_addr learned my addr 172.21.15.40:0/211495206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f064bfff640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 0x7f064c1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f064b7fe640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c0c96f0 0x7f064c0b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 -- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c0c96f0 msgr2=0x7f064c0b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c0c96f0 0x7f064c0b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 -- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 msgr2=0x7f064c1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 0x7f064c1f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 -- 172.21.15.40:0/211495206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0ca610 con 0x7f064c0ba4b0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f064bfff640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 0x7f064c1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0650e30640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0644001370 tx=0x7f0644007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400ea40 con 0x7f064c0ba4b0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064400ebe0 con 0x7f064c0ba4b0 2024-04-26T21:14:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c0ca8f0 con 0x7f064c0ba4b0 2024-04-26T21:14:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400edd0 con 0x7f064c0ba4b0 2024-04-26T21:14:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.591+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c160050 con 0x7f064c0ba4b0 2024-04-26T21:14:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.592+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c0ba930 con 0x7f064c0ba4b0 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.593+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f064401a020 con 0x7f064c0ba4b0 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.593+0000 7f0648ff9640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.593+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f064408b470 con 0x7f064c0ba4b0 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.593+0000 7f064bfff640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.594+0000 7f064bfff640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 0x7f063806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f063c033000 tx=0x7f063c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.597+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0644021020 con 0x7f064c0ba4b0 2024-04-26T21:14:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.782+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f064c06a290 con 0x7f064c0ba4b0 2024-04-26T21:14:24.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.783+0000 7f0648ff9640 1 -- 172.21.15.40:0/211495206 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7f0644059930 con 0x7f064c0ba4b0 2024-04-26T21:14:24.784 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:24.784 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 msgr2=0x7f063806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 0x7f063806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f063c033000 tx=0x7f063c008030 comp rx=0 tx=0).stop 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 msgr2=0x7f064c1f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0644001370 tx=0x7f0644007b10 comp rx=0 tx=0).stop 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 shutdown_connections 2024-04-26T21:14:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f064c0ba4b0 0x7f064c1f8420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f064c0c96f0 0x7f064c0b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 --2- 172.21.15.40:0/211495206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0634003b70 0x7f064c1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 >> 172.21.15.40:0/211495206 conn(0x7f064c08ec10 msgr2=0x7f064c06dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 shutdown_connections 2024-04-26T21:14:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.785+0000 7f0652893640 1 -- 172.21.15.40:0/211495206 wait complete. 2024-04-26T21:14:24.797 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:14:24.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.897+0000 7fc84c27e640 1 Processor -- start 2024-04-26T21:14:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.897+0000 7fc84c27e640 1 -- start start 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.898+0000 7fc84c27e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8440d3ee0 0x7fc8440cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.898+0000 7fc84c27e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc84415f750 0x7fc8440b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.898+0000 7fc84c27e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8440d03a0 con 0x7fc84415f370 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.898+0000 7fc84c27e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc8440b4d10 con 0x7fc84415f750 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.898+0000 7fc84c27e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440d0680 con 0x7fc8440d3ee0 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc84415f750 0x7fc8440b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc84415f750 0x7fc8440b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60094/0 (socket says 172.21.15.40:60094) 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 -- 172.21.15.40:0/98246116 learned_addr learned my addr 172.21.15.40:0/98246116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a01a640 1 --1- 172.21.15.40:0/98246116 >> v1:172.21.15.40:6789/0 conn(0x7fc84415f370 0x7fc8440d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35098/0 (socket says 172.21.15.40:35098) 2024-04-26T21:14:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc849819640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8440d3ee0 0x7fc8440cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 -- 172.21.15.40:0/98246116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8440d3ee0 msgr2=0x7fc8440cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8440d3ee0 0x7fc8440cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 -- 172.21.15.40:0/98246116 >> v1:172.21.15.40:6789/0 conn(0x7fc84415f370 legacy=0x7fc8440d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.899+0000 7fc84a81b640 1 -- 172.21.15.40:0/98246116 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440d0960 con 0x7fc84415f750 2024-04-26T21:14:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84a81b640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440b47d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009e50 tx=0x7fc83c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f08bfc172bcdb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc849018640 1 -- 172.21.15.40:0/98246116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c036070 con 0x7fc84415f750 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc849018640 1 -- 172.21.15.40:0/98246116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83c031600 con 0x7fc84415f750 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc849018640 1 -- 172.21.15.40:0/98246116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0319d0 con 0x7fc84415f750 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 -- 172.21.15.40:0/98246116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 msgr2=0x7fc8440b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440b47d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009e50 tx=0x7fc83c02a710 comp rx=0 tx=0).stop 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 -- 172.21.15.40:0/98246116 shutdown_connections 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440b47d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 --2- 172.21.15.40:0/98246116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8440d3ee0 0x7fc8440cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.900+0000 7fc84c27e640 1 -- 172.21.15.40:0/98246116 >> 172.21.15.40:0/98246116 conn(0x7fc84408ec10 msgr2=0x7fc84409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.901+0000 7fc84c27e640 1 -- 172.21.15.40:0/98246116 shutdown_connections 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.901+0000 7fc84c27e640 1 -- 172.21.15.40:0/98246116 wait complete. 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 Processor -- start 2024-04-26T21:14:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 -- start start 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8440d3ee0 0x7fc8440b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc84415f370 0x7fc8440cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440b51c0 con 0x7fc84415f750 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440d0f20 con 0x7fc8440d3ee0 2024-04-26T21:14:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.902+0000 7fc84c27e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440b5340 con 0x7fc84415f370 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60108/0 (socket says 172.21.15.40:60108) 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 -- 172.21.15.40:0/1593993457 learned_addr learned my addr 172.21.15.40:0/1593993457 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc849819640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc84415f370 0x7fc8440cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a01a640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8440d3ee0 0x7fc8440b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 -- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc84415f370 msgr2=0x7fc8440cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc84415f370 0x7fc8440cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 -- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8440d3ee0 msgr2=0x7fc8440b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8440d3ee0 0x7fc8440b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 -- 172.21.15.40:0/1593993457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440b9de0 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84a81b640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009f70 tx=0x7fc83c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c041020 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83c0077a0 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8440ba0c0 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c036040 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.903+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8441f90e0 con 0x7fc84415f750 2024-04-26T21:14:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.904+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8440ba880 con 0x7fc84415f750 2024-04-26T21:14:24.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.905+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc83c044020 con 0x7fc84415f750 2024-04-26T21:14:24.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.905+0000 7fc83affd640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 0x7fc82806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:24.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.905+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc83c0ac710 con 0x7fc84415f750 2024-04-26T21:14:24.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.905+0000 7fc84a01a640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 0x7fc82806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:24.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.906+0000 7fc84a01a640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 0x7fc82806edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc840000f60 tx=0x7fc840009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:24.910+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8440ba880 con 0x7fc84415f750 2024-04-26T21:14:25.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.133+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8440d3b20 con 0x7fc84415f750 2024-04-26T21:14:25.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.133+0000 7fc83affd640 1 -- 172.21.15.40:0/1593993457 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7fc83c04b030 con 0x7fc84415f750 2024-04-26T21:14:25.135 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:25.135 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:25.135 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:25.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 msgr2=0x7fc82806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 0x7fc82806edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc840000f60 tx=0x7fc840009040 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 msgr2=0x7fc8440cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009f70 tx=0x7fc83c004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 shutdown_connections 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc82806c8f0 0x7fc82806edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84415f750 0x7fc8440cdf80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc84415f370 0x7fc8440cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 --2- 172.21.15.40:0/1593993457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8440d3ee0 0x7fc8440b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 >> 172.21.15.40:0/1593993457 conn(0x7fc84408ec10 msgr2=0x7fc844075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 shutdown_connections 2024-04-26T21:14:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.135+0000 7fc84c27e640 1 -- 172.21.15.40:0/1593993457 wait complete. 2024-04-26T21:14:25.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:25.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.248+0000 7fe019c47640 1 Processor -- start 2024-04-26T21:14:25.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.248+0000 7fe019c47640 1 -- start start 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.248+0000 7fe019c47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe014146790 0x7fe0140cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.248+0000 7fe019c47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140d05b0 con 0x7fe014140350 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe019c47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140d0890 con 0x7fe01413ff70 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe019c47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe01406e450 con 0x7fe014146790 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe014146790 0x7fe0140cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe014146790 0x7fe0140cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60110/0 (socket says 172.21.15.40:60110) 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 -- 172.21.15.40:0/3160829457 learned_addr learned my addr 172.21.15.40:0/3160829457 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:25.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 -- 172.21.15.40:0/3160829457 >> v1:172.21.15.184:6789/0 conn(0x7fe01413ff70 legacy=0x7fe014146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 -- 172.21.15.40:0/3160829457 >> v1:172.21.15.40:6789/0 conn(0x7fe014140350 legacy=0x7fe0140cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.249+0000 7fe013fff640 1 -- 172.21.15.40:0/3160829457 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01406e730 con 0x7fe014146790 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe013fff640 1 --2- 172.21.15.40:0/3160829457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe014146790 0x7fe0140cff20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009e80 tx=0x7fe00c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776b2df95d83b95c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe0127fc640 1 -- 172.21.15.40:0/3160829457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c036070 con 0x7fe014146790 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe0127fc640 1 -- 172.21.15.40:0/3160829457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00c031600 con 0x7fe014146790 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe0127fc640 1 -- 172.21.15.40:0/3160829457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0319d0 con 0x7fe014146790 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 -- 172.21.15.40:0/3160829457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe014146790 msgr2=0x7fe0140cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 --2- 172.21.15.40:0/3160829457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe014146790 0x7fe0140cff20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009e80 tx=0x7fe00c02a710 comp rx=0 tx=0).stop 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 -- 172.21.15.40:0/3160829457 shutdown_connections 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 --2- 172.21.15.40:0/3160829457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe014146790 0x7fe0140cff20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 -- 172.21.15.40:0/3160829457 >> 172.21.15.40:0/3160829457 conn(0x7fe0140d0d30 msgr2=0x7fe0140d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 -- 172.21.15.40:0/3160829457 shutdown_connections 2024-04-26T21:14:25.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.250+0000 7fe019c47640 1 -- 172.21.15.40:0/3160829457 wait complete. 2024-04-26T21:14:25.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 Processor -- start 2024-04-26T21:14:25.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 -- start start 2024-04-26T21:14:25.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe014140350 0x7fe0140b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe014146790 0x7fe0140b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141681c0 con 0x7fe01413ff70 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe014168040 con 0x7fe014140350 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.252+0000 7fe019c47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe014168340 con 0x7fe014146790 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60116/0 (socket says 172.21.15.40:60116) 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 -- 172.21.15.40:0/1988477713 learned_addr learned my addr 172.21.15.40:0/1988477713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe012ffd640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe014140350 0x7fe0140b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe013fff640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe014146790 0x7fe0140b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 -- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe014146790 msgr2=0x7fe0140b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe014146790 0x7fe0140b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 -- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe014140350 msgr2=0x7fe0140b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe014140350 0x7fe0140b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 -- 172.21.15.40:0/1988477713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01415be90 con 0x7fe01413ff70 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe0137fe640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c430 tx=0x7fe00400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004011790 con 0x7fe01413ff70 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe004017070 con 0x7fe01413ff70 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004011bb0 con 0x7fe01413ff70 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01415d980 con 0x7fe01413ff70 2024-04-26T21:14:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.253+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01415df40 con 0x7fe01413ff70 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.254+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01406a290 con 0x7fe01413ff70 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.255+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe00401c460 con 0x7fe01413ff70 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.255+0000 7fdffbfff640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 0x7fe00006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.255+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fe00408bad0 con 0x7fe01413ff70 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.255+0000 7fe012ffd640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 0x7fe00006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.256+0000 7fe012ffd640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 0x7fe00006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe0140b6b10 tx=0x7fe008009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.260+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe004059e80 con 0x7fe01413ff70 2024-04-26T21:14:25.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.484+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0140b45e0 con 0x7fe01413ff70 2024-04-26T21:14:25.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.485+0000 7fdffbfff640 1 -- 172.21.15.40:0/1988477713 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5184 (secure 0 0 0) 0x7fe004059780 con 0x7fe01413ff70 2024-04-26T21:14:25.486 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:25.486 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:10.268813+0000","modified":"2024-04-26T21:14:13.276446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/363892172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":363892172},{"type":"v1","addr":"172.21.15.40:6835","nonce":363892172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5607,"qdb_cluster":[5607]},"id":3}]} 2024-04-26T21:14:25.486 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 msgr2=0x7fe00006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 0x7fe00006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe0140b6b10 tx=0x7fe008009040 comp rx=0 tx=0).stop 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 msgr2=0x7fe01415a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c430 tx=0x7fe00400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 shutdown_connections 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe00006c8f0 0x7fe00006edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe014146790 0x7fe0140b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe014140350 0x7fe0140b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 --2- 172.21.15.40:0/1988477713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe01413ff70 0x7fe01415a130 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 >> 172.21.15.40:0/1988477713 conn(0x7fe0140d0d30 msgr2=0x7fe0140b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 shutdown_connections 2024-04-26T21:14:25.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.487+0000 7fe019c47640 1 -- 172.21.15.40:0/1988477713 wait complete. 2024-04-26T21:14:25.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:25.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.599+0000 7fca556a9640 1 Processor -- start 2024-04-26T21:14:25.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.599+0000 7fca556a9640 1 -- start start 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca556a9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca5015f720 0x7fca500b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca556a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca500d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca556a9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca500b2510 con 0x7fca5015f340 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca556a9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fca500b27f0 con 0x7fca5015f720 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca556a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca500b2ad0 con 0x7fca500d5610 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca5015f340 0x7fca500d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35116/0 (socket says 172.21.15.40:35116) 2024-04-26T21:14:25.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4effd640 1 -- 172.21.15.40:0/2649759241 learned_addr learned my addr 172.21.15.40:0/2649759241 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca500d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca47fff640 1 --2- 172.21.15.40:0/2649759241 >> v2:172.21.15.184:3300/0 conn(0x7fca5015f720 0x7fca500b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257404027 0 0) 0x7fca500b2510 con 0x7fca5015f340 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c003520 con 0x7fca5015f340 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4175247978 0 0) 0x7fca3c003520 con 0x7fca5015f340 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 msgr2=0x7fca500d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca500d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 >> v2:172.21.15.184:3300/0 conn(0x7fca5015f720 msgr2=0x7fca500b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 --2- 172.21.15.40:0/2649759241 >> v2:172.21.15.184:3300/0 conn(0x7fca5015f720 0x7fca500b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca500b2db0 con 0x7fca5015f340 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.600+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fca40001670 con 0x7fca5015f340 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 >> v1:172.21.15.40:6789/0 conn(0x7fca5015f340 legacy=0x7fca500d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca500b27f0 con 0x7fca3c004d40 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca500b2ad0 con 0x7fca3c0039b0 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c00b9c0 con 0x7fca500d5610 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4effd640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca47fff640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 -- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 msgr2=0x7fca3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 -- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0039b0 msgr2=0x7fca3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca4f7fe640 1 -- 172.21.15.40:0/2649759241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c003520 con 0x7fca3c004d40 2024-04-26T21:14:25.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.601+0000 7fca47fff640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500d5610 0x7fca3c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca48009f90 tx=0x7fca3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf40cf7d72ec0425 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48009c10 con 0x7fca3c004d40 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca48036070 con 0x7fca3c004d40 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca4e7fc640 1 -- 172.21.15.40:0/2649759241 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48031810 con 0x7fca3c004d40 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca556a9640 1 -- 172.21.15.40:0/2649759241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004d40 msgr2=0x7fca3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca556a9640 1 --2- 172.21.15.40:0/2649759241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca48009f90 tx=0x7fca3c005830 comp rx=0 tx=0).stop 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca4f7fe640 1 -- 172.21.15.40:0/2649759241 reap_dead start 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca556a9640 1 -- 172.21.15.40:0/2649759241 shutdown_connections 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.602+0000 7fca556a9640 1 -- 172.21.15.40:0/2649759241 >> 172.21.15.40:0/2649759241 conn(0x7fca5008ec70 msgr2=0x7fca5009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.603+0000 7fca556a9640 1 -- 172.21.15.40:0/2649759241 shutdown_connections 2024-04-26T21:14:25.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.603+0000 7fca556a9640 1 -- 172.21.15.40:0/2649759241 wait complete. 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 Processor -- start 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 -- start start 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca50156a20 0x7fca500b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b9ea0 0x7fca5009f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca500d40d0 con 0x7fca500a00f0 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca500d3f50 con 0x7fca50156a20 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca556a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fca500b35b0 con 0x7fca500b9ea0 2024-04-26T21:14:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.604+0000 7fca47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60124/0 (socket says 172.21.15.40:60124) 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 -- 172.21.15.40:0/3088796382 learned_addr learned my addr 172.21.15.40:0/3088796382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca4f7fe640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b9ea0 0x7fca5009f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca4effd640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca50156a20 0x7fca500b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 -- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b9ea0 msgr2=0x7fca5009f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b9ea0 0x7fca5009f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 -- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca50156a20 msgr2=0x7fca500b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca50156a20 0x7fca500b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 -- 172.21.15.40:0/3088796382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca500b9040 con 0x7fca500a00f0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca47fff640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fca38011660 tx=0x7fca38011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800f9c0 con 0x7fca500a00f0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3800fb60 con 0x7fca500a00f0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca500b7fa0 con 0x7fca500a00f0 2024-04-26T21:14:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38015460 con 0x7fca500a00f0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.605+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca500b8560 con 0x7fca500a00f0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.606+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca500ba2b0 con 0x7fca500a00f0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.607+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca38015600 con 0x7fca500a00f0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.607+0000 7fca477fe640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.607+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fca3808c590 con 0x7fca500a00f0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.607+0000 7fca4effd640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.608+0000 7fca4effd640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca40001050 tx=0x7fca40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.611+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca500ba2b0 con 0x7fca500a00f0 2024-04-26T21:14:25.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.798+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca500cbaf0 con 0x7fca500a00f0 2024-04-26T21:14:25.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.799+0000 7fca477fe640 1 -- 172.21.15.40:0/3088796382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16613 (secure 0 0 0) 0x7fca38019030 con 0x7fca500a00f0 2024-04-26T21:14:25.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:25.800 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":47,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:11.908309+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:08.531061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:09.214001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 msgr2=0x7fca3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca40001050 tx=0x7fca40009040 comp rx=0 tx=0).stop 2024-04-26T21:14:25.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 msgr2=0x7fca500b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fca38011660 tx=0x7fca38011b20 comp rx=0 tx=0).stop 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 shutdown_connections 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca500b9ea0 0x7fca5009f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca500a00f0 0x7fca500b9960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 --2- 172.21.15.40:0/3088796382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca50156a20 0x7fca500b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.801+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 >> 172.21.15.40:0/3088796382 conn(0x7fca5008ec70 msgr2=0x7fca500d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.802+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 shutdown_connections 2024-04-26T21:14:25.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.802+0000 7fca556a9640 1 -- 172.21.15.40:0/3088796382 wait complete. 2024-04-26T21:14:25.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:14:25.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.915+0000 7f965c8a0640 1 Processor -- start 2024-04-26T21:14:25.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.915+0000 7f965c8a0640 1 -- start start 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f965c8a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 0x7f96580cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f965c8a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f965815f750 0x7f96580b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f965c8a0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96580d03a0 con 0x7f96580d37d0 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f965c8a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96580b4d10 con 0x7f965815f750 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f965c8a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96580d0680 con 0x7f965815f370 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f9655d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f965815f750 0x7f96580b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f9655d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f965815f750 0x7f96580b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60128/0 (socket says 172.21.15.40:60128) 2024-04-26T21:14:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.916+0000 7f9655d9b640 1 -- 172.21.15.40:0/2402959010 learned_addr learned my addr 172.21.15.40:0/2402959010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f965659c640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 0x7f96580cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f9655d9b640 1 -- 172.21.15.40:0/2402959010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 msgr2=0x7f96580cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f9655d9b640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 0x7f96580cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f9655d9b640 1 -- 172.21.15.40:0/2402959010 >> v1:172.21.15.184:6789/0 conn(0x7f96580d37d0 legacy=0x7f96580d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f9655d9b640 1 -- 172.21.15.40:0/2402959010 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96580d0960 con 0x7f965815f750 2024-04-26T21:14:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f965659c640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 0x7f96580cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.917+0000 7f9655d9b640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f750 0x7f96580b47d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f964c009870 tx=0x7f964c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf3303419fa40d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965559a640 1 -- 172.21.15.40:0/2402959010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c037070 con 0x7f965815f750 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965559a640 1 -- 172.21.15.40:0/2402959010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c032440 con 0x7f965815f750 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965559a640 1 -- 172.21.15.40:0/2402959010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c032810 con 0x7f965815f750 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 -- 172.21.15.40:0/2402959010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f750 msgr2=0x7f96580b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f750 0x7f96580b47d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f964c009870 tx=0x7f964c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 -- 172.21.15.40:0/2402959010 shutdown_connections 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f750 0x7f96580b47d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 --2- 172.21.15.40:0/2402959010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f965815f370 0x7f96580cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 -- 172.21.15.40:0/2402959010 >> 172.21.15.40:0/2402959010 conn(0x7f965808ec10 msgr2=0x7f965809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 -- 172.21.15.40:0/2402959010 shutdown_connections 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.918+0000 7f965c8a0640 1 -- 172.21.15.40:0/2402959010 wait complete. 2024-04-26T21:14:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 Processor -- start 2024-04-26T21:14:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 -- start start 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96580d37d0 0x7f9658153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965815f750 0x7f96580b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96580b51c0 con 0x7f965815f370 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96580d0f20 con 0x7f965815f750 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.920+0000 7f965c8a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96580b5340 con 0x7f96580d37d0 2024-04-26T21:14:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60142/0 (socket says 172.21.15.40:60142) 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 -- 172.21.15.40:0/990628680 learned_addr learned my addr 172.21.15.40:0/990628680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f965659c640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96580d37d0 0x7f9658153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9656d9d640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965815f750 0x7f96580b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 -- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96580d37d0 msgr2=0x7f9658153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96580d37d0 0x7f9658153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 -- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965815f750 msgr2=0x7f96580b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965815f750 0x7f96580b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 -- 172.21.15.40:0/990628680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9658153bb0 con 0x7f965815f370 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9655d9b640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f964c000c00 tx=0x7f964c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c041020 con 0x7f965815f370 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c0306c0 con 0x7f965815f370 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581f4f90 con 0x7f965815f370 2024-04-26T21:14:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c037040 con 0x7f965815f370 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.921+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581f5550 con 0x7f965815f370 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.922+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965806a290 con 0x7f965815f370 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.923+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f964c044020 con 0x7f965815f370 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.923+0000 7f9646ffd640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.923+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f964c0ac2a0 con 0x7f965815f370 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.923+0000 7f965659c640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.924+0000 7f965659c640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 0x7f963806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9648006fd0 tx=0x7f9648008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:25.928+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f964c04b030 con 0x7f965815f370 2024-04-26T21:14:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.151+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9658069690 con 0x7f965815f370 2024-04-26T21:14:26.153 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:14:26.152+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:14:26.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.153+0000 7f9646ffd640 1 -- 172.21.15.40:0/990628680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f9658153bb0 con 0x7f965815f370 2024-04-26T21:14:26.154 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 msgr2=0x7f963806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 0x7f963806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9648006fd0 tx=0x7f9648008040 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 msgr2=0x7f96580ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f964c000c00 tx=0x7f964c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 shutdown_connections 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f965815f750 0x7f96580b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f965815f370 0x7f96580ced30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.155+0000 7f965c8a0640 1 --2- 172.21.15.40:0/990628680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96580d37d0 0x7f9658153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.156+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 >> 172.21.15.40:0/990628680 conn(0x7f965808ec10 msgr2=0x7f96580b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.156+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 shutdown_connections 2024-04-26T21:14:26.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.156+0000 7f965c8a0640 1 -- 172.21.15.40:0/990628680 wait complete. 2024-04-26T21:14:26.158 INFO:tasks.ceph.mds.a.smithi040.stderr: -2275> 2024-04-26T21:14:11.869+0000 7f38c63b4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:14:26.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:14:26.185 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:14:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.272+0000 7f1bcd0c8640 1 Processor -- start 2024-04-26T21:14:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.272+0000 7f1bcd0c8640 1 -- start start 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bcd0c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 0x7f1bc80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bcd0c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1bc8161cb0 0x7f1bc80d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bcd0c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc80cdd40 con 0x7f1bc80d1d90 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bcd0c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc80d2280 con 0x7f1bc8161cb0 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bcd0c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80cdfe0 con 0x7f1bc815e7e0 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1bc8161cb0 0x7f1bc80d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc759e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1bc80d1d90 0x7f1bc8161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35136/0 (socket says 172.21.15.40:35136) 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc759e640 1 -- 172.21.15.40:0/947031829 learned_addr learned my addr 172.21.15.40:0/947031829 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc6d9d640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 0x7f1bc80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc659c640 1 -- 172.21.15.40:0/947031829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 msgr2=0x7f1bc80ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc659c640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 0x7f1bc80ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc659c640 1 -- 172.21.15.40:0/947031829 >> v1:172.21.15.40:6789/0 conn(0x7f1bc80d1d90 legacy=0x7f1bc8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc659c640 1 -- 172.21.15.40:0/947031829 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80ce2c0 con 0x7f1bc8161cb0 2024-04-26T21:14:26.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc5d9b640 1 -- 172.21.15.40:0/947031829 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3777801455 0 0) 0x7f1bc80cdd40 con 0x7f1bc80d1d90 2024-04-26T21:14:26.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.273+0000 7f1bc6d9d640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 0x7f1bc80ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:26.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.274+0000 7f1bc659c640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc8161cb0 0x7f1bc80d1850 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39914f13ffa67b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:26.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.274+0000 7f1bc5d9b640 1 -- 172.21.15.40:0/947031829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037070 con 0x7f1bc8161cb0 2024-04-26T21:14:26.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.274+0000 7f1bc5d9b640 1 -- 172.21.15.40:0/947031829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc032440 con 0x7f1bc8161cb0 2024-04-26T21:14:26.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.274+0000 7f1bc5d9b640 1 -- 172.21.15.40:0/947031829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc032810 con 0x7f1bc8161cb0 2024-04-26T21:14:26.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/947031829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc8161cb0 msgr2=0x7f1bc80d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc8161cb0 0x7f1bc80d1850 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a180 comp rx=0 tx=0).stop 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/947031829 shutdown_connections 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc8161cb0 0x7f1bc80d1850 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/947031829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc815e7e0 0x7f1bc80ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/947031829 >> 172.21.15.40:0/947031829 conn(0x7f1bc809ba40 msgr2=0x7f1bc809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/947031829 shutdown_connections 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.275+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/947031829 wait complete. 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.276+0000 7f1bcd0c8640 1 Processor -- start 2024-04-26T21:14:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.276+0000 7f1bcd0c8640 1 -- start start 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc80d1d90 0x7f1bc814c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc8161cb0 0x7f1bc80b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80cf2e0 con 0x7f1bc815e7e0 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80ce880 con 0x7f1bc80d1d90 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bcd0c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc8161790 con 0x7f1bc8161cb0 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bc659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bc659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60174/0 (socket says 172.21.15.40:60174) 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bc659c640 1 -- 172.21.15.40:0/3751363342 learned_addr learned my addr 172.21.15.40:0/3751363342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bc759e640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc8161cb0 0x7f1bc80b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.277+0000 7f1bc6d9d640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc80d1d90 0x7f1bc814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 -- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc8161cb0 msgr2=0x7f1bc80b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc8161cb0 0x7f1bc80b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 -- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc80d1d90 msgr2=0x7f1bc814c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc80d1d90 0x7f1bc814c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 -- 172.21.15.40:0/3751363342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80ca060 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bc659c640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc035040 tx=0x7f1bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037040 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc030770 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc8160a70 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc030b80 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.278+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc80ca600 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.279+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc806a290 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.280+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bbc044020 con 0x7f1bc815e7e0 2024-04-26T21:14:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.280+0000 7f1baf7fe640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 0x7f1bb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.280+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f1bbc0ac230 con 0x7f1bc815e7e0 2024-04-26T21:14:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.280+0000 7f1bc6d9d640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 0x7f1bb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.281+0000 7f1bc6d9d640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 0x7f1bb406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80075b0 tx=0x7f1bb8007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.284+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bbc032e20 con 0x7f1bc815e7e0 2024-04-26T21:14:26.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:26.509+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1bc80d1010 con 0x7f1bc815e7e0 2024-04-26T21:14:27.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.157+0000 7f1baf7fe640 1 -- 172.21.15.40:0/3751363342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f1bbc07a570 con 0x7f1bc815e7e0 2024-04-26T21:14:27.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 msgr2=0x7f1bb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 0x7f1bb406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80075b0 tx=0x7f1bb8007540 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 msgr2=0x7f1bc8160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc035040 tx=0x7f1bbc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 shutdown_connections 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bb406c8f0 0x7f1bb406edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bc8161cb0 0x7f1bc80b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc815e7e0 0x7f1bc8160530 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 --2- 172.21.15.40:0/3751363342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc80d1d90 0x7f1bc814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 >> 172.21.15.40:0/3751363342 conn(0x7f1bc809ba40 msgr2=0x7f1bc80b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 shutdown_connections 2024-04-26T21:14:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.160+0000 7f1bcd0c8640 1 -- 172.21.15.40:0/3751363342 wait complete. 2024-04-26T21:14:27.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:14:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.328+0000 7f8a618c8640 1 Processor -- start 2024-04-26T21:14:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.328+0000 7f8a618c8640 1 -- start start 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a618c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a5c0cee20 0x7f8a5c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a618c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a5c0d3a30 0x7f8a5c09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a618c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5c0d1240 con 0x7f8a5c09bf80 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a618c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c09c3f0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a618c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c0d1e10 con 0x7f8a5c0cee20 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a5a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a5c0d3a30 0x7f8a5c09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a5b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a5c09bf80 0x7f8a5c1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35146/0 (socket says 172.21.15.40:35146) 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a5b7fe640 1 -- 172.21.15.40:0/2766887912 learned_addr learned my addr 172.21.15.40:0/2766887912 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.329+0000 7f8a5affd640 1 --2- 172.21.15.40:0/2766887912 >> v2:172.21.15.184:3300/0 conn(0x7f8a5c0cee20 0x7f8a5c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a5a7fc640 1 -- 172.21.15.40:0/2766887912 >> v2:172.21.15.184:3300/0 conn(0x7f8a5c0cee20 msgr2=0x7f8a5c06d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a5a7fc640 1 --2- 172.21.15.40:0/2766887912 >> v2:172.21.15.184:3300/0 conn(0x7f8a5c0cee20 0x7f8a5c06d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a5a7fc640 1 -- 172.21.15.40:0/2766887912 >> v1:172.21.15.40:6789/0 conn(0x7f8a5c09bf80 legacy=0x7f8a5c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a5a7fc640 1 -- 172.21.15.40:0/2766887912 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c0d14e0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a59ffb640 1 -- 172.21.15.40:0/2766887912 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879333220 0 0) 0x7f8a5c0d1240 con 0x7f8a5c09bf80 2024-04-26T21:14:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a5a7fc640 1 --2- 172.21.15.40:0/2766887912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c09ba40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c073bd0 tx=0x7f8a5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d971cb76185c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a59ffb640 1 -- 172.21.15.40:0/2766887912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50037070 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a59ffb640 1 -- 172.21.15.40:0/2766887912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a50032440 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.330+0000 7f8a59ffb640 1 -- 172.21.15.40:0/2766887912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50032810 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 -- 172.21.15.40:0/2766887912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 msgr2=0x7f8a5c09ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 --2- 172.21.15.40:0/2766887912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c09ba40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c073bd0 tx=0x7f8a5002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 -- 172.21.15.40:0/2766887912 shutdown_connections 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 --2- 172.21.15.40:0/2766887912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c09ba40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 --2- 172.21.15.40:0/2766887912 >> v2:172.21.15.184:3300/0 conn(0x7f8a5c0cee20 0x7f8a5c06d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 -- 172.21.15.40:0/2766887912 >> 172.21.15.40:0/2766887912 conn(0x7f8a5c0d2f10 msgr2=0x7f8a5c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 -- 172.21.15.40:0/2766887912 shutdown_connections 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.331+0000 7f8a618c8640 1 -- 172.21.15.40:0/2766887912 wait complete. 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.332+0000 7f8a618c8640 1 Processor -- start 2024-04-26T21:14:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.332+0000 7f8a618c8640 1 -- start start 2024-04-26T21:14:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a5c09bf80 0x7f8a5c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 0x7f8a5c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c09c7a0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c0d1aa0 con 0x7f8a5c09bf80 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a618c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c0cf200 con 0x7f8a5c0cee20 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a5b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a5b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60198/0 (socket says 172.21.15.40:60198) 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a5b7fe640 1 -- 172.21.15.40:0/3958300447 learned_addr learned my addr 172.21.15.40:0/3958300447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.333+0000 7f8a5affd640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a5c09bf80 0x7f8a5c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5a7fc640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 0x7f8a5c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 -- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 msgr2=0x7f8a5c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 0x7f8a5c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 -- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a5c09bf80 msgr2=0x7f8a5c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a5c09bf80 0x7f8a5c0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 -- 172.21.15.40:0/3958300447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c15fd40 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5a7fc640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 0x7f8a5c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a5b7fe640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c7a0 tx=0x7f8a5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54018070 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5400a640 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400aab0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c0c85b0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.334+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c0c8220 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.335+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c06a290 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.336+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a5401b070 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.336+0000 7f8a43fff640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.336+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f8a5408b1c0 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.336+0000 7f8a5affd640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.337+0000 7f8a5affd640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 0x7f8a4806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.340+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a54059570 con 0x7f8a5c0d3a30 2024-04-26T21:14:27.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:27.525+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a5c0b4820 con 0x7f8a5c0d3a30 2024-04-26T21:14:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.159+0000 7f8a43fff640 1 -- 172.21.15.40:0/3958300447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f8a54058e70 con 0x7f8a5c0d3a30 2024-04-26T21:14:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:14:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.163+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 msgr2=0x7f8a4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.163+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 0x7f8a4806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c008040 comp rx=0 tx=0).stop 2024-04-26T21:14:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.163+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 msgr2=0x7f8a5c1f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.163+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c7a0 tx=0x7f8a5400cc60 comp rx=0 tx=0).stop 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 shutdown_connections 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a5c0d3a30 0x7f8a5c1f3f50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a5c0cee20 0x7f8a5c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 --2- 172.21.15.40:0/3958300447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a5c09bf80 0x7f8a5c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 >> 172.21.15.40:0/3958300447 conn(0x7f8a5c0d2f10 msgr2=0x7f8a5c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 shutdown_connections 2024-04-26T21:14:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.165+0000 7f8a618c8640 1 -- 172.21.15.40:0/3958300447 wait complete. 2024-04-26T21:14:28.181 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:14:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.291+0000 7f5544625640 1 Processor -- start 2024-04-26T21:14:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.291+0000 7f5544625640 1 -- start start 2024-04-26T21:14:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5544625640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f553c09c150 0x7f553c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5544625640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553c0d1240 con 0x7f553c0cee20 2024-04-26T21:14:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5544625640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553c0d14e0 con 0x7f553c0d3a30 2024-04-26T21:14:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5544625640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f553c0d1e10 con 0x7f553c09c150 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f553c09c150 0x7f553c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f553c09c150 0x7f553c06d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60210/0 (socket says 172.21.15.40:60210) 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f55423c1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f553c0cee20 0x7f553c09ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35154/0 (socket says 172.21.15.40:35154) 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 -- 172.21.15.40:0/1520909109 learned_addr learned my addr 172.21.15.40:0/1520909109 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 -- 172.21.15.40:0/1520909109 >> v1:172.21.15.184:6789/0 conn(0x7f553c0d3a30 legacy=0x7f553c15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 -- 172.21.15.40:0/1520909109 >> v1:172.21.15.40:6789/0 conn(0x7f553c0cee20 legacy=0x7f553c09ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f5541bc0640 1 -- 172.21.15.40:0/1520909109 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0d17c0 con 0x7f553c09c150 2024-04-26T21:14:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.292+0000 7f55413bf640 1 -- 172.21.15.40:0/1520909109 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785616451 0 0) 0x7f553c0d1240 con 0x7f553c0cee20 2024-04-26T21:14:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.293+0000 7f5541bc0640 1 --2- 172.21.15.40:0/1520909109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c06d5c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f552c009870 tx=0x7f552c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e00b0d546b1211d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.293+0000 7f55413bf640 1 -- 172.21.15.40:0/1520909109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c037070 con 0x7f553c09c150 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.293+0000 7f55413bf640 1 -- 172.21.15.40:0/1520909109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c032440 con 0x7f553c09c150 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.293+0000 7f55413bf640 1 -- 172.21.15.40:0/1520909109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c032810 con 0x7f553c09c150 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 -- 172.21.15.40:0/1520909109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 msgr2=0x7f553c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 --2- 172.21.15.40:0/1520909109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c06d5c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f552c009870 tx=0x7f552c02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 -- 172.21.15.40:0/1520909109 shutdown_connections 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 --2- 172.21.15.40:0/1520909109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c06d5c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 -- 172.21.15.40:0/1520909109 >> 172.21.15.40:0/1520909109 conn(0x7f553c0d2f10 msgr2=0x7f553c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 -- 172.21.15.40:0/1520909109 shutdown_connections 2024-04-26T21:14:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.294+0000 7f5544625640 1 -- 172.21.15.40:0/1520909109 wait complete. 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.295+0000 7f5544625640 1 Processor -- start 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.295+0000 7f5544625640 1 -- start start 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553c0cee20 0x7f553c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553c0d3a30 0x7f553c161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f553c0cf200 con 0x7f553c09c150 2024-04-26T21:14:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f553c09c7a0 con 0x7f553c0d3a30 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5544625640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f553c0cf380 con 0x7f553c0cee20 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60216/0 (socket says 172.21.15.40:60216) 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 -- 172.21.15.40:0/2294564686 learned_addr learned my addr 172.21.15.40:0/2294564686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5542bc2640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553c0d3a30 0x7f553c161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f5541bc0640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553c0cee20 0x7f553c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 -- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553c0cee20 msgr2=0x7f553c0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553c0cee20 0x7f553c0c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 -- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553c0d3a30 msgr2=0x7f553c161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553c0d3a30 0x7f553c161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 -- 172.21.15.40:0/2294564686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0ccee0 con 0x7f553c09c150 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.296+0000 7f55423c1640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f553800c4d0 tx=0x7f553800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.297+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538018070 con 0x7f553c09c150 2024-04-26T21:14:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.297+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5538007500 con 0x7f553c09c150 2024-04-26T21:14:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.297+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c161a40 con 0x7f553c09c150 2024-04-26T21:14:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.297+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004a20 con 0x7f553c09c150 2024-04-26T21:14:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.297+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c0c7790 con 0x7f553c09c150 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.298+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c06a290 con 0x7f553c09c150 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.298+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553801b070 con 0x7f553c09c150 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.299+0000 7f5532ffd640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 0x7f552006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.299+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5357+0+0 (secure 0 0 0) 0x7f553808ace0 con 0x7f553c09c150 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.299+0000 7f5541bc0640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 0x7f552006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.299+0000 7f5541bc0640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 0x7f552006ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f552c034000 tx=0x7f552c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.303+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5538059270 con 0x7f553c09c150 2024-04-26T21:14:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:28.489+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f553c0b4820 con 0x7f553c09c150 2024-04-26T21:14:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.165+0000 7f5532ffd640 1 -- 172.21.15.40:0/2294564686 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f5538058b70 con 0x7f553c09c150 2024-04-26T21:14:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:14:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.169+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 msgr2=0x7f552006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.169+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 0x7f552006ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f552c034000 tx=0x7f552c02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.169+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 msgr2=0x7f553c0c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.169+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f553800c4d0 tx=0x7f553800c990 comp rx=0 tx=0).stop 2024-04-26T21:14:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 shutdown_connections 2024-04-26T21:14:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f552006c920 0x7f552006ede0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553c0d3a30 0x7f553c161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f553c0cee20 0x7f553c0c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 --2- 172.21.15.40:0/2294564686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f553c09c150 0x7f553c0c99f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.173+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 >> 172.21.15.40:0/2294564686 conn(0x7f553c0d2f10 msgr2=0x7f553c0b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.174+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 shutdown_connections 2024-04-26T21:14:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.175+0000 7f5544625640 1 -- 172.21.15.40:0/2294564686 wait complete. 2024-04-26T21:14:29.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:14:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.296+0000 7efd1ba47640 1 Processor -- start 2024-04-26T21:14:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.296+0000 7efd1ba47640 1 -- start start 2024-04-26T21:14:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd1415f490 0x7efd140d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd140d0ff0 0x7efd140d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd140ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efd140d2570 con 0x7efd140d0ff0 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efd140d4240 con 0x7efd1415f490 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd1ba47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd140d31a0 con 0x7efd140d2120 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd18fe2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd140d0ff0 0x7efd140d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd18fe2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd140d0ff0 0x7efd140d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60218/0 (socket says 172.21.15.40:60218) 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd18fe2640 1 -- 172.21.15.40:0/378528227 learned_addr learned my addr 172.21.15.40:0/378528227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd197e3640 1 --2- 172.21.15.40:0/378528227 >> v2:172.21.15.184:3300/0 conn(0x7efd1415f490 0x7efd140d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd19fe4640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd140ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.297+0000 7efd18fe2640 1 -- 172.21.15.40:0/378528227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 msgr2=0x7efd140ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd18fe2640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd140ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd18fe2640 1 -- 172.21.15.40:0/378528227 >> v2:172.21.15.184:3300/0 conn(0x7efd1415f490 msgr2=0x7efd140d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd18fe2640 1 --2- 172.21.15.40:0/378528227 >> v2:172.21.15.184:3300/0 conn(0x7efd1415f490 0x7efd140d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd18fe2640 1 -- 172.21.15.40:0/378528227 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd140d3480 con 0x7efd140d0ff0 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd18fe2640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd140d1be0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd140b7cf0 tx=0x7efd0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5752c62a9ae739 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd13fff640 1 -- 172.21.15.40:0/378528227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08037070 con 0x7efd140d0ff0 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd13fff640 1 -- 172.21.15.40:0/378528227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08032440 con 0x7efd140d0ff0 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.298+0000 7efd13fff640 1 -- 172.21.15.40:0/378528227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08032810 con 0x7efd140d0ff0 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 -- 172.21.15.40:0/378528227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 msgr2=0x7efd140d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd140d1be0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd140b7cf0 tx=0x7efd0802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 -- 172.21.15.40:0/378528227 shutdown_connections 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd140ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 --2- 172.21.15.40:0/378528227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd140d1be0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 --2- 172.21.15.40:0/378528227 >> v2:172.21.15.184:3300/0 conn(0x7efd1415f490 0x7efd140d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 -- 172.21.15.40:0/378528227 >> 172.21.15.40:0/378528227 conn(0x7efd1409e530 msgr2=0x7efd140d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 -- 172.21.15.40:0/378528227 shutdown_connections 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.299+0000 7efd1ba47640 1 -- 172.21.15.40:0/378528227 wait complete. 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.300+0000 7efd1ba47640 1 Processor -- start 2024-04-26T21:14:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.300+0000 7efd1ba47640 1 -- start start 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd141524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1415f490 0x7efd1415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd140d3d20 con 0x7efd140d0ff0 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd140d3a40 con 0x7efd1415f490 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd1ba47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd1415f870 con 0x7efd140d2120 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60232/0 (socket says 172.21.15.40:60232) 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 -- 172.21.15.40:0/3506867482 learned_addr learned my addr 172.21.15.40:0/3506867482 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd19fe4640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1415f490 0x7efd1415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd18fe2640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd141524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 -- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 msgr2=0x7efd141524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd141524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 -- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1415f490 msgr2=0x7efd1415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1415f490 0x7efd1415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.301+0000 7efd197e3640 1 -- 172.21.15.40:0/3506867482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd140c9ef0 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd197e3640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007ef0 tx=0x7efd0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c014620 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c019070 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd140ca1d0 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c014a00 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.302+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd140ca790 con 0x7efd140d0ff0 2024-04-26T21:14:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.303+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1406a290 con 0x7efd140d0ff0 2024-04-26T21:14:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.304+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd0c014ba0 con 0x7efd140d0ff0 2024-04-26T21:14:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.304+0000 7efd11ffb640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.304+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7efd0c08c130 con 0x7efd140d0ff0 2024-04-26T21:14:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.304+0000 7efd18fe2640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.305+0000 7efd18fe2640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 0x7efd0006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd08034000 tx=0x7efd0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.308+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd0c05a840 con 0x7efd140d0ff0 2024-04-26T21:14:29.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.492+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efd140d5900 con 0x7efd140d0ff0 2024-04-26T21:14:29.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.495+0000 7efd11ffb640 1 -- 172.21.15.40:0/3506867482 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7efd0c05a140 con 0x7efd140d0ff0 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.497+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 msgr2=0x7efd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.497+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 0x7efd0006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd08034000 tx=0x7efd0802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.497+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 msgr2=0x7efd141f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.497+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007ef0 tx=0x7efd0c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 shutdown_connections 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1415f490 0x7efd1415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd140d2120 0x7efd141524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 --2- 172.21.15.40:0/3506867482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd140d0ff0 0x7efd141f55c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 >> 172.21.15.40:0/3506867482 conn(0x7efd1409e530 msgr2=0x7efd1406e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 shutdown_connections 2024-04-26T21:14:29.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.498+0000 7efd1ba47640 1 -- 172.21.15.40:0/3506867482 wait complete. 2024-04-26T21:14:29.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-04-26T21:14:29.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.609+0000 7f2187ec5640 1 Processor -- start 2024-04-26T21:14:29.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.609+0000 7f2187ec5640 1 -- start start 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2187ec5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2180145980 0x7f2180160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2187ec5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21800b3de0 0x7f21800b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2187ec5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21800b4260 con 0x7f21800b5fd0 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2187ec5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21800b6440 con 0x7f21800b3de0 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2187ec5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21800b5140 con 0x7f2180145980 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21800b3de0 0x7f21800b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21800b3de0 0x7f21800b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60234/0 (socket says 172.21.15.40:60234) 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 -- 172.21.15.40:0/4093843000 learned_addr learned my addr 172.21.15.40:0/4093843000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185c61640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2180145980 0x7f2180160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 -- 172.21.15.40:0/4093843000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2180145980 msgr2=0x7f2180160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2180145980 0x7f2180160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 -- 172.21.15.40:0/4093843000 >> v1:172.21.15.184:6789/0 conn(0x7f21800b5fd0 legacy=0x7f218015c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.610+0000 7f2185460640 1 -- 172.21.15.40:0/4093843000 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21800b4530 con 0x7f21800b3de0 2024-04-26T21:14:29.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.611+0000 7f2185460640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21800b3de0 0x7f21800b5a90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2170009870 tx=0x7f217002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49e29c0e5193f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.611+0000 7f2184c5f640 1 -- 172.21.15.40:0/4093843000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170037070 con 0x7f21800b3de0 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.611+0000 7f2184c5f640 1 -- 172.21.15.40:0/4093843000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2170032440 con 0x7f21800b3de0 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.611+0000 7f2184c5f640 1 -- 172.21.15.40:0/4093843000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170032810 con 0x7f21800b3de0 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 -- 172.21.15.40:0/4093843000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21800b3de0 msgr2=0x7f21800b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21800b3de0 0x7f21800b5a90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2170009870 tx=0x7f217002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 -- 172.21.15.40:0/4093843000 shutdown_connections 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21800b3de0 0x7f21800b5a90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 --2- 172.21.15.40:0/4093843000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2180145980 0x7f2180160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 -- 172.21.15.40:0/4093843000 >> 172.21.15.40:0/4093843000 conn(0x7f21800d0d30 msgr2=0x7f21800d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 -- 172.21.15.40:0/4093843000 shutdown_connections 2024-04-26T21:14:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.612+0000 7f2187ec5640 1 -- 172.21.15.40:0/4093843000 wait complete. 2024-04-26T21:14:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.613+0000 7f2187ec5640 1 Processor -- start 2024-04-26T21:14:29.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.613+0000 7f2187ec5640 1 -- start start 2024-04-26T21:14:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21800b3de0 0x7f21800aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21800b5fd0 0x7f2180159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b4dd0 con 0x7f2180145980 2024-04-26T21:14:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b4af0 con 0x7f21800b3de0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2187ec5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21800730e0 con 0x7f21800b5fd0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60244/0 (socket says 172.21.15.40:60244) 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 -- 172.21.15.40:0/3643853651 learned_addr learned my addr 172.21.15.40:0/3643853651 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2185460640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21800b5fd0 0x7f2180159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2185c61640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21800b3de0 0x7f21800aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 -- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21800b5fd0 msgr2=0x7f2180159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21800b5fd0 0x7f2180159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 -- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21800b3de0 msgr2=0x7f21800aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21800b3de0 0x7f21800aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 -- 172.21.15.40:0/3643853651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21800af590 con 0x7f2180145980 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.614+0000 7f2186462640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f217800c730 tx=0x7f217800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.615+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178003b90 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.615+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178003d30 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.615+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218013d4e0 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.615+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800aa90 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.615+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218013d190 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.616+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218006a290 con 0x7f2180145980 2024-04-26T21:14:29.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.616+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f217800ac30 con 0x7f2180145980 2024-04-26T21:14:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.617+0000 7f21767fc640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.617+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f217808b630 con 0x7f2180145980 2024-04-26T21:14:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.617+0000 7f2185c61640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.617+0000 7f2185c61640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 0x7f216406edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f217c009aa0 tx=0x7f217c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:29.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.621+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2178059700 con 0x7f2180145980 2024-04-26T21:14:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:29.879+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f2180073cc0 con 0x7f2180145980 2024-04-26T21:14:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.166+0000 7f21767fc640 1 -- 172.21.15.40:0/3643853651 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v146) ==== 162+0+0 (secure 0 0 0) 0x7f217805da00 con 0x7f2180145980 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 msgr2=0x7f216406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 0x7f216406edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f217c009aa0 tx=0x7f217c009190 comp rx=0 tx=0).stop 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 msgr2=0x7f21801f90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f217800c730 tx=0x7f217800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 shutdown_connections 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f216406c8f0 0x7f216406edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2180145980 0x7f21801f90d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21800b5fd0 0x7f2180159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 --2- 172.21.15.40:0/3643853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21800b3de0 0x7f21800aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 >> 172.21.15.40:0/3643853651 conn(0x7f21800d0d30 msgr2=0x7f218015b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 shutdown_connections 2024-04-26T21:14:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.169+0000 7f2187ec5640 1 -- 172.21.15.40:0/3643853651 wait complete. 2024-04-26T21:14:30.181 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... skipped 'Require FUSE client to inject client release failure' 2024-04-26T21:14:30.181 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:14:30.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.281+0000 7fb63c751640 1 Processor -- start 2024-04-26T21:14:30.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.281+0000 7fb63c751640 1 -- start start 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.282+0000 7fb63c751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634148a30 0x7fb634073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.282+0000 7fb63c751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6340b3af0 0x7fb6340b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.282+0000 7fb63c751640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6340cfa80 con 0x7fb634160640 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.282+0000 7fb63c751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb6340b44a0 con 0x7fb6340b3af0 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.282+0000 7fb63c751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb6340cfd60 con 0x7fb634148a30 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6340b3af0 0x7fb6340b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6340b3af0 0x7fb6340b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60256/0 (socket says 172.21.15.40:60256) 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb639cec640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb634160640 0x7fb6341636a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35166/0 (socket says 172.21.15.40:35166) 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 -- 172.21.15.40:0/2099563456 learned_addr learned my addr 172.21.15.40:0/2099563456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63a4ed640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634148a30 0x7fb634073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 -- 172.21.15.40:0/2099563456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634148a30 msgr2=0x7fb634073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634148a30 0x7fb634073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 -- 172.21.15.40:0/2099563456 >> v1:172.21.15.40:6789/0 conn(0x7fb634160640 legacy=0x7fb6341636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb63acee640 1 -- 172.21.15.40:0/2099563456 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340d0040 con 0x7fb6340b3af0 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.283+0000 7fb6394eb640 1 -- 172.21.15.40:0/2099563456 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832854125 0 0) 0x7fb6340cfa80 con 0x7fb634160640 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63acee640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6340b3af0 0x7fb6340b3f60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb624009e50 tx=0x7fb62402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38029a1c059d9cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb6394eb640 1 -- 172.21.15.40:0/2099563456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624036070 con 0x7fb6340b3af0 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb6394eb640 1 -- 172.21.15.40:0/2099563456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624031600 con 0x7fb6340b3af0 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb6394eb640 1 -- 172.21.15.40:0/2099563456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240319d0 con 0x7fb6340b3af0 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 -- 172.21.15.40:0/2099563456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6340b3af0 msgr2=0x7fb6340b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6340b3af0 0x7fb6340b3f60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb624009e50 tx=0x7fb62402a680 comp rx=0 tx=0).stop 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 -- 172.21.15.40:0/2099563456 shutdown_connections 2024-04-26T21:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6340b3af0 0x7fb6340b3f60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 --2- 172.21.15.40:0/2099563456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634148a30 0x7fb634073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 -- 172.21.15.40:0/2099563456 >> 172.21.15.40:0/2099563456 conn(0x7fb6340ce8b0 msgr2=0x7fb6340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 -- 172.21.15.40:0/2099563456 shutdown_connections 2024-04-26T21:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.284+0000 7fb63c751640 1 -- 172.21.15.40:0/2099563456 wait complete. 2024-04-26T21:14:30.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 Processor -- start 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 -- start start 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 0x7fb63415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634160640 0x7fb6340b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb634163db0 con 0x7fb634148a30 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb634163180 con 0x7fb6340b3af0 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb63c751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb634074410 con 0x7fb634160640 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.286+0000 7fb639cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60268/0 (socket says 172.21.15.40:60268) 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 -- 172.21.15.40:0/131690063 learned_addr learned my addr 172.21.15.40:0/131690063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb63acee640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634160640 0x7fb6340b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb63a4ed640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 0x7fb63415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 -- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634160640 msgr2=0x7fb6340b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634160640 0x7fb6340b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 -- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 msgr2=0x7fb63415e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 0x7fb63415e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 -- 172.21.15.40:0/131690063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340afb50 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb63a4ed640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 0x7fb63415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb639cec640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb62800c3c0 tx=0x7fb62800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280074e0 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6280047e0 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6340afe30 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628004bd0 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.287+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6340b03f0 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.288+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63406a290 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.289+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb628011070 con 0x7fb634148a30 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.289+0000 7fb632ffd640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 0x7fb62006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.289+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fb62808b3a0 con 0x7fb634148a30 2024-04-26T21:14:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.289+0000 7fb63a4ed640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 0x7fb62006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.290+0000 7fb63a4ed640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 0x7fb62006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ce700 tx=0x7fb62c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.294+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb628059b60 con 0x7fb634148a30 2024-04-26T21:14:30.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.485+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb634163b90 con 0x7fb634148a30 2024-04-26T21:14:30.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.486+0000 7fb632ffd640 1 -- 172.21.15.40:0/131690063 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13018 (secure 0 0 0) 0x7fb628059460 con 0x7fb634148a30 2024-04-26T21:14:30.487 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:30.487 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":50,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:29.160931+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/363892172":"2024-04-27T21:14:26.152841+0000","172.21.15.40:6833/363892172":"2024-04-27T21:14:26.152841+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.487+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 msgr2=0x7fb62006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.487+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 0x7fb62006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ce700 tx=0x7fb62c008040 comp rx=0 tx=0).stop 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.487+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 msgr2=0x7fb6340c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.487+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb62800c3c0 tx=0x7fb62800c880 comp rx=0 tx=0).stop 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 shutdown_connections 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb62006c8f0 0x7fb62006edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb634160640 0x7fb6340b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb634148a30 0x7fb6340c7890 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 --2- 172.21.15.40:0/131690063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6340b3af0 0x7fb63415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 >> 172.21.15.40:0/131690063 conn(0x7fb6340ce8b0 msgr2=0x7fb63406fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 shutdown_connections 2024-04-26T21:14:30.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.488+0000 7fb63c751640 1 -- 172.21.15.40:0/131690063 wait complete. 2024-04-26T21:14:30.500 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:30.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.602+0000 7fb6867f0640 1 Processor -- start 2024-04-26T21:14:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.602+0000 7fb6867f0640 1 -- start start 2024-04-26T21:14:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb6867f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb6800d3820 0x7fb6800b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb6867f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6800b60b0 0x7fb6800d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb6867f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6800d3c90 con 0x7fb68015f680 2024-04-26T21:14:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb6867f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb68015fa60 con 0x7fb6800b60b0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb6867f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb6800d3f50 con 0x7fb6800d3820 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb684d8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6800b60b0 0x7fb6800d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb684d8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6800b60b0 0x7fb6800d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35448/0 (socket says 172.21.15.40:35448) 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb684d8d640 1 -- 172.21.15.40:0/190746733 learned_addr learned my addr 172.21.15.40:0/190746733 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.603+0000 7fb67ffff640 1 --1- 172.21.15.40:0/190746733 >> v1:172.21.15.40:6789/0 conn(0x7fb68015f680 0x7fb680163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39246/0 (socket says 172.21.15.40:39246) 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb67f7fe640 1 --2- 172.21.15.40:0/190746733 >> v2:172.21.15.184:3300/0 conn(0x7fb6800d3820 0x7fb6800b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb684d8d640 1 -- 172.21.15.40:0/190746733 >> v2:172.21.15.184:3300/0 conn(0x7fb6800d3820 msgr2=0x7fb6800b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb684d8d640 1 --2- 172.21.15.40:0/190746733 >> v2:172.21.15.184:3300/0 conn(0x7fb6800d3820 0x7fb6800b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb684d8d640 1 -- 172.21.15.40:0/190746733 >> v1:172.21.15.40:6789/0 conn(0x7fb68015f680 legacy=0x7fb680163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb67effd640 1 -- 172.21.15.40:0/190746733 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726690570 0 0) 0x7fb6800d3c90 con 0x7fb68015f680 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb684d8d640 1 -- 172.21.15.40:0/190746733 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6800b5090 con 0x7fb6800b60b0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.604+0000 7fb684d8d640 1 --2- 172.21.15.40:0/190746733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6800b60b0 0x7fb6800d4f20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb680163040 tx=0x7fb67802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40866e8a716ddaac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb67effd640 1 -- 172.21.15.40:0/190746733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678036070 con 0x7fb6800b60b0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb67effd640 1 -- 172.21.15.40:0/190746733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb678031600 con 0x7fb6800b60b0 2024-04-26T21:14:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb67effd640 1 -- 172.21.15.40:0/190746733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780319d0 con 0x7fb6800b60b0 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 -- 172.21.15.40:0/190746733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6800b60b0 msgr2=0x7fb6800d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 --2- 172.21.15.40:0/190746733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6800b60b0 0x7fb6800d4f20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb680163040 tx=0x7fb67802a730 comp rx=0 tx=0).stop 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 -- 172.21.15.40:0/190746733 shutdown_connections 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 --2- 172.21.15.40:0/190746733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6800b60b0 0x7fb6800d4f20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 --2- 172.21.15.40:0/190746733 >> v2:172.21.15.184:3300/0 conn(0x7fb6800d3820 0x7fb6800b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 -- 172.21.15.40:0/190746733 >> 172.21.15.40:0/190746733 conn(0x7fb68008eca0 msgr2=0x7fb68009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 -- 172.21.15.40:0/190746733 shutdown_connections 2024-04-26T21:14:30.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.605+0000 7fb6867f0640 1 -- 172.21.15.40:0/190746733 wait complete. 2024-04-26T21:14:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.606+0000 7fb6867f0640 1 Processor -- start 2024-04-26T21:14:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 -- start start 2024-04-26T21:14:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 0x7fb6800d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6800d3820 0x7fb680162080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb6800d55e0 con 0x7fb68015f680 2024-04-26T21:14:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6800d5460 con 0x7fb6800d3820 2024-04-26T21:14:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb6867f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb6800d5760 con 0x7fb6800b60b0 2024-04-26T21:14:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb684d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb684d8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35462/0 (socket says 172.21.15.40:35462) 2024-04-26T21:14:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.607+0000 7fb684d8d640 1 -- 172.21.15.40:0/193406147 learned_addr learned my addr 172.21.15.40:0/193406147 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67f7fe640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6800d3820 0x7fb680162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67ffff640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 0x7fb6800d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 -- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 msgr2=0x7fb6800d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 0x7fb6800d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 -- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6800d3820 msgr2=0x7fb680162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6800d3820 0x7fb680162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 -- 172.21.15.40:0/193406147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6800d0e50 con 0x7fb68015f680 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67ffff640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 0x7fb6800d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb684d8d640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb678009f40 tx=0x7fb678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678041020 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb678037040 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6800d1130 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678036050 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.608+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6800d16f0 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.609+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68006a290 con 0x7fb68015f680 2024-04-26T21:14:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.610+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb678007bd0 con 0x7fb68015f680 2024-04-26T21:14:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.610+0000 7fb67cff9640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 0x7fb66006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.610+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fb6780abf10 con 0x7fb68015f680 2024-04-26T21:14:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.610+0000 7fb67ffff640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 0x7fb66006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.611+0000 7fb67ffff640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 0x7fb66006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb668000f60 tx=0x7fb668009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.614+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb67804b060 con 0x7fb68015f680 2024-04-26T21:14:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.798+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6800b4cd0 con 0x7fb68015f680 2024-04-26T21:14:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.798+0000 7fb67cff9640 1 -- 172.21.15.40:0/193406147 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7fb67807a650 con 0x7fb68015f680 2024-04-26T21:14:30.799 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 msgr2=0x7fb66006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 0x7fb66006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb668000f60 tx=0x7fb668009040 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 msgr2=0x7fb680159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb678009f40 tx=0x7fb678004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 shutdown_connections 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb66006c8f0 0x7fb66006edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb68015f680 0x7fb680159b70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6800d3820 0x7fb680162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 --2- 172.21.15.40:0/193406147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6800b60b0 0x7fb6800d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 >> 172.21.15.40:0/193406147 conn(0x7fb68008eca0 msgr2=0x7fb6801630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.800+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 shutdown_connections 2024-04-26T21:14:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.801+0000 7fb6867f0640 1 -- 172.21.15.40:0/193406147 wait complete. 2024-04-26T21:14:30.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.912+0000 7fd73b924640 1 Processor -- start 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 -- start start 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd73415f4a0 0x7fd7340b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7340afbd0 0x7fd7340b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd734074430 con 0x7fd7340b4710 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd734074710 con 0x7fd7340afbd0 2024-04-26T21:14:30.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.913+0000 7fd73b924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7340b0660 con 0x7fd73415f4a0 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7340afbd0 0x7fd7340b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7340afbd0 0x7fd7340b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35472/0 (socket says 172.21.15.40:35472) 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 -- 172.21.15.40:0/3796318587 learned_addr learned my addr 172.21.15.40:0/3796318587 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd7396c0640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd73415f4a0 0x7fd7340b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 -- 172.21.15.40:0/3796318587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd73415f4a0 msgr2=0x7fd7340b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd73415f4a0 0x7fd7340b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 -- 172.21.15.40:0/3796318587 >> v1:172.21.15.184:6789/0 conn(0x7fd7340b4710 legacy=0x7fd7340ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:30.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.914+0000 7fd738ebf640 1 -- 172.21.15.40:0/3796318587 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7340b0940 con 0x7fd7340afbd0 2024-04-26T21:14:30.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd738ebf640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7340afbd0 0x7fd7340b41d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd728009870 tx=0x7fd72802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de469c126281b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd733fff640 1 -- 172.21.15.40:0/3796318587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037070 con 0x7fd7340afbd0 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd733fff640 1 -- 172.21.15.40:0/3796318587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd728032440 con 0x7fd7340afbd0 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd733fff640 1 -- 172.21.15.40:0/3796318587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728032810 con 0x7fd7340afbd0 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 -- 172.21.15.40:0/3796318587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7340afbd0 msgr2=0x7fd7340b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7340afbd0 0x7fd7340b41d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd728009870 tx=0x7fd72802a210 comp rx=0 tx=0).stop 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 -- 172.21.15.40:0/3796318587 shutdown_connections 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7340afbd0 0x7fd7340b41d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 --2- 172.21.15.40:0/3796318587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd73415f4a0 0x7fd7340b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 -- 172.21.15.40:0/3796318587 >> 172.21.15.40:0/3796318587 conn(0x7fd734073910 msgr2=0x7fd734073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 -- 172.21.15.40:0/3796318587 shutdown_connections 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.915+0000 7fd73b924640 1 -- 172.21.15.40:0/3796318587 wait complete. 2024-04-26T21:14:30.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.916+0000 7fd73b924640 1 Processor -- start 2024-04-26T21:14:30.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 -- start start 2024-04-26T21:14:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 0x7fd73409bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 0x7fd7340ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd73415f880 con 0x7fd73415f4a0 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7340b0f00 con 0x7fd7340b4710 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd73b924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd73415fa00 con 0x7fd7340afbd0 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd739ec1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd739ec1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35478/0 (socket says 172.21.15.40:35478) 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.917+0000 7fd739ec1640 1 -- 172.21.15.40:0/1661013619 learned_addr learned my addr 172.21.15.40:0/1661013619 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd738ebf640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 0x7fd7340ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd7396c0640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 0x7fd73409bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 -- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 msgr2=0x7fd73409bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 0x7fd73409bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 -- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 msgr2=0x7fd7340ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:30.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 0x7fd7340ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 -- 172.21.15.40:0/1661013619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7340cd4d0 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd738ebf640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 0x7fd7340ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd7396c0640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 0x7fd73409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd739ec1640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c780 tx=0x7fd72400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724004b60 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724004d00 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73409c2d0 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400aa90 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.918+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73409c890 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.919+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73406a290 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.919+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd72400ac30 con 0x7fd73415f4a0 2024-04-26T21:14:30.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.920+0000 7fd731ffb640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.920+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd72408ba40 con 0x7fd73415f4a0 2024-04-26T21:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.920+0000 7fd7396c0640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.921+0000 7fd7396c0640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 0x7fd72006ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00b820 tx=0x7fd72c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:30.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:30.924+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd724020070 con 0x7fd73415f4a0 2024-04-26T21:14:31.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:31.180+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7fd7340b39a0 con 0x7fd73415f4a0 2024-04-26T21:14:31.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:14:31.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:14:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.172+0000 7fd731ffb640 1 -- 172.21.15.40:0/1661013619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v148) ==== 153+0+0 (secure 0 0 0) 0x7fd72408be20 con 0x7fd73415f4a0 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 msgr2=0x7fd72006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 0x7fd72006ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00b820 tx=0x7fd72c008040 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 msgr2=0x7fd73409d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c780 tx=0x7fd72400cc40 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 shutdown_connections 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73415f4a0 0x7fd73409d430 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340b4710 0x7fd7340ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 --2- 172.21.15.40:0/1661013619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7340afbd0 0x7fd73409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 >> 172.21.15.40:0/1661013619 conn(0x7fd734073910 msgr2=0x7fd73415bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 shutdown_connections 2024-04-26T21:14:32.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.175+0000 7fd73b924640 1 -- 172.21.15.40:0/1661013619 wait complete. 2024-04-26T21:14:32.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:14:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.299+0000 7f8c271bf640 1 Processor -- start 2024-04-26T21:14:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.299+0000 7f8c271bf640 1 -- start start 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c271bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c201631f0 0x7f8c200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c271bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c200d1170 0x7f8c201635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c271bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c2009c790 con 0x7f8c200b3300 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c271bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8c200d15f0 con 0x7f8c200d1170 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c271bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c20163ec0 con 0x7f8c201631f0 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c200d1170 0x7f8c201635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c2575c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c200b3300 0x7f8c200b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39252/0 (socket says 172.21.15.40:39252) 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c2575c640 1 -- 172.21.15.40:0/4246498515 learned_addr learned my addr 172.21.15.40:0/4246498515 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1ffff640 1 -- 172.21.15.40:0/4246498515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c201631f0 msgr2=0x7f8c200b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1ffff640 1 --2- 172.21.15.40:0/4246498515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c201631f0 0x7f8c200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1ffff640 1 -- 172.21.15.40:0/4246498515 >> v1:172.21.15.40:6789/0 conn(0x7f8c200b3300 legacy=0x7f8c200b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1ffff640 1 -- 172.21.15.40:0/4246498515 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c200d1a40 con 0x7f8c200d1170 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.300+0000 7f8c1f7fe640 1 -- 172.21.15.40:0/4246498515 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007987592 0 0) 0x7f8c2009c790 con 0x7f8c200b3300 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.301+0000 7f8c1ffff640 1 --2- 172.21.15.40:0/4246498515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200d1170 0x7f8c201635d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0097a0 tx=0x7f8c0c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980f0abea6db3fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.301+0000 7f8c1f7fe640 1 -- 172.21.15.40:0/4246498515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037070 con 0x7f8c200d1170 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.301+0000 7f8c1f7fe640 1 -- 172.21.15.40:0/4246498515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c02ae50 con 0x7f8c200d1170 2024-04-26T21:14:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.301+0000 7f8c1f7fe640 1 -- 172.21.15.40:0/4246498515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c032660 con 0x7f8c200d1170 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 -- 172.21.15.40:0/4246498515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200d1170 msgr2=0x7f8c201635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 --2- 172.21.15.40:0/4246498515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200d1170 0x7f8c201635d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0097a0 tx=0x7f8c0c02a100 comp rx=0 tx=0).stop 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 -- 172.21.15.40:0/4246498515 shutdown_connections 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 --2- 172.21.15.40:0/4246498515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200d1170 0x7f8c201635d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 --2- 172.21.15.40:0/4246498515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c201631f0 0x7f8c200b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 -- 172.21.15.40:0/4246498515 >> 172.21.15.40:0/4246498515 conn(0x7f8c2009ba40 msgr2=0x7f8c2009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 -- 172.21.15.40:0/4246498515 shutdown_connections 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.302+0000 7f8c271bf640 1 -- 172.21.15.40:0/4246498515 wait complete. 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.303+0000 7f8c271bf640 1 Processor -- start 2024-04-26T21:14:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.303+0000 7f8c271bf640 1 -- start start 2024-04-26T21:14:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c200d1170 0x7f8c2015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c201631f0 0x7f8c201f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c200d0230 con 0x7f8c200b3300 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c200d2000 con 0x7f8c201631f0 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c271bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2015fd30 con 0x7f8c200d1170 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35500/0 (socket says 172.21.15.40:35500) 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 -- 172.21.15.40:0/3302013779 learned_addr learned my addr 172.21.15.40:0/3302013779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c1ffff640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c200d1170 0x7f8c2015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c2575c640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c201631f0 0x7f8c201f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 -- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c200d1170 msgr2=0x7f8c2015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c200d1170 0x7f8c2015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 -- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c201631f0 msgr2=0x7f8c201f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c201631f0 0x7f8c201f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 -- 172.21.15.40:0/3302013779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c20160670 con 0x7f8c200b3300 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.304+0000 7f8c24f5b640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b440 tx=0x7f8c1400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.305+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14015020 con 0x7f8c200b3300 2024-04-26T21:14:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.305+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c14011890 con 0x7f8c200b3300 2024-04-26T21:14:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.305+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c20160950 con 0x7f8c200b3300 2024-04-26T21:14:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.305+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14011c80 con 0x7f8c200b3300 2024-04-26T21:14:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.305+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c200c84d0 con 0x7f8c200b3300 2024-04-26T21:14:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.306+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2006a290 con 0x7f8c200b3300 2024-04-26T21:14:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.306+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c14018020 con 0x7f8c200b3300 2024-04-26T21:14:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.306+0000 7f8c1d7fa640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 0x7f8c0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.307+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f8c1408b940 con 0x7f8c200b3300 2024-04-26T21:14:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.307+0000 7f8c1ffff640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 0x7f8c0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.307+0000 7f8c1ffff640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 0x7f8c0406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c034000 tx=0x7f8c0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.311+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c1401f020 con 0x7f8c200b3300 2024-04-26T21:14:32.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.494+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8c200c6870 con 0x7f8c200b3300 2024-04-26T21:14:32.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.498+0000 7f8c1d7fa640 1 -- 172.21.15.40:0/3302013779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f8c1400d040 con 0x7f8c200b3300 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.500+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 msgr2=0x7f8c0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.500+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 0x7f8c0406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c034000 tx=0x7f8c0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.500+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 msgr2=0x7f8c2015f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.500+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b440 tx=0x7f8c1400b900 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 shutdown_connections 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8c0406c8f0 0x7f8c0406edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c201631f0 0x7f8c201f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c200d1170 0x7f8c2015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 --2- 172.21.15.40:0/3302013779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c200b3300 0x7f8c2015f7f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 >> 172.21.15.40:0/3302013779 conn(0x7f8c2009ba40 msgr2=0x7f8c201eaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 shutdown_connections 2024-04-26T21:14:32.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.501+0000 7f8c271bf640 1 -- 172.21.15.40:0/3302013779 wait complete. 2024-04-26T21:14:32.513 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:14:32.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:32.559 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:32.608 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:14:32.608 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:32.609 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:32.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:32.624 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:32.624 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:32.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:32.641 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:32.641 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:32.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:32.679 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:32.680 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:32.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:32.699 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:14:32.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:32.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.818+0000 7fc0406fc640 1 Processor -- start 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.818+0000 7fc0406fc640 1 -- start start 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc0406fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc03815f340 0x7fc0380b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc0406fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0380b2510 con 0x7fc03815f720 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc0406fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0380b27f0 con 0x7fc0380d5610 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc0406fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380b2ad0 con 0x7fc03815f340 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc03dc97640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc03815f720 0x7fc0380d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39262/0 (socket says 172.21.15.40:39262) 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.819+0000 7fc03dc97640 1 -- 172.21.15.40:0/2909365890 learned_addr learned my addr 172.21.15.40:0/2909365890 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 37318637 0 0) 0x7fc0380b2510 con 0x7fc03815f720 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc024003520 con 0x7fc03815f720 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 --2- 172.21.15.40:0/2909365890 >> v2:172.21.15.184:3300/0 conn(0x7fc03815f340 0x7fc0380b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 835396881 0 0) 0x7fc024003520 con 0x7fc03815f720 2024-04-26T21:14:32.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 >> v2:172.21.15.184:3300/0 conn(0x7fc03815f340 msgr2=0x7fc0380b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 --2- 172.21.15.40:0/2909365890 >> v2:172.21.15.184:3300/0 conn(0x7fc03815f340 0x7fc0380b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 >> v1:172.21.15.184:6789/0 conn(0x7fc0380d5610 legacy=0x7fc0380b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380b2db0 con 0x7fc03815f720 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc028001670 con 0x7fc03815f720 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 >> v1:172.21.15.40:6789/0 conn(0x7fc03815f720 legacy=0x7fc0380d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 0x7fc024003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc024004800 0x7fc024004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0240051c0 0x7fc02400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380b2ad0 con 0x7fc0240039b0 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc02400c160 con 0x7fc0240051c0 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc02400c360 con 0x7fc024004800 2024-04-26T21:14:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 0x7fc024003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03dc97640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc024004800 0x7fc024004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03ec99640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0240051c0 0x7fc02400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 -- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc024004800 msgr2=0x7fc024004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc024004800 0x7fc024004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 -- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0240051c0 msgr2=0x7fc02400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0240051c0 0x7fc02400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03ec99640 1 -- 172.21.15.40:0/2909365890 reap_dead start 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.820+0000 7fc03e498640 1 -- 172.21.15.40:0/2909365890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc024003520 con 0x7fc0240039b0 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.821+0000 7fc03e498640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 0x7fc024003dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc034009ac0 tx=0x7fc024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209698407b716739 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.821+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034036070 con 0x7fc0240039b0 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.821+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034031400 con 0x7fc0240039b0 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.821+0000 7fc03d496640 1 -- 172.21.15.40:0/2909365890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034009690 con 0x7fc0240039b0 2024-04-26T21:14:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 -- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 msgr2=0x7fc024003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 0x7fc024003dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc034009ac0 tx=0x7fc024005830 comp rx=0 tx=0).stop 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 -- 172.21.15.40:0/2909365890 shutdown_connections 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2909365890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0240039b0 0x7fc024003dd0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 -- 172.21.15.40:0/2909365890 >> 172.21.15.40:0/2909365890 conn(0x7fc03808ec70 msgr2=0x7fc03809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 -- 172.21.15.40:0/2909365890 shutdown_connections 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.822+0000 7fc0406fc640 1 -- 172.21.15.40:0/2909365890 wait complete. 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.823+0000 7fc0406fc640 1 Processor -- start 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.823+0000 7fc0406fc640 1 -- start start 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0240039b0 0x7fc0380cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 0x7fc0380cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380b2210 con 0x7fc0381f8af0 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380b2090 con 0x7fc0380caeb0 2024-04-26T21:14:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc0406fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380d3d40 con 0x7fc0240039b0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35524/0 (socket says 172.21.15.40:35524) 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 -- 172.21.15.40:0/2449497980 learned_addr learned my addr 172.21.15.40:0/2449497980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03e498640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0240039b0 0x7fc0380cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03ec99640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 0x7fc0380cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 -- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0240039b0 msgr2=0x7fc0380cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0240039b0 0x7fc0380cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 -- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 msgr2=0x7fc0380cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 0x7fc0380cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 -- 172.21.15.40:0/2449497980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380cc860 con 0x7fc0381f8af0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03ec99640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 0x7fc0380cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.824+0000 7fc03dc97640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc028001eb0 tx=0x7fc02800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02800c630 con 0x7fc0381f8af0 2024-04-26T21:14:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028018070 con 0x7fc0381f8af0 2024-04-26T21:14:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0381f92d0 con 0x7fc0381f8af0 2024-04-26T21:14:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02800ca10 con 0x7fc0381f8af0 2024-04-26T21:14:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381f9890 con 0x7fc0381f8af0 2024-04-26T21:14:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.825+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380d40d0 con 0x7fc0381f8af0 2024-04-26T21:14:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.826+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc02800cbb0 con 0x7fc0381f8af0 2024-04-26T21:14:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.826+0000 7fc02effd640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 0x7fc01c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:32.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.827+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc02808b250 con 0x7fc0381f8af0 2024-04-26T21:14:32.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.827+0000 7fc03e498640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 0x7fc01c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:32.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.827+0000 7fc03e498640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 0x7fc01c06ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc034033000 tx=0x7fc034008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:32.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:32.831+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0380d40d0 con 0x7fc0381f8af0 2024-04-26T21:14:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.056+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc03806a290 con 0x7fc0381f8af0 2024-04-26T21:14:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.056+0000 7fc02effd640 1 -- 172.21.15.40:0/2449497980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3872 (secure 0 0 0) 0x7fc028059980 con 0x7fc0381f8af0 2024-04-26T21:14:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:33.062 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3697398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3697398471},{"type":"v1","addr":"172.21.15.184:6836","nonce":3697398471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":4994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3367808389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3367808389},{"type":"v1","addr":"172.21.15.184:6837","nonce":3367808389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1962748887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1962748887},{"type":"v1","addr":"172.21.15.40:6834","nonce":1962748887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/879571920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":879571920},{"type":"v1","addr":"172.21.15.40:6835","nonce":879571920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 msgr2=0x7fc01c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 0x7fc01c06ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc034033000 tx=0x7fc034008030 comp rx=0 tx=0).stop 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 msgr2=0x7fc0380d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc028001eb0 tx=0x7fc02800eba0 comp rx=0 tx=0).stop 2024-04-26T21:14:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 shutdown_connections 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc01c06c920 0x7fc01c06ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0380caeb0 0x7fc0380cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0381f8af0 0x7fc0380d1170 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 --2- 172.21.15.40:0/2449497980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0240039b0 0x7fc0380cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.058+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 >> 172.21.15.40:0/2449497980 conn(0x7fc03808ec70 msgr2=0x7fc03806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.059+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 shutdown_connections 2024-04-26T21:14:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:33.059+0000 7fc0406fc640 1 -- 172.21.15.40:0/2449497980 wait complete. 2024-04-26T21:14:33.071 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:14:33.072 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:14:33.072 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:14:33.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:33.072 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:14:33.072 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:14:33.072 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:14:33.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:33.073 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:14:33.073 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:14:33.073 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:14:33.073 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:33.073 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:14:33.073 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:14:33.073 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:14:33.073 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:14:33.081 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:14:33.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:14:33.082 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:14:33.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:14:33.087 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:14:33.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:14:33.124 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:14:33.125 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:33.125 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:14:33.125 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:33.130 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:14:33.130 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:33.199 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:14:33.199 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:14:33.200 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:14:33.198+0000 7fe9c00f5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-26T21:14:33.212 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:14:39.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-26T21:14:39.087 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:14:39.088 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:14:39.091 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:14:39.091 DEBUG:teuthology.parallel:result is None 2024-04-26T21:14:39.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:14:39.172 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:14:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.206+0000 7f2d9a0f1640 1 Processor -- start 2024-04-26T21:14:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.206+0000 7f2d9a0f1640 1 -- start start 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d9a0f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f370 0x7f2d940cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d9a0f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d9415f750 0x7f2d940b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d9a0f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d940d03a0 con 0x7f2d940d37d0 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d9a0f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d940b4d10 con 0x7f2d9415f750 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d9a0f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d940d0680 con 0x7f2d9415f370 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d92ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d9415f750 0x7f2d940b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d92ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d9415f750 0x7f2d940b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35570/0 (socket says 172.21.15.40:35570) 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1763023605 learned_addr learned my addr 172.21.15.40:0/1763023605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.207+0000 7f2d937fe640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f370 0x7f2d940cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1763023605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f370 msgr2=0x7f2d940cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:39.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d92ffd640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f370 0x7f2d940cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1763023605 >> v1:172.21.15.184:6789/0 conn(0x7f2d940d37d0 legacy=0x7f2d940d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:39.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1763023605 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940d0960 con 0x7f2d9415f750 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d92ffd640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f750 0x7f2d940b47d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009870 tx=0x7f2d8802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b39afb963c4adf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.208+0000 7f2d927fc640 1 -- 172.21.15.40:0/1763023605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88037070 con 0x7f2d9415f750 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d927fc640 1 -- 172.21.15.40:0/1763023605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d88032440 con 0x7f2d9415f750 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d927fc640 1 -- 172.21.15.40:0/1763023605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88032810 con 0x7f2d9415f750 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1763023605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f750 msgr2=0x7f2d940b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f750 0x7f2d940b47d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009870 tx=0x7f2d8802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1763023605 shutdown_connections 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f750 0x7f2d940b47d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1763023605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f370 0x7f2d940cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1763023605 >> 172.21.15.40:0/1763023605 conn(0x7f2d9408ec10 msgr2=0x7f2d9409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1763023605 shutdown_connections 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.209+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1763023605 wait complete. 2024-04-26T21:14:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.210+0000 7f2d9a0f1640 1 Processor -- start 2024-04-26T21:14:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 -- start start 2024-04-26T21:14:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 0x7f2d941f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 0x7f2d940b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940b51c0 con 0x7f2d9415f370 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940d0f20 con 0x7f2d940d37d0 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d9a0f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d940b5340 con 0x7f2d9415f750 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d92ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35580/0 (socket says 172.21.15.40:35580) 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.211+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1202251672 learned_addr learned my addr 172.21.15.40:0/1202251672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d937fe640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 0x7f2d941f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d93fff640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 0x7f2d940b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 msgr2=0x7f2d940b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 0x7f2d940b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 msgr2=0x7f2d941f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 0x7f2d941f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 -- 172.21.15.40:0/1202251672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940b7e90 con 0x7f2d9415f370 2024-04-26T21:14:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d937fe640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 0x7f2d941f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d93fff640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 0x7f2d940b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d92ffd640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c00 tx=0x7f2d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88041020 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d880306c0 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d940b8170 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88037040 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.212+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d940b8a10 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.213+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d940b8560 con 0x7f2d9415f370 2024-04-26T21:14:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.214+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d88044020 con 0x7f2d9415f370 2024-04-26T21:14:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.214+0000 7f2d7bfff640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 0x7f2d8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.214+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f2d880abf80 con 0x7f2d9415f370 2024-04-26T21:14:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.214+0000 7f2d937fe640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 0x7f2d8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.215+0000 7f2d937fe640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 0x7f2d8006ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.219+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d8804b030 con 0x7f2d9415f370 2024-04-26T21:14:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:39.415+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2d9406a290 con 0x7f2d9415f370 2024-04-26T21:14:40.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.203+0000 7f2d7bfff640 1 -- 172.21.15.40:0/1202251672 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7f2d8807a740 con 0x7f2d9415f370 2024-04-26T21:14:40.205 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:14:40.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 msgr2=0x7f2d8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 0x7f2d8006ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).stop 2024-04-26T21:14:40.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 msgr2=0x7f2d940cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c00 tx=0x7f2d880046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 shutdown_connections 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d8006c920 0x7f2d8006ede0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d9415f750 0x7f2d940b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9415f370 0x7f2d940cd130 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 --2- 172.21.15.40:0/1202251672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d940d37d0 0x7f2d941f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 >> 172.21.15.40:0/1202251672 conn(0x7f2d9408ec10 msgr2=0x7f2d9415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 shutdown_connections 2024-04-26T21:14:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.206+0000 7f2d9a0f1640 1 -- 172.21.15.40:0/1202251672 wait complete. 2024-04-26T21:14:40.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:14:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.327+0000 7f4eca9bf640 1 Processor -- start 2024-04-26T21:14:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.327+0000 7f4eca9bf640 1 -- start start 2024-04-26T21:14:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ec4148a30 0x7f4ec4073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4160640 0x7f4ec41636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec4163be0 0x7f4ec40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40cfa80 con 0x7f4ec4163be0 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40cfd60 con 0x7f4ec4148a30 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4eca9bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40d0040 con 0x7f4ec4160640 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec4163be0 0x7f4ec40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ec4163be0 0x7f4ec40b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35588/0 (socket says 172.21.15.40:35588) 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 -- 172.21.15.40:0/326481468 learned_addr learned my addr 172.21.15.40:0/326481468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec37fe640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4160640 0x7f4ec41636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec3fff640 1 --2- 172.21.15.40:0/326481468 >> v2:172.21.15.184:3300/0 conn(0x7f4ec4148a30 0x7f4ec4073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 -- 172.21.15.40:0/326481468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4160640 msgr2=0x7f4ec41636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4160640 0x7f4ec41636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 -- 172.21.15.40:0/326481468 >> v2:172.21.15.184:3300/0 conn(0x7f4ec4148a30 msgr2=0x7f4ec4073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 --2- 172.21.15.40:0/326481468 >> v2:172.21.15.184:3300/0 conn(0x7f4ec4148a30 0x7f4ec4073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.328+0000 7f4ec8f5c640 1 -- 172.21.15.40:0/326481468 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec4162ea0 con 0x7f4ec4163be0 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.329+0000 7f4ec8f5c640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4163be0 0x7f4ec40b3af0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cf6d0 tx=0x7f4ebc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d211b65a0087d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.329+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/326481468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc036070 con 0x7f4ec4163be0 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.329+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/326481468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc031600 con 0x7f4ec4163be0 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.329+0000 7f4ec2ffd640 1 -- 172.21.15.40:0/326481468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0319d0 con 0x7f4ec4163be0 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 -- 172.21.15.40:0/326481468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4163be0 msgr2=0x7f4ec40b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4163be0 0x7f4ec40b3af0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cf6d0 tx=0x7f4ebc02a610 comp rx=0 tx=0).stop 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 -- 172.21.15.40:0/326481468 shutdown_connections 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4163be0 0x7f4ec40b3af0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/326481468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4160640 0x7f4ec41636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/326481468 >> v2:172.21.15.184:3300/0 conn(0x7f4ec4148a30 0x7f4ec4073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 -- 172.21.15.40:0/326481468 >> 172.21.15.40:0/326481468 conn(0x7f4ec40ce8b0 msgr2=0x7f4ec40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 -- 172.21.15.40:0/326481468 shutdown_connections 2024-04-26T21:14:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.330+0000 7f4eca9bf640 1 -- 172.21.15.40:0/326481468 wait complete. 2024-04-26T21:14:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.331+0000 7f4eca9bf640 1 Processor -- start 2024-04-26T21:14:40.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.331+0000 7f4eca9bf640 1 -- start start 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 0x7f4ec41f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4163be0 0x7f4ec41f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4074590 con 0x7f4ec4148a30 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4074410 con 0x7f4ec4160640 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4eca9bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec4148e10 con 0x7f4ec4163be0 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35590/0 (socket says 172.21.15.40:35590) 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 -- 172.21.15.40:0/2235145700 learned_addr learned my addr 172.21.15.40:0/2235145700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec37fe640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 0x7f4ec41f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec8f5c640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4163be0 0x7f4ec41f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 -- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4163be0 msgr2=0x7f4ec41f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4163be0 0x7f4ec41f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 -- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 msgr2=0x7f4ec41f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 0x7f4ec41f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec3fff640 1 -- 172.21.15.40:0/2235145700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41f6b40 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.332+0000 7f4ec37fe640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 0x7f4ec41f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4ec3fff640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c820 tx=0x7f4eac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac015070 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac004020 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41f6e20 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0043f0 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.333+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41f73e0 con 0x7f4ec4148a30 2024-04-26T21:14:40.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.334+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec4147ab0 con 0x7f4ec4148a30 2024-04-26T21:14:40.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.334+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eac007830 con 0x7f4ec4148a30 2024-04-26T21:14:40.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.335+0000 7f4ec0ff9640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.335+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f4eac08b580 con 0x7f4ec4148a30 2024-04-26T21:14:40.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.335+0000 7f4ec37fe640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.335+0000 7f4ec37fe640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ec413fcd0 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.339+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ec4147ab0 con 0x7f4ec4148a30 2024-04-26T21:14:40.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.592+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4ec415f2f0 con 0x7f4ec4148a30 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.593+0000 7f4ec0ff9640 1 -- 172.21.15.40:0/2235145700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f4eac059d90 con 0x7f4ec4148a30 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:40.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:14:40.598 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.599 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.601 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:14:40.607 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 msgr2=0x7f4ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ec413fcd0 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 msgr2=0x7f4ec41401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c820 tx=0x7f4eac00cce0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 shutdown_connections 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ec4163be0 0x7f4ec41f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ec4160640 0x7f4ec41f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 --2- 172.21.15.40:0/2235145700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec4148a30 0x7f4ec41401e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 >> 172.21.15.40:0/2235145700 conn(0x7f4ec40ce8b0 msgr2=0x7f4ec40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 shutdown_connections 2024-04-26T21:14:40.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.595+0000 7f4eca9bf640 1 -- 172.21.15.40:0/2235145700 wait complete. 2024-04-26T21:14:40.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.709+0000 7eff91524640 1 Processor -- start 2024-04-26T21:14:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.709+0000 7eff91524640 1 -- start start 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff91524640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff8c13ff70 0x7eff8c146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff91524640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff8c140350 0x7eff8c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff91524640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8c0d05b0 con 0x7eff8c0cf810 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff91524640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff8c0d0890 con 0x7eff8c13ff70 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff91524640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7eff8c06e450 con 0x7eff8c140350 2024-04-26T21:14:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff8c13ff70 0x7eff8c146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff8c13ff70 0x7eff8c146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55634/0 (socket says 172.21.15.40:55634) 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 -- 172.21.15.40:0/2692235771 learned_addr learned my addr 172.21.15.40:0/2692235771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 -- 172.21.15.40:0/2692235771 >> v2:172.21.15.184:3300/0 conn(0x7eff8c140350 msgr2=0x7eff8c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 --2- 172.21.15.40:0/2692235771 >> v2:172.21.15.184:3300/0 conn(0x7eff8c140350 0x7eff8c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 -- 172.21.15.40:0/2692235771 >> v1:172.21.15.184:6789/0 conn(0x7eff8c0cf810 legacy=0x7eff8c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:40.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.710+0000 7eff8affd640 1 -- 172.21.15.40:0/2692235771 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c06e730 con 0x7eff8c13ff70 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.711+0000 7eff8affd640 1 --2- 172.21.15.40:0/2692235771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c13ff70 0x7eff8c146080 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0cf1c0 tx=0x7eff8402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93830913b187fcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.711+0000 7eff89ffb640 1 -- 172.21.15.40:0/2692235771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84037070 con 0x7eff8c13ff70 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.711+0000 7eff89ffb640 1 -- 172.21.15.40:0/2692235771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff84032440 con 0x7eff8c13ff70 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.711+0000 7eff89ffb640 1 -- 172.21.15.40:0/2692235771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84032810 con 0x7eff8c13ff70 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 -- 172.21.15.40:0/2692235771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c13ff70 msgr2=0x7eff8c146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 --2- 172.21.15.40:0/2692235771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c13ff70 0x7eff8c146080 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0cf1c0 tx=0x7eff8402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 -- 172.21.15.40:0/2692235771 shutdown_connections 2024-04-26T21:14:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 --2- 172.21.15.40:0/2692235771 >> v2:172.21.15.184:3300/0 conn(0x7eff8c140350 0x7eff8c1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 --2- 172.21.15.40:0/2692235771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c13ff70 0x7eff8c146080 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 -- 172.21.15.40:0/2692235771 >> 172.21.15.40:0/2692235771 conn(0x7eff8c0d0d30 msgr2=0x7eff8c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 -- 172.21.15.40:0/2692235771 shutdown_connections 2024-04-26T21:14:40.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.712+0000 7eff91524640 1 -- 172.21.15.40:0/2692235771 wait complete. 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.713+0000 7eff91524640 1 Processor -- start 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.713+0000 7eff91524640 1 -- start start 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 0x7eff8c1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 0x7eff8c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c06efe0 con 0x7eff8c0cf810 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c06ee60 con 0x7eff8c140350 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c06f160 con 0x7eff8c13ff70 2024-04-26T21:14:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55650/0 (socket says 172.21.15.40:55650) 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 -- 172.21.15.40:0/1846385298 learned_addr learned my addr 172.21.15.40:0/1846385298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8a7fc640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 0x7eff8c1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8b7fe640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 0x7eff8c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 -- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 msgr2=0x7eff8c1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 0x7eff8c1433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 -- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 msgr2=0x7eff8c1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 0x7eff8c1f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 -- 172.21.15.40:0/1846385298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c1443f0 con 0x7eff8c0cf810 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8a7fc640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 0x7eff8c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8b7fe640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 0x7eff8c1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff8affd640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7eff84000c00 tx=0x7eff840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84041070 con 0x7eff8c0cf810 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8403c870 con 0x7eff8c0cf810 2024-04-26T21:14:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84037040 con 0x7eff8c0cf810 2024-04-26T21:14:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.714+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c143b50 con 0x7eff8c0cf810 2024-04-26T21:14:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.715+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c144110 con 0x7eff8c0cf810 2024-04-26T21:14:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.716+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c06a290 con 0x7eff8c0cf810 2024-04-26T21:14:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.716+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff84044070 con 0x7eff8c0cf810 2024-04-26T21:14:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.717+0000 7eff837fe640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 0x7eff6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.717+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7eff840ac3e0 con 0x7eff8c0cf810 2024-04-26T21:14:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.717+0000 7eff8a7fc640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 0x7eff6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.717+0000 7eff8a7fc640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 0x7eff6c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff8c144e70 tx=0x7eff74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:40.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.721+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff8407ad10 con 0x7eff8c0cf810 2024-04-26T21:14:40.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.908+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff8c0ce420 con 0x7eff8c0cf810 2024-04-26T21:14:40.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.909+0000 7eff837fe640 1 -- 172.21.15.40:0/1846385298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+12891 (secure 0 0 0) 0x7eff8407a610 con 0x7eff8c0cf810 2024-04-26T21:14:40.911 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:40.911 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":55,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:40.200242+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 msgr2=0x7eff6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 0x7eff6c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff8c144e70 tx=0x7eff74008040 comp rx=0 tx=0).stop 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 msgr2=0x7eff8c1602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7eff84000c00 tx=0x7eff840046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 shutdown_connections 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff6c06c8f0 0x7eff6c06edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff8c140350 0x7eff8c1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff8c13ff70 0x7eff8c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 --2- 172.21.15.40:0/1846385298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8c0cf810 0x7eff8c1602d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 >> 172.21.15.40:0/1846385298 conn(0x7eff8c0d0d30 msgr2=0x7eff8c140730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 shutdown_connections 2024-04-26T21:14:40.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:40.911+0000 7eff91524640 1 -- 172.21.15.40:0/1846385298 wait complete. 2024-04-26T21:14:40.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:40.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:41.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.026+0000 7f1d238dc640 1 Processor -- start 2024-04-26T21:14:41.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.026+0000 7f1d238dc640 1 -- start start 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d238dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d1c15f260 0x7f1d1c06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d238dc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d1c15f640 con 0x7f1d1c070560 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d238dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d1c06c2a0 con 0x7f1d1c0b4e00 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d238dc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1c06c580 con 0x7f1d1c15f260 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d21678640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d1c15f260 0x7f1d1c06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d20e77640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d1c070560 0x7f1d1c0b46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57578/0 (socket says 172.21.15.40:57578) 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d20e77640 1 -- 172.21.15.40:0/793989334 learned_addr learned my addr 172.21.15.40:0/793989334 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d21678640 1 -- 172.21.15.40:0/793989334 >> v1:172.21.15.184:6789/0 conn(0x7f1d1c0b4e00 legacy=0x7f1d1c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d21678640 1 -- 172.21.15.40:0/793989334 >> v1:172.21.15.40:6789/0 conn(0x7f1d1c070560 legacy=0x7f1d1c0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d21678640 1 -- 172.21.15.40:0/793989334 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c06c860 con 0x7f1d1c15f260 2024-04-26T21:14:41.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.027+0000 7f1d0ffff640 1 -- 172.21.15.40:0/793989334 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132574424 0 0) 0x7f1d1c15f640 con 0x7f1d1c070560 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d21678640 1 --2- 172.21.15.40:0/793989334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c06d620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1d100097a0 tx=0x7f1d1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8e6e7fa3b8d68a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d0ffff640 1 -- 172.21.15.40:0/793989334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10037070 con 0x7f1d1c15f260 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d0ffff640 1 -- 172.21.15.40:0/793989334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d10032440 con 0x7f1d1c15f260 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d0ffff640 1 -- 172.21.15.40:0/793989334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10032810 con 0x7f1d1c15f260 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d238dc640 1 -- 172.21.15.40:0/793989334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 msgr2=0x7f1d1c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d238dc640 1 --2- 172.21.15.40:0/793989334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c06d620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1d100097a0 tx=0x7f1d1002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d238dc640 1 -- 172.21.15.40:0/793989334 shutdown_connections 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d238dc640 1 --2- 172.21.15.40:0/793989334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c06d620 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.028+0000 7f1d238dc640 1 -- 172.21.15.40:0/793989334 >> 172.21.15.40:0/793989334 conn(0x7f1d1c0d1660 msgr2=0x7f1d1c0d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.029+0000 7f1d238dc640 1 -- 172.21.15.40:0/793989334 shutdown_connections 2024-04-26T21:14:41.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.029+0000 7f1d238dc640 1 -- 172.21.15.40:0/793989334 wait complete. 2024-04-26T21:14:41.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 Processor -- start 2024-04-26T21:14:41.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 -- start start 2024-04-26T21:14:41.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c070560 0x7f1d1c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c0b4e00 0x7f1d1c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c06d100 con 0x7f1d1c15f260 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c06ce20 con 0x7f1d1c0b4e00 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.030+0000 7f1d238dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c06db60 con 0x7f1d1c070560 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55674/0 (socket says 172.21.15.40:55674) 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 -- 172.21.15.40:0/6137339 learned_addr learned my addr 172.21.15.40:0/6137339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d20e77640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c0b4e00 0x7f1d1c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21678640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c070560 0x7f1d1c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 -- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c070560 msgr2=0x7f1d1c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c070560 0x7f1d1c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 -- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c0b4e00 msgr2=0x7f1d1c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c0b4e00 0x7f1d1c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 -- 172.21.15.40:0/6137339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c1f9440 con 0x7f1d1c15f260 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d21e79640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14016020 con 0x7f1d1c15f260 2024-04-26T21:14:41.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1400aa50 con 0x7f1d1c15f260 2024-04-26T21:14:41.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1f88c0 con 0x7f1d1c15f260 2024-04-26T21:14:41.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400ce10 con 0x7f1d1c15f260 2024-04-26T21:14:41.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.031+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1f8e80 con 0x7f1d1c15f260 2024-04-26T21:14:41.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.032+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c06a290 con 0x7f1d1c15f260 2024-04-26T21:14:41.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.033+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d14019020 con 0x7f1d1c15f260 2024-04-26T21:14:41.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.033+0000 7f1d0dffb640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 0x7f1d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.033+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f1d1408bad0 con 0x7f1d1c15f260 2024-04-26T21:14:41.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.033+0000 7f1d21678640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 0x7f1d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.034+0000 7f1d21678640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 0x7f1d0006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d10034000 tx=0x7f1d1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.037+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d14020020 con 0x7f1d1c15f260 2024-04-26T21:14:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.221+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d1c0682b0 con 0x7f1d1c15f260 2024-04-26T21:14:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.221+0000 7f1d0dffb640 1 -- 172.21.15.40:0/6137339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7f1d1405a2e0 con 0x7f1d1c15f260 2024-04-26T21:14:41.223 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 msgr2=0x7f1d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 0x7f1d0006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d10034000 tx=0x7f1d1002e000 comp rx=0 tx=0).stop 2024-04-26T21:14:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 msgr2=0x7f1d1c161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 shutdown_connections 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0006c8f0 0x7f1d0006edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d1c15f260 0x7f1d1c161040 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d1c0b4e00 0x7f1d1c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 --2- 172.21.15.40:0/6137339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d1c070560 0x7f1d1c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 >> 172.21.15.40:0/6137339 conn(0x7f1d1c0d1660 msgr2=0x7f1d1c0d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 shutdown_connections 2024-04-26T21:14:41.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.223+0000 7f1d238dc640 1 -- 172.21.15.40:0/6137339 wait complete. 2024-04-26T21:14:41.236 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:14:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.334+0000 7efda663e640 1 Processor -- start 2024-04-26T21:14:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 -- start start 2024-04-26T21:14:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efda015d380 0x7efda00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda00d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda00d0be0 con 0x7efda0160d50 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efda00d0ec0 con 0x7efda015d380 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.335+0000 7efda663e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efda00d11a0 con 0x7efda00b56d0 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efda015d380 0x7efda00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efda4bdb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efda0160d50 0x7efda00b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57590/0 (socket says 172.21.15.40:57590) 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efda4bdb640 1 -- 172.21.15.40:0/2404797004 learned_addr learned my addr 172.21.15.40:0/2404797004 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9f7fe640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda00d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9ffff640 1 -- 172.21.15.40:0/2404797004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 msgr2=0x7efda00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9ffff640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda00d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9ffff640 1 -- 172.21.15.40:0/2404797004 >> v1:172.21.15.40:6789/0 conn(0x7efda0160d50 legacy=0x7efda00b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9ffff640 1 -- 172.21.15.40:0/2404797004 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00d1480 con 0x7efda015d380 2024-04-26T21:14:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.336+0000 7efd9effd640 1 -- 172.21.15.40:0/2404797004 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050464301 0 0) 0x7efda00d0be0 con 0x7efda0160d50 2024-04-26T21:14:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efd9ffff640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00ce590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efd940097a0 tx=0x7efd9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb362d7156ed0488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efd9effd640 1 -- 172.21.15.40:0/2404797004 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94037070 con 0x7efda015d380 2024-04-26T21:14:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efd9effd640 1 -- 172.21.15.40:0/2404797004 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd94032440 con 0x7efda015d380 2024-04-26T21:14:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efd9effd640 1 -- 172.21.15.40:0/2404797004 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94032810 con 0x7efda015d380 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 -- 172.21.15.40:0/2404797004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 msgr2=0x7efda00ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00ce590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efd940097a0 tx=0x7efd9402a180 comp rx=0 tx=0).stop 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 -- 172.21.15.40:0/2404797004 shutdown_connections 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda00d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 --2- 172.21.15.40:0/2404797004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00ce590 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 -- 172.21.15.40:0/2404797004 >> 172.21.15.40:0/2404797004 conn(0x7efda00b4a60 msgr2=0x7efda00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 -- 172.21.15.40:0/2404797004 shutdown_connections 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.337+0000 7efda663e640 1 -- 172.21.15.40:0/2404797004 wait complete. 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.338+0000 7efda663e640 1 Processor -- start 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 -- start start 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda0160d50 0x7efda00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efda00cedc0 con 0x7efda015d380 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efda00d1ee0 con 0x7efda0160d50 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efda663e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efda01611e0 con 0x7efda00b56d0 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55698/0 (socket says 172.21.15.40:55698) 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.339+0000 7efd9f7fe640 1 -- 172.21.15.40:0/3670934458 learned_addr learned my addr 172.21.15.40:0/3670934458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9ffff640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efda4bdb640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda0160d50 0x7efda00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 -- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 msgr2=0x7efda014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda014c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 -- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda0160d50 msgr2=0x7efda00cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda0160d50 0x7efda00cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 -- 172.21.15.40:0/3670934458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda014c8e0 con 0x7efda015d380 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9f7fe640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efd9000d800 tx=0x7efd9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000b660 con 0x7efda015d380 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90019070 con 0x7efda015d380 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000ba20 con 0x7efda015d380 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda00c8680 con 0x7efda015d380 2024-04-26T21:14:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.340+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda00c8c40 con 0x7efda015d380 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.341+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda006a290 con 0x7efda015d380 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.342+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd9000bbc0 con 0x7efda015d380 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.342+0000 7efd9cff9640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 0x7efd8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.342+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7efd9008bc50 con 0x7efda015d380 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.343+0000 7efd9ffff640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 0x7efd8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.343+0000 7efd9ffff640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 0x7efd8c06ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd94034000 tx=0x7efd9402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.347+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd90091020 con 0x7efda015d380 2024-04-26T21:14:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.537+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efda015c870 con 0x7efda015d380 2024-04-26T21:14:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.537+0000 7efd9cff9640 1 -- 172.21.15.40:0/3670934458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7efda014c8e0 con 0x7efda015d380 2024-04-26T21:14:41.539 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:14:41.540 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:14:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 msgr2=0x7efd8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 0x7efd8c06ede0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd94034000 tx=0x7efd9402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 msgr2=0x7efda00cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efd9000d800 tx=0x7efd9000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 shutdown_connections 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd8c06c920 0x7efd8c06ede0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda0160d50 0x7efda00cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efda015d380 0x7efda00cc420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 --2- 172.21.15.40:0/3670934458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efda00b56d0 0x7efda014c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 >> 172.21.15.40:0/3670934458 conn(0x7efda00b4a60 msgr2=0x7efda009ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 shutdown_connections 2024-04-26T21:14:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.539+0000 7efda663e640 1 -- 172.21.15.40:0/3670934458 wait complete. 2024-04-26T21:14:41.552 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:14:41.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:14:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.654+0000 7f588ba85640 1 Processor -- start 2024-04-26T21:14:41.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.654+0000 7f588ba85640 1 -- start start 2024-04-26T21:14:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f588ba85640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58841437f0 0x7f588406f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f588ba85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f58840d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f588ba85640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58841612e0 con 0x7f5884160e70 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f588ba85640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5884073440 con 0x7f58841437f0 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f588ba85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5884161570 con 0x7f58840d1190 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f5889821640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58841437f0 0x7f588406f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f5889821640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58841437f0 0x7f588406f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55714/0 (socket says 172.21.15.40:55714) 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.655+0000 7f5889821640 1 -- 172.21.15.40:0/2479992514 learned_addr learned my addr 172.21.15.40:0/2479992514 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f5889821640 1 -- 172.21.15.40:0/2479992514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 msgr2=0x7f58840d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f5889821640 1 --2- 172.21.15.40:0/2479992514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f58840d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f5889821640 1 -- 172.21.15.40:0/2479992514 >> v1:172.21.15.184:6789/0 conn(0x7f5884160e70 legacy=0x7f5884144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:41.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f5889821640 1 -- 172.21.15.40:0/2479992514 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5884161850 con 0x7f58841437f0 2024-04-26T21:14:41.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f5889821640 1 --2- 172.21.15.40:0/2479992514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588406f600 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446ed30cfe09f58d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f588881f640 1 -- 172.21.15.40:0/2479992514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c036070 con 0x7f58841437f0 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f588881f640 1 -- 172.21.15.40:0/2479992514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587c031600 con 0x7f58841437f0 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.656+0000 7f588881f640 1 -- 172.21.15.40:0/2479992514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0319d0 con 0x7f58841437f0 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 -- 172.21.15.40:0/2479992514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 msgr2=0x7f588406f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 --2- 172.21.15.40:0/2479992514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588406f600 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c02a6c0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 -- 172.21.15.40:0/2479992514 shutdown_connections 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 --2- 172.21.15.40:0/2479992514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f58840d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 --2- 172.21.15.40:0/2479992514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588406f600 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 -- 172.21.15.40:0/2479992514 >> 172.21.15.40:0/2479992514 conn(0x7f588406c690 msgr2=0x7f588406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 -- 172.21.15.40:0/2479992514 shutdown_connections 2024-04-26T21:14:41.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.657+0000 7f588ba85640 1 -- 172.21.15.40:0/2479992514 wait complete. 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.658+0000 7f588ba85640 1 Processor -- start 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.658+0000 7f588ba85640 1 -- start start 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f5884142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5884160e70 0x7f58841e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5884143bd0 con 0x7f58841437f0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58840736a0 con 0x7f5884160e70 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588ba85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5884143d50 con 0x7f58840d1190 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55716/0 (socket says 172.21.15.40:55716) 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 -- 172.21.15.40:0/629219585 learned_addr learned my addr 172.21.15.40:0/629219585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889821640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f5884142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f588a022640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5884160e70 0x7f58841e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 -- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 msgr2=0x7f5884142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f5884142390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 -- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5884160e70 msgr2=0x7f58841e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5884160e70 0x7f58841e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.659+0000 7f5889020640 1 -- 172.21.15.40:0/629219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58841f5ad0 con 0x7f58841437f0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f5889020640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f587800b3e0 tx=0x7f587800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878015020 con 0x7f58841437f0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878011850 con 0x7f58841437f0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58841f5db0 con 0x7f58841437f0 2024-04-26T21:14:41.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878011c40 con 0x7f58841437f0 2024-04-26T21:14:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.660+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58841f6370 con 0x7f58841437f0 2024-04-26T21:14:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.661+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5884073020 con 0x7f58841437f0 2024-04-26T21:14:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.661+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5878018020 con 0x7f58841437f0 2024-04-26T21:14:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.662+0000 7f58827fc640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.662+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f587808b670 con 0x7f58841437f0 2024-04-26T21:14:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.662+0000 7f5889821640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.662+0000 7f5889821640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 0x7f587006edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f587c033000 tx=0x7f587c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:41.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.666+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5884073020 con 0x7f58841437f0 2024-04-26T21:14:41.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:41.850+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f58841f8d70 con 0x7f58841437f0 2024-04-26T21:14:42.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.218+0000 7f58827fc640 1 -- 172.21.15.40:0/629219585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7f587801f020 con 0x7f58841437f0 2024-04-26T21:14:42.220 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 msgr2=0x7f587006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 0x7f587006edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f587c033000 tx=0x7f587c009000 comp rx=0 tx=0).stop 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 msgr2=0x7f588415b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f587800b3e0 tx=0x7f587800b8a0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 shutdown_connections 2024-04-26T21:14:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f587006c8f0 0x7f587006edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5884160e70 0x7f58841e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58841437f0 0x7f588415b500 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 --2- 172.21.15.40:0/629219585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840d1190 0x7f5884142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 >> 172.21.15.40:0/629219585 conn(0x7f588406c690 msgr2=0x7f58840ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 shutdown_connections 2024-04-26T21:14:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.220+0000 7f588ba85640 1 -- 172.21.15.40:0/629219585 wait complete. 2024-04-26T21:14:42.236 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:14:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.343+0000 7f532903f640 1 Processor -- start 2024-04-26T21:14:42.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.343+0000 7f532903f640 1 -- start start 2024-04-26T21:14:42.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532903f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5324162a40 0x7f53240d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532903f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53240d2170 0x7f53240cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532903f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53240d33d0 con 0x7f53240b6050 2024-04-26T21:14:42.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532903f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f53240b49a0 con 0x7f5324162a40 2024-04-26T21:14:42.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532903f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f53240d36b0 con 0x7f53240d2170 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5324162a40 0x7f53240d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5324162a40 0x7f53240d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55724/0 (socket says 172.21.15.40:55724) 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 -- 172.21.15.40:0/995049015 learned_addr learned my addr 172.21.15.40:0/995049015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f532359e640 1 --2- 172.21.15.40:0/995049015 >> v2:172.21.15.184:3300/0 conn(0x7f53240d2170 0x7f53240cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 -- 172.21.15.40:0/995049015 >> v2:172.21.15.184:3300/0 conn(0x7f53240d2170 msgr2=0x7f53240cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 --2- 172.21.15.40:0/995049015 >> v2:172.21.15.184:3300/0 conn(0x7f53240d2170 0x7f53240cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 -- 172.21.15.40:0/995049015 >> v1:172.21.15.184:6789/0 conn(0x7f53240b6050 legacy=0x7f53240d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.344+0000 7f5322d9d640 1 -- 172.21.15.40:0/995049015 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240d3990 con 0x7f5324162a40 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f5322d9d640 1 --2- 172.21.15.40:0/995049015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240d2df0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f53240b6a60 tx=0x7f531402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e0e11db433e076 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532259c640 1 -- 172.21.15.40:0/995049015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314037070 con 0x7f5324162a40 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532259c640 1 -- 172.21.15.40:0/995049015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314032440 con 0x7f5324162a40 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532259c640 1 -- 172.21.15.40:0/995049015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314032810 con 0x7f5324162a40 2024-04-26T21:14:42.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 -- 172.21.15.40:0/995049015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 msgr2=0x7f53240d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 --2- 172.21.15.40:0/995049015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240d2df0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f53240b6a60 tx=0x7f531402a230 comp rx=0 tx=0).stop 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 -- 172.21.15.40:0/995049015 shutdown_connections 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 --2- 172.21.15.40:0/995049015 >> v2:172.21.15.184:3300/0 conn(0x7f53240d2170 0x7f53240cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 --2- 172.21.15.40:0/995049015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240d2df0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.345+0000 7f532903f640 1 -- 172.21.15.40:0/995049015 >> 172.21.15.40:0/995049015 conn(0x7f532409e560 msgr2=0x7f53240d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.346+0000 7f532903f640 1 -- 172.21.15.40:0/995049015 shutdown_connections 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.346+0000 7f532903f640 1 -- 172.21.15.40:0/995049015 wait complete. 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.347+0000 7f532903f640 1 Processor -- start 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.347+0000 7f532903f640 1 -- start start 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b6050 0x7f5324153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240d2170 0x7f5324151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5324162e20 con 0x7f5324162a40 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53240d3f50 con 0x7f53240b6050 2024-04-26T21:14:42.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f53240b6630 con 0x7f53240d2170 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55730/0 (socket says 172.21.15.40:55730) 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 -- 172.21.15.40:0/3771102113 learned_addr learned my addr 172.21.15.40:0/3771102113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f5322d9d640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b6050 0x7f5324153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f531a59c640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240d2170 0x7f5324151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 -- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240d2170 msgr2=0x7f5324151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240d2170 0x7f5324151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 -- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b6050 msgr2=0x7f5324153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b6050 0x7f5324153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:42.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 -- 172.21.15.40:0/3771102113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53241535d0 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532359e640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f531c00cc80 tx=0x7f531c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c00ce30 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c003990 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53240b9270 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.348+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c003da0 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.349+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53240b9830 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.350+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532406a290 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.350+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f531c003f40 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.351+0000 7f531bfff640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 0x7f530406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.351+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7f531c08bdd0 con 0x7f5324162a40 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.351+0000 7f5322d9d640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 0x7f530406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.351+0000 7f5322d9d640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 0x7f530406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:42.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.355+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f531c014030 con 0x7f5324162a40 2024-04-26T21:14:42.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:42.539+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f53240b6af0 con 0x7f5324162a40 2024-04-26T21:14:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.223+0000 7f531bfff640 1 -- 172.21.15.40:0/3771102113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f531c05a3d0 con 0x7f5324162a40 2024-04-26T21:14:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 msgr2=0x7f530406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 0x7f530406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).stop 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 msgr2=0x7f53240cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f531c00cc80 tx=0x7f531c00a5c0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 shutdown_connections 2024-04-26T21:14:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f530406c8f0 0x7f530406edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5324162a40 0x7f53240cd1c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53240d2170 0x7f5324151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 --2- 172.21.15.40:0/3771102113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53240b6050 0x7f5324153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 >> 172.21.15.40:0/3771102113 conn(0x7f532409e560 msgr2=0x7f53240d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.226+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 shutdown_connections 2024-04-26T21:14:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.227+0000 7f532903f640 1 -- 172.21.15.40:0/3771102113 wait complete. 2024-04-26T21:14:43.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:14:43.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.365+0000 7fa3ac6c8640 1 Processor -- start 2024-04-26T21:14:43.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.365+0000 7fa3ac6c8640 1 -- start start 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3ac6c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3a415b930 0x7fa3a415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3ac6c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3a415f4f0 0x7fa3a4073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3ac6c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a415c530 con 0x7fa3a4074030 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3ac6c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a415c810 con 0x7fa3a415b930 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3ac6c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a415caf0 con 0x7fa3a415f4f0 2024-04-26T21:14:43.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3a415b930 0x7fa3a415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3a415b930 0x7fa3a415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55732/0 (socket says 172.21.15.40:55732) 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 -- 172.21.15.40:0/1113968207 learned_addr learned my addr 172.21.15.40:0/1113968207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3a9c63640 1 --2- 172.21.15.40:0/1113968207 >> v2:172.21.15.184:3300/0 conn(0x7fa3a415f4f0 0x7fa3a4073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 -- 172.21.15.40:0/1113968207 >> v2:172.21.15.184:3300/0 conn(0x7fa3a415f4f0 msgr2=0x7fa3a4073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 --2- 172.21.15.40:0/1113968207 >> v2:172.21.15.184:3300/0 conn(0x7fa3a415f4f0 0x7fa3a4073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 -- 172.21.15.40:0/1113968207 >> v1:172.21.15.184:6789/0 conn(0x7fa3a4074030 legacy=0x7fa3a40720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 -- 172.21.15.40:0/1113968207 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a415cdd0 con 0x7fa3a415b930 2024-04-26T21:14:43.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3aa464640 1 --2- 172.21.15.40:0/1113968207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a415b930 0x7fa3a415bd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40b3c10 tx=0x7fa3a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349a26ea9679f88a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:43.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3a9462640 1 -- 172.21.15.40:0/1113968207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0037070 con 0x7fa3a415b930 2024-04-26T21:14:43.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3a9462640 1 -- 172.21.15.40:0/1113968207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a0032440 con 0x7fa3a415b930 2024-04-26T21:14:43.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.367+0000 7fa3a9462640 1 -- 172.21.15.40:0/1113968207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0032810 con 0x7fa3a415b930 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/1113968207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a415b930 msgr2=0x7fa3a415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/1113968207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a415b930 0x7fa3a415bd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40b3c10 tx=0x7fa3a002a270 comp rx=0 tx=0).stop 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/1113968207 shutdown_connections 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/1113968207 >> v2:172.21.15.184:3300/0 conn(0x7fa3a415f4f0 0x7fa3a4073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/1113968207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a415b930 0x7fa3a415bd10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/1113968207 >> 172.21.15.40:0/1113968207 conn(0x7fa3a40cd8a0 msgr2=0x7fa3a40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/1113968207 shutdown_connections 2024-04-26T21:14:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.368+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/1113968207 wait complete. 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.369+0000 7fa3ac6c8640 1 Processor -- start 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.369+0000 7fa3ac6c8640 1 -- start start 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3a415b930 0x7fa3a4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3a415f4f0 0x7fa3a40afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a406fdd0 con 0x7fa3a4074030 2024-04-26T21:14:43.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a415c250 con 0x7fa3a415b930 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a40d0270 con 0x7fa3a415f4f0 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55740/0 (socket says 172.21.15.40:55740) 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 -- 172.21.15.40:0/3551078807 learned_addr learned my addr 172.21.15.40:0/3551078807 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3a9c63640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3a415b930 0x7fa3a4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aac65640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3a415f4f0 0x7fa3a40afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 -- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3a415f4f0 msgr2=0x7fa3a40afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3a415f4f0 0x7fa3a40afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 -- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3a415b930 msgr2=0x7fa3a4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3a415b930 0x7fa3a4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 -- 172.21.15.40:0/3551078807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a41f2fe0 con 0x7fa3a4074030 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3aa464640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0000c00 tx=0x7fa3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0041070 con 0x7fa3a4074030 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a003c870 con 0x7fa3a4074030 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a4155f10 con 0x7fa3a4074030 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0037040 con 0x7fa3a4074030 2024-04-26T21:14:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.370+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41564d0 con 0x7fa3a4074030 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.372+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a40767a0 con 0x7fa3a4074030 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.372+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3a0044070 con 0x7fa3a4074030 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.372+0000 7fa39affd640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 0x7fa38806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.372+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7fa3a00ac470 con 0x7fa3a4074030 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.373+0000 7fa3a9c63640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 0x7fa38806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.373+0000 7fa3a9c63640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 0x7fa38806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3a41f66e0 tx=0x7fa394009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.377+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3a40767a0 con 0x7fa3a4074030 2024-04-26T21:14:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:43.605+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa3a406a290 con 0x7fa3a4074030 2024-04-26T21:14:44.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.226+0000 7fa39affd640 1 -- 172.21.15.40:0/3551078807 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) ==== 172+0+0 (secure 0 0 0) 0x7fa3a00aae20 con 0x7fa3a4074030 2024-04-26T21:14:44.227 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 8 and data pool 9 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.229+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 msgr2=0x7fa38806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.229+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 0x7fa38806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3a41f66e0 tx=0x7fa394009190 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.229+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 msgr2=0x7fa3a41f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.229+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0000c00 tx=0x7fa3a00046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 shutdown_connections 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa38806c8f0 0x7fa38806edb0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3a41f66e0 tx=0x7fa394009190 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3a415f4f0 0x7fa3a40afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3a415b930 0x7fa3a4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 --2- 172.21.15.40:0/3551078807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3a4074030 0x7fa3a41f2aa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 >> 172.21.15.40:0/3551078807 conn(0x7fa3a40cd8a0 msgr2=0x7fa3a40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 shutdown_connections 2024-04-26T21:14:44.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.230+0000 7fa3ac6c8640 1 -- 172.21.15.40:0/3551078807 wait complete. 2024-04-26T21:14:44.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.364+0000 7f960e54e640 1 Processor -- start 2024-04-26T21:14:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.364+0000 7f960e54e640 1 -- start start 2024-04-26T21:14:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.365+0000 7f960e54e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f960815f370 0x7f96080cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.365+0000 7f960e54e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 0x7f96080b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.365+0000 7f960e54e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96080d03a0 con 0x7f960815f750 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.365+0000 7f960e54e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f96080b4d10 con 0x7f960815f370 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.365+0000 7f960e54e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96080d0680 con 0x7f96080d3ee0 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f960caeb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 0x7f96080b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f960815f370 0x7f96080cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f960815f370 0x7f96080cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:47344/0 (socket says 172.21.15.40:47344) 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 -- 172.21.15.40:0/2667962334 learned_addr learned my addr 172.21.15.40:0/2667962334 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f960caeb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 0x7f96080b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:42742/0 (socket says 172.21.15.40:42742) 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 -- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 msgr2=0x7f96080b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 --2- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 0x7f96080b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 -- 172.21.15.40:0/2667962334 >> v1:172.21.15.184:6789/0 conn(0x7f960815f750 legacy=0x7f96080d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.366+0000 7f9607fff640 1 -- 172.21.15.40:0/2667962334 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96080d0960 con 0x7f960815f370 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f9607fff640 1 --2- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960815f370 0x7f96080cfd70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009870 tx=0x7f95f802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3823d02c8e19550 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f9606ffd640 1 -- 172.21.15.40:0/2667962334 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8037070 con 0x7f960815f370 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f9606ffd640 1 -- 172.21.15.40:0/2667962334 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f8032440 con 0x7f960815f370 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f9606ffd640 1 -- 172.21.15.40:0/2667962334 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8032810 con 0x7f960815f370 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 -- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960815f370 msgr2=0x7f96080cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 --2- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960815f370 0x7f96080cfd70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009870 tx=0x7f95f802a270 comp rx=0 tx=0).stop 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 -- 172.21.15.40:0/2667962334 shutdown_connections 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 --2- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96080d3ee0 0x7f96080b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 --2- 172.21.15.40:0/2667962334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f960815f370 0x7f96080cfd70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 -- 172.21.15.40:0/2667962334 >> 172.21.15.40:0/2667962334 conn(0x7f960808ec10 msgr2=0x7f960809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 -- 172.21.15.40:0/2667962334 shutdown_connections 2024-04-26T21:14:44.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.367+0000 7f960e54e640 1 -- 172.21.15.40:0/2667962334 wait complete. 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.369+0000 7f960e54e640 1 Processor -- start 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.369+0000 7f960e54e640 1 -- start start 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.369+0000 7f960e54e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 0x7f9608158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.369+0000 7f960e54e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960815f750 0x7f9608160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96080b51c0 con 0x7f960815f370 2024-04-26T21:14:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96080d0f20 con 0x7f96080d3ee0 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96080b5340 con 0x7f960815f750 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55752/0 (socket says 172.21.15.40:55752) 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 -- 172.21.15.40:0/3119544109 learned_addr learned my addr 172.21.15.40:0/3119544109 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960caeb640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960815f750 0x7f9608160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f9607fff640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 0x7f9608158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 -- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960815f750 msgr2=0x7f9608160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960815f750 0x7f9608160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 -- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 msgr2=0x7f9608158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 0x7f9608158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 -- 172.21.15.40:0/3119544109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96080c8f50 con 0x7f960815f370 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f9607fff640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 0x7f9608158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f96077fe640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c3c0 tx=0x7f95fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0052d0 con 0x7f960815f370 2024-04-26T21:14:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc005490 con 0x7f960815f370 2024-04-26T21:14:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96080c9230 con 0x7f960815f370 2024-04-26T21:14:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc007260 con 0x7f960815f370 2024-04-26T21:14:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.370+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96080c97f0 con 0x7f960815f370 2024-04-26T21:14:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.371+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960806a290 con 0x7f960815f370 2024-04-26T21:14:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.372+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95fc007400 con 0x7f960815f370 2024-04-26T21:14:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.372+0000 7f9604ff9640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.372+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f95fc08aeb0 con 0x7f960815f370 2024-04-26T21:14:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.373+0000 7f9607fff640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.373+0000 7f9607fff640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 0x7f95f406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.377+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95fc058d20 con 0x7f960815f370 2024-04-26T21:14:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.561+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96080c99c0 con 0x7f960815f370 2024-04-26T21:14:44.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.562+0000 7f9604ff9640 1 -- 172.21.15.40:0/3119544109 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7f95fc05f020 con 0x7f960815f370 2024-04-26T21:14:44.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:44.564 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":58,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:43.607325+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 msgr2=0x7f95f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 0x7f95f406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 msgr2=0x7f960815fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c3c0 tx=0x7f95fc00c880 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 shutdown_connections 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f960815f750 0x7f9608160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f960815f370 0x7f960815fc40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 --2- 172.21.15.40:0/3119544109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96080d3ee0 0x7f9608158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.564+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 >> 172.21.15.40:0/3119544109 conn(0x7f960808ec10 msgr2=0x7f960806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.565+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 shutdown_connections 2024-04-26T21:14:44.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.565+0000 7f960e54e640 1 -- 172.21.15.40:0/3119544109 wait complete. 2024-04-26T21:14:44.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:44.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.676+0000 7fe167ec8640 1 Processor -- start 2024-04-26T21:14:44.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.676+0000 7fe167ec8640 1 -- start start 2024-04-26T21:14:44.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe167ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 0x7fe16006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe167ec8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1600d1740 con 0x7fe16006e490 2024-04-26T21:14:44.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe167ec8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1600d1a20 con 0x7fe1600d11b0 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe167ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1600d1d00 con 0x7fe1600cf2e0 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe166465640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe16006e490 0x7fe16015ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57600/0 (socket says 172.21.15.40:57600) 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe166465640 1 -- 172.21.15.40:0/2593300316 learned_addr learned my addr 172.21.15.40:0/2593300316 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe165c64640 1 --2- 172.21.15.40:0/2593300316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 0x7fe16006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe164c62640 1 -- 172.21.15.40:0/2593300316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653076340 0 0) 0x7fe1600d1740 con 0x7fe16006e490 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.677+0000 7fe164c62640 1 -- 172.21.15.40:0/2593300316 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe14c003520 con 0x7fe16006e490 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.678+0000 7fe165c64640 1 -- 172.21.15.40:0/2593300316 >> v1:172.21.15.184:6789/0 conn(0x7fe1600d11b0 legacy=0x7fe16006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.678+0000 7fe165c64640 1 -- 172.21.15.40:0/2593300316 >> v1:172.21.15.40:6789/0 conn(0x7fe16006e490 legacy=0x7fe16015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.678+0000 7fe165c64640 1 -- 172.21.15.40:0/2593300316 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1600d1fe0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.678+0000 7fe165c64640 1 --2- 172.21.15.40:0/2593300316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 0x7fe16006d4d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097d0 tx=0x7fe15c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53f704c5ad062b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe164c62640 1 -- 172.21.15.40:0/2593300316 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c037070 con 0x7fe1600cf2e0 2024-04-26T21:14:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe164c62640 1 -- 172.21.15.40:0/2593300316 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15c032440 con 0x7fe1600cf2e0 2024-04-26T21:14:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe164c62640 1 -- 172.21.15.40:0/2593300316 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c032810 con 0x7fe1600cf2e0 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 -- 172.21.15.40:0/2593300316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 msgr2=0x7fe16006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 --2- 172.21.15.40:0/2593300316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 0x7fe16006d4d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097d0 tx=0x7fe15c02a210 comp rx=0 tx=0).stop 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 -- 172.21.15.40:0/2593300316 shutdown_connections 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 --2- 172.21.15.40:0/2593300316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600cf2e0 0x7fe16006d4d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 -- 172.21.15.40:0/2593300316 >> 172.21.15.40:0/2593300316 conn(0x7fe1600b4a60 msgr2=0x7fe1600b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 -- 172.21.15.40:0/2593300316 shutdown_connections 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.679+0000 7fe167ec8640 1 -- 172.21.15.40:0/2593300316 wait complete. 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.680+0000 7fe167ec8640 1 Processor -- start 2024-04-26T21:14:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.680+0000 7fe167ec8640 1 -- start start 2024-04-26T21:14:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe16006e490 0x7fe160150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 0x7fe1601e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1600cf6c0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe16006da10 con 0x7fe1600d11b0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe167ec8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1600cf840 con 0x7fe16006e490 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe165463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe165463640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55754/0 (socket says 172.21.15.40:55754) 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe165463640 1 -- 172.21.15.40:0/3334710358 learned_addr learned my addr 172.21.15.40:0/3334710358 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe165c64640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe16006e490 0x7fe160150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.681+0000 7fe166465640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 0x7fe1601e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 -- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe16006e490 msgr2=0x7fe160150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe16006e490 0x7fe160150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 -- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 msgr2=0x7fe1601e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 0x7fe1601e9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 -- 172.21.15.40:0/3334710358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601519a0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe166465640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 0x7fe1601e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe165463640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c3c0 tx=0x7fe15000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500052d0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe150005490 con 0x7fe1600cf2e0 2024-04-26T21:14:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601ea910 con 0x7fe1600cf2e0 2024-04-26T21:14:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150007260 con 0x7fe1600cf2e0 2024-04-26T21:14:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.682+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1600c6cf0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.683+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1601f88c0 con 0x7fe1600cf2e0 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.683+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe150007400 con 0x7fe1600cf2e0 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.684+0000 7fe1567fc640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 0x7fe14406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.684+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe15008b040 con 0x7fe1600cf2e0 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.684+0000 7fe165c64640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 0x7fe14406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.685+0000 7fe165c64640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 0x7fe14406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe15c034000 tx=0x7fe15c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.688+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe150059480 con 0x7fe1600cf2e0 2024-04-26T21:14:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.875+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe16006a290 con 0x7fe1600cf2e0 2024-04-26T21:14:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.876+0000 7fe1567fc640 1 -- 172.21.15.40:0/3334710358 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7fe150058d80 con 0x7fe1600cf2e0 2024-04-26T21:14:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:44.877 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":58,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:43.607325+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 msgr2=0x7fe14406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 0x7fe14406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe15c034000 tx=0x7fe15c02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 msgr2=0x7fe1601f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c3c0 tx=0x7fe15000c880 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 shutdown_connections 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe14406c8f0 0x7fe14406edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1600d11b0 0x7fe1601e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600cf2e0 0x7fe1601f54b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 --2- 172.21.15.40:0/3334710358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe16006e490 0x7fe160150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 >> 172.21.15.40:0/3334710358 conn(0x7fe1600b4a60 msgr2=0x7fe1600b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 shutdown_connections 2024-04-26T21:14:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.878+0000 7fe167ec8640 1 -- 172.21.15.40:0/3334710358 wait complete. 2024-04-26T21:14:44.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:14:44.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.993+0000 7fa2fbe45640 1 Processor -- start 2024-04-26T21:14:44.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.993+0000 7fa2fbe45640 1 -- start start 2024-04-26T21:14:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fbe45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f415f490 0x7fa2f40d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fbe45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2f40d22f0 0x7fa2f40ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fbe45640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f40d4240 con 0x7fa2f40d0ff0 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fbe45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f40d31a0 con 0x7fa2f40d22f0 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fbe45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f40d3480 con 0x7fa2f415f490 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fa3e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2f40d22f0 0x7fa2f40ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2f93e0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2f40d0ff0 0x7fa2f40d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57604/0 (socket says 172.21.15.40:57604) 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2f93e0640 1 -- 172.21.15.40:0/735134445 learned_addr learned my addr 172.21.15.40:0/735134445 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2f9be1640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f415f490 0x7fa2f40d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/735134445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f415f490 msgr2=0x7fa2f40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fa3e2640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f415f490 0x7fa2f40d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/735134445 >> v1:172.21.15.40:6789/0 conn(0x7fa2f40d0ff0 legacy=0x7fa2f40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/735134445 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f40d3760 con 0x7fa2f40d22f0 2024-04-26T21:14:44.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.994+0000 7fa2f8bdf640 1 -- 172.21.15.40:0/735134445 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646149167 0 0) 0x7fa2f40d4240 con 0x7fa2f40d0ff0 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.995+0000 7fa2fa3e2640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f40d22f0 0x7fa2f40ba3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009d90 tx=0x7fa2ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88f058b09573ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.995+0000 7fa2f8bdf640 1 -- 172.21.15.40:0/735134445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec007740 con 0x7fa2f40d22f0 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.995+0000 7fa2f8bdf640 1 -- 172.21.15.40:0/735134445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec0078e0 con 0x7fa2f40d22f0 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.995+0000 7fa2f8bdf640 1 -- 172.21.15.40:0/735134445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec007cb0 con 0x7fa2f40d22f0 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 -- 172.21.15.40:0/735134445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f40d22f0 msgr2=0x7fa2f40ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f40d22f0 0x7fa2f40ba3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009d90 tx=0x7fa2ec02a660 comp rx=0 tx=0).stop 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 -- 172.21.15.40:0/735134445 shutdown_connections 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f40d22f0 0x7fa2f40ba3a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/735134445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f415f490 0x7fa2f40d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 -- 172.21.15.40:0/735134445 >> 172.21.15.40:0/735134445 conn(0x7fa2f409e530 msgr2=0x7fa2f40d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 -- 172.21.15.40:0/735134445 shutdown_connections 2024-04-26T21:14:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.996+0000 7fa2fbe45640 1 -- 172.21.15.40:0/735134445 wait complete. 2024-04-26T21:14:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.997+0000 7fa2fbe45640 1 Processor -- start 2024-04-26T21:14:44.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.997+0000 7fa2fbe45640 1 -- start start 2024-04-26T21:14:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 0x7fa2f415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2f40d22f0 0x7fa2f41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f415f870 con 0x7fa2f415f490 2024-04-26T21:14:45.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f40d3d20 con 0x7fa2f40d22f0 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fbe45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f40d2dc0 con 0x7fa2f40d0ff0 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55786/0 (socket says 172.21.15.40:55786) 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/1368505436 learned_addr learned my addr 172.21.15.40:0/1368505436 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2f9be1640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 0x7fa2f415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 msgr2=0x7fa2f415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 0x7fa2f415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2f40d22f0 msgr2=0x7fa2f41581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2f40d22f0 0x7fa2f41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 -- 172.21.15.40:0/1368505436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f40b9690 con 0x7fa2f415f490 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2f9be1640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 0x7fa2f415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.998+0000 7fa2fa3e2640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009d90 tx=0x7fa2ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:45.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.999+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec03f070 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.999+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec031bc0 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.999+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f40b9970 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.999+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec043410 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:44.999+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f40b9f30 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.000+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f406a290 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.000+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2ec007790 con 0x7fa2f415f490 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.001+0000 7fa2ea7fc640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.001+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa2ec0ad050 con 0x7fa2f415f490 2024-04-26T21:14:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.001+0000 7fa2f9be1640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.001+0000 7fa2f9be1640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 0x7fa2d806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0006fd0 tx=0x7fa2f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:45.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.005+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2ec07b480 con 0x7fa2f415f490 2024-04-26T21:14:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.228+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa2f4158700 con 0x7fa2f415f490 2024-04-26T21:14:45.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.239+0000 7fa2ea7fc640 1 -- 172.21.15.40:0/1368505436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fa2ec07ad80 con 0x7fa2f415f490 2024-04-26T21:14:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 msgr2=0x7fa2d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 0x7fa2d806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0006fd0 tx=0x7fa2f0008040 comp rx=0 tx=0).stop 2024-04-26T21:14:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 msgr2=0x7fa2f40b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009d90 tx=0x7fa2ec004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:45.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 shutdown_connections 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2f415f490 0x7fa2f40b8e60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2f40d22f0 0x7fa2f41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 --2- 172.21.15.40:0/1368505436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2f40d0ff0 0x7fa2f415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.241+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 >> 172.21.15.40:0/1368505436 conn(0x7fa2f409e530 msgr2=0x7fa2f4162470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.242+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 shutdown_connections 2024-04-26T21:14:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.242+0000 7fa2fbe45640 1 -- 172.21.15.40:0/1368505436 wait complete. 2024-04-26T21:14:45.253 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:14:45.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:14:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.356+0000 7f6009252640 1 Processor -- start 2024-04-26T21:14:45.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.356+0000 7f6009252640 1 -- start start 2024-04-26T21:14:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6009252640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f60041631f0 0x7f60040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6009252640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60040d1170 0x7f60041635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6009252640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600409c790 con 0x7f60040b3300 2024-04-26T21:14:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6009252640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f60040d15f0 con 0x7f60040d1170 2024-04-26T21:14:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6009252640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6004163ec0 con 0x7f60041631f0 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f600359e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f60040b3300 0x7f60040b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57618/0 (socket says 172.21.15.40:57618) 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f600359e640 1 -- 172.21.15.40:0/3604220506 learned_addr learned my addr 172.21.15.40:0/3604220506 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f600259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60040d1170 0x7f60041635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6002d9d640 1 --2- 172.21.15.40:0/3604220506 >> v2:172.21.15.184:3300/0 conn(0x7f60041631f0 0x7f60040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6001d9b640 1 -- 172.21.15.40:0/3604220506 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522131141 0 0) 0x7f600409c790 con 0x7f60040b3300 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f6001d9b640 1 -- 172.21.15.40:0/3604220506 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff0003520 con 0x7f60040b3300 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f600259c640 1 -- 172.21.15.40:0/3604220506 >> v2:172.21.15.184:3300/0 conn(0x7f60041631f0 msgr2=0x7f60040b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.357+0000 7f600259c640 1 --2- 172.21.15.40:0/3604220506 >> v2:172.21.15.184:3300/0 conn(0x7f60041631f0 0x7f60040b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f600259c640 1 -- 172.21.15.40:0/3604220506 >> v1:172.21.15.40:6789/0 conn(0x7f60040b3300 legacy=0x7f60040b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f600259c640 1 -- 172.21.15.40:0/3604220506 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60040d1a40 con 0x7f60040d1170 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f600259c640 1 --2- 172.21.15.40:0/3604220506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60040d1170 0x7f60041635d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f60040d2ef0 tx=0x7f5fec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1b4b208357c283 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f6001d9b640 1 -- 172.21.15.40:0/3604220506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x7f60040d1170 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f6001d9b640 1 -- 172.21.15.40:0/3604220506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec032440 con 0x7f60040d1170 2024-04-26T21:14:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.358+0000 7f6001d9b640 1 -- 172.21.15.40:0/3604220506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec02ae60 con 0x7f60040d1170 2024-04-26T21:14:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 -- 172.21.15.40:0/3604220506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60040d1170 msgr2=0x7f60041635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 --2- 172.21.15.40:0/3604220506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60040d1170 0x7f60041635d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f60040d2ef0 tx=0x7f5fec02a110 comp rx=0 tx=0).stop 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 -- 172.21.15.40:0/3604220506 shutdown_connections 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 --2- 172.21.15.40:0/3604220506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60040d1170 0x7f60041635d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 --2- 172.21.15.40:0/3604220506 >> v2:172.21.15.184:3300/0 conn(0x7f60041631f0 0x7f60040b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 -- 172.21.15.40:0/3604220506 >> 172.21.15.40:0/3604220506 conn(0x7f600409ba40 msgr2=0x7f600409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 -- 172.21.15.40:0/3604220506 shutdown_connections 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.359+0000 7f6009252640 1 -- 172.21.15.40:0/3604220506 wait complete. 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.360+0000 7f6009252640 1 Processor -- start 2024-04-26T21:14:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 -- start start 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 0x7f6004158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60040d1170 0x7f6004158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60040d0230 con 0x7f60041631f0 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60040d2000 con 0x7f60040d1170 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6009252640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60041f9a80 con 0x7f60040b3300 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f600359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f600359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55808/0 (socket says 172.21.15.40:55808) 2024-04-26T21:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f600359e640 1 -- 172.21.15.40:0/3660250879 learned_addr learned my addr 172.21.15.40:0/3660250879 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f6002d9d640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 0x7f6004158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.361+0000 7f600259c640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60040d1170 0x7f6004158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 -- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 msgr2=0x7f6004158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 0x7f6004158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 -- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60040d1170 msgr2=0x7f6004158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60040d1170 0x7f6004158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 -- 172.21.15.40:0/3660250879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60041f8f00 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f6002d9d640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 0x7f6004158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f600359e640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400cb70 tx=0x7f5ff400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4017070 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff400ad40 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4012620 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60041f91e0 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.362+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60041f97a0 con 0x7f60041631f0 2024-04-26T21:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.363+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600406a290 con 0x7f60041631f0 2024-04-26T21:14:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.364+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ff401a070 con 0x7f60041631f0 2024-04-26T21:14:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.364+0000 7f5ffb7fe640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 0x7f5fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.364+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5ff408b910 con 0x7f60041631f0 2024-04-26T21:14:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.364+0000 7f6002d9d640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 0x7f5fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.365+0000 7f6002d9d640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 0x7f5fe406edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00b9f0 tx=0x7f5ffc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.368+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ff40596f0 con 0x7f60041631f0 2024-04-26T21:14:45.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:45.594+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f60040b06f0 con 0x7f60041631f0 2024-04-26T21:14:45.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-26T21:14:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.247+0000 7f5ffb7fe640 1 -- 172.21.15.40:0/3660250879 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f5ff405d9f0 con 0x7f60041631f0 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 msgr2=0x7f5fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 0x7f5fe406edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00b9f0 tx=0x7f5ffc008040 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 msgr2=0x7f60041592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400cb70 tx=0x7f5ff400a430 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 shutdown_connections 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fe406c8f0 0x7f5fe406edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60041631f0 0x7f60041592c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60040d1170 0x7f6004158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.249+0000 7f6009252640 1 --2- 172.21.15.40:0/3660250879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60040b3300 0x7f6004158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.250+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 >> 172.21.15.40:0/3660250879 conn(0x7f600409ba40 msgr2=0x7f600409be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.250+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 shutdown_connections 2024-04-26T21:14:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.250+0000 7f6009252640 1 -- 172.21.15.40:0/3660250879 wait complete. 2024-04-26T21:14:46.264 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:14:46.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:46.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.371+0000 7f950979d640 1 Processor -- start 2024-04-26T21:14:46.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.371+0000 7f950979d640 1 -- start start 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f950979d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f95040ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f950979d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 0x7f95041611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f950979d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95040d0be0 con 0x7f95040b56d0 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f950979d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f95040d0ec0 con 0x7f9504160d50 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f950979d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95040d11a0 con 0x7f950415d380 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f95027fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95040b56d0 0x7f95040d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57630/0 (socket says 172.21.15.40:57630) 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f95027fc640 1 -- 172.21.15.40:0/610088787 learned_addr learned my addr 172.21.15.40:0/610088787 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f9502ffd640 1 --2- 172.21.15.40:0/610088787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f95040ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.372+0000 7f95037fe640 1 --2- 172.21.15.40:0/610088787 >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 0x7f95041611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9501ffb640 1 -- 172.21.15.40:0/610088787 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682785689 0 0) 0x7f95040d0be0 con 0x7f95040b56d0 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9501ffb640 1 -- 172.21.15.40:0/610088787 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f0003520 con 0x7f95040b56d0 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9502ffd640 1 -- 172.21.15.40:0/610088787 >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 msgr2=0x7f95041611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9502ffd640 1 --2- 172.21.15.40:0/610088787 >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 0x7f95041611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9502ffd640 1 -- 172.21.15.40:0/610088787 >> v1:172.21.15.40:6789/0 conn(0x7f95040b56d0 legacy=0x7f95040d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f9502ffd640 1 -- 172.21.15.40:0/610088787 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95040d1480 con 0x7f950415d380 2024-04-26T21:14:46.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.373+0000 7f95037fe640 1 --2- 172.21.15.40:0/610088787 >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 0x7f95041611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f9502ffd640 1 --2- 172.21.15.40:0/610088787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f95040ce590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0097a0 tx=0x7f94fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1d45b579226888 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f9501ffb640 1 -- 172.21.15.40:0/610088787 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x7f950415d380 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f9501ffb640 1 -- 172.21.15.40:0/610088787 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc02ae50 con 0x7f950415d380 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f9501ffb640 1 -- 172.21.15.40:0/610088787 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc032640 con 0x7f950415d380 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 -- 172.21.15.40:0/610088787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 msgr2=0x7f95040ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 --2- 172.21.15.40:0/610088787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f95040ce590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0097a0 tx=0x7f94fc02a100 comp rx=0 tx=0).stop 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 -- 172.21.15.40:0/610088787 shutdown_connections 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 --2- 172.21.15.40:0/610088787 >> v2:172.21.15.184:3300/0 conn(0x7f9504160d50 0x7f95041611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 --2- 172.21.15.40:0/610088787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f95040ce590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.374+0000 7f950979d640 1 -- 172.21.15.40:0/610088787 >> 172.21.15.40:0/610088787 conn(0x7f95040b4a60 msgr2=0x7f95040b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.375+0000 7f950979d640 1 -- 172.21.15.40:0/610088787 shutdown_connections 2024-04-26T21:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.375+0000 7f950979d640 1 -- 172.21.15.40:0/610088787 wait complete. 2024-04-26T21:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.376+0000 7f950979d640 1 Processor -- start 2024-04-26T21:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.376+0000 7f950979d640 1 -- start start 2024-04-26T21:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040b56d0 0x7f95041f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f950415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95040b5dc0 con 0x7f9504160d50 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95040cedc0 con 0x7f95040b56d0 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95040b5f40 con 0x7f950415d380 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55822/0 (socket says 172.21.15.40:55822) 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 -- 172.21.15.40:0/1531384937 learned_addr learned my addr 172.21.15.40:0/1531384937 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95027fc640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f950415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f9502ffd640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040b56d0 0x7f95041f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 -- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 msgr2=0x7f950415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f950415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 -- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040b56d0 msgr2=0x7f95041f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040b56d0 0x7f95041f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 -- 172.21.15.40:0/1531384937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950415e020 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95027fc640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f950415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f95037fe640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e60 tx=0x7f94f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4018030 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f400cdf0 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950415e300 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.377+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4003bb0 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.378+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950415e8c0 con 0x7f9504160d50 2024-04-26T21:14:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.378+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950406c730 con 0x7f9504160d50 2024-04-26T21:14:46.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.379+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94f4005d10 con 0x7f9504160d50 2024-04-26T21:14:46.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.379+0000 7f94fb7fe640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 0x7f94e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.380+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f94f408c430 con 0x7f9504160d50 2024-04-26T21:14:46.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.380+0000 7f9502ffd640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 0x7f94e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.380+0000 7f9502ffd640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 0x7f94e406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f94fc034000 tx=0x7f94fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.384+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f950406c730 con 0x7f9504160d50 2024-04-26T21:14:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.608+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f950406a290 con 0x7f9504160d50 2024-04-26T21:14:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.609+0000 7f94fb7fe640 1 -- 172.21.15.40:0/1531384937 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7f94f4022070 con 0x7f9504160d50 2024-04-26T21:14:46.610 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:46.610 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:14:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:14:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 msgr2=0x7f94e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 0x7f94e406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f94fc034000 tx=0x7f94fc02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 msgr2=0x7f95040cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e60 tx=0x7f94f400c490 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 shutdown_connections 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94e406c8f0 0x7f94e406edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9504160d50 0x7f95040cc210 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950415d380 0x7f950415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 --2- 172.21.15.40:0/1531384937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040b56d0 0x7f95041f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 >> 172.21.15.40:0/1531384937 conn(0x7f95040b4a60 msgr2=0x7f95040d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 shutdown_connections 2024-04-26T21:14:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.611+0000 7f950979d640 1 -- 172.21.15.40:0/1531384937 wait complete. 2024-04-26T21:14:46.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.723+0000 7fa85ab89640 1 Processor -- start 2024-04-26T21:14:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.723+0000 7fa85ab89640 1 -- start start 2024-04-26T21:14:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa85ab89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa85406e490 0x7fa85415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa85ab89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540d1740 con 0x7fa8540d11b0 2024-04-26T21:14:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa85ab89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540d1a20 con 0x7fa8540cf2e0 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa85ab89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540d1d00 con 0x7fa85406e490 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa853fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8540d11b0 0x7fa85406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57642/0 (socket says 172.21.15.40:57642) 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa853fff640 1 -- 172.21.15.40:0/3251650177 learned_addr learned my addr 172.21.15.40:0/3251650177 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa859126640 1 --2- 172.21.15.40:0/3251650177 >> v2:172.21.15.184:3300/0 conn(0x7fa85406e490 0x7fa85415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa8537fe640 1 -- 172.21.15.40:0/3251650177 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979511745 0 0) 0x7fa8540d1740 con 0x7fa8540d11b0 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa8537fe640 1 -- 172.21.15.40:0/3251650177 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844003520 con 0x7fa8540d11b0 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa859126640 1 -- 172.21.15.40:0/3251650177 >> v1:172.21.15.184:6789/0 conn(0x7fa8540cf2e0 legacy=0x7fa85406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa859126640 1 -- 172.21.15.40:0/3251650177 >> v1:172.21.15.40:6789/0 conn(0x7fa8540d11b0 legacy=0x7fa85406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.724+0000 7fa859126640 1 -- 172.21.15.40:0/3251650177 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540d1fe0 con 0x7fa85406e490 2024-04-26T21:14:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.725+0000 7fa859126640 1 --2- 172.21.15.40:0/3251650177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa85415ebe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009dc0 tx=0x7fa84c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b452272529ca05d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.725+0000 7fa8537fe640 1 -- 172.21.15.40:0/3251650177 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c007740 con 0x7fa85406e490 2024-04-26T21:14:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.725+0000 7fa8537fe640 1 -- 172.21.15.40:0/3251650177 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c0078e0 con 0x7fa85406e490 2024-04-26T21:14:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.725+0000 7fa8537fe640 1 -- 172.21.15.40:0/3251650177 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c007cb0 con 0x7fa85406e490 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 -- 172.21.15.40:0/3251650177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 msgr2=0x7fa85415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 --2- 172.21.15.40:0/3251650177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa85415ebe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009dc0 tx=0x7fa84c02a730 comp rx=0 tx=0).stop 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 -- 172.21.15.40:0/3251650177 shutdown_connections 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 --2- 172.21.15.40:0/3251650177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa85415ebe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 -- 172.21.15.40:0/3251650177 >> 172.21.15.40:0/3251650177 conn(0x7fa8540b4a60 msgr2=0x7fa8540b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 -- 172.21.15.40:0/3251650177 shutdown_connections 2024-04-26T21:14:46.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.726+0000 7fa85ab89640 1 -- 172.21.15.40:0/3251650177 wait complete. 2024-04-26T21:14:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.727+0000 7fa85ab89640 1 Processor -- start 2024-04-26T21:14:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.727+0000 7fa85ab89640 1 -- start start 2024-04-26T21:14:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa8540b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8540d11b0 0x7fa854163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540cf840 con 0x7fa8540cf2e0 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540cf6c0 con 0x7fa85406e490 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa85ab89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8540ce660 con 0x7fa8540d11b0 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55828/0 (socket says 172.21.15.40:55828) 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 -- 172.21.15.40:0/2842284779 learned_addr learned my addr 172.21.15.40:0/2842284779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa859126640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8540d11b0 0x7fa854163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa858925640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa8540b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 -- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8540d11b0 msgr2=0x7fa854163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8540d11b0 0x7fa854163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 -- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 msgr2=0x7fa8540b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa8540b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.728+0000 7fa853fff640 1 -- 172.21.15.40:0/2842284779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85415d6b0 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa853fff640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c410 tx=0x7fa84000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840017070 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840005ce0 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85415ce10 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400072f0 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.729+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85415d3d0 con 0x7fa8540cf2e0 2024-04-26T21:14:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.730+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85406a290 con 0x7fa8540cf2e0 2024-04-26T21:14:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.730+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa84001a070 con 0x7fa8540cf2e0 2024-04-26T21:14:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.731+0000 7fa8517fa640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 0x7fa83806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.731+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa84008af70 con 0x7fa8540cf2e0 2024-04-26T21:14:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.731+0000 7fa858925640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 0x7fa83806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.731+0000 7fa858925640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 0x7fa83806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8480015a0 tx=0x7fa848009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.735+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa840059450 con 0x7fa8540cf2e0 2024-04-26T21:14:46.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.919+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa854163b40 con 0x7fa8540cf2e0 2024-04-26T21:14:46.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.920+0000 7fa8517fa640 1 -- 172.21.15.40:0/2842284779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7fa840058d50 con 0x7fa8540cf2e0 2024-04-26T21:14:46.922 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:46.922 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.922+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 msgr2=0x7fa83806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.922+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 0x7fa83806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8480015a0 tx=0x7fa848009040 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.922+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 msgr2=0x7fa854162ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.922+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c410 tx=0x7fa84000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 shutdown_connections 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa83806c8f0 0x7fa83806edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8540d11b0 0x7fa854163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8540cf2e0 0x7fa854162ee0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 --2- 172.21.15.40:0/2842284779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa85406e490 0x7fa8540b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 >> 172.21.15.40:0/2842284779 conn(0x7fa8540b4a60 msgr2=0x7fa8540b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 shutdown_connections 2024-04-26T21:14:46.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:46.923+0000 7fa85ab89640 1 -- 172.21.15.40:0/2842284779 wait complete. 2024-04-26T21:14:46.935 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:14:46.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.035+0000 7f49182da640 1 Processor -- start 2024-04-26T21:14:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.035+0000 7f49182da640 1 -- start start 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f49182da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49100cf2e0 0x7f491006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f49182da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f491006e490 0x7f491015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f49182da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49100d1740 con 0x7f49100d11b0 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f49182da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f49100d1a20 con 0x7f49100cf2e0 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f49182da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f49100d1d00 con 0x7f491006e490 2024-04-26T21:14:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4916076640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49100cf2e0 0x7f491006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4915875640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49100d11b0 0x7f491006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57644/0 (socket says 172.21.15.40:57644) 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4915875640 1 -- 172.21.15.40:0/2863349742 learned_addr learned my addr 172.21.15.40:0/2863349742 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4916877640 1 --2- 172.21.15.40:0/2863349742 >> v2:172.21.15.184:3300/0 conn(0x7f491006e490 0x7f491015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4916076640 1 -- 172.21.15.40:0/2863349742 >> v2:172.21.15.184:3300/0 conn(0x7f491006e490 msgr2=0x7f491015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4916076640 1 --2- 172.21.15.40:0/2863349742 >> v2:172.21.15.184:3300/0 conn(0x7f491006e490 0x7f491015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4916076640 1 -- 172.21.15.40:0/2863349742 >> v1:172.21.15.40:6789/0 conn(0x7f49100d11b0 legacy=0x7f491006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.037+0000 7f4916076640 1 -- 172.21.15.40:0/2863349742 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100d1fe0 con 0x7f49100cf2e0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.036+0000 7f4915074640 1 -- 172.21.15.40:0/2863349742 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334869371 0 0) 0x7f49100d1740 con 0x7f49100d11b0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.037+0000 7f4916076640 1 --2- 172.21.15.40:0/2863349742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491006d4d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f490802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8a8b5a655e663f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.037+0000 7f4915074640 1 -- 172.21.15.40:0/2863349742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908037070 con 0x7f49100cf2e0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.037+0000 7f4915074640 1 -- 172.21.15.40:0/2863349742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4908032440 con 0x7f49100cf2e0 2024-04-26T21:14:47.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.037+0000 7f4915074640 1 -- 172.21.15.40:0/2863349742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908032810 con 0x7f49100cf2e0 2024-04-26T21:14:47.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 -- 172.21.15.40:0/2863349742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 msgr2=0x7f491006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 --2- 172.21.15.40:0/2863349742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491006d4d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f490802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 -- 172.21.15.40:0/2863349742 shutdown_connections 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 --2- 172.21.15.40:0/2863349742 >> v2:172.21.15.184:3300/0 conn(0x7f491006e490 0x7f491015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 --2- 172.21.15.40:0/2863349742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491006d4d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 -- 172.21.15.40:0/2863349742 >> 172.21.15.40:0/2863349742 conn(0x7f49100b4a60 msgr2=0x7f49100b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 -- 172.21.15.40:0/2863349742 shutdown_connections 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.038+0000 7f49182da640 1 -- 172.21.15.40:0/2863349742 wait complete. 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.039+0000 7f49182da640 1 Processor -- start 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.039+0000 7f49182da640 1 -- start start 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 0x7f49100a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49100d11b0 0x7f4910152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49100cf6c0 con 0x7f49100cf2e0 2024-04-26T21:14:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f491006da10 con 0x7f49100d11b0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f49182da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49100cf840 con 0x7f491006e490 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55848/0 (socket says 172.21.15.40:55848) 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 -- 172.21.15.40:0/1556180496 learned_addr learned my addr 172.21.15.40:0/1556180496 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4916076640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 0x7f49100a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4916877640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49100d11b0 0x7f4910152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 -- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 msgr2=0x7f49100a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 0x7f49100a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 -- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49100d11b0 msgr2=0x7f4910152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49100d11b0 0x7f4910152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.040+0000 7f4915875640 1 -- 172.21.15.40:0/1556180496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910153070 con 0x7f49100cf2e0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f4916076640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 0x7f49100a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f4915875640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f490400c460 tx=0x7f490400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49040074e0 con 0x7f49100cf2e0 2024-04-26T21:14:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49040047e0 con 0x7f49100cf2e0 2024-04-26T21:14:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49100afdc0 con 0x7f49100cf2e0 2024-04-26T21:14:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004bd0 con 0x7f49100cf2e0 2024-04-26T21:14:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.041+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491009e1b0 con 0x7f49100cf2e0 2024-04-26T21:14:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.042+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491009dfd0 con 0x7f49100cf2e0 2024-04-26T21:14:47.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.042+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4904011070 con 0x7f49100cf2e0 2024-04-26T21:14:47.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.043+0000 7f490effd640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 0x7f48fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.043+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f490408b710 con 0x7f49100cf2e0 2024-04-26T21:14:47.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.043+0000 7f4916076640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 0x7f48fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.044+0000 7f4916076640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 0x7f48fc06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.047+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f491009dfd0 con 0x7f49100cf2e0 2024-04-26T21:14:47.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.228+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49100b8ef0 con 0x7f48fc06c8f0 2024-04-26T21:14:47.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.234+0000 7f490effd640 1 -- 172.21.15.40:0/1556180496 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241091 (secure 0 0 0) 0x7f49100b8ef0 con 0x7f48fc06c8f0 2024-04-26T21:14:47.238 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:47.239 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 msgr2=0x7f48fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 0x7f48fc06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).stop 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 msgr2=0x7f491014f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f490400c460 tx=0x7f490400c920 comp rx=0 tx=0).stop 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 shutdown_connections 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f48fc06c8f0 0x7f48fc06edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49100d11b0 0x7f4910152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49100cf2e0 0x7f491014f990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 --2- 172.21.15.40:0/1556180496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f491006e490 0x7f49100a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 >> 172.21.15.40:0/1556180496 conn(0x7f49100b4a60 msgr2=0x7f491009c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 shutdown_connections 2024-04-26T21:14:47.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.237+0000 7f49182da640 1 -- 172.21.15.40:0/1556180496 wait complete. 2024-04-26T21:14:47.249 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-26T21:14:46.956214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234980,"kb_used_data":6336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739740,"statfs":{"total":773094113280,"available":772853493760,"internally_reserved":0,"allocated":6488064,"data_stored":3331905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.730269"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.262358+0000","last_change":"2024-04-26T21:14:44.257792+0000","last_active":"2024-04-26T21:14:44.262358+0000","last_peered":"2024-04-26T21:14:44.262358+0000","last_clean":"2024-04-26T21:14:44.262358+0000","last_became_active":"2024-04-26T21:14:44.257435+0000","last_became_peered":"2024-04-26T21:14:44.257435+0000","last_unstale":"2024-04-26T21:14:44.262358+0000","last_undegraded":"2024-04-26T21:14:44.262358+0000","last_fullsized":"2024-04-26T21:14:44.262358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:46.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.264129+0000","last_change":"2024-04-26T21:14:44.261342+0000","last_active":"2024-04-26T21:14:44.264129+0000","last_peered":"2024-04-26T21:14:44.264129+0000","last_clean":"2024-04-26T21:14:44.264129+0000","last_became_active":"2024-04-26T21:14:44.261052+0000","last_became_peered":"2024-04-26T21:14:44.261052+0000","last_unstale":"2024-04-26T21:14:44.264129+0000","last_undegraded":"2024-04-26T21:14:44.264129+0000","last_fullsized":"2024-04-26T21:14:44.264129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:58.831928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230030+0000","last_change":"2024-04-26T21:14:42.230030+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230030+0000","last_undegraded":"2024-04-26T21:14:42.230030+0000","last_fullsized":"2024-04-26T21:14:42.230030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.259790+0000","last_change":"2024-04-26T21:14:44.256368+0000","last_active":"2024-04-26T21:14:44.259790+0000","last_peered":"2024-04-26T21:14:44.259790+0000","last_clean":"2024-04-26T21:14:44.259790+0000","last_became_active":"2024-04-26T21:14:44.256112+0000","last_became_peered":"2024-04-26T21:14:44.256112+0000","last_unstale":"2024-04-26T21:14:44.259790+0000","last_undegraded":"2024-04-26T21:14:44.259790+0000","last_fullsized":"2024-04-26T21:14:44.259790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:33.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227716+0000","last_change":"2024-04-26T21:14:42.227716+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227716+0000","last_undegraded":"2024-04-26T21:14:42.227716+0000","last_fullsized":"2024-04-26T21:14:42.227716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229537+0000","last_change":"2024-04-26T21:14:42.229537+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229537+0000","last_undegraded":"2024-04-26T21:14:42.229537+0000","last_fullsized":"2024-04-26T21:14:42.229537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228944+0000","last_change":"2024-04-26T21:14:42.228944+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228944+0000","last_undegraded":"2024-04-26T21:14:42.228944+0000","last_fullsized":"2024-04-26T21:14:42.228944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226888+0000","last_change":"2024-04-26T21:14:42.226888+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226888+0000","last_undegraded":"2024-04-26T21:14:42.226888+0000","last_fullsized":"2024-04-26T21:14:42.226888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229534+0000","last_change":"2024-04-26T21:14:42.229534+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229534+0000","last_undegraded":"2024-04-26T21:14:42.229534+0000","last_fullsized":"2024-04-26T21:14:42.229534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228273+0000","last_change":"2024-04-26T21:14:42.228273+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228273+0000","last_undegraded":"2024-04-26T21:14:42.228273+0000","last_fullsized":"2024-04-26T21:14:42.228273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227015+0000","last_change":"2024-04-26T21:14:42.227015+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227015+0000","last_undegraded":"2024-04-26T21:14:42.227015+0000","last_fullsized":"2024-04-26T21:14:42.227015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230387+0000","last_change":"2024-04-26T21:14:42.230387+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230387+0000","last_undegraded":"2024-04-26T21:14:42.230387+0000","last_fullsized":"2024-04-26T21:14:42.230387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266245+0000","last_change":"2024-04-26T21:14:44.263354+0000","last_active":"2024-04-26T21:14:44.266245+0000","last_peered":"2024-04-26T21:14:44.266245+0000","last_clean":"2024-04-26T21:14:44.266245+0000","last_became_active":"2024-04-26T21:14:44.263112+0000","last_became_peered":"2024-04-26T21:14:44.263112+0000","last_unstale":"2024-04-26T21:14:44.266245+0000","last_undegraded":"2024-04-26T21:14:44.266245+0000","last_fullsized":"2024-04-26T21:14:44.266245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.834630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230433+0000","last_change":"2024-04-26T21:14:42.230433+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230433+0000","last_undegraded":"2024-04-26T21:14:42.230433+0000","last_fullsized":"2024-04-26T21:14:42.230433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226951+0000","last_change":"2024-04-26T21:14:42.226951+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226951+0000","last_undegraded":"2024-04-26T21:14:42.226951+0000","last_fullsized":"2024-04-26T21:14:42.226951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227714+0000","last_change":"2024-04-26T21:14:42.227714+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227714+0000","last_undegraded":"2024-04-26T21:14:42.227714+0000","last_fullsized":"2024-04-26T21:14:42.227714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231977+0000","last_change":"2024-04-26T21:14:42.231977+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231977+0000","last_undegraded":"2024-04-26T21:14:42.231977+0000","last_fullsized":"2024-04-26T21:14:42.231977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227763+0000","last_change":"2024-04-26T21:14:42.227763+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227763+0000","last_undegraded":"2024-04-26T21:14:42.227763+0000","last_fullsized":"2024-04-26T21:14:42.227763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230480+0000","last_change":"2024-04-26T21:14:42.230480+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230480+0000","last_undegraded":"2024-04-26T21:14:42.230480+0000","last_fullsized":"2024-04-26T21:14:42.230480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227179+0000","last_change":"2024-04-26T21:14:42.227179+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227179+0000","last_undegraded":"2024-04-26T21:14:42.227179+0000","last_fullsized":"2024-04-26T21:14:42.227179+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230093+0000","last_change":"2024-04-26T21:14:42.230093+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230093+0000","last_undegraded":"2024-04-26T21:14:42.230093+0000","last_fullsized":"2024-04-26T21:14:42.230093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228061+0000","last_change":"2024-04-26T21:14:42.228061+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228061+0000","last_undegraded":"2024-04-26T21:14:42.228061+0000","last_fullsized":"2024-04-26T21:14:42.228061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229662+0000","last_change":"2024-04-26T21:14:42.229662+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229662+0000","last_undegraded":"2024-04-26T21:14:42.229662+0000","last_fullsized":"2024-04-26T21:14:42.229662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237869+0000","last_change":"2024-04-26T21:14:43.241349+0000","last_active":"2024-04-26T21:14:44.237869+0000","last_peered":"2024-04-26T21:14:44.237869+0000","last_clean":"2024-04-26T21:14:44.237869+0000","last_became_active":"2024-04-26T21:14:43.241055+0000","last_became_peered":"2024-04-26T21:14:43.241055+0000","last_unstale":"2024-04-26T21:14:44.237869+0000","last_undegraded":"2024-04-26T21:14:44.237869+0000","last_fullsized":"2024-04-26T21:14:44.237869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.604628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227718+0000","last_change":"2024-04-26T21:14:42.227718+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227718+0000","last_undegraded":"2024-04-26T21:14:42.227718+0000","last_fullsized":"2024-04-26T21:14:42.227718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.263556+0000","last_change":"2024-04-26T21:14:43.247928+0000","last_active":"2024-04-26T21:14:44.263556+0000","last_peered":"2024-04-26T21:14:44.263556+0000","last_clean":"2024-04-26T21:14:44.263556+0000","last_became_active":"2024-04-26T21:14:43.247549+0000","last_became_peered":"2024-04-26T21:14:43.247549+0000","last_unstale":"2024-04-26T21:14:44.263556+0000","last_undegraded":"2024-04-26T21:14:44.263556+0000","last_fullsized":"2024-04-26T21:14:44.263556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:27.702426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229536+0000","last_change":"2024-04-26T21:14:42.229536+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229536+0000","last_undegraded":"2024-04-26T21:14:42.229536+0000","last_fullsized":"2024-04-26T21:14:42.229536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266261+0000","last_change":"2024-04-26T21:14:44.263566+0000","last_active":"2024-04-26T21:14:44.266261+0000","last_peered":"2024-04-26T21:14:44.266261+0000","last_clean":"2024-04-26T21:14:44.266261+0000","last_became_active":"2024-04-26T21:14:44.263259+0000","last_became_peered":"2024-04-26T21:14:44.263259+0000","last_unstale":"2024-04-26T21:14:44.266261+0000","last_undegraded":"2024-04-26T21:14:44.266261+0000","last_fullsized":"2024-04-26T21:14:44.266261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:00.527919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231868+0000","last_change":"2024-04-26T21:14:42.231868+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231868+0000","last_undegraded":"2024-04-26T21:14:42.231868+0000","last_fullsized":"2024-04-26T21:14:42.231868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227738+0000","last_change":"2024-04-26T21:14:42.227738+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227738+0000","last_undegraded":"2024-04-26T21:14:42.227738+0000","last_fullsized":"2024-04-26T21:14:42.227738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237349+0000","last_change":"2024-04-26T21:14:43.243196+0000","last_active":"2024-04-26T21:14:44.237349+0000","last_peered":"2024-04-26T21:14:44.237349+0000","last_clean":"2024-04-26T21:14:44.237349+0000","last_became_active":"2024-04-26T21:14:43.242369+0000","last_became_peered":"2024-04-26T21:14:43.242369+0000","last_unstale":"2024-04-26T21:14:44.237349+0000","last_undegraded":"2024-04-26T21:14:44.237349+0000","last_fullsized":"2024-04-26T21:14:44.237349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:21.948893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226937+0000","last_change":"2024-04-26T21:14:42.226937+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226937+0000","last_undegraded":"2024-04-26T21:14:42.226937+0000","last_fullsized":"2024-04-26T21:14:42.226937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.252042+0000","last_change":"2024-04-26T21:14:44.247167+0000","last_active":"2024-04-26T21:14:44.252042+0000","last_peered":"2024-04-26T21:14:44.252042+0000","last_clean":"2024-04-26T21:14:44.252042+0000","last_became_active":"2024-04-26T21:14:44.246911+0000","last_became_peered":"2024-04-26T21:14:44.246911+0000","last_unstale":"2024-04-26T21:14:44.252042+0000","last_undegraded":"2024-04-26T21:14:44.252042+0000","last_fullsized":"2024-04-26T21:14:44.252042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:15.155730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.236660+0000","last_change":"2024-04-26T21:14:43.241399+0000","last_active":"2024-04-26T21:14:44.236660+0000","last_peered":"2024-04-26T21:14:44.236660+0000","last_clean":"2024-04-26T21:14:44.236660+0000","last_became_active":"2024-04-26T21:14:43.241031+0000","last_became_peered":"2024-04-26T21:14:43.241031+0000","last_unstale":"2024-04-26T21:14:44.236660+0000","last_undegraded":"2024-04-26T21:14:44.236660+0000","last_fullsized":"2024-04-26T21:14:44.236660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:41.265024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227167+0000","last_change":"2024-04-26T21:14:42.227167+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227167+0000","last_undegraded":"2024-04-26T21:14:42.227167+0000","last_fullsized":"2024-04-26T21:14:42.227167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231788+0000","last_change":"2024-04-26T21:14:42.231788+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231788+0000","last_undegraded":"2024-04-26T21:14:42.231788+0000","last_fullsized":"2024-04-26T21:14:42.231788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T21:14:42.230938+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:42.230938+0000","last_peered":"2024-04-26T21:14:42.230938+0000","last_clean":"2024-04-26T21:14:42.230938+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:42.230938+0000","last_undegraded":"2024-04-26T21:14:42.230938+0000","last_fullsized":"2024-04-26T21:14:42.230938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":853987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":282568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:47.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:47.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 Processor -- start 2024-04-26T21:14:47.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 -- start start 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf600b44e0 0x7fbf6006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbf6006d2b0 0x7fbf6006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf600b4a10 con 0x7fbf6015f1e0 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6006dc60 con 0x7fbf600b44e0 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.390+0000 7fbf67015640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6006eeb0 con 0x7fbf6006d2b0 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf600b44e0 0x7fbf6006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf64db1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf6015f1e0 0x7fbf600ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57650/0 (socket says 172.21.15.40:57650) 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf64db1640 1 -- 172.21.15.40:0/502824735 learned_addr learned my addr 172.21.15.40:0/502824735 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5ffff640 1 -- 172.21.15.40:0/502824735 >> v2:172.21.15.184:3300/0 conn(0x7fbf6006d2b0 msgr2=0x7fbf6006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5ffff640 1 --2- 172.21.15.40:0/502824735 >> v2:172.21.15.184:3300/0 conn(0x7fbf6006d2b0 0x7fbf6006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5ffff640 1 -- 172.21.15.40:0/502824735 >> v1:172.21.15.40:6789/0 conn(0x7fbf6015f1e0 legacy=0x7fbf600ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5ffff640 1 -- 172.21.15.40:0/502824735 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600b4cf0 con 0x7fbf600b44e0 2024-04-26T21:14:47.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.391+0000 7fbf5f7fe640 1 -- 172.21.15.40:0/502824735 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318181071 0 0) 0x7fbf600b4a10 con 0x7fbf6015f1e0 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf5ffff640 1 --2- 172.21.15.40:0/502824735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf6006e970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbf600b0ba0 tx=0x7fbf4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc9cc5955ebba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf5f7fe640 1 -- 172.21.15.40:0/502824735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c037070 con 0x7fbf600b44e0 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf5f7fe640 1 -- 172.21.15.40:0/502824735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c032440 con 0x7fbf600b44e0 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf5f7fe640 1 -- 172.21.15.40:0/502824735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c032810 con 0x7fbf600b44e0 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 -- 172.21.15.40:0/502824735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 msgr2=0x7fbf6006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 --2- 172.21.15.40:0/502824735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf6006e970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbf600b0ba0 tx=0x7fbf4c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 -- 172.21.15.40:0/502824735 shutdown_connections 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 --2- 172.21.15.40:0/502824735 >> v2:172.21.15.184:3300/0 conn(0x7fbf6006d2b0 0x7fbf6006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 --2- 172.21.15.40:0/502824735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf6006e970 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 -- 172.21.15.40:0/502824735 >> 172.21.15.40:0/502824735 conn(0x7fbf600728a0 msgr2=0x7fbf60072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 -- 172.21.15.40:0/502824735 shutdown_connections 2024-04-26T21:14:47.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.392+0000 7fbf67015640 1 -- 172.21.15.40:0/502824735 wait complete. 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.393+0000 7fbf67015640 1 Processor -- start 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 -- start start 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 0x7fbf601510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 0x7fbf60155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600b5590 con 0x7fbf600b44e0 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600b52b0 con 0x7fbf6015f1e0 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf67015640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6015f5c0 con 0x7fbf6006d2b0 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.394+0000 7fbf5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55864/0 (socket says 172.21.15.40:55864) 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 -- 172.21.15.40:0/2403636777 learned_addr learned my addr 172.21.15.40:0/2403636777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:47.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf64db1640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 0x7fbf601510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 -- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 msgr2=0x7fbf601510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf655b2640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 0x7fbf60155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 0x7fbf601510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 -- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 msgr2=0x7fbf60155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 0x7fbf60155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 -- 172.21.15.40:0/2403636777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf60152490 con 0x7fbf600b44e0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf64db1640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 0x7fbf601510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf655b2640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 0x7fbf60155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5ffff640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009890 tx=0x7fbf4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c037040 con 0x7fbf600b44e0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c004230 con 0x7fbf600b44e0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf60152770 con 0x7fbf600b44e0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c032e20 con 0x7fbf600b44e0 2024-04-26T21:14:47.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.395+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf60152d30 con 0x7fbf600b44e0 2024-04-26T21:14:47.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.396+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6006a290 con 0x7fbf600b44e0 2024-04-26T21:14:47.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.397+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf4c030770 con 0x7fbf600b44e0 2024-04-26T21:14:47.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.397+0000 7fbf5d7fa640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.397+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbf4c0acd60 con 0x7fbf600b44e0 2024-04-26T21:14:47.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.397+0000 7fbf64db1640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.398+0000 7fbf64db1640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 0x7fbf4406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f60 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.401+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf4c07b220 con 0x7fbf600b44e0 2024-04-26T21:14:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.581+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf60073710 con 0x7fbf4406c8f0 2024-04-26T21:14:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.587+0000 7fbf5d7fa640 1 -- 172.21.15.40:0/2403636777 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241091 (secure 0 0 0) 0x7fbf54001c90 con 0x7fbf4406c8f0 2024-04-26T21:14:47.591 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 msgr2=0x7fbf4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 0x7fbf4406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f60 tx=0x7fbf54009040 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 msgr2=0x7fbf60151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009890 tx=0x7fbf4c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 shutdown_connections 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbf4406c8f0 0x7fbf4406edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf6015f1e0 0x7fbf60155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf600b44e0 0x7fbf60151ad0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 --2- 172.21.15.40:0/2403636777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf6006d2b0 0x7fbf601510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 >> 172.21.15.40:0/2403636777 conn(0x7fbf600728a0 msgr2=0x7fbf6006fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 shutdown_connections 2024-04-26T21:14:47.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.590+0000 7fbf67015640 1 -- 172.21.15.40:0/2403636777 wait complete. 2024-04-26T21:14:47.602 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-26T21:14:46.956214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234980,"kb_used_data":6336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739740,"statfs":{"total":773094113280,"available":772853493760,"internally_reserved":0,"allocated":6488064,"data_stored":3331905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.730269"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.262358+0000","last_change":"2024-04-26T21:14:44.257792+0000","last_active":"2024-04-26T21:14:44.262358+0000","last_peered":"2024-04-26T21:14:44.262358+0000","last_clean":"2024-04-26T21:14:44.262358+0000","last_became_active":"2024-04-26T21:14:44.257435+0000","last_became_peered":"2024-04-26T21:14:44.257435+0000","last_unstale":"2024-04-26T21:14:44.262358+0000","last_undegraded":"2024-04-26T21:14:44.262358+0000","last_fullsized":"2024-04-26T21:14:44.262358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:46.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.264129+0000","last_change":"2024-04-26T21:14:44.261342+0000","last_active":"2024-04-26T21:14:44.264129+0000","last_peered":"2024-04-26T21:14:44.264129+0000","last_clean":"2024-04-26T21:14:44.264129+0000","last_became_active":"2024-04-26T21:14:44.261052+0000","last_became_peered":"2024-04-26T21:14:44.261052+0000","last_unstale":"2024-04-26T21:14:44.264129+0000","last_undegraded":"2024-04-26T21:14:44.264129+0000","last_fullsized":"2024-04-26T21:14:44.264129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:58.831928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230030+0000","last_change":"2024-04-26T21:14:42.230030+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230030+0000","last_undegraded":"2024-04-26T21:14:42.230030+0000","last_fullsized":"2024-04-26T21:14:42.230030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.259790+0000","last_change":"2024-04-26T21:14:44.256368+0000","last_active":"2024-04-26T21:14:44.259790+0000","last_peered":"2024-04-26T21:14:44.259790+0000","last_clean":"2024-04-26T21:14:44.259790+0000","last_became_active":"2024-04-26T21:14:44.256112+0000","last_became_peered":"2024-04-26T21:14:44.256112+0000","last_unstale":"2024-04-26T21:14:44.259790+0000","last_undegraded":"2024-04-26T21:14:44.259790+0000","last_fullsized":"2024-04-26T21:14:44.259790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:33.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227716+0000","last_change":"2024-04-26T21:14:42.227716+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227716+0000","last_undegraded":"2024-04-26T21:14:42.227716+0000","last_fullsized":"2024-04-26T21:14:42.227716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229537+0000","last_change":"2024-04-26T21:14:42.229537+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229537+0000","last_undegraded":"2024-04-26T21:14:42.229537+0000","last_fullsized":"2024-04-26T21:14:42.229537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228944+0000","last_change":"2024-04-26T21:14:42.228944+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228944+0000","last_undegraded":"2024-04-26T21:14:42.228944+0000","last_fullsized":"2024-04-26T21:14:42.228944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226888+0000","last_change":"2024-04-26T21:14:42.226888+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226888+0000","last_undegraded":"2024-04-26T21:14:42.226888+0000","last_fullsized":"2024-04-26T21:14:42.226888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229534+0000","last_change":"2024-04-26T21:14:42.229534+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229534+0000","last_undegraded":"2024-04-26T21:14:42.229534+0000","last_fullsized":"2024-04-26T21:14:42.229534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228273+0000","last_change":"2024-04-26T21:14:42.228273+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228273+0000","last_undegraded":"2024-04-26T21:14:42.228273+0000","last_fullsized":"2024-04-26T21:14:42.228273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227015+0000","last_change":"2024-04-26T21:14:42.227015+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227015+0000","last_undegraded":"2024-04-26T21:14:42.227015+0000","last_fullsized":"2024-04-26T21:14:42.227015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230387+0000","last_change":"2024-04-26T21:14:42.230387+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230387+0000","last_undegraded":"2024-04-26T21:14:42.230387+0000","last_fullsized":"2024-04-26T21:14:42.230387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266245+0000","last_change":"2024-04-26T21:14:44.263354+0000","last_active":"2024-04-26T21:14:44.266245+0000","last_peered":"2024-04-26T21:14:44.266245+0000","last_clean":"2024-04-26T21:14:44.266245+0000","last_became_active":"2024-04-26T21:14:44.263112+0000","last_became_peered":"2024-04-26T21:14:44.263112+0000","last_unstale":"2024-04-26T21:14:44.266245+0000","last_undegraded":"2024-04-26T21:14:44.266245+0000","last_fullsized":"2024-04-26T21:14:44.266245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.834630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230433+0000","last_change":"2024-04-26T21:14:42.230433+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230433+0000","last_undegraded":"2024-04-26T21:14:42.230433+0000","last_fullsized":"2024-04-26T21:14:42.230433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226951+0000","last_change":"2024-04-26T21:14:42.226951+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226951+0000","last_undegraded":"2024-04-26T21:14:42.226951+0000","last_fullsized":"2024-04-26T21:14:42.226951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227714+0000","last_change":"2024-04-26T21:14:42.227714+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227714+0000","last_undegraded":"2024-04-26T21:14:42.227714+0000","last_fullsized":"2024-04-26T21:14:42.227714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231977+0000","last_change":"2024-04-26T21:14:42.231977+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231977+0000","last_undegraded":"2024-04-26T21:14:42.231977+0000","last_fullsized":"2024-04-26T21:14:42.231977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227763+0000","last_change":"2024-04-26T21:14:42.227763+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227763+0000","last_undegraded":"2024-04-26T21:14:42.227763+0000","last_fullsized":"2024-04-26T21:14:42.227763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230480+0000","last_change":"2024-04-26T21:14:42.230480+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230480+0000","last_undegraded":"2024-04-26T21:14:42.230480+0000","last_fullsized":"2024-04-26T21:14:42.230480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227179+0000","last_change":"2024-04-26T21:14:42.227179+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227179+0000","last_undegraded":"2024-04-26T21:14:42.227179+0000","last_fullsized":"2024-04-26T21:14:42.227179+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230093+0000","last_change":"2024-04-26T21:14:42.230093+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230093+0000","last_undegraded":"2024-04-26T21:14:42.230093+0000","last_fullsized":"2024-04-26T21:14:42.230093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228061+0000","last_change":"2024-04-26T21:14:42.228061+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228061+0000","last_undegraded":"2024-04-26T21:14:42.228061+0000","last_fullsized":"2024-04-26T21:14:42.228061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229662+0000","last_change":"2024-04-26T21:14:42.229662+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229662+0000","last_undegraded":"2024-04-26T21:14:42.229662+0000","last_fullsized":"2024-04-26T21:14:42.229662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237869+0000","last_change":"2024-04-26T21:14:43.241349+0000","last_active":"2024-04-26T21:14:44.237869+0000","last_peered":"2024-04-26T21:14:44.237869+0000","last_clean":"2024-04-26T21:14:44.237869+0000","last_became_active":"2024-04-26T21:14:43.241055+0000","last_became_peered":"2024-04-26T21:14:43.241055+0000","last_unstale":"2024-04-26T21:14:44.237869+0000","last_undegraded":"2024-04-26T21:14:44.237869+0000","last_fullsized":"2024-04-26T21:14:44.237869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.604628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227718+0000","last_change":"2024-04-26T21:14:42.227718+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227718+0000","last_undegraded":"2024-04-26T21:14:42.227718+0000","last_fullsized":"2024-04-26T21:14:42.227718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.263556+0000","last_change":"2024-04-26T21:14:43.247928+0000","last_active":"2024-04-26T21:14:44.263556+0000","last_peered":"2024-04-26T21:14:44.263556+0000","last_clean":"2024-04-26T21:14:44.263556+0000","last_became_active":"2024-04-26T21:14:43.247549+0000","last_became_peered":"2024-04-26T21:14:43.247549+0000","last_unstale":"2024-04-26T21:14:44.263556+0000","last_undegraded":"2024-04-26T21:14:44.263556+0000","last_fullsized":"2024-04-26T21:14:44.263556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:27.702426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229536+0000","last_change":"2024-04-26T21:14:42.229536+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229536+0000","last_undegraded":"2024-04-26T21:14:42.229536+0000","last_fullsized":"2024-04-26T21:14:42.229536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266261+0000","last_change":"2024-04-26T21:14:44.263566+0000","last_active":"2024-04-26T21:14:44.266261+0000","last_peered":"2024-04-26T21:14:44.266261+0000","last_clean":"2024-04-26T21:14:44.266261+0000","last_became_active":"2024-04-26T21:14:44.263259+0000","last_became_peered":"2024-04-26T21:14:44.263259+0000","last_unstale":"2024-04-26T21:14:44.266261+0000","last_undegraded":"2024-04-26T21:14:44.266261+0000","last_fullsized":"2024-04-26T21:14:44.266261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:00.527919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231868+0000","last_change":"2024-04-26T21:14:42.231868+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231868+0000","last_undegraded":"2024-04-26T21:14:42.231868+0000","last_fullsized":"2024-04-26T21:14:42.231868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227738+0000","last_change":"2024-04-26T21:14:42.227738+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227738+0000","last_undegraded":"2024-04-26T21:14:42.227738+0000","last_fullsized":"2024-04-26T21:14:42.227738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237349+0000","last_change":"2024-04-26T21:14:43.243196+0000","last_active":"2024-04-26T21:14:44.237349+0000","last_peered":"2024-04-26T21:14:44.237349+0000","last_clean":"2024-04-26T21:14:44.237349+0000","last_became_active":"2024-04-26T21:14:43.242369+0000","last_became_peered":"2024-04-26T21:14:43.242369+0000","last_unstale":"2024-04-26T21:14:44.237349+0000","last_undegraded":"2024-04-26T21:14:44.237349+0000","last_fullsized":"2024-04-26T21:14:44.237349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:21.948893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226937+0000","last_change":"2024-04-26T21:14:42.226937+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226937+0000","last_undegraded":"2024-04-26T21:14:42.226937+0000","last_fullsized":"2024-04-26T21:14:42.226937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.252042+0000","last_change":"2024-04-26T21:14:44.247167+0000","last_active":"2024-04-26T21:14:44.252042+0000","last_peered":"2024-04-26T21:14:44.252042+0000","last_clean":"2024-04-26T21:14:44.252042+0000","last_became_active":"2024-04-26T21:14:44.246911+0000","last_became_peered":"2024-04-26T21:14:44.246911+0000","last_unstale":"2024-04-26T21:14:44.252042+0000","last_undegraded":"2024-04-26T21:14:44.252042+0000","last_fullsized":"2024-04-26T21:14:44.252042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:15.155730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.236660+0000","last_change":"2024-04-26T21:14:43.241399+0000","last_active":"2024-04-26T21:14:44.236660+0000","last_peered":"2024-04-26T21:14:44.236660+0000","last_clean":"2024-04-26T21:14:44.236660+0000","last_became_active":"2024-04-26T21:14:43.241031+0000","last_became_peered":"2024-04-26T21:14:43.241031+0000","last_unstale":"2024-04-26T21:14:44.236660+0000","last_undegraded":"2024-04-26T21:14:44.236660+0000","last_fullsized":"2024-04-26T21:14:44.236660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:41.265024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227167+0000","last_change":"2024-04-26T21:14:42.227167+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227167+0000","last_undegraded":"2024-04-26T21:14:42.227167+0000","last_fullsized":"2024-04-26T21:14:42.227167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231788+0000","last_change":"2024-04-26T21:14:42.231788+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231788+0000","last_undegraded":"2024-04-26T21:14:42.231788+0000","last_fullsized":"2024-04-26T21:14:42.231788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T21:14:42.230938+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:42.230938+0000","last_peered":"2024-04-26T21:14:42.230938+0000","last_clean":"2024-04-26T21:14:42.230938+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:42.230938+0000","last_undegraded":"2024-04-26T21:14:42.230938+0000","last_fullsized":"2024-04-26T21:14:42.230938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":853987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":282568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:47.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.742+0000 7fcebdf46640 1 Processor -- start 2024-04-26T21:14:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.742+0000 7fcebdf46640 1 -- start start 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fcebdf46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fceb806e490 0x7fceb815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fcebdf46640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb80d1740 con 0x7fceb80d11b0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fcebdf46640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb80d1a20 con 0x7fceb80cf2e0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fcebdf46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fceb80d1d00 con 0x7fceb806e490 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fceb80d11b0 0x7fceb806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57660/0 (socket says 172.21.15.40:57660) 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb6ffd640 1 -- 172.21.15.40:0/1602395020 learned_addr learned my addr 172.21.15.40:0/1602395020 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb7fff640 1 --2- 172.21.15.40:0/1602395020 >> v2:172.21.15.184:3300/0 conn(0x7fceb806e490 0x7fceb815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1397567908 0 0) 0x7fceb80d1740 con 0x7fceb80d11b0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcea0003520 con 0x7fceb80d11b0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212614593 0 0) 0x7fceb80d1a20 con 0x7fceb80cf2e0 2024-04-26T21:14:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb80d1740 con 0x7fceb80cf2e0 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4230181662 0 0) 0x7fcea0003520 con 0x7fceb80d11b0 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.743+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 >> v2:172.21.15.184:3300/0 conn(0x7fceb806e490 msgr2=0x7fceb815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 --2- 172.21.15.40:0/1602395020 >> v2:172.21.15.184:3300/0 conn(0x7fceb806e490 0x7fceb815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 >> v1:172.21.15.184:6789/0 conn(0x7fceb80cf2e0 legacy=0x7fceb806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb80d1fe0 con 0x7fceb80d11b0 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fceac001670 con 0x7fceb80d11b0 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 >> v1:172.21.15.40:6789/0 conn(0x7fceb80d11b0 legacy=0x7fceb806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcea0003b40 0x7fcea0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea0004800 0x7fcea0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 0x7fcea000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80d1d00 con 0x7fcea00051c0 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcea000c040 con 0x7fcea0003b40 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcea000c240 con 0x7fcea0004800 2024-04-26T21:14:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 0x7fcea000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb77fe640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcea0003b40 0x7fcea0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb6ffd640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea0004800 0x7fcea0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 -- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea0004800 msgr2=0x7fcea0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea0004800 0x7fcea0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 -- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcea0003b40 msgr2=0x7fcea0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcea0003b40 0x7fcea0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb7fff640 1 -- 172.21.15.40:0/1602395020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea0003520 con 0x7fcea00051c0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.744+0000 7fceb6ffd640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea0004800 0x7fcea0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fceb7fff640 1 -- 172.21.15.40:0/1602395020 reap_dead start 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fceb7fff640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 0x7fcea000b9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fceb8074240 tx=0x7fcea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4778e9f388ac8acc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0009c10 con 0x7fcea00051c0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb0031410 con 0x7fcea00051c0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fceb67fc640 1 -- 172.21.15.40:0/1602395020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb00317e0 con 0x7fcea00051c0 2024-04-26T21:14:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fcebdf46640 1 -- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 msgr2=0x7fcea000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fcebdf46640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 0x7fcea000b9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fceb8074240 tx=0x7fcea0005830 comp rx=0 tx=0).stop 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fcebdf46640 1 -- 172.21.15.40:0/1602395020 shutdown_connections 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fcebdf46640 1 --2- 172.21.15.40:0/1602395020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcea00051c0 0x7fcea000b9c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.745+0000 7fcebdf46640 1 -- 172.21.15.40:0/1602395020 >> 172.21.15.40:0/1602395020 conn(0x7fceb80b4a60 msgr2=0x7fceb80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.746+0000 7fcebdf46640 1 -- 172.21.15.40:0/1602395020 shutdown_connections 2024-04-26T21:14:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.746+0000 7fcebdf46640 1 -- 172.21.15.40:0/1602395020 wait complete. 2024-04-26T21:14:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.747+0000 7fcebdf46640 1 Processor -- start 2024-04-26T21:14:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.747+0000 7fcebdf46640 1 -- start start 2024-04-26T21:14:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea00051c0 0x7fceb8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb80b3ad0 0x7fceb80b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80cf840 con 0x7fceb80b2240 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80cf6c0 con 0x7fceb80b3ad0 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fceb806dd80 con 0x7fcea00051c0 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55884/0 (socket says 172.21.15.40:55884) 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 -- 172.21.15.40:0/3897174924 learned_addr learned my addr 172.21.15.40:0/3897174924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb77fe640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea00051c0 0x7fceb8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb6ffd640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb80b3ad0 0x7fceb80b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 -- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea00051c0 msgr2=0x7fceb8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea00051c0 0x7fceb8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 -- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb80b3ad0 msgr2=0x7fceb80b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb80b3ad0 0x7fceb80b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 -- 172.21.15.40:0/3897174924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb80b2a30 con 0x7fceb80b2240 2024-04-26T21:14:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fceb7fff640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fceb0033040 tx=0x7fceb0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0009c10 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb0043070 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb003c820 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb80b2d10 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.748+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb80b32d0 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.749+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb806a290 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.750+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceb0005d10 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.750+0000 7fce9ffff640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.750+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fceb00ac3a0 con 0x7fceb80b2240 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.750+0000 7fceb77fe640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.751+0000 7fceb77fe640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 0x7fcea406edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.754+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fceb002e050 con 0x7fceb80b2240 2024-04-26T21:14:47.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.936+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fceb80ce900 con 0x7fcea406c8f0 2024-04-26T21:14:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.942+0000 7fce9ffff640 1 -- 172.21.15.40:0/3897174924 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241091 (secure 0 0 0) 0x7fceb80ce900 con 0x7fcea406c8f0 2024-04-26T21:14:47.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.944+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 msgr2=0x7fcea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.944+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 0x7fcea406edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea8009040 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.944+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 msgr2=0x7fceb809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.944+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fceb0033040 tx=0x7fceb0009750 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 shutdown_connections 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcea406c8f0 0x7fcea406edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb80b2240 0x7fceb809dac0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb80b3ad0 0x7fceb80b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 --2- 172.21.15.40:0/3897174924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcea00051c0 0x7fceb8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 >> 172.21.15.40:0/3897174924 conn(0x7fceb80b4a60 msgr2=0x7fceb809c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 shutdown_connections 2024-04-26T21:14:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:47.945+0000 7fcebdf46640 1 -- 172.21.15.40:0/3897174924 wait complete. 2024-04-26T21:14:47.957 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-26T21:14:46.956214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234980,"kb_used_data":6336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739740,"statfs":{"total":773094113280,"available":772853493760,"internally_reserved":0,"allocated":6488064,"data_stored":3331905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.730269"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.262358+0000","last_change":"2024-04-26T21:14:44.257792+0000","last_active":"2024-04-26T21:14:44.262358+0000","last_peered":"2024-04-26T21:14:44.262358+0000","last_clean":"2024-04-26T21:14:44.262358+0000","last_became_active":"2024-04-26T21:14:44.257435+0000","last_became_peered":"2024-04-26T21:14:44.257435+0000","last_unstale":"2024-04-26T21:14:44.262358+0000","last_undegraded":"2024-04-26T21:14:44.262358+0000","last_fullsized":"2024-04-26T21:14:44.262358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:46.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.264129+0000","last_change":"2024-04-26T21:14:44.261342+0000","last_active":"2024-04-26T21:14:44.264129+0000","last_peered":"2024-04-26T21:14:44.264129+0000","last_clean":"2024-04-26T21:14:44.264129+0000","last_became_active":"2024-04-26T21:14:44.261052+0000","last_became_peered":"2024-04-26T21:14:44.261052+0000","last_unstale":"2024-04-26T21:14:44.264129+0000","last_undegraded":"2024-04-26T21:14:44.264129+0000","last_fullsized":"2024-04-26T21:14:44.264129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:58.831928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230030+0000","last_change":"2024-04-26T21:14:42.230030+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230030+0000","last_undegraded":"2024-04-26T21:14:42.230030+0000","last_fullsized":"2024-04-26T21:14:42.230030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.259790+0000","last_change":"2024-04-26T21:14:44.256368+0000","last_active":"2024-04-26T21:14:44.259790+0000","last_peered":"2024-04-26T21:14:44.259790+0000","last_clean":"2024-04-26T21:14:44.259790+0000","last_became_active":"2024-04-26T21:14:44.256112+0000","last_became_peered":"2024-04-26T21:14:44.256112+0000","last_unstale":"2024-04-26T21:14:44.259790+0000","last_undegraded":"2024-04-26T21:14:44.259790+0000","last_fullsized":"2024-04-26T21:14:44.259790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:33.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227716+0000","last_change":"2024-04-26T21:14:42.227716+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227716+0000","last_undegraded":"2024-04-26T21:14:42.227716+0000","last_fullsized":"2024-04-26T21:14:42.227716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229537+0000","last_change":"2024-04-26T21:14:42.229537+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229537+0000","last_undegraded":"2024-04-26T21:14:42.229537+0000","last_fullsized":"2024-04-26T21:14:42.229537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228944+0000","last_change":"2024-04-26T21:14:42.228944+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228944+0000","last_undegraded":"2024-04-26T21:14:42.228944+0000","last_fullsized":"2024-04-26T21:14:42.228944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226888+0000","last_change":"2024-04-26T21:14:42.226888+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226888+0000","last_undegraded":"2024-04-26T21:14:42.226888+0000","last_fullsized":"2024-04-26T21:14:42.226888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229534+0000","last_change":"2024-04-26T21:14:42.229534+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229534+0000","last_undegraded":"2024-04-26T21:14:42.229534+0000","last_fullsized":"2024-04-26T21:14:42.229534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228273+0000","last_change":"2024-04-26T21:14:42.228273+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228273+0000","last_undegraded":"2024-04-26T21:14:42.228273+0000","last_fullsized":"2024-04-26T21:14:42.228273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227015+0000","last_change":"2024-04-26T21:14:42.227015+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227015+0000","last_undegraded":"2024-04-26T21:14:42.227015+0000","last_fullsized":"2024-04-26T21:14:42.227015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230387+0000","last_change":"2024-04-26T21:14:42.230387+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230387+0000","last_undegraded":"2024-04-26T21:14:42.230387+0000","last_fullsized":"2024-04-26T21:14:42.230387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266245+0000","last_change":"2024-04-26T21:14:44.263354+0000","last_active":"2024-04-26T21:14:44.266245+0000","last_peered":"2024-04-26T21:14:44.266245+0000","last_clean":"2024-04-26T21:14:44.266245+0000","last_became_active":"2024-04-26T21:14:44.263112+0000","last_became_peered":"2024-04-26T21:14:44.263112+0000","last_unstale":"2024-04-26T21:14:44.266245+0000","last_undegraded":"2024-04-26T21:14:44.266245+0000","last_fullsized":"2024-04-26T21:14:44.266245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.834630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230433+0000","last_change":"2024-04-26T21:14:42.230433+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230433+0000","last_undegraded":"2024-04-26T21:14:42.230433+0000","last_fullsized":"2024-04-26T21:14:42.230433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226951+0000","last_change":"2024-04-26T21:14:42.226951+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226951+0000","last_undegraded":"2024-04-26T21:14:42.226951+0000","last_fullsized":"2024-04-26T21:14:42.226951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227714+0000","last_change":"2024-04-26T21:14:42.227714+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227714+0000","last_undegraded":"2024-04-26T21:14:42.227714+0000","last_fullsized":"2024-04-26T21:14:42.227714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231977+0000","last_change":"2024-04-26T21:14:42.231977+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231977+0000","last_undegraded":"2024-04-26T21:14:42.231977+0000","last_fullsized":"2024-04-26T21:14:42.231977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227763+0000","last_change":"2024-04-26T21:14:42.227763+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227763+0000","last_undegraded":"2024-04-26T21:14:42.227763+0000","last_fullsized":"2024-04-26T21:14:42.227763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230480+0000","last_change":"2024-04-26T21:14:42.230480+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230480+0000","last_undegraded":"2024-04-26T21:14:42.230480+0000","last_fullsized":"2024-04-26T21:14:42.230480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227179+0000","last_change":"2024-04-26T21:14:42.227179+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227179+0000","last_undegraded":"2024-04-26T21:14:42.227179+0000","last_fullsized":"2024-04-26T21:14:42.227179+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230093+0000","last_change":"2024-04-26T21:14:42.230093+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230093+0000","last_undegraded":"2024-04-26T21:14:42.230093+0000","last_fullsized":"2024-04-26T21:14:42.230093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228061+0000","last_change":"2024-04-26T21:14:42.228061+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228061+0000","last_undegraded":"2024-04-26T21:14:42.228061+0000","last_fullsized":"2024-04-26T21:14:42.228061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229662+0000","last_change":"2024-04-26T21:14:42.229662+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229662+0000","last_undegraded":"2024-04-26T21:14:42.229662+0000","last_fullsized":"2024-04-26T21:14:42.229662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237869+0000","last_change":"2024-04-26T21:14:43.241349+0000","last_active":"2024-04-26T21:14:44.237869+0000","last_peered":"2024-04-26T21:14:44.237869+0000","last_clean":"2024-04-26T21:14:44.237869+0000","last_became_active":"2024-04-26T21:14:43.241055+0000","last_became_peered":"2024-04-26T21:14:43.241055+0000","last_unstale":"2024-04-26T21:14:44.237869+0000","last_undegraded":"2024-04-26T21:14:44.237869+0000","last_fullsized":"2024-04-26T21:14:44.237869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.604628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227718+0000","last_change":"2024-04-26T21:14:42.227718+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227718+0000","last_undegraded":"2024-04-26T21:14:42.227718+0000","last_fullsized":"2024-04-26T21:14:42.227718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.263556+0000","last_change":"2024-04-26T21:14:43.247928+0000","last_active":"2024-04-26T21:14:44.263556+0000","last_peered":"2024-04-26T21:14:44.263556+0000","last_clean":"2024-04-26T21:14:44.263556+0000","last_became_active":"2024-04-26T21:14:43.247549+0000","last_became_peered":"2024-04-26T21:14:43.247549+0000","last_unstale":"2024-04-26T21:14:44.263556+0000","last_undegraded":"2024-04-26T21:14:44.263556+0000","last_fullsized":"2024-04-26T21:14:44.263556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:27.702426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229536+0000","last_change":"2024-04-26T21:14:42.229536+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229536+0000","last_undegraded":"2024-04-26T21:14:42.229536+0000","last_fullsized":"2024-04-26T21:14:42.229536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266261+0000","last_change":"2024-04-26T21:14:44.263566+0000","last_active":"2024-04-26T21:14:44.266261+0000","last_peered":"2024-04-26T21:14:44.266261+0000","last_clean":"2024-04-26T21:14:44.266261+0000","last_became_active":"2024-04-26T21:14:44.263259+0000","last_became_peered":"2024-04-26T21:14:44.263259+0000","last_unstale":"2024-04-26T21:14:44.266261+0000","last_undegraded":"2024-04-26T21:14:44.266261+0000","last_fullsized":"2024-04-26T21:14:44.266261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:00.527919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231868+0000","last_change":"2024-04-26T21:14:42.231868+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231868+0000","last_undegraded":"2024-04-26T21:14:42.231868+0000","last_fullsized":"2024-04-26T21:14:42.231868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227738+0000","last_change":"2024-04-26T21:14:42.227738+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227738+0000","last_undegraded":"2024-04-26T21:14:42.227738+0000","last_fullsized":"2024-04-26T21:14:42.227738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237349+0000","last_change":"2024-04-26T21:14:43.243196+0000","last_active":"2024-04-26T21:14:44.237349+0000","last_peered":"2024-04-26T21:14:44.237349+0000","last_clean":"2024-04-26T21:14:44.237349+0000","last_became_active":"2024-04-26T21:14:43.242369+0000","last_became_peered":"2024-04-26T21:14:43.242369+0000","last_unstale":"2024-04-26T21:14:44.237349+0000","last_undegraded":"2024-04-26T21:14:44.237349+0000","last_fullsized":"2024-04-26T21:14:44.237349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:21.948893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226937+0000","last_change":"2024-04-26T21:14:42.226937+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226937+0000","last_undegraded":"2024-04-26T21:14:42.226937+0000","last_fullsized":"2024-04-26T21:14:42.226937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.252042+0000","last_change":"2024-04-26T21:14:44.247167+0000","last_active":"2024-04-26T21:14:44.252042+0000","last_peered":"2024-04-26T21:14:44.252042+0000","last_clean":"2024-04-26T21:14:44.252042+0000","last_became_active":"2024-04-26T21:14:44.246911+0000","last_became_peered":"2024-04-26T21:14:44.246911+0000","last_unstale":"2024-04-26T21:14:44.252042+0000","last_undegraded":"2024-04-26T21:14:44.252042+0000","last_fullsized":"2024-04-26T21:14:44.252042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:15.155730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.236660+0000","last_change":"2024-04-26T21:14:43.241399+0000","last_active":"2024-04-26T21:14:44.236660+0000","last_peered":"2024-04-26T21:14:44.236660+0000","last_clean":"2024-04-26T21:14:44.236660+0000","last_became_active":"2024-04-26T21:14:43.241031+0000","last_became_peered":"2024-04-26T21:14:43.241031+0000","last_unstale":"2024-04-26T21:14:44.236660+0000","last_undegraded":"2024-04-26T21:14:44.236660+0000","last_fullsized":"2024-04-26T21:14:44.236660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:41.265024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227167+0000","last_change":"2024-04-26T21:14:42.227167+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227167+0000","last_undegraded":"2024-04-26T21:14:42.227167+0000","last_fullsized":"2024-04-26T21:14:42.227167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231788+0000","last_change":"2024-04-26T21:14:42.231788+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231788+0000","last_undegraded":"2024-04-26T21:14:42.231788+0000","last_fullsized":"2024-04-26T21:14:42.231788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T21:14:42.230938+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:42.230938+0000","last_peered":"2024-04-26T21:14:42.230938+0000","last_clean":"2024-04-26T21:14:42.230938+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:42.230938+0000","last_undegraded":"2024-04-26T21:14:42.230938+0000","last_fullsized":"2024-04-26T21:14:42.230938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":853987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":282568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-04-26T21:14:47.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-04-26T21:14:47.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-04-26T21:14:47.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.230030+0000', 'last_change': '2024-04-26T21:14:42.230030+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.230030+0000', 'last_undegraded': '2024-04-26T21:14:42.230030+0000', 'last_fullsized': '2024-04-26T21:14:42.230030+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-26T21:14:47.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227716+0000', 'last_change': '2024-04-26T21:14:42.227716+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227716+0000', 'last_undegraded': '2024-04-26T21:14:42.227716+0000', 'last_fullsized': '2024-04-26T21:14:42.227716+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.229537+0000', 'last_change': '2024-04-26T21:14:42.229537+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.229537+0000', 'last_undegraded': '2024-04-26T21:14:42.229537+0000', 'last_fullsized': '2024-04-26T21:14:42.229537+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.228944+0000', 'last_change': '2024-04-26T21:14:42.228944+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.228944+0000', 'last_undegraded': '2024-04-26T21:14:42.228944+0000', 'last_fullsized': '2024-04-26T21:14:42.228944+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-26T21:14:47.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.226888+0000', 'last_change': '2024-04-26T21:14:42.226888+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.226888+0000', 'last_undegraded': '2024-04-26T21:14:42.226888+0000', 'last_fullsized': '2024-04-26T21:14:42.226888+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.229534+0000', 'last_change': '2024-04-26T21:14:42.229534+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.229534+0000', 'last_undegraded': '2024-04-26T21:14:42.229534+0000', 'last_fullsized': '2024-04-26T21:14:42.229534+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.228273+0000', 'last_change': '2024-04-26T21:14:42.228273+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.228273+0000', 'last_undegraded': '2024-04-26T21:14:42.228273+0000', 'last_fullsized': '2024-04-26T21:14:42.228273+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-26T21:14:47.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227015+0000', 'last_change': '2024-04-26T21:14:42.227015+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227015+0000', 'last_undegraded': '2024-04-26T21:14:42.227015+0000', 'last_fullsized': '2024-04-26T21:14:42.227015+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.230387+0000', 'last_change': '2024-04-26T21:14:42.230387+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.230387+0000', 'last_undegraded': '2024-04-26T21:14:42.230387+0000', 'last_fullsized': '2024-04-26T21:14:42.230387+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227230+0000', 'last_change': '2024-04-26T21:14:42.227230+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227230+0000', 'last_undegraded': '2024-04-26T21:14:42.227230+0000', 'last_fullsized': '2024-04-26T21:14:42.227230+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.230433+0000', 'last_change': '2024-04-26T21:14:42.230433+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.230433+0000', 'last_undegraded': '2024-04-26T21:14:42.230433+0000', 'last_fullsized': '2024-04-26T21:14:42.230433+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-26T21:14:47.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.226951+0000', 'last_change': '2024-04-26T21:14:42.226951+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.226951+0000', 'last_undegraded': '2024-04-26T21:14:42.226951+0000', 'last_fullsized': '2024-04-26T21:14:42.226951+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227714+0000', 'last_change': '2024-04-26T21:14:42.227714+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227714+0000', 'last_undegraded': '2024-04-26T21:14:42.227714+0000', 'last_fullsized': '2024-04-26T21:14:42.227714+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.231977+0000', 'last_change': '2024-04-26T21:14:42.231977+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.231977+0000', 'last_undegraded': '2024-04-26T21:14:42.231977+0000', 'last_fullsized': '2024-04-26T21:14:42.231977+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:14:47.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227763+0000', 'last_change': '2024-04-26T21:14:42.227763+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227763+0000', 'last_undegraded': '2024-04-26T21:14:42.227763+0000', 'last_fullsized': '2024-04-26T21:14:42.227763+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.230480+0000', 'last_change': '2024-04-26T21:14:42.230480+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.230480+0000', 'last_undegraded': '2024-04-26T21:14:42.230480+0000', 'last_fullsized': '2024-04-26T21:14:42.230480+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227179+0000', 'last_change': '2024-04-26T21:14:42.227179+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227179+0000', 'last_undegraded': '2024-04-26T21:14:42.227179+0000', 'last_fullsized': '2024-04-26T21:14:42.227179+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.230093+0000', 'last_change': '2024-04-26T21:14:42.230093+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.230093+0000', 'last_undegraded': '2024-04-26T21:14:42.230093+0000', 'last_fullsized': '2024-04-26T21:14:42.230093+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.228061+0000', 'last_change': '2024-04-26T21:14:42.228061+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.228061+0000', 'last_undegraded': '2024-04-26T21:14:42.228061+0000', 'last_fullsized': '2024-04-26T21:14:42.228061+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-26T21:14:47.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.229662+0000', 'last_change': '2024-04-26T21:14:42.229662+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.229662+0000', 'last_undegraded': '2024-04-26T21:14:42.229662+0000', 'last_fullsized': '2024-04-26T21:14:42.229662+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227230+0000', 'last_change': '2024-04-26T21:14:42.227230+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227230+0000', 'last_undegraded': '2024-04-26T21:14:42.227230+0000', 'last_fullsized': '2024-04-26T21:14:42.227230+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227718+0000', 'last_change': '2024-04-26T21:14:42.227718+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227718+0000', 'last_undegraded': '2024-04-26T21:14:42.227718+0000', 'last_fullsized': '2024-04-26T21:14:42.227718+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-26T21:14:47.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.229536+0000', 'last_change': '2024-04-26T21:14:42.229536+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.229536+0000', 'last_undegraded': '2024-04-26T21:14:42.229536+0000', 'last_fullsized': '2024-04-26T21:14:42.229536+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.231868+0000', 'last_change': '2024-04-26T21:14:42.231868+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.231868+0000', 'last_undegraded': '2024-04-26T21:14:42.231868+0000', 'last_fullsized': '2024-04-26T21:14:42.231868+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227738+0000', 'last_change': '2024-04-26T21:14:42.227738+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227738+0000', 'last_undegraded': '2024-04-26T21:14:42.227738+0000', 'last_fullsized': '2024-04-26T21:14:42.227738+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T21:14:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.226937+0000', 'last_change': '2024-04-26T21:14:42.226937+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.226937+0000', 'last_undegraded': '2024-04-26T21:14:42.226937+0000', 'last_fullsized': '2024-04-26T21:14:42.226937+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.227167+0000', 'last_change': '2024-04-26T21:14:42.227167+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.227167+0000', 'last_undegraded': '2024-04-26T21:14:42.227167+0000', 'last_fullsized': '2024-04-26T21:14:42.227167+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:14:43.220241+0000', 'last_change': '2024-04-26T21:14:43.220241+0000', 'last_active': '2024-04-26T21:14:43.220241+0000', 'last_peered': '2024-04-26T21:14:43.220241+0000', 'last_clean': '2024-04-26T21:14:43.220241+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:43.220241+0000', 'last_undegraded': '2024-04-26T21:14:43.220241+0000', 'last_fullsized': '2024-04-26T21:14:43.220241+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:43.220241+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:14:47.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-26T21:14:47.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:14:42.231788+0000', 'last_change': '2024-04-26T21:14:42.231788+0000', 'last_active': '2024-04-26T21:14:42.210388+0000', 'last_peered': '2024-04-26T21:14:42.210388+0000', 'last_clean': '2024-04-26T21:14:42.210388+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:14:42.231788+0000', 'last_undegraded': '2024-04-26T21:14:42.231788+0000', 'last_fullsized': '2024-04-26T21:14:42.231788+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'last_clean_scrub_stamp': '2024-04-26T21:14:42.210388+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T21:14:47.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:14:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.097+0000 7f4373cda640 1 Processor -- start 2024-04-26T21:14:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.097+0000 7f4373cda640 1 -- start start 2024-04-26T21:14:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4373cda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f436c0d1b80 0x7f436c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4373cda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436c0d0220 con 0x7f436c0cfda0 2024-04-26T21:14:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4373cda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436c0d0500 con 0x7f436c0b60a0 2024-04-26T21:14:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4373cda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f436c0d07e0 con 0x7f436c0d1b80 2024-04-26T21:14:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4371275640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f436c0cfda0 0x7f436c0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57676/0 (socket says 172.21.15.40:57676) 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4371275640 1 -- 172.21.15.40:0/334726690 learned_addr learned my addr 172.21.15.40:0/334726690 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1017274177 0 0) 0x7f436c0d0220 con 0x7f436c0cfda0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4358003520 con 0x7f436c0cfda0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4371a76640 1 --2- 172.21.15.40:0/334726690 >> v2:172.21.15.184:3300/0 conn(0x7f436c0d1b80 0x7f436c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 446468513 0 0) 0x7f4358003520 con 0x7f436c0cfda0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 >> v2:172.21.15.184:3300/0 conn(0x7f436c0d1b80 msgr2=0x7f436c0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 --2- 172.21.15.40:0/334726690 >> v2:172.21.15.184:3300/0 conn(0x7f436c0d1b80 0x7f436c0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 >> v1:172.21.15.184:6789/0 conn(0x7f436c0b60a0 legacy=0x7f436c0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c0d0ac0 con 0x7f436c0cfda0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f435c001670 con 0x7f436c0cfda0 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 >> v1:172.21.15.40:6789/0 conn(0x7f436c0cfda0 legacy=0x7f436c0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43580039b0 0x7f4358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4358004800 0x7f4358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 0x7f435800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0d07e0 con 0x7f43580051c0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f435800c060 con 0x7f4358004800 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.098+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f435800c260 con 0x7f43580039b0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 0x7f435800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4371275640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4358004800 0x7f4358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4371a76640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43580039b0 0x7f4358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 -- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43580039b0 msgr2=0x7f4358003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43580039b0 0x7f4358003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 -- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4358004800 msgr2=0x7f4358004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4358004800 0x7f4358004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 -- 172.21.15.40:0/334726690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4358003520 con 0x7f43580051c0 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4371a76640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43580039b0 0x7f4358003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 -- 172.21.15.40:0/334726690 reap_dead start 2024-04-26T21:14:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.099+0000 7f4372277640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 0x7f435800b9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4364001e50 tx=0x7f4358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170142a84b79d775 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364001a70 con 0x7f43580051c0 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364003970 con 0x7f43580051c0 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4370a74640 1 -- 172.21.15.40:0/334726690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364003d60 con 0x7f43580051c0 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 -- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 msgr2=0x7f435800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 0x7f435800b9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4364001e50 tx=0x7f4358005830 comp rx=0 tx=0).stop 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 -- 172.21.15.40:0/334726690 shutdown_connections 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 --2- 172.21.15.40:0/334726690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580051c0 0x7f435800b9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 -- 172.21.15.40:0/334726690 >> 172.21.15.40:0/334726690 conn(0x7f436c09e540 msgr2=0x7f436c0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 -- 172.21.15.40:0/334726690 shutdown_connections 2024-04-26T21:14:48.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.100+0000 7f4373cda640 1 -- 172.21.15.40:0/334726690 wait complete. 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.101+0000 7f4373cda640 1 Processor -- start 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.101+0000 7f4373cda640 1 -- start start 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 0x7f436c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 0x7f436c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0d20e0 con 0x7f436c1618f0 2024-04-26T21:14:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0d1f60 con 0x7f43580051c0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4373cda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f436c0d2260 con 0x7f436c0a0010 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4371275640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4371275640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55906/0 (socket says 172.21.15.40:55906) 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4371275640 1 -- 172.21.15.40:0/1908412434 learned_addr learned my addr 172.21.15.40:0/1908412434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4371a76640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 0x7f436c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.102+0000 7f4372277640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 0x7f436c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 -- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 msgr2=0x7f436c0a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 0x7f436c0a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 -- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 msgr2=0x7f436c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 0x7f436c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 -- 172.21.15.40:0/1908412434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c1f6290 con 0x7f436c1618f0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371a76640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 0x7f436c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4372277640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 0x7f436c0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4371275640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f435c0072c0 tx=0x7f435c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00f960 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c018070 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c0a0590 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00fd00 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.103+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c0a0b50 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.104+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c06a290 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.104+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f435c01d460 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.105+0000 7f43627fc640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.105+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f435c08bb40 con 0x7f436c1618f0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.105+0000 7f4371a76640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.106+0000 7f4371a76640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 0x7f435006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4368006f80 tx=0x7f4368008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.109+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f436c1f6290 con 0x7f436c1618f0 2024-04-26T21:14:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.367+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f436c09fdf0 con 0x7f436c1618f0 2024-04-26T21:14:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.367+0000 7f43627fc640 1 -- 172.21.15.40:0/1908412434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1590 (secure 0 0 0) 0x7f435c0599b0 con 0x7f436c1618f0 2024-04-26T21:14:48.368 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:48.368 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":191,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":58},{"state_name":"creating+peering","count":28},{"state_name":"active+clean","count":11}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":240619520,"bytes_avail":772853493760,"bytes_total":773094113280,"unknown_pgs_ratio":0.59793812036514282,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":29,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6024}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-26T21:14:26.952511+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 msgr2=0x7f435006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 0x7f435006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4368006f80 tx=0x7f4368008040 comp rx=0 tx=0).stop 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 msgr2=0x7f436c0a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f435c0072c0 tx=0x7f435c007780 comp rx=0 tx=0).stop 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 shutdown_connections 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f436c0a0010 0x7f436c0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f436c1618f0 0x7f436c0a1fa0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 --2- 172.21.15.40:0/1908412434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43580051c0 0x7f436c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 >> 172.21.15.40:0/1908412434 conn(0x7f436c09e540 msgr2=0x7f436c0ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 shutdown_connections 2024-04-26T21:14:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.369+0000 7f4373cda640 1 -- 172.21.15.40:0/1908412434 wait complete. 2024-04-26T21:14:48.381 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:14:48.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.480+0000 7f5cc7274640 1 Processor -- start 2024-04-26T21:14:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.480+0000 7f5cc7274640 1 -- start start 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc7274640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5cc00d3ee0 0x7f5cc00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc7274640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00d03a0 con 0x7f5cc015f750 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc7274640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc00d0680 con 0x7f5cc015f370 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc7274640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc00d0960 con 0x7f5cc00d3ee0 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc5811640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5cc015f750 0x7f5cc00b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57688/0 (socket says 172.21.15.40:57688) 2024-04-26T21:14:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.481+0000 7f5cc5811640 1 -- 172.21.15.40:0/3796240796 learned_addr learned my addr 172.21.15.40:0/3796240796 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> v2:172.21.15.184:3300/0 conn(0x7f5cc00d3ee0 0x7f5cc00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831186382 0 0) 0x7f5cc00d03a0 con 0x7f5cc015f750 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca8003520 con 0x7f5cc015f750 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167350285 0 0) 0x7f5cc00d0680 con 0x7f5cc015f370 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cc00d03a0 con 0x7f5cc015f370 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 735336296 0 0) 0x7f5ca8003520 con 0x7f5cc015f750 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 >> v2:172.21.15.184:3300/0 conn(0x7f5cc00d3ee0 msgr2=0x7f5cc00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 --2- 172.21.15.40:0/3796240796 >> v2:172.21.15.184:3300/0 conn(0x7f5cc00d3ee0 0x7f5cc00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 >> v1:172.21.15.184:6789/0 conn(0x7f5cc015f370 legacy=0x7f5cc00d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc00d0c40 con 0x7f5cc015f750 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5cb00019f0 con 0x7f5cc015f750 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> v2:172.21.15.184:3300/0 conn(0x7f5cc00d3ee0 0x7f5cc00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 >> v1:172.21.15.40:6789/0 conn(0x7f5cc015f750 legacy=0x7f5cc00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8003b40 0x7f5ca8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00d0960 con 0x7f5ca8004800 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca800c040 con 0x7f5ca80051c0 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca800c240 con 0x7f5ca8003b40 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cc5010640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8003b40 0x7f5ca8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.482+0000 7f5cc5811640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 -- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8003b40 msgr2=0x7f5ca8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8003b40 0x7f5ca8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 -- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca80051c0 msgr2=0x7f5ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 -- 172.21.15.40:0/3796240796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca8003520 con 0x7f5ca8004800 2024-04-26T21:14:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc5811640 1 -- 172.21.15.40:0/3796240796 reap_dead start 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5cc008e680 tx=0x7f5ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b34e26ecd2a59c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4036070 con 0x7f5ca8004800 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb4031400 con 0x7f5ca8004800 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.483+0000 7f5cbffff640 1 -- 172.21.15.40:0/3796240796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4009860 con 0x7f5ca8004800 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 -- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 msgr2=0x7f5ca8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5cc008e680 tx=0x7f5ca8005830 comp rx=0 tx=0).stop 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 -- 172.21.15.40:0/3796240796 shutdown_connections 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3796240796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 -- 172.21.15.40:0/3796240796 >> 172.21.15.40:0/3796240796 conn(0x7f5cc008ec10 msgr2=0x7f5cc009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 -- 172.21.15.40:0/3796240796 shutdown_connections 2024-04-26T21:14:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.484+0000 7f5cc7274640 1 -- 172.21.15.40:0/3796240796 wait complete. 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.485+0000 7f5cc7274640 1 Processor -- start 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.485+0000 7f5cc7274640 1 -- start start 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8004800 0x7f5cc00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cc00b9800 0x7f5cc00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00cfef0 con 0x7f5cc00c9500 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc00cfd70 con 0x7f5cc00b9800 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc7274640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc00d0070 con 0x7f5ca8004800 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc5811640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc5811640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55922/0 (socket says 172.21.15.40:55922) 2024-04-26T21:14:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc5811640 1 -- 172.21.15.40:0/3177955528 learned_addr learned my addr 172.21.15.40:0/3177955528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc5010640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8004800 0x7f5cc00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.486+0000 7f5cc480f640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cc00b9800 0x7f5cc00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 -- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8004800 msgr2=0x7f5cc00cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8004800 0x7f5cc00cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 -- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cc00b9800 msgr2=0x7f5cc00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cc00b9800 0x7f5cc00c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 -- 172.21.15.40:0/3177955528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc0158b10 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc5811640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00b7cf0 tx=0x7f5cb0007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000eb00 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb0018070 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc00ba860 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0013420 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.487+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc00b9fc0 con 0x7f5cc00c9500 2024-04-26T21:14:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.488+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc0160c30 con 0x7f5cc00c9500 2024-04-26T21:14:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.488+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cb00135c0 con 0x7f5cc00c9500 2024-04-26T21:14:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.489+0000 7f5cbdffb640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 0x7f5cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.489+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5cb008bdd0 con 0x7f5cc00c9500 2024-04-26T21:14:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.489+0000 7f5cc5010640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 0x7f5cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.490+0000 7f5cc5010640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 0x7f5cac06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.493+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5cc0160c30 con 0x7f5cc00c9500 2024-04-26T21:14:48.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.675+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cc00d16f0 con 0x7f5cac06c8f0 2024-04-26T21:14:48.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.681+0000 7f5cbdffb640 1 -- 172.21.15.40:0/3177955528 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241091 (secure 0 0 0) 0x7f5cc00d16f0 con 0x7f5cac06c8f0 2024-04-26T21:14:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:48.686 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:48.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 msgr2=0x7f5cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 0x7f5cac06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8009040 comp rx=0 tx=0).stop 2024-04-26T21:14:48.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 msgr2=0x7f5cc00c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00b7cf0 tx=0x7f5cb0007bc0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 shutdown_connections 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cac06c8f0 0x7f5cac06edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cc00c9500 0x7f5cc00c9990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cc00b9800 0x7f5cc00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 --2- 172.21.15.40:0/3177955528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ca8004800 0x7f5cc00cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 >> 172.21.15.40:0/3177955528 conn(0x7f5cc008ec10 msgr2=0x7f5cc00d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 shutdown_connections 2024-04-26T21:14:48.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:48.684+0000 7f5cc7274640 1 -- 172.21.15.40:0/3177955528 wait complete. 2024-04-26T21:14:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-26T21:14:46.956214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234980,"kb_used_data":6336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739740,"statfs":{"total":773094113280,"available":772853493760,"internally_reserved":0,"allocated":6488064,"data_stored":3331905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.730269"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.262358+0000","last_change":"2024-04-26T21:14:44.257792+0000","last_active":"2024-04-26T21:14:44.262358+0000","last_peered":"2024-04-26T21:14:44.262358+0000","last_clean":"2024-04-26T21:14:44.262358+0000","last_became_active":"2024-04-26T21:14:44.257435+0000","last_became_peered":"2024-04-26T21:14:44.257435+0000","last_unstale":"2024-04-26T21:14:44.262358+0000","last_undegraded":"2024-04-26T21:14:44.262358+0000","last_fullsized":"2024-04-26T21:14:44.262358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:46.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.264129+0000","last_change":"2024-04-26T21:14:44.261342+0000","last_active":"2024-04-26T21:14:44.264129+0000","last_peered":"2024-04-26T21:14:44.264129+0000","last_clean":"2024-04-26T21:14:44.264129+0000","last_became_active":"2024-04-26T21:14:44.261052+0000","last_became_peered":"2024-04-26T21:14:44.261052+0000","last_unstale":"2024-04-26T21:14:44.264129+0000","last_undegraded":"2024-04-26T21:14:44.264129+0000","last_fullsized":"2024-04-26T21:14:44.264129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:58.831928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230030+0000","last_change":"2024-04-26T21:14:42.230030+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230030+0000","last_undegraded":"2024-04-26T21:14:42.230030+0000","last_fullsized":"2024-04-26T21:14:42.230030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.259790+0000","last_change":"2024-04-26T21:14:44.256368+0000","last_active":"2024-04-26T21:14:44.259790+0000","last_peered":"2024-04-26T21:14:44.259790+0000","last_clean":"2024-04-26T21:14:44.259790+0000","last_became_active":"2024-04-26T21:14:44.256112+0000","last_became_peered":"2024-04-26T21:14:44.256112+0000","last_unstale":"2024-04-26T21:14:44.259790+0000","last_undegraded":"2024-04-26T21:14:44.259790+0000","last_fullsized":"2024-04-26T21:14:44.259790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:33.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227716+0000","last_change":"2024-04-26T21:14:42.227716+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227716+0000","last_undegraded":"2024-04-26T21:14:42.227716+0000","last_fullsized":"2024-04-26T21:14:42.227716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229537+0000","last_change":"2024-04-26T21:14:42.229537+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229537+0000","last_undegraded":"2024-04-26T21:14:42.229537+0000","last_fullsized":"2024-04-26T21:14:42.229537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228944+0000","last_change":"2024-04-26T21:14:42.228944+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228944+0000","last_undegraded":"2024-04-26T21:14:42.228944+0000","last_fullsized":"2024-04-26T21:14:42.228944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226888+0000","last_change":"2024-04-26T21:14:42.226888+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226888+0000","last_undegraded":"2024-04-26T21:14:42.226888+0000","last_fullsized":"2024-04-26T21:14:42.226888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229534+0000","last_change":"2024-04-26T21:14:42.229534+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229534+0000","last_undegraded":"2024-04-26T21:14:42.229534+0000","last_fullsized":"2024-04-26T21:14:42.229534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228273+0000","last_change":"2024-04-26T21:14:42.228273+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228273+0000","last_undegraded":"2024-04-26T21:14:42.228273+0000","last_fullsized":"2024-04-26T21:14:42.228273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227015+0000","last_change":"2024-04-26T21:14:42.227015+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227015+0000","last_undegraded":"2024-04-26T21:14:42.227015+0000","last_fullsized":"2024-04-26T21:14:42.227015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230387+0000","last_change":"2024-04-26T21:14:42.230387+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230387+0000","last_undegraded":"2024-04-26T21:14:42.230387+0000","last_fullsized":"2024-04-26T21:14:42.230387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266245+0000","last_change":"2024-04-26T21:14:44.263354+0000","last_active":"2024-04-26T21:14:44.266245+0000","last_peered":"2024-04-26T21:14:44.266245+0000","last_clean":"2024-04-26T21:14:44.266245+0000","last_became_active":"2024-04-26T21:14:44.263112+0000","last_became_peered":"2024-04-26T21:14:44.263112+0000","last_unstale":"2024-04-26T21:14:44.266245+0000","last_undegraded":"2024-04-26T21:14:44.266245+0000","last_fullsized":"2024-04-26T21:14:44.266245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:32.834630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230433+0000","last_change":"2024-04-26T21:14:42.230433+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230433+0000","last_undegraded":"2024-04-26T21:14:42.230433+0000","last_fullsized":"2024-04-26T21:14:42.230433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226951+0000","last_change":"2024-04-26T21:14:42.226951+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226951+0000","last_undegraded":"2024-04-26T21:14:42.226951+0000","last_fullsized":"2024-04-26T21:14:42.226951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227714+0000","last_change":"2024-04-26T21:14:42.227714+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227714+0000","last_undegraded":"2024-04-26T21:14:42.227714+0000","last_fullsized":"2024-04-26T21:14:42.227714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231977+0000","last_change":"2024-04-26T21:14:42.231977+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231977+0000","last_undegraded":"2024-04-26T21:14:42.231977+0000","last_fullsized":"2024-04-26T21:14:42.231977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227763+0000","last_change":"2024-04-26T21:14:42.227763+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227763+0000","last_undegraded":"2024-04-26T21:14:42.227763+0000","last_fullsized":"2024-04-26T21:14:42.227763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230480+0000","last_change":"2024-04-26T21:14:42.230480+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230480+0000","last_undegraded":"2024-04-26T21:14:42.230480+0000","last_fullsized":"2024-04-26T21:14:42.230480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227179+0000","last_change":"2024-04-26T21:14:42.227179+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227179+0000","last_undegraded":"2024-04-26T21:14:42.227179+0000","last_fullsized":"2024-04-26T21:14:42.227179+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.230093+0000","last_change":"2024-04-26T21:14:42.230093+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.230093+0000","last_undegraded":"2024-04-26T21:14:42.230093+0000","last_fullsized":"2024-04-26T21:14:42.230093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.228061+0000","last_change":"2024-04-26T21:14:42.228061+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.228061+0000","last_undegraded":"2024-04-26T21:14:42.228061+0000","last_fullsized":"2024-04-26T21:14:42.228061+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229662+0000","last_change":"2024-04-26T21:14:42.229662+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229662+0000","last_undegraded":"2024-04-26T21:14:42.229662+0000","last_fullsized":"2024-04-26T21:14:42.229662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227230+0000","last_change":"2024-04-26T21:14:42.227230+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227230+0000","last_undegraded":"2024-04-26T21:14:42.227230+0000","last_fullsized":"2024-04-26T21:14:42.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237869+0000","last_change":"2024-04-26T21:14:43.241349+0000","last_active":"2024-04-26T21:14:44.237869+0000","last_peered":"2024-04-26T21:14:44.237869+0000","last_clean":"2024-04-26T21:14:44.237869+0000","last_became_active":"2024-04-26T21:14:43.241055+0000","last_became_peered":"2024-04-26T21:14:43.241055+0000","last_unstale":"2024-04-26T21:14:44.237869+0000","last_undegraded":"2024-04-26T21:14:44.237869+0000","last_fullsized":"2024-04-26T21:14:44.237869+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.604628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227718+0000","last_change":"2024-04-26T21:14:42.227718+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227718+0000","last_undegraded":"2024-04-26T21:14:42.227718+0000","last_fullsized":"2024-04-26T21:14:42.227718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.263556+0000","last_change":"2024-04-26T21:14:43.247928+0000","last_active":"2024-04-26T21:14:44.263556+0000","last_peered":"2024-04-26T21:14:44.263556+0000","last_clean":"2024-04-26T21:14:44.263556+0000","last_became_active":"2024-04-26T21:14:43.247549+0000","last_became_peered":"2024-04-26T21:14:43.247549+0000","last_unstale":"2024-04-26T21:14:44.263556+0000","last_undegraded":"2024-04-26T21:14:44.263556+0000","last_fullsized":"2024-04-26T21:14:44.263556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:27.702426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.229536+0000","last_change":"2024-04-26T21:14:42.229536+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.229536+0000","last_undegraded":"2024-04-26T21:14:42.229536+0000","last_fullsized":"2024-04-26T21:14:42.229536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.266261+0000","last_change":"2024-04-26T21:14:44.263566+0000","last_active":"2024-04-26T21:14:44.266261+0000","last_peered":"2024-04-26T21:14:44.266261+0000","last_clean":"2024-04-26T21:14:44.266261+0000","last_became_active":"2024-04-26T21:14:44.263259+0000","last_became_peered":"2024-04-26T21:14:44.263259+0000","last_unstale":"2024-04-26T21:14:44.266261+0000","last_undegraded":"2024-04-26T21:14:44.266261+0000","last_fullsized":"2024-04-26T21:14:44.266261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:00.527919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231868+0000","last_change":"2024-04-26T21:14:42.231868+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231868+0000","last_undegraded":"2024-04-26T21:14:42.231868+0000","last_fullsized":"2024-04-26T21:14:42.231868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227738+0000","last_change":"2024-04-26T21:14:42.227738+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227738+0000","last_undegraded":"2024-04-26T21:14:42.227738+0000","last_fullsized":"2024-04-26T21:14:42.227738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.237349+0000","last_change":"2024-04-26T21:14:43.243196+0000","last_active":"2024-04-26T21:14:44.237349+0000","last_peered":"2024-04-26T21:14:44.237349+0000","last_clean":"2024-04-26T21:14:44.237349+0000","last_became_active":"2024-04-26T21:14:43.242369+0000","last_became_peered":"2024-04-26T21:14:43.242369+0000","last_unstale":"2024-04-26T21:14:44.237349+0000","last_undegraded":"2024-04-26T21:14:44.237349+0000","last_fullsized":"2024-04-26T21:14:44.237349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:21.948893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.226937+0000","last_change":"2024-04-26T21:14:42.226937+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.226937+0000","last_undegraded":"2024-04-26T21:14:42.226937+0000","last_fullsized":"2024-04-26T21:14:42.226937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.252042+0000","last_change":"2024-04-26T21:14:44.247167+0000","last_active":"2024-04-26T21:14:44.252042+0000","last_peered":"2024-04-26T21:14:44.252042+0000","last_clean":"2024-04-26T21:14:44.252042+0000","last_became_active":"2024-04-26T21:14:44.246911+0000","last_became_peered":"2024-04-26T21:14:44.246911+0000","last_unstale":"2024-04-26T21:14:44.252042+0000","last_undegraded":"2024-04-26T21:14:44.252042+0000","last_fullsized":"2024-04-26T21:14:44.252042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:39:15.155730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T21:14:44.236660+0000","last_change":"2024-04-26T21:14:43.241399+0000","last_active":"2024-04-26T21:14:44.236660+0000","last_peered":"2024-04-26T21:14:44.236660+0000","last_clean":"2024-04-26T21:14:44.236660+0000","last_became_active":"2024-04-26T21:14:43.241031+0000","last_became_peered":"2024-04-26T21:14:43.241031+0000","last_unstale":"2024-04-26T21:14:44.236660+0000","last_undegraded":"2024-04-26T21:14:44.236660+0000","last_fullsized":"2024-04-26T21:14:44.236660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:41.265024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.227167+0000","last_change":"2024-04-26T21:14:42.227167+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.227167+0000","last_undegraded":"2024-04-26T21:14:42.227167+0000","last_fullsized":"2024-04-26T21:14:42.227167+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:14:43.220241+0000","last_change":"2024-04-26T21:14:43.220241+0000","last_active":"2024-04-26T21:14:43.220241+0000","last_peered":"2024-04-26T21:14:43.220241+0000","last_clean":"2024-04-26T21:14:43.220241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:43.220241+0000","last_undegraded":"2024-04-26T21:14:43.220241+0000","last_fullsized":"2024-04-26T21:14:43.220241+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-26T21:14:42.231788+0000","last_change":"2024-04-26T21:14:42.231788+0000","last_active":"2024-04-26T21:14:42.210388+0000","last_peered":"2024-04-26T21:14:42.210388+0000","last_clean":"2024-04-26T21:14:42.210388+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:14:42.231788+0000","last_undegraded":"2024-04-26T21:14:42.231788+0000","last_fullsized":"2024-04-26T21:14:42.231788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T21:14:42.230938+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:42.230938+0000","last_peered":"2024-04-26T21:14:42.230938+0000","last_clean":"2024-04-26T21:14:42.230938+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:42.230938+0000","last_undegraded":"2024-04-26T21:14:42.230938+0000","last_fullsized":"2024-04-26T21:14:42.230938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":269436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":263619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":853987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":282568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:49.189 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:14:49.188+0000 7f11e461d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:14:51.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:14:51.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.813+0000 7fa59aecf640 1 Processor -- start 2024-04-26T21:14:51.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 -- start start 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5940cf2e0 0x7fa59406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5940d11b0 0x7fa59406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa59406e2c0 0x7fa59415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa59406e750 con 0x7fa5940d11b0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa5940d1740 con 0x7fa5940cf2e0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.814+0000 7fa59aecf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5940d1a20 con 0x7fa59406e2c0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5940d11b0 0x7fa59406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa598c6b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa5940cf2e0 0x7fa59406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa59946c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa59406e2c0 0x7fa59415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa5940d11b0 0x7fa59406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51928/0 (socket says 172.21.15.40:51928) 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 -- 172.21.15.40:0/2418924431 learned_addr learned my addr 172.21.15.40:0/2418924431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 -- 172.21.15.40:0/2418924431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa59406e2c0 msgr2=0x7fa59415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 --2- 172.21.15.40:0/2418924431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa59406e2c0 0x7fa59415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 -- 172.21.15.40:0/2418924431 >> v2:172.21.15.184:3300/0 conn(0x7fa5940cf2e0 msgr2=0x7fa59406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 --2- 172.21.15.40:0/2418924431 >> v2:172.21.15.184:3300/0 conn(0x7fa5940cf2e0 0x7fa59406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.815+0000 7fa593fff640 1 -- 172.21.15.40:0/2418924431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5940d1d00 con 0x7fa5940d11b0 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa593fff640 1 --2- 172.21.15.40:0/2418924431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5940d11b0 0x7fa59406dd80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa580009870 tx=0x7fa58002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb51759a34bd9007 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa5937fe640 1 -- 172.21.15.40:0/2418924431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580037070 con 0x7fa5940d11b0 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa5937fe640 1 -- 172.21.15.40:0/2418924431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa580032440 con 0x7fa5940d11b0 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa5937fe640 1 -- 172.21.15.40:0/2418924431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580032810 con 0x7fa5940d11b0 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa59aecf640 1 -- 172.21.15.40:0/2418924431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5940d11b0 msgr2=0x7fa59406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.816+0000 7fa59aecf640 1 --2- 172.21.15.40:0/2418924431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5940d11b0 0x7fa59406dd80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa580009870 tx=0x7fa58002a210 comp rx=0 tx=0).stop 2024-04-26T21:14:51.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 -- 172.21.15.40:0/2418924431 shutdown_connections 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 --2- 172.21.15.40:0/2418924431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa59406e2c0 0x7fa59415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 --2- 172.21.15.40:0/2418924431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5940d11b0 0x7fa59406dd80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 --2- 172.21.15.40:0/2418924431 >> v2:172.21.15.184:3300/0 conn(0x7fa5940cf2e0 0x7fa59406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 -- 172.21.15.40:0/2418924431 >> 172.21.15.40:0/2418924431 conn(0x7fa5940b4a60 msgr2=0x7fa5940b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 -- 172.21.15.40:0/2418924431 shutdown_connections 2024-04-26T21:14:51.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.817+0000 7fa59aecf640 1 -- 172.21.15.40:0/2418924431 wait complete. 2024-04-26T21:14:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.818+0000 7fa59aecf640 1 Processor -- start 2024-04-26T21:14:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.818+0000 7fa59aecf640 1 -- start start 2024-04-26T21:14:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 0x7fa59409f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 0x7fa59409fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa59406da10 con 0x7fa59406e2c0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5940d22c0 con 0x7fa5940cf2e0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59aecf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5940cf6c0 con 0x7fa5940d11b0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51934/0 (socket says 172.21.15.40:51934) 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 -- 172.21.15.40:0/3783769914 learned_addr learned my addr 172.21.15.40:0/3783769914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59946c640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 0x7fa59409fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa593fff640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 0x7fa59409f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 -- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 msgr2=0x7fa59409fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 0x7fa59409fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 -- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 msgr2=0x7fa59409f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 0x7fa59409f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:51.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 -- 172.21.15.40:0/3783769914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59409e1c0 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa59946c640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 0x7fa59409fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa593fff640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 0x7fa59409f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa598c6b640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9e0 tx=0x7fa58800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588017070 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.819+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58800c9d0 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.820+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59409e4a0 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.820+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58800cdc0 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.820+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59409ea60 con 0x7fa59406e2c0 2024-04-26T21:14:51.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.820+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59406a290 con 0x7fa59406e2c0 2024-04-26T21:14:51.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.821+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa58801a070 con 0x7fa59406e2c0 2024-04-26T21:14:51.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.821+0000 7fa5917fa640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 0x7fa57806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:51.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.822+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa58808be90 con 0x7fa59406e2c0 2024-04-26T21:14:51.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.822+0000 7fa593fff640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 0x7fa57806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:51.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.822+0000 7fa593fff640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 0x7fa57806edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa580034000 tx=0x7fa58002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:51.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:51.826+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa58805a2b0 con 0x7fa59406e2c0 2024-04-26T21:14:52.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.005+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa594161b00 con 0x7fa57806c8f0 2024-04-26T21:14:52.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.012+0000 7fa5917fa640 1 -- 172.21.15.40:0/3783769914 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7fa594161b00 con 0x7fa57806c8f0 2024-04-26T21:14:52.016 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:52.017 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:14:52.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 msgr2=0x7fa57806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 0x7fa57806edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa580034000 tx=0x7fa58002e000 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 msgr2=0x7fa594157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9e0 tx=0x7fa58800eea0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 shutdown_connections 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa57806c8f0 0x7fa57806edb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5940d11b0 0x7fa59409fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5940cf2e0 0x7fa59409f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 --2- 172.21.15.40:0/3783769914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa59406e2c0 0x7fa594157f10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 >> 172.21.15.40:0/3783769914 conn(0x7fa5940b4a60 msgr2=0x7fa594160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 shutdown_connections 2024-04-26T21:14:52.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.015+0000 7fa59aecf640 1 -- 172.21.15.40:0/3783769914 wait complete. 2024-04-26T21:14:52.027 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-26T21:14:50.957444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235368,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739352,"statfs":{"total":773094113280,"available":772853096448,"internally_reserved":0,"allocated":6856704,"data_stored":3495780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.729208"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601505+0000","last_change":"2024-04-26T21:14:44.252210+0000","last_active":"2024-04-26T21:14:45.601505+0000","last_peered":"2024-04-26T21:14:45.601505+0000","last_clean":"2024-04-26T21:14:45.601505+0000","last_became_active":"2024-04-26T21:14:44.251938+0000","last_became_peered":"2024-04-26T21:14:44.251938+0000","last_unstale":"2024-04-26T21:14:45.601505+0000","last_undegraded":"2024-04-26T21:14:45.601505+0000","last_fullsized":"2024-04-26T21:14:45.601505+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:21.390018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602999+0000","last_change":"2024-04-26T21:14:44.259640+0000","last_active":"2024-04-26T21:14:45.602999+0000","last_peered":"2024-04-26T21:14:45.602999+0000","last_clean":"2024-04-26T21:14:45.602999+0000","last_became_active":"2024-04-26T21:14:44.259338+0000","last_became_peered":"2024-04-26T21:14:44.259338+0000","last_unstale":"2024-04-26T21:14:45.602999+0000","last_undegraded":"2024-04-26T21:14:45.602999+0000","last_fullsized":"2024-04-26T21:14:45.602999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:05.685217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298115+0000","last_change":"2024-04-26T21:14:44.251585+0000","last_active":"2024-04-26T21:14:45.298115+0000","last_peered":"2024-04-26T21:14:45.298115+0000","last_clean":"2024-04-26T21:14:45.298115+0000","last_became_active":"2024-04-26T21:14:44.251311+0000","last_became_peered":"2024-04-26T21:14:44.251311+0000","last_unstale":"2024-04-26T21:14:45.298115+0000","last_undegraded":"2024-04-26T21:14:45.298115+0000","last_fullsized":"2024-04-26T21:14:45.298115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:44.482704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245053+0000","last_change":"2024-04-26T21:14:44.245085+0000","last_active":"2024-04-26T21:14:45.245053+0000","last_peered":"2024-04-26T21:14:45.245053+0000","last_clean":"2024-04-26T21:14:45.245053+0000","last_became_active":"2024-04-26T21:14:44.244828+0000","last_became_peered":"2024-04-26T21:14:44.244828+0000","last_unstale":"2024-04-26T21:14:45.245053+0000","last_undegraded":"2024-04-26T21:14:45.245053+0000","last_fullsized":"2024-04-26T21:14:45.245053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:14.343342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601328+0000","last_change":"2024-04-26T21:14:44.244385+0000","last_active":"2024-04-26T21:14:45.601328+0000","last_peered":"2024-04-26T21:14:45.601328+0000","last_clean":"2024-04-26T21:14:45.601328+0000","last_became_active":"2024-04-26T21:14:44.244106+0000","last_became_peered":"2024-04-26T21:14:44.244106+0000","last_unstale":"2024-04-26T21:14:45.601328+0000","last_undegraded":"2024-04-26T21:14:45.601328+0000","last_fullsized":"2024-04-26T21:14:45.601328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:38.090255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602190+0000","last_change":"2024-04-26T21:14:44.244508+0000","last_active":"2024-04-26T21:14:45.602190+0000","last_peered":"2024-04-26T21:14:45.602190+0000","last_clean":"2024-04-26T21:14:45.602190+0000","last_became_active":"2024-04-26T21:14:44.243507+0000","last_became_peered":"2024-04-26T21:14:44.243507+0000","last_unstale":"2024-04-26T21:14:45.602190+0000","last_undegraded":"2024-04-26T21:14:45.602190+0000","last_fullsized":"2024-04-26T21:14:45.602190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:12.604409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245695+0000","last_change":"2024-04-26T21:14:44.242098+0000","last_active":"2024-04-26T21:14:45.245695+0000","last_peered":"2024-04-26T21:14:45.245695+0000","last_clean":"2024-04-26T21:14:45.245695+0000","last_became_active":"2024-04-26T21:14:44.241415+0000","last_became_peered":"2024-04-26T21:14:44.241415+0000","last_unstale":"2024-04-26T21:14:45.245695+0000","last_undegraded":"2024-04-26T21:14:45.245695+0000","last_fullsized":"2024-04-26T21:14:45.245695+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:31.622757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602383+0000","last_change":"2024-04-26T21:14:44.255379+0000","last_active":"2024-04-26T21:14:45.602383+0000","last_peered":"2024-04-26T21:14:45.602383+0000","last_clean":"2024-04-26T21:14:45.602383+0000","last_became_active":"2024-04-26T21:14:44.255085+0000","last_became_peered":"2024-04-26T21:14:44.255085+0000","last_unstale":"2024-04-26T21:14:45.602383+0000","last_undegraded":"2024-04-26T21:14:45.602383+0000","last_fullsized":"2024-04-26T21:14:45.602383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:06.165026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.246178+0000","last_change":"2024-04-26T21:14:44.243662+0000","last_active":"2024-04-26T21:14:45.246178+0000","last_peered":"2024-04-26T21:14:45.246178+0000","last_clean":"2024-04-26T21:14:45.246178+0000","last_became_active":"2024-04-26T21:14:44.243367+0000","last_became_peered":"2024-04-26T21:14:44.243367+0000","last_unstale":"2024-04-26T21:14:45.246178+0000","last_undegraded":"2024-04-26T21:14:45.246178+0000","last_fullsized":"2024-04-26T21:14:45.246178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:26.071658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601831+0000","last_change":"2024-04-26T21:14:44.257792+0000","last_active":"2024-04-26T21:14:45.601831+0000","last_peered":"2024-04-26T21:14:45.601831+0000","last_clean":"2024-04-26T21:14:45.601831+0000","last_became_active":"2024-04-26T21:14:44.257435+0000","last_became_peered":"2024-04-26T21:14:44.257435+0000","last_unstale":"2024-04-26T21:14:45.601831+0000","last_undegraded":"2024-04-26T21:14:45.601831+0000","last_fullsized":"2024-04-26T21:14:45.601831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:25.522176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601567+0000","last_change":"2024-04-26T21:14:44.261342+0000","last_active":"2024-04-26T21:14:45.601567+0000","last_peered":"2024-04-26T21:14:45.601567+0000","last_clean":"2024-04-26T21:14:45.601567+0000","last_became_active":"2024-04-26T21:14:44.261052+0000","last_became_peered":"2024-04-26T21:14:44.261052+0000","last_unstale":"2024-04-26T21:14:45.601567+0000","last_undegraded":"2024-04-26T21:14:45.601567+0000","last_fullsized":"2024-04-26T21:14:45.601567+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:32.995263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245299+0000","last_change":"2024-04-26T21:14:44.248156+0000","last_active":"2024-04-26T21:14:45.245299+0000","last_peered":"2024-04-26T21:14:45.245299+0000","last_clean":"2024-04-26T21:14:45.245299+0000","last_became_active":"2024-04-26T21:14:44.247815+0000","last_became_peered":"2024-04-26T21:14:44.247815+0000","last_unstale":"2024-04-26T21:14:45.245299+0000","last_undegraded":"2024-04-26T21:14:45.245299+0000","last_fullsized":"2024-04-26T21:14:45.245299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:58.018878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244628+0000","last_change":"2024-04-26T21:14:44.247079+0000","last_active":"2024-04-26T21:14:45.244628+0000","last_peered":"2024-04-26T21:14:45.244628+0000","last_clean":"2024-04-26T21:14:45.244628+0000","last_became_active":"2024-04-26T21:14:44.245713+0000","last_became_peered":"2024-04-26T21:14:44.245713+0000","last_unstale":"2024-04-26T21:14:45.244628+0000","last_undegraded":"2024-04-26T21:14:45.244628+0000","last_fullsized":"2024-04-26T21:14:45.244628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.341966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245829+0000","last_change":"2024-04-26T21:14:44.244548+0000","last_active":"2024-04-26T21:14:45.245829+0000","last_peered":"2024-04-26T21:14:45.245829+0000","last_clean":"2024-04-26T21:14:45.245829+0000","last_became_active":"2024-04-26T21:14:44.244254+0000","last_became_peered":"2024-04-26T21:14:44.244254+0000","last_unstale":"2024-04-26T21:14:45.245829+0000","last_undegraded":"2024-04-26T21:14:45.245829+0000","last_fullsized":"2024-04-26T21:14:45.245829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:15.730313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245749+0000","last_change":"2024-04-26T21:14:44.259154+0000","last_active":"2024-04-26T21:14:45.245749+0000","last_peered":"2024-04-26T21:14:45.245749+0000","last_clean":"2024-04-26T21:14:45.245749+0000","last_became_active":"2024-04-26T21:14:44.258866+0000","last_became_peered":"2024-04-26T21:14:44.258866+0000","last_unstale":"2024-04-26T21:14:45.245749+0000","last_undegraded":"2024-04-26T21:14:45.245749+0000","last_fullsized":"2024-04-26T21:14:45.245749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:03.653934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601934+0000","last_change":"2024-04-26T21:14:44.242850+0000","last_active":"2024-04-26T21:14:45.601934+0000","last_peered":"2024-04-26T21:14:45.601934+0000","last_clean":"2024-04-26T21:14:45.601934+0000","last_became_active":"2024-04-26T21:14:44.241878+0000","last_became_peered":"2024-04-26T21:14:44.241878+0000","last_unstale":"2024-04-26T21:14:45.601934+0000","last_undegraded":"2024-04-26T21:14:45.601934+0000","last_fullsized":"2024-04-26T21:14:45.601934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.505318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603230+0000","last_change":"2024-04-26T21:14:44.254016+0000","last_active":"2024-04-26T21:14:45.603230+0000","last_peered":"2024-04-26T21:14:45.603230+0000","last_clean":"2024-04-26T21:14:45.603230+0000","last_became_active":"2024-04-26T21:14:44.253480+0000","last_became_peered":"2024-04-26T21:14:44.253480+0000","last_unstale":"2024-04-26T21:14:45.603230+0000","last_undegraded":"2024-04-26T21:14:45.603230+0000","last_fullsized":"2024-04-26T21:14:45.603230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:03.732240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298369+0000","last_change":"2024-04-26T21:14:44.250956+0000","last_active":"2024-04-26T21:14:45.298369+0000","last_peered":"2024-04-26T21:14:45.298369+0000","last_clean":"2024-04-26T21:14:45.298369+0000","last_became_active":"2024-04-26T21:14:44.250398+0000","last_became_peered":"2024-04-26T21:14:44.250398+0000","last_unstale":"2024-04-26T21:14:45.298369+0000","last_undegraded":"2024-04-26T21:14:45.298369+0000","last_fullsized":"2024-04-26T21:14:45.298369+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:41.363446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601459+0000","last_change":"2024-04-26T21:14:44.253253+0000","last_active":"2024-04-26T21:14:45.601459+0000","last_peered":"2024-04-26T21:14:45.601459+0000","last_clean":"2024-04-26T21:14:45.601459+0000","last_became_active":"2024-04-26T21:14:44.252900+0000","last_became_peered":"2024-04-26T21:14:44.252900+0000","last_unstale":"2024-04-26T21:14:45.601459+0000","last_undegraded":"2024-04-26T21:14:45.601459+0000","last_fullsized":"2024-04-26T21:14:45.601459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:55.839541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602870+0000","last_change":"2024-04-26T21:14:44.255737+0000","last_active":"2024-04-26T21:14:45.602870+0000","last_peered":"2024-04-26T21:14:45.602870+0000","last_clean":"2024-04-26T21:14:45.602870+0000","last_became_active":"2024-04-26T21:14:44.254527+0000","last_became_peered":"2024-04-26T21:14:44.254527+0000","last_unstale":"2024-04-26T21:14:45.602870+0000","last_undegraded":"2024-04-26T21:14:45.602870+0000","last_fullsized":"2024-04-26T21:14:45.602870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:45.647520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.297970+0000","last_change":"2024-04-26T21:14:44.247775+0000","last_active":"2024-04-26T21:14:45.297970+0000","last_peered":"2024-04-26T21:14:45.297970+0000","last_clean":"2024-04-26T21:14:45.297970+0000","last_became_active":"2024-04-26T21:14:44.247480+0000","last_became_peered":"2024-04-26T21:14:44.247480+0000","last_unstale":"2024-04-26T21:14:45.297970+0000","last_undegraded":"2024-04-26T21:14:45.297970+0000","last_fullsized":"2024-04-26T21:14:45.297970+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:32.610470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601097+0000","last_change":"2024-04-26T21:14:44.245912+0000","last_active":"2024-04-26T21:14:45.601097+0000","last_peered":"2024-04-26T21:14:45.601097+0000","last_clean":"2024-04-26T21:14:45.601097+0000","last_became_active":"2024-04-26T21:14:44.245604+0000","last_became_peered":"2024-04-26T21:14:44.245604+0000","last_unstale":"2024-04-26T21:14:45.601097+0000","last_undegraded":"2024-04-26T21:14:45.601097+0000","last_fullsized":"2024-04-26T21:14:45.601097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:33.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298080+0000","last_change":"2024-04-26T21:14:44.254016+0000","last_active":"2024-04-26T21:14:45.298080+0000","last_peered":"2024-04-26T21:14:45.298080+0000","last_clean":"2024-04-26T21:14:45.298080+0000","last_became_active":"2024-04-26T21:14:44.253696+0000","last_became_peered":"2024-04-26T21:14:44.253696+0000","last_unstale":"2024-04-26T21:14:45.298080+0000","last_undegraded":"2024-04-26T21:14:45.298080+0000","last_fullsized":"2024-04-26T21:14:45.298080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:14.343342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244057+0000","last_change":"2024-04-26T21:14:44.245979+0000","last_active":"2024-04-26T21:14:45.244057+0000","last_peered":"2024-04-26T21:14:45.244057+0000","last_clean":"2024-04-26T21:14:45.244057+0000","last_became_active":"2024-04-26T21:14:44.245495+0000","last_became_peered":"2024-04-26T21:14:44.245495+0000","last_unstale":"2024-04-26T21:14:45.244057+0000","last_undegraded":"2024-04-26T21:14:45.244057+0000","last_fullsized":"2024-04-26T21:14:45.244057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:39.045671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601704+0000","last_change":"2024-04-26T21:14:44.259746+0000","last_active":"2024-04-26T21:14:45.601704+0000","last_peered":"2024-04-26T21:14:45.601704+0000","last_clean":"2024-04-26T21:14:45.601704+0000","last_became_active":"2024-04-26T21:14:44.259442+0000","last_became_peered":"2024-04-26T21:14:44.259442+0000","last_unstale":"2024-04-26T21:14:45.601704+0000","last_undegraded":"2024-04-26T21:14:45.601704+0000","last_fullsized":"2024-04-26T21:14:45.601704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:54.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244877+0000","last_change":"2024-04-26T21:14:44.244347+0000","last_active":"2024-04-26T21:14:45.244877+0000","last_peered":"2024-04-26T21:14:45.244877+0000","last_clean":"2024-04-26T21:14:45.244877+0000","last_became_active":"2024-04-26T21:14:44.244124+0000","last_became_peered":"2024-04-26T21:14:44.244124+0000","last_unstale":"2024-04-26T21:14:45.244877+0000","last_undegraded":"2024-04-26T21:14:45.244877+0000","last_fullsized":"2024-04-26T21:14:45.244877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:32.610470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602139+0000","last_change":"2024-04-26T21:14:44.240811+0000","last_active":"2024-04-26T21:14:45.602139+0000","last_peered":"2024-04-26T21:14:45.602139+0000","last_clean":"2024-04-26T21:14:45.602139+0000","last_became_active":"2024-04-26T21:14:44.240507+0000","last_became_peered":"2024-04-26T21:14:44.240507+0000","last_unstale":"2024-04-26T21:14:45.602139+0000","last_undegraded":"2024-04-26T21:14:45.602139+0000","last_fullsized":"2024-04-26T21:14:45.602139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:02.330747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601480+0000","last_change":"2024-04-26T21:14:44.244518+0000","last_active":"2024-04-26T21:14:45.601480+0000","last_peered":"2024-04-26T21:14:45.601480+0000","last_clean":"2024-04-26T21:14:45.601480+0000","last_became_active":"2024-04-26T21:14:44.244237+0000","last_became_peered":"2024-04-26T21:14:44.244237+0000","last_unstale":"2024-04-26T21:14:45.601480+0000","last_undegraded":"2024-04-26T21:14:45.601480+0000","last_fullsized":"2024-04-26T21:14:45.601480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:27.551914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603060+0000","last_change":"2024-04-26T21:14:44.251193+0000","last_active":"2024-04-26T21:14:45.603060+0000","last_peered":"2024-04-26T21:14:45.603060+0000","last_clean":"2024-04-26T21:14:45.603060+0000","last_became_active":"2024-04-26T21:14:44.250915+0000","last_became_peered":"2024-04-26T21:14:44.250915+0000","last_unstale":"2024-04-26T21:14:45.603060+0000","last_undegraded":"2024-04-26T21:14:45.603060+0000","last_fullsized":"2024-04-26T21:14:45.603060+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.416873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298034+0000","last_change":"2024-04-26T21:14:44.249557+0000","last_active":"2024-04-26T21:14:45.298034+0000","last_peered":"2024-04-26T21:14:45.298034+0000","last_clean":"2024-04-26T21:14:45.298034+0000","last_became_active":"2024-04-26T21:14:44.249270+0000","last_became_peered":"2024-04-26T21:14:44.249270+0000","last_unstale":"2024-04-26T21:14:45.298034+0000","last_undegraded":"2024-04-26T21:14:45.298034+0000","last_fullsized":"2024-04-26T21:14:45.298034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:13.579951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.600926+0000","last_change":"2024-04-26T21:14:44.258839+0000","last_active":"2024-04-26T21:14:45.600926+0000","last_peered":"2024-04-26T21:14:45.600926+0000","last_clean":"2024-04-26T21:14:45.600926+0000","last_became_active":"2024-04-26T21:14:44.258556+0000","last_became_peered":"2024-04-26T21:14:44.258556+0000","last_unstale":"2024-04-26T21:14:45.600926+0000","last_undegraded":"2024-04-26T21:14:45.600926+0000","last_fullsized":"2024-04-26T21:14:45.600926+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:52.364393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601944+0000","last_change":"2024-04-26T21:14:44.255699+0000","last_active":"2024-04-26T21:14:45.601944+0000","last_peered":"2024-04-26T21:14:45.601944+0000","last_clean":"2024-04-26T21:14:45.601944+0000","last_became_active":"2024-04-26T21:14:44.255441+0000","last_became_peered":"2024-04-26T21:14:44.255441+0000","last_unstale":"2024-04-26T21:14:45.601944+0000","last_undegraded":"2024-04-26T21:14:45.601944+0000","last_fullsized":"2024-04-26T21:14:45.601944+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:14.343342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602439+0000","last_change":"2024-04-26T21:14:44.242640+0000","last_active":"2024-04-26T21:14:45.602439+0000","last_peered":"2024-04-26T21:14:45.602439+0000","last_clean":"2024-04-26T21:14:45.602439+0000","last_became_active":"2024-04-26T21:14:44.242360+0000","last_became_peered":"2024-04-26T21:14:44.242360+0000","last_unstale":"2024-04-26T21:14:45.602439+0000","last_undegraded":"2024-04-26T21:14:45.602439+0000","last_fullsized":"2024-04-26T21:14:45.602439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:21.907680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602458+0000","last_change":"2024-04-26T21:14:43.245071+0000","last_active":"2024-04-26T21:14:45.602458+0000","last_peered":"2024-04-26T21:14:45.602458+0000","last_clean":"2024-04-26T21:14:45.602458+0000","last_became_active":"2024-04-26T21:14:43.244758+0000","last_became_peered":"2024-04-26T21:14:43.244758+0000","last_unstale":"2024-04-26T21:14:45.602458+0000","last_undegraded":"2024-04-26T21:14:45.602458+0000","last_fullsized":"2024-04-26T21:14:45.602458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:30.612903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602701+0000","last_change":"2024-04-26T21:14:44.256368+0000","last_active":"2024-04-26T21:14:45.602701+0000","last_peered":"2024-04-26T21:14:45.602701+0000","last_clean":"2024-04-26T21:14:45.602701+0000","last_became_active":"2024-04-26T21:14:44.256112+0000","last_became_peered":"2024-04-26T21:14:44.256112+0000","last_unstale":"2024-04-26T21:14:45.602701+0000","last_undegraded":"2024-04-26T21:14:45.602701+0000","last_fullsized":"2024-04-26T21:14:45.602701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:21.201357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603006+0000","last_change":"2024-04-26T21:14:43.243675+0000","last_active":"2024-04-26T21:14:45.603006+0000","last_peered":"2024-04-26T21:14:45.603006+0000","last_clean":"2024-04-26T21:14:45.603006+0000","last_became_active":"2024-04-26T21:14:43.243347+0000","last_became_peered":"2024-04-26T21:14:43.243347+0000","last_unstale":"2024-04-26T21:14:45.603006+0000","last_undegraded":"2024-04-26T21:14:45.603006+0000","last_fullsized":"2024-04-26T21:14:45.603006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:31.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602232+0000","last_change":"2024-04-26T21:14:44.259459+0000","last_active":"2024-04-26T21:14:45.602232+0000","last_peered":"2024-04-26T21:14:45.602232+0000","last_clean":"2024-04-26T21:14:45.602232+0000","last_became_active":"2024-04-26T21:14:44.259165+0000","last_became_peered":"2024-04-26T21:14:44.259165+0000","last_unstale":"2024-04-26T21:14:45.602232+0000","last_undegraded":"2024-04-26T21:14:45.602232+0000","last_fullsized":"2024-04-26T21:14:45.602232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:01.343387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602380+0000","last_change":"2024-04-26T21:14:43.244723+0000","last_active":"2024-04-26T21:14:45.602380+0000","last_peered":"2024-04-26T21:14:45.602380+0000","last_clean":"2024-04-26T21:14:45.602380+0000","last_became_active":"2024-04-26T21:14:43.244073+0000","last_became_peered":"2024-04-26T21:14:43.244073+0000","last_unstale":"2024-04-26T21:14:45.602380+0000","last_undegraded":"2024-04-26T21:14:45.602380+0000","last_fullsized":"2024-04-26T21:14:45.602380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:02.644080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602319+0000","last_change":"2024-04-26T21:14:44.244747+0000","last_active":"2024-04-26T21:14:45.602319+0000","last_peered":"2024-04-26T21:14:45.602319+0000","last_clean":"2024-04-26T21:14:45.602319+0000","last_became_active":"2024-04-26T21:14:44.243422+0000","last_became_peered":"2024-04-26T21:14:44.243422+0000","last_unstale":"2024-04-26T21:14:45.602319+0000","last_undegraded":"2024-04-26T21:14:45.602319+0000","last_fullsized":"2024-04-26T21:14:45.602319+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.674505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245358+0000","last_change":"2024-04-26T21:14:43.245066+0000","last_active":"2024-04-26T21:14:45.245358+0000","last_peered":"2024-04-26T21:14:45.245358+0000","last_clean":"2024-04-26T21:14:45.245358+0000","last_became_active":"2024-04-26T21:14:43.244761+0000","last_became_peered":"2024-04-26T21:14:43.244761+0000","last_unstale":"2024-04-26T21:14:45.245358+0000","last_undegraded":"2024-04-26T21:14:45.245358+0000","last_fullsized":"2024-04-26T21:14:45.245358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:02.644080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602674+0000","last_change":"2024-04-26T21:14:44.245792+0000","last_active":"2024-04-26T21:14:45.602674+0000","last_peered":"2024-04-26T21:14:45.602674+0000","last_clean":"2024-04-26T21:14:45.602674+0000","last_became_active":"2024-04-26T21:14:44.245434+0000","last_became_peered":"2024-04-26T21:14:44.245434+0000","last_unstale":"2024-04-26T21:14:45.602674+0000","last_undegraded":"2024-04-26T21:14:45.602674+0000","last_fullsized":"2024-04-26T21:14:45.602674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:28.252758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245877+0000","last_change":"2024-04-26T21:14:43.240418+0000","last_active":"2024-04-26T21:14:45.245877+0000","last_peered":"2024-04-26T21:14:45.245877+0000","last_clean":"2024-04-26T21:14:45.245877+0000","last_became_active":"2024-04-26T21:14:43.240112+0000","last_became_peered":"2024-04-26T21:14:43.240112+0000","last_unstale":"2024-04-26T21:14:45.245877+0000","last_undegraded":"2024-04-26T21:14:45.245877+0000","last_fullsized":"2024-04-26T21:14:45.245877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:10.969404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602922+0000","last_change":"2024-04-26T21:14:44.251232+0000","last_active":"2024-04-26T21:14:45.602922+0000","last_peered":"2024-04-26T21:14:45.602922+0000","last_clean":"2024-04-26T21:14:45.602922+0000","last_became_active":"2024-04-26T21:14:44.250963+0000","last_became_peered":"2024-04-26T21:14:44.250963+0000","last_unstale":"2024-04-26T21:14:45.602922+0000","last_undegraded":"2024-04-26T21:14:45.602922+0000","last_fullsized":"2024-04-26T21:14:45.602922+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:47.339550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602215+0000","last_change":"2024-04-26T21:14:43.246517+0000","last_active":"2024-04-26T21:14:45.602215+0000","last_peered":"2024-04-26T21:14:45.602215+0000","last_clean":"2024-04-26T21:14:45.602215+0000","last_became_active":"2024-04-26T21:14:43.245502+0000","last_became_peered":"2024-04-26T21:14:43.245502+0000","last_unstale":"2024-04-26T21:14:45.602215+0000","last_undegraded":"2024-04-26T21:14:45.602215+0000","last_fullsized":"2024-04-26T21:14:45.602215+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:18.312427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244104+0000","last_change":"2024-04-26T21:14:44.245754+0000","last_active":"2024-04-26T21:14:45.244104+0000","last_peered":"2024-04-26T21:14:45.244104+0000","last_clean":"2024-04-26T21:14:45.244104+0000","last_became_active":"2024-04-26T21:14:44.245143+0000","last_became_peered":"2024-04-26T21:14:44.245143+0000","last_unstale":"2024-04-26T21:14:45.244104+0000","last_undegraded":"2024-04-26T21:14:45.244104+0000","last_fullsized":"2024-04-26T21:14:45.244104+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:26.895798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244379+0000","last_change":"2024-04-26T21:14:43.240736+0000","last_active":"2024-04-26T21:14:45.244379+0000","last_peered":"2024-04-26T21:14:45.244379+0000","last_clean":"2024-04-26T21:14:45.244379+0000","last_became_active":"2024-04-26T21:14:43.239737+0000","last_became_peered":"2024-04-26T21:14:43.239737+0000","last_unstale":"2024-04-26T21:14:45.244379+0000","last_undegraded":"2024-04-26T21:14:45.244379+0000","last_fullsized":"2024-04-26T21:14:45.244379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:18.312427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601144+0000","last_change":"2024-04-26T21:14:44.254056+0000","last_active":"2024-04-26T21:14:45.601144+0000","last_peered":"2024-04-26T21:14:45.601144+0000","last_clean":"2024-04-26T21:14:45.601144+0000","last_became_active":"2024-04-26T21:14:44.253730+0000","last_became_peered":"2024-04-26T21:14:44.253730+0000","last_unstale":"2024-04-26T21:14:45.601144+0000","last_undegraded":"2024-04-26T21:14:45.601144+0000","last_fullsized":"2024-04-26T21:14:45.601144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:03.464423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245583+0000","last_change":"2024-04-26T21:14:43.248195+0000","last_active":"2024-04-26T21:14:45.245583+0000","last_peered":"2024-04-26T21:14:45.245583+0000","last_clean":"2024-04-26T21:14:45.245583+0000","last_became_active":"2024-04-26T21:14:43.247919+0000","last_became_peered":"2024-04-26T21:14:43.247919+0000","last_unstale":"2024-04-26T21:14:45.245583+0000","last_undegraded":"2024-04-26T21:14:45.245583+0000","last_fullsized":"2024-04-26T21:14:45.245583+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:02.774701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.246207+0000","last_change":"2024-04-26T21:14:44.259493+0000","last_active":"2024-04-26T21:14:45.246207+0000","last_peered":"2024-04-26T21:14:45.246207+0000","last_clean":"2024-04-26T21:14:45.246207+0000","last_became_active":"2024-04-26T21:14:44.259195+0000","last_became_peered":"2024-04-26T21:14:44.259195+0000","last_unstale":"2024-04-26T21:14:45.246207+0000","last_undegraded":"2024-04-26T21:14:45.246207+0000","last_fullsized":"2024-04-26T21:14:45.246207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298447+0000","last_change":"2024-04-26T21:14:43.246064+0000","last_active":"2024-04-26T21:14:45.298447+0000","last_peered":"2024-04-26T21:14:45.298447+0000","last_clean":"2024-04-26T21:14:45.298447+0000","last_became_active":"2024-04-26T21:14:43.245771+0000","last_became_peered":"2024-04-26T21:14:43.245771+0000","last_unstale":"2024-04-26T21:14:45.298447+0000","last_undegraded":"2024-04-26T21:14:45.298447+0000","last_fullsized":"2024-04-26T21:14:45.298447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:18.312427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298517+0000","last_change":"2024-04-26T21:14:44.259013+0000","last_active":"2024-04-26T21:14:45.298517+0000","last_peered":"2024-04-26T21:14:45.298517+0000","last_clean":"2024-04-26T21:14:45.298517+0000","last_became_active":"2024-04-26T21:14:44.258677+0000","last_became_peered":"2024-04-26T21:14:44.258677+0000","last_unstale":"2024-04-26T21:14:45.298517+0000","last_undegraded":"2024-04-26T21:14:45.298517+0000","last_fullsized":"2024-04-26T21:14:45.298517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:21.907680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603048+0000","last_change":"2024-04-26T21:14:43.241603+0000","last_active":"2024-04-26T21:14:45.603048+0000","last_peered":"2024-04-26T21:14:45.603048+0000","last_clean":"2024-04-26T21:14:45.603048+0000","last_became_active":"2024-04-26T21:14:43.238293+0000","last_became_peered":"2024-04-26T21:14:43.238293+0000","last_unstale":"2024-04-26T21:14:45.603048+0000","last_undegraded":"2024-04-26T21:14:45.603048+0000","last_fullsized":"2024-04-26T21:14:45.603048+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:12.570097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601155+0000","last_change":"2024-04-26T21:14:44.263354+0000","last_active":"2024-04-26T21:14:45.601155+0000","last_peered":"2024-04-26T21:14:45.601155+0000","last_clean":"2024-04-26T21:14:45.601155+0000","last_became_active":"2024-04-26T21:14:44.263112+0000","last_became_peered":"2024-04-26T21:14:44.263112+0000","last_unstale":"2024-04-26T21:14:45.601155+0000","last_undegraded":"2024-04-26T21:14:45.601155+0000","last_fullsized":"2024-04-26T21:14:45.601155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:44.830244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298283+0000","last_change":"2024-04-26T21:14:43.244459+0000","last_active":"2024-04-26T21:14:45.298283+0000","last_peered":"2024-04-26T21:14:45.298283+0000","last_clean":"2024-04-26T21:14:45.298283+0000","last_became_active":"2024-04-26T21:14:43.244186+0000","last_became_peered":"2024-04-26T21:14:43.244186+0000","last_unstale":"2024-04-26T21:14:45.298283+0000","last_undegraded":"2024-04-26T21:14:45.298283+0000","last_fullsized":"2024-04-26T21:14:45.298283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:17.993603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298213+0000","last_change":"2024-04-26T21:14:44.258475+0000","last_active":"2024-04-26T21:14:45.298213+0000","last_peered":"2024-04-26T21:14:45.298213+0000","last_clean":"2024-04-26T21:14:45.298213+0000","last_became_active":"2024-04-26T21:14:44.258191+0000","last_became_peered":"2024-04-26T21:14:44.258191+0000","last_unstale":"2024-04-26T21:14:45.298213+0000","last_undegraded":"2024-04-26T21:14:45.298213+0000","last_fullsized":"2024-04-26T21:14:45.298213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:12.604409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.246105+0000","last_change":"2024-04-26T21:14:43.247008+0000","last_active":"2024-04-26T21:14:45.246105+0000","last_peered":"2024-04-26T21:14:45.246105+0000","last_clean":"2024-04-26T21:14:45.246105+0000","last_became_active":"2024-04-26T21:14:43.246743+0000","last_became_peered":"2024-04-26T21:14:43.246743+0000","last_unstale":"2024-04-26T21:14:45.246105+0000","last_undegraded":"2024-04-26T21:14:45.246105+0000","last_fullsized":"2024-04-26T21:14:45.246105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:10.380240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245228+0000","last_change":"2024-04-26T21:14:44.253031+0000","last_active":"2024-04-26T21:14:45.245228+0000","last_peered":"2024-04-26T21:14:45.245228+0000","last_clean":"2024-04-26T21:14:45.245228+0000","last_became_active":"2024-04-26T21:14:44.251048+0000","last_became_peered":"2024-04-26T21:14:44.251048+0000","last_unstale":"2024-04-26T21:14:45.245228+0000","last_undegraded":"2024-04-26T21:14:45.245228+0000","last_fullsized":"2024-04-26T21:14:45.245228+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:02.330747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602945+0000","last_change":"2024-04-26T21:14:43.242813+0000","last_active":"2024-04-26T21:14:45.602945+0000","last_peered":"2024-04-26T21:14:45.602945+0000","last_clean":"2024-04-26T21:14:45.602945+0000","last_became_active":"2024-04-26T21:14:43.242095+0000","last_became_peered":"2024-04-26T21:14:43.242095+0000","last_unstale":"2024-04-26T21:14:45.602945+0000","last_undegraded":"2024-04-26T21:14:45.602945+0000","last_fullsized":"2024-04-26T21:14:45.602945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:13.333488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298273+0000","last_change":"2024-04-26T21:14:44.252847+0000","last_active":"2024-04-26T21:14:45.298273+0000","last_peered":"2024-04-26T21:14:45.298273+0000","last_clean":"2024-04-26T21:14:45.298273+0000","last_became_active":"2024-04-26T21:14:44.252572+0000","last_became_peered":"2024-04-26T21:14:44.252572+0000","last_unstale":"2024-04-26T21:14:45.298273+0000","last_undegraded":"2024-04-26T21:14:45.298273+0000","last_fullsized":"2024-04-26T21:14:45.298273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:02.330747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.243971+0000","last_change":"2024-04-26T21:14:43.245039+0000","last_active":"2024-04-26T21:14:45.243971+0000","last_peered":"2024-04-26T21:14:45.243971+0000","last_clean":"2024-04-26T21:14:45.243971+0000","last_became_active":"2024-04-26T21:14:43.244681+0000","last_became_peered":"2024-04-26T21:14:43.244681+0000","last_unstale":"2024-04-26T21:14:45.243971+0000","last_undegraded":"2024-04-26T21:14:45.243971+0000","last_fullsized":"2024-04-26T21:14:45.243971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:50.492755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245870+0000","last_change":"2024-04-26T21:14:44.243756+0000","last_active":"2024-04-26T21:14:45.245870+0000","last_peered":"2024-04-26T21:14:45.245870+0000","last_clean":"2024-04-26T21:14:45.245870+0000","last_became_active":"2024-04-26T21:14:44.243441+0000","last_became_peered":"2024-04-26T21:14:44.243441+0000","last_unstale":"2024-04-26T21:14:45.245870+0000","last_undegraded":"2024-04-26T21:14:45.245870+0000","last_fullsized":"2024-04-26T21:14:45.245870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:42.915124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244491+0000","last_change":"2024-04-26T21:14:43.244389+0000","last_active":"2024-04-26T21:14:45.244491+0000","last_peered":"2024-04-26T21:14:45.244491+0000","last_clean":"2024-04-26T21:14:45.244491+0000","last_became_active":"2024-04-26T21:14:43.244130+0000","last_became_peered":"2024-04-26T21:14:43.244130+0000","last_unstale":"2024-04-26T21:14:45.244491+0000","last_undegraded":"2024-04-26T21:14:45.244491+0000","last_fullsized":"2024-04-26T21:14:45.244491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:17.993603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298039+0000","last_change":"2024-04-26T21:14:44.257835+0000","last_active":"2024-04-26T21:14:45.298039+0000","last_peered":"2024-04-26T21:14:45.298039+0000","last_clean":"2024-04-26T21:14:45.298039+0000","last_became_active":"2024-04-26T21:14:44.257516+0000","last_became_peered":"2024-04-26T21:14:44.257516+0000","last_unstale":"2024-04-26T21:14:45.298039+0000","last_undegraded":"2024-04-26T21:14:45.298039+0000","last_fullsized":"2024-04-26T21:14:45.298039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.505318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298200+0000","last_change":"2024-04-26T21:14:43.243170+0000","last_active":"2024-04-26T21:14:45.298200+0000","last_peered":"2024-04-26T21:14:45.298200+0000","last_clean":"2024-04-26T21:14:45.298200+0000","last_became_active":"2024-04-26T21:14:43.242458+0000","last_became_peered":"2024-04-26T21:14:43.242458+0000","last_unstale":"2024-04-26T21:14:45.298200+0000","last_undegraded":"2024-04-26T21:14:45.298200+0000","last_fullsized":"2024-04-26T21:14:45.298200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:41.680689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298358+0000","last_change":"2024-04-26T21:14:44.250387+0000","last_active":"2024-04-26T21:14:45.298358+0000","last_peered":"2024-04-26T21:14:45.298358+0000","last_clean":"2024-04-26T21:14:45.298358+0000","last_became_active":"2024-04-26T21:14:44.249821+0000","last_became_peered":"2024-04-26T21:14:44.249821+0000","last_unstale":"2024-04-26T21:14:45.298358+0000","last_undegraded":"2024-04-26T21:14:45.298358+0000","last_fullsized":"2024-04-26T21:14:45.298358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:01.343387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603213+0000","last_change":"2024-04-26T21:14:43.239808+0000","last_active":"2024-04-26T21:14:45.603213+0000","last_peered":"2024-04-26T21:14:45.603213+0000","last_clean":"2024-04-26T21:14:45.603213+0000","last_became_active":"2024-04-26T21:14:43.239524+0000","last_became_peered":"2024-04-26T21:14:43.239524+0000","last_unstale":"2024-04-26T21:14:45.603213+0000","last_undegraded":"2024-04-26T21:14:45.603213+0000","last_fullsized":"2024-04-26T21:14:45.603213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:35.886111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245017+0000","last_change":"2024-04-26T21:14:44.251469+0000","last_active":"2024-04-26T21:14:45.245017+0000","last_peered":"2024-04-26T21:14:45.245017+0000","last_clean":"2024-04-26T21:14:45.245017+0000","last_became_active":"2024-04-26T21:14:44.251207+0000","last_became_peered":"2024-04-26T21:14:44.251207+0000","last_unstale":"2024-04-26T21:14:45.245017+0000","last_undegraded":"2024-04-26T21:14:45.245017+0000","last_fullsized":"2024-04-26T21:14:45.245017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:13.579951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602487+0000","last_change":"2024-04-26T21:14:43.240450+0000","last_active":"2024-04-26T21:14:45.602487+0000","last_peered":"2024-04-26T21:14:45.602487+0000","last_clean":"2024-04-26T21:14:45.602487+0000","last_became_active":"2024-04-26T21:14:43.240147+0000","last_became_peered":"2024-04-26T21:14:43.240147+0000","last_unstale":"2024-04-26T21:14:45.602487+0000","last_undegraded":"2024-04-26T21:14:45.602487+0000","last_fullsized":"2024-04-26T21:14:45.602487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:14.720460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602927+0000","last_change":"2024-04-26T21:14:44.254618+0000","last_active":"2024-04-26T21:14:45.602927+0000","last_peered":"2024-04-26T21:14:45.602927+0000","last_clean":"2024-04-26T21:14:45.602927+0000","last_became_active":"2024-04-26T21:14:44.254283+0000","last_became_peered":"2024-04-26T21:14:44.254283+0000","last_unstale":"2024-04-26T21:14:45.602927+0000","last_undegraded":"2024-04-26T21:14:45.602927+0000","last_fullsized":"2024-04-26T21:14:45.602927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:10.056516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244642+0000","last_change":"2024-04-26T21:14:43.243703+0000","last_active":"2024-04-26T21:14:45.244642+0000","last_peered":"2024-04-26T21:14:45.244642+0000","last_clean":"2024-04-26T21:14:45.244642+0000","last_became_active":"2024-04-26T21:14:43.243435+0000","last_became_peered":"2024-04-26T21:14:43.243435+0000","last_unstale":"2024-04-26T21:14:45.244642+0000","last_undegraded":"2024-04-26T21:14:45.244642+0000","last_fullsized":"2024-04-26T21:14:45.244642+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298363+0000","last_change":"2024-04-26T21:14:44.258552+0000","last_active":"2024-04-26T21:14:45.298363+0000","last_peered":"2024-04-26T21:14:45.298363+0000","last_clean":"2024-04-26T21:14:45.298363+0000","last_became_active":"2024-04-26T21:14:44.258254+0000","last_became_peered":"2024-04-26T21:14:44.258254+0000","last_unstale":"2024-04-26T21:14:45.298363+0000","last_undegraded":"2024-04-26T21:14:45.298363+0000","last_fullsized":"2024-04-26T21:14:45.298363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.674505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602281+0000","last_change":"2024-04-26T21:14:43.249909+0000","last_active":"2024-04-26T21:14:45.602281+0000","last_peered":"2024-04-26T21:14:45.602281+0000","last_clean":"2024-04-26T21:14:45.602281+0000","last_became_active":"2024-04-26T21:14:43.249559+0000","last_became_peered":"2024-04-26T21:14:43.249559+0000","last_unstale":"2024-04-26T21:14:45.602281+0000","last_undegraded":"2024-04-26T21:14:45.602281+0000","last_fullsized":"2024-04-26T21:14:45.602281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245393+0000","last_change":"2024-04-26T21:14:44.257678+0000","last_active":"2024-04-26T21:14:45.245393+0000","last_peered":"2024-04-26T21:14:45.245393+0000","last_clean":"2024-04-26T21:14:45.245393+0000","last_became_active":"2024-04-26T21:14:44.257406+0000","last_became_peered":"2024-04-26T21:14:44.257406+0000","last_unstale":"2024-04-26T21:14:45.245393+0000","last_undegraded":"2024-04-26T21:14:45.245393+0000","last_fullsized":"2024-04-26T21:14:45.245393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:12.604409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.603050+0000","last_change":"2024-04-26T21:14:43.242048+0000","last_active":"2024-04-26T21:14:45.603050+0000","last_peered":"2024-04-26T21:14:45.603050+0000","last_clean":"2024-04-26T21:14:45.603050+0000","last_became_active":"2024-04-26T21:14:43.241625+0000","last_became_peered":"2024-04-26T21:14:43.241625+0000","last_unstale":"2024-04-26T21:14:45.603050+0000","last_undegraded":"2024-04-26T21:14:45.603050+0000","last_fullsized":"2024-04-26T21:14:45.603050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:43.472850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602189+0000","last_change":"2024-04-26T21:14:44.241472+0000","last_active":"2024-04-26T21:14:45.602189+0000","last_peered":"2024-04-26T21:14:45.602189+0000","last_clean":"2024-04-26T21:14:45.602189+0000","last_became_active":"2024-04-26T21:14:44.241194+0000","last_became_peered":"2024-04-26T21:14:44.241194+0000","last_unstale":"2024-04-26T21:14:45.602189+0000","last_undegraded":"2024-04-26T21:14:45.602189+0000","last_fullsized":"2024-04-26T21:14:45.602189+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:58.018878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601355+0000","last_change":"2024-04-26T21:14:43.241349+0000","last_active":"2024-04-26T21:14:45.601355+0000","last_peered":"2024-04-26T21:14:45.601355+0000","last_clean":"2024-04-26T21:14:45.601355+0000","last_became_active":"2024-04-26T21:14:43.241055+0000","last_became_peered":"2024-04-26T21:14:43.241055+0000","last_unstale":"2024-04-26T21:14:45.601355+0000","last_undegraded":"2024-04-26T21:14:45.601355+0000","last_fullsized":"2024-04-26T21:14:45.601355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:35.604628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602003+0000","last_change":"2024-04-26T21:14:44.242587+0000","last_active":"2024-04-26T21:14:45.602003+0000","last_peered":"2024-04-26T21:14:45.602003+0000","last_clean":"2024-04-26T21:14:45.602003+0000","last_became_active":"2024-04-26T21:14:44.241959+0000","last_became_peered":"2024-04-26T21:14:44.241959+0000","last_unstale":"2024-04-26T21:14:45.602003+0000","last_undegraded":"2024-04-26T21:14:45.602003+0000","last_fullsized":"2024-04-26T21:14:45.602003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:44.482704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244806+0000","last_change":"2024-04-26T21:14:43.245289+0000","last_active":"2024-04-26T21:14:45.244806+0000","last_peered":"2024-04-26T21:14:45.244806+0000","last_clean":"2024-04-26T21:14:45.244806+0000","last_became_active":"2024-04-26T21:14:43.245008+0000","last_became_peered":"2024-04-26T21:14:43.245008+0000","last_unstale":"2024-04-26T21:14:45.244806+0000","last_undegraded":"2024-04-26T21:14:45.244806+0000","last_fullsized":"2024-04-26T21:14:45.244806+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:30.612903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245051+0000","last_change":"2024-04-26T21:14:44.244337+0000","last_active":"2024-04-26T21:14:45.245051+0000","last_peered":"2024-04-26T21:14:45.245051+0000","last_clean":"2024-04-26T21:14:45.245051+0000","last_became_active":"2024-04-26T21:14:44.244087+0000","last_became_peered":"2024-04-26T21:14:44.244087+0000","last_unstale":"2024-04-26T21:14:45.245051+0000","last_undegraded":"2024-04-26T21:14:45.245051+0000","last_fullsized":"2024-04-26T21:14:45.245051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:06.505318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602666+0000","last_change":"2024-04-26T21:14:43.247928+0000","last_active":"2024-04-26T21:14:45.602666+0000","last_peered":"2024-04-26T21:14:45.602666+0000","last_clean":"2024-04-26T21:14:45.602666+0000","last_became_active":"2024-04-26T21:14:43.247549+0000","last_became_peered":"2024-04-26T21:14:43.247549+0000","last_unstale":"2024-04-26T21:14:45.602666+0000","last_undegraded":"2024-04-26T21:14:45.602666+0000","last_fullsized":"2024-04-26T21:14:45.602666+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:27.702426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245211+0000","last_change":"2024-04-26T21:14:44.248939+0000","last_active":"2024-04-26T21:14:45.245211+0000","last_peered":"2024-04-26T21:14:45.245211+0000","last_clean":"2024-04-26T21:14:45.245211+0000","last_became_active":"2024-04-26T21:14:44.248656+0000","last_became_peered":"2024-04-26T21:14:44.248656+0000","last_unstale":"2024-04-26T21:14:45.245211+0000","last_undegraded":"2024-04-26T21:14:45.245211+0000","last_fullsized":"2024-04-26T21:14:45.245211+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:44.482704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602619+0000","last_change":"2024-04-26T21:14:43.245782+0000","last_active":"2024-04-26T21:14:45.602619+0000","last_peered":"2024-04-26T21:14:45.602619+0000","last_clean":"2024-04-26T21:14:45.602619+0000","last_became_active":"2024-04-26T21:14:43.245447+0000","last_became_peered":"2024-04-26T21:14:43.245447+0000","last_unstale":"2024-04-26T21:14:45.602619+0000","last_undegraded":"2024-04-26T21:14:45.602619+0000","last_fullsized":"2024-04-26T21:14:45.602619+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:37:41.905270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601312+0000","last_change":"2024-04-26T21:14:44.263566+0000","last_active":"2024-04-26T21:14:45.601312+0000","last_peered":"2024-04-26T21:14:45.601312+0000","last_clean":"2024-04-26T21:14:45.601312+0000","last_became_active":"2024-04-26T21:14:44.263259+0000","last_became_peered":"2024-04-26T21:14:44.263259+0000","last_unstale":"2024-04-26T21:14:45.601312+0000","last_undegraded":"2024-04-26T21:14:45.601312+0000","last_fullsized":"2024-04-26T21:14:45.601312+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:20.846358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244363+0000","last_change":"2024-04-26T21:14:43.244006+0000","last_active":"2024-04-26T21:14:45.244363+0000","last_peered":"2024-04-26T21:14:45.244363+0000","last_clean":"2024-04-26T21:14:45.244363+0000","last_became_active":"2024-04-26T21:14:43.243702+0000","last_became_peered":"2024-04-26T21:14:43.243702+0000","last_unstale":"2024-04-26T21:14:45.244363+0000","last_undegraded":"2024-04-26T21:14:45.244363+0000","last_fullsized":"2024-04-26T21:14:45.244363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:11.562899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602347+0000","last_change":"2024-04-26T21:14:44.243302+0000","last_active":"2024-04-26T21:14:45.602347+0000","last_peered":"2024-04-26T21:14:45.602347+0000","last_clean":"2024-04-26T21:14:45.602347+0000","last_became_active":"2024-04-26T21:14:44.243026+0000","last_became_peered":"2024-04-26T21:14:44.243026+0000","last_unstale":"2024-04-26T21:14:45.602347+0000","last_undegraded":"2024-04-26T21:14:45.602347+0000","last_fullsized":"2024-04-26T21:14:45.602347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:41.363446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245448+0000","last_change":"2024-04-26T21:14:43.240075+0000","last_active":"2024-04-26T21:14:45.245448+0000","last_peered":"2024-04-26T21:14:45.245448+0000","last_clean":"2024-04-26T21:14:45.245448+0000","last_became_active":"2024-04-26T21:14:43.239764+0000","last_became_peered":"2024-04-26T21:14:43.239764+0000","last_unstale":"2024-04-26T21:14:45.245448+0000","last_undegraded":"2024-04-26T21:14:45.245448+0000","last_fullsized":"2024-04-26T21:14:45.245448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:14.720460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602700+0000","last_change":"2024-04-26T21:14:44.256045+0000","last_active":"2024-04-26T21:14:45.602700+0000","last_peered":"2024-04-26T21:14:45.602700+0000","last_clean":"2024-04-26T21:14:45.602700+0000","last_became_active":"2024-04-26T21:14:44.255749+0000","last_became_peered":"2024-04-26T21:14:44.255749+0000","last_unstale":"2024-04-26T21:14:45.602700+0000","last_undegraded":"2024-04-26T21:14:45.602700+0000","last_fullsized":"2024-04-26T21:14:45.602700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:53.269683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601751+0000","last_change":"2024-04-26T21:14:43.243196+0000","last_active":"2024-04-26T21:14:45.601751+0000","last_peered":"2024-04-26T21:14:45.601751+0000","last_clean":"2024-04-26T21:14:45.601751+0000","last_became_active":"2024-04-26T21:14:43.242369+0000","last_became_peered":"2024-04-26T21:14:43.242369+0000","last_unstale":"2024-04-26T21:14:45.601751+0000","last_undegraded":"2024-04-26T21:14:45.601751+0000","last_fullsized":"2024-04-26T21:14:45.601751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:21.948893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.298213+0000","last_change":"2024-04-26T21:14:44.260432+0000","last_active":"2024-04-26T21:14:45.298213+0000","last_peered":"2024-04-26T21:14:45.298213+0000","last_clean":"2024-04-26T21:14:45.298213+0000","last_became_active":"2024-04-26T21:14:44.260195+0000","last_became_peered":"2024-04-26T21:14:44.260195+0000","last_unstale":"2024-04-26T21:14:45.298213+0000","last_undegraded":"2024-04-26T21:14:45.298213+0000","last_fullsized":"2024-04-26T21:14:45.298213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:58.018878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.245702+0000","last_change":"2024-04-26T21:14:43.246304+0000","last_active":"2024-04-26T21:14:45.245702+0000","last_peered":"2024-04-26T21:14:45.245702+0000","last_clean":"2024-04-26T21:14:45.245702+0000","last_became_active":"2024-04-26T21:14:43.246042+0000","last_became_peered":"2024-04-26T21:14:43.246042+0000","last_unstale":"2024-04-26T21:14:45.245702+0000","last_undegraded":"2024-04-26T21:14:45.245702+0000","last_fullsized":"2024-04-26T21:14:45.245702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:55.866035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601249+0000","last_change":"2024-04-26T21:14:44.247167+0000","last_active":"2024-04-26T21:14:45.601249+0000","last_peered":"2024-04-26T21:14:45.601249+0000","last_clean":"2024-04-26T21:14:45.601249+0000","last_became_active":"2024-04-26T21:14:44.246911+0000","last_became_peered":"2024-04-26T21:14:44.246911+0000","last_unstale":"2024-04-26T21:14:45.601249+0000","last_undegraded":"2024-04-26T21:14:45.601249+0000","last_fullsized":"2024-04-26T21:14:45.601249+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:31.466228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601159+0000","last_change":"2024-04-26T21:14:43.241399+0000","last_active":"2024-04-26T21:14:45.601159+0000","last_peered":"2024-04-26T21:14:45.601159+0000","last_clean":"2024-04-26T21:14:45.601159+0000","last_became_active":"2024-04-26T21:14:43.241031+0000","last_became_peered":"2024-04-26T21:14:43.241031+0000","last_unstale":"2024-04-26T21:14:45.601159+0000","last_undegraded":"2024-04-26T21:14:45.601159+0000","last_fullsized":"2024-04-26T21:14:45.601159+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:41.265024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.601160+0000","last_change":"2024-04-26T21:14:44.254356+0000","last_active":"2024-04-26T21:14:45.601160+0000","last_peered":"2024-04-26T21:14:45.601160+0000","last_clean":"2024-04-26T21:14:45.601160+0000","last_became_active":"2024-04-26T21:14:44.253669+0000","last_became_peered":"2024-04-26T21:14:44.253669+0000","last_unstale":"2024-04-26T21:14:45.601160+0000","last_undegraded":"2024-04-26T21:14:45.601160+0000","last_fullsized":"2024-04-26T21:14:45.601160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:45.401321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602433+0000","last_change":"2024-04-26T21:14:43.239309+0000","last_active":"2024-04-26T21:14:45.602433+0000","last_peered":"2024-04-26T21:14:45.602433+0000","last_clean":"2024-04-26T21:14:45.602433+0000","last_became_active":"2024-04-26T21:14:43.238992+0000","last_became_peered":"2024-04-26T21:14:43.238992+0000","last_unstale":"2024-04-26T21:14:45.602433+0000","last_undegraded":"2024-04-26T21:14:45.602433+0000","last_fullsized":"2024-04-26T21:14:45.602433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:05.495464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.602491+0000","last_change":"2024-04-26T21:14:44.250024+0000","last_active":"2024-04-26T21:14:45.602491+0000","last_peered":"2024-04-26T21:14:45.602491+0000","last_clean":"2024-04-26T21:14:45.602491+0000","last_became_active":"2024-04-26T21:14:44.249737+0000","last_became_peered":"2024-04-26T21:14:44.249737+0000","last_unstale":"2024-04-26T21:14:45.602491+0000","last_undegraded":"2024-04-26T21:14:45.602491+0000","last_fullsized":"2024-04-26T21:14:45.602491+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:43.220241+0000","last_clean_scrub_stamp":"2024-04-26T21:14:43.220241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:03.159285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.244179+0000","last_change":"2024-04-26T21:14:43.248267+0000","last_active":"2024-04-26T21:14:45.244179+0000","last_peered":"2024-04-26T21:14:45.244179+0000","last_clean":"2024-04-26T21:14:45.244179+0000","last_became_active":"2024-04-26T21:14:43.247936+0000","last_became_peered":"2024-04-26T21:14:43.247936+0000","last_unstale":"2024-04-26T21:14:45.244179+0000","last_undegraded":"2024-04-26T21:14:45.244179+0000","last_fullsized":"2024-04-26T21:14:45.244179+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:14:42.210388+0000","last_clean_scrub_stamp":"2024-04-26T21:14:42.210388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:19.099279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T21:14:45.297808+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:14:45.297808+0000","last_peered":"2024-04-26T21:14:45.297808+0000","last_clean":"2024-04-26T21:14:45.297808+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:14:45.297808+0000","last_undegraded":"2024-04-26T21:14:45.297808+0000","last_fullsized":"2024-04-26T21:14:45.297808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1302528,"data_stored":879306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":288778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":290390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":288834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1310720,"data_stored":881340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":288868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:14:52.031 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:14:52.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:14:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.174+0000 7fd39f19d640 1 Processor -- start 2024-04-26T21:14:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 -- start start 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd3980cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd3980b4280 0x7fd39809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd39806daf0 con 0x7fd3980b57e0 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd39806d5c0 con 0x7fd3980b4280 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd39f19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd39806d810 con 0x7fd398162af0 2024-04-26T21:14:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd397fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd3980b57e0 0x7fd3980d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59470/0 (socket says 172.21.15.40:59470) 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.175+0000 7fd397fff640 1 -- 172.21.15.40:0/2483181942 learned_addr learned my addr 172.21.15.40:0/2483181942 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd39cf39640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd3980cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd39d73a640 1 --2- 172.21.15.40:0/2483181942 >> v2:172.21.15.184:3300/0 conn(0x7fd3980b4280 0x7fd39809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687984430 0 0) 0x7fd39806daf0 con 0x7fd3980b57e0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd388003520 con 0x7fd3980b57e0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1454401915 0 0) 0x7fd388003520 con 0x7fd3980b57e0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 msgr2=0x7fd3980cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd3980cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 >> v2:172.21.15.184:3300/0 conn(0x7fd3980b4280 msgr2=0x7fd39809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 --2- 172.21.15.40:0/2483181942 >> v2:172.21.15.184:3300/0 conn(0x7fd3980b4280 0x7fd39809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39806de60 con 0x7fd3980b57e0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd384001670 con 0x7fd3980b57e0 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 >> v1:172.21.15.40:6789/0 conn(0x7fd3980b57e0 legacy=0x7fd3980d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3880039b0 0x7fd388003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd388004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd388004d40 0x7fd3880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd39806d5c0 con 0x7fd388004d40 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd39806d810 con 0x7fd3880039b0 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd38800b9c0 con 0x7fd398162af0 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd39d73a640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd388004d40 0x7fd3880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd39cf39640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3880039b0 0x7fd388003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.176+0000 7fd397fff640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd388004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 -- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 msgr2=0x7fd388004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398162af0 0x7fd388004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 -- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3880039b0 msgr2=0x7fd388003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3880039b0 0x7fd388003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 -- 172.21.15.40:0/2483181942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd388003520 con 0x7fd388004d40 2024-04-26T21:14:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd39d73a640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd388004d40 0x7fd3880051c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd390009f90 tx=0x7fd388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438534bfae8089b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390009c10 con 0x7fd388004d40 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd390036070 con 0x7fd388004d40 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.177+0000 7fd3977fe640 1 -- 172.21.15.40:0/2483181942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390031840 con 0x7fd388004d40 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 -- 172.21.15.40:0/2483181942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd388004d40 msgr2=0x7fd3880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 --2- 172.21.15.40:0/2483181942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd388004d40 0x7fd3880051c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd390009f90 tx=0x7fd388005830 comp rx=0 tx=0).stop 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39d73a640 1 -- 172.21.15.40:0/2483181942 reap_dead start 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 -- 172.21.15.40:0/2483181942 shutdown_connections 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 -- 172.21.15.40:0/2483181942 >> 172.21.15.40:0/2483181942 conn(0x7fd3980b4ab0 msgr2=0x7fd3980b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 -- 172.21.15.40:0/2483181942 shutdown_connections 2024-04-26T21:14:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.178+0000 7fd39f19d640 1 -- 172.21.15.40:0/2483181942 wait complete. 2024-04-26T21:14:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.179+0000 7fd39f19d640 1 Processor -- start 2024-04-26T21:14:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.179+0000 7fd39f19d640 1 -- start start 2024-04-26T21:14:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 0x7fd3981596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 0x7fd398148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3980d25c0 con 0x7fd398161e60 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3980d2440 con 0x7fd398147cb0 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39f19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd398162ed0 con 0x7fd398159230 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51950/0 (socket says 172.21.15.40:51950) 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 -- 172.21.15.40:0/90494475 learned_addr learned my addr 172.21.15.40:0/90494475 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd397fff640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 0x7fd3981596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39d73a640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 0x7fd398148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 -- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 msgr2=0x7fd3981596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 0x7fd3981596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 -- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 msgr2=0x7fd398148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 0x7fd398148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39cf39640 1 -- 172.21.15.40:0/90494475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd398149270 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd397fff640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 0x7fd3981596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.180+0000 7fd39d73a640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 0x7fd398148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd39cf39640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0118e0 tx=0x7fd38c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00fa90 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38c00fc30 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd398149550 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c014440 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.181+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd398149b10 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.182+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3980693d0 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.182+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd38c012040 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.183+0000 7fd3957fa640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 0x7fd37c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.183+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd38c08c140 con 0x7fd398161e60 2024-04-26T21:14:52.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.183+0000 7fd397fff640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 0x7fd37c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.184+0000 7fd397fff640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 0x7fd37c06ede0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b8e50 tx=0x7fd384009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.187+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd38c019030 con 0x7fd398161e60 2024-04-26T21:14:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.453+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd398158680 con 0x7fd398161e60 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.455+0000 7fd3957fa640 1 -- 172.21.15.40:0/90494475 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fd38c05a600 con 0x7fd398161e60 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:14:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.457+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 msgr2=0x7fd37c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.457+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 0x7fd37c06ede0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b8e50 tx=0x7fd384009040 comp rx=0 tx=0).stop 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.457+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 msgr2=0x7fd3981588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.457+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0118e0 tx=0x7fd38c011da0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 shutdown_connections 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd37c06c920 0x7fd37c06ede0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd398147cb0 0x7fd398148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd398159230 0x7fd3981596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 --2- 172.21.15.40:0/90494475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd398161e60 0x7fd3981588c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 >> 172.21.15.40:0/90494475 conn(0x7fd3980b4ab0 msgr2=0x7fd3980b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 shutdown_connections 2024-04-26T21:14:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.458+0000 7fd39f19d640 1 -- 172.21.15.40:0/90494475 wait complete. 2024-04-26T21:14:52.469 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.569+0000 7f631e0bd640 1 Processor -- start 2024-04-26T21:14:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.569+0000 7f631e0bd640 1 -- start start 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f631e0bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6318145980 0x7f6318160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f631e0bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 0x7f63180b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f631e0bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63180b4260 con 0x7f63180b5fd0 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f631e0bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f63180b5140 con 0x7f6318145980 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f631e0bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f63180b6410 con 0x7f63180b3de0 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6318145980 0x7f6318160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6318145980 0x7f6318160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51954/0 (socket says 172.21.15.40:51954) 2024-04-26T21:14:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 -- 172.21.15.40:0/718185492 learned_addr learned my addr 172.21.15.40:0/718185492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f6316ffd640 1 --2- 172.21.15.40:0/718185492 >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 0x7f63180b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 -- 172.21.15.40:0/718185492 >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 msgr2=0x7f63180b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 --2- 172.21.15.40:0/718185492 >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 0x7f63180b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 -- 172.21.15.40:0/718185492 >> v1:172.21.15.184:6789/0 conn(0x7f63180b5fd0 legacy=0x7f631815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f63177fe640 1 -- 172.21.15.40:0/718185492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63180b4530 con 0x7f6318145980 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.570+0000 7f6316ffd640 1 --2- 172.21.15.40:0/718185492 >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 0x7f63180b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.571+0000 7f63177fe640 1 --2- 172.21.15.40:0/718185492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f6318160630 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f63180d0d00 tx=0x7f630802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbdd518d2eec6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.571+0000 7f63167fc640 1 -- 172.21.15.40:0/718185492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308032710 con 0x7f6318145980 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.571+0000 7f63167fc640 1 -- 172.21.15.40:0/718185492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63080328b0 con 0x7f6318145980 2024-04-26T21:14:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.571+0000 7f63167fc640 1 -- 172.21.15.40:0/718185492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308032ca0 con 0x7f6318145980 2024-04-26T21:14:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 -- 172.21.15.40:0/718185492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 msgr2=0x7f6318160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 --2- 172.21.15.40:0/718185492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f6318160630 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f63180d0d00 tx=0x7f630802c850 comp rx=0 tx=0).stop 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 -- 172.21.15.40:0/718185492 shutdown_connections 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 --2- 172.21.15.40:0/718185492 >> v2:172.21.15.184:3300/0 conn(0x7f63180b3de0 0x7f63180b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 --2- 172.21.15.40:0/718185492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f6318160630 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 -- 172.21.15.40:0/718185492 >> 172.21.15.40:0/718185492 conn(0x7f63180d0d30 msgr2=0x7f63180d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 -- 172.21.15.40:0/718185492 shutdown_connections 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.572+0000 7f631e0bd640 1 -- 172.21.15.40:0/718185492 wait complete. 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.573+0000 7f631e0bd640 1 Processor -- start 2024-04-26T21:14:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.573+0000 7f631e0bd640 1 -- start start 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63180b3de0 0x7f63181430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63180b5fd0 0x7f63181f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63180b4dd0 con 0x7f6318145980 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63180b4af0 con 0x7f63180b5fd0 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f631e0bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63180730e0 con 0x7f63180b3de0 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51966/0 (socket says 172.21.15.40:51966) 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 -- 172.21.15.40:0/980981460 learned_addr learned my addr 172.21.15.40:0/980981460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6316ffd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63180b5fd0 0x7f63181f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 -- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63180b3de0 msgr2=0x7f63181430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63180b3de0 0x7f63181430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 -- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63180b5fd0 msgr2=0x7f63181f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63180b5fd0 0x7f63181f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.574+0000 7f6317fff640 1 -- 172.21.15.40:0/980981460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63181f5cc0 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f6317fff640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f631000c780 tx=0x7f631000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310004b60 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6310004d00 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63181f5fa0 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000aa90 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.575+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63180aecc0 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.576+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318145150 con 0x7f6318145980 2024-04-26T21:14:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.576+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f631000ac30 con 0x7f6318145980 2024-04-26T21:14:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.577+0000 7f62fffff640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 0x7f630406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.577+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f631008bdc0 con 0x7f6318145980 2024-04-26T21:14:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.577+0000 7f63177fe640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 0x7f630406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.577+0000 7f63177fe640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 0x7f630406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6308037000 tx=0x7f6308031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.581+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6318145150 con 0x7f6318145980 2024-04-26T21:14:52.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.807+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f631806a290 con 0x7f6318145980 2024-04-26T21:14:52.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.807+0000 7f62fffff640 1 -- 172.21.15.40:0/980981460 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7f6310020070 con 0x7f6318145980 2024-04-26T21:14:52.809 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:52.809 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:14:52.809 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.809+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 msgr2=0x7f630406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 0x7f630406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6308037000 tx=0x7f6308031000 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 msgr2=0x7f631813c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f631000c780 tx=0x7f631000cc40 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 shutdown_connections 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f630406c8f0 0x7f630406edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6318145980 0x7f631813c450 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63180b5fd0 0x7f63181f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 --2- 172.21.15.40:0/980981460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63180b3de0 0x7f63181430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 >> 172.21.15.40:0/980981460 conn(0x7f63180d0d30 msgr2=0x7f63180b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:52.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 shutdown_connections 2024-04-26T21:14:52.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:52.810+0000 7f631e0bd640 1 -- 172.21.15.40:0/980981460 wait complete. 2024-04-26T21:14:53.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:14:43.607278+0000', 'modified': '2024-04-26T21:14:45.595782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6024': {'gid': 6024, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6834/555844344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 555844344}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 555844344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6024, 'qdb_cluster': [6024]} 2024-04-26T21:14:53.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:14:53.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:14:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.329+0000 7ff9fe133640 1 Processor -- start 2024-04-26T21:14:53.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9fe133640 1 -- start start 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9fe133640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9f815f370 0x7ff9f80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9fe133640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f80d03a0 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9fe133640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f80d0680 con 0x7ff9f815f750 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9fe133640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80d0960 con 0x7ff9f815f370 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9f7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff9f80d3ee0 0x7ff9f80b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59482/0 (socket says 172.21.15.40:59482) 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.330+0000 7ff9f7fff640 1 -- 172.21.15.40:0/854734513 learned_addr learned my addr 172.21.15.40:0/854734513 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17219832 0 0) 0x7ff9f80d03a0 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/854734513 >> v2:172.21.15.184:3300/0 conn(0x7ff9f815f370 0x7ff9f80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9e0003520 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 956683224 0 0) 0x7ff9e0003520 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 >> v2:172.21.15.184:3300/0 conn(0x7ff9f815f370 msgr2=0x7ff9f80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 --2- 172.21.15.40:0/854734513 >> v2:172.21.15.184:3300/0 conn(0x7ff9f815f370 0x7ff9f80cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 >> v1:172.21.15.184:6789/0 conn(0x7ff9f815f750 legacy=0x7ff9f80d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f80d0c40 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff9f00019f0 con 0x7ff9f80d3ee0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 >> v1:172.21.15.40:6789/0 conn(0x7ff9f80d3ee0 legacy=0x7ff9f80b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e0004800 0x7ff9e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e00051c0 0x7ff9e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80d0960 con 0x7ff9e00039b0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e000c060 con 0x7ff9e0004800 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e000c260 con 0x7ff9e00051c0 2024-04-26T21:14:53.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e0004800 0x7ff9e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e00051c0 0x7ff9e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 -- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e00051c0 msgr2=0x7ff9e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e00051c0 0x7ff9e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 -- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e0004800 msgr2=0x7ff9e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.331+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e0004800 0x7ff9e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f77fe640 1 -- 172.21.15.40:0/854734513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e0003520 con 0x7ff9e00039b0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9e00051c0 0x7ff9e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9e0004800 0x7ff9e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f7fff640 1 -- 172.21.15.40:0/854734513 reap_dead start 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9e0003dd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ac0 tx=0x7ff9e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cdd84f6d318e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8036070 con 0x7ff9e00039b0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8031400 con 0x7ff9e00039b0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.332+0000 7ff9f67fc640 1 -- 172.21.15.40:0/854734513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8009690 con 0x7ff9e00039b0 2024-04-26T21:14:53.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 -- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 msgr2=0x7ff9e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9e0003dd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ac0 tx=0x7ff9e0005830 comp rx=0 tx=0).stop 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 -- 172.21.15.40:0/854734513 shutdown_connections 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 --2- 172.21.15.40:0/854734513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9e0003dd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 -- 172.21.15.40:0/854734513 >> 172.21.15.40:0/854734513 conn(0x7ff9f808ec10 msgr2=0x7ff9f809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 -- 172.21.15.40:0/854734513 shutdown_connections 2024-04-26T21:14:53.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.333+0000 7ff9fe133640 1 -- 172.21.15.40:0/854734513 wait complete. 2024-04-26T21:14:53.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.334+0000 7ff9fe133640 1 Processor -- start 2024-04-26T21:14:53.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.334+0000 7ff9fe133640 1 -- start start 2024-04-26T21:14:53.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8158610 0x7ff9f80b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 0x7ff9f814bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80cfef0 con 0x7ff9e00039b0 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80cfd70 con 0x7ff9f8158610 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9fe133640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f80d0070 con 0x7ff9f814b830 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51992/0 (socket says 172.21.15.40:51992) 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2761947271 learned_addr learned my addr 172.21.15.40:0/2761947271 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8158610 0x7ff9f80b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 0x7ff9f814bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 msgr2=0x7ff9f814bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 0x7ff9f814bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8158610 msgr2=0x7ff9f80b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8158610 0x7ff9f80b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2761947271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f81611c0 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.335+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 0x7ff9f814bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034040 tx=0x7ff9e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9dffff640 1 -- 172.21.15.40:0/2761947271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e803f020 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9dffff640 1 -- 172.21.15.40:0/2761947271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8037040 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f81614a0 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9dffff640 1 -- 172.21.15.40:0/2761947271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8036070 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.336+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f8161a60 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.337+0000 7ff9dffff640 1 -- 172.21.15.40:0/2761947271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9e8042020 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.338+0000 7ff9dffff640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 0x7ff9e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.338+0000 7ff9dffff640 1 -- 172.21.15.40:0/2761947271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e80abbd0 con 0x7ff9e00039b0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.338+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 0x7ff9e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.338+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 0x7ff9e406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.339+0000 7ffa00e4c740 1 Processor -- start 2024-04-26T21:14:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.339+0000 7ffa00e4c740 1 -- start start 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.339+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34229820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.339+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e34247230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.339+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558e34163740 con 0x558e341ff120 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558e340ff3a0 con 0x558e34229440 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558e3415f700 con 0x558e34246ce0 2024-04-26T21:14:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52006/0 (socket says 172.21.15.40:52006) 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 -- 172.21.15.40:0/3236710305 learned_addr learned my addr 172.21.15.40:0/3236710305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34229820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e34247230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 -- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 msgr2=0x558e34247230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e34247230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 -- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 msgr2=0x558e34229820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34229820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 -- 172.21.15.40:0/3236710305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3411b750 con 0x558e341ff120 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34229820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8163080 tx=0x7ff9f000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9194310be9f1b257 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9dd7fa640 1 -- 172.21.15.40:0/3236710305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0018070 con 0x558e341ff120 2024-04-26T21:14:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9dd7fa640 1 -- 172.21.15.40:0/3236710305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f0004b60 con 0x558e341ff120 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.340+0000 7ff9dd7fa640 1 -- 172.21.15.40:0/3236710305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0007110 con 0x558e341ff120 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 -- 172.21.15.40:0/3236710305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 msgr2=0x558e342453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8163080 tx=0x7ff9f000e590 comp rx=0 tx=0).stop 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 -- 172.21.15.40:0/3236710305 shutdown_connections 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e341ff120 0x558e342453a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e34247230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/3236710305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34229820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 -- 172.21.15.40:0/3236710305 >> 172.21.15.40:0/3236710305 conn(0x558e341fba50 msgr2=0x558e34244250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 -- 172.21.15.40:0/3236710305 shutdown_connections 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.341+0000 7ffa00e4c740 1 -- 172.21.15.40:0/3236710305 wait complete. 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.342+0000 7ffa00e4c740 1 -- start start 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.342+0000 7ffa00e4c740 1 Processor -- start 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.342+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.342+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e342431f0 0x558e34245710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558e341ffec0 con 0x558e34229440 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558e340eafe0 con 0x558e342431f0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558e341f9980 con 0x558e34246ce0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 10 client.0 _ll_register_callbacks cb 0x558e34101150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52016/0 (socket says 172.21.15.40:52016) 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 -- 172.21.15.40:0/366526666 learned_addr learned my addr 172.21.15.40:0/366526666 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e342431f0 0x558e34245710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 -- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 msgr2=0x558e342451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 -- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e342431f0 msgr2=0x558e34245710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e342431f0 0x558e34245710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 -- 172.21.15.40:0/366526666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e33e752d0 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ac0 tx=0x7ff9e807dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a3d83c65068e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8094020 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e8082100 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f0004b60 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9e80a7070 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 10 client.6198 fetch_fsmap learned FSMap version 29 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x558e33f3f6b0 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.343+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f0004b60 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7ff9e807ace0 con 0x558e34229440 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 10 client.6198 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 10 client.6198 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 20 client.6198 populate_metadata read hostname 'smithi040' 2024-04-26T21:14:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e342607b0 0x558e34262ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 4 client.6198 mds_command: new command op to 6024 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 --> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e34171350 con 0x558e342607b0 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e342607b0 0x558e34262ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ff9d67fc640 10 client.6198 ms_handle_connect on v2:172.21.15.40:6832/555844344 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.344+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e342607b0 0x558e34262ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00777c63475b2fc server_cookie=69400d283340f9c4 in_seq=0 out_seq=0 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ff9d67fc640 1 -- 172.21.15.40:0/366526666 <== mds.0 v2:172.21.15.40:6832/555844344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e34171350 con 0x558e342607b0 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ff9d67fc640 10 client.6198 handle_command_reply: tid=0 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 client.6198 shutdown 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 msgr2=0x558e34215200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ac0 tx=0x7ff9e807dea0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 shutdown_connections 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e342607b0 0x558e34262ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e342431f0 0x558e34245710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ff9d67fc640 0 client.6198 ms_handle_reset on v2:172.21.15.40:6832/555844344 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/366526666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34229440 0x558e34215200 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 >> 172.21.15.40:0/366526666 conn(0x558e341ff120 msgr2=0x558e342477d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 shutdown_connections 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 1 -- 172.21.15.40:0/366526666 wait complete. 2024-04-26T21:14:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.345+0000 7ffa00e4c740 20 client.6198 trim_cache size 0 max 16384 2024-04-26T21:14:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.363+0000 7ffa00e4c740 1 Processor -- start 2024-04-26T21:14:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.363+0000 7ffa00e4c740 1 -- start start 2024-04-26T21:14:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34254550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f82010a0 con 0x558e342431f0 2024-04-26T21:14:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80ba5d0 con 0x558e34229440 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f8200a80 con 0x558e34246ce0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52024/0 (socket says 172.21.15.40:52024) 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.364+0000 7ff9f6ffd640 1 -- 172.21.15.40:0/2147372217 learned_addr learned my addr 172.21.15.40:0/2147372217 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 -- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 msgr2=0x558e342435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34254550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 -- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 msgr2=0x558e34254550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34254550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 -- 172.21.15.40:0/2147372217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3423c1d0 con 0x558e342431f0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec007fd0 tx=0x7ff9ec00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1ea9364f7794ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9dcff9640 1 -- 172.21.15.40:0/2147372217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec030070 con 0x558e342431f0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9dcff9640 1 -- 172.21.15.40:0/2147372217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec007230 con 0x558e342431f0 2024-04-26T21:14:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.365+0000 7ff9dcff9640 1 -- 172.21.15.40:0/2147372217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0075e0 con 0x558e342431f0 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2147372217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 msgr2=0x558e341f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec007fd0 tx=0x7ff9ec00c000 comp rx=0 tx=0).stop 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2147372217 shutdown_connections 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e34246ce0 0x558e342435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e342431f0 0x558e341f9030 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2147372217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e34254550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2147372217 >> 172.21.15.40:0/2147372217 conn(0x558e341ff120 msgr2=0x558e342450d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2147372217 shutdown_connections 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2147372217 wait complete. 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 -- start start 2024-04-26T21:14:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.366+0000 7ffa00e4c740 1 Processor -- start 2024-04-26T21:14:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e341fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e342431f0 0x558e342b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558e341ffec0 con 0x558e34246ce0 2024-04-26T21:14:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558e340eafe0 con 0x558e34229440 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x558e34225a40 con 0x558e342431f0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ffa00e4c740 10 client.0 _ll_register_callbacks cb 0x7ff9e80799b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52036/0 (socket says 172.21.15.40:52036) 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2931993528 learned_addr learned my addr 172.21.15.40:0/2931993528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e342431f0 0x558e342b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.367+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e341fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e342431f0 msgr2=0x558e342b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e342431f0 0x558e342b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 msgr2=0x558e341fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e341fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2931993528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e342b5010 con 0x558e34246ce0 2024-04-26T21:14:53.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c000 tx=0x7ff9f80d0570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8655595a81949b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0024020 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff9f00075e0 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec007230 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9f002e050 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 10 client.6204 fetch_fsmap learned FSMap version 29 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x558e341ff8c0 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec007230 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7ff9f0031020 con 0x558e34246ce0 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 10 client.6204 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 10 client.6204 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 20 client.6204 populate_metadata read hostname 'smithi040' 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e34267210 0x558e34248840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 4 client.6204 mds_command: new command op to 6024 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.368+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 --> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] -- command(tid 0: {"prefix": "status"}) -- 0x558e3416b1c0 con 0x558e34267210 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e34267210 0x558e34248840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9d67fc640 10 client.6204 ms_handle_connect on v2:172.21.15.40:6832/555844344 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e34267210 0x558e34248840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2a131389f0352f server_cookie=690c489726b3f4a5 in_seq=0 out_seq=0 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9d67fc640 1 -- 172.21.15.40:0/2931993528 <== mds.0 v2:172.21.15.40:6832/555844344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558e3416b1c0 con 0x558e34267210 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9d67fc640 10 client.6204 handle_command_reply: tid=0 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 client.6204 shutdown 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 msgr2=0x558e34227ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c000 tx=0x7ff9f80d0570 comp rx=0 tx=0).stop 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 shutdown_connections 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e34246ce0 0x558e34227ca0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558e342431f0 0x558e342b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.40:6832/555844344,v1:172.21.15.40:6834/555844344] conn(0x558e34267210 0x558e34248840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 --2- 172.21.15.40:0/2931993528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e34229440 0x558e341fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 >> 172.21.15.40:0/2931993528 conn(0x558e341ff120 msgr2=0x558e34268370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ff9d67fc640 0 client.6204 ms_handle_reset on v2:172.21.15.40:6832/555844344 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 shutdown_connections 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 1 -- 172.21.15.40:0/2931993528 wait complete. 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.369+0000 7ffa00e4c740 20 client.6204 trim_cache size 0 max 16384 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6024, 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:14:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 9.1423726260000002, 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 29, 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 59, 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 59, 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 20.169577923999999 2024-04-26T21:14:53.378 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 msgr2=0x7ff9e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 0x7ff9e406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec009040 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 msgr2=0x7ff9f8160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034040 tx=0x7ff9e8004730 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 shutdown_connections 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff9e406c8f0 0x7ff9e406edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9f814b830 0x7ff9f814bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9f8158610 0x7ff9f80b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 --2- 172.21.15.40:0/2761947271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00039b0 0x7ff9f8160aa0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 >> 172.21.15.40:0/2761947271 conn(0x7ff9f808ec10 msgr2=0x7ff9f80b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 shutdown_connections 2024-04-26T21:14:53.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.370+0000 7ff9fe133640 1 -- 172.21.15.40:0/2761947271 wait complete. 2024-04-26T21:14:53.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:14:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.482+0000 7fb2cd738640 1 Processor -- start 2024-04-26T21:14:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.482+0000 7fb2cd738640 1 -- start start 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2cd738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 0x7fb2c80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2cd738640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2c8160d50 0x7fb2c81611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2cd738640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c80d0be0 con 0x7fb2c80b56d0 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2cd738640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c80d0ec0 con 0x7fb2c8160d50 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2cd738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c80d11a0 con 0x7fb2c815d380 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2c77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2c8160d50 0x7fb2c81611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2c6ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 0x7fb2c80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2bffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb2c80b56d0 0x7fb2c80d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41718/0 (socket says 172.21.15.40:41718) 2024-04-26T21:14:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.483+0000 7fb2bffff640 1 -- 172.21.15.40:0/3342383040 learned_addr learned my addr 172.21.15.40:0/3342383040 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c67fc640 1 -- 172.21.15.40:0/3342383040 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814471321 0 0) 0x7fb2c80d0be0 con 0x7fb2c80b56d0 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c67fc640 1 -- 172.21.15.40:0/3342383040 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b4003520 con 0x7fb2c80b56d0 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c77fe640 1 -- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 msgr2=0x7fb2c80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c77fe640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 0x7fb2c80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c77fe640 1 -- 172.21.15.40:0/3342383040 >> v1:172.21.15.184:6789/0 conn(0x7fb2c80b56d0 legacy=0x7fb2c80d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c77fe640 1 -- 172.21.15.40:0/3342383040 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80d1480 con 0x7fb2c8160d50 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.484+0000 7fb2c6ffd640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 0x7fb2c80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2c77fe640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c8160d50 0x7fb2c81611e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf52782681894ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2c67fc640 1 -- 172.21.15.40:0/3342383040 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0031600 con 0x7fb2c8160d50 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2c67fc640 1 -- 172.21.15.40:0/3342383040 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c00317a0 con 0x7fb2c8160d50 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2c67fc640 1 -- 172.21.15.40:0/3342383040 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0031b70 con 0x7fb2c8160d50 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 -- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c8160d50 msgr2=0x7fb2c81611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c8160d50 0x7fb2c81611e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 -- 172.21.15.40:0/3342383040 shutdown_connections 2024-04-26T21:14:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c8160d50 0x7fb2c81611e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 --2- 172.21.15.40:0/3342383040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c815d380 0x7fb2c80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 -- 172.21.15.40:0/3342383040 >> 172.21.15.40:0/3342383040 conn(0x7fb2c80b4a60 msgr2=0x7fb2c80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 -- 172.21.15.40:0/3342383040 shutdown_connections 2024-04-26T21:14:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.485+0000 7fb2cd738640 1 -- 172.21.15.40:0/3342383040 wait complete. 2024-04-26T21:14:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.486+0000 7fb2cd738640 1 Processor -- start 2024-04-26T21:14:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.486+0000 7fb2cd738640 1 -- start start 2024-04-26T21:14:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c80b56d0 0x7fb2c80c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c815d380 0x7fb2c81f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80b5dc0 con 0x7fb2c8160d50 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80cedc0 con 0x7fb2c815d380 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2cd738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c80b5f40 con 0x7fb2c80b56d0 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52038/0 (socket says 172.21.15.40:52038) 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2c77fe640 1 -- 172.21.15.40:0/606366509 learned_addr learned my addr 172.21.15.40:0/606366509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2c6ffd640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c80b56d0 0x7fb2c80c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.487+0000 7fb2bffff640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c815d380 0x7fb2c81f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 -- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c80b56d0 msgr2=0x7fb2c80c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c80b56d0 0x7fb2c80c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 -- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c815d380 msgr2=0x7fb2c81f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c815d380 0x7fb2c81f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 -- 172.21.15.40:0/606366509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80c8100 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2c77fe640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0040070 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c003c830 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c80c7860 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c003cbe0 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.488+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c80c7e20 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.489+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c806a290 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.490+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2c0039040 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.490+0000 7fb2bf7fe640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 0x7fb2a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.490+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c00acaa0 con 0x7fb2c8160d50 2024-04-26T21:14:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.490+0000 7fb2c6ffd640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 0x7fb2a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.491+0000 7fb2c6ffd640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 0x7fb2a806ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2c815a690 tx=0x7fb2b8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:53.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.494+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2c007a910 con 0x7fb2c8160d50 2024-04-26T21:14:53.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.720+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb2c8070bd0 con 0x7fb2c8160d50 2024-04-26T21:14:53.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.720+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/606366509 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7fb2c007ec10 con 0x7fb2c8160d50 2024-04-26T21:14:53.721 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 msgr2=0x7fb2a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 0x7fb2a806ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2c815a690 tx=0x7fb2b8009190 comp rx=0 tx=0).stop 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 msgr2=0x7fb2c81f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c0004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 shutdown_connections 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a806c920 0x7fb2a806ede0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c8160d50 0x7fb2c81f3b00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c815d380 0x7fb2c81f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 --2- 172.21.15.40:0/606366509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c80b56d0 0x7fb2c80c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 >> 172.21.15.40:0/606366509 conn(0x7fb2c80b4a60 msgr2=0x7fb2c80b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 shutdown_connections 2024-04-26T21:14:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53.722+0000 7fb2cd738640 1 -- 172.21.15.40:0/606366509 wait complete. 2024-04-26T21:14:53.733 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:14:53.734 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:53.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:53.734 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:14:53.750 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:14:53.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:53.751 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42576sec preferred_lft 42576sec 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:53.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:53.753 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:14:53.813 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:14:53.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:53.815 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:14:53.871 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:14:53.872 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:14:53.872 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:14:53.872 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:14:53.872 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:14:53.872 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:14:53.872 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:14:53.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:14:53.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:14:53.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:14:53.873 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:14:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:14:53.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:53.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:53.998 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:14:54.014 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:14:54.014 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:14:54.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:54.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.014 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:14:54.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:54.078 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:54.134 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:14:54.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:54.135 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:54.190 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:14:54.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:54.272 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:54.272 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:54.272 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:54.281 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:54.281 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:54.281 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:54.281 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:54.281 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:54.290 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:14:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:14:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:14:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:14:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:14:54.293 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:14:54.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:54.357 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:14:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:14:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:54.459 INFO:teuthology.orchestra.run.smithi040.stdout:client5713 2024-04-26T21:14:54.459 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:54.459 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:54.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:14:54.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.614+0000 7f6cf96cc640 1 Processor -- start 2024-04-26T21:14:54.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.614+0000 7f6cf96cc640 1 -- start start 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf96cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6cf40d3320 0x7f6cf40b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf96cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf40d2440 con 0x7f6cf415f750 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf96cc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf40d2720 con 0x7f6cf40d51a0 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf96cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf40d0030 con 0x7f6cf40d3320 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf27fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6cf40d3320 0x7f6cf40b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6cf415f750 0x7f6cf40d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59490/0 (socket says 172.21.15.40:59490) 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.615+0000 7f6cf2ffd640 1 -- 172.21.15.40:0/1105538464 learned_addr learned my addr 172.21.15.40:0/1105538464 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf27fc640 1 -- 172.21.15.40:0/1105538464 >> v1:172.21.15.184:6789/0 conn(0x7f6cf40d51a0 legacy=0x7f6cf40d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf27fc640 1 -- 172.21.15.40:0/1105538464 >> v1:172.21.15.40:6789/0 conn(0x7f6cf415f750 legacy=0x7f6cf40d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf27fc640 1 -- 172.21.15.40:0/1105538464 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf40d2a00 con 0x7f6cf40d3320 2024-04-26T21:14:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf1ffb640 1 -- 172.21.15.40:0/1105538464 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352191735 0 0) 0x7f6cf40d2440 con 0x7f6cf415f750 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf27fc640 1 --2- 172.21.15.40:0/1105538464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf40d3320 0x7f6cf40b50e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c66a8307dbf375e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.616+0000 7f6cf1ffb640 1 -- 172.21.15.40:0/1105538464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc037070 con 0x7f6cf40d3320 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf1ffb640 1 -- 172.21.15.40:0/1105538464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc032440 con 0x7f6cf40d3320 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf1ffb640 1 -- 172.21.15.40:0/1105538464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc032810 con 0x7f6cf40d3320 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1105538464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf40d3320 msgr2=0x7f6cf40b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1105538464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf40d3320 0x7f6cf40b50e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1105538464 shutdown_connections 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1105538464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf40d3320 0x7f6cf40b50e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1105538464 >> 172.21.15.40:0/1105538464 conn(0x7f6cf408ec20 msgr2=0x7f6cf409e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1105538464 shutdown_connections 2024-04-26T21:14:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.617+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1105538464 wait complete. 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.618+0000 7f6cf96cc640 1 Processor -- start 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.618+0000 7f6cf96cc640 1 -- start start 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 0x7f6cf4161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 0x7f6cf40b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40d5a20 con 0x7f6cf415f750 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40d58a0 con 0x7f6cf40d3320 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf96cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf40d0bd0 con 0x7f6cf40d51a0 2024-04-26T21:14:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52054/0 (socket says 172.21.15.40:52054) 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf37fe640 1 -- 172.21.15.40:0/1147007147 learned_addr learned my addr 172.21.15.40:0/1147007147 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf2ffd640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 0x7f6cf4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.619+0000 7f6cf27fc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 0x7f6cf40b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 -- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 msgr2=0x7f6cf40b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 0x7f6cf40b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 -- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 msgr2=0x7f6cf4161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 0x7f6cf4161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 -- 172.21.15.40:0/1147007147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf40b2d40 con 0x7f6cf415f750 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf27fc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 0x7f6cf40b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf2ffd640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 0x7f6cf4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf37fe640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c780 tx=0x7f6ce400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4017070 con 0x7f6cf415f750 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4003da0 con 0x7f6cf415f750 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40052b0 con 0x7f6cf415f750 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf40b33e0 con 0x7f6cf415f750 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.620+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf40b39a0 con 0x7f6cf415f750 2024-04-26T21:14:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.621+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40693d0 con 0x7f6cf415f750 2024-04-26T21:14:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.621+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ce401a070 con 0x7f6cf415f750 2024-04-26T21:14:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.622+0000 7f6ceb7fe640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 0x7f6cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.622+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6ce408b1c0 con 0x7f6cf415f750 2024-04-26T21:14:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.622+0000 7f6cf2ffd640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 0x7f6cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.623+0000 7f6cf2ffd640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 0x7f6cd406edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6cec000f60 tx=0x7f6cec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.626+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ce40595f0 con 0x7f6cf415f750 2024-04-26T21:14:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.851+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6cf415ad00 con 0x7f6cf415f750 2024-04-26T21:14:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.851+0000 7f6ceb7fe640 1 -- 172.21.15.40:0/1147007147 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7f6ce4058ef0 con 0x7f6cf415f750 2024-04-26T21:14:54.852 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:14:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 msgr2=0x7f6cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 0x7f6cd406edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6cec000f60 tx=0x7f6cec009040 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 msgr2=0x7f6cf41630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c780 tx=0x7f6ce400cc40 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 shutdown_connections 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cd406c8f0 0x7f6cd406edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cf415f750 0x7f6cf41630d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cf40d51a0 0x7f6cf40b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 --2- 172.21.15.40:0/1147007147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf40d3320 0x7f6cf4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 >> 172.21.15.40:0/1147007147 conn(0x7f6cf408ec20 msgr2=0x7f6cf40d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 shutdown_connections 2024-04-26T21:14:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:54.853+0000 7f6cf96cc640 1 -- 172.21.15.40:0/1147007147 wait complete. 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:14:54.865 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:54.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.866 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:14:54.884 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:14:54.884 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:14:54.885 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.886 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:14:54.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.887 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:14:54.951 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:14:54.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:54.953 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:14:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:14:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:14:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:14:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:14:55.011 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:14:55.011 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:14:55.103 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:14:55.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:55.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:55.140 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:14:55.155 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:14:55.155 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:14:55.155 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:55.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:55.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:14:55.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:55.220 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:55.276 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:14:55.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:14:55.277 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:55.331 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:14:55.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:55.332 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:55.413 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:55.413 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:55.413 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:55.432 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:14:55.433 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:14:55.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:14:55.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:55.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:55.501 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:14:55.589 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:14:55.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:14:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:14:55.598 INFO:teuthology.orchestra.run.smithi184.stdout:client6219 2024-04-26T21:14:55.598 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:55.598 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:55.613 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:14:55.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:55.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:14:55.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:14:55.696 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:55.696 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:55.696 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:55.715 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:55.715 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:55.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:55.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.924+0000 7faa299f2640 1 Processor -- start 2024-04-26T21:14:55.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.924+0000 7faa299f2640 1 -- start start 2024-04-26T21:14:55.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa299f2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa240b56d0 0x7faa240d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa299f2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa24160d50 0x7faa241611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa299f2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa240d0be0 con 0x7faa2415d380 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa299f2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faa241616e0 con 0x7faa24160d50 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa299f2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faa240d0ec0 con 0x7faa240b56d0 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa24160d50 0x7faa241611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa22ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa2415d380 0x7faa240ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59496/0 (socket says 172.21.15.40:59496) 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa22ffd640 1 -- 172.21.15.40:0/3900106677 learned_addr learned my addr 172.21.15.40:0/3900106677 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa1bfff640 1 --2- 172.21.15.40:0/3900106677 >> v2:172.21.15.184:3300/0 conn(0x7faa240b56d0 0x7faa240d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa227fc640 1 -- 172.21.15.40:0/3900106677 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414573183 0 0) 0x7faa240d0be0 con 0x7faa2415d380 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa227fc640 1 -- 172.21.15.40:0/3900106677 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa10003520 con 0x7faa2415d380 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa237fe640 1 -- 172.21.15.40:0/3900106677 >> v2:172.21.15.184:3300/0 conn(0x7faa240b56d0 msgr2=0x7faa240d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa237fe640 1 --2- 172.21.15.40:0/3900106677 >> v2:172.21.15.184:3300/0 conn(0x7faa240b56d0 0x7faa240d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa237fe640 1 -- 172.21.15.40:0/3900106677 >> v1:172.21.15.40:6789/0 conn(0x7faa2415d380 legacy=0x7faa240ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:55.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.925+0000 7faa237fe640 1 -- 172.21.15.40:0/3900106677 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240d11a0 con 0x7faa24160d50 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.926+0000 7faa237fe640 1 --2- 172.21.15.40:0/3900106677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241611a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faa24073840 tx=0x7faa1c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b92c932156d1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.926+0000 7faa227fc640 1 -- 172.21.15.40:0/3900106677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007740 con 0x7faa24160d50 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.926+0000 7faa227fc640 1 -- 172.21.15.40:0/3900106677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0078e0 con 0x7faa24160d50 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.926+0000 7faa227fc640 1 -- 172.21.15.40:0/3900106677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007cb0 con 0x7faa24160d50 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 -- 172.21.15.40:0/3900106677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 msgr2=0x7faa241611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 --2- 172.21.15.40:0/3900106677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241611a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faa24073840 tx=0x7faa1c02a680 comp rx=0 tx=0).stop 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 -- 172.21.15.40:0/3900106677 shutdown_connections 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 --2- 172.21.15.40:0/3900106677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241611a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 --2- 172.21.15.40:0/3900106677 >> v2:172.21.15.184:3300/0 conn(0x7faa240b56d0 0x7faa240d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:55.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 -- 172.21.15.40:0/3900106677 >> 172.21.15.40:0/3900106677 conn(0x7faa240b4a60 msgr2=0x7faa240b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:55.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 -- 172.21.15.40:0/3900106677 shutdown_connections 2024-04-26T21:14:55.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.927+0000 7faa299f2640 1 -- 172.21.15.40:0/3900106677 wait complete. 2024-04-26T21:14:55.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.928+0000 7faa299f2640 1 Processor -- start 2024-04-26T21:14:55.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.928+0000 7faa299f2640 1 -- start start 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa240b56d0 0x7faa241f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2415d380 0x7faa240b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa240cedc0 con 0x7faa24160d50 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa240d1ee0 con 0x7faa2415d380 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa299f2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa240b5dc0 con 0x7faa240b56d0 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52074/0 (socket says 172.21.15.40:52074) 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 -- 172.21.15.40:0/123707447 learned_addr learned my addr 172.21.15.40:0/123707447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa22ffd640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa240b56d0 0x7faa241f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa1bfff640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2415d380 0x7faa240b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 -- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa240b56d0 msgr2=0x7faa241f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa240b56d0 0x7faa241f09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 -- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2415d380 msgr2=0x7faa240b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2415d380 0x7faa240b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.929+0000 7faa237fe640 1 -- 172.21.15.40:0/123707447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240c6e00 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa237fe640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faa1c02ac70 tx=0x7faa1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c03f070 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c031bc0 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa240c70e0 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c009450 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.930+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa240c76a0 con 0x7faa24160d50 2024-04-26T21:14:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.931+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2406a290 con 0x7faa24160d50 2024-04-26T21:14:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.932+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa1c044460 con 0x7faa24160d50 2024-04-26T21:14:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.932+0000 7faa20ff9640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 0x7faa0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.932+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7faa1c0ad080 con 0x7faa24160d50 2024-04-26T21:14:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.932+0000 7faa22ffd640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 0x7faa0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.933+0000 7faa22ffd640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 0x7faa0406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa14005880 tx=0x7faa14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:55.937+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa1c07aef0 con 0x7faa24160d50 2024-04-26T21:14:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.123+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa24072140 con 0x7faa24160d50 2024-04-26T21:14:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.125+0000 7faa20ff9640 1 -- 172.21.15.40:0/123707447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7faa1c07f1f0 con 0x7faa24160d50 2024-04-26T21:14:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:56.126 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 msgr2=0x7faa0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 0x7faa0406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa14005880 tx=0x7faa14009040 comp rx=0 tx=0).stop 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 msgr2=0x7faa241f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faa1c02ac70 tx=0x7faa1c004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 shutdown_connections 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faa0406c8f0 0x7faa0406edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa24160d50 0x7faa241f3660 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa2415d380 0x7faa240b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 --2- 172.21.15.40:0/123707447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa240b56d0 0x7faa241f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 >> 172.21.15.40:0/123707447 conn(0x7faa240b4a60 msgr2=0x7faa2415aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 shutdown_connections 2024-04-26T21:14:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.127+0000 7faa299f2640 1 -- 172.21.15.40:0/123707447 wait complete. 2024-04-26T21:14:56.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:41.855331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:42.545028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:56.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 Processor -- start 2024-04-26T21:14:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 -- start start 2024-04-26T21:14:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 0x7f9b0406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9b041638b0 0x7f9b0406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b041485f0 0x7f9b04073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9b04163e20 con 0x7f9b041485f0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9b040b59f0 con 0x7f9b041638b0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.242+0000 7f9b09f46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b04148390 con 0x7f9b04147fb0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b041485f0 0x7f9b04073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b041485f0 0x7f9b04073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52076/0 (socket says 172.21.15.40:52076) 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 -- 172.21.15.40:0/1059072335 learned_addr learned my addr 172.21.15.40:0/1059072335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b02ffd640 1 --2- 172.21.15.40:0/1059072335 >> v2:172.21.15.184:3300/0 conn(0x7f9b041638b0 0x7f9b0406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b037fe640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 0x7f9b0406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 -- 172.21.15.40:0/1059072335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 msgr2=0x7f9b0406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 0x7f9b0406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 -- 172.21.15.40:0/1059072335 >> v2:172.21.15.184:3300/0 conn(0x7f9b041638b0 msgr2=0x7f9b0406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 --2- 172.21.15.40:0/1059072335 >> v2:172.21.15.184:3300/0 conn(0x7f9b041638b0 0x7f9b0406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 -- 172.21.15.40:0/1059072335 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0406fe90 con 0x7f9b041485f0 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b037fe640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 0x7f9b0406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.243+0000 7f9b03fff640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b041485f0 0x7f9b04073150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009e50 tx=0x7f9afc02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd24c3b97ccb4f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b027fc640 1 -- 172.21.15.40:0/1059072335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc036070 con 0x7f9b041485f0 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b027fc640 1 -- 172.21.15.40:0/1059072335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc031600 con 0x7f9b041485f0 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b027fc640 1 -- 172.21.15.40:0/1059072335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0319d0 con 0x7f9b041485f0 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 -- 172.21.15.40:0/1059072335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b041485f0 msgr2=0x7f9b04073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b041485f0 0x7f9b04073150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009e50 tx=0x7f9afc02a640 comp rx=0 tx=0).stop 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 -- 172.21.15.40:0/1059072335 shutdown_connections 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b041485f0 0x7f9b04073150 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 --2- 172.21.15.40:0/1059072335 >> v2:172.21.15.184:3300/0 conn(0x7f9b041638b0 0x7f9b0406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 --2- 172.21.15.40:0/1059072335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b04147fb0 0x7f9b0406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 -- 172.21.15.40:0/1059072335 >> 172.21.15.40:0/1059072335 conn(0x7f9b040d1e90 msgr2=0x7f9b040d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 -- 172.21.15.40:0/1059072335 shutdown_connections 2024-04-26T21:14:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.244+0000 7f9b09f46640 1 -- 172.21.15.40:0/1059072335 wait complete. 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.245+0000 7f9b09f46640 1 Processor -- start 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 -- start start 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b041485f0 0x7f9b04162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b041638b0 0x7f9b04160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04148cd0 con 0x7f9b04147fb0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04070450 con 0x7f9b041485f0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b09f46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b04148e50 con 0x7f9b041638b0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.246+0000 7f9b037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52092/0 (socket says 172.21.15.40:52092) 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 -- 172.21.15.40:0/3444268758 learned_addr learned my addr 172.21.15.40:0/3444268758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b03fff640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b041638b0 0x7f9b04160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b02ffd640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b041485f0 0x7f9b04162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 -- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b041638b0 msgr2=0x7f9b04160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b041638b0 0x7f9b04160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 -- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b041485f0 msgr2=0x7f9b04162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b041485f0 0x7f9b04162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 -- 172.21.15.40:0/3444268758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041eaa80 con 0x7f9b04147fb0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b037fe640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c7c0 tx=0x7f9af400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4015070 con 0x7f9b04147fb0 2024-04-26T21:14:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af4007a30 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041ead60 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4007e00 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.247+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041eb600 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.248+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0406c2a0 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.249+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9af4018070 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.249+0000 7f9aebfff640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.249+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9af408bab0 con 0x7f9b04147fb0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.249+0000 7f9b02ffd640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.250+0000 7f9b02ffd640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 0x7f9af006edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9af8006fd0 tx=0x7f9af8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.254+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b0406c2a0 con 0x7f9b04147fb0 2024-04-26T21:14:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.441+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b040693d0 con 0x7f9b04147fb0 2024-04-26T21:14:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.441+0000 7f9aebfff640 1 -- 172.21.15.40:0/3444268758 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f9af4059ee0 con 0x7f9b04147fb0 2024-04-26T21:14:56.443 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 msgr2=0x7f9af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 0x7f9af006edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9af8006fd0 tx=0x7f9af8008040 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 msgr2=0x7f9b04161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c7c0 tx=0x7f9af400cc80 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 shutdown_connections 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b041638b0 0x7f9b04160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b041485f0 0x7f9b04162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 --2- 172.21.15.40:0/3444268758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b04147fb0 0x7f9b04161ff0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.443+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 >> 172.21.15.40:0/3444268758 conn(0x7f9b040d1e90 msgr2=0x7f9b041427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.444+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 shutdown_connections 2024-04-26T21:14:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.444+0000 7f9b09f46640 1 -- 172.21.15.40:0/3444268758 wait complete. 2024-04-26T21:14:56.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:14:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 Processor -- start 2024-04-26T21:14:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 -- start start 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 0x7f77600d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7760163aa0 0x7f77600b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77600d3f50 con 0x7f77600b60b0 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f776015fa60 con 0x7f7760163aa0 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.559+0000 7f7766cc8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f77600b5090 con 0x7f776015f680 2024-04-26T21:14:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7760163aa0 0x7f77600b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f7765265640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f77600b60b0 0x7f77600d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59510/0 (socket says 172.21.15.40:59510) 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f7765265640 1 -- 172.21.15.40:0/848772807 learned_addr learned my addr 172.21.15.40:0/848772807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f7764a64640 1 --2- 172.21.15.40:0/848772807 >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 0x7f77600d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775ffff640 1 -- 172.21.15.40:0/848772807 >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 msgr2=0x7f77600d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775ffff640 1 --2- 172.21.15.40:0/848772807 >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 0x7f77600d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775ffff640 1 -- 172.21.15.40:0/848772807 >> v1:172.21.15.40:6789/0 conn(0x7f77600b60b0 legacy=0x7f77600d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775ffff640 1 -- 172.21.15.40:0/848772807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77600b5370 con 0x7f7760163aa0 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f775f7fe640 1 -- 172.21.15.40:0/848772807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277450397 0 0) 0x7f77600d3f50 con 0x7f77600b60b0 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.560+0000 7f7764a64640 1 --2- 172.21.15.40:0/848772807 >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 0x7f77600d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f775ffff640 1 --2- 172.21.15.40:0/848772807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7760163aa0 0x7f77600b5b70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f77600d6370 tx=0x7f774c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dadec1a141935da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f775f7fe640 1 -- 172.21.15.40:0/848772807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c037070 con 0x7f7760163aa0 2024-04-26T21:14:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f775f7fe640 1 -- 172.21.15.40:0/848772807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c032440 con 0x7f7760163aa0 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f775f7fe640 1 -- 172.21.15.40:0/848772807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c032810 con 0x7f7760163aa0 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 -- 172.21.15.40:0/848772807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7760163aa0 msgr2=0x7f77600b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 --2- 172.21.15.40:0/848772807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7760163aa0 0x7f77600b5b70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f77600d6370 tx=0x7f774c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 -- 172.21.15.40:0/848772807 shutdown_connections 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 --2- 172.21.15.40:0/848772807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7760163aa0 0x7f77600b5b70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 --2- 172.21.15.40:0/848772807 >> v2:172.21.15.184:3300/0 conn(0x7f776015f680 0x7f77600d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 -- 172.21.15.40:0/848772807 >> 172.21.15.40:0/848772807 conn(0x7f776008eca0 msgr2=0x7f776009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 -- 172.21.15.40:0/848772807 shutdown_connections 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.561+0000 7f7766cc8640 1 -- 172.21.15.40:0/848772807 wait complete. 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.562+0000 7f7766cc8640 1 Processor -- start 2024-04-26T21:14:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 -- start start 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77600b60b0 0x7f77600cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7760163aa0 0x7f77600d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77600d57b0 con 0x7f776015f680 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77600d5630 con 0x7f7760163aa0 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f7766cc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77600d5930 con 0x7f77600b60b0 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f775ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f775ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52116/0 (socket says 172.21.15.40:52116) 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.563+0000 7f775ffff640 1 -- 172.21.15.40:0/1958775890 learned_addr learned my addr 172.21.15.40:0/1958775890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f7765265640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7760163aa0 0x7f77600d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f7764a64640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77600b60b0 0x7f77600cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 -- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77600b60b0 msgr2=0x7f77600cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77600b60b0 0x7f77600cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 -- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7760163aa0 msgr2=0x7f77600d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7760163aa0 0x7f77600d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 -- 172.21.15.40:0/1958775890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77600ceba0 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775ffff640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f774c035040 tx=0x7f774c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c037050 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c038040 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77600cf140 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c004470 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.564+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77601622d0 con 0x7f776015f680 2024-04-26T21:14:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.565+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776006a290 con 0x7f776015f680 2024-04-26T21:14:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.566+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f774c030980 con 0x7f776015f680 2024-04-26T21:14:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.566+0000 7f775d7fa640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.566+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f774c0ac780 con 0x7f776015f680 2024-04-26T21:14:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.566+0000 7f7764a64640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.567+0000 7f7764a64640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 0x7f774406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f775400a990 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.570+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f774c07ac40 con 0x7f776015f680 2024-04-26T21:14:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.754+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f776006eca0 con 0x7f776015f680 2024-04-26T21:14:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.755+0000 7f775d7fa640 1 -- 172.21.15.40:0/1958775890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f774c07a540 con 0x7f776015f680 2024-04-26T21:14:56.756 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.756+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 msgr2=0x7f774406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.756+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 0x7f774406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f775400a990 tx=0x7f7754008040 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.756+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 msgr2=0x7f77600d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.756+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f774c035040 tx=0x7f774c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 shutdown_connections 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7760163aa0 0x7f77600d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f776015f680 0x7f77600d09e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 --2- 172.21.15.40:0/1958775890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77600b60b0 0x7f77600cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 >> 172.21.15.40:0/1958775890 conn(0x7f776008eca0 msgr2=0x7f7760162eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 shutdown_connections 2024-04-26T21:14:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.757+0000 7f7766cc8640 1 -- 172.21.15.40:0/1958775890 wait complete. 2024-04-26T21:14:56.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:14:56.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.871+0000 7f9b620bd640 1 Processor -- start 2024-04-26T21:14:56.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.871+0000 7f9b620bd640 1 -- start start 2024-04-26T21:14:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b620bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0d1be0 0x7f9b5c0d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b620bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b5c0ba3a0 0x7f9b5c0ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b620bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5c0d4240 con 0x7f9b5c15f490 2024-04-26T21:14:56.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b620bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5c0d2580 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b620bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c0d31a0 con 0x7f9b5c0d1be0 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b5c0ba3a0 0x7f9b5c0ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9b5c15f490 0x7f9b5c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59516/0 (socket says 172.21.15.40:59516) 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5b7fe640 1 -- 172.21.15.40:0/289658938 learned_addr learned my addr 172.21.15.40:0/289658938 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5affd640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0d1be0 0x7f9b5c0d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5bfff640 1 -- 172.21.15.40:0/289658938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0d1be0 msgr2=0x7f9b5c0d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5bfff640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0d1be0 0x7f9b5c0d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5bfff640 1 -- 172.21.15.40:0/289658938 >> v1:172.21.15.40:6789/0 conn(0x7f9b5c15f490 legacy=0x7f9b5c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5bfff640 1 -- 172.21.15.40:0/289658938 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c0d3480 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.872+0000 7f9b5a7fc640 1 -- 172.21.15.40:0/289658938 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4199818910 0 0) 0x7f9b5c0d4240 con 0x7f9b5c15f490 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.873+0000 7f9b5bfff640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c0ba3a0 0x7f9b5c0ba7f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009d90 tx=0x7f9b5402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11aca06c7226bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.873+0000 7f9b5a7fc640 1 -- 172.21.15.40:0/289658938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54007740 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.873+0000 7f9b5a7fc640 1 -- 172.21.15.40:0/289658938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b540078e0 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.873+0000 7f9b5a7fc640 1 -- 172.21.15.40:0/289658938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54007cb0 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 -- 172.21.15.40:0/289658938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c0ba3a0 msgr2=0x7f9b5c0ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c0ba3a0 0x7f9b5c0ba7f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009d90 tx=0x7f9b5402a660 comp rx=0 tx=0).stop 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 -- 172.21.15.40:0/289658938 shutdown_connections 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c0ba3a0 0x7f9b5c0ba7f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 --2- 172.21.15.40:0/289658938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0d1be0 0x7f9b5c0d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 -- 172.21.15.40:0/289658938 >> 172.21.15.40:0/289658938 conn(0x7f9b5c09e530 msgr2=0x7f9b5c0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 -- 172.21.15.40:0/289658938 shutdown_connections 2024-04-26T21:14:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.874+0000 7f9b620bd640 1 -- 172.21.15.40:0/289658938 wait complete. 2024-04-26T21:14:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.875+0000 7f9b620bd640 1 Processor -- start 2024-04-26T21:14:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.875+0000 7f9b620bd640 1 -- start start 2024-04-26T21:14:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0ba3a0 0x7f9b5c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c0d1be0 0x7f9b5c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c0d3d20 con 0x7f9b5c15f490 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c0d3a40 con 0x7f9b5c0d1be0 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b620bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c0d11c0 con 0x7f9b5c0ba3a0 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52138/0 (socket says 172.21.15.40:52138) 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 -- 172.21.15.40:0/802543114 learned_addr learned my addr 172.21.15.40:0/802543114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5affd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c0d1be0 0x7f9b5c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5b7fe640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0ba3a0 0x7f9b5c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 -- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0ba3a0 msgr2=0x7f9b5c158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0ba3a0 0x7f9b5c158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 -- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c0d1be0 msgr2=0x7f9b5c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c0d1be0 0x7f9b5c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 -- 172.21.15.40:0/802543114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1602c0 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b5bfff640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.876+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5403f070 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.877+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b54031c40 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.877+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c1605a0 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.877+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54009450 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.877+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c160e40 con 0x7f9b5c15f490 2024-04-26T21:14:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.877+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c160770 con 0x7f9b5c15f490 2024-04-26T21:14:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.878+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b54003990 con 0x7f9b5c15f490 2024-04-26T21:14:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.878+0000 7f9b43fff640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 0x7f9b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.879+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9b540ac8f0 con 0x7f9b5c15f490 2024-04-26T21:14:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.879+0000 7f9b5b7fe640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 0x7f9b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.879+0000 7f9b5b7fe640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 0x7f9b4806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000f60 tx=0x7f9b4c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:56.883+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b5407adb0 con 0x7f9b5c15f490 2024-04-26T21:14:57.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.069+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b5c06a290 con 0x7f9b5c15f490 2024-04-26T21:14:57.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.070+0000 7f9b43fff640 1 -- 172.21.15.40:0/802543114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7f9b5407a6b0 con 0x7f9b5c15f490 2024-04-26T21:14:57.071 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 msgr2=0x7f9b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 0x7f9b4806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000f60 tx=0x7f9b4c007c00 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 msgr2=0x7f9b5c15f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009eb0 tx=0x7f9b54004a40 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 shutdown_connections 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9b4806c8f0 0x7f9b4806edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b5c15f490 0x7f9b5c15f870 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c0d1be0 0x7f9b5c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 --2- 172.21.15.40:0/802543114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b5c0ba3a0 0x7f9b5c158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 >> 172.21.15.40:0/802543114 conn(0x7f9b5c09e530 msgr2=0x7f9b5c0ba780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 shutdown_connections 2024-04-26T21:14:57.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.072+0000 7f9b620bd640 1 -- 172.21.15.40:0/802543114 wait complete. 2024-04-26T21:14:57.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:14:57.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.184+0000 7f40fc4f8640 1 Processor -- start 2024-04-26T21:14:57.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.184+0000 7f40fc4f8640 1 -- start start 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fc4f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f40f40d3ee0 0x7f40f40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fc4f8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f40d03a0 con 0x7f40f415f370 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fc4f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f40d0680 con 0x7f40f415f750 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fc4f8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f40f40d0960 con 0x7f40f40d3ee0 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fa294640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f40f415f370 0x7f40f40d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59518/0 (socket says 172.21.15.40:59518) 2024-04-26T21:14:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.185+0000 7f40fa294640 1 -- 172.21.15.40:0/2353059081 learned_addr learned my addr 172.21.15.40:0/2353059081 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9a93640 1 --2- 172.21.15.40:0/2353059081 >> v2:172.21.15.184:3300/0 conn(0x7f40f40d3ee0 0x7f40f40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868775760 0 0) 0x7f40f40d03a0 con 0x7f40f415f370 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40dc003520 con 0x7f40f415f370 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645039582 0 0) 0x7f40f40d0680 con 0x7f40f415f750 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f40d03a0 con 0x7f40f415f750 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 336087506 0 0) 0x7f40dc003520 con 0x7f40f415f370 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9a93640 1 -- 172.21.15.40:0/2353059081 >> v1:172.21.15.184:6789/0 conn(0x7f40f415f750 legacy=0x7f40f40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9a93640 1 -- 172.21.15.40:0/2353059081 >> v1:172.21.15.40:6789/0 conn(0x7f40f415f370 legacy=0x7f40f40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.186+0000 7f40f9a93640 1 -- 172.21.15.40:0/2353059081 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f40d0c40 con 0x7f40f40d3ee0 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40f9a93640 1 --2- 172.21.15.40:0/2353059081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f40d3ee0 0x7f40f40cfd70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40e80097d0 tx=0x7f40e802a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=850c692c4f6a3ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8037070 con 0x7f40f40d3ee0 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e802ac90 con 0x7f40f40d3ee0 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40f9292640 1 -- 172.21.15.40:0/2353059081 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec001670 con 0x7f40f40d3ee0 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 -- 172.21.15.40:0/2353059081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f40d3ee0 msgr2=0x7f40f40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/2353059081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f40d3ee0 0x7f40f40cfd70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40e80097d0 tx=0x7f40e802a200 comp rx=0 tx=0).stop 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 -- 172.21.15.40:0/2353059081 shutdown_connections 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/2353059081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f40d3ee0 0x7f40f40cfd70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 -- 172.21.15.40:0/2353059081 >> 172.21.15.40:0/2353059081 conn(0x7f40f408ec10 msgr2=0x7f40f409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.187+0000 7f40fc4f8640 1 -- 172.21.15.40:0/2353059081 shutdown_connections 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.188+0000 7f40fc4f8640 1 -- 172.21.15.40:0/2353059081 wait complete. 2024-04-26T21:14:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 Processor -- start 2024-04-26T21:14:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 -- start start 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 0x7f40f40cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 0x7f40f40c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40f40b4950 con 0x7f40f415f370 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40f40b47d0 con 0x7f40f415f750 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.189+0000 7f40fc4f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f40f40b4ad0 con 0x7f40f40d3ee0 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52148/0 (socket says 172.21.15.40:52148) 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 -- 172.21.15.40:0/1093760369 learned_addr learned my addr 172.21.15.40:0/1093760369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40fa294640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 0x7f40f40cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40faa95640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 0x7f40f40c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 -- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 msgr2=0x7f40f40cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 0x7f40f40cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 -- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 msgr2=0x7f40f40c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 0x7f40f40c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 -- 172.21.15.40:0/1093760369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f40baae0 con 0x7f40f415f370 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40fa294640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 0x7f40f40cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40faa95640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 0x7f40f40c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f9a93640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098f0 tx=0x7f40e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8037040 con 0x7f40f415f370 2024-04-26T21:14:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f40badc0 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e8004230 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8006e10 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.190+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f40cf140 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.191+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f406a290 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.192+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40e8003610 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.192+0000 7f40f2ffd640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 0x7f40e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.192+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f40e80ac170 con 0x7f40f415f370 2024-04-26T21:14:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.192+0000 7f40fa294640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 0x7f40e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.193+0000 7f40fa294640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 0x7f40e006ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0017e0 tx=0x7f40ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.197+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40e807a5b0 con 0x7f40f415f370 2024-04-26T21:14:57.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.393+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40f40cfd30 con 0x7f40f415f370 2024-04-26T21:14:57.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.393+0000 7f40f2ffd640 1 -- 172.21.15.40:0/1093760369 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f40e8079eb0 con 0x7f40f415f370 2024-04-26T21:14:57.394 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 msgr2=0x7f40e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 0x7f40e006ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0017e0 tx=0x7f40ec009040 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 msgr2=0x7f40f40cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098f0 tx=0x7f40e80046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 shutdown_connections 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f40e006c920 0x7f40e006ede0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40f415f750 0x7f40f40c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415f370 0x7f40f40cec00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 --2- 172.21.15.40:0/1093760369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40f40d3ee0 0x7f40f40cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 >> 172.21.15.40:0/1093760369 conn(0x7f40f408ec10 msgr2=0x7f40f40b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 shutdown_connections 2024-04-26T21:14:57.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.395+0000 7f40fc4f8640 1 -- 172.21.15.40:0/1093760369 wait complete. 2024-04-26T21:14:57.407 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:14:57.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:57.407 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:14:57.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:14:57.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:57.435 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:14:57.490 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:57.490 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:57.490 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:57.509 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:57.509 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:57.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:57.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.666+0000 7fd69cecc640 1 Processor -- start 2024-04-26T21:14:57.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.666+0000 7fd69cecc640 1 -- start start 2024-04-26T21:14:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69cecc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd698160d50 0x7fd6981611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69cecc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6980d0be0 con 0x7fd69815d380 2024-04-26T21:14:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69cecc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6980d0ec0 con 0x7fd6980b56d0 2024-04-26T21:14:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69cecc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd6981616e0 con 0x7fd698160d50 2024-04-26T21:14:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd696d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd698160d50 0x7fd6981611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd69815d380 0x7fd6980ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59520/0 (socket says 172.21.15.40:59520) 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69659c640 1 -- 172.21.15.40:0/2923297564 learned_addr learned my addr 172.21.15.40:0/2923297564 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd696d9d640 1 -- 172.21.15.40:0/2923297564 >> v1:172.21.15.184:6789/0 conn(0x7fd6980b56d0 legacy=0x7fd6980d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd696d9d640 1 -- 172.21.15.40:0/2923297564 >> v1:172.21.15.40:6789/0 conn(0x7fd69815d380 legacy=0x7fd6980ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd696d9d640 1 -- 172.21.15.40:0/2923297564 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980d11a0 con 0x7fd698160d50 2024-04-26T21:14:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.667+0000 7fd69559a640 1 -- 172.21.15.40:0/2923297564 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1156646236 0 0) 0x7fd6980d0be0 con 0x7fd69815d380 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.668+0000 7fd696d9d640 1 --2- 172.21.15.40:0/2923297564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd698160d50 0x7fd6981611a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd688009d90 tx=0x7fd68802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e5a24db86f8c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.668+0000 7fd69559a640 1 -- 172.21.15.40:0/2923297564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688031600 con 0x7fd698160d50 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.668+0000 7fd69559a640 1 -- 172.21.15.40:0/2923297564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6880317a0 con 0x7fd698160d50 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.668+0000 7fd69559a640 1 -- 172.21.15.40:0/2923297564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688031b70 con 0x7fd698160d50 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 -- 172.21.15.40:0/2923297564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd698160d50 msgr2=0x7fd6981611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 --2- 172.21.15.40:0/2923297564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd698160d50 0x7fd6981611a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd688009d90 tx=0x7fd68802a610 comp rx=0 tx=0).stop 2024-04-26T21:14:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 -- 172.21.15.40:0/2923297564 shutdown_connections 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 --2- 172.21.15.40:0/2923297564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd698160d50 0x7fd6981611a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 -- 172.21.15.40:0/2923297564 >> 172.21.15.40:0/2923297564 conn(0x7fd6980b4a60 msgr2=0x7fd6980b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 -- 172.21.15.40:0/2923297564 shutdown_connections 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.669+0000 7fd69cecc640 1 -- 172.21.15.40:0/2923297564 wait complete. 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.670+0000 7fd69cecc640 1 Processor -- start 2024-04-26T21:14:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.670+0000 7fd69cecc640 1 -- start start 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6980b56d0 0x7fd69815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd698160d50 0x7fd69814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980b5dc0 con 0x7fd69815d380 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980cedc0 con 0x7fd698160d50 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd69cecc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd6980b5f40 con 0x7fd6980b56d0 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52166/0 (socket says 172.21.15.40:52166) 2024-04-26T21:14:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 -- 172.21.15.40:0/3116771450 learned_addr learned my addr 172.21.15.40:0/3116771450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 -- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6980b56d0 msgr2=0x7fd69815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6980b56d0 0x7fd69815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 -- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd698160d50 msgr2=0x7fd69814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.671+0000 7fd695d9b640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd698160d50 0x7fd69814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd695d9b640 1 -- 172.21.15.40:0/3116771450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980cbc70 con 0x7fd69815d380 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd695d9b640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c400 tx=0x7fd68c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c017070 con 0x7fd69815d380 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c0035f0 con 0x7fd69815d380 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6980cb5e0 con 0x7fd69815d380 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c003a20 con 0x7fd69815d380 2024-04-26T21:14:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.672+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69815d9d0 con 0x7fd69815d380 2024-04-26T21:14:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.673+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6980b89c0 con 0x7fd69815d380 2024-04-26T21:14:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.673+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd68c01a070 con 0x7fd69815d380 2024-04-26T21:14:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.674+0000 7fd686ffd640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 0x7fd67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.674+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd68c08ae20 con 0x7fd69815d380 2024-04-26T21:14:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.674+0000 7fd69659c640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 0x7fd67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.675+0000 7fd69659c640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 0x7fd67806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd690007fd0 tx=0x7fd690007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.678+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6980b89c0 con 0x7fd69815d380 2024-04-26T21:14:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.864+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6980cc080 con 0x7fd69815d380 2024-04-26T21:14:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.865+0000 7fd686ffd640 1 -- 172.21.15.40:0/3116771450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7fd6980cc080 con 0x7fd69815d380 2024-04-26T21:14:57.866 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:57.866 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 msgr2=0x7fd67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 0x7fd67806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd690007fd0 tx=0x7fd690007920 comp rx=0 tx=0).stop 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 msgr2=0x7fd6981f3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c400 tx=0x7fd68c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 shutdown_connections 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd67806c8f0 0x7fd67806edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd698160d50 0x7fd69814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69815d380 0x7fd6981f3410 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 --2- 172.21.15.40:0/3116771450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6980b56d0 0x7fd69815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 >> 172.21.15.40:0/3116771450 conn(0x7fd6980b4a60 msgr2=0x7fd69809ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 shutdown_connections 2024-04-26T21:14:57.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.867+0000 7fd69cecc640 1 -- 172.21.15.40:0/3116771450 wait complete. 2024-04-26T21:14:57.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:41.855331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:42.545028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:57.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:14:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.982+0000 7f45b7b51640 1 Processor -- start 2024-04-26T21:14:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.982+0000 7f45b7b51640 1 -- start start 2024-04-26T21:14:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.983+0000 7f45b7b51640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45b015c0e0 0x7f45b015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.983+0000 7f45b7b51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.983+0000 7f45b7b51640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b00cff00 con 0x7f45b006f020 2024-04-26T21:14:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.983+0000 7f45b7b51640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f45b015ca00 con 0x7f45b015c0e0 2024-04-26T21:14:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.983+0000 7f45b7b51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45b0160aa0 con 0x7f45b006dac0 2024-04-26T21:14:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b50ec640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f45b006f020 0x7f45b006d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41752/0 (socket says 172.21.15.40:41752) 2024-04-26T21:14:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b50ec640 1 -- 172.21.15.40:0/1144049607 learned_addr learned my addr 172.21.15.40:0/1144049607 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b58ed640 1 --2- 172.21.15.40:0/1144049607 >> v2:172.21.15.184:3300/0 conn(0x7f45b015c0e0 0x7f45b015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b60ee640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b48eb640 1 -- 172.21.15.40:0/1144049607 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006077054 0 0) 0x7f45b00cff00 con 0x7f45b006f020 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b48eb640 1 -- 172.21.15.40:0/1144049607 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4598003520 con 0x7f45b006f020 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b58ed640 1 -- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 msgr2=0x7f45b015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b58ed640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b58ed640 1 -- 172.21.15.40:0/1144049607 >> v1:172.21.15.184:6789/0 conn(0x7f45b006f020 legacy=0x7f45b006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b58ed640 1 -- 172.21.15.40:0/1144049607 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b015fe90 con 0x7f45b015c0e0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.984+0000 7f45b60ee640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b58ed640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b015c4c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009d90 tx=0x7f45a002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5752296c48f3df08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b48eb640 1 -- 172.21.15.40:0/1144049607 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0007740 con 0x7f45b015c0e0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b48eb640 1 -- 172.21.15.40:0/1144049607 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a00078e0 con 0x7f45b015c0e0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b48eb640 1 -- 172.21.15.40:0/1144049607 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0007c90 con 0x7f45b015c0e0 2024-04-26T21:14:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 -- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 msgr2=0x7f45b015c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b015c4c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009d90 tx=0x7f45a002a660 comp rx=0 tx=0).stop 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 -- 172.21.15.40:0/1144049607 shutdown_connections 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 --2- 172.21.15.40:0/1144049607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b015c4c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.985+0000 7f45b7b51640 1 -- 172.21.15.40:0/1144049607 >> 172.21.15.40:0/1144049607 conn(0x7f45b00d1e40 msgr2=0x7f45b00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.986+0000 7f45b7b51640 1 -- 172.21.15.40:0/1144049607 shutdown_connections 2024-04-26T21:14:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.986+0000 7f45b7b51640 1 -- 172.21.15.40:0/1144049607 wait complete. 2024-04-26T21:14:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 Processor -- start 2024-04-26T21:14:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 -- start start 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b01513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b0073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0160730 con 0x7f45b006f020 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0160450 con 0x7f45b015c0e0 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b7b51640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45b006f480 con 0x7f45b006dac0 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.987+0000 7f45b50ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52172/0 (socket says 172.21.15.40:52172) 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 -- 172.21.15.40:0/174948949 learned_addr learned my addr 172.21.15.40:0/174948949 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b58ed640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b01513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b60ee640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b0073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 -- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 msgr2=0x7f45b01513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b01513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 -- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 msgr2=0x7f45b0073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b0073410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 -- 172.21.15.40:0/174948949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b01f63d0 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b60ee640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b0073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b50ec640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c460 tx=0x7f45a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4017070 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4005ce0 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b01f6670 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4007290 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.988+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01f6c30 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.989+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b00693d0 con 0x7f45b006f020 2024-04-26T21:14:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.990+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45a401a070 con 0x7f45b006f020 2024-04-26T21:14:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.990+0000 7f45ae7fc640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.990+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f45a408ae30 con 0x7f45b006f020 2024-04-26T21:14:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.990+0000 7f45b58ed640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.991+0000 7f45b58ed640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 0x7f459c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f45a0034000 tx=0x7f45a002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:57.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:57.994+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45a40592f0 con 0x7f45b006f020 2024-04-26T21:14:58.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.178+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45b015b4b0 con 0x7f45b006f020 2024-04-26T21:14:58.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.179+0000 7f45ae7fc640 1 -- 172.21.15.40:0/174948949 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f45a4058bf0 con 0x7f45b006f020 2024-04-26T21:14:58.180 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.180+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 msgr2=0x7f459c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.180+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 0x7f459c06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f45a0034000 tx=0x7f45a002f000 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.180+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 msgr2=0x7f45b00b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.180+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c460 tx=0x7f45a400c920 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.180+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 shutdown_connections 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f459c06c8f0 0x7f459c06edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b015c0e0 0x7f45b0073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b006f020 0x7f45b00b23b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 --2- 172.21.15.40:0/174948949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b006dac0 0x7f45b01513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 >> 172.21.15.40:0/174948949 conn(0x7f45b00d1e40 msgr2=0x7f45b00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 shutdown_connections 2024-04-26T21:14:58.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.181+0000 7f45b7b51640 1 -- 172.21.15.40:0/174948949 wait complete. 2024-04-26T21:14:58.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:14:58.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.295+0000 7fd8eb740640 1 Processor -- start 2024-04-26T21:14:58.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.295+0000 7fd8eb740640 1 -- start start 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8eb740640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8e41631f0 0x7fd8e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8eb740640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 0x7fd8e41635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8eb740640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e409c790 con 0x7fd8e40b3300 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8eb740640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e40d15f0 con 0x7fd8e41631f0 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8eb740640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e4163ec0 con 0x7fd8e40d1170 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8e41631f0 0x7fd8e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8e41631f0 0x7fd8e40b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52186/0 (socket says 172.21.15.40:52186) 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 -- 172.21.15.40:0/719658653 learned_addr learned my addr 172.21.15.40:0/719658653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e8cdb640 1 --2- 172.21.15.40:0/719658653 >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 0x7fd8e41635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 -- 172.21.15.40:0/719658653 >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 msgr2=0x7fd8e41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 --2- 172.21.15.40:0/719658653 >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 0x7fd8e41635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 -- 172.21.15.40:0/719658653 >> v1:172.21.15.184:6789/0 conn(0x7fd8e40b3300 legacy=0x7fd8e40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e94dc640 1 -- 172.21.15.40:0/719658653 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e40d1a40 con 0x7fd8e41631f0 2024-04-26T21:14:58.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.296+0000 7fd8e8cdb640 1 --2- 172.21.15.40:0/719658653 >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 0x7fd8e41635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.297+0000 7fd8e94dc640 1 --2- 172.21.15.40:0/719658653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40b2000 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40734b0 tx=0x7fd8dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1a91094de8e86c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.297+0000 7fd8e3fff640 1 -- 172.21.15.40:0/719658653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8e41631f0 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.297+0000 7fd8e3fff640 1 -- 172.21.15.40:0/719658653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc032440 con 0x7fd8e41631f0 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.297+0000 7fd8e3fff640 1 -- 172.21.15.40:0/719658653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032810 con 0x7fd8e41631f0 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 -- 172.21.15.40:0/719658653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 msgr2=0x7fd8e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 --2- 172.21.15.40:0/719658653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40b2000 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40734b0 tx=0x7fd8dc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 -- 172.21.15.40:0/719658653 shutdown_connections 2024-04-26T21:14:58.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 --2- 172.21.15.40:0/719658653 >> v2:172.21.15.184:3300/0 conn(0x7fd8e40d1170 0x7fd8e41635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 --2- 172.21.15.40:0/719658653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40b2000 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 -- 172.21.15.40:0/719658653 >> 172.21.15.40:0/719658653 conn(0x7fd8e409ba40 msgr2=0x7fd8e409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 -- 172.21.15.40:0/719658653 shutdown_connections 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.298+0000 7fd8eb740640 1 -- 172.21.15.40:0/719658653 wait complete. 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.299+0000 7fd8eb740640 1 Processor -- start 2024-04-26T21:14:58.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.299+0000 7fd8eb740640 1 -- start start 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8e40b3300 0x7fd8e415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 0x7fd8e4157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d0230 con 0x7fd8e41631f0 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d2000 con 0x7fd8e40d1170 2024-04-26T21:14:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8eb740640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e4160010 con 0x7fd8e40b3300 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52200/0 (socket says 172.21.15.40:52200) 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 -- 172.21.15.40:0/1945202873 learned_addr learned my addr 172.21.15.40:0/1945202873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e94dc640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8e40b3300 0x7fd8e415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e8cdb640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 0x7fd8e4157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 -- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8e40b3300 msgr2=0x7fd8e415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8e40b3300 0x7fd8e415fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 -- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 msgr2=0x7fd8e4157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.300+0000 7fd8e9cdd640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 0x7fd8e4157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e9cdd640 1 -- 172.21.15.40:0/1945202873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e4157560 con 0x7fd8e41631f0 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e8cdb640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 0x7fd8e4157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e9cdd640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c780 tx=0x7fd8d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4004b60 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d4004d00 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e4159870 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d400aa90 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.301+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e4159e30 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.302+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e406a290 con 0x7fd8e41631f0 2024-04-26T21:14:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.303+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8d400ac30 con 0x7fd8e41631f0 2024-04-26T21:14:58.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.303+0000 7fd8e1ffb640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 0x7fd8d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.303+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd8d408bd50 con 0x7fd8e41631f0 2024-04-26T21:14:58.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.303+0000 7fd8e94dc640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 0x7fd8d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.304+0000 7fd8e94dc640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 0x7fd8d006edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc034000 tx=0x7fd8dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.307+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8d408f030 con 0x7fd8e41631f0 2024-04-26T21:14:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.490+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8e40ce990 con 0x7fd8e41631f0 2024-04-26T21:14:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.490+0000 7fd8e1ffb640 1 -- 172.21.15.40:0/1945202873 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7fd8d4020070 con 0x7fd8e41631f0 2024-04-26T21:14:58.491 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:14:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 msgr2=0x7fd8d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 0x7fd8d006edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc034000 tx=0x7fd8dc02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 msgr2=0x7fd8e40cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c780 tx=0x7fd8d400cc40 comp rx=0 tx=0).stop 2024-04-26T21:14:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 shutdown_connections 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8d006c8f0 0x7fd8d006edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e41631f0 0x7fd8e40cc520 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8e40d1170 0x7fd8e4157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 --2- 172.21.15.40:0/1945202873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8e40b3300 0x7fd8e415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 >> 172.21.15.40:0/1945202873 conn(0x7fd8e409ba40 msgr2=0x7fd8e4075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 shutdown_connections 2024-04-26T21:14:58.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.493+0000 7fd8eb740640 1 -- 172.21.15.40:0/1945202873 wait complete. 2024-04-26T21:14:58.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:14:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.607+0000 7f65761b4640 1 Processor -- start 2024-04-26T21:14:58.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.607+0000 7f65761b4640 1 -- start start 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f65761b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 0x7f657006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f65761b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f65700b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f65761b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f657009c700 con 0x7f65700b3210 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f65761b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f65700b4480 con 0x7f65700d0010 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f65761b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f657006e050 con 0x7f65700b3c20 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f656ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f65700b3210 0x7f657015fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59522/0 (socket says 172.21.15.40:59522) 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f656ffff640 1 -- 172.21.15.40:0/46106671 learned_addr learned my addr 172.21.15.40:0/46106671 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f656effd640 1 --2- 172.21.15.40:0/46106671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f65700b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.608+0000 7f656f7fe640 1 --2- 172.21.15.40:0/46106671 >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 0x7f657006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656e7fc640 1 -- 172.21.15.40:0/46106671 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359513641 0 0) 0x7f657009c700 con 0x7f65700b3210 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656e7fc640 1 -- 172.21.15.40:0/46106671 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6558003520 con 0x7f65700b3210 2024-04-26T21:14:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656effd640 1 -- 172.21.15.40:0/46106671 >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 msgr2=0x7f657006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656effd640 1 --2- 172.21.15.40:0/46106671 >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 0x7f657006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656effd640 1 -- 172.21.15.40:0/46106671 >> v1:172.21.15.40:6789/0 conn(0x7f65700b3210 legacy=0x7f657015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656effd640 1 -- 172.21.15.40:0/46106671 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65700d1c50 con 0x7f65700b3c20 2024-04-26T21:14:58.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656f7fe640 1 --2- 172.21.15.40:0/46106671 >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 0x7f657006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.609+0000 7f656effd640 1 --2- 172.21.15.40:0/46106671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f65700b2cd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65600097a0 tx=0x7f656002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee29daa04314b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f656e7fc640 1 -- 172.21.15.40:0/46106671 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560037070 con 0x7f65700b3c20 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f656e7fc640 1 -- 172.21.15.40:0/46106671 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6560032440 con 0x7f65700b3c20 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f656e7fc640 1 -- 172.21.15.40:0/46106671 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560032810 con 0x7f65700b3c20 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 -- 172.21.15.40:0/46106671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 msgr2=0x7f65700b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 --2- 172.21.15.40:0/46106671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f65700b2cd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65600097a0 tx=0x7f656002a180 comp rx=0 tx=0).stop 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 -- 172.21.15.40:0/46106671 shutdown_connections 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 --2- 172.21.15.40:0/46106671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f65700b2cd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 --2- 172.21.15.40:0/46106671 >> v2:172.21.15.184:3300/0 conn(0x7f65700d0010 0x7f657006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 -- 172.21.15.40:0/46106671 >> 172.21.15.40:0/46106671 conn(0x7f657009ba40 msgr2=0x7f657009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 -- 172.21.15.40:0/46106671 shutdown_connections 2024-04-26T21:14:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.610+0000 7f65761b4640 1 -- 172.21.15.40:0/46106671 wait complete. 2024-04-26T21:14:58.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.611+0000 7f65761b4640 1 Processor -- start 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 -- start start 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65700b3210 0x7f6570153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f6570161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65700b3820 con 0x7f65700d0010 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65700b36a0 con 0x7f65700b3210 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f65761b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65700b39a0 con 0x7f65700b3c20 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f656ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f656ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52214/0 (socket says 172.21.15.40:52214) 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.612+0000 7f656ffff640 1 -- 172.21.15.40:0/4166196968 learned_addr learned my addr 172.21.15.40:0/4166196968 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656effd640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f6570161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656f7fe640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65700b3210 0x7f6570153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 -- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 msgr2=0x7f6570161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f6570161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 -- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65700b3210 msgr2=0x7f6570153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65700b3210 0x7f6570153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 -- 172.21.15.40:0/4166196968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65701610c0 con 0x7f65700d0010 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f656ffff640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f656800ca50 tx=0x7f656800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568017070 con 0x7f65700d0010 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656800abb0 con 0x7f65700d0010 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701613a0 con 0x7f65700d0010 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568012400 con 0x7f65700d0010 2024-04-26T21:14:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.613+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65701623c0 con 0x7f65700d0010 2024-04-26T21:14:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.614+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657006a290 con 0x7f65700d0010 2024-04-26T21:14:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.615+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f656801a070 con 0x7f65700d0010 2024-04-26T21:14:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.615+0000 7f6557fff640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 0x7f655c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.615+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f656808b890 con 0x7f65700d0010 2024-04-26T21:14:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.615+0000 7f656f7fe640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 0x7f655c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.616+0000 7f656f7fe640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 0x7f655c06edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6564009a30 tx=0x7f6564008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.619+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6568059cd0 con 0x7f65700d0010 2024-04-26T21:14:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.805+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6570068e00 con 0x7f65700d0010 2024-04-26T21:14:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.805+0000 7f6557fff640 1 -- 172.21.15.40:0/4166196968 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7f65680595d0 con 0x7f65700d0010 2024-04-26T21:14:58.807 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 msgr2=0x7f655c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 0x7f655c06edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6564009a30 tx=0x7f6564008040 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 msgr2=0x7f657015e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f656800ca50 tx=0x7f656800cf10 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 shutdown_connections 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f655c06c8f0 0x7f655c06edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65700d0010 0x7f657015e090 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65700b3c20 0x7f6570161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 --2- 172.21.15.40:0/4166196968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65700b3210 0x7f6570153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 >> 172.21.15.40:0/4166196968 conn(0x7f657009ba40 msgr2=0x7f65700d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 shutdown_connections 2024-04-26T21:14:58.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.807+0000 7f65761b4640 1 -- 172.21.15.40:0/4166196968 wait complete. 2024-04-26T21:14:58.819 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:14:58.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 Processor -- start 2024-04-26T21:14:58.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 -- start start 2024-04-26T21:14:58.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00f0070d20 0x7f00f0071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 0x7f00f0162f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f00d14c0 con 0x7f00f0070520 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f00f00d1290 con 0x7f00f0070d20 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.920+0000 7f00f6272640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f00f0071640 con 0x7f00f009c400 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00effff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00f0070d20 0x7f00f0071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00f480f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00f0070520 0x7f00f0163470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59524/0 (socket says 172.21.15.40:59524) 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00f480f640 1 -- 172.21.15.40:0/3590149185 learned_addr learned my addr 172.21.15.40:0/3590149185 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/3590149185 >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 0x7f00f0162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00effff640 1 -- 172.21.15.40:0/3590149185 >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 msgr2=0x7f00f0162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00effff640 1 --2- 172.21.15.40:0/3590149185 >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 0x7f00f0162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00effff640 1 -- 172.21.15.40:0/3590149185 >> v1:172.21.15.40:6789/0 conn(0x7f00f0070520 legacy=0x7f00f0163470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00effff640 1 -- 172.21.15.40:0/3590149185 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00ce5b0 con 0x7f00f0070d20 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00eeffd640 1 -- 172.21.15.40:0/3590149185 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574986501 0 0) 0x7f00f00d14c0 con 0x7f00f0070520 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.921+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/3590149185 >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 0x7f00f0162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00effff640 1 --2- 172.21.15.40:0/3590149185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070d20 0x7f00f0071100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f00f00d0cb0 tx=0x7f00e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5823dc2dd440edd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00eeffd640 1 -- 172.21.15.40:0/3590149185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0037070 con 0x7f00f0070d20 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00eeffd640 1 -- 172.21.15.40:0/3590149185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0032440 con 0x7f00f0070d20 2024-04-26T21:14:58.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00eeffd640 1 -- 172.21.15.40:0/3590149185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0032810 con 0x7f00f0070d20 2024-04-26T21:14:58.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 -- 172.21.15.40:0/3590149185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070d20 msgr2=0x7f00f0071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 --2- 172.21.15.40:0/3590149185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070d20 0x7f00f0071100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f00f00d0cb0 tx=0x7f00e002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 -- 172.21.15.40:0/3590149185 shutdown_connections 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 --2- 172.21.15.40:0/3590149185 >> v2:172.21.15.184:3300/0 conn(0x7f00f009c400 0x7f00f0162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 --2- 172.21.15.40:0/3590149185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070d20 0x7f00f0071100 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 -- 172.21.15.40:0/3590149185 >> 172.21.15.40:0/3590149185 conn(0x7f00f00b4a40 msgr2=0x7f00f00b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 -- 172.21.15.40:0/3590149185 shutdown_connections 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.922+0000 7f00f6272640 1 -- 172.21.15.40:0/3590149185 wait complete. 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.923+0000 7f00f6272640 1 Processor -- start 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 -- start start 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f0070d20 0x7f00f0150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f009c400 0x7f00f0151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d1d20 con 0x7f00f0070520 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d1a40 con 0x7f00f0070d20 2024-04-26T21:14:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00f6272640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d2000 con 0x7f00f009c400 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.924+0000 7f00effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52230/0 (socket says 172.21.15.40:52230) 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 -- 172.21.15.40:0/2944036221 learned_addr learned my addr 172.21.15.40:0/2944036221 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f0070d20 0x7f00f0150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 -- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f009c400 msgr2=0x7f00f0151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f009c400 0x7f00f0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 -- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f0070d20 msgr2=0x7f00f0150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f0070d20 0x7f00f0150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 -- 172.21.15.40:0/2944036221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00b61a0 con 0x7f00f0070520 2024-04-26T21:14:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00effff640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0037040 con 0x7f00f0070520 2024-04-26T21:14:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0042070 con 0x7f00f0070520 2024-04-26T21:14:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f00b6480 con 0x7f00f0070520 2024-04-26T21:14:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0030a50 con 0x7f00f0070520 2024-04-26T21:14:58.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.925+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f00b6a40 con 0x7f00f0070520 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.926+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f00693d0 con 0x7f00f0070520 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.927+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00e0030bf0 con 0x7f00f0070520 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.927+0000 7f00ecff9640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.927+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f00e00ac6e0 con 0x7f00f0070520 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.927+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.928+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 0x7f00dc06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f00f00cead0 tx=0x7f00e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:58.931+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00e0045360 con 0x7f00f0070520 2024-04-26T21:14:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.128+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00f0074880 con 0x7f00f0070520 2024-04-26T21:14:59.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.129+0000 7f00ecff9640 1 -- 172.21.15.40:0/2944036221 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f00e007aa80 con 0x7f00f0070520 2024-04-26T21:14:59.130 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 msgr2=0x7f00dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 0x7f00dc06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f00f00cead0 tx=0x7f00e4008040 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 msgr2=0x7f00f00c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 shutdown_connections 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f009c400 0x7f00f0151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f0070d20 0x7f00f0150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 --2- 172.21.15.40:0/2944036221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f0070520 0x7f00f00c6a40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.130+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 >> 172.21.15.40:0/2944036221 conn(0x7f00f00b4a40 msgr2=0x7f00f006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.131+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 shutdown_connections 2024-04-26T21:14:59.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.131+0000 7f00f6272640 1 -- 172.21.15.40:0/2944036221 wait complete. 2024-04-26T21:14:59.143 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:14:59.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:59.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:14:59.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:14:59.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:14:59.176 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:14:59.233 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:59.233 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:14:59.233 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:59.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:14:59.250 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:14:59.250 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:14:59.250 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:14:59.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:14:59.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:14:59.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.390+0000 7fea379e6640 1 Processor -- start 2024-04-26T21:14:59.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.390+0000 7fea379e6640 1 -- start start 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea379e6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea3015f1e0 0x7fea3006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea379e6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 0x7fea3006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea379e6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea300b4a10 con 0x7fea300b44e0 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea379e6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fea3006dc60 con 0x7fea3015f1e0 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea379e6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fea3006eeb0 con 0x7fea3006d2b0 2024-04-26T21:14:59.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35782640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea3015f1e0 0x7fea3006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea34f81640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fea300b44e0 0x7fea300ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59534/0 (socket says 172.21.15.40:59534) 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea34f81640 1 -- 172.21.15.40:0/1805801477 learned_addr learned my addr 172.21.15.40:0/1805801477 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35782640 1 -- 172.21.15.40:0/1805801477 >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 msgr2=0x7fea3006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35f83640 1 --2- 172.21.15.40:0/1805801477 >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 0x7fea3006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea27fff640 1 -- 172.21.15.40:0/1805801477 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021690716 0 0) 0x7fea300b4a10 con 0x7fea300b44e0 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35782640 1 --2- 172.21.15.40:0/1805801477 >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 0x7fea3006d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35782640 1 -- 172.21.15.40:0/1805801477 >> v1:172.21.15.40:6789/0 conn(0x7fea300b44e0 legacy=0x7fea300ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35782640 1 -- 172.21.15.40:0/1805801477 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300b4cf0 con 0x7fea3015f1e0 2024-04-26T21:14:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.391+0000 7fea35f83640 1 --2- 172.21.15.40:0/1805801477 >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 0x7fea3006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea35782640 1 --2- 172.21.15.40:0/1805801477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea3006e970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fea300b0ba0 tx=0x7fea2c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e1b6b035bf2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea27fff640 1 -- 172.21.15.40:0/1805801477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c037070 con 0x7fea3015f1e0 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea27fff640 1 -- 172.21.15.40:0/1805801477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c032440 con 0x7fea3015f1e0 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea27fff640 1 -- 172.21.15.40:0/1805801477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c032810 con 0x7fea3015f1e0 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea379e6640 1 -- 172.21.15.40:0/1805801477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 msgr2=0x7fea3006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.392+0000 7fea379e6640 1 --2- 172.21.15.40:0/1805801477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea3006e970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fea300b0ba0 tx=0x7fea2c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 -- 172.21.15.40:0/1805801477 shutdown_connections 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 --2- 172.21.15.40:0/1805801477 >> v2:172.21.15.184:3300/0 conn(0x7fea3006d2b0 0x7fea3006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 --2- 172.21.15.40:0/1805801477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea3006e970 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 -- 172.21.15.40:0/1805801477 >> 172.21.15.40:0/1805801477 conn(0x7fea300728a0 msgr2=0x7fea30072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 -- 172.21.15.40:0/1805801477 shutdown_connections 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.393+0000 7fea379e6640 1 -- 172.21.15.40:0/1805801477 wait complete. 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.394+0000 7fea379e6640 1 Processor -- start 2024-04-26T21:14:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.394+0000 7fea379e6640 1 -- start start 2024-04-26T21:14:59.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 0x7fea301565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 0x7fea30145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b5590 con 0x7fea3015f1e0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b52b0 con 0x7fea300b44e0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fea3015f5c0 con 0x7fea3006d2b0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52252/0 (socket says 172.21.15.40:52252) 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 -- 172.21.15.40:0/2872562698 learned_addr learned my addr 172.21.15.40:0/2872562698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35782640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 0x7fea301565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea34f81640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 0x7fea30145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 -- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 msgr2=0x7fea301565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 0x7fea301565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 -- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 msgr2=0x7fea30145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 0x7fea30145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 -- 172.21.15.40:0/2872562698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea30159bd0 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea34f81640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 0x7fea30145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35782640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 0x7fea301565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea35f83640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fea2800cf20 tx=0x7fea2800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2801a070 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2800d840 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea30159eb0 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.395+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2800dc30 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.396+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3015a470 con 0x7fea3015f1e0 2024-04-26T21:14:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.397+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea30069760 con 0x7fea3015f1e0 2024-04-26T21:14:59.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.397+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea2801d070 con 0x7fea3015f1e0 2024-04-26T21:14:59.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.398+0000 7fea25ffb640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.398+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea2808c480 con 0x7fea3015f1e0 2024-04-26T21:14:59.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.398+0000 7fea35782640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.399+0000 7fea35782640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 0x7fea1406edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea2c034000 tx=0x7fea2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.402+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea30069760 con 0x7fea3015f1e0 2024-04-26T21:14:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.625+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea3006a290 con 0x7fea3015f1e0 2024-04-26T21:14:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.626+0000 7fea25ffb640 1 -- 172.21.15.40:0/2872562698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7fea2805a820 con 0x7fea3015f1e0 2024-04-26T21:14:59.627 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:59.627 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:14:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 msgr2=0x7fea1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 0x7fea1406edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea2c034000 tx=0x7fea2c02e000 comp rx=0 tx=0).stop 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 msgr2=0x7fea300b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fea2800cf20 tx=0x7fea2800a7d0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 shutdown_connections 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea3015f1e0 0x7fea300b19b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea300b44e0 0x7fea30145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 --2- 172.21.15.40:0/2872562698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3006d2b0 0x7fea301565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 >> 172.21.15.40:0/2872562698 conn(0x7fea300728a0 msgr2=0x7fea30163680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 shutdown_connections 2024-04-26T21:14:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.628+0000 7fea379e6640 1 -- 172.21.15.40:0/2872562698 wait complete. 2024-04-26T21:14:59.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:14:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.739+0000 7f78970ea640 1 Processor -- start 2024-04-26T21:14:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.739+0000 7f78970ea640 1 -- start start 2024-04-26T21:14:59.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f78970ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f789015f720 0x7f78900b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f78970ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78900b2510 con 0x7f78900d5610 2024-04-26T21:14:59.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f78970ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78900b27f0 con 0x7f789015f340 2024-04-26T21:14:59.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f78970ea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f78900b2ad0 con 0x7f789015f720 2024-04-26T21:14:59.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f7895687640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78900d5610 0x7f78900b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59538/0 (socket says 172.21.15.40:59538) 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f7895687640 1 -- 172.21.15.40:0/3209456377 learned_addr learned my addr 172.21.15.40:0/3209456377 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f788ffff640 1 --2- 172.21.15.40:0/3209456377 >> v2:172.21.15.184:3300/0 conn(0x7f789015f720 0x7f78900b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212257953 0 0) 0x7f78900b2510 con 0x7f78900d5610 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.740+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f787c003520 con 0x7f78900d5610 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983549057 0 0) 0x7f78900b27f0 con 0x7f789015f340 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78900b2510 con 0x7f789015f340 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2211630000 0 0) 0x7f787c003520 con 0x7f78900d5610 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 >> v2:172.21.15.184:3300/0 conn(0x7f789015f720 msgr2=0x7f78900b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 --2- 172.21.15.40:0/3209456377 >> v2:172.21.15.184:3300/0 conn(0x7f789015f720 0x7f78900b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 >> v1:172.21.15.184:6789/0 conn(0x7f789015f340 legacy=0x7f78900d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78900b2db0 con 0x7f78900d5610 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788ffff640 1 --2- 172.21.15.40:0/3209456377 >> v2:172.21.15.184:3300/0 conn(0x7f789015f720 0x7f78900b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7878001670 con 0x7f78900d5610 2024-04-26T21:14:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 >> v1:172.21.15.40:6789/0 conn(0x7f78900d5610 legacy=0x7f78900b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c003b40 0x7f787c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c004800 0x7f787c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 0x7f787c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b2ad0 con 0x7f787c0051c0 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00c0f0 con 0x7f787c004800 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f787c00c2f0 con 0x7f787c003b40 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 0x7f787c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f788ffff640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c004800 0x7f787c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7894e86640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c003b40 0x7f787c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 -- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c003b40 msgr2=0x7f787c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c003b40 0x7f787c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 -- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c004800 msgr2=0x7f787c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c004800 0x7f787c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7895687640 1 -- 172.21.15.40:0/3209456377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c003520 con 0x7f787c0051c0 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.741+0000 7f7894e86640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c003b40 0x7f787c003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.742+0000 7f7895687640 1 -- 172.21.15.40:0/3209456377 reap_dead start 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.742+0000 7f7895687640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 0x7f787c00b9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f787800c160 tx=0x7f787c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f036c46776623d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.742+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878007cc0 con 0x7f787c0051c0 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.742+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7878005af0 con 0x7f787c0051c0 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.742+0000 7f788f7fe640 1 -- 172.21.15.40:0/3209456377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878007070 con 0x7f787c0051c0 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 -- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 msgr2=0x7f787c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 0x7f787c00b9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f787800c160 tx=0x7f787c005830 comp rx=0 tx=0).stop 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 -- 172.21.15.40:0/3209456377 shutdown_connections 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 --2- 172.21.15.40:0/3209456377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787c0051c0 0x7f787c00b9c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 -- 172.21.15.40:0/3209456377 >> 172.21.15.40:0/3209456377 conn(0x7f789008ec70 msgr2=0x7f789009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 -- 172.21.15.40:0/3209456377 shutdown_connections 2024-04-26T21:14:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.743+0000 7f78970ea640 1 -- 172.21.15.40:0/3209456377 wait complete. 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.744+0000 7f78970ea640 1 Processor -- start 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.744+0000 7f78970ea640 1 -- start start 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0051c0 0x7f789015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78900b8830 0x7f78900b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b2210 con 0x7f78900b7e70 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b2090 con 0x7f78900b8830 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f78970ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78900d3d40 con 0x7f787c0051c0 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52266/0 (socket says 172.21.15.40:52266) 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 -- 172.21.15.40:0/1175045218 learned_addr learned my addr 172.21.15.40:0/1175045218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f7894e86640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0051c0 0x7f789015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f7895687640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78900b8830 0x7f78900b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 -- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0051c0 msgr2=0x7f789015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0051c0 0x7f789015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 -- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78900b8830 msgr2=0x7f78900b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78900b8830 0x7f78900b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 -- 172.21.15.40:0/1175045218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78900b8cb0 con 0x7f78900b7e70 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.745+0000 7f788ffff640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f788000f490 tx=0x7f788000f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.746+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880003b90 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.746+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7880003d30 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.746+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789015e200 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.746+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880004270 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.746+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78900cad10 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.747+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789006a290 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.747+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7880004410 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.748+0000 7f788d7fa640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.748+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f788008cec0 con 0x7f78900b7e70 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.748+0000 7f7894e86640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.748+0000 7f7894e86640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 0x7f787006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7888007fd0 tx=0x7f7888009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.752+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7880019070 con 0x7f78900b7e70 2024-04-26T21:14:59.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.937+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78900d3020 con 0x7f78900b7e70 2024-04-26T21:14:59.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.938+0000 7f788d7fa640 1 -- 172.21.15.40:0/1175045218 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f7880010040 con 0x7f78900b7e70 2024-04-26T21:14:59.939 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:14:59.939 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 msgr2=0x7f787006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 0x7f787006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7888007fd0 tx=0x7f7888009040 comp rx=0 tx=0).stop 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 msgr2=0x7f78900b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f788000f490 tx=0x7f788000f950 comp rx=0 tx=0).stop 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 shutdown_connections 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78900b8830 0x7f78900b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78900b7e70 0x7f78900b82f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 --2- 172.21.15.40:0/1175045218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f787c0051c0 0x7f789015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:59.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 >> 172.21.15.40:0/1175045218 conn(0x7f789008ec70 msgr2=0x7f789006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:59.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 shutdown_connections 2024-04-26T21:14:59.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:14:59.940+0000 7f78970ea640 1 -- 172.21.15.40:0/1175045218 wait complete. 2024-04-26T21:14:59.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:14:41.855331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:14:42.545028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:14:59.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:00.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.055+0000 7ff67759d640 1 Processor -- start 2024-04-26T21:15:00.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.055+0000 7ff67759d640 1 -- start start 2024-04-26T21:15:00.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff67759d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff67006f340 0x7ff6700b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff67759d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 0x7ff67006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff67759d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff67006fc60 con 0x7ff6700b3e50 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff67759d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff6700b46a0 con 0x7ff67006f340 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff67759d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6700b5db0 con 0x7ff6700d0300 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff674b38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 0x7ff67006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff675339640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff67006f340 0x7ff6700b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff674b38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 0x7ff67006f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35384/0 (socket says 172.21.15.40:35384) 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff675b3a640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff6700b3e50 0x7ff6700720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41788/0 (socket says 172.21.15.40:41788) 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff674b38640 1 -- 172.21.15.40:0/521437827 learned_addr learned my addr 172.21.15.40:0/521437827 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.056+0000 7ff675339640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff67006f340 0x7ff6700b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:51254/0 (socket says 172.21.15.40:51254) 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.057+0000 7ff675339640 1 -- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 msgr2=0x7ff67006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.057+0000 7ff675339640 1 --2- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 0x7ff67006f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.057+0000 7ff675339640 1 -- 172.21.15.40:0/521437827 >> v1:172.21.15.184:6789/0 conn(0x7ff6700b3e50 legacy=0x7ff6700720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.057+0000 7ff675339640 1 -- 172.21.15.40:0/521437827 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700b6090 con 0x7ff67006f340 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.057+0000 7ff675339640 1 --2- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67006f340 0x7ff6700b6b70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097a0 tx=0x7ff66802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6293ce9421ae6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff667fff640 1 -- 172.21.15.40:0/521437827 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668037070 con 0x7ff67006f340 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff667fff640 1 -- 172.21.15.40:0/521437827 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66802adb0 con 0x7ff67006f340 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff667fff640 1 -- 172.21.15.40:0/521437827 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668032660 con 0x7ff67006f340 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 -- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67006f340 msgr2=0x7ff6700b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 --2- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67006f340 0x7ff6700b6b70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097a0 tx=0x7ff66802a060 comp rx=0 tx=0).stop 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 -- 172.21.15.40:0/521437827 shutdown_connections 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 --2- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700d0300 0x7ff67006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 --2- 172.21.15.40:0/521437827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff67006f340 0x7ff6700b6b70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 -- 172.21.15.40:0/521437827 >> 172.21.15.40:0/521437827 conn(0x7ff6700cd8a0 msgr2=0x7ff6700cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 -- 172.21.15.40:0/521437827 shutdown_connections 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.058+0000 7ff67759d640 1 -- 172.21.15.40:0/521437827 wait complete. 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.059+0000 7ff67759d640 1 Processor -- start 2024-04-26T21:15:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.059+0000 7ff67759d640 1 -- start start 2024-04-26T21:15:00.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 0x7ff6700c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6700d0300 0x7ff67015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700b42e0 con 0x7ff67006f340 2024-04-26T21:15:00.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700b6650 con 0x7ff6700d0300 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff67759d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff6700b4460 con 0x7ff6700b3e50 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff675339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff675339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52268/0 (socket says 172.21.15.40:52268) 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff675339640 1 -- 172.21.15.40:0/1495805013 learned_addr learned my addr 172.21.15.40:0/1495805013 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff674b38640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 0x7ff6700c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.060+0000 7ff675b3a640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6700d0300 0x7ff67015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 -- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 msgr2=0x7ff6700c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 0x7ff6700c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 -- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6700d0300 msgr2=0x7ff67015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6700d0300 0x7ff67015b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 -- 172.21.15.40:0/1495805013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700c7f90 con 0x7ff67006f340 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff674b38640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 0x7ff6700c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff675339640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff668035040 tx=0x7ff6680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668041020 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66802abb0 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6700c8270 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668037040 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.061+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6700c8830 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.062+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67006a290 con 0x7ff67006f340 2024-04-26T21:15:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.063+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff668044020 con 0x7ff67006f340 2024-04-26T21:15:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.063+0000 7ff665ffb640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 0x7ff65406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.063+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff6680ac010 con 0x7ff67006f340 2024-04-26T21:15:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.063+0000 7ff674b38640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 0x7ff65406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.064+0000 7ff674b38640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 0x7ff65406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff6701f3de0 tx=0x7ff66000b450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.067+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff66804b030 con 0x7ff67006f340 2024-04-26T21:15:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.252+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff67006d2b0 con 0x7ff67006f340 2024-04-26T21:15:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.252+0000 7ff665ffb640 1 -- 172.21.15.40:0/1495805013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7ff66807a440 con 0x7ff67006f340 2024-04-26T21:15:00.253 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 msgr2=0x7ff65406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 0x7ff65406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff6701f3de0 tx=0x7ff66000b450 comp rx=0 tx=0).stop 2024-04-26T21:15:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 msgr2=0x7ff670133a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff668035040 tx=0x7ff6680046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 shutdown_connections 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff65406c8f0 0x7ff65406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6700d0300 0x7ff67015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6700b3e50 0x7ff6700c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 --2- 172.21.15.40:0/1495805013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff67006f340 0x7ff670133a70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 >> 172.21.15.40:0/1495805013 conn(0x7ff6700cd8a0 msgr2=0x7ff6700cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 shutdown_connections 2024-04-26T21:15:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.254+0000 7ff67759d640 1 -- 172.21.15.40:0/1495805013 wait complete. 2024-04-26T21:15:00.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:15:00.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.369+0000 7f27f07ac640 1 Processor -- start 2024-04-26T21:15:00.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.369+0000 7f27f07ac640 1 -- start start 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27f07ac640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f27e80b2000 0x7f27e806ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27f07ac640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e809c790 con 0x7f27e81631f0 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27f07ac640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e8163ec0 con 0x7f27e80d1170 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27f07ac640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f27e80d1600 con 0x7f27e80b2000 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27ee548640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27e81631f0 0x7f27e81635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59552/0 (socket says 172.21.15.40:59552) 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27ee548640 1 -- 172.21.15.40:0/1818826660 learned_addr learned my addr 172.21.15.40:0/1818826660 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716063795 0 0) 0x7f27e809c790 con 0x7f27e81631f0 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27edd47640 1 --2- 172.21.15.40:0/1818826660 >> v2:172.21.15.184:3300/0 conn(0x7f27e80b2000 0x7f27e806ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.370+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d0003520 con 0x7f27e81631f0 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1082558434 0 0) 0x7f27d0003520 con 0x7f27e81631f0 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 >> v2:172.21.15.184:3300/0 conn(0x7f27e80b2000 msgr2=0x7f27e806ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 --2- 172.21.15.40:0/1818826660 >> v2:172.21.15.184:3300/0 conn(0x7f27e80b2000 0x7f27e806ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 >> v1:172.21.15.184:6789/0 conn(0x7f27e80d1170 legacy=0x7f27e80b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80d1a40 con 0x7f27e81631f0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f27e0001670 con 0x7f27e81631f0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 >> v1:172.21.15.40:6789/0 conn(0x7f27e81631f0 legacy=0x7f27e81635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d00039b0 0x7f27d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27d0004800 0x7f27d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27d000c130 con 0x7f27d00051c0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27d000c2e0 con 0x7f27d00039b0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f27d000c4e0 con 0x7f27d0004800 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27ee548640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d00039b0 0x7f27d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27edd47640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27d0004800 0x7f27d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 -- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27d0004800 msgr2=0x7f27d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27d0004800 0x7f27d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 -- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d00039b0 msgr2=0x7f27d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27d00039b0 0x7f27d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.371+0000 7f27eed49640 1 -- 172.21.15.40:0/1818826660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d0003520 con 0x7f27d00051c0 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27eed49640 1 -- 172.21.15.40:0/1818826660 reap_dead start 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27eed49640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27d000b9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001e50 tx=0x7f27d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4021fa71efcf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8001a70 con 0x7f27d00051c0 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d8003970 con 0x7f27d00051c0 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27ed546640 1 -- 172.21.15.40:0/1818826660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8003d60 con 0x7f27d00051c0 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27f07ac640 1 -- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 msgr2=0x7f27d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.372+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27d000b9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001e50 tx=0x7f27d0005830 comp rx=0 tx=0).stop 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.373+0000 7f27f07ac640 1 -- 172.21.15.40:0/1818826660 shutdown_connections 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.373+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1818826660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27d000b9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.373+0000 7f27f07ac640 1 -- 172.21.15.40:0/1818826660 >> 172.21.15.40:0/1818826660 conn(0x7f27e809ba40 msgr2=0x7f27e809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.373+0000 7f27f07ac640 1 -- 172.21.15.40:0/1818826660 shutdown_connections 2024-04-26T21:15:00.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.373+0000 7f27f07ac640 1 -- 172.21.15.40:0/1818826660 wait complete. 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.374+0000 7f27f07ac640 1 Processor -- start 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.374+0000 7f27f07ac640 1 -- start start 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.374+0000 7f27f07ac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27e815b280 0x7f27e81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 0x7f27e8161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80d0230 con 0x7f27d00051c0 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80d2000 con 0x7f27e8161810 2024-04-26T21:15:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f27e81f8eb0 con 0x7f27e815b280 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52282/0 (socket says 172.21.15.40:52282) 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 -- 172.21.15.40:0/1338627427 learned_addr learned my addr 172.21.15.40:0/1338627427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27edd47640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27e815b280 0x7f27e81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27eed49640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 0x7f27e8161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 -- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27e815b280 msgr2=0x7f27e81582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27e815b280 0x7f27e81582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 -- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 msgr2=0x7f27e8161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 0x7f27e8161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 -- 172.21.15.40:0/1338627427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e81f9150 con 0x7f27d00051c0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27eed49640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 0x7f27e8161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27ee548640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f27e0001050 tx=0x7f27e0007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0004e10 con 0x7f27d00051c0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0018070 con 0x7f27d00051c0 2024-04-26T21:15:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e81f9430 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e000fbc0 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.375+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e81f99f0 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.376+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e806a290 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.377+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27e001d460 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.377+0000 7f27e6ffd640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 0x7f27d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.377+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f27e008bb50 con 0x7f27d00051c0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.378+0000 7f27edd47640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 0x7f27d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.378+0000 7f27edd47640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 0x7f27d406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27e815b810 tx=0x7f27dc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.381+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27e0059f80 con 0x7f27d00051c0 2024-04-26T21:15:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.567+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f27e80d06d0 con 0x7f27d00051c0 2024-04-26T21:15:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.567+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1338627427 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f27e0059880 con 0x7f27d00051c0 2024-04-26T21:15:00.569 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 msgr2=0x7f27d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 0x7f27d406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27e815b810 tx=0x7f27dc009190 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 msgr2=0x7f27e81472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f27e0001050 tx=0x7f27e0007660 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 shutdown_connections 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f27d406c8f0 0x7f27d406edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e8161810 0x7f27e8161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27e815b280 0x7f27e81582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 --2- 172.21.15.40:0/1338627427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d00051c0 0x7f27e81472e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 >> 172.21.15.40:0/1338627427 conn(0x7f27e809ba40 msgr2=0x7f27e8071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 shutdown_connections 2024-04-26T21:15:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.569+0000 7f27f07ac640 1 -- 172.21.15.40:0/1338627427 wait complete. 2024-04-26T21:15:00.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:15:00.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.683+0000 7f2d41808640 1 Processor -- start 2024-04-26T21:15:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.683+0000 7f2d41808640 1 -- start start 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d41808640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c070640 0x7f2d3c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d41808640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d3c071160 0x7f2d3c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d41808640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c0d2470 con 0x7f2d3c162ca0 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d41808640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3c0715d0 con 0x7f2d3c071160 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d41808640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c0cdc40 con 0x7f2d3c070640 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d3b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d3c071160 0x7f2d3c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d3b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d3c071160 0x7f2d3c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39120/0 (socket says 172.21.15.40:39120) 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.684+0000 7f2d3b7fe640 1 -- 172.21.15.40:0/2622752204 learned_addr learned my addr 172.21.15.40:0/2622752204 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3a7fc640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c070640 0x7f2d3c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3b7fe640 1 -- 172.21.15.40:0/2622752204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c070640 msgr2=0x7f2d3c070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3b7fe640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c070640 0x7f2d3c070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3b7fe640 1 -- 172.21.15.40:0/2622752204 >> v1:172.21.15.184:6789/0 conn(0x7f2d3c162ca0 legacy=0x7f2d3c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3b7fe640 1 -- 172.21.15.40:0/2622752204 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c0d2750 con 0x7f2d3c071160 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.685+0000 7f2d3b7fe640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c071160 0x7f2d3c0ce0d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009870 tx=0x7f2d3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c39f0926edb3283 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d39ffb640 1 -- 172.21.15.40:0/2622752204 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34037070 con 0x7f2d3c071160 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d39ffb640 1 -- 172.21.15.40:0/2622752204 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d34032440 con 0x7f2d3c071160 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d39ffb640 1 -- 172.21.15.40:0/2622752204 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34032810 con 0x7f2d3c071160 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 -- 172.21.15.40:0/2622752204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c071160 msgr2=0x7f2d3c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c071160 0x7f2d3c0ce0d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009870 tx=0x7f2d3402a270 comp rx=0 tx=0).stop 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 -- 172.21.15.40:0/2622752204 shutdown_connections 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c071160 0x7f2d3c0ce0d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 --2- 172.21.15.40:0/2622752204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c070640 0x7f2d3c070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 -- 172.21.15.40:0/2622752204 >> 172.21.15.40:0/2622752204 conn(0x7f2d3c09ba40 msgr2=0x7f2d3c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 -- 172.21.15.40:0/2622752204 shutdown_connections 2024-04-26T21:15:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.686+0000 7f2d41808640 1 -- 172.21.15.40:0/2622752204 wait complete. 2024-04-26T21:15:00.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.687+0000 7f2d41808640 1 Processor -- start 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.687+0000 7f2d41808640 1 -- start start 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d3c071160 0x7f2d3c0a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c162ca0 0x7f2d3c09e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0ce610 con 0x7f2d3c070640 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0cd8d0 con 0x7f2d3c071160 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d41808640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c0ce790 con 0x7f2d3c162ca0 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d3affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d3affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39134/0 (socket says 172.21.15.40:39134) 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.688+0000 7f2d3affd640 1 -- 172.21.15.40:0/3880661187 learned_addr learned my addr 172.21.15.40:0/3880661187 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 -- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c162ca0 msgr2=0x7f2d3c09e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3b7fe640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c162ca0 0x7f2d3c09e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3a7fc640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d3c071160 0x7f2d3c0a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c162ca0 0x7f2d3c09e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 -- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d3c071160 msgr2=0x7f2d3c0a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d3c071160 0x7f2d3c0a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 -- 172.21.15.40:0/3880661187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c150270 con 0x7f2d3c070640 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d3affd640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00c760 tx=0x7f2d2c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c016020 con 0x7f2d3c070640 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2c006d50 con 0x7f2d3c070640 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c14d630 con 0x7f2d3c070640 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c003bb0 con 0x7f2d3c070640 2024-04-26T21:15:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.689+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c14ded0 con 0x7f2d3c070640 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.690+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c0693d0 con 0x7f2d3c070640 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.690+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d2c00a850 con 0x7f2d3c070640 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.691+0000 7f2d33fff640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.691+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2d2c08b780 con 0x7f2d3c070640 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.691+0000 7f2d3a7fc640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.692+0000 7f2d3a7fc640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0c9a20 tx=0x7f2d24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.695+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d2c020020 con 0x7f2d3c070640 2024-04-26T21:15:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.879+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d3c0b26e0 con 0x7f2d3c070640 2024-04-26T21:15:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.879+0000 7f2d33fff640 1 -- 172.21.15.40:0/3880661187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7f2d3c150270 con 0x7f2d3c070640 2024-04-26T21:15:00.881 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:15:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 msgr2=0x7f2d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0c9a20 tx=0x7f2d24008040 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 msgr2=0x7f2d3c14fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00c760 tx=0x7f2d2c00cc20 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 shutdown_connections 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d1c06c8f0 0x7f2d1c06edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d3c162ca0 0x7f2d3c09e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d3c071160 0x7f2d3c0a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 --2- 172.21.15.40:0/3880661187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d3c070640 0x7f2d3c14fd30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 >> 172.21.15.40:0/3880661187 conn(0x7f2d3c09ba40 msgr2=0x7f2d3c0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 shutdown_connections 2024-04-26T21:15:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.881+0000 7f2d41808640 1 -- 172.21.15.40:0/3880661187 wait complete. 2024-04-26T21:15:00.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.993+0000 7f8821d71640 1 Processor -- start 2024-04-26T21:15:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.993+0000 7f8821d71640 1 -- start start 2024-04-26T21:15:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f8821d71640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f881c15b4e0 0x7f881c071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f8821d71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c0733e0 con 0x7f881c06f2c0 2024-04-26T21:15:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f8821d71640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c072f00 con 0x7f881c15b8c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f8821d71640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f881c072660 con 0x7f881c15b4e0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f881c06f2c0 0x7f881c0b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51366/0 (socket says 172.21.15.40:51366) 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881bfff640 1 -- 172.21.15.40:0/3144448387 learned_addr learned my addr 172.21.15.40:0/3144448387 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881b7fe640 1 --2- 172.21.15.40:0/3144448387 >> v2:172.21.15.184:3300/0 conn(0x7f881c15b4e0 0x7f881c071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375999528 0 0) 0x7f881c0733e0 con 0x7f881c06f2c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8804003520 con 0x7f881c06f2c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 181855277 0 0) 0x7f8804003520 con 0x7f881c06f2c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 >> v2:172.21.15.184:3300/0 conn(0x7f881c15b4e0 msgr2=0x7f881c071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 --2- 172.21.15.40:0/3144448387 >> v2:172.21.15.184:3300/0 conn(0x7f881c15b4e0 0x7f881c071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 >> v1:172.21.15.184:6789/0 conn(0x7f881c15b8c0 legacy=0x7f881c06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c072940 con 0x7f881c06f2c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f88140019f0 con 0x7f881c06f2c0 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 >> v1:172.21.15.40:6789/0 conn(0x7f881c06f2c0 legacy=0x7f881c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88040039b0 0x7f8804003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8804004800 0x7f8804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 0x7f880400cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f881c072660 con 0x7f88040051c0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f880400d270 con 0x7f88040039b0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.994+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f880400d470 con 0x7f8804004800 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 0x7f880400cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881affd640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8804004800 0x7f8804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881b7fe640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88040039b0 0x7f8804003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 -- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8804004800 msgr2=0x7f8804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8804004800 0x7f8804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 -- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88040039b0 msgr2=0x7f8804003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88040039b0 0x7f8804003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 -- 172.21.15.40:0/3144448387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8804003520 con 0x7f88040051c0 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.995+0000 7f881bfff640 1 -- 172.21.15.40:0/3144448387 reap_dead start 2024-04-26T21:15:00.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f881bfff640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 0x7f880400cad0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f881400a860 tx=0x7f8804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59af37e8164343a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881400a480 con 0x7f88040051c0 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881402f070 con 0x7f88040051c0 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f881a7fc640 1 -- 172.21.15.40:0/3144448387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881402bcc0 con 0x7f88040051c0 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 -- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 msgr2=0x7f880400cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 0x7f880400cad0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f881400a860 tx=0x7f8804005830 comp rx=0 tx=0).stop 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 -- 172.21.15.40:0/3144448387 shutdown_connections 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 --2- 172.21.15.40:0/3144448387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88040051c0 0x7f880400cad0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 -- 172.21.15.40:0/3144448387 >> 172.21.15.40:0/3144448387 conn(0x7f881c0d0d30 msgr2=0x7f881c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 -- 172.21.15.40:0/3144448387 shutdown_connections 2024-04-26T21:15:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.996+0000 7f8821d71640 1 -- 172.21.15.40:0/3144448387 wait complete. 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.997+0000 7f8821d71640 1 Processor -- start 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 -- start start 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 0x7f881c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f881c1597d0 0x7f881c1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f881c073820 con 0x7f881c149c60 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0b3e80 con 0x7f881c1597d0 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.998+0000 7f8821d71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f881c0739a0 con 0x7f88040051c0 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39164/0 (socket says 172.21.15.40:39164) 2024-04-26T21:15:01.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 -- 172.21.15.40:0/2782173044 learned_addr learned my addr 172.21.15.40:0/2782173044 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881b7fe640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 0x7f881c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881bfff640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f881c1597d0 0x7f881c1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 -- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 msgr2=0x7f881c157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 0x7f881c157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 -- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f881c1597d0 msgr2=0x7f881c1e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f881c1597d0 0x7f881c1e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 -- 172.21.15.40:0/2782173044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1e6270 con 0x7f881c149c60 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881b7fe640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 0x7f881c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f881affd640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f881000f5f0 tx=0x7f881000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88100052e0 con 0x7f881c149c60 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810018070 con 0x7f881c149c60 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810003830 con 0x7f881c149c60 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c14ed70 con 0x7f881c149c60 2024-04-26T21:15:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:00.999+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1e6810 con 0x7f881c149c60 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.000+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c06a290 con 0x7f881c149c60 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.001+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88100039d0 con 0x7f881c149c60 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.001+0000 7f8803fff640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 0x7f880806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.001+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f881008b8b0 con 0x7f881c149c60 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.001+0000 7f881b7fe640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 0x7f880806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.002+0000 7f881b7fe640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 0x7f880806ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f880c009b40 tx=0x7f880c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.005+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8810059ce0 con 0x7f881c149c60 2024-04-26T21:15:01.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.230+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f881c0b65f0 con 0x7f881c149c60 2024-04-26T21:15:01.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.230+0000 7f8803fff640 1 -- 172.21.15.40:0/2782173044 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7f88100595e0 con 0x7f881c149c60 2024-04-26T21:15:01.232 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:01.232 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:15:01.232 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:15:01.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 msgr2=0x7f880806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 0x7f880806ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f880c009b40 tx=0x7f880c008040 comp rx=0 tx=0).stop 2024-04-26T21:15:01.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 msgr2=0x7f881c159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f881000f5f0 tx=0x7f881000fab0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 shutdown_connections 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f880806c920 0x7f880806ede0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f881c1597d0 0x7f881c1e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881c149c60 0x7f881c159290 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 --2- 172.21.15.40:0/2782173044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88040051c0 0x7f881c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 >> 172.21.15.40:0/2782173044 conn(0x7f881c0d0d30 msgr2=0x7f881c0ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 shutdown_connections 2024-04-26T21:15:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.232+0000 7f8821d71640 1 -- 172.21.15.40:0/2782173044 wait complete. 2024-04-26T21:15:01.244 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.343+0000 7fe710d6a640 1 Processor -- start 2024-04-26T21:15:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.343+0000 7fe710d6a640 1 -- start start 2024-04-26T21:15:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe710d6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe70c0b61a0 0x7fe70c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe710d6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 0x7fe70c0720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe710d6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70c0b4270 con 0x7fe70c145980 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe710d6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe70c0b5140 con 0x7fe70c0b61a0 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe710d6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe70c0b4530 con 0x7fe70c0b3de0 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe709d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe70c0b61a0 0x7fe70c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe70a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe70c145980 0x7fe70c0b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51378/0 (socket says 172.21.15.40:51378) 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe70a59c640 1 -- 172.21.15.40:0/1636729837 learned_addr learned my addr 172.21.15.40:0/1636729837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe70ad9d640 1 --2- 172.21.15.40:0/1636729837 >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 0x7fe70c0720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe70959a640 1 -- 172.21.15.40:0/1636729837 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1696991023 0 0) 0x7fe70c0b4270 con 0x7fe70c145980 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe70959a640 1 -- 172.21.15.40:0/1636729837 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6f4003520 con 0x7fe70c145980 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe709d9b640 1 -- 172.21.15.40:0/1636729837 >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 msgr2=0x7fe70c0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.344+0000 7fe709d9b640 1 --2- 172.21.15.40:0/1636729837 >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 0x7fe70c0720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe709d9b640 1 -- 172.21.15.40:0/1636729837 >> v1:172.21.15.40:6789/0 conn(0x7fe70c145980 legacy=0x7fe70c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe709d9b640 1 -- 172.21.15.40:0/1636729837 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c0b4810 con 0x7fe70c0b61a0 2024-04-26T21:15:01.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe70ad9d640 1 --2- 172.21.15.40:0/1636729837 >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 0x7fe70c0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe709d9b640 1 --2- 172.21.15.40:0/1636729837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b61a0 0x7fe70c160630 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0b52c0 tx=0x7fe70002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc232380fd9d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe70959a640 1 -- 172.21.15.40:0/1636729837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700037070 con 0x7fe70c0b61a0 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe70959a640 1 -- 172.21.15.40:0/1636729837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700032440 con 0x7fe70c0b61a0 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.345+0000 7fe70959a640 1 -- 172.21.15.40:0/1636729837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70002ae60 con 0x7fe70c0b61a0 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 -- 172.21.15.40:0/1636729837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b61a0 msgr2=0x7fe70c160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 --2- 172.21.15.40:0/1636729837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b61a0 0x7fe70c160630 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0b52c0 tx=0x7fe70002a110 comp rx=0 tx=0).stop 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 -- 172.21.15.40:0/1636729837 shutdown_connections 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 --2- 172.21.15.40:0/1636729837 >> v2:172.21.15.184:3300/0 conn(0x7fe70c0b3de0 0x7fe70c0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 --2- 172.21.15.40:0/1636729837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b61a0 0x7fe70c160630 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 -- 172.21.15.40:0/1636729837 >> 172.21.15.40:0/1636729837 conn(0x7fe70c0d0d30 msgr2=0x7fe70c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 -- 172.21.15.40:0/1636729837 shutdown_connections 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.346+0000 7fe710d6a640 1 -- 172.21.15.40:0/1636729837 wait complete. 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.347+0000 7fe710d6a640 1 Processor -- start 2024-04-26T21:15:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.347+0000 7fe710d6a640 1 -- start start 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe70c0b61a0 0x7fe70c13ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe70c145980 0x7fe70c143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c072610 con 0x7fe70c0b3de0 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0b4dd0 con 0x7fe70c0b61a0 2024-04-26T21:15:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe710d6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe70c072790 con 0x7fe70c145980 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe70a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe70a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39180/0 (socket says 172.21.15.40:39180) 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe70a59c640 1 -- 172.21.15.40:0/545750127 learned_addr learned my addr 172.21.15.40:0/545750127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe70ad9d640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe70c145980 0x7fe70c143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.348+0000 7fe709d9b640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe70c0b61a0 0x7fe70c13ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 -- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe70c145980 msgr2=0x7fe70c143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe70c145980 0x7fe70c143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 -- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe70c0b61a0 msgr2=0x7fe70c13ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe70c0b61a0 0x7fe70c13ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 -- 172.21.15.40:0/545750127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c13cb20 con 0x7fe70c0b3de0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe70a59c640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c390 tx=0x7fe70400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704017070 con 0x7fe70c0b3de0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe704005ce0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040072f0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1f4330 con 0x7fe70c0b3de0 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.349+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1f48f0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.350+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c06a290 con 0x7fe70c0b3de0 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.351+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe70401a070 con 0x7fe70c0b3de0 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.351+0000 7fe6faffd640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 0x7fe6ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.351+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fe70408aec0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.351+0000 7fe709d9b640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 0x7fe6ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.352+0000 7fe709d9b640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 0x7fe6ec06edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe700034000 tx=0x7fe70002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.355+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe704058ca0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:15:01.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:15:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.546+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe70c073cc0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.548+0000 7fe6faffd640 1 -- 172.21.15.40:0/545750127 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7fe70405cfa0 con 0x7fe70c0b3de0 2024-04-26T21:15:01.549 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:01.549 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 msgr2=0x7fe6ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 0x7fe6ec06edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe700034000 tx=0x7fe70002e000 comp rx=0 tx=0).stop 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 msgr2=0x7fe70c139c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c390 tx=0x7fe70400c850 comp rx=0 tx=0).stop 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 shutdown_connections 2024-04-26T21:15:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe6ec06c8f0 0x7fe6ec06edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe70c145980 0x7fe70c143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe70c0b61a0 0x7fe70c13ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 --2- 172.21.15.40:0/545750127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe70c0b3de0 0x7fe70c139c30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 >> 172.21.15.40:0/545750127 conn(0x7fe70c0d0d30 msgr2=0x7fe70c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 shutdown_connections 2024-04-26T21:15:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.550+0000 7fe710d6a640 1 -- 172.21.15.40:0/545750127 wait complete. 2024-04-26T21:15:01.562 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:15:01.562 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:01.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.662+0000 7fc186e28640 1 Processor -- start 2024-04-26T21:15:01.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.662+0000 7fc186e28640 1 -- start start 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc186e28640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc18006e490 0x7fc18015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc186e28640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800d1740 con 0x7fc1800d11b0 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc186e28640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800d1a20 con 0x7fc1800cf2e0 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc186e28640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1800d1d00 con 0x7fc18006e490 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc1853c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc18006e490 0x7fc18015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc17ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1800d11b0 0x7fc18006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51386/0 (socket says 172.21.15.40:51386) 2024-04-26T21:15:01.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc1853c5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc18006e490 0x7fc18015ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39196/0 (socket says 172.21.15.40:39196) 2024-04-26T21:15:01.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc17ffff640 1 -- 172.21.15.40:0/2350929134 learned_addr learned my addr 172.21.15.40:0/2350929134 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:01.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc1853c5640 1 -- 172.21.15.40:0/2350929134 >> v1:172.21.15.184:6789/0 conn(0x7fc1800cf2e0 legacy=0x7fc18006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:01.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc1853c5640 1 -- 172.21.15.40:0/2350929134 >> v1:172.21.15.40:6789/0 conn(0x7fc1800d11b0 legacy=0x7fc18006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.663+0000 7fc1853c5640 1 -- 172.21.15.40:0/2350929134 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800d1fe0 con 0x7fc18006e490 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.664+0000 7fc1853c5640 1 --2- 172.21.15.40:0/2350929134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18006e490 0x7fc18015ebe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc178009dc0 tx=0x7fc17802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aa2af9e52ead03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.664+0000 7fc17f7fe640 1 -- 172.21.15.40:0/2350929134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178007740 con 0x7fc18006e490 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.664+0000 7fc17f7fe640 1 -- 172.21.15.40:0/2350929134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1780078e0 con 0x7fc18006e490 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.664+0000 7fc17f7fe640 1 -- 172.21.15.40:0/2350929134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178007cb0 con 0x7fc18006e490 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 -- 172.21.15.40:0/2350929134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18006e490 msgr2=0x7fc18015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 --2- 172.21.15.40:0/2350929134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18006e490 0x7fc18015ebe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc178009dc0 tx=0x7fc17802a6d0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 -- 172.21.15.40:0/2350929134 shutdown_connections 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 --2- 172.21.15.40:0/2350929134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc18006e490 0x7fc18015ebe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 -- 172.21.15.40:0/2350929134 >> 172.21.15.40:0/2350929134 conn(0x7fc1800b4a60 msgr2=0x7fc1800b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 -- 172.21.15.40:0/2350929134 shutdown_connections 2024-04-26T21:15:01.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.665+0000 7fc186e28640 1 -- 172.21.15.40:0/2350929134 wait complete. 2024-04-26T21:15:01.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.666+0000 7fc186e28640 1 Processor -- start 2024-04-26T21:15:01.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.666+0000 7fc186e28640 1 -- start start 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 0x7fc1801f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1800cf2e0 0x7fc1800cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800cf840 con 0x7fc1800d11b0 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800cf6c0 con 0x7fc18006e490 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc186e28640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1800ce660 con 0x7fc1800cf2e0 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39200/0 (socket says 172.21.15.40:39200) 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 -- 172.21.15.40:0/424573040 learned_addr learned my addr 172.21.15.40:0/424573040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc17ffff640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1800cf2e0 0x7fc1800cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc184bc4640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 0x7fc1801f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 -- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1800cf2e0 msgr2=0x7fc1800cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1800cf2e0 0x7fc1800cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 -- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 msgr2=0x7fc1801f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 0x7fc1801f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 -- 172.21.15.40:0/424573040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800caf50 con 0x7fc1800d11b0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc184bc4640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 0x7fc1801f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc1853c5640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc178009ee0 tx=0x7fc178004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.667+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178040070 con 0x7fc1800d11b0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.668+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc178031c80 con 0x7fc1800d11b0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.668+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178003b90 con 0x7fc1800d11b0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.668+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1800cb230 con 0x7fc1800d11b0 2024-04-26T21:15:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.668+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1800cb7f0 con 0x7fc1800d11b0 2024-04-26T21:15:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.669+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18006a290 con 0x7fc1800d11b0 2024-04-26T21:15:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.669+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc178039040 con 0x7fc1800d11b0 2024-04-26T21:15:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.669+0000 7fc17d7fa640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 0x7fc16406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.670+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc1780acdb0 con 0x7fc1800d11b0 2024-04-26T21:15:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.670+0000 7fc184bc4640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 0x7fc16406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.670+0000 7fc184bc4640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 0x7fc16406edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cd0d0 tx=0x7fc174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.674+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc17807ab90 con 0x7fc1800d11b0 2024-04-26T21:15:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.895+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1800cbbd0 con 0x7fc1800d11b0 2024-04-26T21:15:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.895+0000 7fc17d7fa640 1 -- 172.21.15.40:0/424573040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7fc17807ee90 con 0x7fc1800d11b0 2024-04-26T21:15:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:01.897 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:15:01.897 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:15:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 msgr2=0x7fc16406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 0x7fc16406edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cd0d0 tx=0x7fc174009040 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 msgr2=0x7fc1800c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc178009ee0 tx=0x7fc178004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 shutdown_connections 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc16406c8f0 0x7fc16406edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1800d11b0 0x7fc1800c8a90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1800cf2e0 0x7fc1800cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 --2- 172.21.15.40:0/424573040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18006e490 0x7fc1801f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 >> 172.21.15.40:0/424573040 conn(0x7fc1800b4a60 msgr2=0x7fc1800755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 shutdown_connections 2024-04-26T21:15:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:01.897+0000 7fc186e28640 1 -- 172.21.15.40:0/424573040 wait complete. 2024-04-26T21:15:01.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:02.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 Processor -- start 2024-04-26T21:15:02.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 -- start start 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f302415f750 0x7f30240cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240d03a0 con 0x7f30240d3ee0 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240d0680 con 0x7f302415f370 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f302a59d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f30240d0960 con 0x7f302415f750 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.010+0000 7f3028b3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f302415f750 0x7f30240cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3028b3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f302415f750 0x7f30240cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39210/0 (socket says 172.21.15.40:39210) 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f30237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f30240d3ee0 0x7f30240b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51390/0 (socket says 172.21.15.40:51390) 2024-04-26T21:15:02.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3028b3a640 1 -- 172.21.15.40:0/2604445214 learned_addr learned my addr 172.21.15.40:0/2604445214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:02.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3028b3a640 1 -- 172.21.15.40:0/2604445214 >> v1:172.21.15.184:6789/0 conn(0x7f302415f370 legacy=0x7f30240d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3028b3a640 1 -- 172.21.15.40:0/2604445214 >> v1:172.21.15.40:6789/0 conn(0x7f30240d3ee0 legacy=0x7f30240b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3028b3a640 1 -- 172.21.15.40:0/2604445214 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240d0c40 con 0x7f302415f750 2024-04-26T21:15:02.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.011+0000 7f3022ffd640 1 -- 172.21.15.40:0/2604445214 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169189861 0 0) 0x7f30240d03a0 con 0x7f30240d3ee0 2024-04-26T21:15:02.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f3028b3a640 1 --2- 172.21.15.40:0/2604445214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240cfd70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f301c009e50 tx=0x7f301c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bba43f3b4c06cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f3022ffd640 1 -- 172.21.15.40:0/2604445214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c036070 con 0x7f302415f750 2024-04-26T21:15:02.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f3022ffd640 1 -- 172.21.15.40:0/2604445214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c031600 con 0x7f302415f750 2024-04-26T21:15:02.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f3022ffd640 1 -- 172.21.15.40:0/2604445214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0319d0 con 0x7f302415f750 2024-04-26T21:15:02.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 -- 172.21.15.40:0/2604445214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 msgr2=0x7f30240cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 --2- 172.21.15.40:0/2604445214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240cfd70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f301c009e50 tx=0x7f301c02a760 comp rx=0 tx=0).stop 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 -- 172.21.15.40:0/2604445214 shutdown_connections 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 --2- 172.21.15.40:0/2604445214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240cfd70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 -- 172.21.15.40:0/2604445214 >> 172.21.15.40:0/2604445214 conn(0x7f302408ec10 msgr2=0x7f302409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 -- 172.21.15.40:0/2604445214 shutdown_connections 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.012+0000 7f302a59d640 1 -- 172.21.15.40:0/2604445214 wait complete. 2024-04-26T21:15:02.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.013+0000 7f302a59d640 1 Processor -- start 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 -- start start 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 0x7f302415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 0x7f30240b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b4950 con 0x7f302415f750 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b47d0 con 0x7f302415f370 2024-04-26T21:15:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f302a59d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f30240b4ad0 con 0x7f30240d3ee0 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f3028b3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f3028b3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39224/0 (socket says 172.21.15.40:39224) 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.014+0000 7f3028b3a640 1 -- 172.21.15.40:0/1042190463 learned_addr learned my addr 172.21.15.40:0/1042190463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f30237fe640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 0x7f30240b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3023fff640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 0x7f302415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 -- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 msgr2=0x7f302415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 0x7f302415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 -- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 msgr2=0x7f30240b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 0x7f30240b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 -- 172.21.15.40:0/1042190463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3024161b50 con 0x7f302415f750 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f30237fe640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 0x7f30240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3023fff640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 0x7f302415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3028b3a640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f301c034040 tx=0x7f301c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c041020 con 0x7f302415f750 2024-04-26T21:15:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c007780 con 0x7f302415f750 2024-04-26T21:15:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c036040 con 0x7f302415f750 2024-04-26T21:15:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3024161df0 con 0x7f302415f750 2024-04-26T21:15:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.015+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241623b0 con 0x7f302415f750 2024-04-26T21:15:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.016+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302406a290 con 0x7f302415f750 2024-04-26T21:15:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.017+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f301c044020 con 0x7f302415f750 2024-04-26T21:15:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.017+0000 7f3020ff9640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.017+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f301c0ac830 con 0x7f302415f750 2024-04-26T21:15:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.018+0000 7f3023fff640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.018+0000 7f3023fff640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 0x7f301006edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3014003eb0 tx=0x7f3014009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.022+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f301c04b030 con 0x7f302415f750 2024-04-26T21:15:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.247+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30240b6370 con 0x7f302415f750 2024-04-26T21:15:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.247+0000 7f3020ff9640 1 -- 172.21.15.40:0/1042190463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5184 (secure 0 0 0) 0x7f301c07adb0 con 0x7f302415f750 2024-04-26T21:15:02.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:02.249 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:14:43.607278+0000","modified":"2024-04-26T21:14:45.595782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6024":{"gid":6024,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6834/555844344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":555844344},{"type":"v1","addr":"172.21.15.40:6834","nonce":555844344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6024,"qdb_cluster":[6024]},"id":4}]} 2024-04-26T21:15:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 msgr2=0x7f301006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 0x7f301006edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3014003eb0 tx=0x7f3014009040 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 msgr2=0x7f30240b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f301c034040 tx=0x7f301c004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 shutdown_connections 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f301006c8f0 0x7f301006edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f302415f750 0x7f30240b2e30 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302415f370 0x7f30240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 --2- 172.21.15.40:0/1042190463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30240d3ee0 0x7f302415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.249+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 >> 172.21.15.40:0/1042190463 conn(0x7f302408ec10 msgr2=0x7f302409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.250+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 shutdown_connections 2024-04-26T21:15:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.250+0000 7f302a59d640 1 -- 172.21.15.40:0/1042190463 wait complete. 2024-04-26T21:15:02.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.360+0000 7fdad609b640 1 Processor -- start 2024-04-26T21:15:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.360+0000 7fdad609b640 1 -- start start 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdad609b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdad015f750 0x7fdad00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdad609b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad00d03a0 con 0x7fdad015f370 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdad609b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad00d0680 con 0x7fdad00d3ee0 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdad609b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdad00d0960 con 0x7fdad015f750 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdacf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdad015f370 0x7fdad00d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51396/0 (socket says 172.21.15.40:51396) 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.361+0000 7fdacf7fe640 1 -- 172.21.15.40:0/2587401018 learned_addr learned my addr 172.21.15.40:0/2587401018 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdacffff640 1 --2- 172.21.15.40:0/2587401018 >> v2:172.21.15.184:3300/0 conn(0x7fdad015f750 0x7fdad00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647241034 0 0) 0x7fdad00d03a0 con 0x7fdad015f370 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdab8003520 con 0x7fdad015f370 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849241538 0 0) 0x7fdab8003520 con 0x7fdad015f370 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 >> v2:172.21.15.184:3300/0 conn(0x7fdad015f750 msgr2=0x7fdad00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 --2- 172.21.15.40:0/2587401018 >> v2:172.21.15.184:3300/0 conn(0x7fdad015f750 0x7fdad00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 >> v1:172.21.15.184:6789/0 conn(0x7fdad00d3ee0 legacy=0x7fdad00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad00d0c40 con 0x7fdad015f370 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fdac0001670 con 0x7fdad015f370 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 >> v1:172.21.15.40:6789/0 conn(0x7fdad015f370 legacy=0x7fdad00d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 0x7fdab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00d0960 con 0x7fdab80051c0 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdab800c060 con 0x7fdab80039b0 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdab800c260 con 0x7fdab8004800 2024-04-26T21:15:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdacffff640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 0x7fdab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdacf7fe640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.362+0000 7fdaceffd640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 -- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab8004800 msgr2=0x7fdab8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 -- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab80039b0 msgr2=0x7fdab8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 -- 172.21.15.40:0/2587401018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab8003520 con 0x7fdab80051c0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 -- 172.21.15.40:0/2587401018 reap_dead start 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdacffff640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 0x7fdab800b9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009fc0 tx=0x7fdab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aae3d8a00164d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8035020 con 0x7fdab80051c0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8003b90 con 0x7fdab80051c0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.363+0000 7fdace7fc640 1 -- 172.21.15.40:0/2587401018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8009c10 con 0x7fdab80051c0 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 -- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 msgr2=0x7fdab800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 0x7fdab800b9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009fc0 tx=0x7fdab8005830 comp rx=0 tx=0).stop 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 -- 172.21.15.40:0/2587401018 shutdown_connections 2024-04-26T21:15:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 --2- 172.21.15.40:0/2587401018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdab80051c0 0x7fdab800b9c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 -- 172.21.15.40:0/2587401018 >> 172.21.15.40:0/2587401018 conn(0x7fdad008ec10 msgr2=0x7fdad009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 -- 172.21.15.40:0/2587401018 shutdown_connections 2024-04-26T21:15:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.364+0000 7fdad609b640 1 -- 172.21.15.40:0/2587401018 wait complete. 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.365+0000 7fdad609b640 1 Processor -- start 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.365+0000 7fdad609b640 1 -- start start 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80051c0 0x7fdad00cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad00ce930 0x7fdad014e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00cfef0 con 0x7fdad014f1c0 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00cfd70 con 0x7fdad00ce930 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdad609b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdad00d0070 con 0x7fdab80051c0 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39242/0 (socket says 172.21.15.40:39242) 2024-04-26T21:15:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 -- 172.21.15.40:0/687227057 learned_addr learned my addr 172.21.15.40:0/687227057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdacffff640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad00ce930 0x7fdad014e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdacf7fe640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80051c0 0x7fdad00cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 -- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80051c0 msgr2=0x7fdad00cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80051c0 0x7fdad00cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 -- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad00ce930 msgr2=0x7fdad014e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad00ce930 0x7fdad014e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.366+0000 7fdaceffd640 1 -- 172.21.15.40:0/687227057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad00b8000 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdaceffd640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdac400f5c0 tx=0x7fdac400fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40052e0 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4018070 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad00cde50 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003830 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.367+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad00ce410 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.368+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad006e7f0 con 0x7fdad014f1c0 2024-04-26T21:15:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.368+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdac40039d0 con 0x7fdad014f1c0 2024-04-26T21:15:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.369+0000 7fdab7fff640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 0x7fdabc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.369+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fdac408b910 con 0x7fdad014f1c0 2024-04-26T21:15:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.369+0000 7fdacf7fe640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 0x7fdabc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.370+0000 7fdacf7fe640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 0x7fdabc06ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdac0000ff0 tx=0x7fdac0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.373+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdad006e7f0 con 0x7fdad014f1c0 2024-04-26T21:15:02.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.558+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdad006a290 con 0x7fdad014f1c0 2024-04-26T21:15:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.559+0000 7fdab7fff640 1 -- 172.21.15.40:0/687227057 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7fdac4059d40 con 0x7fdad014f1c0 2024-04-26T21:15:02.561 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:02.561 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:14:45.233854+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:14:41.855331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T21:14:42.545028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 msgr2=0x7fdabc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 0x7fdabc06ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdac0000ff0 tx=0x7fdac0009040 comp rx=0 tx=0).stop 2024-04-26T21:15:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 msgr2=0x7fdad0151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdac400f5c0 tx=0x7fdac400fa80 comp rx=0 tx=0).stop 2024-04-26T21:15:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 shutdown_connections 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdabc06c920 0x7fdabc06ede0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad00ce930 0x7fdad014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdad014f1c0 0x7fdad0151cd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 --2- 172.21.15.40:0/687227057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdab80051c0 0x7fdad00cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 >> 172.21.15.40:0/687227057 conn(0x7fdad008ec10 msgr2=0x7fdad00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 shutdown_connections 2024-04-26T21:15:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.561+0000 7fdad609b640 1 -- 172.21.15.40:0/687227057 wait complete. 2024-04-26T21:15:02.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:15:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 Processor -- start 2024-04-26T21:15:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 -- start start 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f8c15f4a0 0x7f3f8c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f8c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8c0aed90 con 0x7f3f8c0afe30 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c15f880 con 0x7f3f8c15f4a0 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.675+0000 7f3f91a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c0af9a0 con 0x7f3f8c0b6320 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f8c0afe30 0x7f3f8c162f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51410/0 (socket says 172.21.15.40:51410) 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8a7fc640 1 -- 172.21.15.40:0/325085629 learned_addr learned my addr 172.21.15.40:0/325085629 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553636234 0 0) 0x7f3f8c0aed90 con 0x7f3f8c0afe30 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8affd640 1 --2- 172.21.15.40:0/325085629 >> v2:172.21.15.184:3300/0 conn(0x7f3f8c15f4a0 0x7f3f8c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f74003520 con 0x7f3f8c0afe30 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f8c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 156469834 0 0) 0x7f3f74003520 con 0x7f3f8c0afe30 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 msgr2=0x7f3f8c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f8c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 >> v2:172.21.15.184:3300/0 conn(0x7f3f8c15f4a0 msgr2=0x7f3f8c0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 --2- 172.21.15.40:0/325085629 >> v2:172.21.15.184:3300/0 conn(0x7f3f8c15f4a0 0x7f3f8c0b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c0af070 con 0x7f3f8c0afe30 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3f80001670 con 0x7f3f8c0afe30 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 >> v1:172.21.15.40:6789/0 conn(0x7f3f8c0afe30 legacy=0x7f3f8c162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f74003f30 0x7f3f74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f74004d40 0x7f3f740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c15f880 con 0x7f3f74004d40 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0af9a0 con 0x7f3f74003f30 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7400ba10 con 0x7f3f8c0b6320 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8affd640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f74004d40 0x7f3f740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.676+0000 7f3f8a7fc640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f74003f30 0x7f3f74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 -- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 msgr2=0x7f3f740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c0b6320 0x7f3f740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 -- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f74003f30 msgr2=0x7f3f74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f74003f30 0x7f3f74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 -- 172.21.15.40:0/325085629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74003520 con 0x7f3f74004d40 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f74004d40 0x7f3f740051c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009f90 tx=0x7f3f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f708a6ad55cec62f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84009c10 con 0x7f3f74004d40 2024-04-26T21:15:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f84036070 con 0x7f3f74004d40 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.677+0000 7f3f89ffb640 1 -- 172.21.15.40:0/325085629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84031810 con 0x7f3f74004d40 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 -- 172.21.15.40:0/325085629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f74004d40 msgr2=0x7f3f740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/325085629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f74004d40 0x7f3f740051c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009f90 tx=0x7f3f74005830 comp rx=0 tx=0).stop 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f8b7fe640 1 -- 172.21.15.40:0/325085629 reap_dead start 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 -- 172.21.15.40:0/325085629 shutdown_connections 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 -- 172.21.15.40:0/325085629 >> 172.21.15.40:0/325085629 conn(0x7f3f8c0b6800 msgr2=0x7f3f8c0738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 -- 172.21.15.40:0/325085629 shutdown_connections 2024-04-26T21:15:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.678+0000 7f3f91a2a640 1 -- 172.21.15.40:0/325085629 wait complete. 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.679+0000 7f3f91a2a640 1 Processor -- start 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.679+0000 7f3f91a2a640 1 -- start start 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c1593d0 0x7f3f8c161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c0b1dc0 0x7f3f8c0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0aea80 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0af630 con 0x7f3f8c0b1dc0 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f91a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c0b4600 con 0x7f3f8c1593d0 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f8a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f8a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39258/0 (socket says 172.21.15.40:39258) 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.680+0000 7f3f8a7fc640 1 -- 172.21.15.40:0/2378209740 learned_addr learned my addr 172.21.15.40:0/2378209740 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8affd640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c1593d0 0x7f3f8c161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8b7fe640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c0b1dc0 0x7f3f8c0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 -- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c1593d0 msgr2=0x7f3f8c161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c1593d0 0x7f3f8c161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 -- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c0b1dc0 msgr2=0x7f3f8c0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c0b1dc0 0x7f3f8c0b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 -- 172.21.15.40:0/2378209740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c09dca0 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f8a7fc640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3f800072c0 tx=0x7f3f80007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000f9c0 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f80018070 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c09b250 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000fdc0 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.681+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c09b7d0 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.682+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c06a290 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.683+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f8001d460 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.683+0000 7f3f73fff640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 0x7f3f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.683+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f3f8008bbf0 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.683+0000 7f3f8affd640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 0x7f3f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.684+0000 7f3f8affd640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 0x7f3f7806ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.688+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f8005a020 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.909+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3f8c162650 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.910 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:15:02.909+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:15:02.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.910+0000 7f3f73fff640 1 -- 172.21.15.40:0/2378209740 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f3f80059920 con 0x7f3f8c09bfd0 2024-04-26T21:15:02.911 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.913+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 msgr2=0x7f3f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.913+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 0x7f3f7806ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c008040 comp rx=0 tx=0).stop 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.913+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 msgr2=0x7f3f8c09d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.913+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3f800072c0 tx=0x7f3f80007780 comp rx=0 tx=0).stop 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 shutdown_connections 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f7806c920 0x7f3f7806ede0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f8c0b1dc0 0x7f3f8c0b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f8c09bfd0 0x7f3f8c09d760 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 --2- 172.21.15.40:0/2378209740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f8c1593d0 0x7f3f8c161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 >> 172.21.15.40:0/2378209740 conn(0x7f3f8c0b6800 msgr2=0x7f3f8c0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 shutdown_connections 2024-04-26T21:15:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:02.914+0000 7f3f91a2a640 1 -- 172.21.15.40:0/2378209740 wait complete. 2024-04-26T21:15:02.916 INFO:tasks.ceph.mds.c.smithi040.stderr: -2180> 2024-04-26T21:14:49.188+0000 7f11e461d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:15:02.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:15:02.944 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:15:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.029+0000 7f0cf895f640 1 Processor -- start 2024-04-26T21:15:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.029+0000 7f0cf895f640 1 -- start start 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf895f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cf40b3c20 0x7f0cf406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf895f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf409c700 con 0x7f0cf40b33e0 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf895f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf40d1c50 con 0x7f0cf40d0010 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf895f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf40b4480 con 0x7f0cf40b3c20 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cf40b3c20 0x7f0cf406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0cf40b33e0 0x7f0cf4163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51422/0 (socket says 172.21.15.40:51422) 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cebfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cf40b3c20 0x7f0cf406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39278/0 (socket says 172.21.15.40:39278) 2024-04-26T21:15:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf2d9d640 1 -- 172.21.15.40:0/3061060630 learned_addr learned my addr 172.21.15.40:0/3061060630 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cebfff640 1 -- 172.21.15.40:0/3061060630 >> v1:172.21.15.184:6789/0 conn(0x7f0cf40d0010 legacy=0x7f0cf40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cf1d9b640 1 -- 172.21.15.40:0/3061060630 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053567372 0 0) 0x7f0cf409c700 con 0x7f0cf40b33e0 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cebfff640 1 -- 172.21.15.40:0/3061060630 >> v1:172.21.15.40:6789/0 conn(0x7f0cf40b33e0 legacy=0x7f0cf4163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.030+0000 7f0cebfff640 1 -- 172.21.15.40:0/3061060630 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf40d1f30 con 0x7f0cf40b3c20 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.031+0000 7f0cebfff640 1 --2- 172.21.15.40:0/3061060630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40b3c20 0x7f0cf406db10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0097a0 tx=0x7f0cdc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb5955fce0846e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.031+0000 7f0cf1d9b640 1 -- 172.21.15.40:0/3061060630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc037070 con 0x7f0cf40b3c20 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.031+0000 7f0cf1d9b640 1 -- 172.21.15.40:0/3061060630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cdc032440 con 0x7f0cf40b3c20 2024-04-26T21:15:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.031+0000 7f0cf1d9b640 1 -- 172.21.15.40:0/3061060630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc032810 con 0x7f0cf40b3c20 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 -- 172.21.15.40:0/3061060630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40b3c20 msgr2=0x7f0cf406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 --2- 172.21.15.40:0/3061060630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40b3c20 0x7f0cf406db10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0097a0 tx=0x7f0cdc02a230 comp rx=0 tx=0).stop 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 -- 172.21.15.40:0/3061060630 shutdown_connections 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 --2- 172.21.15.40:0/3061060630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40b3c20 0x7f0cf406db10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 -- 172.21.15.40:0/3061060630 >> 172.21.15.40:0/3061060630 conn(0x7f0cf409ba40 msgr2=0x7f0cf409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 -- 172.21.15.40:0/3061060630 shutdown_connections 2024-04-26T21:15:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.032+0000 7f0cf895f640 1 -- 172.21.15.40:0/3061060630 wait complete. 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.033+0000 7f0cf895f640 1 Processor -- start 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.033+0000 7f0cf895f640 1 -- start start 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf40b33e0 0x7f0cf415c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 0x7f0cf415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf406e050 con 0x7f0cf40d0010 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40b3a20 con 0x7f0cf40b33e0 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf895f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf4163990 con 0x7f0cf40b3c20 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39286/0 (socket says 172.21.15.40:39286) 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 -- 172.21.15.40:0/120365312 learned_addr learned my addr 172.21.15.40:0/120365312 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf259c640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf40b33e0 0x7f0cf415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cebfff640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 0x7f0cf415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 -- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 msgr2=0x7f0cf415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 0x7f0cf415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 -- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf40b33e0 msgr2=0x7f0cf415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf40b33e0 0x7f0cf415c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 -- 172.21.15.40:0/120365312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf4158310 con 0x7f0cf40d0010 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cebfff640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 0x7f0cf415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0cf2d9d640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c7a0 tx=0x7f0cec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec018070 con 0x7f0cf40d0010 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cec00a640 con 0x7f0cf40d0010 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.034+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00aab0 con 0x7f0cf40d0010 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.035+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41585f0 con 0x7f0cf40d0010 2024-04-26T21:15:03.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.035+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf4158bb0 con 0x7f0cf40d0010 2024-04-26T21:15:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.035+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf4075730 con 0x7f0cf40d0010 2024-04-26T21:15:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.036+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cec01b070 con 0x7f0cf40d0010 2024-04-26T21:15:03.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.036+0000 7f0ceaffd640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 0x7f0cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.036+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7f0cec08b110 con 0x7f0cf40d0010 2024-04-26T21:15:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.037+0000 7f0cf259c640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 0x7f0cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:03.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.037+0000 7f0cf259c640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 0x7f0cd406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000f60 tx=0x7f0ce4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:03.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.041+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cf4075730 con 0x7f0cf40d0010 2024-04-26T21:15:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.264+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0cf406a290 con 0x7f0cf40d0010 2024-04-26T21:15:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.915+0000 7f0ceaffd640 1 -- 172.21.15.40:0/120365312 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f0cec059570 con 0x7f0cf40d0010 2024-04-26T21:15:03.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 msgr2=0x7f0cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 0x7f0cd406edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000f60 tx=0x7f0ce4009040 comp rx=0 tx=0).stop 2024-04-26T21:15:03.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 msgr2=0x7f0cf41570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:03.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c7a0 tx=0x7f0cec00cc60 comp rx=0 tx=0).stop 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 shutdown_connections 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0cd406c8f0 0x7f0cd406edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40d0010 0x7f0cf41570d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf40b3c20 0x7f0cf415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 --2- 172.21.15.40:0/120365312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf40b33e0 0x7f0cf415c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 >> 172.21.15.40:0/120365312 conn(0x7f0cf409ba40 msgr2=0x7f0cf4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 shutdown_connections 2024-04-26T21:15:03.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:03.918+0000 7f0cf895f640 1 -- 172.21.15.40:0/120365312 wait complete. 2024-04-26T21:15:03.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:15:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.085+0000 7fcfeb293640 1 Processor -- start 2024-04-26T21:15:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.086+0000 7fcfeb293640 1 -- start start 2024-04-26T21:15:04.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfeb293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcfe40d2140 0x7fcfe40d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfeb293640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 0x7fcfe40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfeb293640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe40d33a0 con 0x7fcfe4162a10 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfeb293640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe40b4970 con 0x7fcfe40d2140 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfeb293640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe40d3680 con 0x7fcfe40b6020 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcfe40d2140 0x7fcfe40d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcfe40d2140 0x7fcfe40d2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39300/0 (socket says 172.21.15.40:39300) 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 -- 172.21.15.40:0/4228736751 learned_addr learned my addr 172.21.15.40:0/4228736751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe9830640 1 --2- 172.21.15.40:0/4228736751 >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 0x7fcfe40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 -- 172.21.15.40:0/4228736751 >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 msgr2=0x7fcfe40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 --2- 172.21.15.40:0/4228736751 >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 0x7fcfe40cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 -- 172.21.15.40:0/4228736751 >> v1:172.21.15.184:6789/0 conn(0x7fcfe4162a10 legacy=0x7fcfe40d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:04.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe882e640 1 -- 172.21.15.40:0/4228736751 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe40d3960 con 0x7fcfe40d2140 2024-04-26T21:15:04.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.087+0000 7fcfe9830640 1 --2- 172.21.15.40:0/4228736751 >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 0x7fcfe40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:04.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfe882e640 1 --2- 172.21.15.40:0/4228736751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe40d2140 0x7fcfe40d2dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcfe408e660 tx=0x7fcfd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6eaafce57cff3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:04.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfdbfff640 1 -- 172.21.15.40:0/4228736751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4037070 con 0x7fcfe40d2140 2024-04-26T21:15:04.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfdbfff640 1 -- 172.21.15.40:0/4228736751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd4032440 con 0x7fcfe40d2140 2024-04-26T21:15:04.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfdbfff640 1 -- 172.21.15.40:0/4228736751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4032810 con 0x7fcfe40d2140 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 -- 172.21.15.40:0/4228736751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe40d2140 msgr2=0x7fcfe40d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 --2- 172.21.15.40:0/4228736751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe40d2140 0x7fcfe40d2dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcfe408e660 tx=0x7fcfd402a230 comp rx=0 tx=0).stop 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 -- 172.21.15.40:0/4228736751 shutdown_connections 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 --2- 172.21.15.40:0/4228736751 >> v2:172.21.15.184:3300/0 conn(0x7fcfe40b6020 0x7fcfe40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 --2- 172.21.15.40:0/4228736751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe40d2140 0x7fcfe40d2dc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.088+0000 7fcfeb293640 1 -- 172.21.15.40:0/4228736751 >> 172.21.15.40:0/4228736751 conn(0x7fcfe409e530 msgr2=0x7fcfe40d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:04.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.089+0000 7fcfeb293640 1 -- 172.21.15.40:0/4228736751 shutdown_connections 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.089+0000 7fcfeb293640 1 -- 172.21.15.40:0/4228736751 wait complete. 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.090+0000 7fcfeb293640 1 Processor -- start 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.090+0000 7fcfeb293640 1 -- start start 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe40b6020 0x7fcfe415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfe40d2140 0x7fcfe414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4162df0 con 0x7fcfe4162a10 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe40d3f20 con 0x7fcfe40b6020 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe40b6600 con 0x7fcfe40d2140 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39302/0 (socket says 172.21.15.40:39302) 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 -- 172.21.15.40:0/274121667 learned_addr learned my addr 172.21.15.40:0/274121667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:04.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe882e640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfe40d2140 0x7fcfe414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe902f640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe40b6020 0x7fcfe415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 -- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfe40d2140 msgr2=0x7fcfe414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfe40d2140 0x7fcfe414f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 -- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe40b6020 msgr2=0x7fcfe415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe40b6020 0x7fcfe415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 -- 172.21.15.40:0/274121667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe415a750 con 0x7fcfe4162a10 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfe9830640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000cef0 tx=0x7fcfe000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe001a070 con 0x7fcfe4162a10 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe000d860 con 0x7fcfe4162a10 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000dc50 con 0x7fcfe4162a10 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.091+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe41621e0 con 0x7fcfe4162a10 2024-04-26T21:15:04.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.092+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe40b2bb0 con 0x7fcfe4162a10 2024-04-26T21:15:04.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.093+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe406a290 con 0x7fcfe4162a10 2024-04-26T21:15:04.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.093+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe001d070 con 0x7fcfe4162a10 2024-04-26T21:15:04.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.093+0000 7fcfd9ffb640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 0x7fcfc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:04.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.094+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7fcfe008c300 con 0x7fcfe4162a10 2024-04-26T21:15:04.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.094+0000 7fcfe902f640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 0x7fcfc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:04.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.094+0000 7fcfe902f640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 0x7fcfc806ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc006fd0 tx=0x7fcfdc009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:04.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.098+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcfe005a1d0 con 0x7fcfe4162a10 2024-04-26T21:15:04.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.283+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcfe40d27a0 con 0x7fcfe4162a10 2024-04-26T21:15:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.917+0000 7fcfd9ffb640 1 -- 172.21.15.40:0/274121667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7fcfe005e4d0 con 0x7fcfe4162a10 2024-04-26T21:15:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:15:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.921+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 msgr2=0x7fcfc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.921+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 0x7fcfc806ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc006fd0 tx=0x7fcfdc009060 comp rx=0 tx=0).stop 2024-04-26T21:15:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.921+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 msgr2=0x7fcfe40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.921+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000cef0 tx=0x7fcfe000a710 comp rx=0 tx=0).stop 2024-04-26T21:15:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 shutdown_connections 2024-04-26T21:15:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcfc806c920 0x7fcfc806ede0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfe4162a10 0x7fcfe40cd190 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfe40d2140 0x7fcfe414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 --2- 172.21.15.40:0/274121667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe40b6020 0x7fcfe415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.923+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 >> 172.21.15.40:0/274121667 conn(0x7fcfe409e530 msgr2=0x7fcfe406df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:04.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.927+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 shutdown_connections 2024-04-26T21:15:04.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:04.927+0000 7fcfeb293640 1 -- 172.21.15.40:0/274121667 wait complete. 2024-04-26T21:15:04.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:15:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.053+0000 7fd8f23a4640 1 Processor -- start 2024-04-26T21:15:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.053+0000 7fd8f23a4640 1 -- start start 2024-04-26T21:15:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f23a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0d2140 0x7fd8ec0d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f23a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f23a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec0d33a0 con 0x7fd8ec162a10 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f23a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec0b4970 con 0x7fd8ec0b6020 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f23a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d3680 con 0x7fd8ec0d2140 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39308/0 (socket says 172.21.15.40:39308) 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 -- 172.21.15.40:0/1798363539 learned_addr learned my addr 172.21.15.40:0/1798363539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8eb7fe640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0d2140 0x7fd8ec0d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 -- 172.21.15.40:0/1798363539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0d2140 msgr2=0x7fd8ec0d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0d2140 0x7fd8ec0d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 -- 172.21.15.40:0/1798363539 >> v1:172.21.15.184:6789/0 conn(0x7fd8ec162a10 legacy=0x7fd8ec0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.054+0000 7fd8f0941640 1 -- 172.21.15.40:0/1798363539 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec0d3960 con 0x7fd8ec0b6020 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.055+0000 7fd8f0941640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009e50 tx=0x7fd8e402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9027ef0b1ae01997 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.055+0000 7fd8eaffd640 1 -- 172.21.15.40:0/1798363539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4036070 con 0x7fd8ec0b6020 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.055+0000 7fd8eaffd640 1 -- 172.21.15.40:0/1798363539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4031600 con 0x7fd8ec0b6020 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.055+0000 7fd8eaffd640 1 -- 172.21.15.40:0/1798363539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40319d0 con 0x7fd8ec0b6020 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 -- 172.21.15.40:0/1798363539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0b6020 msgr2=0x7fd8ec0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009e50 tx=0x7fd8e402a710 comp rx=0 tx=0).stop 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 -- 172.21.15.40:0/1798363539 shutdown_connections 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0b6020 0x7fd8ec0cffb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/1798363539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0d2140 0x7fd8ec0d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 -- 172.21.15.40:0/1798363539 >> 172.21.15.40:0/1798363539 conn(0x7fd8ec09e530 msgr2=0x7fd8ec0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 -- 172.21.15.40:0/1798363539 shutdown_connections 2024-04-26T21:15:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.056+0000 7fd8f23a4640 1 -- 172.21.15.40:0/1798363539 wait complete. 2024-04-26T21:15:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.057+0000 7fd8f23a4640 1 Processor -- start 2024-04-26T21:15:05.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.057+0000 7fd8f23a4640 1 -- start start 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 0x7fd8ec15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 0x7fd8ec160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec162df0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d3f20 con 0x7fd8ec162a10 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f23a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec0b6600 con 0x7fd8ec0b6020 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39324/0 (socket says 172.21.15.40:39324) 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 -- 172.21.15.40:0/4243031204 learned_addr learned my addr 172.21.15.40:0/4243031204 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f0941640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 0x7fd8ec160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8ebfff640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 0x7fd8ec15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 -- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 msgr2=0x7fd8ec15a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 0x7fd8ec15a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 -- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 msgr2=0x7fd8ec160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 0x7fd8ec160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8eb7fe640 1 -- 172.21.15.40:0/4243031204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec15a7d0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8ebfff640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 0x7fd8ec15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.058+0000 7fd8f0941640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 0x7fd8ec160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8eb7fe640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00d5f0 tx=0x7fd8dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0125f0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc019070 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0129d0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec0b2480 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.059+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec0b2a40 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.060+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec06a290 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.060+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8dc012b70 con 0x7fd8ec0d2140 2024-04-26T21:15:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.061+0000 7fd8e8ff9640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 0x7fd8cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.061+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5357+0+0 (secure 0 0 0) 0x7fd8dc08bdf0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.061+0000 7fd8ebfff640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 0x7fd8cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.061+0000 7fd8ebfff640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 0x7fd8cc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40015e0 tx=0x7fd8d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.065+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8dc05a380 con 0x7fd8ec0d2140 2024-04-26T21:15:05.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.251+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd8ec0d27a0 con 0x7fd8ec0d2140 2024-04-26T21:15:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.924+0000 7fd8e8ff9640 1 -- 172.21.15.40:0/4243031204 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7fd8dc059c80 con 0x7fd8ec0d2140 2024-04-26T21:15:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.926+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 msgr2=0x7fd8cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.926+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 0x7fd8cc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40015e0 tx=0x7fd8d4009040 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.926+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 msgr2=0x7fd8ec0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.926+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00d5f0 tx=0x7fd8dc00dab0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 shutdown_connections 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8cc06c8f0 0x7fd8cc06edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ec162a10 0x7fd8ec160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8ec0d2140 0x7fd8ec0b3520 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 --2- 172.21.15.40:0/4243031204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ec0b6020 0x7fd8ec15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 >> 172.21.15.40:0/4243031204 conn(0x7fd8ec09e530 msgr2=0x7fd8ec162f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 shutdown_connections 2024-04-26T21:15:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:05.927+0000 7fd8f23a4640 1 -- 172.21.15.40:0/4243031204 wait complete. 2024-04-26T21:15:05.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:15:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 Processor -- start 2024-04-26T21:15:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 -- start start 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd0406f340 0x7efd040b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efd0406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0406fc60 con 0x7efd040b3e50 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efd040b46a0 con 0x7efd0406f340 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.052+0000 7efd0ae1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd040b5db0 con 0x7efd040d0300 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd093ba640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd040b3e50 0x7efd040720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51430/0 (socket says 172.21.15.40:51430) 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd093ba640 1 -- 172.21.15.40:0/2365352868 learned_addr learned my addr 172.21.15.40:0/2365352868 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd03fff640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efd0406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd08bb9640 1 --2- 172.21.15.40:0/2365352868 >> v2:172.21.15.184:3300/0 conn(0x7efd0406f340 0x7efd040b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472124040 0 0) 0x7efd0406fc60 con 0x7efd040b3e50 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcec003520 con 0x7efd040b3e50 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 754673690 0 0) 0x7efcec003520 con 0x7efd040b3e50 2024-04-26T21:15:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 msgr2=0x7efd0406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efd0406f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 >> v2:172.21.15.184:3300/0 conn(0x7efd0406f340 msgr2=0x7efd040b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 --2- 172.21.15.40:0/2365352868 >> v2:172.21.15.184:3300/0 conn(0x7efd0406f340 0x7efd040b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd040b6090 con 0x7efd040b3e50 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7efcf4001670 con 0x7efd040b3e50 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 >> v1:172.21.15.40:6789/0 conn(0x7efd040b3e50 legacy=0x7efd040720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcec0039b0 0x7efcec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efcec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcec004d40 0x7efcec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd040b46a0 con 0x7efcec004d40 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd040b5db0 con 0x7efcec0039b0 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efcec00ba40 con 0x7efd040d0300 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.053+0000 7efd08bb9640 1 --2- 172.21.15.40:0/2365352868 >> v2:172.21.15.184:3300/0 conn(0x7efd0406f340 0x7efd040b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcec004d40 0x7efcec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd03fff640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efcec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd08bb9640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcec0039b0 0x7efcec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 -- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 msgr2=0x7efcec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040d0300 0x7efcec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 -- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcec0039b0 msgr2=0x7efcec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcec0039b0 0x7efcec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 -- 172.21.15.40:0/2365352868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec003520 con 0x7efcec004d40 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.054+0000 7efd093ba640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcec004d40 0x7efcec0051c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c240 tx=0x7efcec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb02d13c7ffb8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007d60 con 0x7efcec004d40 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf4005ad0 con 0x7efcec004d40 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd037fe640 1 -- 172.21.15.40:0/2365352868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007070 con 0x7efcec004d40 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 -- 172.21.15.40:0/2365352868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcec004d40 msgr2=0x7efcec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/2365352868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcec004d40 0x7efcec0051c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c240 tx=0x7efcec005830 comp rx=0 tx=0).stop 2024-04-26T21:15:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd093ba640 1 -- 172.21.15.40:0/2365352868 reap_dead start 2024-04-26T21:15:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 -- 172.21.15.40:0/2365352868 shutdown_connections 2024-04-26T21:15:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 -- 172.21.15.40:0/2365352868 >> 172.21.15.40:0/2365352868 conn(0x7efd040cd8a0 msgr2=0x7efd040cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 -- 172.21.15.40:0/2365352868 shutdown_connections 2024-04-26T21:15:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.055+0000 7efd0ae1d640 1 -- 172.21.15.40:0/2365352868 wait complete. 2024-04-26T21:15:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.056+0000 7efd0ae1d640 1 Processor -- start 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.056+0000 7efd0ae1d640 1 -- start start 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 0x7efd040c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 0x7efd041350d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd040b42e0 con 0x7efd0415ac90 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd040b6650 con 0x7efd040c8af0 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd0ae1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd040b4460 con 0x7efd040aa9e0 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd08bb9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd08bb9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39356/0 (socket says 172.21.15.40:39356) 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.057+0000 7efd08bb9640 1 -- 172.21.15.40:0/1424361341 learned_addr learned my addr 172.21.15.40:0/1424361341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd03fff640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 0x7efd040c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd093ba640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 0x7efd041350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 -- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 msgr2=0x7efd040c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 0x7efd040c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 -- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 msgr2=0x7efd041350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 0x7efd041350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 -- 172.21.15.40:0/1424361341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd040c9320 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd093ba640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 0x7efd041350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd03fff640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 0x7efd040c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd08bb9640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00f430 tx=0x7efcfc00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc003b90 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc003d30 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd040c9600 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc015860 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.058+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd04133120 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.059+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0406a290 con 0x7efd0415ac90 2024-04-26T21:15:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.060+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcfc015a00 con 0x7efd0415ac90 2024-04-26T21:15:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.060+0000 7efd017fa640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 0x7efcf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.060+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7efcfc08c9b0 con 0x7efd0415ac90 2024-04-26T21:15:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.061+0000 7efd03fff640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 0x7efcf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.061+0000 7efd03fff640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 0x7efcf006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd0415d220 tx=0x7efcf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.064+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efcfc019070 con 0x7efd0415ac90 2024-04-26T21:15:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.251+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efd040aadc0 con 0x7efd0415ac90 2024-04-26T21:15:06.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.254+0000 7efd017fa640 1 -- 172.21.15.40:0/1424361341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7efcfc010040 con 0x7efd0415ac90 2024-04-26T21:15:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 msgr2=0x7efcf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 0x7efcf006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd0415d220 tx=0x7efcf8008040 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 msgr2=0x7efd040aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00f430 tx=0x7efcfc00f8f0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 shutdown_connections 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efcf006c8f0 0x7efcf006edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd040c8af0 0x7efd041350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd040aa9e0 0x7efd040c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 --2- 172.21.15.40:0/1424361341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd0415ac90 0x7efd040aa4a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 >> 172.21.15.40:0/1424361341 conn(0x7efd040cd8a0 msgr2=0x7efd040ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 shutdown_connections 2024-04-26T21:15:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.256+0000 7efd0ae1d640 1 -- 172.21.15.40:0/1424361341 wait complete. 2024-04-26T21:15:06.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-04-26T21:15:06.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.368+0000 7f85f6d9d640 1 Processor -- start 2024-04-26T21:15:06.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.368+0000 7f85f6d9d640 1 -- start start 2024-04-26T21:15:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f6d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85f0070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f6d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85f00b5530 0x7f85f00d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f6d9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85f015fce0 con 0x7f85f009ba40 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f6d9d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f85f00b59c0 con 0x7f85f00b5530 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f6d9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85f0160610 con 0x7f85f015f6a0 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f4b39640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85f009ba40 0x7f85f009be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51436/0 (socket says 172.21.15.40:51436) 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f4b39640 1 -- 172.21.15.40:0/212481786 learned_addr learned my addr 172.21.15.40:0/212481786 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103647557 0 0) 0x7f85f015fce0 con 0x7f85f009ba40 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85dc003520 con 0x7f85f009ba40 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85effff640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85f0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.369+0000 7f85f533a640 1 --2- 172.21.15.40:0/212481786 >> v2:172.21.15.184:3300/0 conn(0x7f85f00b5530 0x7f85f00d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116250572 0 0) 0x7f85dc003520 con 0x7f85f009ba40 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 msgr2=0x7f85f0070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85f0070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 >> v2:172.21.15.184:3300/0 conn(0x7f85f00b5530 msgr2=0x7f85f00d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 --2- 172.21.15.40:0/212481786 >> v2:172.21.15.184:3300/0 conn(0x7f85f00b5530 0x7f85f00d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f015ffc0 con 0x7f85f009ba40 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f85e80019f0 con 0x7f85f009ba40 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 >> v1:172.21.15.40:6789/0 conn(0x7f85f009ba40 legacy=0x7f85f009be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0039b0 0x7f85dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc004d40 0x7f85dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85f00b59c0 con 0x7f85dc0039b0 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0160610 con 0x7f85dc004d40 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85dc00b9c0 con 0x7f85f015f6a0 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0039b0 0x7f85dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85effff640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f533a640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc004d40 0x7f85dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 -- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 msgr2=0x7f85dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f015f6a0 0x7f85dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 -- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc004d40 msgr2=0x7f85dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc004d40 0x7f85dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.370+0000 7f85f4b39640 1 -- 172.21.15.40:0/212481786 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc003520 con 0x7f85dc0039b0 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85f4b39640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0039b0 0x7f85dc003dd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f85e800a850 tx=0x7f85dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2edc676454d7b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e800a4a0 con 0x7f85dc0039b0 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e802f070 con 0x7f85dc0039b0 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85ef7fe640 1 -- 172.21.15.40:0/212481786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e802bce0 con 0x7f85dc0039b0 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85f6d9d640 1 -- 172.21.15.40:0/212481786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0039b0 msgr2=0x7f85dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.371+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/212481786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0039b0 0x7f85dc003dd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f85e800a850 tx=0x7f85dc005830 comp rx=0 tx=0).stop 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.372+0000 7f85f533a640 1 -- 172.21.15.40:0/212481786 reap_dead start 2024-04-26T21:15:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.372+0000 7f85f6d9d640 1 -- 172.21.15.40:0/212481786 shutdown_connections 2024-04-26T21:15:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.372+0000 7f85f6d9d640 1 -- 172.21.15.40:0/212481786 >> 172.21.15.40:0/212481786 conn(0x7f85f00b4a10 msgr2=0x7f85f00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.372+0000 7f85f6d9d640 1 -- 172.21.15.40:0/212481786 shutdown_connections 2024-04-26T21:15:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.372+0000 7f85f6d9d640 1 -- 172.21.15.40:0/212481786 wait complete. 2024-04-26T21:15:06.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.373+0000 7f85f6d9d640 1 Processor -- start 2024-04-26T21:15:06.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.373+0000 7f85f6d9d640 1 -- start start 2024-04-26T21:15:06.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 0x7f85f01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f00b1df0 0x7f85f0161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85f00706f0 con 0x7f85f00b6550 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0070570 con 0x7f85dc0039b0 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f85f0070870 con 0x7f85f00b1df0 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39372/0 (socket says 172.21.15.40:39372) 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 -- 172.21.15.40:0/1702163379 learned_addr learned my addr 172.21.15.40:0/1702163379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f533a640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f00b1df0 0x7f85f0161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f4b39640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 0x7f85f01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 -- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f00b1df0 msgr2=0x7f85f0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f00b1df0 0x7f85f0161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 -- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 msgr2=0x7f85f01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 0x7f85f01566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 -- 172.21.15.40:0/1702163379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f00b8940 con 0x7f85f00b6550 2024-04-26T21:15:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f4b39640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 0x7f85f01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85effff640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f85d80119d0 tx=0x7f85d8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d800fa90 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d800fc30 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f0156be0 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.374+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8014440 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.375+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f0160a40 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.375+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f00696b0 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.376+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85d8012040 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.376+0000 7f85ed7fa640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 0x7f85d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.377+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f85d808bd60 con 0x7f85f00b6550 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.377+0000 7f85f4b39640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 0x7f85d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.377+0000 7f85f4b39640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 0x7f85d006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85e8033000 tx=0x7f85e8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:06.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.381+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85f00696b0 con 0x7f85f00b6550 2024-04-26T21:15:06.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.640+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f85f00b21d0 con 0x7f85f00b6550 2024-04-26T21:15:06.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.925+0000 7f85ed7fa640 1 -- 172.21.15.40:0/1702163379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v180) ==== 150+0+0 (secure 0 0 0) 0x7f85d8019030 con 0x7f85f00b6550 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.927+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 msgr2=0x7f85d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.927+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 0x7f85d006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85e8033000 tx=0x7f85e8009030 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.927+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 msgr2=0x7f85f00b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.927+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f85d80119d0 tx=0x7f85d8011dd0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.927+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 shutdown_connections 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f85d006c920 0x7f85d006ede0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f00b1df0 0x7f85f0161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85f00b6550 0x7f85f00b18b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 --2- 172.21.15.40:0/1702163379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85dc0039b0 0x7f85f01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 >> 172.21.15.40:0/1702163379 conn(0x7f85f00b4a10 msgr2=0x7f85f006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 shutdown_connections 2024-04-26T21:15:06.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:06.928+0000 7f85f6d9d640 1 -- 172.21.15.40:0/1702163379 wait complete. 2024-04-26T21:15:06.940 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... skipped 'Require FUSE client to inject client release failure' 2024-04-26T21:15:06.941 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2024-04-26T21:15:06.941 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:15:06.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:07.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.042+0000 7fa151de4640 1 Processor -- start 2024-04-26T21:15:07.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.042+0000 7fa151de4640 1 -- start start 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa151de4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa14c15f340 0x7fa14c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa151de4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0b2510 con 0x7fa14c15f720 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa151de4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0b27f0 con 0x7fa14c0d5610 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa151de4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa14c0b2ad0 con 0x7fa14c15f340 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa14affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa14c15f720 0x7fa14c0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51440/0 (socket says 172.21.15.40:51440) 2024-04-26T21:15:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.043+0000 7fa14affd640 1 -- 172.21.15.40:0/1771124970 learned_addr learned my addr 172.21.15.40:0/1771124970 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/1771124970 >> v2:172.21.15.184:3300/0 conn(0x7fa14c15f340 0x7fa14c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239052757 0 0) 0x7fa14c0b2510 con 0x7fa14c15f720 2024-04-26T21:15:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa134003520 con 0x7fa14c15f720 2024-04-26T21:15:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505003035 0 0) 0x7fa134003520 con 0x7fa14c15f720 2024-04-26T21:15:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 >> v2:172.21.15.184:3300/0 conn(0x7fa14c15f340 msgr2=0x7fa14c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 --2- 172.21.15.40:0/1771124970 >> v2:172.21.15.184:3300/0 conn(0x7fa14c15f340 0x7fa14c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 >> v1:172.21.15.184:6789/0 conn(0x7fa14c0d5610 legacy=0x7fa14c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c0b2db0 con 0x7fa14c15f720 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa140001670 con 0x7fa14c15f720 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 >> v1:172.21.15.40:6789/0 conn(0x7fa14c15f720 legacy=0x7fa14c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 0x7fa134003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa134004800 0x7fa134004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1340051c0 0x7fa13400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c0b2ad0 con 0x7fa1340039b0 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa13400c160 con 0x7fa134004800 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa13400c360 con 0x7fa1340051c0 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 0x7fa134003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14affd640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa134004800 0x7fa134004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14bfff640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1340051c0 0x7fa13400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 -- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1340051c0 msgr2=0x7fa13400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1340051c0 0x7fa13400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 -- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa134004800 msgr2=0x7fa134004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa134004800 0x7fa134004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14b7fe640 1 -- 172.21.15.40:0/1771124970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa134003520 con 0x7fa1340039b0 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14affd640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa134004800 0x7fa134004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.044+0000 7fa14bfff640 1 -- 172.21.15.40:0/1771124970 reap_dead start 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.045+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 0x7fa134003dd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ac0 tx=0x7fa134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5e5d1206e9eca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.045+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c036070 con 0x7fa1340039b0 2024-04-26T21:15:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.045+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c031400 con 0x7fa1340039b0 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.045+0000 7fa14a7fc640 1 -- 172.21.15.40:0/1771124970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c009690 con 0x7fa1340039b0 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 -- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 msgr2=0x7fa134003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 0x7fa134003dd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ac0 tx=0x7fa134005830 comp rx=0 tx=0).stop 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 -- 172.21.15.40:0/1771124970 shutdown_connections 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 --2- 172.21.15.40:0/1771124970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1340039b0 0x7fa134003dd0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 -- 172.21.15.40:0/1771124970 >> 172.21.15.40:0/1771124970 conn(0x7fa14c08ec70 msgr2=0x7fa14c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 -- 172.21.15.40:0/1771124970 shutdown_connections 2024-04-26T21:15:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.046+0000 7fa151de4640 1 -- 172.21.15.40:0/1771124970 wait complete. 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.047+0000 7fa151de4640 1 Processor -- start 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.047+0000 7fa151de4640 1 -- start start 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1340039b0 0x7fa14c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa14c0b7ee0 0x7fa14c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c0b2210 con 0x7fa14c0d1330 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c0b2090 con 0x7fa1340039b0 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa151de4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa14c0d3d40 con 0x7fa14c0b7ee0 2024-04-26T21:15:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39394/0 (socket says 172.21.15.40:39394) 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 -- 172.21.15.40:0/677061457 learned_addr learned my addr 172.21.15.40:0/677061457 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14affd640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa14c0b7ee0 0x7fa14c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1340039b0 0x7fa14c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 -- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa14c0b7ee0 msgr2=0x7fa14c1f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa14c0b7ee0 0x7fa14c1f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 -- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1340039b0 msgr2=0x7fa14c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1340039b0 0x7fa14c1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 -- 172.21.15.40:0/677061457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c1f5b20 con 0x7fa14c0d1330 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa14bfff640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa144007fd0 tx=0x7fa14400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.048+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144003970 con 0x7fa14c0d1330 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.049+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa144018070 con 0x7fa14c0d1330 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.049+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c1f5e00 con 0x7fa14c0d1330 2024-04-26T21:15:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.049+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144003d70 con 0x7fa14c0d1330 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.049+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c153c10 con 0x7fa14c0d1330 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.050+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c1f6320 con 0x7fa14c0d1330 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.050+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1440050d0 con 0x7fa14c0d1330 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.051+0000 7fa133fff640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 0x7fa13806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.051+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fa14408b3f0 con 0x7fa14c0d1330 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.051+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 0x7fa13806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.051+0000 7fa14b7fe640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 0x7fa13806ede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa13c033000 tx=0x7fa13c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.055+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa144059440 con 0x7fa14c0d1330 2024-04-26T21:15:07.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.240+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa14c06a290 con 0x7fa14c0d1330 2024-04-26T21:15:07.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.241+0000 7fa133fff640 1 -- 172.21.15.40:0/677061457 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13018 (secure 0 0 0) 0x7fa14405d740 con 0x7fa14c0d1330 2024-04-26T21:15:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:07.242 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":62,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:05.918499+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/555844344":"2024-04-27T21:15:02.910208+0000","172.21.15.40:6832/555844344":"2024-04-27T21:15:02.910208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 msgr2=0x7fa13806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 0x7fa13806ede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa13c033000 tx=0x7fa13c008030 comp rx=0 tx=0).stop 2024-04-26T21:15:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 msgr2=0x7fa14c0d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa144007fd0 tx=0x7fa14400ec10 comp rx=0 tx=0).stop 2024-04-26T21:15:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 shutdown_connections 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa13806c920 0x7fa13806ede0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa14c0d1330 0x7fa14c0d17b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa14c0b7ee0 0x7fa14c1f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 --2- 172.21.15.40:0/677061457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1340039b0 0x7fa14c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 >> 172.21.15.40:0/677061457 conn(0x7fa14c08ec70 msgr2=0x7fa14c0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 shutdown_connections 2024-04-26T21:15:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.243+0000 7fa151de4640 1 -- 172.21.15.40:0/677061457 wait complete. 2024-04-26T21:15:07.255 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:07.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:07.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.358+0000 7f5dfca1d640 1 Processor -- start 2024-04-26T21:15:07.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.358+0000 7f5dfca1d640 1 -- start start 2024-04-26T21:15:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5dfca1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 0x7f5df806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5dfca1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df809c700 con 0x7f5df80b33e0 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5dfca1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80d1c50 con 0x7f5df80d0010 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5dfca1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80b4480 con 0x7f5df80b3c20 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5df80b33e0 0x7f5df8163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51452/0 (socket says 172.21.15.40:51452) 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df6d9d640 1 -- 172.21.15.40:0/3393999849 learned_addr learned my addr 172.21.15.40:0/3393999849 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5deffff640 1 --2- 172.21.15.40:0/3393999849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 0x7f5df806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063822392 0 0) 0x7f5df809c700 con 0x7f5df80b33e0 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5de4003520 con 0x7f5df80b33e0 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006542338 0 0) 0x7f5df80d1c50 con 0x7f5df80d0010 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df809c700 con 0x7f5df80d0010 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2673665021 0 0) 0x7f5de4003520 con 0x7f5df80b33e0 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5deffff640 1 -- 172.21.15.40:0/3393999849 >> v1:172.21.15.184:6789/0 conn(0x7f5df80d0010 legacy=0x7f5df80b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5deffff640 1 -- 172.21.15.40:0/3393999849 >> v1:172.21.15.40:6789/0 conn(0x7f5df80b33e0 legacy=0x7f5df8163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.359+0000 7f5deffff640 1 -- 172.21.15.40:0/3393999849 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80d1f30 con 0x7f5df80b3c20 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.360+0000 7f5deffff640 1 --2- 172.21.15.40:0/3393999849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 0x7f5df806db10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5de00097d0 tx=0x7f5de002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26684eacfa69efb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.360+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0037070 con 0x7f5df80b3c20 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.360+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de002ac30 con 0x7f5df80b3c20 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.360+0000 7f5df5d9b640 1 -- 172.21.15.40:0/3393999849 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00019f0 con 0x7f5df80b3c20 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 -- 172.21.15.40:0/3393999849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 msgr2=0x7f5df806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/3393999849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 0x7f5df806db10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5de00097d0 tx=0x7f5de002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 -- 172.21.15.40:0/3393999849 shutdown_connections 2024-04-26T21:15:07.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/3393999849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b3c20 0x7f5df806db10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 -- 172.21.15.40:0/3393999849 >> 172.21.15.40:0/3393999849 conn(0x7f5df809ba40 msgr2=0x7f5df809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 -- 172.21.15.40:0/3393999849 shutdown_connections 2024-04-26T21:15:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.361+0000 7f5dfca1d640 1 -- 172.21.15.40:0/3393999849 wait complete. 2024-04-26T21:15:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.362+0000 7f5dfca1d640 1 Processor -- start 2024-04-26T21:15:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.362+0000 7f5dfca1d640 1 -- start start 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 0x7f5df81458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df80d0010 0x7f5df8143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5df806e050 con 0x7f5df80b3c20 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80b3a80 con 0x7f5df80d0010 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5dfca1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5df8163990 con 0x7f5df80b33e0 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39410/0 (socket says 172.21.15.40:39410) 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 -- 172.21.15.40:0/143691538 learned_addr learned my addr 172.21.15.40:0/143691538 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5df659c640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 0x7f5df81458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5df6d9d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df80d0010 0x7f5df8143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 -- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 msgr2=0x7f5df81458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 0x7f5df81458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 -- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df80d0010 msgr2=0x7f5df8143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df80d0010 0x7f5df8143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5deffff640 1 -- 172.21.15.40:0/143691538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8144810 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.363+0000 7f5df659c640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 0x7f5df81458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5deffff640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5de00097d0 tx=0x7f5de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0037050 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de0038040 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df8144af0 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0006e10 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.364+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df81450b0 con 0x7f5df80b3c20 2024-04-26T21:15:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.365+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df806a290 con 0x7f5df80b3c20 2024-04-26T21:15:07.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.365+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5de0045020 con 0x7f5df80b3c20 2024-04-26T21:15:07.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.366+0000 7f5deeffd640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 0x7f5dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.366+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f5de00ab8a0 con 0x7f5df80b3c20 2024-04-26T21:15:07.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.366+0000 7f5df659c640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 0x7f5dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.366+0000 7f5df659c640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 0x7f5dd806edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5de80011e0 tx=0x7f5de8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.370+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5de004c020 con 0x7f5df80b3c20 2024-04-26T21:15:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.554+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5df8068e00 con 0x7f5df80b3c20 2024-04-26T21:15:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.554+0000 7f5deeffd640 1 -- 172.21.15.40:0/143691538 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7f5de0079fd0 con 0x7f5df80b3c20 2024-04-26T21:15:07.555 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 msgr2=0x7f5dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 0x7f5dd806edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5de80011e0 tx=0x7f5de8009040 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 msgr2=0x7f5df815f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5de00097d0 tx=0x7f5de00046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 shutdown_connections 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dd806c8f0 0x7f5dd806edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df80d0010 0x7f5df8143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5df80b3c20 0x7f5df815f000 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 --2- 172.21.15.40:0/143691538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df80b33e0 0x7f5df81458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 >> 172.21.15.40:0/143691538 conn(0x7f5df809ba40 msgr2=0x7f5df809be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 shutdown_connections 2024-04-26T21:15:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.556+0000 7f5dfca1d640 1 -- 172.21.15.40:0/143691538 wait complete. 2024-04-26T21:15:07.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-04-26T21:15:07.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 Processor -- start 2024-04-26T21:15:07.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 -- start start 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8341629b0 0x7fb8340d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb834162d90 0x7fb8340d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8340d5de0 0x7fb8340d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8340cfca0 con 0x7fb8340d5de0 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8340d0ae0 con 0x7fb834162d90 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.669+0000 7fb838ab0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8340cfed0 con 0x7fb8341629b0 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8340d5de0 0x7fb8340d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8340d5de0 0x7fb8340d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39418/0 (socket says 172.21.15.40:39418) 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 -- 172.21.15.40:0/3214748738 learned_addr learned my addr 172.21.15.40:0/3214748738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb82bfff640 1 --2- 172.21.15.40:0/3214748738 >> v2:172.21.15.184:3300/0 conn(0x7fb834162d90 0x7fb8340d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb83259c640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8341629b0 0x7fb8340d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 -- 172.21.15.40:0/3214748738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8341629b0 msgr2=0x7fb8340d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:07.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8341629b0 0x7fb8340d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 -- 172.21.15.40:0/3214748738 >> v2:172.21.15.184:3300/0 conn(0x7fb834162d90 msgr2=0x7fb8340d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:07.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 --2- 172.21.15.40:0/3214748738 >> v2:172.21.15.184:3300/0 conn(0x7fb834162d90 0x7fb8340d58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 -- 172.21.15.40:0/3214748738 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8340d01b0 con 0x7fb8340d5de0 2024-04-26T21:15:07.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.670+0000 7fb832d9d640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8340d19d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb83408de20 tx=0x7fb82c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e4a031da123d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb831d9b640 1 -- 172.21.15.40:0/3214748738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c036070 con 0x7fb8340d5de0 2024-04-26T21:15:07.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb831d9b640 1 -- 172.21.15.40:0/3214748738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c031600 con 0x7fb8340d5de0 2024-04-26T21:15:07.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb831d9b640 1 -- 172.21.15.40:0/3214748738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0319d0 con 0x7fb8340d5de0 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 -- 172.21.15.40:0/3214748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 msgr2=0x7fb8340d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8340d19d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb83408de20 tx=0x7fb82c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 -- 172.21.15.40:0/3214748738 shutdown_connections 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8340d19d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 --2- 172.21.15.40:0/3214748738 >> v2:172.21.15.184:3300/0 conn(0x7fb834162d90 0x7fb8340d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 --2- 172.21.15.40:0/3214748738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8341629b0 0x7fb8340d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 -- 172.21.15.40:0/3214748738 >> 172.21.15.40:0/3214748738 conn(0x7fb83409e4d0 msgr2=0x7fb8340d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 -- 172.21.15.40:0/3214748738 shutdown_connections 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.671+0000 7fb838ab0640 1 -- 172.21.15.40:0/3214748738 wait complete. 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 Processor -- start 2024-04-26T21:15:07.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 -- start start 2024-04-26T21:15:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8341629b0 0x7fb834158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 0x7fb8340b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340d3fe0 con 0x7fb8340d5de0 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340d0770 con 0x7fb8341629b0 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb838ab0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8340d1f10 con 0x7fb834162d90 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb83259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.673+0000 7fb83259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39420/0 (socket says 172.21.15.40:39420) 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 -- 172.21.15.40:0/4144955200 learned_addr learned my addr 172.21.15.40:0/4144955200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb82bfff640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8341629b0 0x7fb834158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb832d9d640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 0x7fb8340b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 -- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 msgr2=0x7fb8340b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 0x7fb8340b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 -- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8341629b0 msgr2=0x7fb834158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8341629b0 0x7fb834158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 -- 172.21.15.40:0/4144955200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8340b9a30 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb832d9d640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 0x7fb8340b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb83259c640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d800 tx=0x7fb82400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824018020 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8240096a0 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb834158d50 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824009a70 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.674+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8340b8ba0 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.675+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8340693d0 con 0x7fb8340d5de0 2024-04-26T21:15:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.676+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb82401b020 con 0x7fb8340d5de0 2024-04-26T21:15:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.676+0000 7fb82affd640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 0x7fb81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.676+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fb82408ba40 con 0x7fb8340d5de0 2024-04-26T21:15:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.676+0000 7fb82bfff640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 0x7fb81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.677+0000 7fb82bfff640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 0x7fb81406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb81c006fd0 tx=0x7fb81c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.680+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb824022030 con 0x7fb8340d5de0 2024-04-26T21:15:07.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:07.942+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7fb8340d22e0 con 0x7fb8340d5de0 2024-04-26T21:15:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.931+0000 7fb82affd640 1 -- 172.21.15.40:0/4144955200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v182) ==== 151+0+0 (secure 0 0 0) 0x7fb82405a170 con 0x7fb8340d5de0 2024-04-26T21:15:08.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 msgr2=0x7fb81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 0x7fb81406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb81c006fd0 tx=0x7fb81c008040 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 msgr2=0x7fb8341f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d800 tx=0x7fb82400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 shutdown_connections 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb81406c8f0 0x7fb81406edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb834162d90 0x7fb8340b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8341629b0 0x7fb834158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 --2- 172.21.15.40:0/4144955200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8340d5de0 0x7fb8341f5250 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 >> 172.21.15.40:0/4144955200 conn(0x7fb83409e4d0 msgr2=0x7fb8340ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 shutdown_connections 2024-04-26T21:15:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:08.934+0000 7fb838ab0640 1 -- 172.21.15.40:0/4144955200 wait complete. 2024-04-26T21:15:08.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:15:09.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.059+0000 7fb1153af640 1 Processor -- start 2024-04-26T21:15:09.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.059+0000 7fb1153af640 1 -- start start 2024-04-26T21:15:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb1153af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1100d0010 0x7fb11006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb1153af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1100b3c20 0x7fb1100b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb1153af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11009c700 con 0x7fb1100b3210 2024-04-26T21:15:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb1153af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb1100b4480 con 0x7fb1100d0010 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb1153af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb11006e050 con 0x7fb1100b3c20 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1100d0010 0x7fb11006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb1100b3210 0x7fb11015fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51456/0 (socket says 172.21.15.40:51456) 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10f7fe640 1 -- 172.21.15.40:0/2299539753 learned_addr learned my addr 172.21.15.40:0/2299539753 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10e7fc640 1 --2- 172.21.15.40:0/2299539753 >> v2:172.21.15.184:3300/0 conn(0x7fb1100b3c20 0x7fb1100b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10effd640 1 -- 172.21.15.40:0/2299539753 >> v2:172.21.15.184:3300/0 conn(0x7fb1100b3c20 msgr2=0x7fb1100b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10effd640 1 --2- 172.21.15.40:0/2299539753 >> v2:172.21.15.184:3300/0 conn(0x7fb1100b3c20 0x7fb1100b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10effd640 1 -- 172.21.15.40:0/2299539753 >> v1:172.21.15.40:6789/0 conn(0x7fb1100b3210 legacy=0x7fb11015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10dffb640 1 -- 172.21.15.40:0/2299539753 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934455297 0 0) 0x7fb11009c700 con 0x7fb1100b3210 2024-04-26T21:15:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.060+0000 7fb10effd640 1 -- 172.21.15.40:0/2299539753 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1100d1c50 con 0x7fb1100d0010 2024-04-26T21:15:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.061+0000 7fb10effd640 1 --2- 172.21.15.40:0/2299539753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100d0010 0x7fb11006db10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb110073930 tx=0x7fb10002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3dba762a43bed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.061+0000 7fb10dffb640 1 -- 172.21.15.40:0/2299539753 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100037070 con 0x7fb1100d0010 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.061+0000 7fb10dffb640 1 -- 172.21.15.40:0/2299539753 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100032440 con 0x7fb1100d0010 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.061+0000 7fb10dffb640 1 -- 172.21.15.40:0/2299539753 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100032810 con 0x7fb1100d0010 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 -- 172.21.15.40:0/2299539753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100d0010 msgr2=0x7fb11006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 --2- 172.21.15.40:0/2299539753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100d0010 0x7fb11006db10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb110073930 tx=0x7fb10002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 -- 172.21.15.40:0/2299539753 shutdown_connections 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 --2- 172.21.15.40:0/2299539753 >> v2:172.21.15.184:3300/0 conn(0x7fb1100b3c20 0x7fb1100b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 --2- 172.21.15.40:0/2299539753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100d0010 0x7fb11006db10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 -- 172.21.15.40:0/2299539753 >> 172.21.15.40:0/2299539753 conn(0x7fb11009ba40 msgr2=0x7fb11009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 -- 172.21.15.40:0/2299539753 shutdown_connections 2024-04-26T21:15:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.062+0000 7fb1153af640 1 -- 172.21.15.40:0/2299539753 wait complete. 2024-04-26T21:15:09.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.063+0000 7fb1153af640 1 Processor -- start 2024-04-26T21:15:09.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.063+0000 7fb1153af640 1 -- start start 2024-04-26T21:15:09.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100b3210 0x7fb110157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1100d0010 0x7fb110161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1100b37c0 con 0x7fb1100b3c20 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1100b3640 con 0x7fb1100d0010 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb1153af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb1100b3940 con 0x7fb1100b3210 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39442/0 (socket says 172.21.15.40:39442) 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 -- 172.21.15.40:0/3648652377 learned_addr learned my addr 172.21.15.40:0/3648652377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10f7fe640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1100d0010 0x7fb110161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10effd640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100b3210 0x7fb110157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 -- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100b3210 msgr2=0x7fb110157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100b3210 0x7fb110157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 -- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1100d0010 msgr2=0x7fb110161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1100d0010 0x7fb110161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.064+0000 7fb10e7fc640 1 -- 172.21.15.40:0/3648652377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11015ae80 con 0x7fb1100b3c20 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb10e7fc640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb10400d5f0 tx=0x7fb10400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040125f0 con 0x7fb1100b3c20 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb104019070 con 0x7fb1100b3c20 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11015b160 con 0x7fb1100b3c20 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040129d0 con 0x7fb1100b3c20 2024-04-26T21:15:09.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.065+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb110155640 con 0x7fb1100b3c20 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.066+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb110155170 con 0x7fb1100b3c20 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.066+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb104012b70 con 0x7fb1100b3c20 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.067+0000 7fb0f77fe640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 0x7fb0fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.067+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fb10408bc10 con 0x7fb1100b3c20 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.067+0000 7fb10effd640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 0x7fb0fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.068+0000 7fb10effd640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 0x7fb0fc06edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb100034000 tx=0x7fb10002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.071+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb10405a3d0 con 0x7fb1100b3c20 2024-04-26T21:15:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.257+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb110068e00 con 0x7fb1100b3c20 2024-04-26T21:15:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.261+0000 7fb0f77fe640 1 -- 172.21.15.40:0/3648652377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fb104059cd0 con 0x7fb1100b3c20 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 msgr2=0x7fb0fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 0x7fb0fc06edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb100034000 tx=0x7fb10002e000 comp rx=0 tx=0).stop 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 msgr2=0x7fb110154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb10400d5f0 tx=0x7fb10400dab0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 shutdown_connections 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb0fc06c8f0 0x7fb0fc06edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1100d0010 0x7fb110161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1100b3c20 0x7fb110154400 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 --2- 172.21.15.40:0/3648652377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100b3210 0x7fb110157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 >> 172.21.15.40:0/3648652377 conn(0x7fb11009ba40 msgr2=0x7fb1100d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 shutdown_connections 2024-04-26T21:15:09.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.263+0000 7fb1153af640 1 -- 172.21.15.40:0/3648652377 wait complete. 2024-04-26T21:15:09.275 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:15:09.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:09.314 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:09.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:15:09.371 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:09.371 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:09.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:15:09.386 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:09.386 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:09.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:15:09.404 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:09.404 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:09.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:15:09.442 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:09.442 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:09.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:15:09.460 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:15:09.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.582+0000 7f405fd9d640 1 Processor -- start 2024-04-26T21:15:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.582+0000 7f405fd9d640 1 -- start start 2024-04-26T21:15:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405fd9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4058160630 0x7f40580b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405fd9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40580b4530 con 0x7f4058145980 2024-04-26T21:15:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405fd9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40580b4810 con 0x7f40580b61a0 2024-04-26T21:15:09.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405fd9d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4058160ac0 con 0x7f4058160630 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405e33a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4058160630 0x7f40580b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405db39640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4058145980 0x7f40580b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51460/0 (socket says 172.21.15.40:51460) 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.583+0000 7f405db39640 1 -- 172.21.15.40:0/1081660515 learned_addr learned my addr 172.21.15.40:0/1081660515 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405e33a640 1 -- 172.21.15.40:0/1081660515 >> v1:172.21.15.184:6789/0 conn(0x7f40580b61a0 legacy=0x7f40580720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405e33a640 1 -- 172.21.15.40:0/1081660515 >> v1:172.21.15.40:6789/0 conn(0x7f4058145980 legacy=0x7f40580b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405e33a640 1 -- 172.21.15.40:0/1081660515 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40580b4af0 con 0x7f4058160630 2024-04-26T21:15:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405e33a640 1 --2- 172.21.15.40:0/1081660515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058160630 0x7f40580b3de0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4048009d90 tx=0x7f404802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af73d610fe095513 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405cb37640 1 -- 172.21.15.40:0/1081660515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048031600 con 0x7f4058160630 2024-04-26T21:15:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405cb37640 1 -- 172.21.15.40:0/1081660515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40480317a0 con 0x7f4058160630 2024-04-26T21:15:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.584+0000 7f405cb37640 1 -- 172.21.15.40:0/1081660515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048031b70 con 0x7f4058160630 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 -- 172.21.15.40:0/1081660515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058160630 msgr2=0x7f40580b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 --2- 172.21.15.40:0/1081660515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058160630 0x7f40580b3de0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4048009d90 tx=0x7f404802a610 comp rx=0 tx=0).stop 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 -- 172.21.15.40:0/1081660515 shutdown_connections 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 --2- 172.21.15.40:0/1081660515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058160630 0x7f40580b3de0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 -- 172.21.15.40:0/1081660515 >> 172.21.15.40:0/1081660515 conn(0x7f40580d0d30 msgr2=0x7f40580d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 -- 172.21.15.40:0/1081660515 shutdown_connections 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.585+0000 7f405fd9d640 1 -- 172.21.15.40:0/1081660515 wait complete. 2024-04-26T21:15:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.586+0000 7f405fd9d640 1 Processor -- start 2024-04-26T21:15:09.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.586+0000 7f405fd9d640 1 -- start start 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40580b61a0 0x7f40580b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4058160630 0x7f40581545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40580727e0 con 0x7f4058145980 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40580b5140 con 0x7f40580b61a0 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405fd9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4058072960 con 0x7f4058160630 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39452/0 (socket says 172.21.15.40:39452) 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 -- 172.21.15.40:0/2500087505 learned_addr learned my addr 172.21.15.40:0/2500087505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405db39640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40580b61a0 0x7f40580b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 -- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4058160630 msgr2=0x7f40581545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4058160630 0x7f40581545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 -- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40580b61a0 msgr2=0x7f40580b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40580b61a0 0x7f40580b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.587+0000 7f405d338640 1 -- 172.21.15.40:0/2500087505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405813bb90 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f405d338640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c410 tx=0x7f404c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c017070 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c0035f0 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0039e0 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405813be70 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.588+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405813c430 con 0x7f4058145980 2024-04-26T21:15:09.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.589+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40580b4270 con 0x7f4058145980 2024-04-26T21:15:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.589+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f404c01a070 con 0x7f4058145980 2024-04-26T21:15:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.590+0000 7f40567fc640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 0x7f404406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.590+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f404c08ad90 con 0x7f4058145980 2024-04-26T21:15:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.590+0000 7f405db39640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 0x7f404406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.591+0000 7f405db39640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 0x7f404406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4050000f60 tx=0x7f4050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:09.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.594+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40580b4270 con 0x7f4058145980 2024-04-26T21:15:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.819+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4058069be0 con 0x7f4058145980 2024-04-26T21:15:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.819+0000 7f40567fc640 1 -- 172.21.15.40:0/2500087505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+3875 (secure 0 0 0) 0x7f4058069be0 con 0x7f4058145980 2024-04-26T21:15:09.821 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:09.821 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2719059895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2719059895},{"type":"v1","addr":"172.21.15.184:6835","nonce":2719059895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1155980532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1155980532},{"type":"v1","addr":"172.21.15.184:6837","nonce":1155980532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2444218952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2444218952},{"type":"v1","addr":"172.21.15.40:6835","nonce":2444218952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":6369,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1232242948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1232242948},{"type":"v1","addr":"172.21.15.40:6834","nonce":1232242948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-04-26T21:15:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 31 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 msgr2=0x7f404406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 0x7f404406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4050000f60 tx=0x7f4050009040 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 msgr2=0x7f405813b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c410 tx=0x7f404c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 shutdown_connections 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f404406c8f0 0x7f404406edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4058160630 0x7f40581545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4058145980 0x7f405813b590 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 --2- 172.21.15.40:0/2500087505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40580b61a0 0x7f40580b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 >> 172.21.15.40:0/2500087505 conn(0x7f40580d0d30 msgr2=0x7f405815d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 shutdown_connections 2024-04-26T21:15:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.821+0000 7f405fd9d640 1 -- 172.21.15.40:0/2500087505 wait complete. 2024-04-26T21:15:09.833 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:15:09.833 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:15:09.834 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:15:09.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:09.834 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:15:09.834 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:15:09.834 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:15:09.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:09.835 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:15:09.835 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:15:09.835 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:15:09.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:09.835 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:15:09.835 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:15:09.835 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:15:09.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:09.846 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:15:09.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:15:09.847 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:15:09.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:15:09.848 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:15:09.848 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:15:09.849 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:15:09.849 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:15:09.889 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:15:09.889 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:09.890 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:15:09.891 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:09.892 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:15:09.892 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:09.892 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:15:09.893 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:09.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:15:09.966 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:15:09.966 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:15:09.975 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:15:09.977 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:15:09.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 Processor -- start 2024-04-26T21:15:09.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 -- start start 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc824070d20 0x7fc824071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070520 0x7fc824163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8240d14c0 con 0x7fc82409c400 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc8240d1290 con 0x7fc824070d20 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.998+0000 7fc829f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc824071640 con 0x7fc824070520 2024-04-26T21:15:10.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc822ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc82409c400 0x7fc824162f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51488/0 (socket says 172.21.15.40:51488) 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc822ffd640 1 -- 172.21.15.40:0/866548170 learned_addr learned my addr 172.21.15.40:0/866548170 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc823fff640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070520 0x7fc824163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8237fe640 1 --2- 172.21.15.40:0/866548170 >> v2:172.21.15.184:3300/0 conn(0x7fc824070d20 0x7fc824071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711469475 0 0) 0x7fc8240d14c0 con 0x7fc82409c400 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc80c003520 con 0x7fc82409c400 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1897401106 0 0) 0x7fc80c003520 con 0x7fc82409c400 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8237fe640 1 -- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070520 msgr2=0x7fc824163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8237fe640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070520 0x7fc824163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8237fe640 1 -- 172.21.15.40:0/866548170 >> v1:172.21.15.40:6789/0 conn(0x7fc82409c400 legacy=0x7fc824162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:09.999+0000 7fc8237fe640 1 -- 172.21.15.40:0/866548170 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8240ce5b0 con 0x7fc824070d20 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc8237fe640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc824070d20 0x7fc824071100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8140097a0 tx=0x7fc81402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb987a9480fc34c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814037070 con 0x7fc824070d20 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81402aba0 con 0x7fc824070d20 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc8227fc640 1 -- 172.21.15.40:0/866548170 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818001670 con 0x7fc824070d20 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc829f11640 1 -- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc824070d20 msgr2=0x7fc824071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.000+0000 7fc829f11640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc824070d20 0x7fc824071100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8140097a0 tx=0x7fc81402a110 comp rx=0 tx=0).stop 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 -- 172.21.15.40:0/866548170 shutdown_connections 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070520 0x7fc824163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 --2- 172.21.15.40:0/866548170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc824070d20 0x7fc824071100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 -- 172.21.15.40:0/866548170 >> 172.21.15.40:0/866548170 conn(0x7fc8240b4a40 msgr2=0x7fc8240b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 -- 172.21.15.40:0/866548170 shutdown_connections 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.001+0000 7fc829f11640 1 -- 172.21.15.40:0/866548170 wait complete. 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.002+0000 7fc829f11640 1 Processor -- start 2024-04-26T21:15:10.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.002+0000 7fc829f11640 1 -- start start 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070d20 0x7fc8240b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc82409c400 0x7fc8240b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240d1d20 con 0x7fc824070520 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240d1a40 con 0x7fc82409c400 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8240d2000 con 0x7fc824070d20 2024-04-26T21:15:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39514/0 (socket says 172.21.15.40:39514) 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 -- 172.21.15.40:0/3461511544 learned_addr learned my addr 172.21.15.40:0/3461511544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 -- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070d20 msgr2=0x7fc8240b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070d20 0x7fc8240b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 -- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc82409c400 msgr2=0x7fc8240b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc82409c400 0x7fc8240b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 -- 172.21.15.40:0/3461511544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8240c9290 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc8237fe640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc814035040 tx=0x7fc8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814041020 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81403d6e0 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.003+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8240c9570 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.004+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814037040 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.004+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8240b5ef0 con 0x7fc824070520 2024-04-26T21:15:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.004+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8240693d0 con 0x7fc824070520 2024-04-26T21:15:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.005+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc814044020 con 0x7fc824070520 2024-04-26T21:15:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.005+0000 7fc80bfff640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 0x7fc81006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.005+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fc8140abf40 con 0x7fc824070520 2024-04-26T21:15:10.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.006+0000 7fc822ffd640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 0x7fc81006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:10.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.006+0000 7fc822ffd640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 0x7fc81006ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8240c8370 tx=0x7fc818001180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:10.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.010+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc81404b030 con 0x7fc824070520 2024-04-26T21:15:10.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.205+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc824075c00 con 0x7fc824070520 2024-04-26T21:15:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.949+0000 7fc80bfff640 1 -- 172.21.15.40:0/3461511544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) ==== 97+0+0 (secure 0 0 0) 0x7fc81407a700 con 0x7fc824070520 2024-04-26T21:15:10.950 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 msgr2=0x7fc81006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 0x7fc81006ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8240c8370 tx=0x7fc818001180 comp rx=0 tx=0).stop 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 msgr2=0x7fc824160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc814035040 tx=0x7fc8140046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 shutdown_connections 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c920 0x7fc81006ede0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc82409c400 0x7fc8240b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc824070d20 0x7fc8240b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 --2- 172.21.15.40:0/3461511544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc824070520 0x7fc824160550 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:10.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 >> 172.21.15.40:0/3461511544 conn(0x7fc8240b4a40 msgr2=0x7fc8240b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:10.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 shutdown_connections 2024-04-26T21:15:10.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:10.952+0000 7fc829f11640 1 -- 172.21.15.40:0/3461511544 wait complete. 2024-04-26T21:15:10.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:15:11.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.073+0000 7f59dc647640 1 Processor -- start 2024-04-26T21:15:11.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.073+0000 7f59dc647640 1 -- start start 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dc647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dc647640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59d415f750 0x7f59d40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dc647640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d40d03a0 con 0x7f59d415f370 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dc647640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f59d40b4d10 con 0x7f59d415f750 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dc647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59d40d0680 con 0x7f59d40d3ee0 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dabe4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59d415f750 0x7f59d40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dabe4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59d415f750 0x7f59d40b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50988/0 (socket says 172.21.15.40:50988) 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.074+0000 7f59dabe4640 1 -- 172.21.15.40:0/2222792760 learned_addr learned my addr 172.21.15.40:0/2222792760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59d9be2640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59dabe4640 1 -- 172.21.15.40:0/2222792760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 msgr2=0x7f59d40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59dabe4640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59dabe4640 1 -- 172.21.15.40:0/2222792760 >> v1:172.21.15.184:6789/0 conn(0x7f59d415f370 legacy=0x7f59d40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59dabe4640 1 -- 172.21.15.40:0/2222792760 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d40d0960 con 0x7f59d415f750 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.075+0000 7f59dabe4640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40b47d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009e50 tx=0x7f59c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882bd03dc83c017f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59d93e1640 1 -- 172.21.15.40:0/2222792760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4036070 con 0x7f59d415f750 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59d93e1640 1 -- 172.21.15.40:0/2222792760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4031600 con 0x7f59d415f750 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59d93e1640 1 -- 172.21.15.40:0/2222792760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c40319d0 con 0x7f59d415f750 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 -- 172.21.15.40:0/2222792760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 msgr2=0x7f59d40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40b47d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009e50 tx=0x7f59c402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 -- 172.21.15.40:0/2222792760 shutdown_connections 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40b47d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 --2- 172.21.15.40:0/2222792760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 -- 172.21.15.40:0/2222792760 >> 172.21.15.40:0/2222792760 conn(0x7f59d408ec10 msgr2=0x7f59d409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 -- 172.21.15.40:0/2222792760 shutdown_connections 2024-04-26T21:15:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.076+0000 7f59dc647640 1 -- 172.21.15.40:0/2222792760 wait complete. 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 Processor -- start 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 -- start start 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59d415f370 0x7f59d4157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40b51c0 con 0x7f59d415f750 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40d0f20 con 0x7f59d415f370 2024-04-26T21:15:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dc647640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f59d40b5340 con 0x7f59d40d3ee0 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dabe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dabe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50998/0 (socket says 172.21.15.40:50998) 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.078+0000 7f59dabe4640 1 -- 172.21.15.40:0/2322238409 learned_addr learned my addr 172.21.15.40:0/2322238409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59da3e3640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59d9be2640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59d415f370 0x7f59d4157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 -- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 msgr2=0x7f59d40b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 -- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59d415f370 msgr2=0x7f59d4157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59d415f370 0x7f59d4157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 -- 172.21.15.40:0/2322238409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d40b8de0 con 0x7f59d415f750 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59da3e3640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dabe4640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009e50 tx=0x7f59c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4041020 con 0x7f59d415f750 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4007780 con 0x7f59d415f750 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4036040 con 0x7f59d415f750 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d40b90c0 con 0x7f59d415f750 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.079+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d40c8d80 con 0x7f59d415f750 2024-04-26T21:15:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.080+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d40d2900 con 0x7f59d415f750 2024-04-26T21:15:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.081+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59c4044020 con 0x7f59d415f750 2024-04-26T21:15:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.081+0000 7f59d2ffd640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 0x7f59c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.081+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f59c40ac4c0 con 0x7f59d415f750 2024-04-26T21:15:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.082+0000 7f59da3e3640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 0x7f59c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.082+0000 7f59da3e3640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 0x7f59c006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ceed0 tx=0x7f59cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.085+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59d40d2900 con 0x7f59d415f750 2024-04-26T21:15:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.348+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f59d40d2560 con 0x7f59d415f750 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.349+0000 7f59d2ffd640 1 -- 172.21.15.40:0/2322238409 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f59c404b030 con 0x7f59d415f750 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:15:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:15:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:15:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:15:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:15:11.362 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:15:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 msgr2=0x7f59c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 0x7f59c006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ceed0 tx=0x7f59cc009040 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 msgr2=0x7f59d40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009e50 tx=0x7f59c4004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 shutdown_connections 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f59c006c8f0 0x7f59c006edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59d415f750 0x7f59d40cb5e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59d415f370 0x7f59d4157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 --2- 172.21.15.40:0/2322238409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59d40d3ee0 0x7f59d40b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 >> 172.21.15.40:0/2322238409 conn(0x7f59d408ec10 msgr2=0x7f59d40d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 shutdown_connections 2024-04-26T21:15:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.351+0000 7f59dc647640 1 -- 172.21.15.40:0/2322238409 wait complete. 2024-04-26T21:15:11.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:11.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.464+0000 7f9f4fcb8640 1 Processor -- start 2024-04-26T21:15:11.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.464+0000 7f9f4fcb8640 1 -- start start 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4fcb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9f480cf2e0 0x7f9f4806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4fcb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f4806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4fcb8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f480d1740 con 0x7f9f4806e2c0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4fcb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4806e750 con 0x7f9f480cf2e0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4fcb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9f480d1a20 con 0x7f9f480d11b0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4e255640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9f4806e2c0 0x7f9f4815ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38986/0 (socket says 172.21.15.40:38986) 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4e255640 1 -- 172.21.15.40:0/1193275600 learned_addr learned my addr 172.21.15.40:0/1193275600 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254855425 0 0) 0x7f9f480d1740 con 0x7f9f4806e2c0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4d253640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f4806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f34003520 con 0x7f9f4806e2c0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4da54640 1 --2- 172.21.15.40:0/1193275600 >> v2:172.21.15.184:3300/0 conn(0x7f9f480cf2e0 0x7f9f4806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.465+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1299956992 0 0) 0x7f9f34003520 con 0x7f9f4806e2c0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 msgr2=0x7f9f4806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f4806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 >> v2:172.21.15.184:3300/0 conn(0x7f9f480cf2e0 msgr2=0x7f9f4806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 --2- 172.21.15.40:0/1193275600 >> v2:172.21.15.184:3300/0 conn(0x7f9f480cf2e0 0x7f9f4806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f480d1d00 con 0x7f9f4806e2c0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9f400019f0 con 0x7f9f4806e2c0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4d253640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f4806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 >> v1:172.21.15.40:6789/0 conn(0x7f9f4806e2c0 legacy=0x7f9f4815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f340039b0 0x7f9f34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f34004800 0x7f9f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4806e750 con 0x7f9f340039b0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f480d1a20 con 0x7f9f34004800 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3400b9c0 con 0x7f9f480d11b0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f340039b0 0x7f9f34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4d253640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f34004800 0x7f9f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4e255640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 -- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 msgr2=0x7f9f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 -- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f34004800 msgr2=0x7f9f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f34004800 0x7f9f34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4da54640 1 -- 172.21.15.40:0/1193275600 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34003520 con 0x7f9f340039b0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.466+0000 7f9f4e255640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f480d11b0 0x7f9f340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4da54640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f340039b0 0x7f9f34003dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009af0 tx=0x7f9f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e458ff238c6cf6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44036070 con 0x7f9f340039b0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f44031400 con 0x7f9f340039b0 2024-04-26T21:15:11.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4ca52640 1 -- 172.21.15.40:0/1193275600 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44009690 con 0x7f9f340039b0 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/1193275600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f340039b0 msgr2=0x7f9f34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/1193275600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f340039b0 0x7f9f34003dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009af0 tx=0x7f9f34005830 comp rx=0 tx=0).stop 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4e255640 1 -- 172.21.15.40:0/1193275600 reap_dead start 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/1193275600 shutdown_connections 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/1193275600 >> 172.21.15.40:0/1193275600 conn(0x7f9f480b4a60 msgr2=0x7f9f480b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.467+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/1193275600 shutdown_connections 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.468+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/1193275600 wait complete. 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 Processor -- start 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 -- start start 2024-04-26T21:15:11.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f340039b0 0x7f9f481f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f4815d910 0x7f9f481f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4806da10 con 0x7f9f480afd20 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f480d22c0 con 0x7f9f4815d910 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4fcb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9f480cf6c0 con 0x7f9f340039b0 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4e255640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.469+0000 7f9f4e255640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51014/0 (socket says 172.21.15.40:51014) 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 -- 172.21.15.40:0/189931051 learned_addr learned my addr 172.21.15.40:0/189931051 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4d253640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f4815d910 0x7f9f481f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4da54640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f340039b0 0x7f9f481f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 -- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f340039b0 msgr2=0x7f9f481f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f340039b0 0x7f9f481f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 -- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f4815d910 msgr2=0x7f9f481f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f4815d910 0x7f9f481f7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 -- 172.21.15.40:0/189931051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f480b08f0 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4e255640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9f40000f80 tx=0x7f9f4000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000eb90 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f40018070 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f48160fb0 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40013420 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.470+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f480b0e70 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.471+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f480b0710 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.471+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f400135c0 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.472+0000 7f9f3e7fc640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 0x7f9f2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.472+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f9f4008bc30 con 0x7f9f480afd20 2024-04-26T21:15:11.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.472+0000 7f9f4da54640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 0x7f9f2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.473+0000 7f9f4da54640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 0x7f9f2c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f44033000 tx=0x7f9f44008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.476+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f480b0710 con 0x7f9f480afd20 2024-04-26T21:15:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.663+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f4806a290 con 0x7f9f480afd20 2024-04-26T21:15:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.664+0000 7f9f3e7fc640 1 -- 172.21.15.40:0/189931051 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+12891 (secure 0 0 0) 0x7f9f4005a320 con 0x7f9f480afd20 2024-04-26T21:15:11.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:11.665 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:10.962711+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.666+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 msgr2=0x7f9f2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.666+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 0x7f9f2c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f44033000 tx=0x7f9f44008030 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 msgr2=0x7f9f480b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9f40000f80 tx=0x7f9f4000e530 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 shutdown_connections 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9f2c06c8f0 0x7f9f2c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f480afd20 0x7f9f480b01d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f4815d910 0x7f9f481f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 --2- 172.21.15.40:0/189931051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f340039b0 0x7f9f481f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 >> 172.21.15.40:0/189931051 conn(0x7f9f480b4a60 msgr2=0x7f9f480b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 shutdown_connections 2024-04-26T21:15:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.667+0000 7f9f4fcb8640 1 -- 172.21.15.40:0/189931051 wait complete. 2024-04-26T21:15:11.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:11.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:11.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.781+0000 7f6545e93640 1 Processor -- start 2024-04-26T21:15:11.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.781+0000 7f6545e93640 1 -- start start 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f6545e93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f65401631f0 0x7f65400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f6545e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f65400b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f6545e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654009c790 con 0x7f65400d1170 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f6545e93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f654006eec0 con 0x7f65401631f0 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f6545e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65400d15f0 con 0x7f654006ea30 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f653effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f65400d1170 0x7f65401635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39002/0 (socket says 172.21.15.40:39002) 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f653effd640 1 -- 172.21.15.40:0/1457933197 learned_addr learned my addr 172.21.15.40:0/1457933197 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f653ffff640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f65400b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1457933197 >> v2:172.21.15.184:3300/0 conn(0x7f65401631f0 0x7f65400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.782+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30678470 0 0) 0x7f654009c790 con 0x7f65400d1170 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6528003520 con 0x7f65400d1170 2024-04-26T21:15:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1559548295 0 0) 0x7f6528003520 con 0x7f65400d1170 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 msgr2=0x7f65400b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f65400b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 >> v2:172.21.15.184:3300/0 conn(0x7f65401631f0 msgr2=0x7f65400b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 --2- 172.21.15.40:0/1457933197 >> v2:172.21.15.184:3300/0 conn(0x7f65401631f0 0x7f65400b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6540163ec0 con 0x7f65400d1170 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f6534001670 con 0x7f65400d1170 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 >> v1:172.21.15.40:6789/0 conn(0x7f65400d1170 legacy=0x7f65401635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65280039e0 0x7f6528003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f6528004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6528004d40 0x7f65280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d15f0 con 0x7f65280039e0 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f652800b9c0 con 0x7f6528004d40 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f652800bb90 con 0x7f654006ea30 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65280039e0 0x7f6528003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653effd640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f6528004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653ffff640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6528004d40 0x7f65280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 -- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 msgr2=0x7f6528004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f654006ea30 0x7f6528004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 -- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6528004d40 msgr2=0x7f65280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6528004d40 0x7f65280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653f7fe640 1 -- 172.21.15.40:0/1457933197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6528003520 con 0x7f65280039e0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.783+0000 7f653ffff640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6528004d40 0x7f65280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.784+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65280039e0 0x7f6528003e00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6530009af0 tx=0x7f6528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078bfc62d0c7307 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.784+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530036070 con 0x7f65280039e0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.784+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6530031400 con 0x7f65280039e0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.784+0000 7f653e7fc640 1 -- 172.21.15.40:0/1457933197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530009690 con 0x7f65280039e0 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 -- 172.21.15.40:0/1457933197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65280039e0 msgr2=0x7f6528003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 --2- 172.21.15.40:0/1457933197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65280039e0 0x7f6528003e00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6530009af0 tx=0x7f6528005830 comp rx=0 tx=0).stop 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f653ffff640 1 -- 172.21.15.40:0/1457933197 reap_dead start 2024-04-26T21:15:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 -- 172.21.15.40:0/1457933197 shutdown_connections 2024-04-26T21:15:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 -- 172.21.15.40:0/1457933197 >> 172.21.15.40:0/1457933197 conn(0x7f654009ba40 msgr2=0x7f654009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 -- 172.21.15.40:0/1457933197 shutdown_connections 2024-04-26T21:15:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.785+0000 7f6545e93640 1 -- 172.21.15.40:0/1457933197 wait complete. 2024-04-26T21:15:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.786+0000 7f6545e93640 1 Processor -- start 2024-04-26T21:15:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.786+0000 7f6545e93640 1 -- start start 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 0x7f65400cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65400b0a00 0x7f65400b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d2000 con 0x7f65400ca180 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d1d20 con 0x7f65280039e0 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6545e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f65400d0230 con 0x7f65400b0a00 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51028/0 (socket says 172.21.15.40:51028) 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 -- 172.21.15.40:0/1029815776 learned_addr learned my addr 172.21.15.40:0/1029815776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 0x7f65400cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 -- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65400b0a00 msgr2=0x7f65400b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653ffff640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65400b0a00 0x7f65400b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65400b0a00 0x7f65400b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 -- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 msgr2=0x7f65400cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 0x7f65400cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 -- 172.21.15.40:0/1029815776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65401f4fe0 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 0x7f65400cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f653effd640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f653400e780 tx=0x7f653400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400c660 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6534018070 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.787+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400ca40 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.788+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65401f52c0 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.788+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401f5880 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.789+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654006a290 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.789+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f653400cbe0 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.789+0000 7f6527fff640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 0x7f652c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.790+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f653408b1f0 con 0x7f65400ca180 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.790+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 0x7f652c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.790+0000 7f653f7fe640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 0x7f652c06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6530033000 tx=0x7f6530008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:11.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.794+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6534059a00 con 0x7f65400ca180 2024-04-26T21:15:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.981+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65400b8f10 con 0x7f65400ca180 2024-04-26T21:15:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.981+0000 7f6527fff640 1 -- 172.21.15.40:0/1029815776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f6534059300 con 0x7f65400ca180 2024-04-26T21:15:11.982 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 msgr2=0x7f652c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 0x7f652c06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6530033000 tx=0x7f6530008030 comp rx=0 tx=0).stop 2024-04-26T21:15:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 msgr2=0x7f65400ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f653400e780 tx=0x7f653400ec40 comp rx=0 tx=0).stop 2024-04-26T21:15:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 shutdown_connections 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f652c06c920 0x7f652c06ede0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65400b0a00 0x7f65400b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65400ca180 0x7f65400ca600 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 --2- 172.21.15.40:0/1029815776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65280039e0 0x7f65400cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 >> 172.21.15.40:0/1029815776 conn(0x7f654009ba40 msgr2=0x7f65400711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 shutdown_connections 2024-04-26T21:15:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:11.983+0000 7f6545e93640 1 -- 172.21.15.40:0/1029815776 wait complete. 2024-04-26T21:15:11.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:15:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.096+0000 7f706d42a640 1 Processor -- start 2024-04-26T21:15:12.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.096+0000 7f706d42a640 1 -- start start 2024-04-26T21:15:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f706d42a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f706806e490 0x7f706815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f706d42a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680d1740 con 0x7f70680cf2e0 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f706d42a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680d1a20 con 0x7f70680d11b0 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f706d42a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f70680d1d00 con 0x7f706806e490 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f70677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f706806e490 0x7f706815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f7066ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f70680cf2e0 0x7f706806d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39008/0 (socket says 172.21.15.40:39008) 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f7066ffd640 1 -- 172.21.15.40:0/675817559 learned_addr learned my addr 172.21.15.40:0/675817559 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f70677fe640 1 -- 172.21.15.40:0/675817559 >> v1:172.21.15.184:6789/0 conn(0x7f70680d11b0 legacy=0x7f706806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f70677fe640 1 -- 172.21.15.40:0/675817559 >> v1:172.21.15.40:6789/0 conn(0x7f70680cf2e0 legacy=0x7f706806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.097+0000 7f70677fe640 1 -- 172.21.15.40:0/675817559 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680d1fe0 con 0x7f706806e490 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.098+0000 7f70677fe640 1 --2- 172.21.15.40:0/675817559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f706815ebe0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7058009dc0 tx=0x7f705802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3033bf4e6dbb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.098+0000 7f7065ffb640 1 -- 172.21.15.40:0/675817559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058007740 con 0x7f706806e490 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.098+0000 7f7065ffb640 1 -- 172.21.15.40:0/675817559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70580078e0 con 0x7f706806e490 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.098+0000 7f7065ffb640 1 -- 172.21.15.40:0/675817559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058007cb0 con 0x7f706806e490 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 -- 172.21.15.40:0/675817559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 msgr2=0x7f706815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 --2- 172.21.15.40:0/675817559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f706815ebe0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7058009dc0 tx=0x7f705802a730 comp rx=0 tx=0).stop 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 -- 172.21.15.40:0/675817559 shutdown_connections 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 --2- 172.21.15.40:0/675817559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f706815ebe0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 -- 172.21.15.40:0/675817559 >> 172.21.15.40:0/675817559 conn(0x7f70680b4a60 msgr2=0x7f70680b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 -- 172.21.15.40:0/675817559 shutdown_connections 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.099+0000 7f706d42a640 1 -- 172.21.15.40:0/675817559 wait complete. 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.100+0000 7f706d42a640 1 Processor -- start 2024-04-26T21:15:12.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.100+0000 7f706d42a640 1 -- start start 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 0x7f706815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70680d11b0 0x7f70680b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70680cf840 con 0x7f706806e490 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70680cf6c0 con 0x7f70680cf2e0 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f706d42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70680ce660 con 0x7f70680d11b0 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51034/0 (socket says 172.21.15.40:51034) 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 -- 172.21.15.40:0/3415329410 learned_addr learned my addr 172.21.15.40:0/3415329410 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f70677fe640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70680d11b0 0x7f70680b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f70667fc640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 0x7f706815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 -- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70680d11b0 msgr2=0x7f70680b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70680d11b0 0x7f70680b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 -- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 msgr2=0x7f706815a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 0x7f706815a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 -- 172.21.15.40:0/3415329410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680b5cd0 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f70667fc640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 0x7f706815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f7066ffd640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f706000c4a0 tx=0x7f706000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.101+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060017070 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.102+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7060005ce0 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.102+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70680b5f70 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.102+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007270 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.102+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70680b6530 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.103+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706806a290 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.103+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f706001a070 con 0x7f706806e490 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.104+0000 7f705f7fe640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 0x7f704806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.104+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f706008aa20 con 0x7f706806e490 2024-04-26T21:15:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.104+0000 7f70667fc640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 0x7f704806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.104+0000 7f70667fc640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 0x7f704806ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f706809bf50 tx=0x7f7050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.108+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7060058b40 con 0x7f706806e490 2024-04-26T21:15:12.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.295+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f70680c9e00 con 0x7f706806e490 2024-04-26T21:15:12.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.295+0000 7f705f7fe640 1 -- 172.21.15.40:0/3415329410 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7f706005ce40 con 0x7f706806e490 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:12.297 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 msgr2=0x7f704806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 0x7f704806ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f706809bf50 tx=0x7f7050009040 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 msgr2=0x7f70681f9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f706000c4a0 tx=0x7f706000c960 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 shutdown_connections 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f704806c920 0x7f704806ede0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70680d11b0 0x7f70680b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70680cf2e0 0x7f706815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 --2- 172.21.15.40:0/3415329410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f706806e490 0x7f70681f9b90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 >> 172.21.15.40:0/3415329410 conn(0x7f70680b4a60 msgr2=0x7f706806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 shutdown_connections 2024-04-26T21:15:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.297+0000 7f706d42a640 1 -- 172.21.15.40:0/3415329410 wait complete. 2024-04-26T21:15:12.309 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:15:12.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:15:12.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.413+0000 7fd07c9b4640 1 Processor -- start 2024-04-26T21:15:12.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.413+0000 7fd07c9b4640 1 -- start start 2024-04-26T21:15:12.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07c9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0781437f0 0x7fd07806f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07c9b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0780d1190 0x7fd0780d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07c9b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0781612b0 con 0x7fd078160e70 2024-04-26T21:15:12.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07c9b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0780d1b30 con 0x7fd0780d1190 2024-04-26T21:15:12.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07c9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd078073440 con 0x7fd0781437f0 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0780d1190 0x7fd0780d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0780d1190 0x7fd0780d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51036/0 (socket says 172.21.15.40:51036) 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 -- 172.21.15.40:0/1348584981 learned_addr learned my addr 172.21.15.40:0/1348584981 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd07659c640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0781437f0 0x7fd07806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 -- 172.21.15.40:0/1348584981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0781437f0 msgr2=0x7fd07806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0781437f0 0x7fd07806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 -- 172.21.15.40:0/1348584981 >> v1:172.21.15.184:6789/0 conn(0x7fd078160e70 legacy=0x7fd078144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.414+0000 7fd076d9d640 1 -- 172.21.15.40:0/1348584981 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078161570 con 0x7fd0780d1190 2024-04-26T21:15:12.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.415+0000 7fd076d9d640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0780d1190 0x7fd0780d15f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd070009870 tx=0x7fd07002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcc92fbd85b27aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.415+0000 7fd07559a640 1 -- 172.21.15.40:0/1348584981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070037070 con 0x7fd0780d1190 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.415+0000 7fd07559a640 1 -- 172.21.15.40:0/1348584981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070032440 con 0x7fd0780d1190 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.415+0000 7fd07559a640 1 -- 172.21.15.40:0/1348584981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070032810 con 0x7fd0780d1190 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1348584981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0780d1190 msgr2=0x7fd0780d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0780d1190 0x7fd0780d15f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd070009870 tx=0x7fd07002a210 comp rx=0 tx=0).stop 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1348584981 shutdown_connections 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0780d1190 0x7fd0780d15f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1348584981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0781437f0 0x7fd07806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1348584981 >> 172.21.15.40:0/1348584981 conn(0x7fd07806c690 msgr2=0x7fd07806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1348584981 shutdown_connections 2024-04-26T21:15:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.416+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1348584981 wait complete. 2024-04-26T21:15:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.417+0000 7fd07c9b4640 1 Processor -- start 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.417+0000 7fd07c9b4640 1 -- start start 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0780d1190 0x7fd0780b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 0x7fd0781e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780736a0 con 0x7fd0781437f0 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd078161b30 con 0x7fd078160e70 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07c9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd078143bd0 con 0x7fd0780d1190 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51046/0 (socket says 172.21.15.40:51046) 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 -- 172.21.15.40:0/1689319115 learned_addr learned my addr 172.21.15.40:0/1689319115 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd07659c640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0780d1190 0x7fd0780b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 -- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0780d1190 msgr2=0x7fd0780b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0780d1190 0x7fd0780b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 -- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 msgr2=0x7fd0781e79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd076d9d640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 0x7fd0781e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 0x7fd0781e79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd075d9b640 1 -- 172.21.15.40:0/1689319115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0780b4b00 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.418+0000 7fd076d9d640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 0x7fd0781e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd075d9b640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd06000b410 tx=0x7fd06000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060015020 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060011830 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0780b4de0 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060011c20 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.419+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0780b53a0 con 0x7fd0781437f0 2024-04-26T21:15:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.420+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07806a290 con 0x7fd0781437f0 2024-04-26T21:15:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.420+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd060018020 con 0x7fd0781437f0 2024-04-26T21:15:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.421+0000 7fd06effd640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.421+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fd06008b720 con 0x7fd0781437f0 2024-04-26T21:15:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.421+0000 7fd07659c640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.422+0000 7fd07659c640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 0x7fd05806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd068009f30 tx=0x7fd06800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.425+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd06001f020 con 0x7fd0781437f0 2024-04-26T21:15:12.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.611+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd07806eb80 con 0x7fd0781437f0 2024-04-26T21:15:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.968+0000 7fd06effd640 1 -- 172.21.15.40:0/1689319115 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7fd0600172c0 con 0x7fd0781437f0 2024-04-26T21:15:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 msgr2=0x7fd05806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 0x7fd05806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd068009f30 tx=0x7fd06800b040 comp rx=0 tx=0).stop 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 msgr2=0x7fd078141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd06000b410 tx=0x7fd06000b8d0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 shutdown_connections 2024-04-26T21:15:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd05806c8f0 0x7fd05806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd078160e70 0x7fd0781e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781437f0 0x7fd078141d90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 --2- 172.21.15.40:0/1689319115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0780d1190 0x7fd0780b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 >> 172.21.15.40:0/1689319115 conn(0x7fd07806c690 msgr2=0x7fd07815da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.971+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 shutdown_connections 2024-04-26T21:15:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:12.972+0000 7fd07c9b4640 1 -- 172.21.15.40:0/1689319115 wait complete. 2024-04-26T21:15:12.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:15:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.094+0000 7f203faf9640 1 Processor -- start 2024-04-26T21:15:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.094+0000 7f203faf9640 1 -- start start 2024-04-26T21:15:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203faf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203faf9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 0x7f203806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203faf9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20380b46f0 con 0x7f20380b3ac0 2024-04-26T21:15:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203faf9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f203806d3b0 con 0x7f203809c150 2024-04-26T21:15:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203faf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20380d1f60 con 0x7f20380b3620 2024-04-26T21:15:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203d895640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203e096640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 0x7f203806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203d895640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51752/0 (socket says 172.21.15.40:51752) 2024-04-26T21:15:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.095+0000 7f203d895640 1 -- 172.21.15.40:0/2305339296 learned_addr learned my addr 172.21.15.40:0/2305339296 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:13.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.096+0000 7f203d895640 1 -- 172.21.15.40:0/2305339296 >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 msgr2=0x7f203806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:13.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.096+0000 7f203d895640 1 --2- 172.21.15.40:0/2305339296 >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 0x7f203806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.096+0000 7f203d895640 1 -- 172.21.15.40:0/2305339296 >> v1:172.21.15.184:6789/0 conn(0x7f20380b3ac0 legacy=0x7f203809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.096+0000 7f203d895640 1 -- 172.21.15.40:0/2305339296 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380d2240 con 0x7f20380b3620 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.096+0000 7f203e096640 1 --2- 172.21.15.40:0/2305339296 >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 0x7f203806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203d895640 1 --2- 172.21.15.40:0/2305339296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2034009e50 tx=0x7f203402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da084c30456647a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203c893640 1 -- 172.21.15.40:0/2305339296 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034036070 con 0x7f20380b3620 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203c893640 1 -- 172.21.15.40:0/2305339296 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034031600 con 0x7f20380b3620 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203c893640 1 -- 172.21.15.40:0/2305339296 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20340319d0 con 0x7f20380b3620 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 -- 172.21.15.40:0/2305339296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 msgr2=0x7f2038160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 --2- 172.21.15.40:0/2305339296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2034009e50 tx=0x7f203402a710 comp rx=0 tx=0).stop 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 -- 172.21.15.40:0/2305339296 shutdown_connections 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 --2- 172.21.15.40:0/2305339296 >> v2:172.21.15.184:3300/0 conn(0x7f203809c150 0x7f203806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 --2- 172.21.15.40:0/2305339296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20380b3620 0x7f2038160f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 -- 172.21.15.40:0/2305339296 >> 172.21.15.40:0/2305339296 conn(0x7f20380b4ab0 msgr2=0x7f20380b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 -- 172.21.15.40:0/2305339296 shutdown_connections 2024-04-26T21:15:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.097+0000 7f203faf9640 1 -- 172.21.15.40:0/2305339296 wait complete. 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.098+0000 7f203faf9640 1 Processor -- start 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 -- start start 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f203809c150 0x7f203815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 0x7f2038070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20380b3ff0 con 0x7f20380b3620 2024-04-26T21:15:13.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f203806d600 con 0x7f20380b3ac0 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203faf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20380b4170 con 0x7f203809c150 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203d094640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203d094640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51062/0 (socket says 172.21.15.40:51062) 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.099+0000 7f203d094640 1 -- 172.21.15.40:0/2553081557 learned_addr learned my addr 172.21.15.40:0/2553081557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d895640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f203809c150 0x7f203815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203e096640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 0x7f2038070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 -- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f203809c150 msgr2=0x7f203815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f203809c150 0x7f203815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 -- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 msgr2=0x7f2038070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 0x7f2038070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 -- 172.21.15.40:0/2553081557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203809f8a0 con 0x7f20380b3620 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203e096640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 0x7f2038070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203d094640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f202800c470 tx=0x7f2028001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280052d0 con 0x7f20380b3620 2024-04-26T21:15:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2028005470 con 0x7f20380b3620 2024-04-26T21:15:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028007260 con 0x7f20380b3620 2024-04-26T21:15:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20380b0fe0 con 0x7f20380b3620 2024-04-26T21:15:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.100+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381f94d0 con 0x7f20380b3620 2024-04-26T21:15:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.101+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203806a290 con 0x7f20380b3620 2024-04-26T21:15:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.102+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2028007400 con 0x7f20380b3620 2024-04-26T21:15:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.102+0000 7f202e7fc640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 0x7f201c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.102+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5227+0+0 (secure 0 0 0) 0x7f202808aed0 con 0x7f20380b3620 2024-04-26T21:15:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.102+0000 7f203d895640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 0x7f201c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.103+0000 7f203d895640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 0x7f201c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2034033000 tx=0x7f2034009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:13.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.107+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20380b46f0 con 0x7f20380b3620 2024-04-26T21:15:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.293+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20380745c0 con 0x7f20380b3620 2024-04-26T21:15:13.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.974+0000 7f202e7fc640 1 -- 172.21.15.40:0/2553081557 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f20280594e0 con 0x7f20380b3620 2024-04-26T21:15:13.975 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:15:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.979+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 msgr2=0x7f201c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.979+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 0x7f201c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2034033000 tx=0x7f2034009000 comp rx=0 tx=0).stop 2024-04-26T21:15:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.979+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 msgr2=0x7f20380707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.979+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f202800c470 tx=0x7f2028001e70 comp rx=0 tx=0).stop 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 shutdown_connections 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f201c06c8f0 0x7f201c06edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20380b3ac0 0x7f2038070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20380b3620 0x7f20380707c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 --2- 172.21.15.40:0/2553081557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f203809c150 0x7f203815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.985+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 >> 172.21.15.40:0/2553081557 conn(0x7f20380b4ab0 msgr2=0x7f203809ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.986+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 shutdown_connections 2024-04-26T21:15:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:13.987+0000 7f203faf9640 1 -- 172.21.15.40:0/2553081557 wait complete. 2024-04-26T21:15:14.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:15:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.114+0000 7f1b10197640 1 Processor -- start 2024-04-26T21:15:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.115+0000 7f1b10197640 1 -- start start 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.115+0000 7f1b10197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0815f490 0x7f1b080d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.115+0000 7f1b10197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b080d0ff0 0x7f1b080d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b10197640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b080d2120 0x7f1b080ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b10197640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080d2570 con 0x7f1b080d0ff0 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b10197640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080d4240 con 0x7f1b080d2120 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b10197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b080d31a0 con 0x7f1b0815f490 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b080d0ff0 0x7f1b080d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b080d0ff0 0x7f1b080d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51066/0 (socket says 172.21.15.40:51066) 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 -- 172.21.15.40:0/999595527 learned_addr learned my addr 172.21.15.40:0/999595527 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0df33640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0815f490 0x7f1b080d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 -- 172.21.15.40:0/999595527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0815f490 msgr2=0x7f1b080d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0e734640 1 --2- 172.21.15.40:0/999595527 >> v2:172.21.15.184:3300/0 conn(0x7f1b080d2120 0x7f1b080ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0815f490 0x7f1b080d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 -- 172.21.15.40:0/999595527 >> v2:172.21.15.184:3300/0 conn(0x7f1b080d2120 msgr2=0x7f1b080ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 --2- 172.21.15.40:0/999595527 >> v2:172.21.15.184:3300/0 conn(0x7f1b080d2120 0x7f1b080ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.116+0000 7f1b0d732640 1 -- 172.21.15.40:0/999595527 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080d3480 con 0x7f1b080d0ff0 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b0d732640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b080d0ff0 0x7f1b080d1be0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1b0808d6f0 tx=0x7f1af802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ce2f633ddbab48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b0cf31640 1 -- 172.21.15.40:0/999595527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8037070 con 0x7f1b080d0ff0 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b0cf31640 1 -- 172.21.15.40:0/999595527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af8032440 con 0x7f1b080d0ff0 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b0cf31640 1 -- 172.21.15.40:0/999595527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8032810 con 0x7f1b080d0ff0 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b10197640 1 -- 172.21.15.40:0/999595527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b080d0ff0 msgr2=0x7f1b080d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.117+0000 7f1b10197640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b080d0ff0 0x7f1b080d1be0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1b0808d6f0 tx=0x7f1af802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 -- 172.21.15.40:0/999595527 shutdown_connections 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 --2- 172.21.15.40:0/999595527 >> v2:172.21.15.184:3300/0 conn(0x7f1b080d2120 0x7f1b080ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b080d0ff0 0x7f1b080d1be0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 --2- 172.21.15.40:0/999595527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0815f490 0x7f1b080d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 -- 172.21.15.40:0/999595527 >> 172.21.15.40:0/999595527 conn(0x7f1b0809e530 msgr2=0x7f1b080d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 -- 172.21.15.40:0/999595527 shutdown_connections 2024-04-26T21:15:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.118+0000 7f1b10197640 1 -- 172.21.15.40:0/999595527 wait complete. 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.119+0000 7f1b10197640 1 Processor -- start 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.119+0000 7f1b10197640 1 -- start start 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b080d0ff0 0x7f1b080cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b080d2120 0x7f1b080ba550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080d3d20 con 0x7f1b0815f490 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080d3a40 con 0x7f1b080d0ff0 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0815f870 con 0x7f1b080d2120 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51072/0 (socket says 172.21.15.40:51072) 2024-04-26T21:15:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 -- 172.21.15.40:0/3834364751 learned_addr learned my addr 172.21.15.40:0/3834364751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0d732640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b080d2120 0x7f1b080ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0df33640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b080d0ff0 0x7f1b080cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 -- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b080d2120 msgr2=0x7f1b080ba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b080d2120 0x7f1b080ba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 -- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b080d0ff0 msgr2=0x7f1b080cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b080d0ff0 0x7f1b080cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 -- 172.21.15.40:0/3834364751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0815bd20 con 0x7f1b0815f490 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b0e734640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000cc60 tx=0x7f1b0000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00003990 con 0x7f1b0815f490 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b00003b30 con 0x7f1b0815f490 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00004020 con 0x7f1b0815f490 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.120+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b080b94b0 con 0x7f1b0815f490 2024-04-26T21:15:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.121+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b080b9a70 con 0x7f1b0815f490 2024-04-26T21:15:14.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.121+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0806a290 con 0x7f1b0815f490 2024-04-26T21:15:14.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.122+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b000041c0 con 0x7f1b0815f490 2024-04-26T21:15:14.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.122+0000 7f1afe7fc640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:14.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.122+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5609+0+0 (secure 0 0 0) 0x7f1b0008bf50 con 0x7f1b0815f490 2024-04-26T21:15:14.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.123+0000 7f1b0df33640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:14.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.123+0000 7f1b0df33640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 0x7f1aec06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400a8f0 tx=0x7f1b04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:14.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.127+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b00014030 con 0x7f1b0815f490 2024-04-26T21:15:14.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.353+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1b081614b0 con 0x7f1b0815f490 2024-04-26T21:15:14.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.975+0000 7f1afe7fc640 1 -- 172.21.15.40:0/3834364751 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v34) ==== 174+0+0 (secure 0 0 0) 0x7f1b0005a470 con 0x7f1b0815f490 2024-04-26T21:15:14.977 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 10 and data pool 11 2024-04-26T21:15:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.981+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 msgr2=0x7f1aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.981+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 0x7f1aec06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400a8f0 tx=0x7f1b04008040 comp rx=0 tx=0).stop 2024-04-26T21:15:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.981+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 msgr2=0x7f1b080cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:14.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.981+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000cc60 tx=0x7f1b0000a5c0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 shutdown_connections 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1aec06c8f0 0x7f1aec06edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b0815f490 0x7f1b080cf110 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b080d2120 0x7f1b080ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 --2- 172.21.15.40:0/3834364751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b080d0ff0 0x7f1b080cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.982+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 >> 172.21.15.40:0/3834364751 conn(0x7f1b0809e530 msgr2=0x7f1b0806e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.983+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 shutdown_connections 2024-04-26T21:15:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:14.983+0000 7f1b10197640 1 -- 172.21.15.40:0/3834364751 wait complete. 2024-04-26T21:15:15.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 Processor -- start 2024-04-26T21:15:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 -- start start 2024-04-26T21:15:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f355c0cf2e0 0x7f355c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f355c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f355c0d1740 con 0x7f355c0d11b0 2024-04-26T21:15:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f355c0d1a20 con 0x7f355c0cf2e0 2024-04-26T21:15:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.116+0000 7f3561311640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f355c0d1d00 con 0x7f355c06e490 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f355c0d11b0 0x7f355c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39024/0 (socket says 172.21.15.40:39024) 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355a7fc640 1 -- 172.21.15.40:0/3393647691 learned_addr learned my addr 172.21.15.40:0/3393647691 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813452069 0 0) 0x7f355c0d1740 con 0x7f355c0d11b0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355affd640 1 --2- 172.21.15.40:0/3393647691 >> v2:172.21.15.184:3300/0 conn(0x7f355c0cf2e0 0x7f355c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355b7fe640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f355c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3544003520 con 0x7f355c0d11b0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3255536284 0 0) 0x7f3544003520 con 0x7f355c0d11b0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 msgr2=0x7f355c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f355c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 >> v2:172.21.15.184:3300/0 conn(0x7f355c0cf2e0 msgr2=0x7f355c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 --2- 172.21.15.40:0/3393647691 >> v2:172.21.15.184:3300/0 conn(0x7f355c0cf2e0 0x7f355c06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c0d1fe0 con 0x7f355c0d11b0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3550001670 con 0x7f355c0d11b0 2024-04-26T21:15:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 >> v1:172.21.15.40:6789/0 conn(0x7f355c0d11b0 legacy=0x7f355c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f3544004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3544004d40 0x7f35440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0d1a20 con 0x7f35440039b0 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0d1d00 con 0x7f3544004d40 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f354400b9c0 con 0x7f355c06e490 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355affd640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.117+0000 7f355a7fc640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f3544004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355b7fe640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3544004d40 0x7f35440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 -- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 msgr2=0x7f3544004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f355c06e490 0x7f3544004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 -- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3544004d40 msgr2=0x7f35440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3544004d40 0x7f35440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 -- 172.21.15.40:0/3393647691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3544003520 con 0x7f35440039b0 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.118+0000 7f355affd640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f354c009ac0 tx=0x7f3544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e319189f7d00d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c036070 con 0x7f35440039b0 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c031400 con 0x7f35440039b0 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3559ffb640 1 -- 172.21.15.40:0/3393647691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c009690 con 0x7f35440039b0 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 -- 172.21.15.40:0/3393647691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35440039b0 msgr2=0x7f3544003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 --2- 172.21.15.40:0/3393647691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f354c009ac0 tx=0x7f3544005830 comp rx=0 tx=0).stop 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f355b7fe640 1 -- 172.21.15.40:0/3393647691 reap_dead start 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 -- 172.21.15.40:0/3393647691 shutdown_connections 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 -- 172.21.15.40:0/3393647691 >> 172.21.15.40:0/3393647691 conn(0x7f355c0b4a60 msgr2=0x7f355c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 -- 172.21.15.40:0/3393647691 shutdown_connections 2024-04-26T21:15:15.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.119+0000 7f3561311640 1 -- 172.21.15.40:0/3393647691 wait complete. 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 Processor -- start 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 -- start start 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35440039b0 0x7f355c1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f355c0a0420 0x7f355c0a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0cf6c0 con 0x7f355c09e430 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f355c06da10 con 0x7f355c0a0420 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f3561311640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f355c0cf840 con 0x7f35440039b0 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51096/0 (socket says 172.21.15.40:51096) 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.121+0000 7f355a7fc640 1 -- 172.21.15.40:0/1519708897 learned_addr learned my addr 172.21.15.40:0/1519708897 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355affd640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35440039b0 0x7f355c1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355b7fe640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f355c0a0420 0x7f355c0a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 -- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35440039b0 msgr2=0x7f355c1f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35440039b0 0x7f355c1f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 -- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f355c0a0420 msgr2=0x7f355c0a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f355c0a0420 0x7f355c0a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 -- 172.21.15.40:0/1519708897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1f6020 con 0x7f355c09e430 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f355a7fc640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f35500072c0 tx=0x7f3550007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355000f8e0 con 0x7f355c09e430 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3550018070 con 0x7f355c09e430 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355000fc80 con 0x7f355c09e430 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c160a00 con 0x7f355c09e430 2024-04-26T21:15:15.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.122+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c09ecb0 con 0x7f355c09e430 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.123+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c06a290 con 0x7f355c09e430 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.124+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f355001d460 con 0x7f355c09e430 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.124+0000 7f35437fe640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.124+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f355008bb00 con 0x7f355c09e430 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.124+0000 7f355affd640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.125+0000 7f355affd640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f354c033000 tx=0x7f354c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.128+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f355c1f6020 con 0x7f355c09e430 2024-04-26T21:15:15.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.315+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f355c0b8ef0 con 0x7f355c09e430 2024-04-26T21:15:15.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.316+0000 7f35437fe640 1 -- 172.21.15.40:0/1519708897 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16681 (secure 0 0 0) 0x7f3550059fc0 con 0x7f355c09e430 2024-04-26T21:15:15.317 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:15.317 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":69,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:14.354358+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 msgr2=0x7f354806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f354c033000 tx=0x7f354c008030 comp rx=0 tx=0).stop 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 msgr2=0x7f355c1604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f35500072c0 tx=0x7f3550007780 comp rx=0 tx=0).stop 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 shutdown_connections 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f355c0a0420 0x7f355c0a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f355c09e430 0x7f355c1604c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 --2- 172.21.15.40:0/1519708897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35440039b0 0x7f355c1f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 >> 172.21.15.40:0/1519708897 conn(0x7f355c0b4a60 msgr2=0x7f355c0705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:15.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 shutdown_connections 2024-04-26T21:15:15.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.318+0000 7f3561311640 1 -- 172.21.15.40:0/1519708897 wait complete. 2024-04-26T21:15:15.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.431+0000 7eff6b645640 1 Processor -- start 2024-04-26T21:15:15.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 -- start start 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff640d3ee0 0x7eff640cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff6415f750 0x7eff640b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff640d03a0 con 0x7eff6415f370 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff640b4d10 con 0x7eff6415f750 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.432+0000 7eff6b645640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7eff640d0680 con 0x7eff640d3ee0 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff69be2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff6415f750 0x7eff640b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff693e1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff6415f370 0x7eff640d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39038/0 (socket says 172.21.15.40:39038) 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff693e1640 1 -- 172.21.15.40:0/2882399257 learned_addr learned my addr 172.21.15.40:0/2882399257 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff68be0640 1 --2- 172.21.15.40:0/2882399257 >> v2:172.21.15.184:3300/0 conn(0x7eff640d3ee0 0x7eff640cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff69be2640 1 -- 172.21.15.40:0/2882399257 >> v2:172.21.15.184:3300/0 conn(0x7eff640d3ee0 msgr2=0x7eff640cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff69be2640 1 --2- 172.21.15.40:0/2882399257 >> v2:172.21.15.184:3300/0 conn(0x7eff640d3ee0 0x7eff640cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff69be2640 1 -- 172.21.15.40:0/2882399257 >> v1:172.21.15.40:6789/0 conn(0x7eff6415f370 legacy=0x7eff640d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff5bfff640 1 -- 172.21.15.40:0/2882399257 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714015738 0 0) 0x7eff640d03a0 con 0x7eff6415f370 2024-04-26T21:15:15.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.433+0000 7eff69be2640 1 -- 172.21.15.40:0/2882399257 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff640d0960 con 0x7eff6415f750 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff69be2640 1 --2- 172.21.15.40:0/2882399257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f750 0x7eff640b47d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009d90 tx=0x7eff5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78617c72af1c634 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff5bfff640 1 -- 172.21.15.40:0/2882399257 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007740 con 0x7eff6415f750 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff5bfff640 1 -- 172.21.15.40:0/2882399257 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5c0078e0 con 0x7eff6415f750 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff5bfff640 1 -- 172.21.15.40:0/2882399257 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007cb0 con 0x7eff6415f750 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 -- 172.21.15.40:0/2882399257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f750 msgr2=0x7eff640b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 --2- 172.21.15.40:0/2882399257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f750 0x7eff640b47d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009d90 tx=0x7eff5c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 -- 172.21.15.40:0/2882399257 shutdown_connections 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 --2- 172.21.15.40:0/2882399257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f750 0x7eff640b47d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 --2- 172.21.15.40:0/2882399257 >> v2:172.21.15.184:3300/0 conn(0x7eff640d3ee0 0x7eff640cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.434+0000 7eff6b645640 1 -- 172.21.15.40:0/2882399257 >> 172.21.15.40:0/2882399257 conn(0x7eff6408ec10 msgr2=0x7eff6409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.435+0000 7eff6b645640 1 -- 172.21.15.40:0/2882399257 shutdown_connections 2024-04-26T21:15:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.435+0000 7eff6b645640 1 -- 172.21.15.40:0/2882399257 wait complete. 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 Processor -- start 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 -- start start 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 0x7eff640b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff6415f750 0x7eff640b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff640b51c0 con 0x7eff6415f370 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff640d0f20 con 0x7eff640d3ee0 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.436+0000 7eff6b645640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff640b5340 con 0x7eff6415f750 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51122/0 (socket says 172.21.15.40:51122) 2024-04-26T21:15:15.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 -- 172.21.15.40:0/1603262876 learned_addr learned my addr 172.21.15.40:0/1603262876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff693e1640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 0x7eff640b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff69be2640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff6415f750 0x7eff640b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 -- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff6415f750 msgr2=0x7eff640b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff6415f750 0x7eff640b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 -- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 msgr2=0x7eff640b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 0x7eff640b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 -- 172.21.15.40:0/1603262876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff641f50a0 con 0x7eff6415f370 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff693e1640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 0x7eff640b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff68be0640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d800 tx=0x7eff5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400b6c0 con 0x7eff6415f370 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff54019070 con 0x7eff6415f370 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff64161e50 con 0x7eff6415f370 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400baa0 con 0x7eff6415f370 2024-04-26T21:15:15.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.437+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff64162410 con 0x7eff6415f370 2024-04-26T21:15:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.439+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6406a290 con 0x7eff6415f370 2024-04-26T21:15:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.439+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff5400bc40 con 0x7eff6415f370 2024-04-26T21:15:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.439+0000 7eff59ffb640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 0x7eff4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.439+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7eff5408bf40 con 0x7eff6415f370 2024-04-26T21:15:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.440+0000 7eff693e1640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 0x7eff4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.440+0000 7eff693e1640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 0x7eff4806ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff60000f60 tx=0x7eff60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.443+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff5405a370 con 0x7eff6415f370 2024-04-26T21:15:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.635+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff640d4cc0 con 0x7eff6415f370 2024-04-26T21:15:15.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.637+0000 7eff59ffb640 1 -- 172.21.15.40:0/1603262876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16681 (secure 0 0 0) 0x7eff54060030 con 0x7eff6415f370 2024-04-26T21:15:15.638 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:15.638 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":69,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:14.354358+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 msgr2=0x7eff4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 0x7eff4806ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff60000f60 tx=0x7eff60009040 comp rx=0 tx=0).stop 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 msgr2=0x7eff640b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d800 tx=0x7eff5400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 shutdown_connections 2024-04-26T21:15:15.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4806c920 0x7eff4806ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff6415f750 0x7eff640b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff6415f370 0x7eff640b60a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 --2- 172.21.15.40:0/1603262876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff640d3ee0 0x7eff640b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 >> 172.21.15.40:0/1603262876 conn(0x7eff6408ec10 msgr2=0x7eff640d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 shutdown_connections 2024-04-26T21:15:15.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.639+0000 7eff6b645640 1 -- 172.21.15.40:0/1603262876 wait complete. 2024-04-26T21:15:15.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:15:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.754+0000 7f061107e640 1 Processor -- start 2024-04-26T21:15:15.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.754+0000 7f061107e640 1 -- start start 2024-04-26T21:15:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f061107e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0d0010 0x7f060c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f061107e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f060c0b33e0 0x7f060c163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f061107e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c09c700 con 0x7f060c0b3c20 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f061107e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f060c0b4480 con 0x7f060c0b33e0 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f061107e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f060c06e050 con 0x7f060c0d0010 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060b59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f060c0b33e0 0x7f060c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f0603fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f060c0b3c20 0x7f060c0b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39052/0 (socket says 172.21.15.40:39052) 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f0603fff640 1 -- 172.21.15.40:0/475942008 learned_addr learned my addr 172.21.15.40:0/475942008 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060ad9d640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0d0010 0x7f060c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060b59e640 1 -- 172.21.15.40:0/475942008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0d0010 msgr2=0x7f060c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060b59e640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0d0010 0x7f060c06db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060b59e640 1 -- 172.21.15.40:0/475942008 >> v1:172.21.15.40:6789/0 conn(0x7f060c0b3c20 legacy=0x7f060c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060a59c640 1 -- 172.21.15.40:0/475942008 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219049508 0 0) 0x7f060c09c700 con 0x7f060c0b3c20 2024-04-26T21:15:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.755+0000 7f060b59e640 1 -- 172.21.15.40:0/475942008 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c0d1c50 con 0x7f060c0b33e0 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.756+0000 7f060b59e640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b33e0 0x7f060c163280 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0604009d90 tx=0x7f060402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2f964cc9ae4ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.756+0000 7f060a59c640 1 -- 172.21.15.40:0/475942008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604007740 con 0x7f060c0b33e0 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.756+0000 7f060a59c640 1 -- 172.21.15.40:0/475942008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06040078e0 con 0x7f060c0b33e0 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.756+0000 7f060a59c640 1 -- 172.21.15.40:0/475942008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604007cb0 con 0x7f060c0b33e0 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 -- 172.21.15.40:0/475942008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b33e0 msgr2=0x7f060c163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b33e0 0x7f060c163280 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0604009d90 tx=0x7f060402a660 comp rx=0 tx=0).stop 2024-04-26T21:15:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 -- 172.21.15.40:0/475942008 shutdown_connections 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b33e0 0x7f060c163280 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 --2- 172.21.15.40:0/475942008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0d0010 0x7f060c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 -- 172.21.15.40:0/475942008 >> 172.21.15.40:0/475942008 conn(0x7f060c09ba40 msgr2=0x7f060c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 -- 172.21.15.40:0/475942008 shutdown_connections 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.757+0000 7f061107e640 1 -- 172.21.15.40:0/475942008 wait complete. 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.758+0000 7f061107e640 1 Processor -- start 2024-04-26T21:15:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.758+0000 7f061107e640 1 -- start start 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0b33e0 0x7f060c1f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 0x7f060c0a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f060c0b39f0 con 0x7f060c0b3c20 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f060c0b3870 con 0x7f060c0d0010 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f061107e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f060c1637c0 con 0x7f060c0b33e0 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51142/0 (socket says 172.21.15.40:51142) 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 -- 172.21.15.40:0/134147521 learned_addr learned my addr 172.21.15.40:0/134147521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f060ad9d640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0b33e0 0x7f060c1f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f060b59e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 0x7f060c0a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 -- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0b33e0 msgr2=0x7f060c1f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0b33e0 0x7f060c1f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 -- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 msgr2=0x7f060c0a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 0x7f060c0a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f0603fff640 1 -- 172.21.15.40:0/134147521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c1f95c0 con 0x7f060c0b3c20 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.759+0000 7f060b59e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 0x7f060c0a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f0603fff640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c4d0 tx=0x7f05f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4018070 con 0x7f060c0b3c20 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4007500 con 0x7f060c0b3c20 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c0b64f0 con 0x7f060c0b3c20 2024-04-26T21:15:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004a20 con 0x7f060c0b3c20 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.760+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c0b6ab0 con 0x7f060c0b3c20 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.761+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c06a290 con 0x7f060c0b3c20 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.761+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05f401b070 con 0x7f060c0b3c20 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.762+0000 7f06037fe640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 0x7f05ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.762+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f05f408ae20 con 0x7f060c0b3c20 2024-04-26T21:15:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.762+0000 7f060ad9d640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 0x7f05ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.762+0000 7f060ad9d640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 0x7f05ec06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006fd0 tx=0x7f05fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.766+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05f40592e0 con 0x7f060c0b3c20 2024-04-26T21:15:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:15.992+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f060c068e00 con 0x7f060c0b3c20 2024-04-26T21:15:16.953 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:15:16.953+0000 7f72f9ae4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:15:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.992+0000 7f06037fe640 1 -- 172.21.15.40:0/134147521 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) ==== 118+0+0 (secure 0 0 0) 0x7f05f4058be0 con 0x7f060c0b3c20 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 msgr2=0x7f05ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 0x7f05ec06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006fd0 tx=0x7f05fc008040 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 msgr2=0x7f060c15bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c4d0 tx=0x7f05f400c990 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 shutdown_connections 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05ec06c8f0 0x7f05ec06edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f060c0d0010 0x7f060c0a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f060c0b3c20 0x7f060c15bbd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 --2- 172.21.15.40:0/134147521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f060c0b33e0 0x7f060c1f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 >> 172.21.15.40:0/134147521 conn(0x7f060c09ba40 msgr2=0x7f060c15f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:16.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 shutdown_connections 2024-04-26T21:15:16.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:16.995+0000 7f061107e640 1 -- 172.21.15.40:0/134147521 wait complete. 2024-04-26T21:15:17.010 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:15:17.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:15:17.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.119+0000 7f606f50a640 1 Processor -- start 2024-04-26T21:15:17.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.119+0000 7f606f50a640 1 -- start start 2024-04-26T21:15:17.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606f50a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60680d36b0 0x7f60680d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606f50a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f60680d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606f50a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60680d5cd0 con 0x7f60680d50c0 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606f50a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f60680d5a90 con 0x7f60680d36b0 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606f50a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60680d3fd0 con 0x7f60680d0e30 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60680d36b0 0x7f60680d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60680d36b0 0x7f60680d3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51154/0 (socket says 172.21.15.40:51154) 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 -- 172.21.15.40:0/2129471699 learned_addr learned my addr 172.21.15.40:0/2129471699 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606daa7640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f60680d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 -- 172.21.15.40:0/2129471699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 msgr2=0x7f60680d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f60680d5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 -- 172.21.15.40:0/2129471699 >> v1:172.21.15.184:6789/0 conn(0x7f60680d50c0 legacy=0x7f60680d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.120+0000 7f606d2a6640 1 -- 172.21.15.40:0/2129471699 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680b50e0 con 0x7f60680d36b0 2024-04-26T21:15:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.121+0000 7f606d2a6640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680d3a90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6060009870 tx=0x7f606002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecb0e016ede8ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.121+0000 7f6067fff640 1 -- 172.21.15.40:0/2129471699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060037070 con 0x7f60680d36b0 2024-04-26T21:15:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.121+0000 7f6067fff640 1 -- 172.21.15.40:0/2129471699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6060032440 con 0x7f60680d36b0 2024-04-26T21:15:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.121+0000 7f6067fff640 1 -- 172.21.15.40:0/2129471699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060032810 con 0x7f60680d36b0 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 -- 172.21.15.40:0/2129471699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 msgr2=0x7f60680d3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680d3a90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6060009870 tx=0x7f606002a210 comp rx=0 tx=0).stop 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 -- 172.21.15.40:0/2129471699 shutdown_connections 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f60680d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 --2- 172.21.15.40:0/2129471699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680d3a90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 -- 172.21.15.40:0/2129471699 >> 172.21.15.40:0/2129471699 conn(0x7f606808ec30 msgr2=0x7f606809e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 -- 172.21.15.40:0/2129471699 shutdown_connections 2024-04-26T21:15:17.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.122+0000 7f606f50a640 1 -- 172.21.15.40:0/2129471699 wait complete. 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.123+0000 7f606f50a640 1 Processor -- start 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.123+0000 7f606f50a640 1 -- start start 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f606814c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60680d50c0 0x7f60680caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60680b64c0 con 0x7f60680d36b0 2024-04-26T21:15:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60680b61e0 con 0x7f60680d50c0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606f50a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60680b67a0 con 0x7f60680d0e30 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51160/0 (socket says 172.21.15.40:51160) 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 -- 172.21.15.40:0/3638842071 learned_addr learned my addr 172.21.15.40:0/3638842071 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606d2a6640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f606814c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606daa7640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60680d50c0 0x7f60680caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 -- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 msgr2=0x7f606814c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f606814c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 -- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60680d50c0 msgr2=0x7f60680caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60680d50c0 0x7f60680caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 -- 172.21.15.40:0/3638842071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606814c630 con 0x7f60680d36b0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.124+0000 7f606caa5640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f605c00b420 tx=0x7f605c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.125+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c011830 con 0x7f60680d36b0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.125+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c016070 con 0x7f60680d36b0 2024-04-26T21:15:17.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.125+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60680b9dc0 con 0x7f60680d36b0 2024-04-26T21:15:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.125+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c011c10 con 0x7f60680d36b0 2024-04-26T21:15:17.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.125+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60680ba380 con 0x7f60680d36b0 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.126+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6068069ae0 con 0x7f60680d36b0 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.126+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f605c01b460 con 0x7f60680d36b0 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.127+0000 7f6065ffb640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.127+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f605c08b5a0 con 0x7f60680d36b0 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.127+0000 7f606d2a6640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:17.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.128+0000 7f606d2a6640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 0x7f605406edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6060034000 tx=0x7f606002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:17.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.132+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6068069ae0 con 0x7f60680d36b0 2024-04-26T21:15:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:17.357+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f606806a290 con 0x7f60680d36b0 2024-04-26T21:15:18.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.005+0000 7f6065ffb640 1 -- 172.21.15.40:0/3638842071 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f605c020020 con 0x7f60680d36b0 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 msgr2=0x7f605406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 0x7f605406edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6060034000 tx=0x7f606002e000 comp rx=0 tx=0).stop 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 msgr2=0x7f60680b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f605c00b420 tx=0x7f605c00b8e0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 shutdown_connections 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60680d50c0 0x7f60680caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60680d36b0 0x7f60680b7f90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.007+0000 7f606f50a640 1 --2- 172.21.15.40:0/3638842071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60680d0e30 0x7f606814c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.008+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 >> 172.21.15.40:0/3638842071 conn(0x7f606808ec30 msgr2=0x7f60680b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.008+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 shutdown_connections 2024-04-26T21:15:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.008+0000 7f606f50a640 1 -- 172.21.15.40:0/3638842071 wait complete. 2024-04-26T21:15:18.022 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:15:18.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.128+0000 7fa0466c8640 1 Processor -- start 2024-04-26T21:15:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.128+0000 7fa0466c8640 1 -- start start 2024-04-26T21:15:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa0466c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0400b56d0 0x7fa0400d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa0466c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa040160d50 0x7fa0401611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa0466c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0400d0be0 con 0x7fa04015d380 2024-04-26T21:15:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa0466c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa0400d0ec0 con 0x7fa0400b56d0 2024-04-26T21:15:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa0466c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0400d11a0 con 0x7fa040160d50 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0400b56d0 0x7fa0400d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0400b56d0 0x7fa0400d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51162/0 (socket says 172.21.15.40:51162) 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 -- 172.21.15.40:0/532202855 learned_addr learned my addr 172.21.15.40:0/532202855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa044c65640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa040160d50 0x7fa0401611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 -- 172.21.15.40:0/532202855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa040160d50 msgr2=0x7fa0401611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa040160d50 0x7fa0401611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 -- 172.21.15.40:0/532202855 >> v1:172.21.15.184:6789/0 conn(0x7fa04015d380 legacy=0x7fa0400ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.129+0000 7fa03f7fe640 1 -- 172.21.15.40:0/532202855 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400d1480 con 0x7fa0400b56d0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.130+0000 7fa03f7fe640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400b56d0 0x7fa0400d19a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa030009870 tx=0x7fa03002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3d6df2d905e255 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.130+0000 7fa03effd640 1 -- 172.21.15.40:0/532202855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030037070 con 0x7fa0400b56d0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.130+0000 7fa03effd640 1 -- 172.21.15.40:0/532202855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa030032440 con 0x7fa0400b56d0 2024-04-26T21:15:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.130+0000 7fa03effd640 1 -- 172.21.15.40:0/532202855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030032810 con 0x7fa0400b56d0 2024-04-26T21:15:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 -- 172.21.15.40:0/532202855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400b56d0 msgr2=0x7fa0400d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400b56d0 0x7fa0400d19a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa030009870 tx=0x7fa03002a270 comp rx=0 tx=0).stop 2024-04-26T21:15:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 -- 172.21.15.40:0/532202855 shutdown_connections 2024-04-26T21:15:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa040160d50 0x7fa0401611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 --2- 172.21.15.40:0/532202855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0400b56d0 0x7fa0400d19a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 -- 172.21.15.40:0/532202855 >> 172.21.15.40:0/532202855 conn(0x7fa0400b4a60 msgr2=0x7fa0400b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 -- 172.21.15.40:0/532202855 shutdown_connections 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.131+0000 7fa0466c8640 1 -- 172.21.15.40:0/532202855 wait complete. 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.132+0000 7fa0466c8640 1 Processor -- start 2024-04-26T21:15:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.132+0000 7fa0466c8640 1 -- start start 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b56d0 0x7fa0400a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d380 0x7fa0400c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400b5dc0 con 0x7fa040160d50 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400cedc0 con 0x7fa04015d380 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa0466c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0400b5f40 con 0x7fa0400b56d0 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa044c65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa044c65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51172/0 (socket says 172.21.15.40:51172) 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa044c65640 1 -- 172.21.15.40:0/4230083721 learned_addr learned my addr 172.21.15.40:0/4230083721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa03ffff640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b56d0 0x7fa0400a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.133+0000 7fa03f7fe640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d380 0x7fa0400c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 -- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b56d0 msgr2=0x7fa0400a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b56d0 0x7fa0400a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 -- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d380 msgr2=0x7fa0400c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d380 0x7fa0400c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 -- 172.21.15.40:0/4230083721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400c8d70 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa044c65640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa038007e60 tx=0x7fa03800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038014420 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa038019070 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0400c9050 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038014820 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.134+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0400c88a0 con 0x7fa040160d50 2024-04-26T21:15:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.135+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0400c8ad0 con 0x7fa040160d50 2024-04-26T21:15:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.135+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0380149c0 con 0x7fa040160d50 2024-04-26T21:15:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.136+0000 7fa03cff9640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.136+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fa03808c210 con 0x7fa040160d50 2024-04-26T21:15:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.136+0000 7fa03ffff640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.137+0000 7fa03ffff640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa0400cd0f0 tx=0x7fa028009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.140+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa038059ff0 con 0x7fa040160d50 2024-04-26T21:15:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.361+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa04006a290 con 0x7fa040160d50 2024-04-26T21:15:18.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.362+0000 7fa03cff9640 1 -- 172.21.15.40:0/4230083721 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7fa03805e2f0 con 0x7fa040160d50 2024-04-26T21:15:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:18.363 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:18.363 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 msgr2=0x7fa02006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa0400cd0f0 tx=0x7fa028009040 comp rx=0 tx=0).stop 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 msgr2=0x7fa04015fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa038007e60 tx=0x7fa03800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 shutdown_connections 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040160d50 0x7fa04015fbf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04015d380 0x7fa0400c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 --2- 172.21.15.40:0/4230083721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0400b56d0 0x7fa0400a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 >> 172.21.15.40:0/4230083721 conn(0x7fa0400b4a60 msgr2=0x7fa04015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 shutdown_connections 2024-04-26T21:15:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.364+0000 7fa0466c8640 1 -- 172.21.15.40:0/4230083721 wait complete. 2024-04-26T21:15:18.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:18.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.475+0000 7f363da37640 1 Processor -- start 2024-04-26T21:15:18.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.475+0000 7f363da37640 1 -- start start 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f363da37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36380d3ee0 0x7f36380cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f363da37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f36380b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f363da37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36380d03a0 con 0x7f363815f370 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f363da37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36380b4d10 con 0x7f36380d3ee0 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f363da37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36380d0680 con 0x7f363815f750 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f3636ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f363815f370 0x7f36380d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39062/0 (socket says 172.21.15.40:39062) 2024-04-26T21:15:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.476+0000 7f3636ffd640 1 -- 172.21.15.40:0/1157366008 learned_addr learned my addr 172.21.15.40:0/1157366008 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36367fc640 1 --2- 172.21.15.40:0/1157366008 >> v2:172.21.15.184:3300/0 conn(0x7f36380d3ee0 0x7f36380cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f36380b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741839418 0 0) 0x7f36380d03a0 con 0x7f363815f370 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3624003520 con 0x7f363815f370 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2069217196 0 0) 0x7f3624003520 con 0x7f363815f370 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 msgr2=0x7f36380b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f36380b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 >> v2:172.21.15.184:3300/0 conn(0x7f36380d3ee0 msgr2=0x7f36380cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 --2- 172.21.15.40:0/1157366008 >> v2:172.21.15.184:3300/0 conn(0x7f36380d3ee0 0x7f36380cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380d0960 con 0x7f363815f370 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3630001670 con 0x7f363815f370 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36367fc640 1 --2- 172.21.15.40:0/1157366008 >> v2:172.21.15.184:3300/0 conn(0x7f36380d3ee0 0x7f36380cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 >> v1:172.21.15.40:6789/0 conn(0x7f363815f370 legacy=0x7f36380d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f3624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3624004d40 0x7f36240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b4d10 con 0x7f3624004d40 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36380d0680 con 0x7f36240039b0 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f362400b9c0 con 0x7f363815f750 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3624004d40 0x7f36240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36367fc640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f3624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f3636ffd640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 -- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 msgr2=0x7f3624004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f363815f750 0x7f3624004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 -- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36240039b0 msgr2=0x7f3624003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.477+0000 7f36377fe640 1 -- 172.21.15.40:0/1157366008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3624003520 con 0x7f3624004d40 2024-04-26T21:15:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.478+0000 7f36377fe640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3624004d40 0x7f36240051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f362800e170 tx=0x7f3624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d75184acf502c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.478+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628034020 con 0x7f3624004d40 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.478+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36280305f0 con 0x7f3624004d40 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.478+0000 7f3635ffb640 1 -- 172.21.15.40:0/1157366008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628009c10 con 0x7f3624004d40 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 -- 172.21.15.40:0/1157366008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3624004d40 msgr2=0x7f36240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 --2- 172.21.15.40:0/1157366008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3624004d40 0x7f36240051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f362800e170 tx=0x7f3624005830 comp rx=0 tx=0).stop 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f36377fe640 1 -- 172.21.15.40:0/1157366008 reap_dead start 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 -- 172.21.15.40:0/1157366008 shutdown_connections 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 -- 172.21.15.40:0/1157366008 >> 172.21.15.40:0/1157366008 conn(0x7f363808ec10 msgr2=0x7f363809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 -- 172.21.15.40:0/1157366008 shutdown_connections 2024-04-26T21:15:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.479+0000 7f363da37640 1 -- 172.21.15.40:0/1157366008 wait complete. 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.480+0000 7f363da37640 1 Processor -- start 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.480+0000 7f363da37640 1 -- start start 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3638158de0 0x7f36380b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 0x7f36380d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b47d0 con 0x7f3638162780 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36380d0f20 con 0x7f3638158de0 2024-04-26T21:15:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f363da37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36380b4950 con 0x7f36381591c0 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51186/0 (socket says 172.21.15.40:51186) 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 -- 172.21.15.40:0/130482954 learned_addr learned my addr 172.21.15.40:0/130482954 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36367fc640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 0x7f36380d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f3636ffd640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3638158de0 0x7f36380b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 -- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 msgr2=0x7f36380d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 0x7f36380d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 -- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3638158de0 msgr2=0x7f36380b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3638158de0 0x7f36380b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36377fe640 1 -- 172.21.15.40:0/130482954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380ba4b0 con 0x7f3638162780 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.481+0000 7f36367fc640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 0x7f36380d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f36377fe640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f362800e290 tx=0x7f3628009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628034030 con 0x7f3638162780 2024-04-26T21:15:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3628003990 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36380ba790 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628009c10 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.482+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36380b26c0 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.483+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363806a290 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.483+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3628037020 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.484+0000 7f362ffff640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 0x7f361806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.484+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f36280ac210 con 0x7f3638162780 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.484+0000 7f3636ffd640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 0x7f361806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.485+0000 7f3636ffd640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 0x7f361806ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3630004840 tx=0x7f3630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.488+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f362807a5a0 con 0x7f3638162780 2024-04-26T21:15:18.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.676+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3638075fb0 con 0x7f3638162780 2024-04-26T21:15:18.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.677+0000 7f362ffff640 1 -- 172.21.15.40:0/130482954 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16668 (secure 0 0 0) 0x7f3628079ea0 con 0x7f3638162780 2024-04-26T21:15:18.678 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:15.986368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 msgr2=0x7f361806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 0x7f361806ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3630004840 tx=0x7f3630009040 comp rx=0 tx=0).stop 2024-04-26T21:15:18.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 msgr2=0x7f36380d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f362800e290 tx=0x7f3628009750 comp rx=0 tx=0).stop 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 shutdown_connections 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f361806c920 0x7f361806ede0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3638162780 0x7f36380d20e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36381591c0 0x7f36380d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 --2- 172.21.15.40:0/130482954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3638158de0 0x7f36380b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 >> 172.21.15.40:0/130482954 conn(0x7f363808ec10 msgr2=0x7f363815e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 shutdown_connections 2024-04-26T21:15:18.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.679+0000 7f363da37640 1 -- 172.21.15.40:0/130482954 wait complete. 2024-04-26T21:15:18.692 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:15:18.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:15:18.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.791+0000 7faee8938640 1 Processor -- start 2024-04-26T21:15:18.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.791+0000 7faee8938640 1 -- start start 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.792+0000 7faee8938640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faee40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.792+0000 7faee8938640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40d03a0 con 0x7faee415f750 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.792+0000 7faee8938640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40d0680 con 0x7faee415f370 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.792+0000 7faee8938640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d0960 con 0x7faee40d3ee0 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.792+0000 7faee2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faee415f750 0x7faee40b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39068/0 (socket says 172.21.15.40:39068) 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee2d9d640 1 -- 172.21.15.40:0/1961733616 learned_addr learned my addr 172.21.15.40:0/1961733616 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee1d9b640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faee40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964852858 0 0) 0x7faee40d03a0 con 0x7faee415f750 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed0003520 con 0x7faee415f750 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 243058082 0 0) 0x7faed0003520 con 0x7faee415f750 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 msgr2=0x7faee40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faee40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 >> v1:172.21.15.184:6789/0 conn(0x7faee415f370 legacy=0x7faee40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40d0c40 con 0x7faee415f750 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7faed40019f0 con 0x7faee415f750 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 >> v1:172.21.15.40:6789/0 conn(0x7faee415f750 legacy=0x7faee40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed00039e0 0x7faed0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faed0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faed00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d0960 con 0x7faed0004390 2024-04-26T21:15:18.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faed000b9c0 con 0x7faed00039e0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faed000bbc0 con 0x7faee40d3ee0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee1d9b640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faed0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee259c640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed00039e0 0x7faed0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.793+0000 7faee2d9d640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faed00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 -- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 msgr2=0x7faed00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee40d3ee0 0x7faed00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 -- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed00039e0 msgr2=0x7faed0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed00039e0 0x7faed0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 -- 172.21.15.40:0/1961733616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed0003520 con 0x7faed0004390 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee259c640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed00039e0 0x7faed0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee1d9b640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faed0004860 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faecc009ac0 tx=0x7faed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b5226841030d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc036070 con 0x7faed0004390 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc031400 con 0x7faed0004390 2024-04-26T21:15:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.794+0000 7faee159a640 1 -- 172.21.15.40:0/1961733616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc009690 con 0x7faed0004390 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 -- 172.21.15.40:0/1961733616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 msgr2=0x7faed0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 --2- 172.21.15.40:0/1961733616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faed0004860 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faecc009ac0 tx=0x7faed0005830 comp rx=0 tx=0).stop 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee2d9d640 1 -- 172.21.15.40:0/1961733616 reap_dead start 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 -- 172.21.15.40:0/1961733616 shutdown_connections 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 -- 172.21.15.40:0/1961733616 >> 172.21.15.40:0/1961733616 conn(0x7faee408ec10 msgr2=0x7faee409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 -- 172.21.15.40:0/1961733616 shutdown_connections 2024-04-26T21:15:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.795+0000 7faee8938640 1 -- 172.21.15.40:0/1961733616 wait complete. 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.796+0000 7faee8938640 1 Processor -- start 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 -- start start 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 0x7faee40b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee41621d0 0x7faee40b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faee40cfef0 con 0x7faed0004390 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faee40cfd70 con 0x7faee4160650 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee8938640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d0070 con 0x7faee41621d0 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51216/0 (socket says 172.21.15.40:51216) 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.797+0000 7faee259c640 1 -- 172.21.15.40:0/2578013274 learned_addr learned my addr 172.21.15.40:0/2578013274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:18.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee2d9d640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee41621d0 0x7faee40b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee1d9b640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 0x7faee40b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 -- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee41621d0 msgr2=0x7faee40b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee41621d0 0x7faee40b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 -- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 msgr2=0x7faee40b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 0x7faee40b9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 -- 172.21.15.40:0/2578013274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee4161410 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee1d9b640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 0x7faee40b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee259c640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faedc00fb90 tx=0x7faedc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00fe90 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faedc018070 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee41616f0 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00de50 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.798+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee4160ea0 con 0x7faed0004390 2024-04-26T21:15:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.799+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee406a290 con 0x7faed0004390 2024-04-26T21:15:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.800+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faedc0050d0 con 0x7faed0004390 2024-04-26T21:15:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.800+0000 7faedaffd640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 0x7faec406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.800+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faedc08bc60 con 0x7faed0004390 2024-04-26T21:15:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.800+0000 7faee1d9b640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 0x7faec406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.801+0000 7faee1d9b640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 0x7faec406ef50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faecc009ac0 tx=0x7faecc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.804+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faedc05a120 con 0x7faed0004390 2024-04-26T21:15:18.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.984+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faee40d54f0 con 0x7faec406ca90 2024-04-26T21:15:18.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.990+0000 7faedaffd640 1 -- 172.21.15.40:0/2578013274 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251642 (secure 0 0 0) 0x7faee40d54f0 con 0x7faec406ca90 2024-04-26T21:15:18.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:18.996 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 msgr2=0x7faec406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 0x7faec406ef50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faecc009ac0 tx=0x7faecc008030 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 msgr2=0x7faee40cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faedc00fb90 tx=0x7faedc00d520 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 shutdown_connections 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faec406ca90 0x7faec406ef50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.993+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee41621d0 0x7faee40b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.994+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee4160650 0x7faee40b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.994+0000 7faee8938640 1 --2- 172.21.15.40:0/2578013274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faed0004390 0x7faee40cb7b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:18.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.994+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 >> 172.21.15.40:0/2578013274 conn(0x7faee408ec10 msgr2=0x7faee40b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.994+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 shutdown_connections 2024-04-26T21:15:18.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:18.994+0000 7faee8938640 1 -- 172.21.15.40:0/2578013274 wait complete. 2024-04-26T21:15:19.006 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-26T21:15:18.963801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240620,"kb_used_data":7772,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734100,"statfs":{"total":773094113280,"available":772847718400,"internally_reserved":0,"allocated":7958528,"data_stored":3986077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993743"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594048+0000","last_change":"2024-04-26T21:15:15.008421+0000","last_active":"2024-04-26T21:15:16.594048+0000","last_peered":"2024-04-26T21:15:16.594048+0000","last_clean":"2024-04-26T21:15:16.594048+0000","last_became_active":"2024-04-26T21:15:15.008122+0000","last_became_peered":"2024-04-26T21:15:15.008122+0000","last_unstale":"2024-04-26T21:15:16.594048+0000","last_undegraded":"2024-04-26T21:15:16.594048+0000","last_fullsized":"2024-04-26T21:15:16.594048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:35.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998310+0000","last_change":"2024-04-26T21:15:15.004592+0000","last_active":"2024-04-26T21:15:15.998310+0000","last_peered":"2024-04-26T21:15:15.998310+0000","last_clean":"2024-04-26T21:15:15.998310+0000","last_became_active":"2024-04-26T21:15:15.004314+0000","last_became_peered":"2024-04-26T21:15:15.004314+0000","last_unstale":"2024-04-26T21:15:15.998310+0000","last_undegraded":"2024-04-26T21:15:15.998310+0000","last_fullsized":"2024-04-26T21:15:15.998310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595104+0000","last_change":"2024-04-26T21:15:14.984552+0000","last_active":"2024-04-26T21:15:16.595104+0000","last_peered":"2024-04-26T21:15:16.595104+0000","last_clean":"2024-04-26T21:15:16.595104+0000","last_became_active":"2024-04-26T21:15:14.984224+0000","last_became_peered":"2024-04-26T21:15:14.984224+0000","last_unstale":"2024-04-26T21:15:16.595104+0000","last_undegraded":"2024-04-26T21:15:16.595104+0000","last_fullsized":"2024-04-26T21:15:16.595104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:52.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.986510+0000","last_change":"2024-04-26T21:15:13.986510+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.986510+0000","last_undegraded":"2024-04-26T21:15:13.986510+0000","last_fullsized":"2024-04-26T21:15:13.986510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993499+0000","last_change":"2024-04-26T21:15:15.008338+0000","last_active":"2024-04-26T21:15:15.993499+0000","last_peered":"2024-04-26T21:15:15.993499+0000","last_clean":"2024-04-26T21:15:15.993499+0000","last_became_active":"2024-04-26T21:15:15.008029+0000","last_became_peered":"2024-04-26T21:15:15.008029+0000","last_unstale":"2024-04-26T21:15:15.993499+0000","last_undegraded":"2024-04-26T21:15:15.993499+0000","last_fullsized":"2024-04-26T21:15:15.993499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:02.371753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997414+0000","last_change":"2024-04-26T21:15:15.003006+0000","last_active":"2024-04-26T21:15:15.997414+0000","last_peered":"2024-04-26T21:15:15.997414+0000","last_clean":"2024-04-26T21:15:15.997414+0000","last_became_active":"2024-04-26T21:15:15.002731+0000","last_became_peered":"2024-04-26T21:15:15.002731+0000","last_unstale":"2024-04-26T21:15:15.997414+0000","last_undegraded":"2024-04-26T21:15:15.997414+0000","last_fullsized":"2024-04-26T21:15:15.997414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:45.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996492+0000","last_change":"2024-04-26T21:15:15.000633+0000","last_active":"2024-04-26T21:15:15.996492+0000","last_peered":"2024-04-26T21:15:15.996492+0000","last_clean":"2024-04-26T21:15:15.996492+0000","last_became_active":"2024-04-26T21:15:15.000275+0000","last_became_peered":"2024-04-26T21:15:15.000275+0000","last_unstale":"2024-04-26T21:15:15.996492+0000","last_undegraded":"2024-04-26T21:15:15.996492+0000","last_fullsized":"2024-04-26T21:15:15.996492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:11.788021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997948+0000","last_change":"2024-04-26T21:15:15.002868+0000","last_active":"2024-04-26T21:15:15.997948+0000","last_peered":"2024-04-26T21:15:15.997948+0000","last_clean":"2024-04-26T21:15:15.997948+0000","last_became_active":"2024-04-26T21:15:15.002561+0000","last_became_peered":"2024-04-26T21:15:15.002561+0000","last_unstale":"2024-04-26T21:15:15.997948+0000","last_undegraded":"2024-04-26T21:15:15.997948+0000","last_fullsized":"2024-04-26T21:15:15.997948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994128+0000","last_change":"2024-04-26T21:15:15.004629+0000","last_active":"2024-04-26T21:15:15.994128+0000","last_peered":"2024-04-26T21:15:15.994128+0000","last_clean":"2024-04-26T21:15:15.994128+0000","last_became_active":"2024-04-26T21:15:15.004365+0000","last_became_peered":"2024-04-26T21:15:15.004365+0000","last_unstale":"2024-04-26T21:15:15.994128+0000","last_undegraded":"2024-04-26T21:15:15.994128+0000","last_fullsized":"2024-04-26T21:15:15.994128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:33.056050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594298+0000","last_change":"2024-04-26T21:15:15.005575+0000","last_active":"2024-04-26T21:15:16.594298+0000","last_peered":"2024-04-26T21:15:16.594298+0000","last_clean":"2024-04-26T21:15:16.594298+0000","last_became_active":"2024-04-26T21:15:15.005289+0000","last_became_peered":"2024-04-26T21:15:15.005289+0000","last_unstale":"2024-04-26T21:15:16.594298+0000","last_undegraded":"2024-04-26T21:15:16.594298+0000","last_fullsized":"2024-04-26T21:15:16.594298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:18.088546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594550+0000","last_change":"2024-04-26T21:15:15.000155+0000","last_active":"2024-04-26T21:15:16.594550+0000","last_peered":"2024-04-26T21:15:16.594550+0000","last_clean":"2024-04-26T21:15:16.594550+0000","last_became_active":"2024-04-26T21:15:14.999858+0000","last_became_peered":"2024-04-26T21:15:14.999858+0000","last_unstale":"2024-04-26T21:15:16.594550+0000","last_undegraded":"2024-04-26T21:15:16.594550+0000","last_fullsized":"2024-04-26T21:15:16.594550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:01.080096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985514+0000","last_change":"2024-04-26T21:15:13.985514+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985514+0000","last_undegraded":"2024-04-26T21:15:13.985514+0000","last_fullsized":"2024-04-26T21:15:13.985514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593769+0000","last_change":"2024-04-26T21:15:15.001688+0000","last_active":"2024-04-26T21:15:16.593769+0000","last_peered":"2024-04-26T21:15:16.593769+0000","last_clean":"2024-04-26T21:15:16.593769+0000","last_became_active":"2024-04-26T21:15:15.001372+0000","last_became_peered":"2024-04-26T21:15:15.001372+0000","last_unstale":"2024-04-26T21:15:16.593769+0000","last_undegraded":"2024-04-26T21:15:16.593769+0000","last_fullsized":"2024-04-26T21:15:16.593769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993853+0000","last_change":"2024-04-26T21:15:14.995986+0000","last_active":"2024-04-26T21:15:15.993853+0000","last_peered":"2024-04-26T21:15:15.993853+0000","last_clean":"2024-04-26T21:15:15.993853+0000","last_became_active":"2024-04-26T21:15:14.995673+0000","last_became_peered":"2024-04-26T21:15:14.995673+0000","last_unstale":"2024-04-26T21:15:15.993853+0000","last_undegraded":"2024-04-26T21:15:15.993853+0000","last_fullsized":"2024-04-26T21:15:15.993853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593933+0000","last_change":"2024-04-26T21:15:14.996331+0000","last_active":"2024-04-26T21:15:16.593933+0000","last_peered":"2024-04-26T21:15:16.593933+0000","last_clean":"2024-04-26T21:15:16.593933+0000","last_became_active":"2024-04-26T21:15:14.996044+0000","last_became_peered":"2024-04-26T21:15:14.996044+0000","last_unstale":"2024-04-26T21:15:16.593933+0000","last_undegraded":"2024-04-26T21:15:16.593933+0000","last_fullsized":"2024-04-26T21:15:16.593933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593583+0000","last_change":"2024-04-26T21:15:15.007557+0000","last_active":"2024-04-26T21:15:16.593583+0000","last_peered":"2024-04-26T21:15:16.593583+0000","last_clean":"2024-04-26T21:15:16.593583+0000","last_became_active":"2024-04-26T21:15:15.007269+0000","last_became_peered":"2024-04-26T21:15:15.007269+0000","last_unstale":"2024-04-26T21:15:16.593583+0000","last_undegraded":"2024-04-26T21:15:16.593583+0000","last_fullsized":"2024-04-26T21:15:16.593583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994100+0000","last_change":"2024-04-26T21:15:15.003937+0000","last_active":"2024-04-26T21:15:15.994100+0000","last_peered":"2024-04-26T21:15:15.994100+0000","last_clean":"2024-04-26T21:15:15.994100+0000","last_became_active":"2024-04-26T21:15:15.003652+0000","last_became_peered":"2024-04-26T21:15:15.003652+0000","last_unstale":"2024-04-26T21:15:15.994100+0000","last_undegraded":"2024-04-26T21:15:15.994100+0000","last_fullsized":"2024-04-26T21:15:15.994100+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.457492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985659+0000","last_change":"2024-04-26T21:15:13.985659+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985659+0000","last_undegraded":"2024-04-26T21:15:13.985659+0000","last_fullsized":"2024-04-26T21:15:13.985659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997884+0000","last_change":"2024-04-26T21:15:15.003066+0000","last_active":"2024-04-26T21:15:15.997884+0000","last_peered":"2024-04-26T21:15:15.997884+0000","last_clean":"2024-04-26T21:15:15.997884+0000","last_became_active":"2024-04-26T21:15:15.002597+0000","last_became_peered":"2024-04-26T21:15:15.002597+0000","last_unstale":"2024-04-26T21:15:15.997884+0000","last_undegraded":"2024-04-26T21:15:15.997884+0000","last_fullsized":"2024-04-26T21:15:15.997884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593869+0000","last_change":"2024-04-26T21:15:14.996454+0000","last_active":"2024-04-26T21:15:16.593869+0000","last_peered":"2024-04-26T21:15:16.593869+0000","last_clean":"2024-04-26T21:15:16.593869+0000","last_became_active":"2024-04-26T21:15:14.996140+0000","last_became_peered":"2024-04-26T21:15:14.996140+0000","last_unstale":"2024-04-26T21:15:16.593869+0000","last_undegraded":"2024-04-26T21:15:16.593869+0000","last_fullsized":"2024-04-26T21:15:16.593869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593784+0000","last_change":"2024-04-26T21:15:15.004956+0000","last_active":"2024-04-26T21:15:16.593784+0000","last_peered":"2024-04-26T21:15:16.593784+0000","last_clean":"2024-04-26T21:15:16.593784+0000","last_became_active":"2024-04-26T21:15:15.004666+0000","last_became_peered":"2024-04-26T21:15:15.004666+0000","last_unstale":"2024-04-26T21:15:16.593784+0000","last_undegraded":"2024-04-26T21:15:16.593784+0000","last_fullsized":"2024-04-26T21:15:16.593784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593651+0000","last_change":"2024-04-26T21:15:14.998310+0000","last_active":"2024-04-26T21:15:16.593651+0000","last_peered":"2024-04-26T21:15:16.593651+0000","last_clean":"2024-04-26T21:15:16.593651+0000","last_became_active":"2024-04-26T21:15:14.998039+0000","last_became_peered":"2024-04-26T21:15:14.998039+0000","last_unstale":"2024-04-26T21:15:16.593651+0000","last_undegraded":"2024-04-26T21:15:16.593651+0000","last_fullsized":"2024-04-26T21:15:16.593651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:14.997132+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:14.996824+0000","last_became_peered":"2024-04-26T21:15:14.996824+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593965+0000","last_change":"2024-04-26T21:15:15.003818+0000","last_active":"2024-04-26T21:15:16.593965+0000","last_peered":"2024-04-26T21:15:16.593965+0000","last_clean":"2024-04-26T21:15:16.593965+0000","last_became_active":"2024-04-26T21:15:15.003428+0000","last_became_peered":"2024-04-26T21:15:15.003428+0000","last_unstale":"2024-04-26T21:15:16.593965+0000","last_undegraded":"2024-04-26T21:15:16.593965+0000","last_fullsized":"2024-04-26T21:15:16.593965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594069+0000","last_change":"2024-04-26T21:15:15.004314+0000","last_active":"2024-04-26T21:15:16.594069+0000","last_peered":"2024-04-26T21:15:16.594069+0000","last_clean":"2024-04-26T21:15:16.594069+0000","last_became_active":"2024-04-26T21:15:15.004022+0000","last_became_peered":"2024-04-26T21:15:15.004022+0000","last_unstale":"2024-04-26T21:15:16.594069+0000","last_undegraded":"2024-04-26T21:15:16.594069+0000","last_fullsized":"2024-04-26T21:15:16.594069+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996953+0000","last_change":"2024-04-26T21:15:15.006413+0000","last_active":"2024-04-26T21:15:15.996953+0000","last_peered":"2024-04-26T21:15:15.996953+0000","last_clean":"2024-04-26T21:15:15.996953+0000","last_became_active":"2024-04-26T21:15:15.006136+0000","last_became_peered":"2024-04-26T21:15:15.006136+0000","last_unstale":"2024-04-26T21:15:15.996953+0000","last_undegraded":"2024-04-26T21:15:15.996953+0000","last_fullsized":"2024-04-26T21:15:15.996953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595237+0000","last_change":"2024-04-26T21:15:14.999635+0000","last_active":"2024-04-26T21:15:16.595237+0000","last_peered":"2024-04-26T21:15:16.595237+0000","last_clean":"2024-04-26T21:15:16.595237+0000","last_became_active":"2024-04-26T21:15:14.999351+0000","last_became_peered":"2024-04-26T21:15:14.999351+0000","last_unstale":"2024-04-26T21:15:16.595237+0000","last_undegraded":"2024-04-26T21:15:16.595237+0000","last_fullsized":"2024-04-26T21:15:16.595237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:23.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594169+0000","last_change":"2024-04-26T21:15:15.006522+0000","last_active":"2024-04-26T21:15:16.594169+0000","last_peered":"2024-04-26T21:15:16.594169+0000","last_clean":"2024-04-26T21:15:16.594169+0000","last_became_active":"2024-04-26T21:15:15.005849+0000","last_became_peered":"2024-04-26T21:15:15.005849+0000","last_unstale":"2024-04-26T21:15:16.594169+0000","last_undegraded":"2024-04-26T21:15:16.594169+0000","last_fullsized":"2024-04-26T21:15:16.594169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:16.396516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594834+0000","last_change":"2024-04-26T21:15:14.988314+0000","last_active":"2024-04-26T21:15:16.594834+0000","last_peered":"2024-04-26T21:15:16.594834+0000","last_clean":"2024-04-26T21:15:16.594834+0000","last_became_active":"2024-04-26T21:15:14.987983+0000","last_became_peered":"2024-04-26T21:15:14.987983+0000","last_unstale":"2024-04-26T21:15:16.594834+0000","last_undegraded":"2024-04-26T21:15:16.594834+0000","last_fullsized":"2024-04-26T21:15:16.594834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:56.347854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996624+0000","last_change":"2024-04-26T21:15:15.001705+0000","last_active":"2024-04-26T21:15:15.996624+0000","last_peered":"2024-04-26T21:15:15.996624+0000","last_clean":"2024-04-26T21:15:15.996624+0000","last_became_active":"2024-04-26T21:15:15.001398+0000","last_became_peered":"2024-04-26T21:15:15.001398+0000","last_unstale":"2024-04-26T21:15:15.996624+0000","last_undegraded":"2024-04-26T21:15:15.996624+0000","last_fullsized":"2024-04-26T21:15:15.996624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996883+0000","last_change":"2024-04-26T21:15:14.996277+0000","last_active":"2024-04-26T21:15:15.996883+0000","last_peered":"2024-04-26T21:15:15.996883+0000","last_clean":"2024-04-26T21:15:15.996883+0000","last_became_active":"2024-04-26T21:15:14.996006+0000","last_became_peered":"2024-04-26T21:15:14.996006+0000","last_unstale":"2024-04-26T21:15:15.996883+0000","last_undegraded":"2024-04-26T21:15:15.996883+0000","last_fullsized":"2024-04-26T21:15:15.996883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997797+0000","last_change":"2024-04-26T21:15:15.005406+0000","last_active":"2024-04-26T21:15:15.997797+0000","last_peered":"2024-04-26T21:15:15.997797+0000","last_clean":"2024-04-26T21:15:15.997797+0000","last_became_active":"2024-04-26T21:15:15.005116+0000","last_became_peered":"2024-04-26T21:15:15.005116+0000","last_unstale":"2024-04-26T21:15:15.997797+0000","last_undegraded":"2024-04-26T21:15:15.997797+0000","last_fullsized":"2024-04-26T21:15:15.997797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:16.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594603+0000","last_change":"2024-04-26T21:15:15.008320+0000","last_active":"2024-04-26T21:15:16.594603+0000","last_peered":"2024-04-26T21:15:16.594603+0000","last_clean":"2024-04-26T21:15:16.594603+0000","last_became_active":"2024-04-26T21:15:15.008048+0000","last_became_peered":"2024-04-26T21:15:15.008048+0000","last_unstale":"2024-04-26T21:15:16.594603+0000","last_undegraded":"2024-04-26T21:15:16.594603+0000","last_fullsized":"2024-04-26T21:15:16.594603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.353647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594502+0000","last_change":"2024-04-26T21:15:13.991660+0000","last_active":"2024-04-26T21:15:16.594502+0000","last_peered":"2024-04-26T21:15:16.594502+0000","last_clean":"2024-04-26T21:15:16.594502+0000","last_became_active":"2024-04-26T21:15:13.991361+0000","last_became_peered":"2024-04-26T21:15:13.991361+0000","last_unstale":"2024-04-26T21:15:16.594502+0000","last_undegraded":"2024-04-26T21:15:16.594502+0000","last_fullsized":"2024-04-26T21:15:16.594502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994190+0000","last_change":"2024-04-26T21:15:15.003269+0000","last_active":"2024-04-26T21:15:15.994190+0000","last_peered":"2024-04-26T21:15:15.994190+0000","last_clean":"2024-04-26T21:15:15.994190+0000","last_became_active":"2024-04-26T21:15:15.002973+0000","last_became_peered":"2024-04-26T21:15:15.002973+0000","last_unstale":"2024-04-26T21:15:15.994190+0000","last_undegraded":"2024-04-26T21:15:15.994190+0000","last_fullsized":"2024-04-26T21:15:15.994190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:30.756090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593977+0000","last_change":"2024-04-26T21:15:13.990425+0000","last_active":"2024-04-26T21:15:16.593977+0000","last_peered":"2024-04-26T21:15:16.593977+0000","last_clean":"2024-04-26T21:15:16.593977+0000","last_became_active":"2024-04-26T21:15:13.990113+0000","last_became_peered":"2024-04-26T21:15:13.990113+0000","last_unstale":"2024-04-26T21:15:16.593977+0000","last_undegraded":"2024-04-26T21:15:16.593977+0000","last_fullsized":"2024-04-26T21:15:16.593977+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997715+0000","last_change":"2024-04-26T21:15:15.004144+0000","last_active":"2024-04-26T21:15:15.997715+0000","last_peered":"2024-04-26T21:15:15.997715+0000","last_clean":"2024-04-26T21:15:15.997715+0000","last_became_active":"2024-04-26T21:15:15.003804+0000","last_became_peered":"2024-04-26T21:15:15.003804+0000","last_unstale":"2024-04-26T21:15:15.997715+0000","last_undegraded":"2024-04-26T21:15:15.997715+0000","last_fullsized":"2024-04-26T21:15:15.997715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:34.213418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594471+0000","last_change":"2024-04-26T21:15:13.996784+0000","last_active":"2024-04-26T21:15:16.594471+0000","last_peered":"2024-04-26T21:15:16.594471+0000","last_clean":"2024-04-26T21:15:16.594471+0000","last_became_active":"2024-04-26T21:15:13.996508+0000","last_became_peered":"2024-04-26T21:15:13.996508+0000","last_unstale":"2024-04-26T21:15:16.594471+0000","last_undegraded":"2024-04-26T21:15:16.594471+0000","last_fullsized":"2024-04-26T21:15:16.594471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:31.090506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594503+0000","last_change":"2024-04-26T21:15:14.998880+0000","last_active":"2024-04-26T21:15:16.594503+0000","last_peered":"2024-04-26T21:15:16.594503+0000","last_clean":"2024-04-26T21:15:16.594503+0000","last_became_active":"2024-04-26T21:15:14.998406+0000","last_became_peered":"2024-04-26T21:15:14.998406+0000","last_unstale":"2024-04-26T21:15:16.594503+0000","last_undegraded":"2024-04-26T21:15:16.594503+0000","last_fullsized":"2024-04-26T21:15:16.594503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:32.918934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997990+0000","last_change":"2024-04-26T21:15:13.990545+0000","last_active":"2024-04-26T21:15:15.997990+0000","last_peered":"2024-04-26T21:15:15.997990+0000","last_clean":"2024-04-26T21:15:15.997990+0000","last_became_active":"2024-04-26T21:15:13.990151+0000","last_became_peered":"2024-04-26T21:15:13.990151+0000","last_unstale":"2024-04-26T21:15:15.997990+0000","last_undegraded":"2024-04-26T21:15:15.997990+0000","last_fullsized":"2024-04-26T21:15:15.997990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:43.327070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996712+0000","last_change":"2024-04-26T21:15:14.999455+0000","last_active":"2024-04-26T21:15:15.996712+0000","last_peered":"2024-04-26T21:15:15.996712+0000","last_clean":"2024-04-26T21:15:15.996712+0000","last_became_active":"2024-04-26T21:15:14.999063+0000","last_became_peered":"2024-04-26T21:15:14.999063+0000","last_unstale":"2024-04-26T21:15:15.996712+0000","last_undegraded":"2024-04-26T21:15:15.996712+0000","last_fullsized":"2024-04-26T21:15:15.996712+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595004+0000","last_change":"2024-04-26T21:15:14.000741+0000","last_active":"2024-04-26T21:15:16.595004+0000","last_peered":"2024-04-26T21:15:16.595004+0000","last_clean":"2024-04-26T21:15:16.595004+0000","last_became_active":"2024-04-26T21:15:14.000484+0000","last_became_peered":"2024-04-26T21:15:14.000484+0000","last_unstale":"2024-04-26T21:15:16.595004+0000","last_undegraded":"2024-04-26T21:15:16.595004+0000","last_fullsized":"2024-04-26T21:15:16.595004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:34.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:15.003590+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:15.003252+0000","last_became_peered":"2024-04-26T21:15:15.003252+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:40.805512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997494+0000","last_change":"2024-04-26T21:15:13.987191+0000","last_active":"2024-04-26T21:15:15.997494+0000","last_peered":"2024-04-26T21:15:15.997494+0000","last_clean":"2024-04-26T21:15:15.997494+0000","last_became_active":"2024-04-26T21:15:13.986872+0000","last_became_peered":"2024-04-26T21:15:13.986872+0000","last_unstale":"2024-04-26T21:15:15.997494+0000","last_undegraded":"2024-04-26T21:15:15.997494+0000","last_fullsized":"2024-04-26T21:15:15.997494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:02.357589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993432+0000","last_change":"2024-04-26T21:15:15.002302+0000","last_active":"2024-04-26T21:15:15.993432+0000","last_peered":"2024-04-26T21:15:15.993432+0000","last_clean":"2024-04-26T21:15:15.993432+0000","last_became_active":"2024-04-26T21:15:15.001311+0000","last_became_peered":"2024-04-26T21:15:15.001311+0000","last_unstale":"2024-04-26T21:15:15.993432+0000","last_undegraded":"2024-04-26T21:15:15.993432+0000","last_fullsized":"2024-04-26T21:15:15.993432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:08.309553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003614+0000","last_change":"2024-04-26T21:15:13.998731+0000","last_active":"2024-04-26T21:15:14.003614+0000","last_peered":"2024-04-26T21:15:14.003614+0000","last_clean":"2024-04-26T21:15:14.003614+0000","last_became_active":"2024-04-26T21:15:13.998420+0000","last_became_peered":"2024-04-26T21:15:13.998420+0000","last_unstale":"2024-04-26T21:15:14.003614+0000","last_undegraded":"2024-04-26T21:15:14.003614+0000","last_fullsized":"2024-04-26T21:15:14.003614+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:03.412018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593528+0000","last_change":"2024-04-26T21:15:14.999599+0000","last_active":"2024-04-26T21:15:16.593528+0000","last_peered":"2024-04-26T21:15:16.593528+0000","last_clean":"2024-04-26T21:15:16.593528+0000","last_became_active":"2024-04-26T21:15:14.998302+0000","last_became_peered":"2024-04-26T21:15:14.998302+0000","last_unstale":"2024-04-26T21:15:16.593528+0000","last_undegraded":"2024-04-26T21:15:16.593528+0000","last_fullsized":"2024-04-26T21:15:16.593528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993426+0000","last_change":"2024-04-26T21:15:13.989877+0000","last_active":"2024-04-26T21:15:15.993426+0000","last_peered":"2024-04-26T21:15:15.993426+0000","last_clean":"2024-04-26T21:15:15.993426+0000","last_became_active":"2024-04-26T21:15:13.989441+0000","last_became_peered":"2024-04-26T21:15:13.989441+0000","last_unstale":"2024-04-26T21:15:15.993426+0000","last_undegraded":"2024-04-26T21:15:15.993426+0000","last_fullsized":"2024-04-26T21:15:15.993426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:41.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997022+0000","last_change":"2024-04-26T21:15:14.999473+0000","last_active":"2024-04-26T21:15:15.997022+0000","last_peered":"2024-04-26T21:15:15.997022+0000","last_clean":"2024-04-26T21:15:15.997022+0000","last_became_active":"2024-04-26T21:15:14.999179+0000","last_became_peered":"2024-04-26T21:15:14.999179+0000","last_unstale":"2024-04-26T21:15:15.997022+0000","last_undegraded":"2024-04-26T21:15:15.997022+0000","last_fullsized":"2024-04-26T21:15:15.997022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594171+0000","last_change":"2024-04-26T21:15:13.999674+0000","last_active":"2024-04-26T21:15:16.594171+0000","last_peered":"2024-04-26T21:15:16.594171+0000","last_clean":"2024-04-26T21:15:16.594171+0000","last_became_active":"2024-04-26T21:15:13.999384+0000","last_became_peered":"2024-04-26T21:15:13.999384+0000","last_unstale":"2024-04-26T21:15:16.594171+0000","last_undegraded":"2024-04-26T21:15:16.594171+0000","last_fullsized":"2024-04-26T21:15:16.594171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:41.421624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594232+0000","last_change":"2024-04-26T21:15:15.004056+0000","last_active":"2024-04-26T21:15:16.594232+0000","last_peered":"2024-04-26T21:15:16.594232+0000","last_clean":"2024-04-26T21:15:16.594232+0000","last_became_active":"2024-04-26T21:15:15.003109+0000","last_became_peered":"2024-04-26T21:15:15.003109+0000","last_unstale":"2024-04-26T21:15:16.594232+0000","last_undegraded":"2024-04-26T21:15:16.594232+0000","last_fullsized":"2024-04-26T21:15:16.594232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.332709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998354+0000","last_change":"2024-04-26T21:15:13.988307+0000","last_active":"2024-04-26T21:15:15.998354+0000","last_peered":"2024-04-26T21:15:15.998354+0000","last_clean":"2024-04-26T21:15:15.998354+0000","last_became_active":"2024-04-26T21:15:13.988018+0000","last_became_peered":"2024-04-26T21:15:13.988018+0000","last_unstale":"2024-04-26T21:15:15.998354+0000","last_undegraded":"2024-04-26T21:15:15.998354+0000","last_fullsized":"2024-04-26T21:15:15.998354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.077866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997865+0000","last_change":"2024-04-26T21:15:15.004543+0000","last_active":"2024-04-26T21:15:15.997865+0000","last_peered":"2024-04-26T21:15:15.997865+0000","last_clean":"2024-04-26T21:15:15.997865+0000","last_became_active":"2024-04-26T21:15:15.004281+0000","last_became_peered":"2024-04-26T21:15:15.004281+0000","last_unstale":"2024-04-26T21:15:15.997865+0000","last_undegraded":"2024-04-26T21:15:15.997865+0000","last_fullsized":"2024-04-26T21:15:15.997865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595209+0000","last_change":"2024-04-26T21:15:13.991260+0000","last_active":"2024-04-26T21:15:16.595209+0000","last_peered":"2024-04-26T21:15:16.595209+0000","last_clean":"2024-04-26T21:15:16.595209+0000","last_became_active":"2024-04-26T21:15:13.990969+0000","last_became_peered":"2024-04-26T21:15:13.990969+0000","last_unstale":"2024-04-26T21:15:16.595209+0000","last_undegraded":"2024-04-26T21:15:16.595209+0000","last_fullsized":"2024-04-26T21:15:16.595209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:35.822660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993865+0000","last_change":"2024-04-26T21:15:14.997788+0000","last_active":"2024-04-26T21:15:15.993865+0000","last_peered":"2024-04-26T21:15:15.993865+0000","last_clean":"2024-04-26T21:15:15.993865+0000","last_became_active":"2024-04-26T21:15:14.997464+0000","last_became_peered":"2024-04-26T21:15:14.997464+0000","last_unstale":"2024-04-26T21:15:15.993865+0000","last_undegraded":"2024-04-26T21:15:15.993865+0000","last_fullsized":"2024-04-26T21:15:15.993865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:56.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994753+0000","last_change":"2024-04-26T21:15:13.991679+0000","last_active":"2024-04-26T21:15:13.994753+0000","last_peered":"2024-04-26T21:15:13.994753+0000","last_clean":"2024-04-26T21:15:13.994753+0000","last_became_active":"2024-04-26T21:15:13.991338+0000","last_became_peered":"2024-04-26T21:15:13.991338+0000","last_unstale":"2024-04-26T21:15:13.994753+0000","last_undegraded":"2024-04-26T21:15:13.994753+0000","last_fullsized":"2024-04-26T21:15:13.994753+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:32.732498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:14.999960+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:14.999673+0000","last_became_peered":"2024-04-26T21:15:14.999673+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:36.824537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994639+0000","last_change":"2024-04-26T21:15:13.990261+0000","last_active":"2024-04-26T21:15:13.994639+0000","last_peered":"2024-04-26T21:15:13.994639+0000","last_clean":"2024-04-26T21:15:13.994639+0000","last_became_active":"2024-04-26T21:15:13.989888+0000","last_became_peered":"2024-04-26T21:15:13.989888+0000","last_unstale":"2024-04-26T21:15:13.994639+0000","last_undegraded":"2024-04-26T21:15:13.994639+0000","last_fullsized":"2024-04-26T21:15:13.994639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:22.729662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996901+0000","last_change":"2024-04-26T21:15:15.000394+0000","last_active":"2024-04-26T21:15:15.996901+0000","last_peered":"2024-04-26T21:15:15.996901+0000","last_clean":"2024-04-26T21:15:15.996901+0000","last_became_active":"2024-04-26T21:15:14.999894+0000","last_became_peered":"2024-04-26T21:15:14.999894+0000","last_unstale":"2024-04-26T21:15:15.996901+0000","last_undegraded":"2024-04-26T21:15:15.996901+0000","last_fullsized":"2024-04-26T21:15:15.996901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997598+0000","last_change":"2024-04-26T21:15:13.989609+0000","last_active":"2024-04-26T21:15:15.997598+0000","last_peered":"2024-04-26T21:15:15.997598+0000","last_clean":"2024-04-26T21:15:15.997598+0000","last_became_active":"2024-04-26T21:15:13.989419+0000","last_became_peered":"2024-04-26T21:15:13.989419+0000","last_unstale":"2024-04-26T21:15:15.997598+0000","last_undegraded":"2024-04-26T21:15:15.997598+0000","last_fullsized":"2024-04-26T21:15:15.997598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:36.252437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593466+0000","last_change":"2024-04-26T21:15:15.005165+0000","last_active":"2024-04-26T21:15:16.593466+0000","last_peered":"2024-04-26T21:15:16.593466+0000","last_clean":"2024-04-26T21:15:16.593466+0000","last_became_active":"2024-04-26T21:15:15.004821+0000","last_became_peered":"2024-04-26T21:15:15.004821+0000","last_unstale":"2024-04-26T21:15:16.593466+0000","last_undegraded":"2024-04-26T21:15:16.593466+0000","last_fullsized":"2024-04-26T21:15:16.593466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996529+0000","last_change":"2024-04-26T21:15:13.984946+0000","last_active":"2024-04-26T21:15:15.996529+0000","last_peered":"2024-04-26T21:15:15.996529+0000","last_clean":"2024-04-26T21:15:15.996529+0000","last_became_active":"2024-04-26T21:15:13.984641+0000","last_became_peered":"2024-04-26T21:15:13.984641+0000","last_unstale":"2024-04-26T21:15:15.996529+0000","last_undegraded":"2024-04-26T21:15:15.996529+0000","last_fullsized":"2024-04-26T21:15:15.996529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993537+0000","last_change":"2024-04-26T21:15:15.007343+0000","last_active":"2024-04-26T21:15:15.993537+0000","last_peered":"2024-04-26T21:15:15.993537+0000","last_clean":"2024-04-26T21:15:15.993537+0000","last_became_active":"2024-04-26T21:15:15.007015+0000","last_became_peered":"2024-04-26T21:15:15.007015+0000","last_unstale":"2024-04-26T21:15:15.993537+0000","last_undegraded":"2024-04-26T21:15:15.993537+0000","last_fullsized":"2024-04-26T21:15:15.993537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:34.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593601+0000","last_change":"2024-04-26T21:15:13.996573+0000","last_active":"2024-04-26T21:15:16.593601+0000","last_peered":"2024-04-26T21:15:16.593601+0000","last_clean":"2024-04-26T21:15:16.593601+0000","last_became_active":"2024-04-26T21:15:13.996295+0000","last_became_peered":"2024-04-26T21:15:13.996295+0000","last_unstale":"2024-04-26T21:15:16.593601+0000","last_undegraded":"2024-04-26T21:15:16.593601+0000","last_fullsized":"2024-04-26T21:15:16.593601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:42.351528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594435+0000","last_change":"2024-04-26T21:15:15.001042+0000","last_active":"2024-04-26T21:15:16.594435+0000","last_peered":"2024-04-26T21:15:16.594435+0000","last_clean":"2024-04-26T21:15:16.594435+0000","last_became_active":"2024-04-26T21:15:15.000753+0000","last_became_peered":"2024-04-26T21:15:15.000753+0000","last_unstale":"2024-04-26T21:15:16.594435+0000","last_undegraded":"2024-04-26T21:15:16.594435+0000","last_fullsized":"2024-04-26T21:15:16.594435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.434213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998075+0000","last_change":"2024-04-26T21:15:13.993704+0000","last_active":"2024-04-26T21:15:15.998075+0000","last_peered":"2024-04-26T21:15:15.998075+0000","last_clean":"2024-04-26T21:15:15.998075+0000","last_became_active":"2024-04-26T21:15:13.993410+0000","last_became_peered":"2024-04-26T21:15:13.993410+0000","last_unstale":"2024-04-26T21:15:15.998075+0000","last_undegraded":"2024-04-26T21:15:15.998075+0000","last_fullsized":"2024-04-26T21:15:15.998075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594517+0000","last_change":"2024-04-26T21:15:14.998082+0000","last_active":"2024-04-26T21:15:16.594517+0000","last_peered":"2024-04-26T21:15:16.594517+0000","last_clean":"2024-04-26T21:15:16.594517+0000","last_became_active":"2024-04-26T21:15:14.997733+0000","last_became_peered":"2024-04-26T21:15:14.997733+0000","last_unstale":"2024-04-26T21:15:16.594517+0000","last_undegraded":"2024-04-26T21:15:16.594517+0000","last_fullsized":"2024-04-26T21:15:16.594517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:07.215468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594639+0000","last_change":"2024-04-26T21:15:14.001697+0000","last_active":"2024-04-26T21:15:16.594639+0000","last_peered":"2024-04-26T21:15:16.594639+0000","last_clean":"2024-04-26T21:15:16.594639+0000","last_became_active":"2024-04-26T21:15:14.001416+0000","last_became_peered":"2024-04-26T21:15:14.001416+0000","last_unstale":"2024-04-26T21:15:16.594639+0000","last_undegraded":"2024-04-26T21:15:16.594639+0000","last_fullsized":"2024-04-26T21:15:16.594639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:11.110564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985044+0000","last_change":"2024-04-26T21:15:13.985044+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985044+0000","last_undegraded":"2024-04-26T21:15:13.985044+0000","last_fullsized":"2024-04-26T21:15:13.985044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997916+0000","last_change":"2024-04-26T21:15:13.997192+0000","last_active":"2024-04-26T21:15:15.997916+0000","last_peered":"2024-04-26T21:15:15.997916+0000","last_clean":"2024-04-26T21:15:15.997916+0000","last_became_active":"2024-04-26T21:15:13.996946+0000","last_became_peered":"2024-04-26T21:15:13.996946+0000","last_unstale":"2024-04-26T21:15:15.997916+0000","last_undegraded":"2024-04-26T21:15:15.997916+0000","last_fullsized":"2024-04-26T21:15:15.997916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:33.005058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993826+0000","last_change":"2024-04-26T21:15:15.000551+0000","last_active":"2024-04-26T21:15:15.993826+0000","last_peered":"2024-04-26T21:15:15.993826+0000","last_clean":"2024-04-26T21:15:15.993826+0000","last_became_active":"2024-04-26T21:15:15.000269+0000","last_became_peered":"2024-04-26T21:15:15.000269+0000","last_unstale":"2024-04-26T21:15:15.993826+0000","last_undegraded":"2024-04-26T21:15:15.993826+0000","last_fullsized":"2024-04-26T21:15:15.993826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:46.479309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594507+0000","last_change":"2024-04-26T21:15:14.000103+0000","last_active":"2024-04-26T21:15:16.594507+0000","last_peered":"2024-04-26T21:15:16.594507+0000","last_clean":"2024-04-26T21:15:16.594507+0000","last_became_active":"2024-04-26T21:15:13.999824+0000","last_became_peered":"2024-04-26T21:15:13.999824+0000","last_unstale":"2024-04-26T21:15:16.594507+0000","last_undegraded":"2024-04-26T21:15:16.594507+0000","last_fullsized":"2024-04-26T21:15:16.594507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595067+0000","last_change":"2024-04-26T21:15:14.999573+0000","last_active":"2024-04-26T21:15:16.595067+0000","last_peered":"2024-04-26T21:15:16.595067+0000","last_clean":"2024-04-26T21:15:16.595067+0000","last_became_active":"2024-04-26T21:15:14.999265+0000","last_became_peered":"2024-04-26T21:15:14.999265+0000","last_unstale":"2024-04-26T21:15:16.595067+0000","last_undegraded":"2024-04-26T21:15:16.595067+0000","last_fullsized":"2024-04-26T21:15:16.595067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:49.825951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997655+0000","last_change":"2024-04-26T21:15:13.987004+0000","last_active":"2024-04-26T21:15:15.997655+0000","last_peered":"2024-04-26T21:15:15.997655+0000","last_clean":"2024-04-26T21:15:15.997655+0000","last_became_active":"2024-04-26T21:15:13.986676+0000","last_became_peered":"2024-04-26T21:15:13.986676+0000","last_unstale":"2024-04-26T21:15:15.997655+0000","last_undegraded":"2024-04-26T21:15:15.997655+0000","last_fullsized":"2024-04-26T21:15:15.997655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:44.090460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594645+0000","last_change":"2024-04-26T21:15:15.004474+0000","last_active":"2024-04-26T21:15:16.594645+0000","last_peered":"2024-04-26T21:15:16.594645+0000","last_clean":"2024-04-26T21:15:16.594645+0000","last_became_active":"2024-04-26T21:15:15.004185+0000","last_became_peered":"2024-04-26T21:15:15.004185+0000","last_unstale":"2024-04-26T21:15:16.594645+0000","last_undegraded":"2024-04-26T21:15:16.594645+0000","last_fullsized":"2024-04-26T21:15:16.594645+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.298412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996654+0000","last_change":"2024-04-26T21:15:13.988682+0000","last_active":"2024-04-26T21:15:15.996654+0000","last_peered":"2024-04-26T21:15:15.996654+0000","last_clean":"2024-04-26T21:15:15.996654+0000","last_became_active":"2024-04-26T21:15:13.988377+0000","last_became_peered":"2024-04-26T21:15:13.988377+0000","last_unstale":"2024-04-26T21:15:15.996654+0000","last_undegraded":"2024-04-26T21:15:15.996654+0000","last_fullsized":"2024-04-26T21:15:15.996654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:15.394639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594757+0000","last_change":"2024-04-26T21:15:15.005522+0000","last_active":"2024-04-26T21:15:16.594757+0000","last_peered":"2024-04-26T21:15:16.594757+0000","last_clean":"2024-04-26T21:15:16.594757+0000","last_became_active":"2024-04-26T21:15:15.005254+0000","last_became_peered":"2024-04-26T21:15:15.005254+0000","last_unstale":"2024-04-26T21:15:16.594757+0000","last_undegraded":"2024-04-26T21:15:16.594757+0000","last_fullsized":"2024-04-26T21:15:16.594757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.398672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593386+0000","last_change":"2024-04-26T21:15:13.990434+0000","last_active":"2024-04-26T21:15:16.593386+0000","last_peered":"2024-04-26T21:15:16.593386+0000","last_clean":"2024-04-26T21:15:16.593386+0000","last_became_active":"2024-04-26T21:15:13.990190+0000","last_became_peered":"2024-04-26T21:15:13.990190+0000","last_unstale":"2024-04-26T21:15:16.593386+0000","last_undegraded":"2024-04-26T21:15:16.593386+0000","last_fullsized":"2024-04-26T21:15:16.593386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:39.803635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593847+0000","last_change":"2024-04-26T21:15:14.997159+0000","last_active":"2024-04-26T21:15:16.593847+0000","last_peered":"2024-04-26T21:15:16.593847+0000","last_clean":"2024-04-26T21:15:16.593847+0000","last_became_active":"2024-04-26T21:15:14.996826+0000","last_became_peered":"2024-04-26T21:15:14.996826+0000","last_unstale":"2024-04-26T21:15:16.593847+0000","last_undegraded":"2024-04-26T21:15:16.593847+0000","last_fullsized":"2024-04-26T21:15:16.593847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998071+0000","last_change":"2024-04-26T21:15:13.995264+0000","last_active":"2024-04-26T21:15:15.998071+0000","last_peered":"2024-04-26T21:15:15.998071+0000","last_clean":"2024-04-26T21:15:15.998071+0000","last_became_active":"2024-04-26T21:15:13.994989+0000","last_became_peered":"2024-04-26T21:15:13.994989+0000","last_unstale":"2024-04-26T21:15:15.998071+0000","last_undegraded":"2024-04-26T21:15:15.998071+0000","last_fullsized":"2024-04-26T21:15:15.998071+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:35.044683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996782+0000","last_change":"2024-04-26T21:15:15.000876+0000","last_active":"2024-04-26T21:15:15.996782+0000","last_peered":"2024-04-26T21:15:15.996782+0000","last_clean":"2024-04-26T21:15:15.996782+0000","last_became_active":"2024-04-26T21:15:15.000234+0000","last_became_peered":"2024-04-26T21:15:15.000234+0000","last_unstale":"2024-04-26T21:15:15.996782+0000","last_undegraded":"2024-04-26T21:15:15.996782+0000","last_fullsized":"2024-04-26T21:15:15.996782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.985094+0000","last_change":"2024-04-26T21:15:13.988753+0000","last_active":"2024-04-26T21:15:16.985094+0000","last_peered":"2024-04-26T21:15:16.985094+0000","last_clean":"2024-04-26T21:15:16.985094+0000","last_became_active":"2024-04-26T21:15:13.988472+0000","last_became_peered":"2024-04-26T21:15:13.988472+0000","last_unstale":"2024-04-26T21:15:16.985094+0000","last_undegraded":"2024-04-26T21:15:16.985094+0000","last_fullsized":"2024-04-26T21:15:16.985094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:17.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594646+0000","last_change":"2024-04-26T21:15:15.007784+0000","last_active":"2024-04-26T21:15:16.594646+0000","last_peered":"2024-04-26T21:15:16.594646+0000","last_clean":"2024-04-26T21:15:16.594646+0000","last_became_active":"2024-04-26T21:15:15.007373+0000","last_became_peered":"2024-04-26T21:15:15.007373+0000","last_unstale":"2024-04-26T21:15:16.594646+0000","last_undegraded":"2024-04-26T21:15:16.594646+0000","last_fullsized":"2024-04-26T21:15:16.594646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:27.459629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993931+0000","last_change":"2024-04-26T21:15:13.985808+0000","last_active":"2024-04-26T21:15:15.993931+0000","last_peered":"2024-04-26T21:15:15.993931+0000","last_clean":"2024-04-26T21:15:15.993931+0000","last_became_active":"2024-04-26T21:15:13.985482+0000","last_became_peered":"2024-04-26T21:15:13.985482+0000","last_unstale":"2024-04-26T21:15:15.993931+0000","last_undegraded":"2024-04-26T21:15:15.993931+0000","last_fullsized":"2024-04-26T21:15:15.993931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:56.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594703+0000","last_change":"2024-04-26T21:15:14.997195+0000","last_active":"2024-04-26T21:15:16.594703+0000","last_peered":"2024-04-26T21:15:16.594703+0000","last_clean":"2024-04-26T21:15:16.594703+0000","last_became_active":"2024-04-26T21:15:14.996889+0000","last_became_peered":"2024-04-26T21:15:14.996889+0000","last_unstale":"2024-04-26T21:15:16.594703+0000","last_undegraded":"2024-04-26T21:15:16.594703+0000","last_fullsized":"2024-04-26T21:15:16.594703+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:03.261175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593581+0000","last_change":"2024-04-26T21:15:14.000033+0000","last_active":"2024-04-26T21:15:16.593581+0000","last_peered":"2024-04-26T21:15:16.593581+0000","last_clean":"2024-04-26T21:15:16.593581+0000","last_became_active":"2024-04-26T21:15:13.999642+0000","last_became_peered":"2024-04-26T21:15:13.999642+0000","last_unstale":"2024-04-26T21:15:16.593581+0000","last_undegraded":"2024-04-26T21:15:16.593581+0000","last_fullsized":"2024-04-26T21:15:16.593581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993953+0000","last_change":"2024-04-26T21:15:14.998038+0000","last_active":"2024-04-26T21:15:15.993953+0000","last_peered":"2024-04-26T21:15:15.993953+0000","last_clean":"2024-04-26T21:15:15.993953+0000","last_became_active":"2024-04-26T21:15:14.997735+0000","last_became_peered":"2024-04-26T21:15:14.997735+0000","last_unstale":"2024-04-26T21:15:15.993953+0000","last_undegraded":"2024-04-26T21:15:15.993953+0000","last_fullsized":"2024-04-26T21:15:15.993953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993592+0000","last_change":"2024-04-26T21:15:13.985339+0000","last_active":"2024-04-26T21:15:15.993592+0000","last_peered":"2024-04-26T21:15:15.993592+0000","last_clean":"2024-04-26T21:15:15.993592+0000","last_became_active":"2024-04-26T21:15:13.984804+0000","last_became_peered":"2024-04-26T21:15:13.984804+0000","last_unstale":"2024-04-26T21:15:15.993592+0000","last_undegraded":"2024-04-26T21:15:15.993592+0000","last_fullsized":"2024-04-26T21:15:15.993592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:08.792790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997529+0000","last_change":"2024-04-26T21:15:15.004362+0000","last_active":"2024-04-26T21:15:15.997529+0000","last_peered":"2024-04-26T21:15:15.997529+0000","last_clean":"2024-04-26T21:15:15.997529+0000","last_became_active":"2024-04-26T21:15:15.003059+0000","last_became_peered":"2024-04-26T21:15:15.003059+0000","last_unstale":"2024-04-26T21:15:15.997529+0000","last_undegraded":"2024-04-26T21:15:15.997529+0000","last_fullsized":"2024-04-26T21:15:15.997529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:04.403950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003690+0000","last_change":"2024-04-26T21:15:14.000183+0000","last_active":"2024-04-26T21:15:14.003690+0000","last_peered":"2024-04-26T21:15:14.003690+0000","last_clean":"2024-04-26T21:15:14.003690+0000","last_became_active":"2024-04-26T21:15:13.999874+0000","last_became_peered":"2024-04-26T21:15:13.999874+0000","last_unstale":"2024-04-26T21:15:14.003690+0000","last_undegraded":"2024-04-26T21:15:14.003690+0000","last_fullsized":"2024-04-26T21:15:14.003690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:18.312063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985817+0000","last_change":"2024-04-26T21:15:13.985817+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985817+0000","last_undegraded":"2024-04-26T21:15:13.985817+0000","last_fullsized":"2024-04-26T21:15:13.985817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593678+0000","last_change":"2024-04-26T21:15:14.000123+0000","last_active":"2024-04-26T21:15:16.593678+0000","last_peered":"2024-04-26T21:15:16.593678+0000","last_clean":"2024-04-26T21:15:16.593678+0000","last_became_active":"2024-04-26T21:15:13.999730+0000","last_became_peered":"2024-04-26T21:15:13.999730+0000","last_unstale":"2024-04-26T21:15:16.593678+0000","last_undegraded":"2024-04-26T21:15:16.593678+0000","last_fullsized":"2024-04-26T21:15:16.593678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:51.654799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997415+0000","last_change":"2024-04-26T21:15:15.004122+0000","last_active":"2024-04-26T21:15:15.997415+0000","last_peered":"2024-04-26T21:15:15.997415+0000","last_clean":"2024-04-26T21:15:15.997415+0000","last_became_active":"2024-04-26T21:15:15.003876+0000","last_became_peered":"2024-04-26T21:15:15.003876+0000","last_unstale":"2024-04-26T21:15:15.997415+0000","last_undegraded":"2024-04-26T21:15:15.997415+0000","last_fullsized":"2024-04-26T21:15:15.997415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:55.716624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998034+0000","last_change":"2024-04-26T21:15:13.992777+0000","last_active":"2024-04-26T21:15:15.998034+0000","last_peered":"2024-04-26T21:15:15.998034+0000","last_clean":"2024-04-26T21:15:15.998034+0000","last_became_active":"2024-04-26T21:15:13.992544+0000","last_became_peered":"2024-04-26T21:15:13.992544+0000","last_unstale":"2024-04-26T21:15:15.998034+0000","last_undegraded":"2024-04-26T21:15:15.998034+0000","last_fullsized":"2024-04-26T21:15:15.998034+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:06.643084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594821+0000","last_change":"2024-04-26T21:15:15.005232+0000","last_active":"2024-04-26T21:15:16.594821+0000","last_peered":"2024-04-26T21:15:16.594821+0000","last_clean":"2024-04-26T21:15:16.594821+0000","last_became_active":"2024-04-26T21:15:15.004904+0000","last_became_peered":"2024-04-26T21:15:15.004904+0000","last_unstale":"2024-04-26T21:15:16.594821+0000","last_undegraded":"2024-04-26T21:15:16.594821+0000","last_fullsized":"2024-04-26T21:15:16.594821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:48.293671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997514+0000","last_change":"2024-04-26T21:15:13.991822+0000","last_active":"2024-04-26T21:15:15.997514+0000","last_peered":"2024-04-26T21:15:15.997514+0000","last_clean":"2024-04-26T21:15:15.997514+0000","last_became_active":"2024-04-26T21:15:13.991508+0000","last_became_peered":"2024-04-26T21:15:13.991508+0000","last_unstale":"2024-04-26T21:15:15.997514+0000","last_undegraded":"2024-04-26T21:15:15.997514+0000","last_fullsized":"2024-04-26T21:15:15.997514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:29.754213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":255,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593633+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:16.593633+0000","last_peered":"2024-04-26T21:15:16.593633+0000","last_clean":"2024-04-26T21:15:16.593633+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:16.593633+0000","last_undegraded":"2024-04-26T21:15:16.593633+0000","last_fullsized":"2024-04-26T21:15:16.593633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":942962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":352070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":352090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":353590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":942346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":338903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:15:19.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:15:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.145+0000 7fcd380ea640 1 Processor -- start 2024-04-26T21:15:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.145+0000 7fcd380ea640 1 -- start start 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd380ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd300b61a0 0x7fcd30160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd380ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b3de0 0x7fcd300720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd380ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd300b4270 con 0x7fcd30145980 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd380ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcd300b5140 con 0x7fcd300b61a0 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd380ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd300b4530 con 0x7fcd300b3de0 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd35685640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd300b61a0 0x7fcd30160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd35685640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd300b61a0 0x7fcd30160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51218/0 (socket says 172.21.15.40:51218) 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd35685640 1 -- 172.21.15.40:0/3470764398 learned_addr learned my addr 172.21.15.40:0/3470764398 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd35685640 1 -- 172.21.15.40:0/3470764398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b3de0 msgr2=0x7fcd300720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.146+0000 7fcd36687640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b3de0 0x7fcd300720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd35685640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b3de0 0x7fcd300720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd35685640 1 -- 172.21.15.40:0/3470764398 >> v1:172.21.15.184:6789/0 conn(0x7fcd30145980 legacy=0x7fcd300b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd35685640 1 -- 172.21.15.40:0/3470764398 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300b4810 con 0x7fcd300b61a0 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd35685640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd300b61a0 0x7fcd30160630 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcd300d0d00 tx=0x7fcd2402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d5885c2c823b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd34e84640 1 -- 172.21.15.40:0/3470764398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24037070 con 0x7fcd300b61a0 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd34e84640 1 -- 172.21.15.40:0/3470764398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2402ae50 con 0x7fcd300b61a0 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.147+0000 7fcd34e84640 1 -- 172.21.15.40:0/3470764398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24032660 con 0x7fcd300b61a0 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 -- 172.21.15.40:0/3470764398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd300b61a0 msgr2=0x7fcd30160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd300b61a0 0x7fcd30160630 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcd300d0d00 tx=0x7fcd2402a140 comp rx=0 tx=0).stop 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 -- 172.21.15.40:0/3470764398 shutdown_connections 2024-04-26T21:15:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b3de0 0x7fcd300720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 --2- 172.21.15.40:0/3470764398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd300b61a0 0x7fcd30160630 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 -- 172.21.15.40:0/3470764398 >> 172.21.15.40:0/3470764398 conn(0x7fcd300d0d30 msgr2=0x7fcd300d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 -- 172.21.15.40:0/3470764398 shutdown_connections 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.148+0000 7fcd380ea640 1 -- 172.21.15.40:0/3470764398 wait complete. 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.149+0000 7fcd380ea640 1 Processor -- start 2024-04-26T21:15:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.149+0000 7fcd380ea640 1 -- start start 2024-04-26T21:15:19.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 0x7fcd3013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 0x7fcd300b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30072610 con 0x7fcd30145980 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300b4dd0 con 0x7fcd300b3de0 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd380ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd30072790 con 0x7fcd300b61a0 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd36687640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd36687640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51234/0 (socket says 172.21.15.40:51234) 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd36687640 1 -- 172.21.15.40:0/1412587517 learned_addr learned my addr 172.21.15.40:0/1412587517 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd35685640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 0x7fcd300b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.150+0000 7fcd35e86640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 0x7fcd3013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 -- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 msgr2=0x7fcd300b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 0x7fcd300b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 -- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 msgr2=0x7fcd3013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 0x7fcd3013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 -- 172.21.15.40:0/1412587517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3013e630 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd35685640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 0x7fcd300b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd35e86640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 0x7fcd3013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd36687640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000cd70 tx=0x7fcd2000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003b90 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd20003d30 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20004240 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3013e910 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.151+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3013eed0 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.152+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3013f0d0 con 0x7fcd30145980 2024-04-26T21:15:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.152+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd200043e0 con 0x7fcd30145980 2024-04-26T21:15:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.153+0000 7fcd2e7fc640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 0x7fcd1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.153+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fcd2008c740 con 0x7fcd30145980 2024-04-26T21:15:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.153+0000 7fcd35e86640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 0x7fcd1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.154+0000 7fcd35e86640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 0x7fcd1c06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800a830 tx=0x7fcd28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.157+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd3013f0d0 con 0x7fcd30145980 2024-04-26T21:15:19.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.338+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd3013e3e0 con 0x7fcd1c06c8f0 2024-04-26T21:15:19.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.345+0000 7fcd2e7fc640 1 -- 172.21.15.40:0/1412587517 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251642 (secure 0 0 0) 0x7fcd3013e3e0 con 0x7fcd1c06c8f0 2024-04-26T21:15:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:19.350 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 msgr2=0x7fcd1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 0x7fcd1c06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800a830 tx=0x7fcd28008040 comp rx=0 tx=0).stop 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 msgr2=0x7fcd3013d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000cd70 tx=0x7fcd2000a660 comp rx=0 tx=0).stop 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 shutdown_connections 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd1c06c8f0 0x7fcd1c06edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd30145980 0x7fcd3013d070 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd300b61a0 0x7fcd300b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 --2- 172.21.15.40:0/1412587517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd300b3de0 0x7fcd3013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 >> 172.21.15.40:0/1412587517 conn(0x7fcd300d0d30 msgr2=0x7fcd300d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 shutdown_connections 2024-04-26T21:15:19.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.348+0000 7fcd380ea640 1 -- 172.21.15.40:0/1412587517 wait complete. 2024-04-26T21:15:19.362 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-26T21:15:18.963801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240620,"kb_used_data":7772,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734100,"statfs":{"total":773094113280,"available":772847718400,"internally_reserved":0,"allocated":7958528,"data_stored":3986077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993743"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594048+0000","last_change":"2024-04-26T21:15:15.008421+0000","last_active":"2024-04-26T21:15:16.594048+0000","last_peered":"2024-04-26T21:15:16.594048+0000","last_clean":"2024-04-26T21:15:16.594048+0000","last_became_active":"2024-04-26T21:15:15.008122+0000","last_became_peered":"2024-04-26T21:15:15.008122+0000","last_unstale":"2024-04-26T21:15:16.594048+0000","last_undegraded":"2024-04-26T21:15:16.594048+0000","last_fullsized":"2024-04-26T21:15:16.594048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:35.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998310+0000","last_change":"2024-04-26T21:15:15.004592+0000","last_active":"2024-04-26T21:15:15.998310+0000","last_peered":"2024-04-26T21:15:15.998310+0000","last_clean":"2024-04-26T21:15:15.998310+0000","last_became_active":"2024-04-26T21:15:15.004314+0000","last_became_peered":"2024-04-26T21:15:15.004314+0000","last_unstale":"2024-04-26T21:15:15.998310+0000","last_undegraded":"2024-04-26T21:15:15.998310+0000","last_fullsized":"2024-04-26T21:15:15.998310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595104+0000","last_change":"2024-04-26T21:15:14.984552+0000","last_active":"2024-04-26T21:15:16.595104+0000","last_peered":"2024-04-26T21:15:16.595104+0000","last_clean":"2024-04-26T21:15:16.595104+0000","last_became_active":"2024-04-26T21:15:14.984224+0000","last_became_peered":"2024-04-26T21:15:14.984224+0000","last_unstale":"2024-04-26T21:15:16.595104+0000","last_undegraded":"2024-04-26T21:15:16.595104+0000","last_fullsized":"2024-04-26T21:15:16.595104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:52.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.986510+0000","last_change":"2024-04-26T21:15:13.986510+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.986510+0000","last_undegraded":"2024-04-26T21:15:13.986510+0000","last_fullsized":"2024-04-26T21:15:13.986510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993499+0000","last_change":"2024-04-26T21:15:15.008338+0000","last_active":"2024-04-26T21:15:15.993499+0000","last_peered":"2024-04-26T21:15:15.993499+0000","last_clean":"2024-04-26T21:15:15.993499+0000","last_became_active":"2024-04-26T21:15:15.008029+0000","last_became_peered":"2024-04-26T21:15:15.008029+0000","last_unstale":"2024-04-26T21:15:15.993499+0000","last_undegraded":"2024-04-26T21:15:15.993499+0000","last_fullsized":"2024-04-26T21:15:15.993499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:02.371753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997414+0000","last_change":"2024-04-26T21:15:15.003006+0000","last_active":"2024-04-26T21:15:15.997414+0000","last_peered":"2024-04-26T21:15:15.997414+0000","last_clean":"2024-04-26T21:15:15.997414+0000","last_became_active":"2024-04-26T21:15:15.002731+0000","last_became_peered":"2024-04-26T21:15:15.002731+0000","last_unstale":"2024-04-26T21:15:15.997414+0000","last_undegraded":"2024-04-26T21:15:15.997414+0000","last_fullsized":"2024-04-26T21:15:15.997414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:45.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996492+0000","last_change":"2024-04-26T21:15:15.000633+0000","last_active":"2024-04-26T21:15:15.996492+0000","last_peered":"2024-04-26T21:15:15.996492+0000","last_clean":"2024-04-26T21:15:15.996492+0000","last_became_active":"2024-04-26T21:15:15.000275+0000","last_became_peered":"2024-04-26T21:15:15.000275+0000","last_unstale":"2024-04-26T21:15:15.996492+0000","last_undegraded":"2024-04-26T21:15:15.996492+0000","last_fullsized":"2024-04-26T21:15:15.996492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:11.788021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997948+0000","last_change":"2024-04-26T21:15:15.002868+0000","last_active":"2024-04-26T21:15:15.997948+0000","last_peered":"2024-04-26T21:15:15.997948+0000","last_clean":"2024-04-26T21:15:15.997948+0000","last_became_active":"2024-04-26T21:15:15.002561+0000","last_became_peered":"2024-04-26T21:15:15.002561+0000","last_unstale":"2024-04-26T21:15:15.997948+0000","last_undegraded":"2024-04-26T21:15:15.997948+0000","last_fullsized":"2024-04-26T21:15:15.997948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994128+0000","last_change":"2024-04-26T21:15:15.004629+0000","last_active":"2024-04-26T21:15:15.994128+0000","last_peered":"2024-04-26T21:15:15.994128+0000","last_clean":"2024-04-26T21:15:15.994128+0000","last_became_active":"2024-04-26T21:15:15.004365+0000","last_became_peered":"2024-04-26T21:15:15.004365+0000","last_unstale":"2024-04-26T21:15:15.994128+0000","last_undegraded":"2024-04-26T21:15:15.994128+0000","last_fullsized":"2024-04-26T21:15:15.994128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:33.056050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594298+0000","last_change":"2024-04-26T21:15:15.005575+0000","last_active":"2024-04-26T21:15:16.594298+0000","last_peered":"2024-04-26T21:15:16.594298+0000","last_clean":"2024-04-26T21:15:16.594298+0000","last_became_active":"2024-04-26T21:15:15.005289+0000","last_became_peered":"2024-04-26T21:15:15.005289+0000","last_unstale":"2024-04-26T21:15:16.594298+0000","last_undegraded":"2024-04-26T21:15:16.594298+0000","last_fullsized":"2024-04-26T21:15:16.594298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:18.088546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594550+0000","last_change":"2024-04-26T21:15:15.000155+0000","last_active":"2024-04-26T21:15:16.594550+0000","last_peered":"2024-04-26T21:15:16.594550+0000","last_clean":"2024-04-26T21:15:16.594550+0000","last_became_active":"2024-04-26T21:15:14.999858+0000","last_became_peered":"2024-04-26T21:15:14.999858+0000","last_unstale":"2024-04-26T21:15:16.594550+0000","last_undegraded":"2024-04-26T21:15:16.594550+0000","last_fullsized":"2024-04-26T21:15:16.594550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:01.080096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985514+0000","last_change":"2024-04-26T21:15:13.985514+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985514+0000","last_undegraded":"2024-04-26T21:15:13.985514+0000","last_fullsized":"2024-04-26T21:15:13.985514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593769+0000","last_change":"2024-04-26T21:15:15.001688+0000","last_active":"2024-04-26T21:15:16.593769+0000","last_peered":"2024-04-26T21:15:16.593769+0000","last_clean":"2024-04-26T21:15:16.593769+0000","last_became_active":"2024-04-26T21:15:15.001372+0000","last_became_peered":"2024-04-26T21:15:15.001372+0000","last_unstale":"2024-04-26T21:15:16.593769+0000","last_undegraded":"2024-04-26T21:15:16.593769+0000","last_fullsized":"2024-04-26T21:15:16.593769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993853+0000","last_change":"2024-04-26T21:15:14.995986+0000","last_active":"2024-04-26T21:15:15.993853+0000","last_peered":"2024-04-26T21:15:15.993853+0000","last_clean":"2024-04-26T21:15:15.993853+0000","last_became_active":"2024-04-26T21:15:14.995673+0000","last_became_peered":"2024-04-26T21:15:14.995673+0000","last_unstale":"2024-04-26T21:15:15.993853+0000","last_undegraded":"2024-04-26T21:15:15.993853+0000","last_fullsized":"2024-04-26T21:15:15.993853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593933+0000","last_change":"2024-04-26T21:15:14.996331+0000","last_active":"2024-04-26T21:15:16.593933+0000","last_peered":"2024-04-26T21:15:16.593933+0000","last_clean":"2024-04-26T21:15:16.593933+0000","last_became_active":"2024-04-26T21:15:14.996044+0000","last_became_peered":"2024-04-26T21:15:14.996044+0000","last_unstale":"2024-04-26T21:15:16.593933+0000","last_undegraded":"2024-04-26T21:15:16.593933+0000","last_fullsized":"2024-04-26T21:15:16.593933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593583+0000","last_change":"2024-04-26T21:15:15.007557+0000","last_active":"2024-04-26T21:15:16.593583+0000","last_peered":"2024-04-26T21:15:16.593583+0000","last_clean":"2024-04-26T21:15:16.593583+0000","last_became_active":"2024-04-26T21:15:15.007269+0000","last_became_peered":"2024-04-26T21:15:15.007269+0000","last_unstale":"2024-04-26T21:15:16.593583+0000","last_undegraded":"2024-04-26T21:15:16.593583+0000","last_fullsized":"2024-04-26T21:15:16.593583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994100+0000","last_change":"2024-04-26T21:15:15.003937+0000","last_active":"2024-04-26T21:15:15.994100+0000","last_peered":"2024-04-26T21:15:15.994100+0000","last_clean":"2024-04-26T21:15:15.994100+0000","last_became_active":"2024-04-26T21:15:15.003652+0000","last_became_peered":"2024-04-26T21:15:15.003652+0000","last_unstale":"2024-04-26T21:15:15.994100+0000","last_undegraded":"2024-04-26T21:15:15.994100+0000","last_fullsized":"2024-04-26T21:15:15.994100+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.457492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985659+0000","last_change":"2024-04-26T21:15:13.985659+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985659+0000","last_undegraded":"2024-04-26T21:15:13.985659+0000","last_fullsized":"2024-04-26T21:15:13.985659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997884+0000","last_change":"2024-04-26T21:15:15.003066+0000","last_active":"2024-04-26T21:15:15.997884+0000","last_peered":"2024-04-26T21:15:15.997884+0000","last_clean":"2024-04-26T21:15:15.997884+0000","last_became_active":"2024-04-26T21:15:15.002597+0000","last_became_peered":"2024-04-26T21:15:15.002597+0000","last_unstale":"2024-04-26T21:15:15.997884+0000","last_undegraded":"2024-04-26T21:15:15.997884+0000","last_fullsized":"2024-04-26T21:15:15.997884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593869+0000","last_change":"2024-04-26T21:15:14.996454+0000","last_active":"2024-04-26T21:15:16.593869+0000","last_peered":"2024-04-26T21:15:16.593869+0000","last_clean":"2024-04-26T21:15:16.593869+0000","last_became_active":"2024-04-26T21:15:14.996140+0000","last_became_peered":"2024-04-26T21:15:14.996140+0000","last_unstale":"2024-04-26T21:15:16.593869+0000","last_undegraded":"2024-04-26T21:15:16.593869+0000","last_fullsized":"2024-04-26T21:15:16.593869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593784+0000","last_change":"2024-04-26T21:15:15.004956+0000","last_active":"2024-04-26T21:15:16.593784+0000","last_peered":"2024-04-26T21:15:16.593784+0000","last_clean":"2024-04-26T21:15:16.593784+0000","last_became_active":"2024-04-26T21:15:15.004666+0000","last_became_peered":"2024-04-26T21:15:15.004666+0000","last_unstale":"2024-04-26T21:15:16.593784+0000","last_undegraded":"2024-04-26T21:15:16.593784+0000","last_fullsized":"2024-04-26T21:15:16.593784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593651+0000","last_change":"2024-04-26T21:15:14.998310+0000","last_active":"2024-04-26T21:15:16.593651+0000","last_peered":"2024-04-26T21:15:16.593651+0000","last_clean":"2024-04-26T21:15:16.593651+0000","last_became_active":"2024-04-26T21:15:14.998039+0000","last_became_peered":"2024-04-26T21:15:14.998039+0000","last_unstale":"2024-04-26T21:15:16.593651+0000","last_undegraded":"2024-04-26T21:15:16.593651+0000","last_fullsized":"2024-04-26T21:15:16.593651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:14.997132+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:14.996824+0000","last_became_peered":"2024-04-26T21:15:14.996824+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593965+0000","last_change":"2024-04-26T21:15:15.003818+0000","last_active":"2024-04-26T21:15:16.593965+0000","last_peered":"2024-04-26T21:15:16.593965+0000","last_clean":"2024-04-26T21:15:16.593965+0000","last_became_active":"2024-04-26T21:15:15.003428+0000","last_became_peered":"2024-04-26T21:15:15.003428+0000","last_unstale":"2024-04-26T21:15:16.593965+0000","last_undegraded":"2024-04-26T21:15:16.593965+0000","last_fullsized":"2024-04-26T21:15:16.593965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594069+0000","last_change":"2024-04-26T21:15:15.004314+0000","last_active":"2024-04-26T21:15:16.594069+0000","last_peered":"2024-04-26T21:15:16.594069+0000","last_clean":"2024-04-26T21:15:16.594069+0000","last_became_active":"2024-04-26T21:15:15.004022+0000","last_became_peered":"2024-04-26T21:15:15.004022+0000","last_unstale":"2024-04-26T21:15:16.594069+0000","last_undegraded":"2024-04-26T21:15:16.594069+0000","last_fullsized":"2024-04-26T21:15:16.594069+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996953+0000","last_change":"2024-04-26T21:15:15.006413+0000","last_active":"2024-04-26T21:15:15.996953+0000","last_peered":"2024-04-26T21:15:15.996953+0000","last_clean":"2024-04-26T21:15:15.996953+0000","last_became_active":"2024-04-26T21:15:15.006136+0000","last_became_peered":"2024-04-26T21:15:15.006136+0000","last_unstale":"2024-04-26T21:15:15.996953+0000","last_undegraded":"2024-04-26T21:15:15.996953+0000","last_fullsized":"2024-04-26T21:15:15.996953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595237+0000","last_change":"2024-04-26T21:15:14.999635+0000","last_active":"2024-04-26T21:15:16.595237+0000","last_peered":"2024-04-26T21:15:16.595237+0000","last_clean":"2024-04-26T21:15:16.595237+0000","last_became_active":"2024-04-26T21:15:14.999351+0000","last_became_peered":"2024-04-26T21:15:14.999351+0000","last_unstale":"2024-04-26T21:15:16.595237+0000","last_undegraded":"2024-04-26T21:15:16.595237+0000","last_fullsized":"2024-04-26T21:15:16.595237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:23.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594169+0000","last_change":"2024-04-26T21:15:15.006522+0000","last_active":"2024-04-26T21:15:16.594169+0000","last_peered":"2024-04-26T21:15:16.594169+0000","last_clean":"2024-04-26T21:15:16.594169+0000","last_became_active":"2024-04-26T21:15:15.005849+0000","last_became_peered":"2024-04-26T21:15:15.005849+0000","last_unstale":"2024-04-26T21:15:16.594169+0000","last_undegraded":"2024-04-26T21:15:16.594169+0000","last_fullsized":"2024-04-26T21:15:16.594169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:16.396516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594834+0000","last_change":"2024-04-26T21:15:14.988314+0000","last_active":"2024-04-26T21:15:16.594834+0000","last_peered":"2024-04-26T21:15:16.594834+0000","last_clean":"2024-04-26T21:15:16.594834+0000","last_became_active":"2024-04-26T21:15:14.987983+0000","last_became_peered":"2024-04-26T21:15:14.987983+0000","last_unstale":"2024-04-26T21:15:16.594834+0000","last_undegraded":"2024-04-26T21:15:16.594834+0000","last_fullsized":"2024-04-26T21:15:16.594834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:56.347854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996624+0000","last_change":"2024-04-26T21:15:15.001705+0000","last_active":"2024-04-26T21:15:15.996624+0000","last_peered":"2024-04-26T21:15:15.996624+0000","last_clean":"2024-04-26T21:15:15.996624+0000","last_became_active":"2024-04-26T21:15:15.001398+0000","last_became_peered":"2024-04-26T21:15:15.001398+0000","last_unstale":"2024-04-26T21:15:15.996624+0000","last_undegraded":"2024-04-26T21:15:15.996624+0000","last_fullsized":"2024-04-26T21:15:15.996624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996883+0000","last_change":"2024-04-26T21:15:14.996277+0000","last_active":"2024-04-26T21:15:15.996883+0000","last_peered":"2024-04-26T21:15:15.996883+0000","last_clean":"2024-04-26T21:15:15.996883+0000","last_became_active":"2024-04-26T21:15:14.996006+0000","last_became_peered":"2024-04-26T21:15:14.996006+0000","last_unstale":"2024-04-26T21:15:15.996883+0000","last_undegraded":"2024-04-26T21:15:15.996883+0000","last_fullsized":"2024-04-26T21:15:15.996883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997797+0000","last_change":"2024-04-26T21:15:15.005406+0000","last_active":"2024-04-26T21:15:15.997797+0000","last_peered":"2024-04-26T21:15:15.997797+0000","last_clean":"2024-04-26T21:15:15.997797+0000","last_became_active":"2024-04-26T21:15:15.005116+0000","last_became_peered":"2024-04-26T21:15:15.005116+0000","last_unstale":"2024-04-26T21:15:15.997797+0000","last_undegraded":"2024-04-26T21:15:15.997797+0000","last_fullsized":"2024-04-26T21:15:15.997797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:16.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594603+0000","last_change":"2024-04-26T21:15:15.008320+0000","last_active":"2024-04-26T21:15:16.594603+0000","last_peered":"2024-04-26T21:15:16.594603+0000","last_clean":"2024-04-26T21:15:16.594603+0000","last_became_active":"2024-04-26T21:15:15.008048+0000","last_became_peered":"2024-04-26T21:15:15.008048+0000","last_unstale":"2024-04-26T21:15:16.594603+0000","last_undegraded":"2024-04-26T21:15:16.594603+0000","last_fullsized":"2024-04-26T21:15:16.594603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.353647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594502+0000","last_change":"2024-04-26T21:15:13.991660+0000","last_active":"2024-04-26T21:15:16.594502+0000","last_peered":"2024-04-26T21:15:16.594502+0000","last_clean":"2024-04-26T21:15:16.594502+0000","last_became_active":"2024-04-26T21:15:13.991361+0000","last_became_peered":"2024-04-26T21:15:13.991361+0000","last_unstale":"2024-04-26T21:15:16.594502+0000","last_undegraded":"2024-04-26T21:15:16.594502+0000","last_fullsized":"2024-04-26T21:15:16.594502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994190+0000","last_change":"2024-04-26T21:15:15.003269+0000","last_active":"2024-04-26T21:15:15.994190+0000","last_peered":"2024-04-26T21:15:15.994190+0000","last_clean":"2024-04-26T21:15:15.994190+0000","last_became_active":"2024-04-26T21:15:15.002973+0000","last_became_peered":"2024-04-26T21:15:15.002973+0000","last_unstale":"2024-04-26T21:15:15.994190+0000","last_undegraded":"2024-04-26T21:15:15.994190+0000","last_fullsized":"2024-04-26T21:15:15.994190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:30.756090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593977+0000","last_change":"2024-04-26T21:15:13.990425+0000","last_active":"2024-04-26T21:15:16.593977+0000","last_peered":"2024-04-26T21:15:16.593977+0000","last_clean":"2024-04-26T21:15:16.593977+0000","last_became_active":"2024-04-26T21:15:13.990113+0000","last_became_peered":"2024-04-26T21:15:13.990113+0000","last_unstale":"2024-04-26T21:15:16.593977+0000","last_undegraded":"2024-04-26T21:15:16.593977+0000","last_fullsized":"2024-04-26T21:15:16.593977+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997715+0000","last_change":"2024-04-26T21:15:15.004144+0000","last_active":"2024-04-26T21:15:15.997715+0000","last_peered":"2024-04-26T21:15:15.997715+0000","last_clean":"2024-04-26T21:15:15.997715+0000","last_became_active":"2024-04-26T21:15:15.003804+0000","last_became_peered":"2024-04-26T21:15:15.003804+0000","last_unstale":"2024-04-26T21:15:15.997715+0000","last_undegraded":"2024-04-26T21:15:15.997715+0000","last_fullsized":"2024-04-26T21:15:15.997715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:34.213418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594471+0000","last_change":"2024-04-26T21:15:13.996784+0000","last_active":"2024-04-26T21:15:16.594471+0000","last_peered":"2024-04-26T21:15:16.594471+0000","last_clean":"2024-04-26T21:15:16.594471+0000","last_became_active":"2024-04-26T21:15:13.996508+0000","last_became_peered":"2024-04-26T21:15:13.996508+0000","last_unstale":"2024-04-26T21:15:16.594471+0000","last_undegraded":"2024-04-26T21:15:16.594471+0000","last_fullsized":"2024-04-26T21:15:16.594471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:31.090506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594503+0000","last_change":"2024-04-26T21:15:14.998880+0000","last_active":"2024-04-26T21:15:16.594503+0000","last_peered":"2024-04-26T21:15:16.594503+0000","last_clean":"2024-04-26T21:15:16.594503+0000","last_became_active":"2024-04-26T21:15:14.998406+0000","last_became_peered":"2024-04-26T21:15:14.998406+0000","last_unstale":"2024-04-26T21:15:16.594503+0000","last_undegraded":"2024-04-26T21:15:16.594503+0000","last_fullsized":"2024-04-26T21:15:16.594503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:32.918934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997990+0000","last_change":"2024-04-26T21:15:13.990545+0000","last_active":"2024-04-26T21:15:15.997990+0000","last_peered":"2024-04-26T21:15:15.997990+0000","last_clean":"2024-04-26T21:15:15.997990+0000","last_became_active":"2024-04-26T21:15:13.990151+0000","last_became_peered":"2024-04-26T21:15:13.990151+0000","last_unstale":"2024-04-26T21:15:15.997990+0000","last_undegraded":"2024-04-26T21:15:15.997990+0000","last_fullsized":"2024-04-26T21:15:15.997990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:43.327070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996712+0000","last_change":"2024-04-26T21:15:14.999455+0000","last_active":"2024-04-26T21:15:15.996712+0000","last_peered":"2024-04-26T21:15:15.996712+0000","last_clean":"2024-04-26T21:15:15.996712+0000","last_became_active":"2024-04-26T21:15:14.999063+0000","last_became_peered":"2024-04-26T21:15:14.999063+0000","last_unstale":"2024-04-26T21:15:15.996712+0000","last_undegraded":"2024-04-26T21:15:15.996712+0000","last_fullsized":"2024-04-26T21:15:15.996712+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595004+0000","last_change":"2024-04-26T21:15:14.000741+0000","last_active":"2024-04-26T21:15:16.595004+0000","last_peered":"2024-04-26T21:15:16.595004+0000","last_clean":"2024-04-26T21:15:16.595004+0000","last_became_active":"2024-04-26T21:15:14.000484+0000","last_became_peered":"2024-04-26T21:15:14.000484+0000","last_unstale":"2024-04-26T21:15:16.595004+0000","last_undegraded":"2024-04-26T21:15:16.595004+0000","last_fullsized":"2024-04-26T21:15:16.595004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:34.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:15.003590+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:15.003252+0000","last_became_peered":"2024-04-26T21:15:15.003252+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:40.805512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997494+0000","last_change":"2024-04-26T21:15:13.987191+0000","last_active":"2024-04-26T21:15:15.997494+0000","last_peered":"2024-04-26T21:15:15.997494+0000","last_clean":"2024-04-26T21:15:15.997494+0000","last_became_active":"2024-04-26T21:15:13.986872+0000","last_became_peered":"2024-04-26T21:15:13.986872+0000","last_unstale":"2024-04-26T21:15:15.997494+0000","last_undegraded":"2024-04-26T21:15:15.997494+0000","last_fullsized":"2024-04-26T21:15:15.997494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:02.357589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993432+0000","last_change":"2024-04-26T21:15:15.002302+0000","last_active":"2024-04-26T21:15:15.993432+0000","last_peered":"2024-04-26T21:15:15.993432+0000","last_clean":"2024-04-26T21:15:15.993432+0000","last_became_active":"2024-04-26T21:15:15.001311+0000","last_became_peered":"2024-04-26T21:15:15.001311+0000","last_unstale":"2024-04-26T21:15:15.993432+0000","last_undegraded":"2024-04-26T21:15:15.993432+0000","last_fullsized":"2024-04-26T21:15:15.993432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:08.309553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003614+0000","last_change":"2024-04-26T21:15:13.998731+0000","last_active":"2024-04-26T21:15:14.003614+0000","last_peered":"2024-04-26T21:15:14.003614+0000","last_clean":"2024-04-26T21:15:14.003614+0000","last_became_active":"2024-04-26T21:15:13.998420+0000","last_became_peered":"2024-04-26T21:15:13.998420+0000","last_unstale":"2024-04-26T21:15:14.003614+0000","last_undegraded":"2024-04-26T21:15:14.003614+0000","last_fullsized":"2024-04-26T21:15:14.003614+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:03.412018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593528+0000","last_change":"2024-04-26T21:15:14.999599+0000","last_active":"2024-04-26T21:15:16.593528+0000","last_peered":"2024-04-26T21:15:16.593528+0000","last_clean":"2024-04-26T21:15:16.593528+0000","last_became_active":"2024-04-26T21:15:14.998302+0000","last_became_peered":"2024-04-26T21:15:14.998302+0000","last_unstale":"2024-04-26T21:15:16.593528+0000","last_undegraded":"2024-04-26T21:15:16.593528+0000","last_fullsized":"2024-04-26T21:15:16.593528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993426+0000","last_change":"2024-04-26T21:15:13.989877+0000","last_active":"2024-04-26T21:15:15.993426+0000","last_peered":"2024-04-26T21:15:15.993426+0000","last_clean":"2024-04-26T21:15:15.993426+0000","last_became_active":"2024-04-26T21:15:13.989441+0000","last_became_peered":"2024-04-26T21:15:13.989441+0000","last_unstale":"2024-04-26T21:15:15.993426+0000","last_undegraded":"2024-04-26T21:15:15.993426+0000","last_fullsized":"2024-04-26T21:15:15.993426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:41.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997022+0000","last_change":"2024-04-26T21:15:14.999473+0000","last_active":"2024-04-26T21:15:15.997022+0000","last_peered":"2024-04-26T21:15:15.997022+0000","last_clean":"2024-04-26T21:15:15.997022+0000","last_became_active":"2024-04-26T21:15:14.999179+0000","last_became_peered":"2024-04-26T21:15:14.999179+0000","last_unstale":"2024-04-26T21:15:15.997022+0000","last_undegraded":"2024-04-26T21:15:15.997022+0000","last_fullsized":"2024-04-26T21:15:15.997022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594171+0000","last_change":"2024-04-26T21:15:13.999674+0000","last_active":"2024-04-26T21:15:16.594171+0000","last_peered":"2024-04-26T21:15:16.594171+0000","last_clean":"2024-04-26T21:15:16.594171+0000","last_became_active":"2024-04-26T21:15:13.999384+0000","last_became_peered":"2024-04-26T21:15:13.999384+0000","last_unstale":"2024-04-26T21:15:16.594171+0000","last_undegraded":"2024-04-26T21:15:16.594171+0000","last_fullsized":"2024-04-26T21:15:16.594171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:41.421624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594232+0000","last_change":"2024-04-26T21:15:15.004056+0000","last_active":"2024-04-26T21:15:16.594232+0000","last_peered":"2024-04-26T21:15:16.594232+0000","last_clean":"2024-04-26T21:15:16.594232+0000","last_became_active":"2024-04-26T21:15:15.003109+0000","last_became_peered":"2024-04-26T21:15:15.003109+0000","last_unstale":"2024-04-26T21:15:16.594232+0000","last_undegraded":"2024-04-26T21:15:16.594232+0000","last_fullsized":"2024-04-26T21:15:16.594232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.332709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998354+0000","last_change":"2024-04-26T21:15:13.988307+0000","last_active":"2024-04-26T21:15:15.998354+0000","last_peered":"2024-04-26T21:15:15.998354+0000","last_clean":"2024-04-26T21:15:15.998354+0000","last_became_active":"2024-04-26T21:15:13.988018+0000","last_became_peered":"2024-04-26T21:15:13.988018+0000","last_unstale":"2024-04-26T21:15:15.998354+0000","last_undegraded":"2024-04-26T21:15:15.998354+0000","last_fullsized":"2024-04-26T21:15:15.998354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.077866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997865+0000","last_change":"2024-04-26T21:15:15.004543+0000","last_active":"2024-04-26T21:15:15.997865+0000","last_peered":"2024-04-26T21:15:15.997865+0000","last_clean":"2024-04-26T21:15:15.997865+0000","last_became_active":"2024-04-26T21:15:15.004281+0000","last_became_peered":"2024-04-26T21:15:15.004281+0000","last_unstale":"2024-04-26T21:15:15.997865+0000","last_undegraded":"2024-04-26T21:15:15.997865+0000","last_fullsized":"2024-04-26T21:15:15.997865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595209+0000","last_change":"2024-04-26T21:15:13.991260+0000","last_active":"2024-04-26T21:15:16.595209+0000","last_peered":"2024-04-26T21:15:16.595209+0000","last_clean":"2024-04-26T21:15:16.595209+0000","last_became_active":"2024-04-26T21:15:13.990969+0000","last_became_peered":"2024-04-26T21:15:13.990969+0000","last_unstale":"2024-04-26T21:15:16.595209+0000","last_undegraded":"2024-04-26T21:15:16.595209+0000","last_fullsized":"2024-04-26T21:15:16.595209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:35.822660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993865+0000","last_change":"2024-04-26T21:15:14.997788+0000","last_active":"2024-04-26T21:15:15.993865+0000","last_peered":"2024-04-26T21:15:15.993865+0000","last_clean":"2024-04-26T21:15:15.993865+0000","last_became_active":"2024-04-26T21:15:14.997464+0000","last_became_peered":"2024-04-26T21:15:14.997464+0000","last_unstale":"2024-04-26T21:15:15.993865+0000","last_undegraded":"2024-04-26T21:15:15.993865+0000","last_fullsized":"2024-04-26T21:15:15.993865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:56.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994753+0000","last_change":"2024-04-26T21:15:13.991679+0000","last_active":"2024-04-26T21:15:13.994753+0000","last_peered":"2024-04-26T21:15:13.994753+0000","last_clean":"2024-04-26T21:15:13.994753+0000","last_became_active":"2024-04-26T21:15:13.991338+0000","last_became_peered":"2024-04-26T21:15:13.991338+0000","last_unstale":"2024-04-26T21:15:13.994753+0000","last_undegraded":"2024-04-26T21:15:13.994753+0000","last_fullsized":"2024-04-26T21:15:13.994753+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:32.732498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:14.999960+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:14.999673+0000","last_became_peered":"2024-04-26T21:15:14.999673+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:36.824537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994639+0000","last_change":"2024-04-26T21:15:13.990261+0000","last_active":"2024-04-26T21:15:13.994639+0000","last_peered":"2024-04-26T21:15:13.994639+0000","last_clean":"2024-04-26T21:15:13.994639+0000","last_became_active":"2024-04-26T21:15:13.989888+0000","last_became_peered":"2024-04-26T21:15:13.989888+0000","last_unstale":"2024-04-26T21:15:13.994639+0000","last_undegraded":"2024-04-26T21:15:13.994639+0000","last_fullsized":"2024-04-26T21:15:13.994639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:22.729662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996901+0000","last_change":"2024-04-26T21:15:15.000394+0000","last_active":"2024-04-26T21:15:15.996901+0000","last_peered":"2024-04-26T21:15:15.996901+0000","last_clean":"2024-04-26T21:15:15.996901+0000","last_became_active":"2024-04-26T21:15:14.999894+0000","last_became_peered":"2024-04-26T21:15:14.999894+0000","last_unstale":"2024-04-26T21:15:15.996901+0000","last_undegraded":"2024-04-26T21:15:15.996901+0000","last_fullsized":"2024-04-26T21:15:15.996901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997598+0000","last_change":"2024-04-26T21:15:13.989609+0000","last_active":"2024-04-26T21:15:15.997598+0000","last_peered":"2024-04-26T21:15:15.997598+0000","last_clean":"2024-04-26T21:15:15.997598+0000","last_became_active":"2024-04-26T21:15:13.989419+0000","last_became_peered":"2024-04-26T21:15:13.989419+0000","last_unstale":"2024-04-26T21:15:15.997598+0000","last_undegraded":"2024-04-26T21:15:15.997598+0000","last_fullsized":"2024-04-26T21:15:15.997598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:36.252437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593466+0000","last_change":"2024-04-26T21:15:15.005165+0000","last_active":"2024-04-26T21:15:16.593466+0000","last_peered":"2024-04-26T21:15:16.593466+0000","last_clean":"2024-04-26T21:15:16.593466+0000","last_became_active":"2024-04-26T21:15:15.004821+0000","last_became_peered":"2024-04-26T21:15:15.004821+0000","last_unstale":"2024-04-26T21:15:16.593466+0000","last_undegraded":"2024-04-26T21:15:16.593466+0000","last_fullsized":"2024-04-26T21:15:16.593466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996529+0000","last_change":"2024-04-26T21:15:13.984946+0000","last_active":"2024-04-26T21:15:15.996529+0000","last_peered":"2024-04-26T21:15:15.996529+0000","last_clean":"2024-04-26T21:15:15.996529+0000","last_became_active":"2024-04-26T21:15:13.984641+0000","last_became_peered":"2024-04-26T21:15:13.984641+0000","last_unstale":"2024-04-26T21:15:15.996529+0000","last_undegraded":"2024-04-26T21:15:15.996529+0000","last_fullsized":"2024-04-26T21:15:15.996529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993537+0000","last_change":"2024-04-26T21:15:15.007343+0000","last_active":"2024-04-26T21:15:15.993537+0000","last_peered":"2024-04-26T21:15:15.993537+0000","last_clean":"2024-04-26T21:15:15.993537+0000","last_became_active":"2024-04-26T21:15:15.007015+0000","last_became_peered":"2024-04-26T21:15:15.007015+0000","last_unstale":"2024-04-26T21:15:15.993537+0000","last_undegraded":"2024-04-26T21:15:15.993537+0000","last_fullsized":"2024-04-26T21:15:15.993537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:34.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593601+0000","last_change":"2024-04-26T21:15:13.996573+0000","last_active":"2024-04-26T21:15:16.593601+0000","last_peered":"2024-04-26T21:15:16.593601+0000","last_clean":"2024-04-26T21:15:16.593601+0000","last_became_active":"2024-04-26T21:15:13.996295+0000","last_became_peered":"2024-04-26T21:15:13.996295+0000","last_unstale":"2024-04-26T21:15:16.593601+0000","last_undegraded":"2024-04-26T21:15:16.593601+0000","last_fullsized":"2024-04-26T21:15:16.593601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:42.351528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594435+0000","last_change":"2024-04-26T21:15:15.001042+0000","last_active":"2024-04-26T21:15:16.594435+0000","last_peered":"2024-04-26T21:15:16.594435+0000","last_clean":"2024-04-26T21:15:16.594435+0000","last_became_active":"2024-04-26T21:15:15.000753+0000","last_became_peered":"2024-04-26T21:15:15.000753+0000","last_unstale":"2024-04-26T21:15:16.594435+0000","last_undegraded":"2024-04-26T21:15:16.594435+0000","last_fullsized":"2024-04-26T21:15:16.594435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.434213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998075+0000","last_change":"2024-04-26T21:15:13.993704+0000","last_active":"2024-04-26T21:15:15.998075+0000","last_peered":"2024-04-26T21:15:15.998075+0000","last_clean":"2024-04-26T21:15:15.998075+0000","last_became_active":"2024-04-26T21:15:13.993410+0000","last_became_peered":"2024-04-26T21:15:13.993410+0000","last_unstale":"2024-04-26T21:15:15.998075+0000","last_undegraded":"2024-04-26T21:15:15.998075+0000","last_fullsized":"2024-04-26T21:15:15.998075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594517+0000","last_change":"2024-04-26T21:15:14.998082+0000","last_active":"2024-04-26T21:15:16.594517+0000","last_peered":"2024-04-26T21:15:16.594517+0000","last_clean":"2024-04-26T21:15:16.594517+0000","last_became_active":"2024-04-26T21:15:14.997733+0000","last_became_peered":"2024-04-26T21:15:14.997733+0000","last_unstale":"2024-04-26T21:15:16.594517+0000","last_undegraded":"2024-04-26T21:15:16.594517+0000","last_fullsized":"2024-04-26T21:15:16.594517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:07.215468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594639+0000","last_change":"2024-04-26T21:15:14.001697+0000","last_active":"2024-04-26T21:15:16.594639+0000","last_peered":"2024-04-26T21:15:16.594639+0000","last_clean":"2024-04-26T21:15:16.594639+0000","last_became_active":"2024-04-26T21:15:14.001416+0000","last_became_peered":"2024-04-26T21:15:14.001416+0000","last_unstale":"2024-04-26T21:15:16.594639+0000","last_undegraded":"2024-04-26T21:15:16.594639+0000","last_fullsized":"2024-04-26T21:15:16.594639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:11.110564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985044+0000","last_change":"2024-04-26T21:15:13.985044+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985044+0000","last_undegraded":"2024-04-26T21:15:13.985044+0000","last_fullsized":"2024-04-26T21:15:13.985044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997916+0000","last_change":"2024-04-26T21:15:13.997192+0000","last_active":"2024-04-26T21:15:15.997916+0000","last_peered":"2024-04-26T21:15:15.997916+0000","last_clean":"2024-04-26T21:15:15.997916+0000","last_became_active":"2024-04-26T21:15:13.996946+0000","last_became_peered":"2024-04-26T21:15:13.996946+0000","last_unstale":"2024-04-26T21:15:15.997916+0000","last_undegraded":"2024-04-26T21:15:15.997916+0000","last_fullsized":"2024-04-26T21:15:15.997916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:33.005058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993826+0000","last_change":"2024-04-26T21:15:15.000551+0000","last_active":"2024-04-26T21:15:15.993826+0000","last_peered":"2024-04-26T21:15:15.993826+0000","last_clean":"2024-04-26T21:15:15.993826+0000","last_became_active":"2024-04-26T21:15:15.000269+0000","last_became_peered":"2024-04-26T21:15:15.000269+0000","last_unstale":"2024-04-26T21:15:15.993826+0000","last_undegraded":"2024-04-26T21:15:15.993826+0000","last_fullsized":"2024-04-26T21:15:15.993826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:46.479309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594507+0000","last_change":"2024-04-26T21:15:14.000103+0000","last_active":"2024-04-26T21:15:16.594507+0000","last_peered":"2024-04-26T21:15:16.594507+0000","last_clean":"2024-04-26T21:15:16.594507+0000","last_became_active":"2024-04-26T21:15:13.999824+0000","last_became_peered":"2024-04-26T21:15:13.999824+0000","last_unstale":"2024-04-26T21:15:16.594507+0000","last_undegraded":"2024-04-26T21:15:16.594507+0000","last_fullsized":"2024-04-26T21:15:16.594507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595067+0000","last_change":"2024-04-26T21:15:14.999573+0000","last_active":"2024-04-26T21:15:16.595067+0000","last_peered":"2024-04-26T21:15:16.595067+0000","last_clean":"2024-04-26T21:15:16.595067+0000","last_became_active":"2024-04-26T21:15:14.999265+0000","last_became_peered":"2024-04-26T21:15:14.999265+0000","last_unstale":"2024-04-26T21:15:16.595067+0000","last_undegraded":"2024-04-26T21:15:16.595067+0000","last_fullsized":"2024-04-26T21:15:16.595067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:49.825951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997655+0000","last_change":"2024-04-26T21:15:13.987004+0000","last_active":"2024-04-26T21:15:15.997655+0000","last_peered":"2024-04-26T21:15:15.997655+0000","last_clean":"2024-04-26T21:15:15.997655+0000","last_became_active":"2024-04-26T21:15:13.986676+0000","last_became_peered":"2024-04-26T21:15:13.986676+0000","last_unstale":"2024-04-26T21:15:15.997655+0000","last_undegraded":"2024-04-26T21:15:15.997655+0000","last_fullsized":"2024-04-26T21:15:15.997655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:44.090460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594645+0000","last_change":"2024-04-26T21:15:15.004474+0000","last_active":"2024-04-26T21:15:16.594645+0000","last_peered":"2024-04-26T21:15:16.594645+0000","last_clean":"2024-04-26T21:15:16.594645+0000","last_became_active":"2024-04-26T21:15:15.004185+0000","last_became_peered":"2024-04-26T21:15:15.004185+0000","last_unstale":"2024-04-26T21:15:16.594645+0000","last_undegraded":"2024-04-26T21:15:16.594645+0000","last_fullsized":"2024-04-26T21:15:16.594645+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.298412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996654+0000","last_change":"2024-04-26T21:15:13.988682+0000","last_active":"2024-04-26T21:15:15.996654+0000","last_peered":"2024-04-26T21:15:15.996654+0000","last_clean":"2024-04-26T21:15:15.996654+0000","last_became_active":"2024-04-26T21:15:13.988377+0000","last_became_peered":"2024-04-26T21:15:13.988377+0000","last_unstale":"2024-04-26T21:15:15.996654+0000","last_undegraded":"2024-04-26T21:15:15.996654+0000","last_fullsized":"2024-04-26T21:15:15.996654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:15.394639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594757+0000","last_change":"2024-04-26T21:15:15.005522+0000","last_active":"2024-04-26T21:15:16.594757+0000","last_peered":"2024-04-26T21:15:16.594757+0000","last_clean":"2024-04-26T21:15:16.594757+0000","last_became_active":"2024-04-26T21:15:15.005254+0000","last_became_peered":"2024-04-26T21:15:15.005254+0000","last_unstale":"2024-04-26T21:15:16.594757+0000","last_undegraded":"2024-04-26T21:15:16.594757+0000","last_fullsized":"2024-04-26T21:15:16.594757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.398672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593386+0000","last_change":"2024-04-26T21:15:13.990434+0000","last_active":"2024-04-26T21:15:16.593386+0000","last_peered":"2024-04-26T21:15:16.593386+0000","last_clean":"2024-04-26T21:15:16.593386+0000","last_became_active":"2024-04-26T21:15:13.990190+0000","last_became_peered":"2024-04-26T21:15:13.990190+0000","last_unstale":"2024-04-26T21:15:16.593386+0000","last_undegraded":"2024-04-26T21:15:16.593386+0000","last_fullsized":"2024-04-26T21:15:16.593386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:39.803635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593847+0000","last_change":"2024-04-26T21:15:14.997159+0000","last_active":"2024-04-26T21:15:16.593847+0000","last_peered":"2024-04-26T21:15:16.593847+0000","last_clean":"2024-04-26T21:15:16.593847+0000","last_became_active":"2024-04-26T21:15:14.996826+0000","last_became_peered":"2024-04-26T21:15:14.996826+0000","last_unstale":"2024-04-26T21:15:16.593847+0000","last_undegraded":"2024-04-26T21:15:16.593847+0000","last_fullsized":"2024-04-26T21:15:16.593847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998071+0000","last_change":"2024-04-26T21:15:13.995264+0000","last_active":"2024-04-26T21:15:15.998071+0000","last_peered":"2024-04-26T21:15:15.998071+0000","last_clean":"2024-04-26T21:15:15.998071+0000","last_became_active":"2024-04-26T21:15:13.994989+0000","last_became_peered":"2024-04-26T21:15:13.994989+0000","last_unstale":"2024-04-26T21:15:15.998071+0000","last_undegraded":"2024-04-26T21:15:15.998071+0000","last_fullsized":"2024-04-26T21:15:15.998071+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:35.044683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996782+0000","last_change":"2024-04-26T21:15:15.000876+0000","last_active":"2024-04-26T21:15:15.996782+0000","last_peered":"2024-04-26T21:15:15.996782+0000","last_clean":"2024-04-26T21:15:15.996782+0000","last_became_active":"2024-04-26T21:15:15.000234+0000","last_became_peered":"2024-04-26T21:15:15.000234+0000","last_unstale":"2024-04-26T21:15:15.996782+0000","last_undegraded":"2024-04-26T21:15:15.996782+0000","last_fullsized":"2024-04-26T21:15:15.996782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.985094+0000","last_change":"2024-04-26T21:15:13.988753+0000","last_active":"2024-04-26T21:15:16.985094+0000","last_peered":"2024-04-26T21:15:16.985094+0000","last_clean":"2024-04-26T21:15:16.985094+0000","last_became_active":"2024-04-26T21:15:13.988472+0000","last_became_peered":"2024-04-26T21:15:13.988472+0000","last_unstale":"2024-04-26T21:15:16.985094+0000","last_undegraded":"2024-04-26T21:15:16.985094+0000","last_fullsized":"2024-04-26T21:15:16.985094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:17.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594646+0000","last_change":"2024-04-26T21:15:15.007784+0000","last_active":"2024-04-26T21:15:16.594646+0000","last_peered":"2024-04-26T21:15:16.594646+0000","last_clean":"2024-04-26T21:15:16.594646+0000","last_became_active":"2024-04-26T21:15:15.007373+0000","last_became_peered":"2024-04-26T21:15:15.007373+0000","last_unstale":"2024-04-26T21:15:16.594646+0000","last_undegraded":"2024-04-26T21:15:16.594646+0000","last_fullsized":"2024-04-26T21:15:16.594646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:27.459629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993931+0000","last_change":"2024-04-26T21:15:13.985808+0000","last_active":"2024-04-26T21:15:15.993931+0000","last_peered":"2024-04-26T21:15:15.993931+0000","last_clean":"2024-04-26T21:15:15.993931+0000","last_became_active":"2024-04-26T21:15:13.985482+0000","last_became_peered":"2024-04-26T21:15:13.985482+0000","last_unstale":"2024-04-26T21:15:15.993931+0000","last_undegraded":"2024-04-26T21:15:15.993931+0000","last_fullsized":"2024-04-26T21:15:15.993931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:56.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594703+0000","last_change":"2024-04-26T21:15:14.997195+0000","last_active":"2024-04-26T21:15:16.594703+0000","last_peered":"2024-04-26T21:15:16.594703+0000","last_clean":"2024-04-26T21:15:16.594703+0000","last_became_active":"2024-04-26T21:15:14.996889+0000","last_became_peered":"2024-04-26T21:15:14.996889+0000","last_unstale":"2024-04-26T21:15:16.594703+0000","last_undegraded":"2024-04-26T21:15:16.594703+0000","last_fullsized":"2024-04-26T21:15:16.594703+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:03.261175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593581+0000","last_change":"2024-04-26T21:15:14.000033+0000","last_active":"2024-04-26T21:15:16.593581+0000","last_peered":"2024-04-26T21:15:16.593581+0000","last_clean":"2024-04-26T21:15:16.593581+0000","last_became_active":"2024-04-26T21:15:13.999642+0000","last_became_peered":"2024-04-26T21:15:13.999642+0000","last_unstale":"2024-04-26T21:15:16.593581+0000","last_undegraded":"2024-04-26T21:15:16.593581+0000","last_fullsized":"2024-04-26T21:15:16.593581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993953+0000","last_change":"2024-04-26T21:15:14.998038+0000","last_active":"2024-04-26T21:15:15.993953+0000","last_peered":"2024-04-26T21:15:15.993953+0000","last_clean":"2024-04-26T21:15:15.993953+0000","last_became_active":"2024-04-26T21:15:14.997735+0000","last_became_peered":"2024-04-26T21:15:14.997735+0000","last_unstale":"2024-04-26T21:15:15.993953+0000","last_undegraded":"2024-04-26T21:15:15.993953+0000","last_fullsized":"2024-04-26T21:15:15.993953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993592+0000","last_change":"2024-04-26T21:15:13.985339+0000","last_active":"2024-04-26T21:15:15.993592+0000","last_peered":"2024-04-26T21:15:15.993592+0000","last_clean":"2024-04-26T21:15:15.993592+0000","last_became_active":"2024-04-26T21:15:13.984804+0000","last_became_peered":"2024-04-26T21:15:13.984804+0000","last_unstale":"2024-04-26T21:15:15.993592+0000","last_undegraded":"2024-04-26T21:15:15.993592+0000","last_fullsized":"2024-04-26T21:15:15.993592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:08.792790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997529+0000","last_change":"2024-04-26T21:15:15.004362+0000","last_active":"2024-04-26T21:15:15.997529+0000","last_peered":"2024-04-26T21:15:15.997529+0000","last_clean":"2024-04-26T21:15:15.997529+0000","last_became_active":"2024-04-26T21:15:15.003059+0000","last_became_peered":"2024-04-26T21:15:15.003059+0000","last_unstale":"2024-04-26T21:15:15.997529+0000","last_undegraded":"2024-04-26T21:15:15.997529+0000","last_fullsized":"2024-04-26T21:15:15.997529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:04.403950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003690+0000","last_change":"2024-04-26T21:15:14.000183+0000","last_active":"2024-04-26T21:15:14.003690+0000","last_peered":"2024-04-26T21:15:14.003690+0000","last_clean":"2024-04-26T21:15:14.003690+0000","last_became_active":"2024-04-26T21:15:13.999874+0000","last_became_peered":"2024-04-26T21:15:13.999874+0000","last_unstale":"2024-04-26T21:15:14.003690+0000","last_undegraded":"2024-04-26T21:15:14.003690+0000","last_fullsized":"2024-04-26T21:15:14.003690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:18.312063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985817+0000","last_change":"2024-04-26T21:15:13.985817+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985817+0000","last_undegraded":"2024-04-26T21:15:13.985817+0000","last_fullsized":"2024-04-26T21:15:13.985817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593678+0000","last_change":"2024-04-26T21:15:14.000123+0000","last_active":"2024-04-26T21:15:16.593678+0000","last_peered":"2024-04-26T21:15:16.593678+0000","last_clean":"2024-04-26T21:15:16.593678+0000","last_became_active":"2024-04-26T21:15:13.999730+0000","last_became_peered":"2024-04-26T21:15:13.999730+0000","last_unstale":"2024-04-26T21:15:16.593678+0000","last_undegraded":"2024-04-26T21:15:16.593678+0000","last_fullsized":"2024-04-26T21:15:16.593678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:51.654799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997415+0000","last_change":"2024-04-26T21:15:15.004122+0000","last_active":"2024-04-26T21:15:15.997415+0000","last_peered":"2024-04-26T21:15:15.997415+0000","last_clean":"2024-04-26T21:15:15.997415+0000","last_became_active":"2024-04-26T21:15:15.003876+0000","last_became_peered":"2024-04-26T21:15:15.003876+0000","last_unstale":"2024-04-26T21:15:15.997415+0000","last_undegraded":"2024-04-26T21:15:15.997415+0000","last_fullsized":"2024-04-26T21:15:15.997415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:55.716624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998034+0000","last_change":"2024-04-26T21:15:13.992777+0000","last_active":"2024-04-26T21:15:15.998034+0000","last_peered":"2024-04-26T21:15:15.998034+0000","last_clean":"2024-04-26T21:15:15.998034+0000","last_became_active":"2024-04-26T21:15:13.992544+0000","last_became_peered":"2024-04-26T21:15:13.992544+0000","last_unstale":"2024-04-26T21:15:15.998034+0000","last_undegraded":"2024-04-26T21:15:15.998034+0000","last_fullsized":"2024-04-26T21:15:15.998034+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:06.643084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594821+0000","last_change":"2024-04-26T21:15:15.005232+0000","last_active":"2024-04-26T21:15:16.594821+0000","last_peered":"2024-04-26T21:15:16.594821+0000","last_clean":"2024-04-26T21:15:16.594821+0000","last_became_active":"2024-04-26T21:15:15.004904+0000","last_became_peered":"2024-04-26T21:15:15.004904+0000","last_unstale":"2024-04-26T21:15:16.594821+0000","last_undegraded":"2024-04-26T21:15:16.594821+0000","last_fullsized":"2024-04-26T21:15:16.594821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:48.293671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997514+0000","last_change":"2024-04-26T21:15:13.991822+0000","last_active":"2024-04-26T21:15:15.997514+0000","last_peered":"2024-04-26T21:15:15.997514+0000","last_clean":"2024-04-26T21:15:15.997514+0000","last_became_active":"2024-04-26T21:15:13.991508+0000","last_became_peered":"2024-04-26T21:15:13.991508+0000","last_unstale":"2024-04-26T21:15:15.997514+0000","last_undegraded":"2024-04-26T21:15:15.997514+0000","last_fullsized":"2024-04-26T21:15:15.997514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:29.754213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":255,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593633+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:16.593633+0000","last_peered":"2024-04-26T21:15:16.593633+0000","last_clean":"2024-04-26T21:15:16.593633+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:16.593633+0000","last_undegraded":"2024-04-26T21:15:16.593633+0000","last_fullsized":"2024-04-26T21:15:16.593633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":942962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":352070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":352090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":353590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":942346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":338903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:15:19.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:15:19.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.502+0000 7ff97075f640 1 Processor -- start 2024-04-26T21:15:19.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.502+0000 7ff97075f640 1 -- start start 2024-04-26T21:15:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9680cf2e0 0x7ff96806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680d11b0 0x7ff96806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 0x7ff96815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff96806e750 con 0x7ff9680d11b0 2024-04-26T21:15:19.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff9680d1740 con 0x7ff96806e2c0 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff97075f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680d1a20 con 0x7ff9680cf2e0 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff96dcfa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680d11b0 0x7ff96806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff96dcfa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680d11b0 0x7ff96806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51236/0 (socket says 172.21.15.40:51236) 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff96dcfa640 1 -- 172.21.15.40:0/3953672699 learned_addr learned my addr 172.21.15.40:0/3953672699 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff96e4fb640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9680cf2e0 0x7ff96806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.503+0000 7ff96ecfc640 1 --2- 172.21.15.40:0/3953672699 >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 0x7ff96815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 -- 172.21.15.40:0/3953672699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9680cf2e0 msgr2=0x7ff96806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9680cf2e0 0x7ff96806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 -- 172.21.15.40:0/3953672699 >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 msgr2=0x7ff96815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/3953672699 >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 0x7ff96815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 -- 172.21.15.40:0/3953672699 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9680d1d00 con 0x7ff9680d11b0 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96ecfc640 1 --2- 172.21.15.40:0/3953672699 >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 0x7ff96815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.504+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680d11b0 0x7ff96806dd80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff968163ff0 tx=0x7ff95c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60272acdd6674e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff96d4f9640 1 -- 172.21.15.40:0/3953672699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff9680d11b0 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff96d4f9640 1 -- 172.21.15.40:0/3953672699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c032440 con 0x7ff9680d11b0 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff96d4f9640 1 -- 172.21.15.40:0/3953672699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032810 con 0x7ff9680d11b0 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 -- 172.21.15.40:0/3953672699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680d11b0 msgr2=0x7ff96806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680d11b0 0x7ff96806dd80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff968163ff0 tx=0x7ff95c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 -- 172.21.15.40:0/3953672699 shutdown_connections 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 --2- 172.21.15.40:0/3953672699 >> v2:172.21.15.184:3300/0 conn(0x7ff96806e2c0 0x7ff96815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680d11b0 0x7ff96806dd80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 --2- 172.21.15.40:0/3953672699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9680cf2e0 0x7ff96806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 -- 172.21.15.40:0/3953672699 >> 172.21.15.40:0/3953672699 conn(0x7ff9680b4a60 msgr2=0x7ff9680b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 -- 172.21.15.40:0/3953672699 shutdown_connections 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.505+0000 7ff97075f640 1 -- 172.21.15.40:0/3953672699 wait complete. 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.506+0000 7ff97075f640 1 Processor -- start 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.506+0000 7ff97075f640 1 -- start start 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96806e2c0 0x7ff9681602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 0x7ff968156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff96806da10 con 0x7ff9680cf2e0 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9680d22c0 con 0x7ff9680d11b0 2024-04-26T21:15:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff97075f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680cf6c0 con 0x7ff96806e2c0 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96dcfa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96dcfa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51252/0 (socket says 172.21.15.40:51252) 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96dcfa640 1 -- 172.21.15.40:0/2823480714 learned_addr learned my addr 172.21.15.40:0/2823480714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96dcfa640 1 -- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96806e2c0 msgr2=0x7ff9681602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96e4fb640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96806e2c0 0x7ff9681602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.507+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96806e2c0 0x7ff9681602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96dcfa640 1 -- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 msgr2=0x7ff968156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96ecfc640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 0x7ff968156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 0x7ff968156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96dcfa640 1 -- 172.21.15.40:0/2823480714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96809f650 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96ecfc640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 0x7ff968156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff96dcfa640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0090b0 tx=0x7ff95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c041070 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c03c890 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96809f930 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037040 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.508+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96809fef0 con 0x7ff9680cf2e0 2024-04-26T21:15:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.509+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96806a290 con 0x7ff9680cf2e0 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.509+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff95c044070 con 0x7ff9680cf2e0 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.510+0000 7ff966ffd640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 0x7ff95406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.510+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff95c0ac550 con 0x7ff9680cf2e0 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.510+0000 7ff96e4fb640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 0x7ff95406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.510+0000 7ff96e4fb640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 0x7ff95406edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9580073f0 tx=0x7ff95800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.514+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff95c07a980 con 0x7ff9680cf2e0 2024-04-26T21:15:19.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.694+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9681625d0 con 0x7ff95406c8f0 2024-04-26T21:15:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.700+0000 7ff966ffd640 1 -- 172.21.15.40:0/2823480714 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251642 (secure 0 0 0) 0x7ff9681625d0 con 0x7ff95406c8f0 2024-04-26T21:15:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:19.705 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:15:19.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 msgr2=0x7ff95406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 0x7ff95406edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9580073f0 tx=0x7ff95800b040 comp rx=0 tx=0).stop 2024-04-26T21:15:19.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 msgr2=0x7ff96809ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0090b0 tx=0x7ff95c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 shutdown_connections 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95406c8f0 0x7ff95406edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9680d11b0 0x7ff968156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680cf2e0 0x7ff96809ed70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 --2- 172.21.15.40:0/2823480714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96806e2c0 0x7ff9681602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 >> 172.21.15.40:0/2823480714 conn(0x7ff9680b4a60 msgr2=0x7ff9681611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 shutdown_connections 2024-04-26T21:15:19.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.703+0000 7ff97075f640 1 -- 172.21.15.40:0/2823480714 wait complete. 2024-04-26T21:15:19.715 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-26T21:15:18.963801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240620,"kb_used_data":7772,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734100,"statfs":{"total":773094113280,"available":772847718400,"internally_reserved":0,"allocated":7958528,"data_stored":3986077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993743"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594048+0000","last_change":"2024-04-26T21:15:15.008421+0000","last_active":"2024-04-26T21:15:16.594048+0000","last_peered":"2024-04-26T21:15:16.594048+0000","last_clean":"2024-04-26T21:15:16.594048+0000","last_became_active":"2024-04-26T21:15:15.008122+0000","last_became_peered":"2024-04-26T21:15:15.008122+0000","last_unstale":"2024-04-26T21:15:16.594048+0000","last_undegraded":"2024-04-26T21:15:16.594048+0000","last_fullsized":"2024-04-26T21:15:16.594048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:35.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998310+0000","last_change":"2024-04-26T21:15:15.004592+0000","last_active":"2024-04-26T21:15:15.998310+0000","last_peered":"2024-04-26T21:15:15.998310+0000","last_clean":"2024-04-26T21:15:15.998310+0000","last_became_active":"2024-04-26T21:15:15.004314+0000","last_became_peered":"2024-04-26T21:15:15.004314+0000","last_unstale":"2024-04-26T21:15:15.998310+0000","last_undegraded":"2024-04-26T21:15:15.998310+0000","last_fullsized":"2024-04-26T21:15:15.998310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595104+0000","last_change":"2024-04-26T21:15:14.984552+0000","last_active":"2024-04-26T21:15:16.595104+0000","last_peered":"2024-04-26T21:15:16.595104+0000","last_clean":"2024-04-26T21:15:16.595104+0000","last_became_active":"2024-04-26T21:15:14.984224+0000","last_became_peered":"2024-04-26T21:15:14.984224+0000","last_unstale":"2024-04-26T21:15:16.595104+0000","last_undegraded":"2024-04-26T21:15:16.595104+0000","last_fullsized":"2024-04-26T21:15:16.595104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:52.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.986510+0000","last_change":"2024-04-26T21:15:13.986510+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.986510+0000","last_undegraded":"2024-04-26T21:15:13.986510+0000","last_fullsized":"2024-04-26T21:15:13.986510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993499+0000","last_change":"2024-04-26T21:15:15.008338+0000","last_active":"2024-04-26T21:15:15.993499+0000","last_peered":"2024-04-26T21:15:15.993499+0000","last_clean":"2024-04-26T21:15:15.993499+0000","last_became_active":"2024-04-26T21:15:15.008029+0000","last_became_peered":"2024-04-26T21:15:15.008029+0000","last_unstale":"2024-04-26T21:15:15.993499+0000","last_undegraded":"2024-04-26T21:15:15.993499+0000","last_fullsized":"2024-04-26T21:15:15.993499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:02.371753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997414+0000","last_change":"2024-04-26T21:15:15.003006+0000","last_active":"2024-04-26T21:15:15.997414+0000","last_peered":"2024-04-26T21:15:15.997414+0000","last_clean":"2024-04-26T21:15:15.997414+0000","last_became_active":"2024-04-26T21:15:15.002731+0000","last_became_peered":"2024-04-26T21:15:15.002731+0000","last_unstale":"2024-04-26T21:15:15.997414+0000","last_undegraded":"2024-04-26T21:15:15.997414+0000","last_fullsized":"2024-04-26T21:15:15.997414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:45.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996492+0000","last_change":"2024-04-26T21:15:15.000633+0000","last_active":"2024-04-26T21:15:15.996492+0000","last_peered":"2024-04-26T21:15:15.996492+0000","last_clean":"2024-04-26T21:15:15.996492+0000","last_became_active":"2024-04-26T21:15:15.000275+0000","last_became_peered":"2024-04-26T21:15:15.000275+0000","last_unstale":"2024-04-26T21:15:15.996492+0000","last_undegraded":"2024-04-26T21:15:15.996492+0000","last_fullsized":"2024-04-26T21:15:15.996492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:11.788021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997948+0000","last_change":"2024-04-26T21:15:15.002868+0000","last_active":"2024-04-26T21:15:15.997948+0000","last_peered":"2024-04-26T21:15:15.997948+0000","last_clean":"2024-04-26T21:15:15.997948+0000","last_became_active":"2024-04-26T21:15:15.002561+0000","last_became_peered":"2024-04-26T21:15:15.002561+0000","last_unstale":"2024-04-26T21:15:15.997948+0000","last_undegraded":"2024-04-26T21:15:15.997948+0000","last_fullsized":"2024-04-26T21:15:15.997948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994128+0000","last_change":"2024-04-26T21:15:15.004629+0000","last_active":"2024-04-26T21:15:15.994128+0000","last_peered":"2024-04-26T21:15:15.994128+0000","last_clean":"2024-04-26T21:15:15.994128+0000","last_became_active":"2024-04-26T21:15:15.004365+0000","last_became_peered":"2024-04-26T21:15:15.004365+0000","last_unstale":"2024-04-26T21:15:15.994128+0000","last_undegraded":"2024-04-26T21:15:15.994128+0000","last_fullsized":"2024-04-26T21:15:15.994128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:33.056050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594298+0000","last_change":"2024-04-26T21:15:15.005575+0000","last_active":"2024-04-26T21:15:16.594298+0000","last_peered":"2024-04-26T21:15:16.594298+0000","last_clean":"2024-04-26T21:15:16.594298+0000","last_became_active":"2024-04-26T21:15:15.005289+0000","last_became_peered":"2024-04-26T21:15:15.005289+0000","last_unstale":"2024-04-26T21:15:16.594298+0000","last_undegraded":"2024-04-26T21:15:16.594298+0000","last_fullsized":"2024-04-26T21:15:16.594298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:18.088546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594550+0000","last_change":"2024-04-26T21:15:15.000155+0000","last_active":"2024-04-26T21:15:16.594550+0000","last_peered":"2024-04-26T21:15:16.594550+0000","last_clean":"2024-04-26T21:15:16.594550+0000","last_became_active":"2024-04-26T21:15:14.999858+0000","last_became_peered":"2024-04-26T21:15:14.999858+0000","last_unstale":"2024-04-26T21:15:16.594550+0000","last_undegraded":"2024-04-26T21:15:16.594550+0000","last_fullsized":"2024-04-26T21:15:16.594550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:01.080096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985514+0000","last_change":"2024-04-26T21:15:13.985514+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985514+0000","last_undegraded":"2024-04-26T21:15:13.985514+0000","last_fullsized":"2024-04-26T21:15:13.985514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593769+0000","last_change":"2024-04-26T21:15:15.001688+0000","last_active":"2024-04-26T21:15:16.593769+0000","last_peered":"2024-04-26T21:15:16.593769+0000","last_clean":"2024-04-26T21:15:16.593769+0000","last_became_active":"2024-04-26T21:15:15.001372+0000","last_became_peered":"2024-04-26T21:15:15.001372+0000","last_unstale":"2024-04-26T21:15:16.593769+0000","last_undegraded":"2024-04-26T21:15:16.593769+0000","last_fullsized":"2024-04-26T21:15:16.593769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993853+0000","last_change":"2024-04-26T21:15:14.995986+0000","last_active":"2024-04-26T21:15:15.993853+0000","last_peered":"2024-04-26T21:15:15.993853+0000","last_clean":"2024-04-26T21:15:15.993853+0000","last_became_active":"2024-04-26T21:15:14.995673+0000","last_became_peered":"2024-04-26T21:15:14.995673+0000","last_unstale":"2024-04-26T21:15:15.993853+0000","last_undegraded":"2024-04-26T21:15:15.993853+0000","last_fullsized":"2024-04-26T21:15:15.993853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593933+0000","last_change":"2024-04-26T21:15:14.996331+0000","last_active":"2024-04-26T21:15:16.593933+0000","last_peered":"2024-04-26T21:15:16.593933+0000","last_clean":"2024-04-26T21:15:16.593933+0000","last_became_active":"2024-04-26T21:15:14.996044+0000","last_became_peered":"2024-04-26T21:15:14.996044+0000","last_unstale":"2024-04-26T21:15:16.593933+0000","last_undegraded":"2024-04-26T21:15:16.593933+0000","last_fullsized":"2024-04-26T21:15:16.593933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593583+0000","last_change":"2024-04-26T21:15:15.007557+0000","last_active":"2024-04-26T21:15:16.593583+0000","last_peered":"2024-04-26T21:15:16.593583+0000","last_clean":"2024-04-26T21:15:16.593583+0000","last_became_active":"2024-04-26T21:15:15.007269+0000","last_became_peered":"2024-04-26T21:15:15.007269+0000","last_unstale":"2024-04-26T21:15:16.593583+0000","last_undegraded":"2024-04-26T21:15:16.593583+0000","last_fullsized":"2024-04-26T21:15:16.593583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994100+0000","last_change":"2024-04-26T21:15:15.003937+0000","last_active":"2024-04-26T21:15:15.994100+0000","last_peered":"2024-04-26T21:15:15.994100+0000","last_clean":"2024-04-26T21:15:15.994100+0000","last_became_active":"2024-04-26T21:15:15.003652+0000","last_became_peered":"2024-04-26T21:15:15.003652+0000","last_unstale":"2024-04-26T21:15:15.994100+0000","last_undegraded":"2024-04-26T21:15:15.994100+0000","last_fullsized":"2024-04-26T21:15:15.994100+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.457492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985659+0000","last_change":"2024-04-26T21:15:13.985659+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985659+0000","last_undegraded":"2024-04-26T21:15:13.985659+0000","last_fullsized":"2024-04-26T21:15:13.985659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997884+0000","last_change":"2024-04-26T21:15:15.003066+0000","last_active":"2024-04-26T21:15:15.997884+0000","last_peered":"2024-04-26T21:15:15.997884+0000","last_clean":"2024-04-26T21:15:15.997884+0000","last_became_active":"2024-04-26T21:15:15.002597+0000","last_became_peered":"2024-04-26T21:15:15.002597+0000","last_unstale":"2024-04-26T21:15:15.997884+0000","last_undegraded":"2024-04-26T21:15:15.997884+0000","last_fullsized":"2024-04-26T21:15:15.997884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593869+0000","last_change":"2024-04-26T21:15:14.996454+0000","last_active":"2024-04-26T21:15:16.593869+0000","last_peered":"2024-04-26T21:15:16.593869+0000","last_clean":"2024-04-26T21:15:16.593869+0000","last_became_active":"2024-04-26T21:15:14.996140+0000","last_became_peered":"2024-04-26T21:15:14.996140+0000","last_unstale":"2024-04-26T21:15:16.593869+0000","last_undegraded":"2024-04-26T21:15:16.593869+0000","last_fullsized":"2024-04-26T21:15:16.593869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593784+0000","last_change":"2024-04-26T21:15:15.004956+0000","last_active":"2024-04-26T21:15:16.593784+0000","last_peered":"2024-04-26T21:15:16.593784+0000","last_clean":"2024-04-26T21:15:16.593784+0000","last_became_active":"2024-04-26T21:15:15.004666+0000","last_became_peered":"2024-04-26T21:15:15.004666+0000","last_unstale":"2024-04-26T21:15:16.593784+0000","last_undegraded":"2024-04-26T21:15:16.593784+0000","last_fullsized":"2024-04-26T21:15:16.593784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593651+0000","last_change":"2024-04-26T21:15:14.998310+0000","last_active":"2024-04-26T21:15:16.593651+0000","last_peered":"2024-04-26T21:15:16.593651+0000","last_clean":"2024-04-26T21:15:16.593651+0000","last_became_active":"2024-04-26T21:15:14.998039+0000","last_became_peered":"2024-04-26T21:15:14.998039+0000","last_unstale":"2024-04-26T21:15:16.593651+0000","last_undegraded":"2024-04-26T21:15:16.593651+0000","last_fullsized":"2024-04-26T21:15:16.593651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:14.997132+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:14.996824+0000","last_became_peered":"2024-04-26T21:15:14.996824+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593965+0000","last_change":"2024-04-26T21:15:15.003818+0000","last_active":"2024-04-26T21:15:16.593965+0000","last_peered":"2024-04-26T21:15:16.593965+0000","last_clean":"2024-04-26T21:15:16.593965+0000","last_became_active":"2024-04-26T21:15:15.003428+0000","last_became_peered":"2024-04-26T21:15:15.003428+0000","last_unstale":"2024-04-26T21:15:16.593965+0000","last_undegraded":"2024-04-26T21:15:16.593965+0000","last_fullsized":"2024-04-26T21:15:16.593965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594069+0000","last_change":"2024-04-26T21:15:15.004314+0000","last_active":"2024-04-26T21:15:16.594069+0000","last_peered":"2024-04-26T21:15:16.594069+0000","last_clean":"2024-04-26T21:15:16.594069+0000","last_became_active":"2024-04-26T21:15:15.004022+0000","last_became_peered":"2024-04-26T21:15:15.004022+0000","last_unstale":"2024-04-26T21:15:16.594069+0000","last_undegraded":"2024-04-26T21:15:16.594069+0000","last_fullsized":"2024-04-26T21:15:16.594069+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996953+0000","last_change":"2024-04-26T21:15:15.006413+0000","last_active":"2024-04-26T21:15:15.996953+0000","last_peered":"2024-04-26T21:15:15.996953+0000","last_clean":"2024-04-26T21:15:15.996953+0000","last_became_active":"2024-04-26T21:15:15.006136+0000","last_became_peered":"2024-04-26T21:15:15.006136+0000","last_unstale":"2024-04-26T21:15:15.996953+0000","last_undegraded":"2024-04-26T21:15:15.996953+0000","last_fullsized":"2024-04-26T21:15:15.996953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595237+0000","last_change":"2024-04-26T21:15:14.999635+0000","last_active":"2024-04-26T21:15:16.595237+0000","last_peered":"2024-04-26T21:15:16.595237+0000","last_clean":"2024-04-26T21:15:16.595237+0000","last_became_active":"2024-04-26T21:15:14.999351+0000","last_became_peered":"2024-04-26T21:15:14.999351+0000","last_unstale":"2024-04-26T21:15:16.595237+0000","last_undegraded":"2024-04-26T21:15:16.595237+0000","last_fullsized":"2024-04-26T21:15:16.595237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:23.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594169+0000","last_change":"2024-04-26T21:15:15.006522+0000","last_active":"2024-04-26T21:15:16.594169+0000","last_peered":"2024-04-26T21:15:16.594169+0000","last_clean":"2024-04-26T21:15:16.594169+0000","last_became_active":"2024-04-26T21:15:15.005849+0000","last_became_peered":"2024-04-26T21:15:15.005849+0000","last_unstale":"2024-04-26T21:15:16.594169+0000","last_undegraded":"2024-04-26T21:15:16.594169+0000","last_fullsized":"2024-04-26T21:15:16.594169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:16.396516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594834+0000","last_change":"2024-04-26T21:15:14.988314+0000","last_active":"2024-04-26T21:15:16.594834+0000","last_peered":"2024-04-26T21:15:16.594834+0000","last_clean":"2024-04-26T21:15:16.594834+0000","last_became_active":"2024-04-26T21:15:14.987983+0000","last_became_peered":"2024-04-26T21:15:14.987983+0000","last_unstale":"2024-04-26T21:15:16.594834+0000","last_undegraded":"2024-04-26T21:15:16.594834+0000","last_fullsized":"2024-04-26T21:15:16.594834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:56.347854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996624+0000","last_change":"2024-04-26T21:15:15.001705+0000","last_active":"2024-04-26T21:15:15.996624+0000","last_peered":"2024-04-26T21:15:15.996624+0000","last_clean":"2024-04-26T21:15:15.996624+0000","last_became_active":"2024-04-26T21:15:15.001398+0000","last_became_peered":"2024-04-26T21:15:15.001398+0000","last_unstale":"2024-04-26T21:15:15.996624+0000","last_undegraded":"2024-04-26T21:15:15.996624+0000","last_fullsized":"2024-04-26T21:15:15.996624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996883+0000","last_change":"2024-04-26T21:15:14.996277+0000","last_active":"2024-04-26T21:15:15.996883+0000","last_peered":"2024-04-26T21:15:15.996883+0000","last_clean":"2024-04-26T21:15:15.996883+0000","last_became_active":"2024-04-26T21:15:14.996006+0000","last_became_peered":"2024-04-26T21:15:14.996006+0000","last_unstale":"2024-04-26T21:15:15.996883+0000","last_undegraded":"2024-04-26T21:15:15.996883+0000","last_fullsized":"2024-04-26T21:15:15.996883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997797+0000","last_change":"2024-04-26T21:15:15.005406+0000","last_active":"2024-04-26T21:15:15.997797+0000","last_peered":"2024-04-26T21:15:15.997797+0000","last_clean":"2024-04-26T21:15:15.997797+0000","last_became_active":"2024-04-26T21:15:15.005116+0000","last_became_peered":"2024-04-26T21:15:15.005116+0000","last_unstale":"2024-04-26T21:15:15.997797+0000","last_undegraded":"2024-04-26T21:15:15.997797+0000","last_fullsized":"2024-04-26T21:15:15.997797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:16.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594603+0000","last_change":"2024-04-26T21:15:15.008320+0000","last_active":"2024-04-26T21:15:16.594603+0000","last_peered":"2024-04-26T21:15:16.594603+0000","last_clean":"2024-04-26T21:15:16.594603+0000","last_became_active":"2024-04-26T21:15:15.008048+0000","last_became_peered":"2024-04-26T21:15:15.008048+0000","last_unstale":"2024-04-26T21:15:16.594603+0000","last_undegraded":"2024-04-26T21:15:16.594603+0000","last_fullsized":"2024-04-26T21:15:16.594603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.353647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594502+0000","last_change":"2024-04-26T21:15:13.991660+0000","last_active":"2024-04-26T21:15:16.594502+0000","last_peered":"2024-04-26T21:15:16.594502+0000","last_clean":"2024-04-26T21:15:16.594502+0000","last_became_active":"2024-04-26T21:15:13.991361+0000","last_became_peered":"2024-04-26T21:15:13.991361+0000","last_unstale":"2024-04-26T21:15:16.594502+0000","last_undegraded":"2024-04-26T21:15:16.594502+0000","last_fullsized":"2024-04-26T21:15:16.594502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994190+0000","last_change":"2024-04-26T21:15:15.003269+0000","last_active":"2024-04-26T21:15:15.994190+0000","last_peered":"2024-04-26T21:15:15.994190+0000","last_clean":"2024-04-26T21:15:15.994190+0000","last_became_active":"2024-04-26T21:15:15.002973+0000","last_became_peered":"2024-04-26T21:15:15.002973+0000","last_unstale":"2024-04-26T21:15:15.994190+0000","last_undegraded":"2024-04-26T21:15:15.994190+0000","last_fullsized":"2024-04-26T21:15:15.994190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:30.756090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593977+0000","last_change":"2024-04-26T21:15:13.990425+0000","last_active":"2024-04-26T21:15:16.593977+0000","last_peered":"2024-04-26T21:15:16.593977+0000","last_clean":"2024-04-26T21:15:16.593977+0000","last_became_active":"2024-04-26T21:15:13.990113+0000","last_became_peered":"2024-04-26T21:15:13.990113+0000","last_unstale":"2024-04-26T21:15:16.593977+0000","last_undegraded":"2024-04-26T21:15:16.593977+0000","last_fullsized":"2024-04-26T21:15:16.593977+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997715+0000","last_change":"2024-04-26T21:15:15.004144+0000","last_active":"2024-04-26T21:15:15.997715+0000","last_peered":"2024-04-26T21:15:15.997715+0000","last_clean":"2024-04-26T21:15:15.997715+0000","last_became_active":"2024-04-26T21:15:15.003804+0000","last_became_peered":"2024-04-26T21:15:15.003804+0000","last_unstale":"2024-04-26T21:15:15.997715+0000","last_undegraded":"2024-04-26T21:15:15.997715+0000","last_fullsized":"2024-04-26T21:15:15.997715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:34.213418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594471+0000","last_change":"2024-04-26T21:15:13.996784+0000","last_active":"2024-04-26T21:15:16.594471+0000","last_peered":"2024-04-26T21:15:16.594471+0000","last_clean":"2024-04-26T21:15:16.594471+0000","last_became_active":"2024-04-26T21:15:13.996508+0000","last_became_peered":"2024-04-26T21:15:13.996508+0000","last_unstale":"2024-04-26T21:15:16.594471+0000","last_undegraded":"2024-04-26T21:15:16.594471+0000","last_fullsized":"2024-04-26T21:15:16.594471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:31.090506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594503+0000","last_change":"2024-04-26T21:15:14.998880+0000","last_active":"2024-04-26T21:15:16.594503+0000","last_peered":"2024-04-26T21:15:16.594503+0000","last_clean":"2024-04-26T21:15:16.594503+0000","last_became_active":"2024-04-26T21:15:14.998406+0000","last_became_peered":"2024-04-26T21:15:14.998406+0000","last_unstale":"2024-04-26T21:15:16.594503+0000","last_undegraded":"2024-04-26T21:15:16.594503+0000","last_fullsized":"2024-04-26T21:15:16.594503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:32.918934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997990+0000","last_change":"2024-04-26T21:15:13.990545+0000","last_active":"2024-04-26T21:15:15.997990+0000","last_peered":"2024-04-26T21:15:15.997990+0000","last_clean":"2024-04-26T21:15:15.997990+0000","last_became_active":"2024-04-26T21:15:13.990151+0000","last_became_peered":"2024-04-26T21:15:13.990151+0000","last_unstale":"2024-04-26T21:15:15.997990+0000","last_undegraded":"2024-04-26T21:15:15.997990+0000","last_fullsized":"2024-04-26T21:15:15.997990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:43.327070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996712+0000","last_change":"2024-04-26T21:15:14.999455+0000","last_active":"2024-04-26T21:15:15.996712+0000","last_peered":"2024-04-26T21:15:15.996712+0000","last_clean":"2024-04-26T21:15:15.996712+0000","last_became_active":"2024-04-26T21:15:14.999063+0000","last_became_peered":"2024-04-26T21:15:14.999063+0000","last_unstale":"2024-04-26T21:15:15.996712+0000","last_undegraded":"2024-04-26T21:15:15.996712+0000","last_fullsized":"2024-04-26T21:15:15.996712+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595004+0000","last_change":"2024-04-26T21:15:14.000741+0000","last_active":"2024-04-26T21:15:16.595004+0000","last_peered":"2024-04-26T21:15:16.595004+0000","last_clean":"2024-04-26T21:15:16.595004+0000","last_became_active":"2024-04-26T21:15:14.000484+0000","last_became_peered":"2024-04-26T21:15:14.000484+0000","last_unstale":"2024-04-26T21:15:16.595004+0000","last_undegraded":"2024-04-26T21:15:16.595004+0000","last_fullsized":"2024-04-26T21:15:16.595004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:34.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:15.003590+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:15.003252+0000","last_became_peered":"2024-04-26T21:15:15.003252+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:40.805512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997494+0000","last_change":"2024-04-26T21:15:13.987191+0000","last_active":"2024-04-26T21:15:15.997494+0000","last_peered":"2024-04-26T21:15:15.997494+0000","last_clean":"2024-04-26T21:15:15.997494+0000","last_became_active":"2024-04-26T21:15:13.986872+0000","last_became_peered":"2024-04-26T21:15:13.986872+0000","last_unstale":"2024-04-26T21:15:15.997494+0000","last_undegraded":"2024-04-26T21:15:15.997494+0000","last_fullsized":"2024-04-26T21:15:15.997494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:02.357589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993432+0000","last_change":"2024-04-26T21:15:15.002302+0000","last_active":"2024-04-26T21:15:15.993432+0000","last_peered":"2024-04-26T21:15:15.993432+0000","last_clean":"2024-04-26T21:15:15.993432+0000","last_became_active":"2024-04-26T21:15:15.001311+0000","last_became_peered":"2024-04-26T21:15:15.001311+0000","last_unstale":"2024-04-26T21:15:15.993432+0000","last_undegraded":"2024-04-26T21:15:15.993432+0000","last_fullsized":"2024-04-26T21:15:15.993432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:08.309553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003614+0000","last_change":"2024-04-26T21:15:13.998731+0000","last_active":"2024-04-26T21:15:14.003614+0000","last_peered":"2024-04-26T21:15:14.003614+0000","last_clean":"2024-04-26T21:15:14.003614+0000","last_became_active":"2024-04-26T21:15:13.998420+0000","last_became_peered":"2024-04-26T21:15:13.998420+0000","last_unstale":"2024-04-26T21:15:14.003614+0000","last_undegraded":"2024-04-26T21:15:14.003614+0000","last_fullsized":"2024-04-26T21:15:14.003614+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:03.412018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593528+0000","last_change":"2024-04-26T21:15:14.999599+0000","last_active":"2024-04-26T21:15:16.593528+0000","last_peered":"2024-04-26T21:15:16.593528+0000","last_clean":"2024-04-26T21:15:16.593528+0000","last_became_active":"2024-04-26T21:15:14.998302+0000","last_became_peered":"2024-04-26T21:15:14.998302+0000","last_unstale":"2024-04-26T21:15:16.593528+0000","last_undegraded":"2024-04-26T21:15:16.593528+0000","last_fullsized":"2024-04-26T21:15:16.593528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993426+0000","last_change":"2024-04-26T21:15:13.989877+0000","last_active":"2024-04-26T21:15:15.993426+0000","last_peered":"2024-04-26T21:15:15.993426+0000","last_clean":"2024-04-26T21:15:15.993426+0000","last_became_active":"2024-04-26T21:15:13.989441+0000","last_became_peered":"2024-04-26T21:15:13.989441+0000","last_unstale":"2024-04-26T21:15:15.993426+0000","last_undegraded":"2024-04-26T21:15:15.993426+0000","last_fullsized":"2024-04-26T21:15:15.993426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:41.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997022+0000","last_change":"2024-04-26T21:15:14.999473+0000","last_active":"2024-04-26T21:15:15.997022+0000","last_peered":"2024-04-26T21:15:15.997022+0000","last_clean":"2024-04-26T21:15:15.997022+0000","last_became_active":"2024-04-26T21:15:14.999179+0000","last_became_peered":"2024-04-26T21:15:14.999179+0000","last_unstale":"2024-04-26T21:15:15.997022+0000","last_undegraded":"2024-04-26T21:15:15.997022+0000","last_fullsized":"2024-04-26T21:15:15.997022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594171+0000","last_change":"2024-04-26T21:15:13.999674+0000","last_active":"2024-04-26T21:15:16.594171+0000","last_peered":"2024-04-26T21:15:16.594171+0000","last_clean":"2024-04-26T21:15:16.594171+0000","last_became_active":"2024-04-26T21:15:13.999384+0000","last_became_peered":"2024-04-26T21:15:13.999384+0000","last_unstale":"2024-04-26T21:15:16.594171+0000","last_undegraded":"2024-04-26T21:15:16.594171+0000","last_fullsized":"2024-04-26T21:15:16.594171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:41.421624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594232+0000","last_change":"2024-04-26T21:15:15.004056+0000","last_active":"2024-04-26T21:15:16.594232+0000","last_peered":"2024-04-26T21:15:16.594232+0000","last_clean":"2024-04-26T21:15:16.594232+0000","last_became_active":"2024-04-26T21:15:15.003109+0000","last_became_peered":"2024-04-26T21:15:15.003109+0000","last_unstale":"2024-04-26T21:15:16.594232+0000","last_undegraded":"2024-04-26T21:15:16.594232+0000","last_fullsized":"2024-04-26T21:15:16.594232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.332709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998354+0000","last_change":"2024-04-26T21:15:13.988307+0000","last_active":"2024-04-26T21:15:15.998354+0000","last_peered":"2024-04-26T21:15:15.998354+0000","last_clean":"2024-04-26T21:15:15.998354+0000","last_became_active":"2024-04-26T21:15:13.988018+0000","last_became_peered":"2024-04-26T21:15:13.988018+0000","last_unstale":"2024-04-26T21:15:15.998354+0000","last_undegraded":"2024-04-26T21:15:15.998354+0000","last_fullsized":"2024-04-26T21:15:15.998354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.077866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997865+0000","last_change":"2024-04-26T21:15:15.004543+0000","last_active":"2024-04-26T21:15:15.997865+0000","last_peered":"2024-04-26T21:15:15.997865+0000","last_clean":"2024-04-26T21:15:15.997865+0000","last_became_active":"2024-04-26T21:15:15.004281+0000","last_became_peered":"2024-04-26T21:15:15.004281+0000","last_unstale":"2024-04-26T21:15:15.997865+0000","last_undegraded":"2024-04-26T21:15:15.997865+0000","last_fullsized":"2024-04-26T21:15:15.997865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595209+0000","last_change":"2024-04-26T21:15:13.991260+0000","last_active":"2024-04-26T21:15:16.595209+0000","last_peered":"2024-04-26T21:15:16.595209+0000","last_clean":"2024-04-26T21:15:16.595209+0000","last_became_active":"2024-04-26T21:15:13.990969+0000","last_became_peered":"2024-04-26T21:15:13.990969+0000","last_unstale":"2024-04-26T21:15:16.595209+0000","last_undegraded":"2024-04-26T21:15:16.595209+0000","last_fullsized":"2024-04-26T21:15:16.595209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:35.822660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993865+0000","last_change":"2024-04-26T21:15:14.997788+0000","last_active":"2024-04-26T21:15:15.993865+0000","last_peered":"2024-04-26T21:15:15.993865+0000","last_clean":"2024-04-26T21:15:15.993865+0000","last_became_active":"2024-04-26T21:15:14.997464+0000","last_became_peered":"2024-04-26T21:15:14.997464+0000","last_unstale":"2024-04-26T21:15:15.993865+0000","last_undegraded":"2024-04-26T21:15:15.993865+0000","last_fullsized":"2024-04-26T21:15:15.993865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:56.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994753+0000","last_change":"2024-04-26T21:15:13.991679+0000","last_active":"2024-04-26T21:15:13.994753+0000","last_peered":"2024-04-26T21:15:13.994753+0000","last_clean":"2024-04-26T21:15:13.994753+0000","last_became_active":"2024-04-26T21:15:13.991338+0000","last_became_peered":"2024-04-26T21:15:13.991338+0000","last_unstale":"2024-04-26T21:15:13.994753+0000","last_undegraded":"2024-04-26T21:15:13.994753+0000","last_fullsized":"2024-04-26T21:15:13.994753+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:32.732498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:14.999960+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:14.999673+0000","last_became_peered":"2024-04-26T21:15:14.999673+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:36.824537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994639+0000","last_change":"2024-04-26T21:15:13.990261+0000","last_active":"2024-04-26T21:15:13.994639+0000","last_peered":"2024-04-26T21:15:13.994639+0000","last_clean":"2024-04-26T21:15:13.994639+0000","last_became_active":"2024-04-26T21:15:13.989888+0000","last_became_peered":"2024-04-26T21:15:13.989888+0000","last_unstale":"2024-04-26T21:15:13.994639+0000","last_undegraded":"2024-04-26T21:15:13.994639+0000","last_fullsized":"2024-04-26T21:15:13.994639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:22.729662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996901+0000","last_change":"2024-04-26T21:15:15.000394+0000","last_active":"2024-04-26T21:15:15.996901+0000","last_peered":"2024-04-26T21:15:15.996901+0000","last_clean":"2024-04-26T21:15:15.996901+0000","last_became_active":"2024-04-26T21:15:14.999894+0000","last_became_peered":"2024-04-26T21:15:14.999894+0000","last_unstale":"2024-04-26T21:15:15.996901+0000","last_undegraded":"2024-04-26T21:15:15.996901+0000","last_fullsized":"2024-04-26T21:15:15.996901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997598+0000","last_change":"2024-04-26T21:15:13.989609+0000","last_active":"2024-04-26T21:15:15.997598+0000","last_peered":"2024-04-26T21:15:15.997598+0000","last_clean":"2024-04-26T21:15:15.997598+0000","last_became_active":"2024-04-26T21:15:13.989419+0000","last_became_peered":"2024-04-26T21:15:13.989419+0000","last_unstale":"2024-04-26T21:15:15.997598+0000","last_undegraded":"2024-04-26T21:15:15.997598+0000","last_fullsized":"2024-04-26T21:15:15.997598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:36.252437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593466+0000","last_change":"2024-04-26T21:15:15.005165+0000","last_active":"2024-04-26T21:15:16.593466+0000","last_peered":"2024-04-26T21:15:16.593466+0000","last_clean":"2024-04-26T21:15:16.593466+0000","last_became_active":"2024-04-26T21:15:15.004821+0000","last_became_peered":"2024-04-26T21:15:15.004821+0000","last_unstale":"2024-04-26T21:15:16.593466+0000","last_undegraded":"2024-04-26T21:15:16.593466+0000","last_fullsized":"2024-04-26T21:15:16.593466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996529+0000","last_change":"2024-04-26T21:15:13.984946+0000","last_active":"2024-04-26T21:15:15.996529+0000","last_peered":"2024-04-26T21:15:15.996529+0000","last_clean":"2024-04-26T21:15:15.996529+0000","last_became_active":"2024-04-26T21:15:13.984641+0000","last_became_peered":"2024-04-26T21:15:13.984641+0000","last_unstale":"2024-04-26T21:15:15.996529+0000","last_undegraded":"2024-04-26T21:15:15.996529+0000","last_fullsized":"2024-04-26T21:15:15.996529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993537+0000","last_change":"2024-04-26T21:15:15.007343+0000","last_active":"2024-04-26T21:15:15.993537+0000","last_peered":"2024-04-26T21:15:15.993537+0000","last_clean":"2024-04-26T21:15:15.993537+0000","last_became_active":"2024-04-26T21:15:15.007015+0000","last_became_peered":"2024-04-26T21:15:15.007015+0000","last_unstale":"2024-04-26T21:15:15.993537+0000","last_undegraded":"2024-04-26T21:15:15.993537+0000","last_fullsized":"2024-04-26T21:15:15.993537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:34.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593601+0000","last_change":"2024-04-26T21:15:13.996573+0000","last_active":"2024-04-26T21:15:16.593601+0000","last_peered":"2024-04-26T21:15:16.593601+0000","last_clean":"2024-04-26T21:15:16.593601+0000","last_became_active":"2024-04-26T21:15:13.996295+0000","last_became_peered":"2024-04-26T21:15:13.996295+0000","last_unstale":"2024-04-26T21:15:16.593601+0000","last_undegraded":"2024-04-26T21:15:16.593601+0000","last_fullsized":"2024-04-26T21:15:16.593601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:42.351528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594435+0000","last_change":"2024-04-26T21:15:15.001042+0000","last_active":"2024-04-26T21:15:16.594435+0000","last_peered":"2024-04-26T21:15:16.594435+0000","last_clean":"2024-04-26T21:15:16.594435+0000","last_became_active":"2024-04-26T21:15:15.000753+0000","last_became_peered":"2024-04-26T21:15:15.000753+0000","last_unstale":"2024-04-26T21:15:16.594435+0000","last_undegraded":"2024-04-26T21:15:16.594435+0000","last_fullsized":"2024-04-26T21:15:16.594435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.434213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998075+0000","last_change":"2024-04-26T21:15:13.993704+0000","last_active":"2024-04-26T21:15:15.998075+0000","last_peered":"2024-04-26T21:15:15.998075+0000","last_clean":"2024-04-26T21:15:15.998075+0000","last_became_active":"2024-04-26T21:15:13.993410+0000","last_became_peered":"2024-04-26T21:15:13.993410+0000","last_unstale":"2024-04-26T21:15:15.998075+0000","last_undegraded":"2024-04-26T21:15:15.998075+0000","last_fullsized":"2024-04-26T21:15:15.998075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594517+0000","last_change":"2024-04-26T21:15:14.998082+0000","last_active":"2024-04-26T21:15:16.594517+0000","last_peered":"2024-04-26T21:15:16.594517+0000","last_clean":"2024-04-26T21:15:16.594517+0000","last_became_active":"2024-04-26T21:15:14.997733+0000","last_became_peered":"2024-04-26T21:15:14.997733+0000","last_unstale":"2024-04-26T21:15:16.594517+0000","last_undegraded":"2024-04-26T21:15:16.594517+0000","last_fullsized":"2024-04-26T21:15:16.594517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:07.215468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594639+0000","last_change":"2024-04-26T21:15:14.001697+0000","last_active":"2024-04-26T21:15:16.594639+0000","last_peered":"2024-04-26T21:15:16.594639+0000","last_clean":"2024-04-26T21:15:16.594639+0000","last_became_active":"2024-04-26T21:15:14.001416+0000","last_became_peered":"2024-04-26T21:15:14.001416+0000","last_unstale":"2024-04-26T21:15:16.594639+0000","last_undegraded":"2024-04-26T21:15:16.594639+0000","last_fullsized":"2024-04-26T21:15:16.594639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:11.110564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985044+0000","last_change":"2024-04-26T21:15:13.985044+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985044+0000","last_undegraded":"2024-04-26T21:15:13.985044+0000","last_fullsized":"2024-04-26T21:15:13.985044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997916+0000","last_change":"2024-04-26T21:15:13.997192+0000","last_active":"2024-04-26T21:15:15.997916+0000","last_peered":"2024-04-26T21:15:15.997916+0000","last_clean":"2024-04-26T21:15:15.997916+0000","last_became_active":"2024-04-26T21:15:13.996946+0000","last_became_peered":"2024-04-26T21:15:13.996946+0000","last_unstale":"2024-04-26T21:15:15.997916+0000","last_undegraded":"2024-04-26T21:15:15.997916+0000","last_fullsized":"2024-04-26T21:15:15.997916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:33.005058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993826+0000","last_change":"2024-04-26T21:15:15.000551+0000","last_active":"2024-04-26T21:15:15.993826+0000","last_peered":"2024-04-26T21:15:15.993826+0000","last_clean":"2024-04-26T21:15:15.993826+0000","last_became_active":"2024-04-26T21:15:15.000269+0000","last_became_peered":"2024-04-26T21:15:15.000269+0000","last_unstale":"2024-04-26T21:15:15.993826+0000","last_undegraded":"2024-04-26T21:15:15.993826+0000","last_fullsized":"2024-04-26T21:15:15.993826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:46.479309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594507+0000","last_change":"2024-04-26T21:15:14.000103+0000","last_active":"2024-04-26T21:15:16.594507+0000","last_peered":"2024-04-26T21:15:16.594507+0000","last_clean":"2024-04-26T21:15:16.594507+0000","last_became_active":"2024-04-26T21:15:13.999824+0000","last_became_peered":"2024-04-26T21:15:13.999824+0000","last_unstale":"2024-04-26T21:15:16.594507+0000","last_undegraded":"2024-04-26T21:15:16.594507+0000","last_fullsized":"2024-04-26T21:15:16.594507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595067+0000","last_change":"2024-04-26T21:15:14.999573+0000","last_active":"2024-04-26T21:15:16.595067+0000","last_peered":"2024-04-26T21:15:16.595067+0000","last_clean":"2024-04-26T21:15:16.595067+0000","last_became_active":"2024-04-26T21:15:14.999265+0000","last_became_peered":"2024-04-26T21:15:14.999265+0000","last_unstale":"2024-04-26T21:15:16.595067+0000","last_undegraded":"2024-04-26T21:15:16.595067+0000","last_fullsized":"2024-04-26T21:15:16.595067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:49.825951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997655+0000","last_change":"2024-04-26T21:15:13.987004+0000","last_active":"2024-04-26T21:15:15.997655+0000","last_peered":"2024-04-26T21:15:15.997655+0000","last_clean":"2024-04-26T21:15:15.997655+0000","last_became_active":"2024-04-26T21:15:13.986676+0000","last_became_peered":"2024-04-26T21:15:13.986676+0000","last_unstale":"2024-04-26T21:15:15.997655+0000","last_undegraded":"2024-04-26T21:15:15.997655+0000","last_fullsized":"2024-04-26T21:15:15.997655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:44.090460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594645+0000","last_change":"2024-04-26T21:15:15.004474+0000","last_active":"2024-04-26T21:15:16.594645+0000","last_peered":"2024-04-26T21:15:16.594645+0000","last_clean":"2024-04-26T21:15:16.594645+0000","last_became_active":"2024-04-26T21:15:15.004185+0000","last_became_peered":"2024-04-26T21:15:15.004185+0000","last_unstale":"2024-04-26T21:15:16.594645+0000","last_undegraded":"2024-04-26T21:15:16.594645+0000","last_fullsized":"2024-04-26T21:15:16.594645+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.298412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996654+0000","last_change":"2024-04-26T21:15:13.988682+0000","last_active":"2024-04-26T21:15:15.996654+0000","last_peered":"2024-04-26T21:15:15.996654+0000","last_clean":"2024-04-26T21:15:15.996654+0000","last_became_active":"2024-04-26T21:15:13.988377+0000","last_became_peered":"2024-04-26T21:15:13.988377+0000","last_unstale":"2024-04-26T21:15:15.996654+0000","last_undegraded":"2024-04-26T21:15:15.996654+0000","last_fullsized":"2024-04-26T21:15:15.996654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:15.394639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594757+0000","last_change":"2024-04-26T21:15:15.005522+0000","last_active":"2024-04-26T21:15:16.594757+0000","last_peered":"2024-04-26T21:15:16.594757+0000","last_clean":"2024-04-26T21:15:16.594757+0000","last_became_active":"2024-04-26T21:15:15.005254+0000","last_became_peered":"2024-04-26T21:15:15.005254+0000","last_unstale":"2024-04-26T21:15:16.594757+0000","last_undegraded":"2024-04-26T21:15:16.594757+0000","last_fullsized":"2024-04-26T21:15:16.594757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.398672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593386+0000","last_change":"2024-04-26T21:15:13.990434+0000","last_active":"2024-04-26T21:15:16.593386+0000","last_peered":"2024-04-26T21:15:16.593386+0000","last_clean":"2024-04-26T21:15:16.593386+0000","last_became_active":"2024-04-26T21:15:13.990190+0000","last_became_peered":"2024-04-26T21:15:13.990190+0000","last_unstale":"2024-04-26T21:15:16.593386+0000","last_undegraded":"2024-04-26T21:15:16.593386+0000","last_fullsized":"2024-04-26T21:15:16.593386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:39.803635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593847+0000","last_change":"2024-04-26T21:15:14.997159+0000","last_active":"2024-04-26T21:15:16.593847+0000","last_peered":"2024-04-26T21:15:16.593847+0000","last_clean":"2024-04-26T21:15:16.593847+0000","last_became_active":"2024-04-26T21:15:14.996826+0000","last_became_peered":"2024-04-26T21:15:14.996826+0000","last_unstale":"2024-04-26T21:15:16.593847+0000","last_undegraded":"2024-04-26T21:15:16.593847+0000","last_fullsized":"2024-04-26T21:15:16.593847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998071+0000","last_change":"2024-04-26T21:15:13.995264+0000","last_active":"2024-04-26T21:15:15.998071+0000","last_peered":"2024-04-26T21:15:15.998071+0000","last_clean":"2024-04-26T21:15:15.998071+0000","last_became_active":"2024-04-26T21:15:13.994989+0000","last_became_peered":"2024-04-26T21:15:13.994989+0000","last_unstale":"2024-04-26T21:15:15.998071+0000","last_undegraded":"2024-04-26T21:15:15.998071+0000","last_fullsized":"2024-04-26T21:15:15.998071+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:35.044683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996782+0000","last_change":"2024-04-26T21:15:15.000876+0000","last_active":"2024-04-26T21:15:15.996782+0000","last_peered":"2024-04-26T21:15:15.996782+0000","last_clean":"2024-04-26T21:15:15.996782+0000","last_became_active":"2024-04-26T21:15:15.000234+0000","last_became_peered":"2024-04-26T21:15:15.000234+0000","last_unstale":"2024-04-26T21:15:15.996782+0000","last_undegraded":"2024-04-26T21:15:15.996782+0000","last_fullsized":"2024-04-26T21:15:15.996782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.985094+0000","last_change":"2024-04-26T21:15:13.988753+0000","last_active":"2024-04-26T21:15:16.985094+0000","last_peered":"2024-04-26T21:15:16.985094+0000","last_clean":"2024-04-26T21:15:16.985094+0000","last_became_active":"2024-04-26T21:15:13.988472+0000","last_became_peered":"2024-04-26T21:15:13.988472+0000","last_unstale":"2024-04-26T21:15:16.985094+0000","last_undegraded":"2024-04-26T21:15:16.985094+0000","last_fullsized":"2024-04-26T21:15:16.985094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:17.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594646+0000","last_change":"2024-04-26T21:15:15.007784+0000","last_active":"2024-04-26T21:15:16.594646+0000","last_peered":"2024-04-26T21:15:16.594646+0000","last_clean":"2024-04-26T21:15:16.594646+0000","last_became_active":"2024-04-26T21:15:15.007373+0000","last_became_peered":"2024-04-26T21:15:15.007373+0000","last_unstale":"2024-04-26T21:15:16.594646+0000","last_undegraded":"2024-04-26T21:15:16.594646+0000","last_fullsized":"2024-04-26T21:15:16.594646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:27.459629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993931+0000","last_change":"2024-04-26T21:15:13.985808+0000","last_active":"2024-04-26T21:15:15.993931+0000","last_peered":"2024-04-26T21:15:15.993931+0000","last_clean":"2024-04-26T21:15:15.993931+0000","last_became_active":"2024-04-26T21:15:13.985482+0000","last_became_peered":"2024-04-26T21:15:13.985482+0000","last_unstale":"2024-04-26T21:15:15.993931+0000","last_undegraded":"2024-04-26T21:15:15.993931+0000","last_fullsized":"2024-04-26T21:15:15.993931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:56.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594703+0000","last_change":"2024-04-26T21:15:14.997195+0000","last_active":"2024-04-26T21:15:16.594703+0000","last_peered":"2024-04-26T21:15:16.594703+0000","last_clean":"2024-04-26T21:15:16.594703+0000","last_became_active":"2024-04-26T21:15:14.996889+0000","last_became_peered":"2024-04-26T21:15:14.996889+0000","last_unstale":"2024-04-26T21:15:16.594703+0000","last_undegraded":"2024-04-26T21:15:16.594703+0000","last_fullsized":"2024-04-26T21:15:16.594703+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:03.261175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593581+0000","last_change":"2024-04-26T21:15:14.000033+0000","last_active":"2024-04-26T21:15:16.593581+0000","last_peered":"2024-04-26T21:15:16.593581+0000","last_clean":"2024-04-26T21:15:16.593581+0000","last_became_active":"2024-04-26T21:15:13.999642+0000","last_became_peered":"2024-04-26T21:15:13.999642+0000","last_unstale":"2024-04-26T21:15:16.593581+0000","last_undegraded":"2024-04-26T21:15:16.593581+0000","last_fullsized":"2024-04-26T21:15:16.593581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993953+0000","last_change":"2024-04-26T21:15:14.998038+0000","last_active":"2024-04-26T21:15:15.993953+0000","last_peered":"2024-04-26T21:15:15.993953+0000","last_clean":"2024-04-26T21:15:15.993953+0000","last_became_active":"2024-04-26T21:15:14.997735+0000","last_became_peered":"2024-04-26T21:15:14.997735+0000","last_unstale":"2024-04-26T21:15:15.993953+0000","last_undegraded":"2024-04-26T21:15:15.993953+0000","last_fullsized":"2024-04-26T21:15:15.993953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993592+0000","last_change":"2024-04-26T21:15:13.985339+0000","last_active":"2024-04-26T21:15:15.993592+0000","last_peered":"2024-04-26T21:15:15.993592+0000","last_clean":"2024-04-26T21:15:15.993592+0000","last_became_active":"2024-04-26T21:15:13.984804+0000","last_became_peered":"2024-04-26T21:15:13.984804+0000","last_unstale":"2024-04-26T21:15:15.993592+0000","last_undegraded":"2024-04-26T21:15:15.993592+0000","last_fullsized":"2024-04-26T21:15:15.993592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:08.792790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997529+0000","last_change":"2024-04-26T21:15:15.004362+0000","last_active":"2024-04-26T21:15:15.997529+0000","last_peered":"2024-04-26T21:15:15.997529+0000","last_clean":"2024-04-26T21:15:15.997529+0000","last_became_active":"2024-04-26T21:15:15.003059+0000","last_became_peered":"2024-04-26T21:15:15.003059+0000","last_unstale":"2024-04-26T21:15:15.997529+0000","last_undegraded":"2024-04-26T21:15:15.997529+0000","last_fullsized":"2024-04-26T21:15:15.997529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:04.403950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003690+0000","last_change":"2024-04-26T21:15:14.000183+0000","last_active":"2024-04-26T21:15:14.003690+0000","last_peered":"2024-04-26T21:15:14.003690+0000","last_clean":"2024-04-26T21:15:14.003690+0000","last_became_active":"2024-04-26T21:15:13.999874+0000","last_became_peered":"2024-04-26T21:15:13.999874+0000","last_unstale":"2024-04-26T21:15:14.003690+0000","last_undegraded":"2024-04-26T21:15:14.003690+0000","last_fullsized":"2024-04-26T21:15:14.003690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:18.312063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985817+0000","last_change":"2024-04-26T21:15:13.985817+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985817+0000","last_undegraded":"2024-04-26T21:15:13.985817+0000","last_fullsized":"2024-04-26T21:15:13.985817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593678+0000","last_change":"2024-04-26T21:15:14.000123+0000","last_active":"2024-04-26T21:15:16.593678+0000","last_peered":"2024-04-26T21:15:16.593678+0000","last_clean":"2024-04-26T21:15:16.593678+0000","last_became_active":"2024-04-26T21:15:13.999730+0000","last_became_peered":"2024-04-26T21:15:13.999730+0000","last_unstale":"2024-04-26T21:15:16.593678+0000","last_undegraded":"2024-04-26T21:15:16.593678+0000","last_fullsized":"2024-04-26T21:15:16.593678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:51.654799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997415+0000","last_change":"2024-04-26T21:15:15.004122+0000","last_active":"2024-04-26T21:15:15.997415+0000","last_peered":"2024-04-26T21:15:15.997415+0000","last_clean":"2024-04-26T21:15:15.997415+0000","last_became_active":"2024-04-26T21:15:15.003876+0000","last_became_peered":"2024-04-26T21:15:15.003876+0000","last_unstale":"2024-04-26T21:15:15.997415+0000","last_undegraded":"2024-04-26T21:15:15.997415+0000","last_fullsized":"2024-04-26T21:15:15.997415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:55.716624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998034+0000","last_change":"2024-04-26T21:15:13.992777+0000","last_active":"2024-04-26T21:15:15.998034+0000","last_peered":"2024-04-26T21:15:15.998034+0000","last_clean":"2024-04-26T21:15:15.998034+0000","last_became_active":"2024-04-26T21:15:13.992544+0000","last_became_peered":"2024-04-26T21:15:13.992544+0000","last_unstale":"2024-04-26T21:15:15.998034+0000","last_undegraded":"2024-04-26T21:15:15.998034+0000","last_fullsized":"2024-04-26T21:15:15.998034+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:06.643084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594821+0000","last_change":"2024-04-26T21:15:15.005232+0000","last_active":"2024-04-26T21:15:16.594821+0000","last_peered":"2024-04-26T21:15:16.594821+0000","last_clean":"2024-04-26T21:15:16.594821+0000","last_became_active":"2024-04-26T21:15:15.004904+0000","last_became_peered":"2024-04-26T21:15:15.004904+0000","last_unstale":"2024-04-26T21:15:16.594821+0000","last_undegraded":"2024-04-26T21:15:16.594821+0000","last_fullsized":"2024-04-26T21:15:16.594821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:48.293671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997514+0000","last_change":"2024-04-26T21:15:13.991822+0000","last_active":"2024-04-26T21:15:15.997514+0000","last_peered":"2024-04-26T21:15:15.997514+0000","last_clean":"2024-04-26T21:15:15.997514+0000","last_became_active":"2024-04-26T21:15:13.991508+0000","last_became_peered":"2024-04-26T21:15:13.991508+0000","last_unstale":"2024-04-26T21:15:15.997514+0000","last_undegraded":"2024-04-26T21:15:15.997514+0000","last_fullsized":"2024-04-26T21:15:15.997514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:29.754213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":255,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593633+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:16.593633+0000","last_peered":"2024-04-26T21:15:16.593633+0000","last_clean":"2024-04-26T21:15:16.593633+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:16.593633+0000","last_undegraded":"2024-04-26T21:15:16.593633+0000","last_fullsized":"2024-04-26T21:15:16.593633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":942962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":352070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":352090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":353590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":942346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":338903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 68, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:15:13.986510+0000', 'last_change': '2024-04-26T21:15:13.986510+0000', 'last_active': '2024-04-26T21:15:13.969237+0000', 'last_peered': '2024-04-26T21:15:13.969237+0000', 'last_clean': '2024-04-26T21:15:13.969237+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:15:13.986510+0000', 'last_undegraded': '2024-04-26T21:15:13.986510+0000', 'last_fullsized': '2024-04-26T21:15:13.986510+0000', 'mapping_epoch': 68, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 68, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_clean_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 68, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:15:13.985514+0000', 'last_change': '2024-04-26T21:15:13.985514+0000', 'last_active': '2024-04-26T21:15:13.969237+0000', 'last_peered': '2024-04-26T21:15:13.969237+0000', 'last_clean': '2024-04-26T21:15:13.969237+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:15:13.985514+0000', 'last_undegraded': '2024-04-26T21:15:13.985514+0000', 'last_fullsized': '2024-04-26T21:15:13.985514+0000', 'mapping_epoch': 68, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 68, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_clean_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 68, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:15:13.985659+0000', 'last_change': '2024-04-26T21:15:13.985659+0000', 'last_active': '2024-04-26T21:15:13.969237+0000', 'last_peered': '2024-04-26T21:15:13.969237+0000', 'last_clean': '2024-04-26T21:15:13.969237+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:15:13.985659+0000', 'last_undegraded': '2024-04-26T21:15:13.985659+0000', 'last_fullsized': '2024-04-26T21:15:13.985659+0000', 'mapping_epoch': 68, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 68, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_clean_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-26T21:15:19.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 68, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:15:13.985044+0000', 'last_change': '2024-04-26T21:15:13.985044+0000', 'last_active': '2024-04-26T21:15:13.969237+0000', 'last_peered': '2024-04-26T21:15:13.969237+0000', 'last_clean': '2024-04-26T21:15:13.969237+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:15:13.985044+0000', 'last_undegraded': '2024-04-26T21:15:13.985044+0000', 'last_fullsized': '2024-04-26T21:15:13.985044+0000', 'mapping_epoch': 68, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 68, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_clean_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:15:19.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-26T21:15:19.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 68, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:15:13.985817+0000', 'last_change': '2024-04-26T21:15:13.985817+0000', 'last_active': '2024-04-26T21:15:13.969237+0000', 'last_peered': '2024-04-26T21:15:13.969237+0000', 'last_clean': '2024-04-26T21:15:13.969237+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:15:13.985817+0000', 'last_undegraded': '2024-04-26T21:15:13.985817+0000', 'last_fullsized': '2024-04-26T21:15:13.985817+0000', 'mapping_epoch': 68, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 68, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'last_clean_scrub_stamp': '2024-04-26T21:15:13.969237+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:15:19.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:15:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.855+0000 7fd0e1a83640 1 Processor -- start 2024-04-26T21:15:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.855+0000 7fd0e1a83640 1 -- start start 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0dc0d1b80 0x7fd0dc0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 0x7fd0dc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0dc0d0220 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd0dc0d0500 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0e1a83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc0d07e0 con 0x7fd0dc0d3fa0 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51256/0 (socket says 172.21.15.40:51256) 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 -- 172.21.15.40:0/2448167302 learned_addr learned my addr 172.21.15.40:0/2448167302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0dbfff640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 0x7fd0dc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0db7fe640 1 --2- 172.21.15.40:0/2448167302 >> v2:172.21.15.184:3300/0 conn(0x7fd0dc0d1b80 0x7fd0dc0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 -- 172.21.15.40:0/2448167302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 msgr2=0x7fd0dc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 0x7fd0dc0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 -- 172.21.15.40:0/2448167302 >> v2:172.21.15.184:3300/0 conn(0x7fd0dc0d1b80 msgr2=0x7fd0dc0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 --2- 172.21.15.40:0/2448167302 >> v2:172.21.15.184:3300/0 conn(0x7fd0dc0d1b80 0x7fd0dc0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0daffd640 1 -- 172.21.15.40:0/2448167302 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc0d0ac0 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.856+0000 7fd0dbfff640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 0x7fd0dc0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0daffd640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc08f3d0 tx=0x7fd0d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb93d7a7804d765 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0da7fc640 1 -- 172.21.15.40:0/2448167302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0037070 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0da7fc640 1 -- 172.21.15.40:0/2448167302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0032440 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0da7fc640 1 -- 172.21.15.40:0/2448167302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0032810 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 -- 172.21.15.40:0/2448167302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0cfda0 msgr2=0x7fd0dc0d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc08f3d0 tx=0x7fd0d002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 -- 172.21.15.40:0/2448167302 shutdown_connections 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0d3fa0 0x7fd0dc0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/2448167302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0cfda0 0x7fd0dc0d3a60 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/2448167302 >> v2:172.21.15.184:3300/0 conn(0x7fd0dc0d1b80 0x7fd0dc0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.857+0000 7fd0e1a83640 1 -- 172.21.15.40:0/2448167302 >> 172.21.15.40:0/2448167302 conn(0x7fd0dc09e540 msgr2=0x7fd0dc0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.858+0000 7fd0e1a83640 1 -- 172.21.15.40:0/2448167302 shutdown_connections 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.858+0000 7fd0e1a83640 1 -- 172.21.15.40:0/2448167302 wait complete. 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.859+0000 7fd0e1a83640 1 Processor -- start 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.859+0000 7fd0e1a83640 1 -- start start 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0cfda0 0x7fd0dc15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0dc0d3fa0 0x7fd0dc159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc0b6760 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc0b65e0 con 0x7fd0dc0d3fa0 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc0b68e0 con 0x7fd0dc0cfda0 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51268/0 (socket says 172.21.15.40:51268) 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 -- 172.21.15.40:0/1033848623 learned_addr learned my addr 172.21.15.40:0/1033848623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0db7fe640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0cfda0 0x7fd0dc15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0dbfff640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0dc0d3fa0 0x7fd0dc159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 -- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0cfda0 msgr2=0x7fd0dc15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0cfda0 0x7fd0dc15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 -- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0dc0d3fa0 msgr2=0x7fd0dc159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0dc0d3fa0 0x7fd0dc159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 -- 172.21.15.40:0/1033848623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc15a4e0 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0daffd640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0000c00 tx=0x7fd0d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0041020 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d00306c0 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc15fa40 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0037040 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.860+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc0cd850 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.862+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc06a290 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.862+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0d0044020 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.862+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.862+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d00ac370 con 0x7fd0dc0d1b80 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.863+0000 7fd0db7fe640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.863+0000 7fd0db7fe640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 0x7fd0c806edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc006fd0 tx=0x7fd0cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:19.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:19.867+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0d004b030 con 0x7fd0dc0d1b80 2024-04-26T21:15:20.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.122+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd0dc0d27b0 con 0x7fd0dc0d1b80 2024-04-26T21:15:20.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.123+0000 7fd0c3fff640 1 -- 172.21.15.40:0/1033848623 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1517 (secure 0 0 0) 0x7fd0d007a7b0 con 0x7fd0dc0d1b80 2024-04-26T21:15:20.124 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:20.124 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":223,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":591132,"bytes_used":246394880,"bytes_avail":772847718400,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":683,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":38,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6453}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-26T21:15:02.960374+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 msgr2=0x7fd0c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 0x7fd0c806edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc006fd0 tx=0x7fd0cc008040 comp rx=0 tx=0).stop 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 msgr2=0x7fd0dc157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0000c00 tx=0x7fd0d00046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 shutdown_connections 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0dc0d3fa0 0x7fd0dc159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0dc0d1b80 0x7fd0dc157ba0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 --2- 172.21.15.40:0/1033848623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0dc0cfda0 0x7fd0dc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 >> 172.21.15.40:0/1033848623 conn(0x7fd0dc09e540 msgr2=0x7fd0dc0b9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:20.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 shutdown_connections 2024-04-26T21:15:20.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.125+0000 7fd0e1a83640 1 -- 172.21.15.40:0/1033848623 wait complete. 2024-04-26T21:15:20.136 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:15:20.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:15:20.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.236+0000 7f6e59ba4640 1 Processor -- start 2024-04-26T21:15:20.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.236+0000 7f6e59ba4640 1 -- start start 2024-04-26T21:15:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e59ba4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5413ff70 0x7f6e54146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e59ba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e59ba4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e540d05b0 con 0x7f6e54140350 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e59ba4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6e540d0890 con 0x7f6e5413ff70 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e59ba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5406e450 con 0x7f6e540cff20 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e53fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e537fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5413ff70 0x7f6e54146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e537fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5413ff70 0x7f6e54146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:36498/0 (socket says 172.21.15.40:36498) 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e53fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:51950/0 (socket says 172.21.15.40:51950) 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.237+0000 7f6e537fe640 1 -- 172.21.15.40:0/2251206260 learned_addr learned my addr 172.21.15.40:0/2251206260 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e537fe640 1 -- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 msgr2=0x7f6e541465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e537fe640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e537fe640 1 -- 172.21.15.40:0/2251206260 >> v1:172.21.15.184:6789/0 conn(0x7f6e54140350 legacy=0x7f6e540cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e537fe640 1 -- 172.21.15.40:0/2251206260 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5406e730 con 0x7f6e5413ff70 2024-04-26T21:15:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e53fff640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e537fe640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5413ff70 0x7f6e54146080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009870 tx=0x7f6e4402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0946b8cbdf3a433 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.238+0000 7f6e527fc640 1 -- 172.21.15.40:0/2251206260 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44037070 con 0x7f6e5413ff70 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e527fc640 1 -- 172.21.15.40:0/2251206260 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44032440 con 0x7f6e5413ff70 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e527fc640 1 -- 172.21.15.40:0/2251206260 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44032810 con 0x7f6e5413ff70 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 -- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5413ff70 msgr2=0x7f6e54146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5413ff70 0x7f6e54146080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009870 tx=0x7f6e4402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 -- 172.21.15.40:0/2251206260 shutdown_connections 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540cff20 0x7f6e541465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/2251206260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5413ff70 0x7f6e54146080 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 -- 172.21.15.40:0/2251206260 >> 172.21.15.40:0/2251206260 conn(0x7f6e540d0d30 msgr2=0x7f6e540d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 -- 172.21.15.40:0/2251206260 shutdown_connections 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.239+0000 7f6e59ba4640 1 -- 172.21.15.40:0/2251206260 wait complete. 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.240+0000 7f6e59ba4640 1 Processor -- start 2024-04-26T21:15:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.240+0000 7f6e59ba4640 1 -- start start 2024-04-26T21:15:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5413ff70 0x7f6e540b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54140350 0x7f6e54142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5406efe0 con 0x7f6e540cff20 2024-04-26T21:15:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5406ee60 con 0x7f6e54140350 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e59ba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5406f160 con 0x7f6e5413ff70 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51270/0 (socket says 172.21.15.40:51270) 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.241+0000 7f6e537fe640 1 -- 172.21.15.40:0/1892241930 learned_addr learned my addr 172.21.15.40:0/1892241930 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e52ffd640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5413ff70 0x7f6e540b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e53fff640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54140350 0x7f6e54142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 -- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5413ff70 msgr2=0x7f6e540b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5413ff70 0x7f6e540b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 -- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54140350 msgr2=0x7f6e54142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54140350 0x7f6e54142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 -- 172.21.15.40:0/1892241930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e540ae8b0 con 0x7f6e540cff20 2024-04-26T21:15:20.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e537fe640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6e44000c00 tx=0x7f6e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44037040 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44042070 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e540aeb90 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44030880 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.242+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e540af150 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.243+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e541f6250 con 0x7f6e540cff20 2024-04-26T21:15:20.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.244+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e44030a20 con 0x7f6e540cff20 2024-04-26T21:15:20.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.244+0000 7f6e3bfff640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 0x7f6e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:20.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.244+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f6e440ac3a0 con 0x7f6e540cff20 2024-04-26T21:15:20.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.244+0000 7f6e52ffd640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 0x7f6e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:20.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.245+0000 7f6e52ffd640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 0x7f6e4006edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007fd0 tx=0x7f6e48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:20.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.248+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e4407a7c0 con 0x7f6e540cff20 2024-04-26T21:15:20.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.430+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e540b45e0 con 0x7f6e4006c8f0 2024-04-26T21:15:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.437+0000 7f6e3bfff640 1 -- 172.21.15.40:0/1892241930 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251642 (secure 0 0 0) 0x7f6e540b45e0 con 0x7f6e4006c8f0 2024-04-26T21:15:20.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:20.441 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 msgr2=0x7f6e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 0x7f6e4006edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007fd0 tx=0x7f6e48009040 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 msgr2=0x7f6e54073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6e44000c00 tx=0x7f6e440046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 shutdown_connections 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4006c8f0 0x7f6e4006edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54140350 0x7f6e54142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5413ff70 0x7f6e540b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 --2- 172.21.15.40:0/1892241930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e540cff20 0x7f6e54073800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:20.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 >> 172.21.15.40:0/1892241930 conn(0x7f6e540d0d30 msgr2=0x7f6e540b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 shutdown_connections 2024-04-26T21:15:20.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:20.440+0000 7f6e59ba4640 1 -- 172.21.15.40:0/1892241930 wait complete. 2024-04-26T21:15:20.452 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-26T21:15:18.963801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240620,"kb_used_data":7772,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734100,"statfs":{"total":773094113280,"available":772847718400,"internally_reserved":0,"allocated":7958528,"data_stored":3986077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993743"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594048+0000","last_change":"2024-04-26T21:15:15.008421+0000","last_active":"2024-04-26T21:15:16.594048+0000","last_peered":"2024-04-26T21:15:16.594048+0000","last_clean":"2024-04-26T21:15:16.594048+0000","last_became_active":"2024-04-26T21:15:15.008122+0000","last_became_peered":"2024-04-26T21:15:15.008122+0000","last_unstale":"2024-04-26T21:15:16.594048+0000","last_undegraded":"2024-04-26T21:15:16.594048+0000","last_fullsized":"2024-04-26T21:15:16.594048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:35.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998310+0000","last_change":"2024-04-26T21:15:15.004592+0000","last_active":"2024-04-26T21:15:15.998310+0000","last_peered":"2024-04-26T21:15:15.998310+0000","last_clean":"2024-04-26T21:15:15.998310+0000","last_became_active":"2024-04-26T21:15:15.004314+0000","last_became_peered":"2024-04-26T21:15:15.004314+0000","last_unstale":"2024-04-26T21:15:15.998310+0000","last_undegraded":"2024-04-26T21:15:15.998310+0000","last_fullsized":"2024-04-26T21:15:15.998310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595104+0000","last_change":"2024-04-26T21:15:14.984552+0000","last_active":"2024-04-26T21:15:16.595104+0000","last_peered":"2024-04-26T21:15:16.595104+0000","last_clean":"2024-04-26T21:15:16.595104+0000","last_became_active":"2024-04-26T21:15:14.984224+0000","last_became_peered":"2024-04-26T21:15:14.984224+0000","last_unstale":"2024-04-26T21:15:16.595104+0000","last_undegraded":"2024-04-26T21:15:16.595104+0000","last_fullsized":"2024-04-26T21:15:16.595104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:52.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.986510+0000","last_change":"2024-04-26T21:15:13.986510+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.986510+0000","last_undegraded":"2024-04-26T21:15:13.986510+0000","last_fullsized":"2024-04-26T21:15:13.986510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993499+0000","last_change":"2024-04-26T21:15:15.008338+0000","last_active":"2024-04-26T21:15:15.993499+0000","last_peered":"2024-04-26T21:15:15.993499+0000","last_clean":"2024-04-26T21:15:15.993499+0000","last_became_active":"2024-04-26T21:15:15.008029+0000","last_became_peered":"2024-04-26T21:15:15.008029+0000","last_unstale":"2024-04-26T21:15:15.993499+0000","last_undegraded":"2024-04-26T21:15:15.993499+0000","last_fullsized":"2024-04-26T21:15:15.993499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:02.371753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997414+0000","last_change":"2024-04-26T21:15:15.003006+0000","last_active":"2024-04-26T21:15:15.997414+0000","last_peered":"2024-04-26T21:15:15.997414+0000","last_clean":"2024-04-26T21:15:15.997414+0000","last_became_active":"2024-04-26T21:15:15.002731+0000","last_became_peered":"2024-04-26T21:15:15.002731+0000","last_unstale":"2024-04-26T21:15:15.997414+0000","last_undegraded":"2024-04-26T21:15:15.997414+0000","last_fullsized":"2024-04-26T21:15:15.997414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:45.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996492+0000","last_change":"2024-04-26T21:15:15.000633+0000","last_active":"2024-04-26T21:15:15.996492+0000","last_peered":"2024-04-26T21:15:15.996492+0000","last_clean":"2024-04-26T21:15:15.996492+0000","last_became_active":"2024-04-26T21:15:15.000275+0000","last_became_peered":"2024-04-26T21:15:15.000275+0000","last_unstale":"2024-04-26T21:15:15.996492+0000","last_undegraded":"2024-04-26T21:15:15.996492+0000","last_fullsized":"2024-04-26T21:15:15.996492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:11.788021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997948+0000","last_change":"2024-04-26T21:15:15.002868+0000","last_active":"2024-04-26T21:15:15.997948+0000","last_peered":"2024-04-26T21:15:15.997948+0000","last_clean":"2024-04-26T21:15:15.997948+0000","last_became_active":"2024-04-26T21:15:15.002561+0000","last_became_peered":"2024-04-26T21:15:15.002561+0000","last_unstale":"2024-04-26T21:15:15.997948+0000","last_undegraded":"2024-04-26T21:15:15.997948+0000","last_fullsized":"2024-04-26T21:15:15.997948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994128+0000","last_change":"2024-04-26T21:15:15.004629+0000","last_active":"2024-04-26T21:15:15.994128+0000","last_peered":"2024-04-26T21:15:15.994128+0000","last_clean":"2024-04-26T21:15:15.994128+0000","last_became_active":"2024-04-26T21:15:15.004365+0000","last_became_peered":"2024-04-26T21:15:15.004365+0000","last_unstale":"2024-04-26T21:15:15.994128+0000","last_undegraded":"2024-04-26T21:15:15.994128+0000","last_fullsized":"2024-04-26T21:15:15.994128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:33.056050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594298+0000","last_change":"2024-04-26T21:15:15.005575+0000","last_active":"2024-04-26T21:15:16.594298+0000","last_peered":"2024-04-26T21:15:16.594298+0000","last_clean":"2024-04-26T21:15:16.594298+0000","last_became_active":"2024-04-26T21:15:15.005289+0000","last_became_peered":"2024-04-26T21:15:15.005289+0000","last_unstale":"2024-04-26T21:15:16.594298+0000","last_undegraded":"2024-04-26T21:15:16.594298+0000","last_fullsized":"2024-04-26T21:15:16.594298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:18.088546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594550+0000","last_change":"2024-04-26T21:15:15.000155+0000","last_active":"2024-04-26T21:15:16.594550+0000","last_peered":"2024-04-26T21:15:16.594550+0000","last_clean":"2024-04-26T21:15:16.594550+0000","last_became_active":"2024-04-26T21:15:14.999858+0000","last_became_peered":"2024-04-26T21:15:14.999858+0000","last_unstale":"2024-04-26T21:15:16.594550+0000","last_undegraded":"2024-04-26T21:15:16.594550+0000","last_fullsized":"2024-04-26T21:15:16.594550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:01.080096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985514+0000","last_change":"2024-04-26T21:15:13.985514+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985514+0000","last_undegraded":"2024-04-26T21:15:13.985514+0000","last_fullsized":"2024-04-26T21:15:13.985514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593769+0000","last_change":"2024-04-26T21:15:15.001688+0000","last_active":"2024-04-26T21:15:16.593769+0000","last_peered":"2024-04-26T21:15:16.593769+0000","last_clean":"2024-04-26T21:15:16.593769+0000","last_became_active":"2024-04-26T21:15:15.001372+0000","last_became_peered":"2024-04-26T21:15:15.001372+0000","last_unstale":"2024-04-26T21:15:16.593769+0000","last_undegraded":"2024-04-26T21:15:16.593769+0000","last_fullsized":"2024-04-26T21:15:16.593769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993853+0000","last_change":"2024-04-26T21:15:14.995986+0000","last_active":"2024-04-26T21:15:15.993853+0000","last_peered":"2024-04-26T21:15:15.993853+0000","last_clean":"2024-04-26T21:15:15.993853+0000","last_became_active":"2024-04-26T21:15:14.995673+0000","last_became_peered":"2024-04-26T21:15:14.995673+0000","last_unstale":"2024-04-26T21:15:15.993853+0000","last_undegraded":"2024-04-26T21:15:15.993853+0000","last_fullsized":"2024-04-26T21:15:15.993853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593933+0000","last_change":"2024-04-26T21:15:14.996331+0000","last_active":"2024-04-26T21:15:16.593933+0000","last_peered":"2024-04-26T21:15:16.593933+0000","last_clean":"2024-04-26T21:15:16.593933+0000","last_became_active":"2024-04-26T21:15:14.996044+0000","last_became_peered":"2024-04-26T21:15:14.996044+0000","last_unstale":"2024-04-26T21:15:16.593933+0000","last_undegraded":"2024-04-26T21:15:16.593933+0000","last_fullsized":"2024-04-26T21:15:16.593933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593583+0000","last_change":"2024-04-26T21:15:15.007557+0000","last_active":"2024-04-26T21:15:16.593583+0000","last_peered":"2024-04-26T21:15:16.593583+0000","last_clean":"2024-04-26T21:15:16.593583+0000","last_became_active":"2024-04-26T21:15:15.007269+0000","last_became_peered":"2024-04-26T21:15:15.007269+0000","last_unstale":"2024-04-26T21:15:16.593583+0000","last_undegraded":"2024-04-26T21:15:16.593583+0000","last_fullsized":"2024-04-26T21:15:16.593583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994100+0000","last_change":"2024-04-26T21:15:15.003937+0000","last_active":"2024-04-26T21:15:15.994100+0000","last_peered":"2024-04-26T21:15:15.994100+0000","last_clean":"2024-04-26T21:15:15.994100+0000","last_became_active":"2024-04-26T21:15:15.003652+0000","last_became_peered":"2024-04-26T21:15:15.003652+0000","last_unstale":"2024-04-26T21:15:15.994100+0000","last_undegraded":"2024-04-26T21:15:15.994100+0000","last_fullsized":"2024-04-26T21:15:15.994100+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.457492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985659+0000","last_change":"2024-04-26T21:15:13.985659+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985659+0000","last_undegraded":"2024-04-26T21:15:13.985659+0000","last_fullsized":"2024-04-26T21:15:13.985659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997884+0000","last_change":"2024-04-26T21:15:15.003066+0000","last_active":"2024-04-26T21:15:15.997884+0000","last_peered":"2024-04-26T21:15:15.997884+0000","last_clean":"2024-04-26T21:15:15.997884+0000","last_became_active":"2024-04-26T21:15:15.002597+0000","last_became_peered":"2024-04-26T21:15:15.002597+0000","last_unstale":"2024-04-26T21:15:15.997884+0000","last_undegraded":"2024-04-26T21:15:15.997884+0000","last_fullsized":"2024-04-26T21:15:15.997884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593869+0000","last_change":"2024-04-26T21:15:14.996454+0000","last_active":"2024-04-26T21:15:16.593869+0000","last_peered":"2024-04-26T21:15:16.593869+0000","last_clean":"2024-04-26T21:15:16.593869+0000","last_became_active":"2024-04-26T21:15:14.996140+0000","last_became_peered":"2024-04-26T21:15:14.996140+0000","last_unstale":"2024-04-26T21:15:16.593869+0000","last_undegraded":"2024-04-26T21:15:16.593869+0000","last_fullsized":"2024-04-26T21:15:16.593869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593784+0000","last_change":"2024-04-26T21:15:15.004956+0000","last_active":"2024-04-26T21:15:16.593784+0000","last_peered":"2024-04-26T21:15:16.593784+0000","last_clean":"2024-04-26T21:15:16.593784+0000","last_became_active":"2024-04-26T21:15:15.004666+0000","last_became_peered":"2024-04-26T21:15:15.004666+0000","last_unstale":"2024-04-26T21:15:16.593784+0000","last_undegraded":"2024-04-26T21:15:16.593784+0000","last_fullsized":"2024-04-26T21:15:16.593784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593651+0000","last_change":"2024-04-26T21:15:14.998310+0000","last_active":"2024-04-26T21:15:16.593651+0000","last_peered":"2024-04-26T21:15:16.593651+0000","last_clean":"2024-04-26T21:15:16.593651+0000","last_became_active":"2024-04-26T21:15:14.998039+0000","last_became_peered":"2024-04-26T21:15:14.998039+0000","last_unstale":"2024-04-26T21:15:16.593651+0000","last_undegraded":"2024-04-26T21:15:16.593651+0000","last_fullsized":"2024-04-26T21:15:16.593651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:14.997132+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:14.996824+0000","last_became_peered":"2024-04-26T21:15:14.996824+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593965+0000","last_change":"2024-04-26T21:15:15.003818+0000","last_active":"2024-04-26T21:15:16.593965+0000","last_peered":"2024-04-26T21:15:16.593965+0000","last_clean":"2024-04-26T21:15:16.593965+0000","last_became_active":"2024-04-26T21:15:15.003428+0000","last_became_peered":"2024-04-26T21:15:15.003428+0000","last_unstale":"2024-04-26T21:15:16.593965+0000","last_undegraded":"2024-04-26T21:15:16.593965+0000","last_fullsized":"2024-04-26T21:15:16.593965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594069+0000","last_change":"2024-04-26T21:15:15.004314+0000","last_active":"2024-04-26T21:15:16.594069+0000","last_peered":"2024-04-26T21:15:16.594069+0000","last_clean":"2024-04-26T21:15:16.594069+0000","last_became_active":"2024-04-26T21:15:15.004022+0000","last_became_peered":"2024-04-26T21:15:15.004022+0000","last_unstale":"2024-04-26T21:15:16.594069+0000","last_undegraded":"2024-04-26T21:15:16.594069+0000","last_fullsized":"2024-04-26T21:15:16.594069+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996953+0000","last_change":"2024-04-26T21:15:15.006413+0000","last_active":"2024-04-26T21:15:15.996953+0000","last_peered":"2024-04-26T21:15:15.996953+0000","last_clean":"2024-04-26T21:15:15.996953+0000","last_became_active":"2024-04-26T21:15:15.006136+0000","last_became_peered":"2024-04-26T21:15:15.006136+0000","last_unstale":"2024-04-26T21:15:15.996953+0000","last_undegraded":"2024-04-26T21:15:15.996953+0000","last_fullsized":"2024-04-26T21:15:15.996953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595237+0000","last_change":"2024-04-26T21:15:14.999635+0000","last_active":"2024-04-26T21:15:16.595237+0000","last_peered":"2024-04-26T21:15:16.595237+0000","last_clean":"2024-04-26T21:15:16.595237+0000","last_became_active":"2024-04-26T21:15:14.999351+0000","last_became_peered":"2024-04-26T21:15:14.999351+0000","last_unstale":"2024-04-26T21:15:16.595237+0000","last_undegraded":"2024-04-26T21:15:16.595237+0000","last_fullsized":"2024-04-26T21:15:16.595237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:23.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594169+0000","last_change":"2024-04-26T21:15:15.006522+0000","last_active":"2024-04-26T21:15:16.594169+0000","last_peered":"2024-04-26T21:15:16.594169+0000","last_clean":"2024-04-26T21:15:16.594169+0000","last_became_active":"2024-04-26T21:15:15.005849+0000","last_became_peered":"2024-04-26T21:15:15.005849+0000","last_unstale":"2024-04-26T21:15:16.594169+0000","last_undegraded":"2024-04-26T21:15:16.594169+0000","last_fullsized":"2024-04-26T21:15:16.594169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:16.396516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594834+0000","last_change":"2024-04-26T21:15:14.988314+0000","last_active":"2024-04-26T21:15:16.594834+0000","last_peered":"2024-04-26T21:15:16.594834+0000","last_clean":"2024-04-26T21:15:16.594834+0000","last_became_active":"2024-04-26T21:15:14.987983+0000","last_became_peered":"2024-04-26T21:15:14.987983+0000","last_unstale":"2024-04-26T21:15:16.594834+0000","last_undegraded":"2024-04-26T21:15:16.594834+0000","last_fullsized":"2024-04-26T21:15:16.594834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:56.347854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996624+0000","last_change":"2024-04-26T21:15:15.001705+0000","last_active":"2024-04-26T21:15:15.996624+0000","last_peered":"2024-04-26T21:15:15.996624+0000","last_clean":"2024-04-26T21:15:15.996624+0000","last_became_active":"2024-04-26T21:15:15.001398+0000","last_became_peered":"2024-04-26T21:15:15.001398+0000","last_unstale":"2024-04-26T21:15:15.996624+0000","last_undegraded":"2024-04-26T21:15:15.996624+0000","last_fullsized":"2024-04-26T21:15:15.996624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996883+0000","last_change":"2024-04-26T21:15:14.996277+0000","last_active":"2024-04-26T21:15:15.996883+0000","last_peered":"2024-04-26T21:15:15.996883+0000","last_clean":"2024-04-26T21:15:15.996883+0000","last_became_active":"2024-04-26T21:15:14.996006+0000","last_became_peered":"2024-04-26T21:15:14.996006+0000","last_unstale":"2024-04-26T21:15:15.996883+0000","last_undegraded":"2024-04-26T21:15:15.996883+0000","last_fullsized":"2024-04-26T21:15:15.996883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997797+0000","last_change":"2024-04-26T21:15:15.005406+0000","last_active":"2024-04-26T21:15:15.997797+0000","last_peered":"2024-04-26T21:15:15.997797+0000","last_clean":"2024-04-26T21:15:15.997797+0000","last_became_active":"2024-04-26T21:15:15.005116+0000","last_became_peered":"2024-04-26T21:15:15.005116+0000","last_unstale":"2024-04-26T21:15:15.997797+0000","last_undegraded":"2024-04-26T21:15:15.997797+0000","last_fullsized":"2024-04-26T21:15:15.997797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:16.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594603+0000","last_change":"2024-04-26T21:15:15.008320+0000","last_active":"2024-04-26T21:15:16.594603+0000","last_peered":"2024-04-26T21:15:16.594603+0000","last_clean":"2024-04-26T21:15:16.594603+0000","last_became_active":"2024-04-26T21:15:15.008048+0000","last_became_peered":"2024-04-26T21:15:15.008048+0000","last_unstale":"2024-04-26T21:15:16.594603+0000","last_undegraded":"2024-04-26T21:15:16.594603+0000","last_fullsized":"2024-04-26T21:15:16.594603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.353647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594502+0000","last_change":"2024-04-26T21:15:13.991660+0000","last_active":"2024-04-26T21:15:16.594502+0000","last_peered":"2024-04-26T21:15:16.594502+0000","last_clean":"2024-04-26T21:15:16.594502+0000","last_became_active":"2024-04-26T21:15:13.991361+0000","last_became_peered":"2024-04-26T21:15:13.991361+0000","last_unstale":"2024-04-26T21:15:16.594502+0000","last_undegraded":"2024-04-26T21:15:16.594502+0000","last_fullsized":"2024-04-26T21:15:16.594502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994190+0000","last_change":"2024-04-26T21:15:15.003269+0000","last_active":"2024-04-26T21:15:15.994190+0000","last_peered":"2024-04-26T21:15:15.994190+0000","last_clean":"2024-04-26T21:15:15.994190+0000","last_became_active":"2024-04-26T21:15:15.002973+0000","last_became_peered":"2024-04-26T21:15:15.002973+0000","last_unstale":"2024-04-26T21:15:15.994190+0000","last_undegraded":"2024-04-26T21:15:15.994190+0000","last_fullsized":"2024-04-26T21:15:15.994190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:30.756090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593977+0000","last_change":"2024-04-26T21:15:13.990425+0000","last_active":"2024-04-26T21:15:16.593977+0000","last_peered":"2024-04-26T21:15:16.593977+0000","last_clean":"2024-04-26T21:15:16.593977+0000","last_became_active":"2024-04-26T21:15:13.990113+0000","last_became_peered":"2024-04-26T21:15:13.990113+0000","last_unstale":"2024-04-26T21:15:16.593977+0000","last_undegraded":"2024-04-26T21:15:16.593977+0000","last_fullsized":"2024-04-26T21:15:16.593977+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997715+0000","last_change":"2024-04-26T21:15:15.004144+0000","last_active":"2024-04-26T21:15:15.997715+0000","last_peered":"2024-04-26T21:15:15.997715+0000","last_clean":"2024-04-26T21:15:15.997715+0000","last_became_active":"2024-04-26T21:15:15.003804+0000","last_became_peered":"2024-04-26T21:15:15.003804+0000","last_unstale":"2024-04-26T21:15:15.997715+0000","last_undegraded":"2024-04-26T21:15:15.997715+0000","last_fullsized":"2024-04-26T21:15:15.997715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:34.213418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594471+0000","last_change":"2024-04-26T21:15:13.996784+0000","last_active":"2024-04-26T21:15:16.594471+0000","last_peered":"2024-04-26T21:15:16.594471+0000","last_clean":"2024-04-26T21:15:16.594471+0000","last_became_active":"2024-04-26T21:15:13.996508+0000","last_became_peered":"2024-04-26T21:15:13.996508+0000","last_unstale":"2024-04-26T21:15:16.594471+0000","last_undegraded":"2024-04-26T21:15:16.594471+0000","last_fullsized":"2024-04-26T21:15:16.594471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:31.090506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594503+0000","last_change":"2024-04-26T21:15:14.998880+0000","last_active":"2024-04-26T21:15:16.594503+0000","last_peered":"2024-04-26T21:15:16.594503+0000","last_clean":"2024-04-26T21:15:16.594503+0000","last_became_active":"2024-04-26T21:15:14.998406+0000","last_became_peered":"2024-04-26T21:15:14.998406+0000","last_unstale":"2024-04-26T21:15:16.594503+0000","last_undegraded":"2024-04-26T21:15:16.594503+0000","last_fullsized":"2024-04-26T21:15:16.594503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:32.918934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997990+0000","last_change":"2024-04-26T21:15:13.990545+0000","last_active":"2024-04-26T21:15:15.997990+0000","last_peered":"2024-04-26T21:15:15.997990+0000","last_clean":"2024-04-26T21:15:15.997990+0000","last_became_active":"2024-04-26T21:15:13.990151+0000","last_became_peered":"2024-04-26T21:15:13.990151+0000","last_unstale":"2024-04-26T21:15:15.997990+0000","last_undegraded":"2024-04-26T21:15:15.997990+0000","last_fullsized":"2024-04-26T21:15:15.997990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:43.327070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996712+0000","last_change":"2024-04-26T21:15:14.999455+0000","last_active":"2024-04-26T21:15:15.996712+0000","last_peered":"2024-04-26T21:15:15.996712+0000","last_clean":"2024-04-26T21:15:15.996712+0000","last_became_active":"2024-04-26T21:15:14.999063+0000","last_became_peered":"2024-04-26T21:15:14.999063+0000","last_unstale":"2024-04-26T21:15:15.996712+0000","last_undegraded":"2024-04-26T21:15:15.996712+0000","last_fullsized":"2024-04-26T21:15:15.996712+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595004+0000","last_change":"2024-04-26T21:15:14.000741+0000","last_active":"2024-04-26T21:15:16.595004+0000","last_peered":"2024-04-26T21:15:16.595004+0000","last_clean":"2024-04-26T21:15:16.595004+0000","last_became_active":"2024-04-26T21:15:14.000484+0000","last_became_peered":"2024-04-26T21:15:14.000484+0000","last_unstale":"2024-04-26T21:15:16.595004+0000","last_undegraded":"2024-04-26T21:15:16.595004+0000","last_fullsized":"2024-04-26T21:15:16.595004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:34.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:15.003590+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:15.003252+0000","last_became_peered":"2024-04-26T21:15:15.003252+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:40.805512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997494+0000","last_change":"2024-04-26T21:15:13.987191+0000","last_active":"2024-04-26T21:15:15.997494+0000","last_peered":"2024-04-26T21:15:15.997494+0000","last_clean":"2024-04-26T21:15:15.997494+0000","last_became_active":"2024-04-26T21:15:13.986872+0000","last_became_peered":"2024-04-26T21:15:13.986872+0000","last_unstale":"2024-04-26T21:15:15.997494+0000","last_undegraded":"2024-04-26T21:15:15.997494+0000","last_fullsized":"2024-04-26T21:15:15.997494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:02.357589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993432+0000","last_change":"2024-04-26T21:15:15.002302+0000","last_active":"2024-04-26T21:15:15.993432+0000","last_peered":"2024-04-26T21:15:15.993432+0000","last_clean":"2024-04-26T21:15:15.993432+0000","last_became_active":"2024-04-26T21:15:15.001311+0000","last_became_peered":"2024-04-26T21:15:15.001311+0000","last_unstale":"2024-04-26T21:15:15.993432+0000","last_undegraded":"2024-04-26T21:15:15.993432+0000","last_fullsized":"2024-04-26T21:15:15.993432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:08.309553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003614+0000","last_change":"2024-04-26T21:15:13.998731+0000","last_active":"2024-04-26T21:15:14.003614+0000","last_peered":"2024-04-26T21:15:14.003614+0000","last_clean":"2024-04-26T21:15:14.003614+0000","last_became_active":"2024-04-26T21:15:13.998420+0000","last_became_peered":"2024-04-26T21:15:13.998420+0000","last_unstale":"2024-04-26T21:15:14.003614+0000","last_undegraded":"2024-04-26T21:15:14.003614+0000","last_fullsized":"2024-04-26T21:15:14.003614+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:03.412018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593528+0000","last_change":"2024-04-26T21:15:14.999599+0000","last_active":"2024-04-26T21:15:16.593528+0000","last_peered":"2024-04-26T21:15:16.593528+0000","last_clean":"2024-04-26T21:15:16.593528+0000","last_became_active":"2024-04-26T21:15:14.998302+0000","last_became_peered":"2024-04-26T21:15:14.998302+0000","last_unstale":"2024-04-26T21:15:16.593528+0000","last_undegraded":"2024-04-26T21:15:16.593528+0000","last_fullsized":"2024-04-26T21:15:16.593528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993426+0000","last_change":"2024-04-26T21:15:13.989877+0000","last_active":"2024-04-26T21:15:15.993426+0000","last_peered":"2024-04-26T21:15:15.993426+0000","last_clean":"2024-04-26T21:15:15.993426+0000","last_became_active":"2024-04-26T21:15:13.989441+0000","last_became_peered":"2024-04-26T21:15:13.989441+0000","last_unstale":"2024-04-26T21:15:15.993426+0000","last_undegraded":"2024-04-26T21:15:15.993426+0000","last_fullsized":"2024-04-26T21:15:15.993426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:41.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997022+0000","last_change":"2024-04-26T21:15:14.999473+0000","last_active":"2024-04-26T21:15:15.997022+0000","last_peered":"2024-04-26T21:15:15.997022+0000","last_clean":"2024-04-26T21:15:15.997022+0000","last_became_active":"2024-04-26T21:15:14.999179+0000","last_became_peered":"2024-04-26T21:15:14.999179+0000","last_unstale":"2024-04-26T21:15:15.997022+0000","last_undegraded":"2024-04-26T21:15:15.997022+0000","last_fullsized":"2024-04-26T21:15:15.997022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594171+0000","last_change":"2024-04-26T21:15:13.999674+0000","last_active":"2024-04-26T21:15:16.594171+0000","last_peered":"2024-04-26T21:15:16.594171+0000","last_clean":"2024-04-26T21:15:16.594171+0000","last_became_active":"2024-04-26T21:15:13.999384+0000","last_became_peered":"2024-04-26T21:15:13.999384+0000","last_unstale":"2024-04-26T21:15:16.594171+0000","last_undegraded":"2024-04-26T21:15:16.594171+0000","last_fullsized":"2024-04-26T21:15:16.594171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:41.421624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594232+0000","last_change":"2024-04-26T21:15:15.004056+0000","last_active":"2024-04-26T21:15:16.594232+0000","last_peered":"2024-04-26T21:15:16.594232+0000","last_clean":"2024-04-26T21:15:16.594232+0000","last_became_active":"2024-04-26T21:15:15.003109+0000","last_became_peered":"2024-04-26T21:15:15.003109+0000","last_unstale":"2024-04-26T21:15:16.594232+0000","last_undegraded":"2024-04-26T21:15:16.594232+0000","last_fullsized":"2024-04-26T21:15:16.594232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.332709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998354+0000","last_change":"2024-04-26T21:15:13.988307+0000","last_active":"2024-04-26T21:15:15.998354+0000","last_peered":"2024-04-26T21:15:15.998354+0000","last_clean":"2024-04-26T21:15:15.998354+0000","last_became_active":"2024-04-26T21:15:13.988018+0000","last_became_peered":"2024-04-26T21:15:13.988018+0000","last_unstale":"2024-04-26T21:15:15.998354+0000","last_undegraded":"2024-04-26T21:15:15.998354+0000","last_fullsized":"2024-04-26T21:15:15.998354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.077866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997865+0000","last_change":"2024-04-26T21:15:15.004543+0000","last_active":"2024-04-26T21:15:15.997865+0000","last_peered":"2024-04-26T21:15:15.997865+0000","last_clean":"2024-04-26T21:15:15.997865+0000","last_became_active":"2024-04-26T21:15:15.004281+0000","last_became_peered":"2024-04-26T21:15:15.004281+0000","last_unstale":"2024-04-26T21:15:15.997865+0000","last_undegraded":"2024-04-26T21:15:15.997865+0000","last_fullsized":"2024-04-26T21:15:15.997865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595209+0000","last_change":"2024-04-26T21:15:13.991260+0000","last_active":"2024-04-26T21:15:16.595209+0000","last_peered":"2024-04-26T21:15:16.595209+0000","last_clean":"2024-04-26T21:15:16.595209+0000","last_became_active":"2024-04-26T21:15:13.990969+0000","last_became_peered":"2024-04-26T21:15:13.990969+0000","last_unstale":"2024-04-26T21:15:16.595209+0000","last_undegraded":"2024-04-26T21:15:16.595209+0000","last_fullsized":"2024-04-26T21:15:16.595209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:35.822660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993865+0000","last_change":"2024-04-26T21:15:14.997788+0000","last_active":"2024-04-26T21:15:15.993865+0000","last_peered":"2024-04-26T21:15:15.993865+0000","last_clean":"2024-04-26T21:15:15.993865+0000","last_became_active":"2024-04-26T21:15:14.997464+0000","last_became_peered":"2024-04-26T21:15:14.997464+0000","last_unstale":"2024-04-26T21:15:15.993865+0000","last_undegraded":"2024-04-26T21:15:15.993865+0000","last_fullsized":"2024-04-26T21:15:15.993865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:56.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994753+0000","last_change":"2024-04-26T21:15:13.991679+0000","last_active":"2024-04-26T21:15:13.994753+0000","last_peered":"2024-04-26T21:15:13.994753+0000","last_clean":"2024-04-26T21:15:13.994753+0000","last_became_active":"2024-04-26T21:15:13.991338+0000","last_became_peered":"2024-04-26T21:15:13.991338+0000","last_unstale":"2024-04-26T21:15:13.994753+0000","last_undegraded":"2024-04-26T21:15:13.994753+0000","last_fullsized":"2024-04-26T21:15:13.994753+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:32.732498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:14.999960+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:14.999673+0000","last_became_peered":"2024-04-26T21:15:14.999673+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:36.824537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:13.994639+0000","last_change":"2024-04-26T21:15:13.990261+0000","last_active":"2024-04-26T21:15:13.994639+0000","last_peered":"2024-04-26T21:15:13.994639+0000","last_clean":"2024-04-26T21:15:13.994639+0000","last_became_active":"2024-04-26T21:15:13.989888+0000","last_became_peered":"2024-04-26T21:15:13.989888+0000","last_unstale":"2024-04-26T21:15:13.994639+0000","last_undegraded":"2024-04-26T21:15:13.994639+0000","last_fullsized":"2024-04-26T21:15:13.994639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:15:22.729662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996901+0000","last_change":"2024-04-26T21:15:15.000394+0000","last_active":"2024-04-26T21:15:15.996901+0000","last_peered":"2024-04-26T21:15:15.996901+0000","last_clean":"2024-04-26T21:15:15.996901+0000","last_became_active":"2024-04-26T21:15:14.999894+0000","last_became_peered":"2024-04-26T21:15:14.999894+0000","last_unstale":"2024-04-26T21:15:15.996901+0000","last_undegraded":"2024-04-26T21:15:15.996901+0000","last_fullsized":"2024-04-26T21:15:15.996901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997598+0000","last_change":"2024-04-26T21:15:13.989609+0000","last_active":"2024-04-26T21:15:15.997598+0000","last_peered":"2024-04-26T21:15:15.997598+0000","last_clean":"2024-04-26T21:15:15.997598+0000","last_became_active":"2024-04-26T21:15:13.989419+0000","last_became_peered":"2024-04-26T21:15:13.989419+0000","last_unstale":"2024-04-26T21:15:15.997598+0000","last_undegraded":"2024-04-26T21:15:15.997598+0000","last_fullsized":"2024-04-26T21:15:15.997598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:36.252437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593466+0000","last_change":"2024-04-26T21:15:15.005165+0000","last_active":"2024-04-26T21:15:16.593466+0000","last_peered":"2024-04-26T21:15:16.593466+0000","last_clean":"2024-04-26T21:15:16.593466+0000","last_became_active":"2024-04-26T21:15:15.004821+0000","last_became_peered":"2024-04-26T21:15:15.004821+0000","last_unstale":"2024-04-26T21:15:16.593466+0000","last_undegraded":"2024-04-26T21:15:16.593466+0000","last_fullsized":"2024-04-26T21:15:16.593466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996529+0000","last_change":"2024-04-26T21:15:13.984946+0000","last_active":"2024-04-26T21:15:15.996529+0000","last_peered":"2024-04-26T21:15:15.996529+0000","last_clean":"2024-04-26T21:15:15.996529+0000","last_became_active":"2024-04-26T21:15:13.984641+0000","last_became_peered":"2024-04-26T21:15:13.984641+0000","last_unstale":"2024-04-26T21:15:15.996529+0000","last_undegraded":"2024-04-26T21:15:15.996529+0000","last_fullsized":"2024-04-26T21:15:15.996529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993537+0000","last_change":"2024-04-26T21:15:15.007343+0000","last_active":"2024-04-26T21:15:15.993537+0000","last_peered":"2024-04-26T21:15:15.993537+0000","last_clean":"2024-04-26T21:15:15.993537+0000","last_became_active":"2024-04-26T21:15:15.007015+0000","last_became_peered":"2024-04-26T21:15:15.007015+0000","last_unstale":"2024-04-26T21:15:15.993537+0000","last_undegraded":"2024-04-26T21:15:15.993537+0000","last_fullsized":"2024-04-26T21:15:15.993537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:34.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593601+0000","last_change":"2024-04-26T21:15:13.996573+0000","last_active":"2024-04-26T21:15:16.593601+0000","last_peered":"2024-04-26T21:15:16.593601+0000","last_clean":"2024-04-26T21:15:16.593601+0000","last_became_active":"2024-04-26T21:15:13.996295+0000","last_became_peered":"2024-04-26T21:15:13.996295+0000","last_unstale":"2024-04-26T21:15:16.593601+0000","last_undegraded":"2024-04-26T21:15:16.593601+0000","last_fullsized":"2024-04-26T21:15:16.593601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:42.351528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594435+0000","last_change":"2024-04-26T21:15:15.001042+0000","last_active":"2024-04-26T21:15:16.594435+0000","last_peered":"2024-04-26T21:15:16.594435+0000","last_clean":"2024-04-26T21:15:16.594435+0000","last_became_active":"2024-04-26T21:15:15.000753+0000","last_became_peered":"2024-04-26T21:15:15.000753+0000","last_unstale":"2024-04-26T21:15:16.594435+0000","last_undegraded":"2024-04-26T21:15:16.594435+0000","last_fullsized":"2024-04-26T21:15:16.594435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.434213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998075+0000","last_change":"2024-04-26T21:15:13.993704+0000","last_active":"2024-04-26T21:15:15.998075+0000","last_peered":"2024-04-26T21:15:15.998075+0000","last_clean":"2024-04-26T21:15:15.998075+0000","last_became_active":"2024-04-26T21:15:13.993410+0000","last_became_peered":"2024-04-26T21:15:13.993410+0000","last_unstale":"2024-04-26T21:15:15.998075+0000","last_undegraded":"2024-04-26T21:15:15.998075+0000","last_fullsized":"2024-04-26T21:15:15.998075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594517+0000","last_change":"2024-04-26T21:15:14.998082+0000","last_active":"2024-04-26T21:15:16.594517+0000","last_peered":"2024-04-26T21:15:16.594517+0000","last_clean":"2024-04-26T21:15:16.594517+0000","last_became_active":"2024-04-26T21:15:14.997733+0000","last_became_peered":"2024-04-26T21:15:14.997733+0000","last_unstale":"2024-04-26T21:15:16.594517+0000","last_undegraded":"2024-04-26T21:15:16.594517+0000","last_fullsized":"2024-04-26T21:15:16.594517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:07.215468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594639+0000","last_change":"2024-04-26T21:15:14.001697+0000","last_active":"2024-04-26T21:15:16.594639+0000","last_peered":"2024-04-26T21:15:16.594639+0000","last_clean":"2024-04-26T21:15:16.594639+0000","last_became_active":"2024-04-26T21:15:14.001416+0000","last_became_peered":"2024-04-26T21:15:14.001416+0000","last_unstale":"2024-04-26T21:15:16.594639+0000","last_undegraded":"2024-04-26T21:15:16.594639+0000","last_fullsized":"2024-04-26T21:15:16.594639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:11.110564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985044+0000","last_change":"2024-04-26T21:15:13.985044+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985044+0000","last_undegraded":"2024-04-26T21:15:13.985044+0000","last_fullsized":"2024-04-26T21:15:13.985044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997916+0000","last_change":"2024-04-26T21:15:13.997192+0000","last_active":"2024-04-26T21:15:15.997916+0000","last_peered":"2024-04-26T21:15:15.997916+0000","last_clean":"2024-04-26T21:15:15.997916+0000","last_became_active":"2024-04-26T21:15:13.996946+0000","last_became_peered":"2024-04-26T21:15:13.996946+0000","last_unstale":"2024-04-26T21:15:15.997916+0000","last_undegraded":"2024-04-26T21:15:15.997916+0000","last_fullsized":"2024-04-26T21:15:15.997916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:33.005058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993826+0000","last_change":"2024-04-26T21:15:15.000551+0000","last_active":"2024-04-26T21:15:15.993826+0000","last_peered":"2024-04-26T21:15:15.993826+0000","last_clean":"2024-04-26T21:15:15.993826+0000","last_became_active":"2024-04-26T21:15:15.000269+0000","last_became_peered":"2024-04-26T21:15:15.000269+0000","last_unstale":"2024-04-26T21:15:15.993826+0000","last_undegraded":"2024-04-26T21:15:15.993826+0000","last_fullsized":"2024-04-26T21:15:15.993826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:46.479309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594507+0000","last_change":"2024-04-26T21:15:14.000103+0000","last_active":"2024-04-26T21:15:16.594507+0000","last_peered":"2024-04-26T21:15:16.594507+0000","last_clean":"2024-04-26T21:15:16.594507+0000","last_became_active":"2024-04-26T21:15:13.999824+0000","last_became_peered":"2024-04-26T21:15:13.999824+0000","last_unstale":"2024-04-26T21:15:16.594507+0000","last_undegraded":"2024-04-26T21:15:16.594507+0000","last_fullsized":"2024-04-26T21:15:16.594507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595067+0000","last_change":"2024-04-26T21:15:14.999573+0000","last_active":"2024-04-26T21:15:16.595067+0000","last_peered":"2024-04-26T21:15:16.595067+0000","last_clean":"2024-04-26T21:15:16.595067+0000","last_became_active":"2024-04-26T21:15:14.999265+0000","last_became_peered":"2024-04-26T21:15:14.999265+0000","last_unstale":"2024-04-26T21:15:16.595067+0000","last_undegraded":"2024-04-26T21:15:16.595067+0000","last_fullsized":"2024-04-26T21:15:16.595067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:49.825951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997655+0000","last_change":"2024-04-26T21:15:13.987004+0000","last_active":"2024-04-26T21:15:15.997655+0000","last_peered":"2024-04-26T21:15:15.997655+0000","last_clean":"2024-04-26T21:15:15.997655+0000","last_became_active":"2024-04-26T21:15:13.986676+0000","last_became_peered":"2024-04-26T21:15:13.986676+0000","last_unstale":"2024-04-26T21:15:15.997655+0000","last_undegraded":"2024-04-26T21:15:15.997655+0000","last_fullsized":"2024-04-26T21:15:15.997655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:44.090460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594645+0000","last_change":"2024-04-26T21:15:15.004474+0000","last_active":"2024-04-26T21:15:16.594645+0000","last_peered":"2024-04-26T21:15:16.594645+0000","last_clean":"2024-04-26T21:15:16.594645+0000","last_became_active":"2024-04-26T21:15:15.004185+0000","last_became_peered":"2024-04-26T21:15:15.004185+0000","last_unstale":"2024-04-26T21:15:16.594645+0000","last_undegraded":"2024-04-26T21:15:16.594645+0000","last_fullsized":"2024-04-26T21:15:16.594645+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.298412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996654+0000","last_change":"2024-04-26T21:15:13.988682+0000","last_active":"2024-04-26T21:15:15.996654+0000","last_peered":"2024-04-26T21:15:15.996654+0000","last_clean":"2024-04-26T21:15:15.996654+0000","last_became_active":"2024-04-26T21:15:13.988377+0000","last_became_peered":"2024-04-26T21:15:13.988377+0000","last_unstale":"2024-04-26T21:15:15.996654+0000","last_undegraded":"2024-04-26T21:15:15.996654+0000","last_fullsized":"2024-04-26T21:15:15.996654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:15.394639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594757+0000","last_change":"2024-04-26T21:15:15.005522+0000","last_active":"2024-04-26T21:15:16.594757+0000","last_peered":"2024-04-26T21:15:16.594757+0000","last_clean":"2024-04-26T21:15:16.594757+0000","last_became_active":"2024-04-26T21:15:15.005254+0000","last_became_peered":"2024-04-26T21:15:15.005254+0000","last_unstale":"2024-04-26T21:15:16.594757+0000","last_undegraded":"2024-04-26T21:15:16.594757+0000","last_fullsized":"2024-04-26T21:15:16.594757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.398672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593386+0000","last_change":"2024-04-26T21:15:13.990434+0000","last_active":"2024-04-26T21:15:16.593386+0000","last_peered":"2024-04-26T21:15:16.593386+0000","last_clean":"2024-04-26T21:15:16.593386+0000","last_became_active":"2024-04-26T21:15:13.990190+0000","last_became_peered":"2024-04-26T21:15:13.990190+0000","last_unstale":"2024-04-26T21:15:16.593386+0000","last_undegraded":"2024-04-26T21:15:16.593386+0000","last_fullsized":"2024-04-26T21:15:16.593386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:39.803635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593847+0000","last_change":"2024-04-26T21:15:14.997159+0000","last_active":"2024-04-26T21:15:16.593847+0000","last_peered":"2024-04-26T21:15:16.593847+0000","last_clean":"2024-04-26T21:15:16.593847+0000","last_became_active":"2024-04-26T21:15:14.996826+0000","last_became_peered":"2024-04-26T21:15:14.996826+0000","last_unstale":"2024-04-26T21:15:16.593847+0000","last_undegraded":"2024-04-26T21:15:16.593847+0000","last_fullsized":"2024-04-26T21:15:16.593847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998071+0000","last_change":"2024-04-26T21:15:13.995264+0000","last_active":"2024-04-26T21:15:15.998071+0000","last_peered":"2024-04-26T21:15:15.998071+0000","last_clean":"2024-04-26T21:15:15.998071+0000","last_became_active":"2024-04-26T21:15:13.994989+0000","last_became_peered":"2024-04-26T21:15:13.994989+0000","last_unstale":"2024-04-26T21:15:15.998071+0000","last_undegraded":"2024-04-26T21:15:15.998071+0000","last_fullsized":"2024-04-26T21:15:15.998071+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:35.044683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996782+0000","last_change":"2024-04-26T21:15:15.000876+0000","last_active":"2024-04-26T21:15:15.996782+0000","last_peered":"2024-04-26T21:15:15.996782+0000","last_clean":"2024-04-26T21:15:15.996782+0000","last_became_active":"2024-04-26T21:15:15.000234+0000","last_became_peered":"2024-04-26T21:15:15.000234+0000","last_unstale":"2024-04-26T21:15:15.996782+0000","last_undegraded":"2024-04-26T21:15:15.996782+0000","last_fullsized":"2024-04-26T21:15:15.996782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.985094+0000","last_change":"2024-04-26T21:15:13.988753+0000","last_active":"2024-04-26T21:15:16.985094+0000","last_peered":"2024-04-26T21:15:16.985094+0000","last_clean":"2024-04-26T21:15:16.985094+0000","last_became_active":"2024-04-26T21:15:13.988472+0000","last_became_peered":"2024-04-26T21:15:13.988472+0000","last_unstale":"2024-04-26T21:15:16.985094+0000","last_undegraded":"2024-04-26T21:15:16.985094+0000","last_fullsized":"2024-04-26T21:15:16.985094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:17.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594646+0000","last_change":"2024-04-26T21:15:15.007784+0000","last_active":"2024-04-26T21:15:16.594646+0000","last_peered":"2024-04-26T21:15:16.594646+0000","last_clean":"2024-04-26T21:15:16.594646+0000","last_became_active":"2024-04-26T21:15:15.007373+0000","last_became_peered":"2024-04-26T21:15:15.007373+0000","last_unstale":"2024-04-26T21:15:16.594646+0000","last_undegraded":"2024-04-26T21:15:16.594646+0000","last_fullsized":"2024-04-26T21:15:16.594646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:27.459629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993931+0000","last_change":"2024-04-26T21:15:13.985808+0000","last_active":"2024-04-26T21:15:15.993931+0000","last_peered":"2024-04-26T21:15:15.993931+0000","last_clean":"2024-04-26T21:15:15.993931+0000","last_became_active":"2024-04-26T21:15:13.985482+0000","last_became_peered":"2024-04-26T21:15:13.985482+0000","last_unstale":"2024-04-26T21:15:15.993931+0000","last_undegraded":"2024-04-26T21:15:15.993931+0000","last_fullsized":"2024-04-26T21:15:15.993931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:56.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594703+0000","last_change":"2024-04-26T21:15:14.997195+0000","last_active":"2024-04-26T21:15:16.594703+0000","last_peered":"2024-04-26T21:15:16.594703+0000","last_clean":"2024-04-26T21:15:16.594703+0000","last_became_active":"2024-04-26T21:15:14.996889+0000","last_became_peered":"2024-04-26T21:15:14.996889+0000","last_unstale":"2024-04-26T21:15:16.594703+0000","last_undegraded":"2024-04-26T21:15:16.594703+0000","last_fullsized":"2024-04-26T21:15:16.594703+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:03.261175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593581+0000","last_change":"2024-04-26T21:15:14.000033+0000","last_active":"2024-04-26T21:15:16.593581+0000","last_peered":"2024-04-26T21:15:16.593581+0000","last_clean":"2024-04-26T21:15:16.593581+0000","last_became_active":"2024-04-26T21:15:13.999642+0000","last_became_peered":"2024-04-26T21:15:13.999642+0000","last_unstale":"2024-04-26T21:15:16.593581+0000","last_undegraded":"2024-04-26T21:15:16.593581+0000","last_fullsized":"2024-04-26T21:15:16.593581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993953+0000","last_change":"2024-04-26T21:15:14.998038+0000","last_active":"2024-04-26T21:15:15.993953+0000","last_peered":"2024-04-26T21:15:15.993953+0000","last_clean":"2024-04-26T21:15:15.993953+0000","last_became_active":"2024-04-26T21:15:14.997735+0000","last_became_peered":"2024-04-26T21:15:14.997735+0000","last_unstale":"2024-04-26T21:15:15.993953+0000","last_undegraded":"2024-04-26T21:15:15.993953+0000","last_fullsized":"2024-04-26T21:15:15.993953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993592+0000","last_change":"2024-04-26T21:15:13.985339+0000","last_active":"2024-04-26T21:15:15.993592+0000","last_peered":"2024-04-26T21:15:15.993592+0000","last_clean":"2024-04-26T21:15:15.993592+0000","last_became_active":"2024-04-26T21:15:13.984804+0000","last_became_peered":"2024-04-26T21:15:13.984804+0000","last_unstale":"2024-04-26T21:15:15.993592+0000","last_undegraded":"2024-04-26T21:15:15.993592+0000","last_fullsized":"2024-04-26T21:15:15.993592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:08.792790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997529+0000","last_change":"2024-04-26T21:15:15.004362+0000","last_active":"2024-04-26T21:15:15.997529+0000","last_peered":"2024-04-26T21:15:15.997529+0000","last_clean":"2024-04-26T21:15:15.997529+0000","last_became_active":"2024-04-26T21:15:15.003059+0000","last_became_peered":"2024-04-26T21:15:15.003059+0000","last_unstale":"2024-04-26T21:15:15.997529+0000","last_undegraded":"2024-04-26T21:15:15.997529+0000","last_fullsized":"2024-04-26T21:15:15.997529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:04.403950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T21:15:14.003690+0000","last_change":"2024-04-26T21:15:14.000183+0000","last_active":"2024-04-26T21:15:14.003690+0000","last_peered":"2024-04-26T21:15:14.003690+0000","last_clean":"2024-04-26T21:15:14.003690+0000","last_became_active":"2024-04-26T21:15:13.999874+0000","last_became_peered":"2024-04-26T21:15:13.999874+0000","last_unstale":"2024-04-26T21:15:14.003690+0000","last_undegraded":"2024-04-26T21:15:14.003690+0000","last_fullsized":"2024-04-26T21:15:14.003690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:18.312063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":68,"state":"creating+peering","last_fresh":"2024-04-26T21:15:13.985817+0000","last_change":"2024-04-26T21:15:13.985817+0000","last_active":"2024-04-26T21:15:13.969237+0000","last_peered":"2024-04-26T21:15:13.969237+0000","last_clean":"2024-04-26T21:15:13.969237+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:15:13.985817+0000","last_undegraded":"2024-04-26T21:15:13.985817+0000","last_fullsized":"2024-04-26T21:15:13.985817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593678+0000","last_change":"2024-04-26T21:15:14.000123+0000","last_active":"2024-04-26T21:15:16.593678+0000","last_peered":"2024-04-26T21:15:16.593678+0000","last_clean":"2024-04-26T21:15:16.593678+0000","last_became_active":"2024-04-26T21:15:13.999730+0000","last_became_peered":"2024-04-26T21:15:13.999730+0000","last_unstale":"2024-04-26T21:15:16.593678+0000","last_undegraded":"2024-04-26T21:15:16.593678+0000","last_fullsized":"2024-04-26T21:15:16.593678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:51.654799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997415+0000","last_change":"2024-04-26T21:15:15.004122+0000","last_active":"2024-04-26T21:15:15.997415+0000","last_peered":"2024-04-26T21:15:15.997415+0000","last_clean":"2024-04-26T21:15:15.997415+0000","last_became_active":"2024-04-26T21:15:15.003876+0000","last_became_peered":"2024-04-26T21:15:15.003876+0000","last_unstale":"2024-04-26T21:15:15.997415+0000","last_undegraded":"2024-04-26T21:15:15.997415+0000","last_fullsized":"2024-04-26T21:15:15.997415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:55.716624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998034+0000","last_change":"2024-04-26T21:15:13.992777+0000","last_active":"2024-04-26T21:15:15.998034+0000","last_peered":"2024-04-26T21:15:15.998034+0000","last_clean":"2024-04-26T21:15:15.998034+0000","last_became_active":"2024-04-26T21:15:13.992544+0000","last_became_peered":"2024-04-26T21:15:13.992544+0000","last_unstale":"2024-04-26T21:15:15.998034+0000","last_undegraded":"2024-04-26T21:15:15.998034+0000","last_fullsized":"2024-04-26T21:15:15.998034+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:06.643084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594821+0000","last_change":"2024-04-26T21:15:15.005232+0000","last_active":"2024-04-26T21:15:16.594821+0000","last_peered":"2024-04-26T21:15:16.594821+0000","last_clean":"2024-04-26T21:15:16.594821+0000","last_became_active":"2024-04-26T21:15:15.004904+0000","last_became_peered":"2024-04-26T21:15:15.004904+0000","last_unstale":"2024-04-26T21:15:16.594821+0000","last_undegraded":"2024-04-26T21:15:16.594821+0000","last_fullsized":"2024-04-26T21:15:16.594821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:48.293671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997514+0000","last_change":"2024-04-26T21:15:13.991822+0000","last_active":"2024-04-26T21:15:15.997514+0000","last_peered":"2024-04-26T21:15:15.997514+0000","last_clean":"2024-04-26T21:15:15.997514+0000","last_became_active":"2024-04-26T21:15:13.991508+0000","last_became_peered":"2024-04-26T21:15:13.991508+0000","last_unstale":"2024-04-26T21:15:15.997514+0000","last_undegraded":"2024-04-26T21:15:15.997514+0000","last_fullsized":"2024-04-26T21:15:15.997514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:29.754213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":255,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593633+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:16.593633+0000","last_peered":"2024-04-26T21:15:16.593633+0000","last_clean":"2024-04-26T21:15:16.593633+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:16.593633+0000","last_undegraded":"2024-04-26T21:15:16.593633+0000","last_fullsized":"2024-04-26T21:15:16.593633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":942962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":352070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":352090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":353590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":942346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":338903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:15:23.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:15:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 Processor -- start 2024-04-26T21:15:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 -- start start 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc73815f370 0x7fc7380cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 0x7fc7380b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7380d03a0 con 0x7fc73815f750 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc7380b4d10 con 0x7fc73815f370 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.571+0000 7fc73c9af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc7380d0680 con 0x7fc7380d3ee0 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc73659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc73815f370 0x7fc7380cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc72ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc73815f750 0x7fc7380d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48418/0 (socket says 172.21.15.40:48418) 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc72ffff640 1 -- 172.21.15.40:0/3594910321 learned_addr learned my addr 172.21.15.40:0/3594910321 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc73659c640 1 -- 172.21.15.40:0/3594910321 >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 msgr2=0x7fc7380b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc736d9d640 1 --2- 172.21.15.40:0/3594910321 >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 0x7fc7380b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc73659c640 1 --2- 172.21.15.40:0/3594910321 >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 0x7fc7380b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc73659c640 1 -- 172.21.15.40:0/3594910321 >> v1:172.21.15.40:6789/0 conn(0x7fc73815f750 legacy=0x7fc7380d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc735d9b640 1 -- 172.21.15.40:0/3594910321 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986468031 0 0) 0x7fc7380d03a0 con 0x7fc73815f750 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc73659c640 1 -- 172.21.15.40:0/3594910321 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7380d0960 con 0x7fc73815f370 2024-04-26T21:15:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.572+0000 7fc736d9d640 1 --2- 172.21.15.40:0/3594910321 >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 0x7fc7380b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc73659c640 1 --2- 172.21.15.40:0/3594910321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc73815f370 0x7fc7380cfd70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc7380c8cf0 tx=0x7fc72802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e5ecc064a0c39d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc735d9b640 1 -- 172.21.15.40:0/3594910321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728037070 con 0x7fc73815f370 2024-04-26T21:15:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc735d9b640 1 -- 172.21.15.40:0/3594910321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc728032440 con 0x7fc73815f370 2024-04-26T21:15:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc735d9b640 1 -- 172.21.15.40:0/3594910321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728032810 con 0x7fc73815f370 2024-04-26T21:15:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc73c9af640 1 -- 172.21.15.40:0/3594910321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc73815f370 msgr2=0x7fc7380cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.573+0000 7fc73c9af640 1 --2- 172.21.15.40:0/3594910321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc73815f370 0x7fc7380cfd70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc7380c8cf0 tx=0x7fc72802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 -- 172.21.15.40:0/3594910321 shutdown_connections 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 --2- 172.21.15.40:0/3594910321 >> v2:172.21.15.184:3300/0 conn(0x7fc7380d3ee0 0x7fc7380b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 --2- 172.21.15.40:0/3594910321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc73815f370 0x7fc7380cfd70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 -- 172.21.15.40:0/3594910321 >> 172.21.15.40:0/3594910321 conn(0x7fc73808ec10 msgr2=0x7fc73809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 -- 172.21.15.40:0/3594910321 shutdown_connections 2024-04-26T21:15:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.574+0000 7fc73c9af640 1 -- 172.21.15.40:0/3594910321 wait complete. 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.575+0000 7fc73c9af640 1 Processor -- start 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.575+0000 7fc73c9af640 1 -- start start 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 0x7fc7380b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc73815f750 0x7fc7380ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380b51c0 con 0x7fc7380d3ee0 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380d0f20 con 0x7fc73815f370 2024-04-26T21:15:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73c9af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc7380b5340 con 0x7fc73815f750 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46760/0 (socket says 172.21.15.40:46760) 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 -- 172.21.15.40:0/728070056 learned_addr learned my addr 172.21.15.40:0/728070056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc736d9d640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc73815f750 0x7fc7380ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 -- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc73815f750 msgr2=0x7fc7380ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc73815f750 0x7fc7380ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc72ffff640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 0x7fc7380b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 -- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 msgr2=0x7fc7380b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 0x7fc7380b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 -- 172.21.15.40:0/728070056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7380b9b60 con 0x7fc7380d3ee0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc72ffff640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 0x7fc7380b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.576+0000 7fc73659c640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc728000c00 tx=0x7fc7280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.577+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728037050 con 0x7fc7380d3ee0 2024-04-26T21:15:23.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.577+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc728038040 con 0x7fc7380d3ee0 2024-04-26T21:15:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.577+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7380b9e40 con 0x7fc7380d3ee0 2024-04-26T21:15:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.577+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728004450 con 0x7fc7380d3ee0 2024-04-26T21:15:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.577+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7380ba400 con 0x7fc7380d3ee0 2024-04-26T21:15:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.578+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73806a290 con 0x7fc7380d3ee0 2024-04-26T21:15:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.578+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc728030980 con 0x7fc7380d3ee0 2024-04-26T21:15:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.578+0000 7fc72effd640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.579+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc7280ac6a0 con 0x7fc7380d3ee0 2024-04-26T21:15:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.579+0000 7fc72ffff640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.579+0000 7fc72ffff640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc738162580 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.583+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc72807ab60 con 0x7fc7380d3ee0 2024-04-26T21:15:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.765+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7380d1610 con 0x7fc71806c8f0 2024-04-26T21:15:23.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.772+0000 7fc72effd640 1 -- 172.21.15.40:0/728070056 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251981 (secure 0 0 0) 0x7fc7380d1610 con 0x7fc71806c8f0 2024-04-26T21:15:23.775 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:23.778 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 msgr2=0x7fc71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc738162580 tx=0x7fc720009040 comp rx=0 tx=0).stop 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 msgr2=0x7fc73815cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc728000c00 tx=0x7fc7280046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 shutdown_connections 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc73815f750 0x7fc7380ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc73815f370 0x7fc7380b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 --2- 172.21.15.40:0/728070056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7380d3ee0 0x7fc73815cd50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 >> 172.21.15.40:0/728070056 conn(0x7fc73808ec10 msgr2=0x7fc7380d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 shutdown_connections 2024-04-26T21:15:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.775+0000 7fc73c9af640 1 -- 172.21.15.40:0/728070056 wait complete. 2024-04-26T21:15:23.787 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-26T21:15:22.964868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240660,"kb_used_data":7816,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734060,"statfs":{"total":773094113280,"available":772847677440,"internally_reserved":0,"allocated":8003584,"data_stored":4001380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001949"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594048+0000","last_change":"2024-04-26T21:15:15.008421+0000","last_active":"2024-04-26T21:15:16.594048+0000","last_peered":"2024-04-26T21:15:16.594048+0000","last_clean":"2024-04-26T21:15:16.594048+0000","last_became_active":"2024-04-26T21:15:15.008122+0000","last_became_peered":"2024-04-26T21:15:15.008122+0000","last_unstale":"2024-04-26T21:15:16.594048+0000","last_undegraded":"2024-04-26T21:15:16.594048+0000","last_fullsized":"2024-04-26T21:15:16.594048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:35.373822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998310+0000","last_change":"2024-04-26T21:15:15.004592+0000","last_active":"2024-04-26T21:15:15.998310+0000","last_peered":"2024-04-26T21:15:15.998310+0000","last_clean":"2024-04-26T21:15:15.998310+0000","last_became_active":"2024-04-26T21:15:15.004314+0000","last_became_peered":"2024-04-26T21:15:15.004314+0000","last_unstale":"2024-04-26T21:15:15.998310+0000","last_undegraded":"2024-04-26T21:15:15.998310+0000","last_fullsized":"2024-04-26T21:15:15.998310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595104+0000","last_change":"2024-04-26T21:15:14.984552+0000","last_active":"2024-04-26T21:15:16.595104+0000","last_peered":"2024-04-26T21:15:16.595104+0000","last_clean":"2024-04-26T21:15:16.595104+0000","last_became_active":"2024-04-26T21:15:14.984224+0000","last_became_peered":"2024-04-26T21:15:14.984224+0000","last_unstale":"2024-04-26T21:15:16.595104+0000","last_undegraded":"2024-04-26T21:15:16.595104+0000","last_fullsized":"2024-04-26T21:15:16.595104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:52.413000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594172+0000","last_change":"2024-04-26T21:15:15.008395+0000","last_active":"2024-04-26T21:15:16.594172+0000","last_peered":"2024-04-26T21:15:16.594172+0000","last_clean":"2024-04-26T21:15:16.594172+0000","last_became_active":"2024-04-26T21:15:15.008108+0000","last_became_peered":"2024-04-26T21:15:15.008108+0000","last_unstale":"2024-04-26T21:15:16.594172+0000","last_undegraded":"2024-04-26T21:15:16.594172+0000","last_fullsized":"2024-04-26T21:15:16.594172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:08.349839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993499+0000","last_change":"2024-04-26T21:15:15.008338+0000","last_active":"2024-04-26T21:15:15.993499+0000","last_peered":"2024-04-26T21:15:15.993499+0000","last_clean":"2024-04-26T21:15:15.993499+0000","last_became_active":"2024-04-26T21:15:15.008029+0000","last_became_peered":"2024-04-26T21:15:15.008029+0000","last_unstale":"2024-04-26T21:15:15.993499+0000","last_undegraded":"2024-04-26T21:15:15.993499+0000","last_fullsized":"2024-04-26T21:15:15.993499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:02.371753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997414+0000","last_change":"2024-04-26T21:15:15.003006+0000","last_active":"2024-04-26T21:15:15.997414+0000","last_peered":"2024-04-26T21:15:15.997414+0000","last_clean":"2024-04-26T21:15:15.997414+0000","last_became_active":"2024-04-26T21:15:15.002731+0000","last_became_peered":"2024-04-26T21:15:15.002731+0000","last_unstale":"2024-04-26T21:15:15.997414+0000","last_undegraded":"2024-04-26T21:15:15.997414+0000","last_fullsized":"2024-04-26T21:15:15.997414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:45.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996492+0000","last_change":"2024-04-26T21:15:15.000633+0000","last_active":"2024-04-26T21:15:15.996492+0000","last_peered":"2024-04-26T21:15:15.996492+0000","last_clean":"2024-04-26T21:15:15.996492+0000","last_became_active":"2024-04-26T21:15:15.000275+0000","last_became_peered":"2024-04-26T21:15:15.000275+0000","last_unstale":"2024-04-26T21:15:15.996492+0000","last_undegraded":"2024-04-26T21:15:15.996492+0000","last_fullsized":"2024-04-26T21:15:15.996492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:11.788021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997948+0000","last_change":"2024-04-26T21:15:15.002868+0000","last_active":"2024-04-26T21:15:15.997948+0000","last_peered":"2024-04-26T21:15:15.997948+0000","last_clean":"2024-04-26T21:15:15.997948+0000","last_became_active":"2024-04-26T21:15:15.002561+0000","last_became_peered":"2024-04-26T21:15:15.002561+0000","last_unstale":"2024-04-26T21:15:15.997948+0000","last_undegraded":"2024-04-26T21:15:15.997948+0000","last_fullsized":"2024-04-26T21:15:15.997948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994128+0000","last_change":"2024-04-26T21:15:15.004629+0000","last_active":"2024-04-26T21:15:15.994128+0000","last_peered":"2024-04-26T21:15:15.994128+0000","last_clean":"2024-04-26T21:15:15.994128+0000","last_became_active":"2024-04-26T21:15:15.004365+0000","last_became_peered":"2024-04-26T21:15:15.004365+0000","last_unstale":"2024-04-26T21:15:15.994128+0000","last_undegraded":"2024-04-26T21:15:15.994128+0000","last_fullsized":"2024-04-26T21:15:15.994128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:33.056050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594298+0000","last_change":"2024-04-26T21:15:15.005575+0000","last_active":"2024-04-26T21:15:16.594298+0000","last_peered":"2024-04-26T21:15:16.594298+0000","last_clean":"2024-04-26T21:15:16.594298+0000","last_became_active":"2024-04-26T21:15:15.005289+0000","last_became_peered":"2024-04-26T21:15:15.005289+0000","last_unstale":"2024-04-26T21:15:16.594298+0000","last_undegraded":"2024-04-26T21:15:16.594298+0000","last_fullsized":"2024-04-26T21:15:16.594298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:18.088546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594550+0000","last_change":"2024-04-26T21:15:15.000155+0000","last_active":"2024-04-26T21:15:16.594550+0000","last_peered":"2024-04-26T21:15:16.594550+0000","last_clean":"2024-04-26T21:15:16.594550+0000","last_became_active":"2024-04-26T21:15:14.999858+0000","last_became_peered":"2024-04-26T21:15:14.999858+0000","last_unstale":"2024-04-26T21:15:16.594550+0000","last_undegraded":"2024-04-26T21:15:16.594550+0000","last_fullsized":"2024-04-26T21:15:16.594550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:01.080096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594165+0000","last_change":"2024-04-26T21:15:15.001502+0000","last_active":"2024-04-26T21:15:16.594165+0000","last_peered":"2024-04-26T21:15:16.594165+0000","last_clean":"2024-04-26T21:15:16.594165+0000","last_became_active":"2024-04-26T21:15:15.000795+0000","last_became_peered":"2024-04-26T21:15:15.000795+0000","last_unstale":"2024-04-26T21:15:16.594165+0000","last_undegraded":"2024-04-26T21:15:16.594165+0000","last_fullsized":"2024-04-26T21:15:16.594165+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:13.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593769+0000","last_change":"2024-04-26T21:15:15.001688+0000","last_active":"2024-04-26T21:15:16.593769+0000","last_peered":"2024-04-26T21:15:16.593769+0000","last_clean":"2024-04-26T21:15:16.593769+0000","last_became_active":"2024-04-26T21:15:15.001372+0000","last_became_peered":"2024-04-26T21:15:15.001372+0000","last_unstale":"2024-04-26T21:15:16.593769+0000","last_undegraded":"2024-04-26T21:15:16.593769+0000","last_fullsized":"2024-04-26T21:15:16.593769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993853+0000","last_change":"2024-04-26T21:15:14.995986+0000","last_active":"2024-04-26T21:15:15.993853+0000","last_peered":"2024-04-26T21:15:15.993853+0000","last_clean":"2024-04-26T21:15:15.993853+0000","last_became_active":"2024-04-26T21:15:14.995673+0000","last_became_peered":"2024-04-26T21:15:14.995673+0000","last_unstale":"2024-04-26T21:15:15.993853+0000","last_undegraded":"2024-04-26T21:15:15.993853+0000","last_fullsized":"2024-04-26T21:15:15.993853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593933+0000","last_change":"2024-04-26T21:15:14.996331+0000","last_active":"2024-04-26T21:15:16.593933+0000","last_peered":"2024-04-26T21:15:16.593933+0000","last_clean":"2024-04-26T21:15:16.593933+0000","last_became_active":"2024-04-26T21:15:14.996044+0000","last_became_peered":"2024-04-26T21:15:14.996044+0000","last_unstale":"2024-04-26T21:15:16.593933+0000","last_undegraded":"2024-04-26T21:15:16.593933+0000","last_fullsized":"2024-04-26T21:15:16.593933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593583+0000","last_change":"2024-04-26T21:15:15.007557+0000","last_active":"2024-04-26T21:15:16.593583+0000","last_peered":"2024-04-26T21:15:16.593583+0000","last_clean":"2024-04-26T21:15:16.593583+0000","last_became_active":"2024-04-26T21:15:15.007269+0000","last_became_peered":"2024-04-26T21:15:15.007269+0000","last_unstale":"2024-04-26T21:15:16.593583+0000","last_undegraded":"2024-04-26T21:15:16.593583+0000","last_fullsized":"2024-04-26T21:15:16.593583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994100+0000","last_change":"2024-04-26T21:15:15.003937+0000","last_active":"2024-04-26T21:15:15.994100+0000","last_peered":"2024-04-26T21:15:15.994100+0000","last_clean":"2024-04-26T21:15:15.994100+0000","last_became_active":"2024-04-26T21:15:15.003652+0000","last_became_peered":"2024-04-26T21:15:15.003652+0000","last_unstale":"2024-04-26T21:15:15.994100+0000","last_undegraded":"2024-04-26T21:15:15.994100+0000","last_fullsized":"2024-04-26T21:15:15.994100+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.457492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:15.007673+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:15.007408+0000","last_became_peered":"2024-04-26T21:15:15.007408+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:24.287344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997884+0000","last_change":"2024-04-26T21:15:15.003066+0000","last_active":"2024-04-26T21:15:15.997884+0000","last_peered":"2024-04-26T21:15:15.997884+0000","last_clean":"2024-04-26T21:15:15.997884+0000","last_became_active":"2024-04-26T21:15:15.002597+0000","last_became_peered":"2024-04-26T21:15:15.002597+0000","last_unstale":"2024-04-26T21:15:15.997884+0000","last_undegraded":"2024-04-26T21:15:15.997884+0000","last_fullsized":"2024-04-26T21:15:15.997884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593869+0000","last_change":"2024-04-26T21:15:14.996454+0000","last_active":"2024-04-26T21:15:16.593869+0000","last_peered":"2024-04-26T21:15:16.593869+0000","last_clean":"2024-04-26T21:15:16.593869+0000","last_became_active":"2024-04-26T21:15:14.996140+0000","last_became_peered":"2024-04-26T21:15:14.996140+0000","last_unstale":"2024-04-26T21:15:16.593869+0000","last_undegraded":"2024-04-26T21:15:16.593869+0000","last_fullsized":"2024-04-26T21:15:16.593869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593784+0000","last_change":"2024-04-26T21:15:15.004956+0000","last_active":"2024-04-26T21:15:16.593784+0000","last_peered":"2024-04-26T21:15:16.593784+0000","last_clean":"2024-04-26T21:15:16.593784+0000","last_became_active":"2024-04-26T21:15:15.004666+0000","last_became_peered":"2024-04-26T21:15:15.004666+0000","last_unstale":"2024-04-26T21:15:16.593784+0000","last_undegraded":"2024-04-26T21:15:16.593784+0000","last_fullsized":"2024-04-26T21:15:16.593784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:59.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593651+0000","last_change":"2024-04-26T21:15:14.998310+0000","last_active":"2024-04-26T21:15:16.593651+0000","last_peered":"2024-04-26T21:15:16.593651+0000","last_clean":"2024-04-26T21:15:16.593651+0000","last_became_active":"2024-04-26T21:15:14.998039+0000","last_became_peered":"2024-04-26T21:15:14.998039+0000","last_unstale":"2024-04-26T21:15:16.593651+0000","last_undegraded":"2024-04-26T21:15:16.593651+0000","last_fullsized":"2024-04-26T21:15:16.593651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:33.908281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:14.997132+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:14.996824+0000","last_became_peered":"2024-04-26T21:15:14.996824+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593965+0000","last_change":"2024-04-26T21:15:15.003818+0000","last_active":"2024-04-26T21:15:16.593965+0000","last_peered":"2024-04-26T21:15:16.593965+0000","last_clean":"2024-04-26T21:15:16.593965+0000","last_became_active":"2024-04-26T21:15:15.003428+0000","last_became_peered":"2024-04-26T21:15:15.003428+0000","last_unstale":"2024-04-26T21:15:16.593965+0000","last_undegraded":"2024-04-26T21:15:16.593965+0000","last_fullsized":"2024-04-26T21:15:16.593965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594069+0000","last_change":"2024-04-26T21:15:15.004314+0000","last_active":"2024-04-26T21:15:16.594069+0000","last_peered":"2024-04-26T21:15:16.594069+0000","last_clean":"2024-04-26T21:15:16.594069+0000","last_became_active":"2024-04-26T21:15:15.004022+0000","last_became_peered":"2024-04-26T21:15:15.004022+0000","last_unstale":"2024-04-26T21:15:16.594069+0000","last_undegraded":"2024-04-26T21:15:16.594069+0000","last_fullsized":"2024-04-26T21:15:16.594069+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:24.018679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996953+0000","last_change":"2024-04-26T21:15:15.006413+0000","last_active":"2024-04-26T21:15:15.996953+0000","last_peered":"2024-04-26T21:15:15.996953+0000","last_clean":"2024-04-26T21:15:15.996953+0000","last_became_active":"2024-04-26T21:15:15.006136+0000","last_became_peered":"2024-04-26T21:15:15.006136+0000","last_unstale":"2024-04-26T21:15:15.996953+0000","last_undegraded":"2024-04-26T21:15:15.996953+0000","last_fullsized":"2024-04-26T21:15:15.996953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:39.315392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595237+0000","last_change":"2024-04-26T21:15:14.999635+0000","last_active":"2024-04-26T21:15:16.595237+0000","last_peered":"2024-04-26T21:15:16.595237+0000","last_clean":"2024-04-26T21:15:16.595237+0000","last_became_active":"2024-04-26T21:15:14.999351+0000","last_became_peered":"2024-04-26T21:15:14.999351+0000","last_unstale":"2024-04-26T21:15:16.595237+0000","last_undegraded":"2024-04-26T21:15:16.595237+0000","last_fullsized":"2024-04-26T21:15:16.595237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:23.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594169+0000","last_change":"2024-04-26T21:15:15.006522+0000","last_active":"2024-04-26T21:15:16.594169+0000","last_peered":"2024-04-26T21:15:16.594169+0000","last_clean":"2024-04-26T21:15:16.594169+0000","last_became_active":"2024-04-26T21:15:15.005849+0000","last_became_peered":"2024-04-26T21:15:15.005849+0000","last_unstale":"2024-04-26T21:15:16.594169+0000","last_undegraded":"2024-04-26T21:15:16.594169+0000","last_fullsized":"2024-04-26T21:15:16.594169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:16.396516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594834+0000","last_change":"2024-04-26T21:15:14.988314+0000","last_active":"2024-04-26T21:15:16.594834+0000","last_peered":"2024-04-26T21:15:16.594834+0000","last_clean":"2024-04-26T21:15:16.594834+0000","last_became_active":"2024-04-26T21:15:14.987983+0000","last_became_peered":"2024-04-26T21:15:14.987983+0000","last_unstale":"2024-04-26T21:15:16.594834+0000","last_undegraded":"2024-04-26T21:15:16.594834+0000","last_fullsized":"2024-04-26T21:15:16.594834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:56.347854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996624+0000","last_change":"2024-04-26T21:15:15.001705+0000","last_active":"2024-04-26T21:15:15.996624+0000","last_peered":"2024-04-26T21:15:15.996624+0000","last_clean":"2024-04-26T21:15:15.996624+0000","last_became_active":"2024-04-26T21:15:15.001398+0000","last_became_peered":"2024-04-26T21:15:15.001398+0000","last_unstale":"2024-04-26T21:15:15.996624+0000","last_undegraded":"2024-04-26T21:15:15.996624+0000","last_fullsized":"2024-04-26T21:15:15.996624+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996883+0000","last_change":"2024-04-26T21:15:14.996277+0000","last_active":"2024-04-26T21:15:15.996883+0000","last_peered":"2024-04-26T21:15:15.996883+0000","last_clean":"2024-04-26T21:15:15.996883+0000","last_became_active":"2024-04-26T21:15:14.996006+0000","last_became_peered":"2024-04-26T21:15:14.996006+0000","last_unstale":"2024-04-26T21:15:15.996883+0000","last_undegraded":"2024-04-26T21:15:15.996883+0000","last_fullsized":"2024-04-26T21:15:15.996883+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997797+0000","last_change":"2024-04-26T21:15:15.005406+0000","last_active":"2024-04-26T21:15:15.997797+0000","last_peered":"2024-04-26T21:15:15.997797+0000","last_clean":"2024-04-26T21:15:15.997797+0000","last_became_active":"2024-04-26T21:15:15.005116+0000","last_became_peered":"2024-04-26T21:15:15.005116+0000","last_unstale":"2024-04-26T21:15:15.997797+0000","last_undegraded":"2024-04-26T21:15:15.997797+0000","last_fullsized":"2024-04-26T21:15:15.997797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:16.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594603+0000","last_change":"2024-04-26T21:15:15.008320+0000","last_active":"2024-04-26T21:15:16.594603+0000","last_peered":"2024-04-26T21:15:16.594603+0000","last_clean":"2024-04-26T21:15:16.594603+0000","last_became_active":"2024-04-26T21:15:15.008048+0000","last_became_peered":"2024-04-26T21:15:15.008048+0000","last_unstale":"2024-04-26T21:15:16.594603+0000","last_undegraded":"2024-04-26T21:15:16.594603+0000","last_fullsized":"2024-04-26T21:15:16.594603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.353647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594502+0000","last_change":"2024-04-26T21:15:13.991660+0000","last_active":"2024-04-26T21:15:16.594502+0000","last_peered":"2024-04-26T21:15:16.594502+0000","last_clean":"2024-04-26T21:15:16.594502+0000","last_became_active":"2024-04-26T21:15:13.991361+0000","last_became_peered":"2024-04-26T21:15:13.991361+0000","last_unstale":"2024-04-26T21:15:16.594502+0000","last_undegraded":"2024-04-26T21:15:16.594502+0000","last_fullsized":"2024-04-26T21:15:16.594502+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.994190+0000","last_change":"2024-04-26T21:15:15.003269+0000","last_active":"2024-04-26T21:15:15.994190+0000","last_peered":"2024-04-26T21:15:15.994190+0000","last_clean":"2024-04-26T21:15:15.994190+0000","last_became_active":"2024-04-26T21:15:15.002973+0000","last_became_peered":"2024-04-26T21:15:15.002973+0000","last_unstale":"2024-04-26T21:15:15.994190+0000","last_undegraded":"2024-04-26T21:15:15.994190+0000","last_fullsized":"2024-04-26T21:15:15.994190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:30.756090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593977+0000","last_change":"2024-04-26T21:15:13.990425+0000","last_active":"2024-04-26T21:15:16.593977+0000","last_peered":"2024-04-26T21:15:16.593977+0000","last_clean":"2024-04-26T21:15:16.593977+0000","last_became_active":"2024-04-26T21:15:13.990113+0000","last_became_peered":"2024-04-26T21:15:13.990113+0000","last_unstale":"2024-04-26T21:15:16.593977+0000","last_undegraded":"2024-04-26T21:15:16.593977+0000","last_fullsized":"2024-04-26T21:15:16.593977+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:39.163991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997715+0000","last_change":"2024-04-26T21:15:15.004144+0000","last_active":"2024-04-26T21:15:15.997715+0000","last_peered":"2024-04-26T21:15:15.997715+0000","last_clean":"2024-04-26T21:15:15.997715+0000","last_became_active":"2024-04-26T21:15:15.003804+0000","last_became_peered":"2024-04-26T21:15:15.003804+0000","last_unstale":"2024-04-26T21:15:15.997715+0000","last_undegraded":"2024-04-26T21:15:15.997715+0000","last_fullsized":"2024-04-26T21:15:15.997715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:34.213418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594471+0000","last_change":"2024-04-26T21:15:13.996784+0000","last_active":"2024-04-26T21:15:16.594471+0000","last_peered":"2024-04-26T21:15:16.594471+0000","last_clean":"2024-04-26T21:15:16.594471+0000","last_became_active":"2024-04-26T21:15:13.996508+0000","last_became_peered":"2024-04-26T21:15:13.996508+0000","last_unstale":"2024-04-26T21:15:16.594471+0000","last_undegraded":"2024-04-26T21:15:16.594471+0000","last_fullsized":"2024-04-26T21:15:16.594471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:31.090506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594503+0000","last_change":"2024-04-26T21:15:14.998880+0000","last_active":"2024-04-26T21:15:16.594503+0000","last_peered":"2024-04-26T21:15:16.594503+0000","last_clean":"2024-04-26T21:15:16.594503+0000","last_became_active":"2024-04-26T21:15:14.998406+0000","last_became_peered":"2024-04-26T21:15:14.998406+0000","last_unstale":"2024-04-26T21:15:16.594503+0000","last_undegraded":"2024-04-26T21:15:16.594503+0000","last_fullsized":"2024-04-26T21:15:16.594503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:32.918934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997990+0000","last_change":"2024-04-26T21:15:13.990545+0000","last_active":"2024-04-26T21:15:15.997990+0000","last_peered":"2024-04-26T21:15:15.997990+0000","last_clean":"2024-04-26T21:15:15.997990+0000","last_became_active":"2024-04-26T21:15:13.990151+0000","last_became_peered":"2024-04-26T21:15:13.990151+0000","last_unstale":"2024-04-26T21:15:15.997990+0000","last_undegraded":"2024-04-26T21:15:15.997990+0000","last_fullsized":"2024-04-26T21:15:15.997990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:43.327070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996712+0000","last_change":"2024-04-26T21:15:14.999455+0000","last_active":"2024-04-26T21:15:15.996712+0000","last_peered":"2024-04-26T21:15:15.996712+0000","last_clean":"2024-04-26T21:15:15.996712+0000","last_became_active":"2024-04-26T21:15:14.999063+0000","last_became_peered":"2024-04-26T21:15:14.999063+0000","last_unstale":"2024-04-26T21:15:15.996712+0000","last_undegraded":"2024-04-26T21:15:15.996712+0000","last_fullsized":"2024-04-26T21:15:15.996712+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595004+0000","last_change":"2024-04-26T21:15:14.000741+0000","last_active":"2024-04-26T21:15:16.595004+0000","last_peered":"2024-04-26T21:15:16.595004+0000","last_clean":"2024-04-26T21:15:16.595004+0000","last_became_active":"2024-04-26T21:15:14.000484+0000","last_became_peered":"2024-04-26T21:15:14.000484+0000","last_unstale":"2024-04-26T21:15:16.595004+0000","last_undegraded":"2024-04-26T21:15:16.595004+0000","last_fullsized":"2024-04-26T21:15:16.595004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:34.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594360+0000","last_change":"2024-04-26T21:15:15.003590+0000","last_active":"2024-04-26T21:15:16.594360+0000","last_peered":"2024-04-26T21:15:16.594360+0000","last_clean":"2024-04-26T21:15:16.594360+0000","last_became_active":"2024-04-26T21:15:15.003252+0000","last_became_peered":"2024-04-26T21:15:15.003252+0000","last_unstale":"2024-04-26T21:15:16.594360+0000","last_undegraded":"2024-04-26T21:15:16.594360+0000","last_fullsized":"2024-04-26T21:15:16.594360+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:40.805512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997494+0000","last_change":"2024-04-26T21:15:13.987191+0000","last_active":"2024-04-26T21:15:15.997494+0000","last_peered":"2024-04-26T21:15:15.997494+0000","last_clean":"2024-04-26T21:15:15.997494+0000","last_became_active":"2024-04-26T21:15:13.986872+0000","last_became_peered":"2024-04-26T21:15:13.986872+0000","last_unstale":"2024-04-26T21:15:15.997494+0000","last_undegraded":"2024-04-26T21:15:15.997494+0000","last_fullsized":"2024-04-26T21:15:15.997494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:02.357589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993432+0000","last_change":"2024-04-26T21:15:15.002302+0000","last_active":"2024-04-26T21:15:15.993432+0000","last_peered":"2024-04-26T21:15:15.993432+0000","last_clean":"2024-04-26T21:15:15.993432+0000","last_became_active":"2024-04-26T21:15:15.001311+0000","last_became_peered":"2024-04-26T21:15:15.001311+0000","last_unstale":"2024-04-26T21:15:15.993432+0000","last_undegraded":"2024-04-26T21:15:15.993432+0000","last_fullsized":"2024-04-26T21:15:15.993432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:08.309553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593817+0000","last_change":"2024-04-26T21:15:13.998731+0000","last_active":"2024-04-26T21:15:16.593817+0000","last_peered":"2024-04-26T21:15:16.593817+0000","last_clean":"2024-04-26T21:15:16.593817+0000","last_became_active":"2024-04-26T21:15:13.998420+0000","last_became_peered":"2024-04-26T21:15:13.998420+0000","last_unstale":"2024-04-26T21:15:16.593817+0000","last_undegraded":"2024-04-26T21:15:16.593817+0000","last_fullsized":"2024-04-26T21:15:16.593817+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:51.325314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593528+0000","last_change":"2024-04-26T21:15:14.999599+0000","last_active":"2024-04-26T21:15:16.593528+0000","last_peered":"2024-04-26T21:15:16.593528+0000","last_clean":"2024-04-26T21:15:16.593528+0000","last_became_active":"2024-04-26T21:15:14.998302+0000","last_became_peered":"2024-04-26T21:15:14.998302+0000","last_unstale":"2024-04-26T21:15:16.593528+0000","last_undegraded":"2024-04-26T21:15:16.593528+0000","last_fullsized":"2024-04-26T21:15:16.593528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:11.746138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993426+0000","last_change":"2024-04-26T21:15:13.989877+0000","last_active":"2024-04-26T21:15:15.993426+0000","last_peered":"2024-04-26T21:15:15.993426+0000","last_clean":"2024-04-26T21:15:15.993426+0000","last_became_active":"2024-04-26T21:15:13.989441+0000","last_became_peered":"2024-04-26T21:15:13.989441+0000","last_unstale":"2024-04-26T21:15:15.993426+0000","last_undegraded":"2024-04-26T21:15:15.993426+0000","last_fullsized":"2024-04-26T21:15:15.993426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:41.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997022+0000","last_change":"2024-04-26T21:15:14.999473+0000","last_active":"2024-04-26T21:15:15.997022+0000","last_peered":"2024-04-26T21:15:15.997022+0000","last_clean":"2024-04-26T21:15:15.997022+0000","last_became_active":"2024-04-26T21:15:14.999179+0000","last_became_peered":"2024-04-26T21:15:14.999179+0000","last_unstale":"2024-04-26T21:15:15.997022+0000","last_undegraded":"2024-04-26T21:15:15.997022+0000","last_fullsized":"2024-04-26T21:15:15.997022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:13.687381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594171+0000","last_change":"2024-04-26T21:15:13.999674+0000","last_active":"2024-04-26T21:15:16.594171+0000","last_peered":"2024-04-26T21:15:16.594171+0000","last_clean":"2024-04-26T21:15:16.594171+0000","last_became_active":"2024-04-26T21:15:13.999384+0000","last_became_peered":"2024-04-26T21:15:13.999384+0000","last_unstale":"2024-04-26T21:15:16.594171+0000","last_undegraded":"2024-04-26T21:15:16.594171+0000","last_fullsized":"2024-04-26T21:15:16.594171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:41.421624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594232+0000","last_change":"2024-04-26T21:15:15.004056+0000","last_active":"2024-04-26T21:15:16.594232+0000","last_peered":"2024-04-26T21:15:16.594232+0000","last_clean":"2024-04-26T21:15:16.594232+0000","last_became_active":"2024-04-26T21:15:15.003109+0000","last_became_peered":"2024-04-26T21:15:15.003109+0000","last_unstale":"2024-04-26T21:15:16.594232+0000","last_undegraded":"2024-04-26T21:15:16.594232+0000","last_fullsized":"2024-04-26T21:15:16.594232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.332709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998354+0000","last_change":"2024-04-26T21:15:13.988307+0000","last_active":"2024-04-26T21:15:15.998354+0000","last_peered":"2024-04-26T21:15:15.998354+0000","last_clean":"2024-04-26T21:15:15.998354+0000","last_became_active":"2024-04-26T21:15:13.988018+0000","last_became_peered":"2024-04-26T21:15:13.988018+0000","last_unstale":"2024-04-26T21:15:15.998354+0000","last_undegraded":"2024-04-26T21:15:15.998354+0000","last_fullsized":"2024-04-26T21:15:15.998354+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:32.077866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997865+0000","last_change":"2024-04-26T21:15:15.004543+0000","last_active":"2024-04-26T21:15:15.997865+0000","last_peered":"2024-04-26T21:15:15.997865+0000","last_clean":"2024-04-26T21:15:15.997865+0000","last_became_active":"2024-04-26T21:15:15.004281+0000","last_became_peered":"2024-04-26T21:15:15.004281+0000","last_unstale":"2024-04-26T21:15:15.997865+0000","last_undegraded":"2024-04-26T21:15:15.997865+0000","last_fullsized":"2024-04-26T21:15:15.997865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:36.914022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595209+0000","last_change":"2024-04-26T21:15:13.991260+0000","last_active":"2024-04-26T21:15:16.595209+0000","last_peered":"2024-04-26T21:15:16.595209+0000","last_clean":"2024-04-26T21:15:16.595209+0000","last_became_active":"2024-04-26T21:15:13.990969+0000","last_became_peered":"2024-04-26T21:15:13.990969+0000","last_unstale":"2024-04-26T21:15:16.595209+0000","last_undegraded":"2024-04-26T21:15:16.595209+0000","last_fullsized":"2024-04-26T21:15:16.595209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:35.822660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993865+0000","last_change":"2024-04-26T21:15:14.997788+0000","last_active":"2024-04-26T21:15:15.993865+0000","last_peered":"2024-04-26T21:15:15.993865+0000","last_clean":"2024-04-26T21:15:15.993865+0000","last_became_active":"2024-04-26T21:15:14.997464+0000","last_became_peered":"2024-04-26T21:15:14.997464+0000","last_unstale":"2024-04-26T21:15:15.993865+0000","last_undegraded":"2024-04-26T21:15:15.993865+0000","last_fullsized":"2024-04-26T21:15:15.993865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:56.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594341+0000","last_change":"2024-04-26T21:15:13.991679+0000","last_active":"2024-04-26T21:15:16.594341+0000","last_peered":"2024-04-26T21:15:16.594341+0000","last_clean":"2024-04-26T21:15:16.594341+0000","last_became_active":"2024-04-26T21:15:13.991338+0000","last_became_peered":"2024-04-26T21:15:13.991338+0000","last_unstale":"2024-04-26T21:15:16.594341+0000","last_undegraded":"2024-04-26T21:15:16.594341+0000","last_fullsized":"2024-04-26T21:15:16.594341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:18.190785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593976+0000","last_change":"2024-04-26T21:15:14.999960+0000","last_active":"2024-04-26T21:15:16.593976+0000","last_peered":"2024-04-26T21:15:16.593976+0000","last_clean":"2024-04-26T21:15:16.593976+0000","last_became_active":"2024-04-26T21:15:14.999673+0000","last_became_peered":"2024-04-26T21:15:14.999673+0000","last_unstale":"2024-04-26T21:15:16.593976+0000","last_undegraded":"2024-04-26T21:15:16.593976+0000","last_fullsized":"2024-04-26T21:15:16.593976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:36.824537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593810+0000","last_change":"2024-04-26T21:15:13.990261+0000","last_active":"2024-04-26T21:15:16.593810+0000","last_peered":"2024-04-26T21:15:16.593810+0000","last_clean":"2024-04-26T21:15:16.593810+0000","last_became_active":"2024-04-26T21:15:13.989888+0000","last_became_peered":"2024-04-26T21:15:13.989888+0000","last_unstale":"2024-04-26T21:15:16.593810+0000","last_undegraded":"2024-04-26T21:15:16.593810+0000","last_fullsized":"2024-04-26T21:15:16.593810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:17.366680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996901+0000","last_change":"2024-04-26T21:15:15.000394+0000","last_active":"2024-04-26T21:15:15.996901+0000","last_peered":"2024-04-26T21:15:15.996901+0000","last_clean":"2024-04-26T21:15:15.996901+0000","last_became_active":"2024-04-26T21:15:14.999894+0000","last_became_peered":"2024-04-26T21:15:14.999894+0000","last_unstale":"2024-04-26T21:15:15.996901+0000","last_undegraded":"2024-04-26T21:15:15.996901+0000","last_fullsized":"2024-04-26T21:15:15.996901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:40.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997598+0000","last_change":"2024-04-26T21:15:13.989609+0000","last_active":"2024-04-26T21:15:15.997598+0000","last_peered":"2024-04-26T21:15:15.997598+0000","last_clean":"2024-04-26T21:15:15.997598+0000","last_became_active":"2024-04-26T21:15:13.989419+0000","last_became_peered":"2024-04-26T21:15:13.989419+0000","last_unstale":"2024-04-26T21:15:15.997598+0000","last_undegraded":"2024-04-26T21:15:15.997598+0000","last_fullsized":"2024-04-26T21:15:15.997598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:36.252437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593466+0000","last_change":"2024-04-26T21:15:15.005165+0000","last_active":"2024-04-26T21:15:16.593466+0000","last_peered":"2024-04-26T21:15:16.593466+0000","last_clean":"2024-04-26T21:15:16.593466+0000","last_became_active":"2024-04-26T21:15:15.004821+0000","last_became_peered":"2024-04-26T21:15:15.004821+0000","last_unstale":"2024-04-26T21:15:16.593466+0000","last_undegraded":"2024-04-26T21:15:16.593466+0000","last_fullsized":"2024-04-26T21:15:16.593466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:16.876777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996529+0000","last_change":"2024-04-26T21:15:13.984946+0000","last_active":"2024-04-26T21:15:15.996529+0000","last_peered":"2024-04-26T21:15:15.996529+0000","last_clean":"2024-04-26T21:15:15.996529+0000","last_became_active":"2024-04-26T21:15:13.984641+0000","last_became_peered":"2024-04-26T21:15:13.984641+0000","last_unstale":"2024-04-26T21:15:15.996529+0000","last_undegraded":"2024-04-26T21:15:15.996529+0000","last_fullsized":"2024-04-26T21:15:15.996529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993537+0000","last_change":"2024-04-26T21:15:15.007343+0000","last_active":"2024-04-26T21:15:15.993537+0000","last_peered":"2024-04-26T21:15:15.993537+0000","last_clean":"2024-04-26T21:15:15.993537+0000","last_became_active":"2024-04-26T21:15:15.007015+0000","last_became_peered":"2024-04-26T21:15:15.007015+0000","last_unstale":"2024-04-26T21:15:15.993537+0000","last_undegraded":"2024-04-26T21:15:15.993537+0000","last_fullsized":"2024-04-26T21:15:15.993537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:34.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593601+0000","last_change":"2024-04-26T21:15:13.996573+0000","last_active":"2024-04-26T21:15:16.593601+0000","last_peered":"2024-04-26T21:15:16.593601+0000","last_clean":"2024-04-26T21:15:16.593601+0000","last_became_active":"2024-04-26T21:15:13.996295+0000","last_became_peered":"2024-04-26T21:15:13.996295+0000","last_unstale":"2024-04-26T21:15:16.593601+0000","last_undegraded":"2024-04-26T21:15:16.593601+0000","last_fullsized":"2024-04-26T21:15:16.593601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:42.351528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594435+0000","last_change":"2024-04-26T21:15:15.001042+0000","last_active":"2024-04-26T21:15:16.594435+0000","last_peered":"2024-04-26T21:15:16.594435+0000","last_clean":"2024-04-26T21:15:16.594435+0000","last_became_active":"2024-04-26T21:15:15.000753+0000","last_became_peered":"2024-04-26T21:15:15.000753+0000","last_unstale":"2024-04-26T21:15:16.594435+0000","last_undegraded":"2024-04-26T21:15:16.594435+0000","last_fullsized":"2024-04-26T21:15:16.594435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.434213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998075+0000","last_change":"2024-04-26T21:15:13.993704+0000","last_active":"2024-04-26T21:15:15.998075+0000","last_peered":"2024-04-26T21:15:15.998075+0000","last_clean":"2024-04-26T21:15:15.998075+0000","last_became_active":"2024-04-26T21:15:13.993410+0000","last_became_peered":"2024-04-26T21:15:13.993410+0000","last_unstale":"2024-04-26T21:15:15.998075+0000","last_undegraded":"2024-04-26T21:15:15.998075+0000","last_fullsized":"2024-04-26T21:15:15.998075+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594517+0000","last_change":"2024-04-26T21:15:14.998082+0000","last_active":"2024-04-26T21:15:16.594517+0000","last_peered":"2024-04-26T21:15:16.594517+0000","last_clean":"2024-04-26T21:15:16.594517+0000","last_became_active":"2024-04-26T21:15:14.997733+0000","last_became_peered":"2024-04-26T21:15:14.997733+0000","last_unstale":"2024-04-26T21:15:16.594517+0000","last_undegraded":"2024-04-26T21:15:16.594517+0000","last_fullsized":"2024-04-26T21:15:16.594517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:07.215468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594639+0000","last_change":"2024-04-26T21:15:14.001697+0000","last_active":"2024-04-26T21:15:16.594639+0000","last_peered":"2024-04-26T21:15:16.594639+0000","last_clean":"2024-04-26T21:15:16.594639+0000","last_became_active":"2024-04-26T21:15:14.001416+0000","last_became_peered":"2024-04-26T21:15:14.001416+0000","last_unstale":"2024-04-26T21:15:16.594639+0000","last_undegraded":"2024-04-26T21:15:16.594639+0000","last_fullsized":"2024-04-26T21:15:16.594639+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:11.110564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594295+0000","last_change":"2024-04-26T21:15:15.001253+0000","last_active":"2024-04-26T21:15:16.594295+0000","last_peered":"2024-04-26T21:15:16.594295+0000","last_clean":"2024-04-26T21:15:16.594295+0000","last_became_active":"2024-04-26T21:15:15.000396+0000","last_became_peered":"2024-04-26T21:15:15.000396+0000","last_unstale":"2024-04-26T21:15:16.594295+0000","last_undegraded":"2024-04-26T21:15:16.594295+0000","last_fullsized":"2024-04-26T21:15:16.594295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:14.062366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997916+0000","last_change":"2024-04-26T21:15:13.997192+0000","last_active":"2024-04-26T21:15:15.997916+0000","last_peered":"2024-04-26T21:15:15.997916+0000","last_clean":"2024-04-26T21:15:15.997916+0000","last_became_active":"2024-04-26T21:15:13.996946+0000","last_became_peered":"2024-04-26T21:15:13.996946+0000","last_unstale":"2024-04-26T21:15:15.997916+0000","last_undegraded":"2024-04-26T21:15:15.997916+0000","last_fullsized":"2024-04-26T21:15:15.997916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:33.005058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993826+0000","last_change":"2024-04-26T21:15:15.000551+0000","last_active":"2024-04-26T21:15:15.993826+0000","last_peered":"2024-04-26T21:15:15.993826+0000","last_clean":"2024-04-26T21:15:15.993826+0000","last_became_active":"2024-04-26T21:15:15.000269+0000","last_became_peered":"2024-04-26T21:15:15.000269+0000","last_unstale":"2024-04-26T21:15:15.993826+0000","last_undegraded":"2024-04-26T21:15:15.993826+0000","last_fullsized":"2024-04-26T21:15:15.993826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:46.479309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594507+0000","last_change":"2024-04-26T21:15:14.000103+0000","last_active":"2024-04-26T21:15:16.594507+0000","last_peered":"2024-04-26T21:15:16.594507+0000","last_clean":"2024-04-26T21:15:16.594507+0000","last_became_active":"2024-04-26T21:15:13.999824+0000","last_became_peered":"2024-04-26T21:15:13.999824+0000","last_unstale":"2024-04-26T21:15:16.594507+0000","last_undegraded":"2024-04-26T21:15:16.594507+0000","last_fullsized":"2024-04-26T21:15:16.594507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.595067+0000","last_change":"2024-04-26T21:15:14.999573+0000","last_active":"2024-04-26T21:15:16.595067+0000","last_peered":"2024-04-26T21:15:16.595067+0000","last_clean":"2024-04-26T21:15:16.595067+0000","last_became_active":"2024-04-26T21:15:14.999265+0000","last_became_peered":"2024-04-26T21:15:14.999265+0000","last_unstale":"2024-04-26T21:15:16.595067+0000","last_undegraded":"2024-04-26T21:15:16.595067+0000","last_fullsized":"2024-04-26T21:15:16.595067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:49.825951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997655+0000","last_change":"2024-04-26T21:15:13.987004+0000","last_active":"2024-04-26T21:15:15.997655+0000","last_peered":"2024-04-26T21:15:15.997655+0000","last_clean":"2024-04-26T21:15:15.997655+0000","last_became_active":"2024-04-26T21:15:13.986676+0000","last_became_peered":"2024-04-26T21:15:13.986676+0000","last_unstale":"2024-04-26T21:15:15.997655+0000","last_undegraded":"2024-04-26T21:15:15.997655+0000","last_fullsized":"2024-04-26T21:15:15.997655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:44.090460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594645+0000","last_change":"2024-04-26T21:15:15.004474+0000","last_active":"2024-04-26T21:15:16.594645+0000","last_peered":"2024-04-26T21:15:16.594645+0000","last_clean":"2024-04-26T21:15:16.594645+0000","last_became_active":"2024-04-26T21:15:15.004185+0000","last_became_peered":"2024-04-26T21:15:15.004185+0000","last_unstale":"2024-04-26T21:15:16.594645+0000","last_undegraded":"2024-04-26T21:15:16.594645+0000","last_fullsized":"2024-04-26T21:15:16.594645+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.298412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996654+0000","last_change":"2024-04-26T21:15:13.988682+0000","last_active":"2024-04-26T21:15:15.996654+0000","last_peered":"2024-04-26T21:15:15.996654+0000","last_clean":"2024-04-26T21:15:15.996654+0000","last_became_active":"2024-04-26T21:15:13.988377+0000","last_became_peered":"2024-04-26T21:15:13.988377+0000","last_unstale":"2024-04-26T21:15:15.996654+0000","last_undegraded":"2024-04-26T21:15:15.996654+0000","last_fullsized":"2024-04-26T21:15:15.996654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:15.394639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594757+0000","last_change":"2024-04-26T21:15:15.005522+0000","last_active":"2024-04-26T21:15:16.594757+0000","last_peered":"2024-04-26T21:15:16.594757+0000","last_clean":"2024-04-26T21:15:16.594757+0000","last_became_active":"2024-04-26T21:15:15.005254+0000","last_became_peered":"2024-04-26T21:15:15.005254+0000","last_unstale":"2024-04-26T21:15:16.594757+0000","last_undegraded":"2024-04-26T21:15:16.594757+0000","last_fullsized":"2024-04-26T21:15:16.594757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:47.398672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593386+0000","last_change":"2024-04-26T21:15:13.990434+0000","last_active":"2024-04-26T21:15:16.593386+0000","last_peered":"2024-04-26T21:15:16.593386+0000","last_clean":"2024-04-26T21:15:16.593386+0000","last_became_active":"2024-04-26T21:15:13.990190+0000","last_became_peered":"2024-04-26T21:15:13.990190+0000","last_unstale":"2024-04-26T21:15:16.593386+0000","last_undegraded":"2024-04-26T21:15:16.593386+0000","last_fullsized":"2024-04-26T21:15:16.593386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:39.803635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593847+0000","last_change":"2024-04-26T21:15:14.997159+0000","last_active":"2024-04-26T21:15:16.593847+0000","last_peered":"2024-04-26T21:15:16.593847+0000","last_clean":"2024-04-26T21:15:16.593847+0000","last_became_active":"2024-04-26T21:15:14.996826+0000","last_became_peered":"2024-04-26T21:15:14.996826+0000","last_unstale":"2024-04-26T21:15:16.593847+0000","last_undegraded":"2024-04-26T21:15:16.593847+0000","last_fullsized":"2024-04-26T21:15:16.593847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:34.999923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998071+0000","last_change":"2024-04-26T21:15:13.995264+0000","last_active":"2024-04-26T21:15:15.998071+0000","last_peered":"2024-04-26T21:15:15.998071+0000","last_clean":"2024-04-26T21:15:15.998071+0000","last_became_active":"2024-04-26T21:15:13.994989+0000","last_became_peered":"2024-04-26T21:15:13.994989+0000","last_unstale":"2024-04-26T21:15:15.998071+0000","last_undegraded":"2024-04-26T21:15:15.998071+0000","last_fullsized":"2024-04-26T21:15:15.998071+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:35.044683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.996782+0000","last_change":"2024-04-26T21:15:15.000876+0000","last_active":"2024-04-26T21:15:15.996782+0000","last_peered":"2024-04-26T21:15:15.996782+0000","last_clean":"2024-04-26T21:15:15.996782+0000","last_became_active":"2024-04-26T21:15:15.000234+0000","last_became_peered":"2024-04-26T21:15:15.000234+0000","last_unstale":"2024-04-26T21:15:15.996782+0000","last_undegraded":"2024-04-26T21:15:15.996782+0000","last_fullsized":"2024-04-26T21:15:15.996782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:41.172208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.985094+0000","last_change":"2024-04-26T21:15:13.988753+0000","last_active":"2024-04-26T21:15:16.985094+0000","last_peered":"2024-04-26T21:15:16.985094+0000","last_clean":"2024-04-26T21:15:16.985094+0000","last_became_active":"2024-04-26T21:15:13.988472+0000","last_became_peered":"2024-04-26T21:15:13.988472+0000","last_unstale":"2024-04-26T21:15:16.985094+0000","last_undegraded":"2024-04-26T21:15:16.985094+0000","last_fullsized":"2024-04-26T21:15:16.985094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:17.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594646+0000","last_change":"2024-04-26T21:15:15.007784+0000","last_active":"2024-04-26T21:15:16.594646+0000","last_peered":"2024-04-26T21:15:16.594646+0000","last_clean":"2024-04-26T21:15:16.594646+0000","last_became_active":"2024-04-26T21:15:15.007373+0000","last_became_peered":"2024-04-26T21:15:15.007373+0000","last_unstale":"2024-04-26T21:15:16.594646+0000","last_undegraded":"2024-04-26T21:15:16.594646+0000","last_fullsized":"2024-04-26T21:15:16.594646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:27.459629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993931+0000","last_change":"2024-04-26T21:15:13.985808+0000","last_active":"2024-04-26T21:15:15.993931+0000","last_peered":"2024-04-26T21:15:15.993931+0000","last_clean":"2024-04-26T21:15:15.993931+0000","last_became_active":"2024-04-26T21:15:13.985482+0000","last_became_peered":"2024-04-26T21:15:13.985482+0000","last_unstale":"2024-04-26T21:15:15.993931+0000","last_undegraded":"2024-04-26T21:15:15.993931+0000","last_fullsized":"2024-04-26T21:15:15.993931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:56.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594703+0000","last_change":"2024-04-26T21:15:14.997195+0000","last_active":"2024-04-26T21:15:16.594703+0000","last_peered":"2024-04-26T21:15:16.594703+0000","last_clean":"2024-04-26T21:15:16.594703+0000","last_became_active":"2024-04-26T21:15:14.996889+0000","last_became_peered":"2024-04-26T21:15:14.996889+0000","last_unstale":"2024-04-26T21:15:16.594703+0000","last_undegraded":"2024-04-26T21:15:16.594703+0000","last_fullsized":"2024-04-26T21:15:16.594703+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:03.261175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593581+0000","last_change":"2024-04-26T21:15:14.000033+0000","last_active":"2024-04-26T21:15:16.593581+0000","last_peered":"2024-04-26T21:15:16.593581+0000","last_clean":"2024-04-26T21:15:16.593581+0000","last_became_active":"2024-04-26T21:15:13.999642+0000","last_became_peered":"2024-04-26T21:15:13.999642+0000","last_unstale":"2024-04-26T21:15:16.593581+0000","last_undegraded":"2024-04-26T21:15:16.593581+0000","last_fullsized":"2024-04-26T21:15:16.593581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:35.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993953+0000","last_change":"2024-04-26T21:15:14.998038+0000","last_active":"2024-04-26T21:15:15.993953+0000","last_peered":"2024-04-26T21:15:15.993953+0000","last_clean":"2024-04-26T21:15:15.993953+0000","last_became_active":"2024-04-26T21:15:14.997735+0000","last_became_peered":"2024-04-26T21:15:14.997735+0000","last_unstale":"2024-04-26T21:15:15.993953+0000","last_undegraded":"2024-04-26T21:15:15.993953+0000","last_fullsized":"2024-04-26T21:15:15.993953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:01.073894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.993592+0000","last_change":"2024-04-26T21:15:13.985339+0000","last_active":"2024-04-26T21:15:15.993592+0000","last_peered":"2024-04-26T21:15:15.993592+0000","last_clean":"2024-04-26T21:15:15.993592+0000","last_became_active":"2024-04-26T21:15:13.984804+0000","last_became_peered":"2024-04-26T21:15:13.984804+0000","last_unstale":"2024-04-26T21:15:15.993592+0000","last_undegraded":"2024-04-26T21:15:15.993592+0000","last_fullsized":"2024-04-26T21:15:15.993592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:08.792790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997529+0000","last_change":"2024-04-26T21:15:15.004362+0000","last_active":"2024-04-26T21:15:15.997529+0000","last_peered":"2024-04-26T21:15:15.997529+0000","last_clean":"2024-04-26T21:15:15.997529+0000","last_became_active":"2024-04-26T21:15:15.003059+0000","last_became_peered":"2024-04-26T21:15:15.003059+0000","last_unstale":"2024-04-26T21:15:15.997529+0000","last_undegraded":"2024-04-26T21:15:15.997529+0000","last_fullsized":"2024-04-26T21:15:15.997529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:04.403950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593592+0000","last_change":"2024-04-26T21:15:14.000183+0000","last_active":"2024-04-26T21:15:16.593592+0000","last_peered":"2024-04-26T21:15:16.593592+0000","last_clean":"2024-04-26T21:15:16.593592+0000","last_became_active":"2024-04-26T21:15:13.999874+0000","last_became_peered":"2024-04-26T21:15:13.999874+0000","last_unstale":"2024-04-26T21:15:16.593592+0000","last_undegraded":"2024-04-26T21:15:16.593592+0000","last_fullsized":"2024-04-26T21:15:16.593592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:38.452300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593825+0000","last_change":"2024-04-26T21:15:15.000223+0000","last_active":"2024-04-26T21:15:16.593825+0000","last_peered":"2024-04-26T21:15:16.593825+0000","last_clean":"2024-04-26T21:15:16.593825+0000","last_became_active":"2024-04-26T21:15:14.999913+0000","last_became_peered":"2024-04-26T21:15:14.999913+0000","last_unstale":"2024-04-26T21:15:16.593825+0000","last_undegraded":"2024-04-26T21:15:16.593825+0000","last_fullsized":"2024-04-26T21:15:16.593825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:25.746525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593678+0000","last_change":"2024-04-26T21:15:14.000123+0000","last_active":"2024-04-26T21:15:16.593678+0000","last_peered":"2024-04-26T21:15:16.593678+0000","last_clean":"2024-04-26T21:15:16.593678+0000","last_became_active":"2024-04-26T21:15:13.999730+0000","last_became_peered":"2024-04-26T21:15:13.999730+0000","last_unstale":"2024-04-26T21:15:16.593678+0000","last_undegraded":"2024-04-26T21:15:16.593678+0000","last_fullsized":"2024-04-26T21:15:16.593678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:51.654799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997415+0000","last_change":"2024-04-26T21:15:15.004122+0000","last_active":"2024-04-26T21:15:15.997415+0000","last_peered":"2024-04-26T21:15:15.997415+0000","last_clean":"2024-04-26T21:15:15.997415+0000","last_became_active":"2024-04-26T21:15:15.003876+0000","last_became_peered":"2024-04-26T21:15:15.003876+0000","last_unstale":"2024-04-26T21:15:15.997415+0000","last_undegraded":"2024-04-26T21:15:15.997415+0000","last_fullsized":"2024-04-26T21:15:15.997415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:55.716624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.998034+0000","last_change":"2024-04-26T21:15:13.992777+0000","last_active":"2024-04-26T21:15:15.998034+0000","last_peered":"2024-04-26T21:15:15.998034+0000","last_clean":"2024-04-26T21:15:15.998034+0000","last_became_active":"2024-04-26T21:15:13.992544+0000","last_became_peered":"2024-04-26T21:15:13.992544+0000","last_unstale":"2024-04-26T21:15:15.998034+0000","last_undegraded":"2024-04-26T21:15:15.998034+0000","last_fullsized":"2024-04-26T21:15:15.998034+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:06.643084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.594821+0000","last_change":"2024-04-26T21:15:15.005232+0000","last_active":"2024-04-26T21:15:16.594821+0000","last_peered":"2024-04-26T21:15:16.594821+0000","last_clean":"2024-04-26T21:15:16.594821+0000","last_became_active":"2024-04-26T21:15:15.004904+0000","last_became_peered":"2024-04-26T21:15:15.004904+0000","last_unstale":"2024-04-26T21:15:16.594821+0000","last_undegraded":"2024-04-26T21:15:16.594821+0000","last_fullsized":"2024-04-26T21:15:16.594821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:13.969237+0000","last_clean_scrub_stamp":"2024-04-26T21:15:13.969237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:48.293671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:15.997514+0000","last_change":"2024-04-26T21:15:13.991822+0000","last_active":"2024-04-26T21:15:15.997514+0000","last_peered":"2024-04-26T21:15:15.997514+0000","last_clean":"2024-04-26T21:15:15.997514+0000","last_became_active":"2024-04-26T21:15:13.991508+0000","last_became_peered":"2024-04-26T21:15:13.991508+0000","last_unstale":"2024-04-26T21:15:15.997514+0000","last_undegraded":"2024-04-26T21:15:15.997514+0000","last_fullsized":"2024-04-26T21:15:15.997514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:12.967360+0000","last_clean_scrub_stamp":"2024-04-26T21:15:12.967360+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:29.754213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":255,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T21:15:16.593633+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:16.593633+0000","last_peered":"2024-04-26T21:15:16.593633+0000","last_clean":"2024-04-26T21:15:16.593633+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:16.593633+0000","last_undegraded":"2024-04-26T21:15:16.593633+0000","last_fullsized":"2024-04-26T21:15:16.593633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":942962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":352070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":352048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":352090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":353590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":942346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":354206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:15:23.791 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:15:23.792 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:15:23.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.935+0000 7f950962a640 1 Processor -- start 2024-04-26T21:15:23.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.935+0000 7f950962a640 1 -- start start 2024-04-26T21:15:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f950962a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f950415f630 0x7f95040ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f950962a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95040d04e0 0x7f95040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f950962a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f950415fa10 con 0x7f9504069ae0 2024-04-26T21:15:23.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f950962a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95040b5230 con 0x7f95040d04e0 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f950962a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f95040cafd0 con 0x7f950415f630 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95040d04e0 0x7f95040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95040d04e0 0x7f95040caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46762/0 (socket says 172.21.15.40:46762) 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95027fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9504069ae0 0x7f95040b56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48424/0 (socket says 172.21.15.40:48424) 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 -- 172.21.15.40:0/1657091649 learned_addr learned my addr 172.21.15.40:0/1657091649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f9502ffd640 1 --2- 172.21.15.40:0/1657091649 >> v2:172.21.15.184:3300/0 conn(0x7f950415f630 0x7f95040ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 -- 172.21.15.40:0/1657091649 >> v2:172.21.15.184:3300/0 conn(0x7f950415f630 msgr2=0x7f95040ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 --2- 172.21.15.40:0/1657091649 >> v2:172.21.15.184:3300/0 conn(0x7f950415f630 0x7f95040ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 -- 172.21.15.40:0/1657091649 >> v1:172.21.15.40:6789/0 conn(0x7f9504069ae0 legacy=0x7f95040b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f95037fe640 1 -- 172.21.15.40:0/1657091649 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95040cb220 con 0x7f95040d04e0 2024-04-26T21:15:23.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.936+0000 7f9501ffb640 1 -- 172.21.15.40:0/1657091649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372618743 0 0) 0x7f950415fa10 con 0x7f9504069ae0 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.937+0000 7f95037fe640 1 --2- 172.21.15.40:0/1657091649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95040d04e0 0x7f95040caa90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9504071bf0 tx=0x7f94fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bec83d939ea971 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.937+0000 7f9501ffb640 1 -- 172.21.15.40:0/1657091649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc036070 con 0x7f95040d04e0 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.937+0000 7f9501ffb640 1 -- 172.21.15.40:0/1657091649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc031600 con 0x7f95040d04e0 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.937+0000 7f9501ffb640 1 -- 172.21.15.40:0/1657091649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0319d0 con 0x7f95040d04e0 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 -- 172.21.15.40:0/1657091649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95040d04e0 msgr2=0x7f95040caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 --2- 172.21.15.40:0/1657091649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95040d04e0 0x7f95040caa90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9504071bf0 tx=0x7f94fc02a730 comp rx=0 tx=0).stop 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 -- 172.21.15.40:0/1657091649 shutdown_connections 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 --2- 172.21.15.40:0/1657091649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95040d04e0 0x7f95040caa90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 --2- 172.21.15.40:0/1657091649 >> v2:172.21.15.184:3300/0 conn(0x7f950415f630 0x7f95040ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 -- 172.21.15.40:0/1657091649 >> 172.21.15.40:0/1657091649 conn(0x7f95040b2ed0 msgr2=0x7f95040b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 -- 172.21.15.40:0/1657091649 shutdown_connections 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.938+0000 7f950962a640 1 -- 172.21.15.40:0/1657091649 wait complete. 2024-04-26T21:15:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.939+0000 7f950962a640 1 Processor -- start 2024-04-26T21:15:23.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.939+0000 7f950962a640 1 -- start start 2024-04-26T21:15:23.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9504069ae0 0x7f9504155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040d04e0 0x7f950415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95040b4ec0 con 0x7f950415f630 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95040b4be0 con 0x7f95040d04e0 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f950962a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f950406d000 con 0x7f9504069ae0 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46774/0 (socket says 172.21.15.40:46774) 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 -- 172.21.15.40:0/80928708 learned_addr learned my addr 172.21.15.40:0/80928708 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f9502ffd640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9504069ae0 0x7f9504155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95027fc640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040d04e0 0x7f950415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 -- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9504069ae0 msgr2=0x7f9504155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9504069ae0 0x7f9504155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 -- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040d04e0 msgr2=0x7f950415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040d04e0 0x7f950415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 -- 172.21.15.40:0/80928708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9504156010 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.940+0000 7f95037fe640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009f40 tx=0x7f94fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.941+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc036050 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.941+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc042070 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.941+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc009510 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.941+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95041562f0 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.941+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9504153f90 con 0x7f950415f630 2024-04-26T21:15:23.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.942+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950406a290 con 0x7f950415f630 2024-04-26T21:15:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.942+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94fc007bf0 con 0x7f950415f630 2024-04-26T21:15:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.942+0000 7f94eb7fe640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 0x7f94f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.943+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f94fc0ac710 con 0x7f950415f630 2024-04-26T21:15:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.943+0000 7f9502ffd640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 0x7f94f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.943+0000 7f9502ffd640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 0x7f94f006ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009b40 tx=0x7f94f4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:23.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:23.947+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94fc07a4f0 con 0x7f950415f630 2024-04-26T21:15:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.200+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f95041ea370 con 0x7f950415f630 2024-04-26T21:15:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.202+0000 7f94eb7fe640 1 -- 172.21.15.40:0/80928708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f94fc07e7f0 con 0x7f950415f630 2024-04-26T21:15:24.203 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:15:24.204 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:15:24.204 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:15:24.204 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:15:24.204 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:15:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 msgr2=0x7f94f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 0x7f94f006ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009b40 tx=0x7f94f4009190 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 msgr2=0x7f9504153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009f40 tx=0x7f94fc004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 shutdown_connections 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f94f006c920 0x7f94f006ede0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950415f630 0x7f9504153a50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95040d04e0 0x7f950415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 --2- 172.21.15.40:0/80928708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9504069ae0 0x7f9504155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 >> 172.21.15.40:0/80928708 conn(0x7f95040b2ed0 msgr2=0x7f950406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 shutdown_connections 2024-04-26T21:15:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.204+0000 7f950962a640 1 -- 172.21.15.40:0/80928708 wait complete. 2024-04-26T21:15:24.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.316+0000 7f90f57af640 1 Processor -- start 2024-04-26T21:15:24.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.316+0000 7f90f57af640 1 -- start start 2024-04-26T21:15:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90f57af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90f015f370 0x7f90f00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90f57af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00d03a0 con 0x7f90f00d3ee0 2024-04-26T21:15:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90f57af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00d0680 con 0x7f90f015f750 2024-04-26T21:15:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90f57af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f90f00d0960 con 0x7f90f015f370 2024-04-26T21:15:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90ef7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90f00d3ee0 0x7f90f00b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48440/0 (socket says 172.21.15.40:48440) 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90ef7fe640 1 -- 172.21.15.40:0/2311712909 learned_addr learned my addr 172.21.15.40:0/2311712909 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.317+0000 7f90eeffd640 1 --2- 172.21.15.40:0/2311712909 >> v2:172.21.15.184:3300/0 conn(0x7f90f015f370 0x7f90f00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907371864 0 0) 0x7f90f00d03a0 con 0x7f90f00d3ee0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90dc003520 con 0x7f90f00d3ee0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1883031828 0 0) 0x7f90dc003520 con 0x7f90f00d3ee0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 >> v2:172.21.15.184:3300/0 conn(0x7f90f015f370 msgr2=0x7f90f00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 --2- 172.21.15.40:0/2311712909 >> v2:172.21.15.184:3300/0 conn(0x7f90f015f370 0x7f90f00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 >> v1:172.21.15.184:6789/0 conn(0x7f90f015f750 legacy=0x7f90f00d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00d0c40 con 0x7f90f00d3ee0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f90e00019f0 con 0x7f90f00d3ee0 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90eeffd640 1 --2- 172.21.15.40:0/2311712909 >> v2:172.21.15.184:3300/0 conn(0x7f90f015f370 0x7f90f00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 >> v1:172.21.15.40:6789/0 conn(0x7f90f00d3ee0 legacy=0x7f90f00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90dc0039b0 0x7f90dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 0x7f90dc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc005080 0x7f90dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00d0960 con 0x7f90dc0043b0 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc00bfe0 con 0x7f90dc005080 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc00c1e0 con 0x7f90dc0039b0 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 0x7f90dc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90eeffd640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90dc0039b0 0x7f90dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ef7fe640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc005080 0x7f90dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 -- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90dc0039b0 msgr2=0x7f90dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90dc0039b0 0x7f90dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 -- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc005080 msgr2=0x7f90dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc005080 0x7f90dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ee7fc640 1 -- 172.21.15.40:0/2311712909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc003520 con 0x7f90dc0043b0 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90eeffd640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90dc0039b0 0x7f90dc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ef7fe640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc005080 0x7f90dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.318+0000 7f90ef7fe640 1 -- 172.21.15.40:0/2311712909 reap_dead start 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.319+0000 7f90ee7fc640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 0x7f90dc004880 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001860 tx=0x7f90dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19b0d63eeaf7266 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.319+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8037070 con 0x7f90dc0043b0 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.319+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d802abe0 con 0x7f90dc0043b0 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.319+0000 7f90edffb640 1 -- 172.21.15.40:0/2311712909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8001430 con 0x7f90dc0043b0 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 -- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 msgr2=0x7f90dc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 0x7f90dc004880 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001860 tx=0x7f90dc005830 comp rx=0 tx=0).stop 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 -- 172.21.15.40:0/2311712909 shutdown_connections 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 --2- 172.21.15.40:0/2311712909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90dc0043b0 0x7f90dc004880 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 -- 172.21.15.40:0/2311712909 >> 172.21.15.40:0/2311712909 conn(0x7f90f008ec10 msgr2=0x7f90f009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 -- 172.21.15.40:0/2311712909 shutdown_connections 2024-04-26T21:15:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.320+0000 7f90f57af640 1 -- 172.21.15.40:0/2311712909 wait complete. 2024-04-26T21:15:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.321+0000 7f90f57af640 1 Processor -- start 2024-04-26T21:15:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.321+0000 7f90f57af640 1 -- start start 2024-04-26T21:15:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc0043b0 0x7f90f00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f015fc40 0x7f90f014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00cfef0 con 0x7f90f00b9c50 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00cfd70 con 0x7f90dc0043b0 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90f57af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90f00d0070 con 0x7f90f015fc40 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46790/0 (socket says 172.21.15.40:46790) 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 -- 172.21.15.40:0/1049279162 learned_addr learned my addr 172.21.15.40:0/1049279162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ee7fc640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f015fc40 0x7f90f014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90eeffd640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc0043b0 0x7f90f00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 -- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f015fc40 msgr2=0x7f90f014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f015fc40 0x7f90f014e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 -- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc0043b0 msgr2=0x7f90f00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc0043b0 0x7f90f00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.322+0000 7f90ef7fe640 1 -- 172.21.15.40:0/1049279162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f0160140 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90ef7fe640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f90f0163080 tx=0x7f90e000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000ec10 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e000edb0 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f00b9170 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0014650 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.323+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f015bf80 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.324+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f00b8fa0 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.324+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90e00147f0 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.325+0000 7f90e77fe640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 0x7f90d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.325+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f90e008b9b0 con 0x7f90f00b9c50 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.325+0000 7f90eeffd640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 0x7f90d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.326+0000 7f90eeffd640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 0x7f90d006ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f90f006b1f0 tx=0x7f90e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:24.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.329+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90e0059d40 con 0x7f90f00b9c50 2024-04-26T21:15:24.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.550+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90f006a290 con 0x7f90f00b9c50 2024-04-26T21:15:24.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.551+0000 7f90e77fe640 1 -- 172.21.15.40:0/1049279162 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7f90e0059640 con 0x7f90f00b9c50 2024-04-26T21:15:24.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:24.552 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:24.552 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 msgr2=0x7f90d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 0x7f90d006ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f90f006b1f0 tx=0x7f90e8008040 comp rx=0 tx=0).stop 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 msgr2=0x7f90f014ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f90f0163080 tx=0x7f90e000e530 comp rx=0 tx=0).stop 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 shutdown_connections 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90d006c920 0x7f90d006ede0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90f00b9c50 0x7f90f014ec20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90f015fc40 0x7f90f014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 --2- 172.21.15.40:0/1049279162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90dc0043b0 0x7f90f00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:24.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 >> 172.21.15.40:0/1049279162 conn(0x7f90f008ec10 msgr2=0x7f90f00b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:24.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 shutdown_connections 2024-04-26T21:15:24.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:24.553+0000 7f90f57af640 1 -- 172.21.15.40:0/1049279162 wait complete. 2024-04-26T21:15:24.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:15:14.354319+0000', 'modified': '2024-04-26T21:15:17.358700+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6453}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6453': {'gid': 6453, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/2174268870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 2174268870}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2174268870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6453, 'qdb_cluster': [6453]} 2024-04-26T21:15:24.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:15:24.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:15:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.074+0000 7f696fd38640 1 Processor -- start 2024-04-26T21:15:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 -- start start 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f696815f370 0x7f69680cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696815f750 0x7f69680b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69680d03a0 con 0x7f69680d37d0 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f69680b4d10 con 0x7f696815f370 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696fd38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69680d0680 con 0x7f696815f750 2024-04-26T21:15:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696dad4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f696815f370 0x7f69680cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.075+0000 7f696e2d5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f69680d37d0 0x7f69680d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48442/0 (socket says 172.21.15.40:48442) 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696e2d5640 1 -- 172.21.15.40:0/1224039891 learned_addr learned my addr 172.21.15.40:0/1224039891 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696d2d3640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696815f750 0x7f69680b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696dad4640 1 -- 172.21.15.40:0/1224039891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696815f750 msgr2=0x7f69680b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696dad4640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696815f750 0x7f69680b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696dad4640 1 -- 172.21.15.40:0/1224039891 >> v1:172.21.15.40:6789/0 conn(0x7f69680d37d0 legacy=0x7f69680d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696cad2640 1 -- 172.21.15.40:0/1224039891 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214323066 0 0) 0x7f69680d03a0 con 0x7f69680d37d0 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.076+0000 7f696dad4640 1 -- 172.21.15.40:0/1224039891 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69680d0960 con 0x7f696815f370 2024-04-26T21:15:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696dad4640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f370 0x7f69680cfd70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e957a1694afd325 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696cad2640 1 -- 172.21.15.40:0/1224039891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958037070 con 0x7f696815f370 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696cad2640 1 -- 172.21.15.40:0/1224039891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695802ae50 con 0x7f696815f370 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696cad2640 1 -- 172.21.15.40:0/1224039891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958032660 con 0x7f696815f370 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 -- 172.21.15.40:0/1224039891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f370 msgr2=0x7f69680cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f370 0x7f69680cfd70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802a100 comp rx=0 tx=0).stop 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 -- 172.21.15.40:0/1224039891 shutdown_connections 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696815f750 0x7f69680b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 --2- 172.21.15.40:0/1224039891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f370 0x7f69680cfd70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 -- 172.21.15.40:0/1224039891 >> 172.21.15.40:0/1224039891 conn(0x7f696808ec10 msgr2=0x7f696809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 -- 172.21.15.40:0/1224039891 shutdown_connections 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.077+0000 7f696fd38640 1 -- 172.21.15.40:0/1224039891 wait complete. 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.078+0000 7f696fd38640 1 Processor -- start 2024-04-26T21:15:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 -- start start 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69680d37d0 0x7f69680ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696815f370 0x7f69681f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69680b51c0 con 0x7f696815f750 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69680d0f20 con 0x7f696815f370 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696fd38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69680b5340 con 0x7f69680d37d0 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46802/0 (socket says 172.21.15.40:46802) 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696e2d5640 1 -- 172.21.15.40:0/1422869263 learned_addr learned my addr 172.21.15.40:0/1422869263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696dad4640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69680d37d0 0x7f69680ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.079+0000 7f696d2d3640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696815f370 0x7f69681f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 -- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69680d37d0 msgr2=0x7f69680ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69680d37d0 0x7f69680ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 -- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696815f370 msgr2=0x7f69681f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696815f370 0x7f69681f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 -- 172.21.15.40:0/1422869263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681e70f0 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696e2d5640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f696000c810 tx=0x7f696000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f695e7fc640 1 -- 172.21.15.40:0/1422869263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960018070 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f695e7fc640 1 -- 172.21.15.40:0/1422869263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696000a640 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f695e7fc640 1 -- 172.21.15.40:0/1422869263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696000aa30 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69680cb190 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.080+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681e73b0 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.081+0000 7f695e7fc640 1 -- 172.21.15.40:0/1422869263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f696001b070 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.082+0000 7f695e7fc640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 0x7f694c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.082+0000 7f695e7fc640 1 -- 172.21.15.40:0/1422869263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f696008b0e0 con 0x7f696815f750 2024-04-26T21:15:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.082+0000 7f696dad4640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 0x7f694c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.083+0000 7f696dad4640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 0x7f694c06edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6958034000 tx=0x7f695802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.083+0000 7f6972a55740 1 Processor -- start 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.083+0000 7f6972a55740 1 -- start start 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bb9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bd6ce0 0x55cf37bd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf37af3740 con 0x55cf37b8f120 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf37a8f3a0 con 0x55cf37bb9440 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55cf37aef700 con 0x55cf37bd6ce0 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46806/0 (socket says 172.21.15.40:46806) 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 -- 172.21.15.40:0/3781073184 learned_addr learned my addr 172.21.15.40:0/3781073184 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696d2d3640 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bd6ce0 0x55cf37bd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696dad4640 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bb9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 -- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bd6ce0 msgr2=0x55cf37bd7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bd6ce0 0x55cf37bd7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 -- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 msgr2=0x55cf37bb9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bb9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 -- 172.21.15.40:0/3781073184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf37aab750 con 0x55cf37b8f120 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f696e2d5640 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f69600014a0 tx=0x7f6960059660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de374e3e892d2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f694b7fe640 1 -- 172.21.15.40:0/3781073184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960061a80 con 0x55cf37b8f120 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f694b7fe640 1 -- 172.21.15.40:0/3781073184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960056f40 con 0x55cf37b8f120 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.084+0000 7f694b7fe640 1 -- 172.21.15.40:0/3781073184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960055510 con 0x55cf37b8f120 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 -- 172.21.15.40:0/3781073184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 msgr2=0x55cf37bd53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f69600014a0 tx=0x7f6960059660 comp rx=0 tx=0).stop 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 -- 172.21.15.40:0/3781073184 shutdown_connections 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37b8f120 0x55cf37bd53a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bd6ce0 0x55cf37bd7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 --2- 172.21.15.40:0/3781073184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bb9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 -- 172.21.15.40:0/3781073184 >> 172.21.15.40:0/3781073184 conn(0x55cf37b8ba50 msgr2=0x55cf37bd4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 -- 172.21.15.40:0/3781073184 shutdown_connections 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.085+0000 7f6972a55740 1 -- 172.21.15.40:0/3781073184 wait complete. 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.086+0000 7f6972a55740 1 -- start start 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.086+0000 7f6972a55740 1 Processor -- start 2024-04-26T21:15:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.086+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bdb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb89e0 0x55cf37bb50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf37b8fec0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf37a7afe0 con 0x55cf37bb9440 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55cf37b89980 con 0x55cf37bb89e0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 10 client.0 _ll_register_callbacks cb 0x55cf37a91150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46822/0 (socket says 172.21.15.40:46822) 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 -- 172.21.15.40:0/370449926 learned_addr learned my addr 172.21.15.40:0/370449926 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696e2d5640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb89e0 0x55cf37bb50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696dad4640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bdb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 -- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb89e0 msgr2=0x55cf37bb50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb89e0 0x55cf37bb50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 -- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 msgr2=0x55cf37bdb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bdb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 -- 172.21.15.40:0/370449926 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf378052d0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f696d2d3640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f696400bf50 tx=0x7f6964012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc360fbd51dccaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964019070 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f6964004240 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6960056f40 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6964024050 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 10 client.6621 fetch_fsmap learned FSMap version 38 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.087+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55cf378cf6b0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960056f40 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 38) ==== 3494+0+0 (secure 0 0 0) 0x7f6964027020 con 0x55cf37bd6ce0 2024-04-26T21:15:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 10 client.6621 fetch_fsmap finished waiting for FSMap version 38 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 10 client.6621 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 20 client.6621 populate_metadata read hostname 'smithi040' 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37b88930 0x55cf37bd9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 4 client.6621 mds_command: new command op to 6453 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 --> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf37b01350 con 0x55cf37b88930 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f696dad4640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37b88930 0x55cf37bd9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f696dad4640 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37b88930 0x55cf37bd9760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3903df6df6749d94 server_cookie=5ef6df4dc7a2de8e in_seq=0 out_seq=0 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.088+0000 7f6948ff9640 10 client.6621 ms_handle_connect on v2:172.21.15.40:6833/2174268870 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6948ff9640 1 -- 172.21.15.40:0/370449926 <== mds.0 v2:172.21.15.40:6833/2174268870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf37b01350 con 0x55cf37b88930 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6948ff9640 10 client.6621 handle_command_reply: tid=0 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 client.6621 shutdown 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 msgr2=0x55cf37bb84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f696400bf50 tx=0x7f6964012760 comp rx=0 tx=0).stop 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 shutdown_connections 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37b88930 0x55cf37bd9760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb89e0 0x55cf37bb50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bb84a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6948ff9640 0 client.6621 ms_handle_reset on v2:172.21.15.40:6833/2174268870 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 --2- 172.21.15.40:0/370449926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb9440 0x55cf37bdb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 >> 172.21.15.40:0/370449926 conn(0x55cf37b8f120 msgr2=0x55cf37bd77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 shutdown_connections 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 1 -- 172.21.15.40:0/370449926 wait complete. 2024-04-26T21:15:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.089+0000 7f6972a55740 20 client.6621 trim_cache size 0 max 16384 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 Processor -- start 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 -- start start 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37bbb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bf7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6968200770 con 0x55cf37bd6ce0 2024-04-26T21:15:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ffb00 con 0x55cf37bb89e0 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.108+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6968200120 con 0x55cf37bb9440 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46824/0 (socket says 172.21.15.40:46824) 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 -- 172.21.15.40:0/2488341460 learned_addr learned my addr 172.21.15.40:0/2488341460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696d2d3640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bf7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 -- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 msgr2=0x55cf37bf7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696dad4640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37bbb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bf7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 -- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 msgr2=0x55cf37bbb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37bbb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 -- 172.21.15.40:0/2488341460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf37bd6030 con 0x55cf37bd6ce0 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696d2d3640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bf7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696dad4640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37bbb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f69600012e0 tx=0x7f696001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bed299d6ef73a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f694affd640 1 -- 172.21.15.40:0/2488341460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960061a80 con 0x55cf37bd6ce0 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f694affd640 1 -- 172.21.15.40:0/2488341460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960089360 con 0x55cf37bd6ce0 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.109+0000 7f694affd640 1 -- 172.21.15.40:0/2488341460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960056f40 con 0x55cf37bd6ce0 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- 172.21.15.40:0/2488341460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 msgr2=0x55cf37bf75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f69600012e0 tx=0x7f696001e000 comp rx=0 tx=0).stop 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- 172.21.15.40:0/2488341460 shutdown_connections 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37bf75a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bf7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 --2- 172.21.15.40:0/2488341460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37bbb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- 172.21.15.40:0/2488341460 >> 172.21.15.40:0/2488341460 conn(0x55cf37b8f120 msgr2=0x55cf37b84f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- 172.21.15.40:0/2488341460 shutdown_connections 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- 172.21.15.40:0/2488341460 wait complete. 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 -- start start 2024-04-26T21:15:25.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.110+0000 7f6972a55740 1 Processor -- start 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37be7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bb4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55cf37b887b0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf37bda3b0 con 0x55cf37bb89e0 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55cf37b88d40 con 0x55cf37bb9440 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f6972a55740 10 client.0 _ll_register_callbacks cb 0x7f696400b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:15:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f696e2d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46840/0 (socket says 172.21.15.40:46840) 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f696e2d5640 1 -- 172.21.15.40:0/2273124690 learned_addr learned my addr 172.21.15.40:0/2273124690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f696d2d3640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bb4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.111+0000 7f696dad4640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37be7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 -- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 msgr2=0x55cf37bb4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bb4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 -- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 msgr2=0x55cf37be7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37be7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 -- 172.21.15.40:0/2273124690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf37be6630 con 0x55cf37bd6ce0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f696e2d5640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6960001330 tx=0x7f6960004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c61062a3a35a1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960061a80 con 0x55cf37bd6ce0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f6960087040 con 0x55cf37bd6ce0 2024-04-26T21:15:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6960089360 con 0x55cf37bd6ce0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6960055510 con 0x55cf37bd6ce0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 10 client.6627 fetch_fsmap learned FSMap version 38 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55cf37c4f1c0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960089360 con 0x55cf37bd6ce0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 38) ==== 3494+0+0 (secure 0 0 0) 0x7f69600615f0 con 0x55cf37bd6ce0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 10 client.6627 fetch_fsmap finished waiting for FSMap version 38 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 10 client.6627 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 20 client.6627 populate_metadata read hostname 'smithi040' 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37bf7060 0x55cf37bf7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 4 client.6627 mds_command: new command op to 6453 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.112+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 --> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] -- command(tid 0: {"prefix": "status"}) -- 0x55cf37bba470 con 0x55cf37bf7060 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f696dad4640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37bf7060 0x55cf37bf7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6948ff9640 10 client.6627 ms_handle_connect on v2:172.21.15.40:6833/2174268870 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f696dad4640 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37bf7060 0x55cf37bf7440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4460fb38649792ec server_cookie=473e777a0f3346f1 in_seq=0 out_seq=0 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6948ff9640 1 -- 172.21.15.40:0/2273124690 <== mds.0 v2:172.21.15.40:6833/2174268870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cf37bba470 con 0x55cf37bf7060 2024-04-26T21:15:25.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6948ff9640 10 client.6627 handle_command_reply: tid=0 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 client.6627 shutdown 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 msgr2=0x55cf37c4e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6960001330 tx=0x7f6960004f90 comp rx=0 tx=0).stop 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 shutdown_connections 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cf37bd6ce0 0x55cf37c4e6e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.40:6833/2174268870,v1:172.21.15.40:6835/2174268870] conn(0x55cf37bf7060 0x55cf37bf7440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55cf37bb9440 0x55cf37bb4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 --2- 172.21.15.40:0/2273124690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf37bb89e0 0x55cf37be7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6948ff9640 0 client.6627 ms_handle_reset on v2:172.21.15.40:6833/2174268870 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 >> 172.21.15.40:0/2273124690 conn(0x55cf37b8f120 msgr2=0x55cf37bbb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 shutdown_connections 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.113+0000 7f6972a55740 1 -- 172.21.15.40:0/2273124690 wait complete. 2024-04-26T21:15:25.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f6972a55740 20 client.6627 trim_cache size 0 max 16384 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6453, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.137165726999999, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 38, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 70, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 70, 2024-04-26T21:15:25.121 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.146341206000001 2024-04-26T21:15:25.122 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:15:25.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 msgr2=0x7f694c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 0x7f694c06edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6958034000 tx=0x7f695802e000 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 msgr2=0x7f69681f8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f696000c810 tx=0x7f696000ccd0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 shutdown_connections 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694c06c8f0 0x7f694c06edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696815f750 0x7f69681f8640 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696815f370 0x7f69681f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 --2- 172.21.15.40:0/1422869263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69680d37d0 0x7f69680ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 >> 172.21.15.40:0/1422869263 conn(0x7f696808ec10 msgr2=0x7f696806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 shutdown_connections 2024-04-26T21:15:25.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.114+0000 7f696fd38640 1 -- 172.21.15.40:0/1422869263 wait complete. 2024-04-26T21:15:25.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:15:25.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.226+0000 7fece83bf640 1 Processor -- start 2024-04-26T21:15:25.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.226+0000 7fece83bf640 1 -- start start 2024-04-26T21:15:25.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece83bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fece00d5610 0x7fece00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece83bf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece00b2510 con 0x7fece015f340 2024-04-26T21:15:25.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece83bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece00b27f0 con 0x7fece015f720 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece83bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fece00b2ad0 con 0x7fece00d5610 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece695c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fece00d5610 0x7fece00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece615b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fece015f340 0x7fece00d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48450/0 (socket says 172.21.15.40:48450) 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece615b640 1 -- 172.21.15.40:0/826782402 learned_addr learned my addr 172.21.15.40:0/826782402 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.227+0000 7fece695c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fece00d5610 0x7fece00b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46846/0 (socket says 172.21.15.40:46846) 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece695c640 1 -- 172.21.15.40:0/826782402 >> v1:172.21.15.184:6789/0 conn(0x7fece015f720 legacy=0x7fece00b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece695c640 1 -- 172.21.15.40:0/826782402 >> v1:172.21.15.40:6789/0 conn(0x7fece015f340 legacy=0x7fece00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece695c640 1 -- 172.21.15.40:0/826782402 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece00b2db0 con 0x7fece00d5610 2024-04-26T21:15:25.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece695c640 1 --2- 172.21.15.40:0/826782402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d5610 0x7fece00b1e80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097a0 tx=0x7fecd802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca497dc214b3147 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece5159640 1 -- 172.21.15.40:0/826782402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8037070 con 0x7fece00d5610 2024-04-26T21:15:25.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece5159640 1 -- 172.21.15.40:0/826782402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd802ae50 con 0x7fece00d5610 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.228+0000 7fece5159640 1 -- 172.21.15.40:0/826782402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8032660 con 0x7fece00d5610 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 -- 172.21.15.40:0/826782402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d5610 msgr2=0x7fece00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 --2- 172.21.15.40:0/826782402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d5610 0x7fece00b1e80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097a0 tx=0x7fecd802a140 comp rx=0 tx=0).stop 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 -- 172.21.15.40:0/826782402 shutdown_connections 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 --2- 172.21.15.40:0/826782402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d5610 0x7fece00b1e80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 -- 172.21.15.40:0/826782402 >> 172.21.15.40:0/826782402 conn(0x7fece008ec70 msgr2=0x7fece009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 -- 172.21.15.40:0/826782402 shutdown_connections 2024-04-26T21:15:25.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.229+0000 7fece83bf640 1 -- 172.21.15.40:0/826782402 wait complete. 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.230+0000 7fece83bf640 1 Processor -- start 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.230+0000 7fece83bf640 1 -- start start 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 0x7fece0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece015f720 0x7fece00d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fece00d40d0 con 0x7fece015f340 2024-04-26T21:15:25.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fece00d3f50 con 0x7fece00d5610 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece83bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fece00b3cc0 con 0x7fece015f720 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46854/0 (socket says 172.21.15.40:46854) 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 -- 172.21.15.40:0/1438640051 learned_addr learned my addr 172.21.15.40:0/1438640051 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece615b640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 0x7fece0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece695c640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece015f720 0x7fece00d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 -- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece015f720 msgr2=0x7fece00d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece015f720 0x7fece00d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 -- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 msgr2=0x7fece0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 0x7fece0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.231+0000 7fece595a640 1 -- 172.21.15.40:0/1438640051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece0159b40 con 0x7fece015f340 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fece615b640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 0x7fece0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fece595a640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fecd400b3f0 tx=0x7fecd400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4015020 con 0x7fece015f340 2024-04-26T21:15:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd4011830 con 0x7fece015f340 2024-04-26T21:15:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4011c20 con 0x7fece015f340 2024-04-26T21:15:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece00c9f50 con 0x7fece015f340 2024-04-26T21:15:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.232+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece00c9190 con 0x7fece015f340 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.233+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece006a290 con 0x7fece015f340 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.233+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd4018020 con 0x7fece015f340 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.234+0000 7fecdeffd640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.234+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fecd408b670 con 0x7fece015f340 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.234+0000 7fece615b640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.234+0000 7fece615b640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecd00012e0 tx=0x7fecd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:25.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.238+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecd401f030 con 0x7fece015f340 2024-04-26T21:15:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.463+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fece009e730 con 0x7fece015f340 2024-04-26T21:15:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.463+0000 7fecdeffd640 1 -- 172.21.15.40:0/1438640051 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7fecd40172c0 con 0x7fece015f340 2024-04-26T21:15:25.464 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:15:25.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 msgr2=0x7feccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecd00012e0 tx=0x7fecd0009040 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 msgr2=0x7fece01605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fecd400b3f0 tx=0x7fecd400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 shutdown_connections 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece015f720 0x7fece00d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015f340 0x7fece01605d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 --2- 172.21.15.40:0/1438640051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece00d5610 0x7fece0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 >> 172.21.15.40:0/1438640051 conn(0x7fece008ec70 msgr2=0x7fece00b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 shutdown_connections 2024-04-26T21:15:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25.465+0000 7fece83bf640 1 -- 172.21.15.40:0/1438640051 wait complete. 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:15:25.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:15:25.478 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:15:25.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.478 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:25.494 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:25.495 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:15:25.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:15:25.497 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:25.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.497 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:15:25.556 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:15:25.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.558 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:15:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:15:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:15:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:15:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:15:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:15:25.615 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:15:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:15:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:25.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.741 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:15:25.757 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:15:25.757 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:15:25.757 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:15:25.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:15:25.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:15:25.822 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:15:25.877 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:15:25.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:15:25.877 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:15:25.932 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:15:25.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:25.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:15:26.015 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:15:26.016 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:15:26.016 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:15:26.024 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:15:26.024 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:15:26.024 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:15:26.024 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:15:26.024 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:15:26.029 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:15:26.029 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:15:26.029 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:15:26.029 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:15:26.030 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:15:26.030 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:15:26.030 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:15:26.030 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:15:26.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:15:26.031 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:15:26.031 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:15:26.031 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:15:26.031 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:15:26.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:15:26.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:15:26.095 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:15:26.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:15:26.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:26.198 INFO:teuthology.orchestra.run.smithi040.stdout:client5477 2024-04-26T21:15:26.199 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:26.199 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:26.214 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:15:26.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.354+0000 7fc229cbd640 1 Processor -- start 2024-04-26T21:15:26.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.354+0000 7fc229cbd640 1 -- start start 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc229cbd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc224148a30 0x7fc224073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc229cbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc2241636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc229cbd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2240cfa80 con 0x7fc224163be0 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc229cbd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc2240cfd60 con 0x7fc224148a30 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc229cbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc2240d0040 con 0x7fc224160640 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc223fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc224163be0 0x7fc2240b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48466/0 (socket says 172.21.15.40:48466) 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.355+0000 7fc223fff640 1 -- 172.21.15.40:0/2969946035 learned_addr learned my addr 172.21.15.40:0/2969946035 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002720318 0 0) 0x7fc2240cfa80 con 0x7fc224163be0 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc20c003520 con 0x7fc224163be0 2024-04-26T21:15:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2969946035 >> v2:172.21.15.184:3300/0 conn(0x7fc224148a30 0x7fc224073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc222ffd640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc2241636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3710393487 0 0) 0x7fc20c003520 con 0x7fc224163be0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 msgr2=0x7fc2241636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc2241636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 >> v2:172.21.15.184:3300/0 conn(0x7fc224148a30 msgr2=0x7fc224073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 --2- 172.21.15.40:0/2969946035 >> v2:172.21.15.184:3300/0 conn(0x7fc224148a30 0x7fc224073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc224162ea0 con 0x7fc224163be0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc21c0019f0 con 0x7fc224163be0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 >> v1:172.21.15.40:6789/0 conn(0x7fc224163be0 legacy=0x7fc2240b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0039b0 0x7fc20c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc20c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2240cfd60 con 0x7fc20c0039b0 2024-04-26T21:15:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2240d0040 con 0x7fc20c004800 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c00b9c0 con 0x7fc224160640 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0039b0 0x7fc20c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc222ffd640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc223fff640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc20c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 -- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 msgr2=0x7fc20c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc224160640 0x7fc20c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 -- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c004800 msgr2=0x7fc20c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c004800 0x7fc20c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.356+0000 7fc2237fe640 1 -- 172.21.15.40:0/2969946035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c003520 con 0x7fc20c0039b0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.357+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0039b0 0x7fc20c003dd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc214009af0 tx=0x7fc20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d19314a0d89d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.357+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214036070 con 0x7fc20c0039b0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.357+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc214031400 con 0x7fc20c0039b0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.357+0000 7fc2227fc640 1 -- 172.21.15.40:0/2969946035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214009690 con 0x7fc20c0039b0 2024-04-26T21:15:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 -- 172.21.15.40:0/2969946035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0039b0 msgr2=0x7fc20c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2969946035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0039b0 0x7fc20c003dd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc214009af0 tx=0x7fc20c005830 comp rx=0 tx=0).stop 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc223fff640 1 -- 172.21.15.40:0/2969946035 reap_dead start 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 -- 172.21.15.40:0/2969946035 shutdown_connections 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 -- 172.21.15.40:0/2969946035 >> 172.21.15.40:0/2969946035 conn(0x7fc2240ce8b0 msgr2=0x7fc2240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 -- 172.21.15.40:0/2969946035 shutdown_connections 2024-04-26T21:15:26.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.358+0000 7fc229cbd640 1 -- 172.21.15.40:0/2969946035 wait complete. 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.359+0000 7fc229cbd640 1 Processor -- start 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.359+0000 7fc229cbd640 1 -- start start 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0039b0 0x7fc224143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2240b7500 0x7fc2240ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc224074590 con 0x7fc2241f4d30 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc224074410 con 0x7fc2240b7500 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc229cbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc224148e10 con 0x7fc20c0039b0 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46874/0 (socket says 172.21.15.40:46874) 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 -- 172.21.15.40:0/2361291250 learned_addr learned my addr 172.21.15.40:0/2361291250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc222ffd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2240b7500 0x7fc2240ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0039b0 0x7fc224143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 -- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0039b0 msgr2=0x7fc224143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0039b0 0x7fc224143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 -- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2240b7500 msgr2=0x7fc2240ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2240b7500 0x7fc2240ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 -- 172.21.15.40:0/2361291250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2240b7960 con 0x7fc2241f4d30 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.360+0000 7fc223fff640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc224163fc0 tx=0x7fc21c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.361+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00ee20 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.361+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c018070 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.361+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2240af000 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.361+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c013660 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.361+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2240af5c0 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.362+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22406a290 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.362+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc21c013800 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.362+0000 7fc20bfff640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 0x7fc21006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.363+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc21c08bf10 con 0x7fc2241f4d30 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.363+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 0x7fc21006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.363+0000 7fc2237fe640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 0x7fc21006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc214033000 tx=0x7fc214008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.367+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc21c05a330 con 0x7fc2241f4d30 2024-04-26T21:15:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.590+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2240739f0 con 0x7fc2241f4d30 2024-04-26T21:15:26.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.591+0000 7fc20bfff640 1 -- 172.21.15.40:0/2361291250 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7fc21c059c30 con 0x7fc2241f4d30 2024-04-26T21:15:26.592 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 msgr2=0x7fc21006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 0x7fc21006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc214033000 tx=0x7fc214008030 comp rx=0 tx=0).stop 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 msgr2=0x7fc22406f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc224163fc0 tx=0x7fc21c00e530 comp rx=0 tx=0).stop 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 shutdown_connections 2024-04-26T21:15:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc21006c8f0 0x7fc21006edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2241f4d30 0x7fc22406f330 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2240b7500 0x7fc2240ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 --2- 172.21.15.40:0/2361291250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0039b0 0x7fc224143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 >> 172.21.15.40:0/2361291250 conn(0x7fc2240ce8b0 msgr2=0x7fc224147d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 shutdown_connections 2024-04-26T21:15:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:26.593+0000 7fc229cbd640 1 -- 172.21.15.40:0/2361291250 wait complete. 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:15:26.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:15:26.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.606 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:15:26.625 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:15:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:15:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:15:26.625 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:26.626 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42547sec preferred_lft 42547sec 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:15:26.627 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:15:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:15:26.628 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:15:26.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.628 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:15:26.692 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:15:26.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.694 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:15:26.752 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:15:26.752 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:15:26.752 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:15:26.752 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:15:26.752 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:15:26.752 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:15:26.752 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:15:26.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:15:26.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:15:26.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:15:26.753 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:15:26.844 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:15:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:15:26.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:15:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:26.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.882 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:15:26.897 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:15:26.897 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:15:26.897 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:15:26.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:26.898 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:15:26.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:15:26.962 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:15:27.016 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:15:27.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:15:27.017 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:15:27.071 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:15:27.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:27.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:15:27.153 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:15:27.153 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:15:27.153 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:15:27.161 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:15:27.161 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:15:27.162 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:15:27.162 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:15:27.162 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:15:27.167 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:15:27.167 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:15:27.167 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:15:27.167 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:15:27.167 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:15:27.168 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:15:27.168 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:15:27.168 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:15:27.170 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:15:27.170 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:15:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:15:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:15:27.171 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:15:27.171 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:15:27.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:27.171 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:15:27.234 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:15:27.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:15:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:15:27.325 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:27.331 INFO:teuthology.orchestra.run.smithi184.stdout:client6645 2024-04-26T21:15:27.332 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:27.332 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:27.348 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-26T21:15:27.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.461+0000 7f7209714640 1 Processor -- start 2024-04-26T21:15:27.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.461+0000 7f7209714640 1 -- start start 2024-04-26T21:15:27.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7209714640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f720415f950 0x7f72040d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7209714640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72040d0d10 0x7f72040d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7209714640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72040b63e0 con 0x7f72040d2230 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7209714640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f72040d5ed0 con 0x7f720415f950 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7209714640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f72040d3120 con 0x7f72040d0d10 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7202ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f720415f950 0x7f72040d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7202ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f720415f950 0x7f72040d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46878/0 (socket says 172.21.15.40:46878) 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.462+0000 7f7202ffd640 1 -- 172.21.15.40:0/71944029 learned_addr learned my addr 172.21.15.40:0/71944029 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.463+0000 7f72037fe640 1 --2- 172.21.15.40:0/71944029 >> v2:172.21.15.184:3300/0 conn(0x7f72040d0d10 0x7f72040d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.463+0000 7f7202ffd640 1 -- 172.21.15.40:0/71944029 >> v2:172.21.15.184:3300/0 conn(0x7f72040d0d10 msgr2=0x7f72040d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.463+0000 7f7202ffd640 1 --2- 172.21.15.40:0/71944029 >> v2:172.21.15.184:3300/0 conn(0x7f72040d0d10 0x7f72040d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.463+0000 7f7202ffd640 1 -- 172.21.15.40:0/71944029 >> v1:172.21.15.184:6789/0 conn(0x7f72040d2230 legacy=0x7f72040c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.463+0000 7f7202ffd640 1 -- 172.21.15.40:0/71944029 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72040b69c0 con 0x7f720415f950 2024-04-26T21:15:27.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7202ffd640 1 --2- 172.21.15.40:0/71944029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f720415f950 0x7f72040d3740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f72040d2c40 tx=0x7f71fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a068813b58593dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7201ffb640 1 -- 172.21.15.40:0/71944029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc037070 con 0x7f720415f950 2024-04-26T21:15:27.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7201ffb640 1 -- 172.21.15.40:0/71944029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc032440 con 0x7f720415f950 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7201ffb640 1 -- 172.21.15.40:0/71944029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc032810 con 0x7f720415f950 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 -- 172.21.15.40:0/71944029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f720415f950 msgr2=0x7f72040d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 --2- 172.21.15.40:0/71944029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f720415f950 0x7f72040d3740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f72040d2c40 tx=0x7f71fc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 -- 172.21.15.40:0/71944029 shutdown_connections 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 --2- 172.21.15.40:0/71944029 >> v2:172.21.15.184:3300/0 conn(0x7f72040d0d10 0x7f72040d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 --2- 172.21.15.40:0/71944029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f720415f950 0x7f72040d3740 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 -- 172.21.15.40:0/71944029 >> 172.21.15.40:0/71944029 conn(0x7f72040d5160 msgr2=0x7f72040d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 -- 172.21.15.40:0/71944029 shutdown_connections 2024-04-26T21:15:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.464+0000 7f7209714640 1 -- 172.21.15.40:0/71944029 wait complete. 2024-04-26T21:15:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.466+0000 7f7209714640 1 Processor -- start 2024-04-26T21:15:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.466+0000 7f7209714640 1 -- start start 2024-04-26T21:15:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.466+0000 7f7209714640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7209714640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72040d2230 0x7f720406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7209714640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f720415f950 0x7f720406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7209714640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72040c9850 con 0x7f72040d0d10 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7209714640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72040c96d0 con 0x7f720415f950 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7209714640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f72040c99d0 con 0x7f72040d2230 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46886/0 (socket says 172.21.15.40:46886) 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 -- 172.21.15.40:0/1409824295 learned_addr learned my addr 172.21.15.40:0/1409824295 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f72037fe640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f720415f950 0x7f720406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f72027fc640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72040d2230 0x7f720406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 -- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72040d2230 msgr2=0x7f720406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72040d2230 0x7f720406ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 -- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f720415f950 msgr2=0x7f720406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f720415f950 0x7f720406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 -- 172.21.15.40:0/1409824295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72040cc960 con 0x7f72040d0d10 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f7202ffd640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f71fc02a220 tx=0x7f71fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc037040 con 0x7f72040d0d10 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.467+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc042070 con 0x7f72040d0d10 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.468+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72040ccc40 con 0x7f72040d0d10 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.468+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc030880 con 0x7f72040d0d10 2024-04-26T21:15:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.468+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72040cd200 con 0x7f72040d0d10 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.469+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72040693d0 con 0x7f72040d0d10 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.469+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71fc030a20 con 0x7f72040d0d10 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.470+0000 7f71fb7fe640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.470+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f71fc0ac310 con 0x7f72040d0d10 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.470+0000 7f72027fc640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.470+0000 7f72027fc640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7204162620 tx=0x7f71ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.474+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71fc07a050 con 0x7f72040d0d10 2024-04-26T21:15:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.659+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f720415bc60 con 0x7f72040d0d10 2024-04-26T21:15:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.659+0000 7f71fb7fe640 1 -- 172.21.15.40:0/1409824295 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v20) ==== 106+0+4 (secure 0 0 0) 0x7f71fc07e350 con 0x7f72040d0d10 2024-04-26T21:15:27.660 INFO:teuthology.orchestra.run.smithi040.stdout:100 2024-04-26T21:15:27.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 msgr2=0x7f71e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7204162620 tx=0x7f71ec009040 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 msgr2=0x7f72040cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f71fc02a220 tx=0x7f71fc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 shutdown_connections 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f720415f950 0x7f720406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72040d2230 0x7f720406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 --2- 172.21.15.40:0/1409824295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72040d0d10 0x7f72040cc230 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 >> 172.21.15.40:0/1409824295 conn(0x7f72040d5160 msgr2=0x7f720406cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 shutdown_connections 2024-04-26T21:15:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.661+0000 7f7209714640 1 -- 172.21.15.40:0/1409824295 wait complete. 2024-04-26T21:15:27.673 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2024-04-26T21:15:27.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.775+0000 7fc0d0445640 1 Processor -- start 2024-04-26T21:15:27.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.775+0000 7fc0d0445640 1 -- start start 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0d0445640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc0c815f680 0x7fc0c80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0d0445640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0c8163aa0 0x7fc0c80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0d0445640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80d3f50 con 0x7fc0c80b60b0 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0d0445640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c815fa60 con 0x7fc0c8163aa0 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0d0445640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c80b5090 con 0x7fc0c815f680 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0cd9e0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0c8163aa0 0x7fc0c80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0ce9e2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc0c80b60b0 0x7fc0c80d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48468/0 (socket says 172.21.15.40:48468) 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.776+0000 7fc0ce9e2640 1 -- 172.21.15.40:0/284110553 learned_addr learned my addr 172.21.15.40:0/284110553 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0ce1e1640 1 --2- 172.21.15.40:0/284110553 >> v2:172.21.15.184:3300/0 conn(0x7fc0c815f680 0x7fc0c80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd9e0640 1 -- 172.21.15.40:0/284110553 >> v2:172.21.15.184:3300/0 conn(0x7fc0c815f680 msgr2=0x7fc0c80d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd9e0640 1 --2- 172.21.15.40:0/284110553 >> v2:172.21.15.184:3300/0 conn(0x7fc0c815f680 0x7fc0c80d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd9e0640 1 -- 172.21.15.40:0/284110553 >> v1:172.21.15.40:6789/0 conn(0x7fc0c80b60b0 legacy=0x7fc0c80d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd9e0640 1 -- 172.21.15.40:0/284110553 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80b5370 con 0x7fc0c8163aa0 2024-04-26T21:15:27.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd1df640 1 -- 172.21.15.40:0/284110553 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133329049 0 0) 0x7fc0c80d3f50 con 0x7fc0c80b60b0 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.777+0000 7fc0cd9e0640 1 --2- 172.21.15.40:0/284110553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b5b70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80d6370 tx=0x7fc0bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e094bae746a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0cd1df640 1 -- 172.21.15.40:0/284110553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc037070 con 0x7fc0c8163aa0 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0cd1df640 1 -- 172.21.15.40:0/284110553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc032440 con 0x7fc0c8163aa0 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0cd1df640 1 -- 172.21.15.40:0/284110553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc032810 con 0x7fc0c8163aa0 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 -- 172.21.15.40:0/284110553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 msgr2=0x7fc0c80b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 --2- 172.21.15.40:0/284110553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b5b70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80d6370 tx=0x7fc0bc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 -- 172.21.15.40:0/284110553 shutdown_connections 2024-04-26T21:15:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 --2- 172.21.15.40:0/284110553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b5b70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 --2- 172.21.15.40:0/284110553 >> v2:172.21.15.184:3300/0 conn(0x7fc0c815f680 0x7fc0c80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 -- 172.21.15.40:0/284110553 >> 172.21.15.40:0/284110553 conn(0x7fc0c808eca0 msgr2=0x7fc0c809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 -- 172.21.15.40:0/284110553 shutdown_connections 2024-04-26T21:15:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.778+0000 7fc0d0445640 1 -- 172.21.15.40:0/284110553 wait complete. 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.779+0000 7fc0d0445640 1 Processor -- start 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 -- start start 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0c80b60b0 0x7fc0c8162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 0x7fc0c80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80d57b0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80d5630 con 0x7fc0c815f680 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0d0445640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d5930 con 0x7fc0c80b60b0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0ce9e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0ce9e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46902/0 (socket says 172.21.15.40:46902) 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.780+0000 7fc0ce9e2640 1 -- 172.21.15.40:0/807280365 learned_addr learned my addr 172.21.15.40:0/807280365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0cd9e0640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 0x7fc0c80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce1e1640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0c80b60b0 0x7fc0c8162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 -- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0c80b60b0 msgr2=0x7fc0c8162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0c80b60b0 0x7fc0c8162450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 -- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 msgr2=0x7fc0c80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 0x7fc0c80ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 -- 172.21.15.40:0/807280365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80cc470 con 0x7fc0c8163aa0 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0cd9e0640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 0x7fc0c80ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0ce9e2640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800c480 tx=0x7fc0b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8018070 con 0x7fc0c8163aa0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80074e0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c80cc750 con 0x7fc0c8163aa0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004a20 con 0x7fc0c8163aa0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.781+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c80c8da0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.782+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8153fa0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.783+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0b801b070 con 0x7fc0c8163aa0 2024-04-26T21:15:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.783+0000 7fc0c6ffd640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.783+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc0b808ad20 con 0x7fc0c8163aa0 2024-04-26T21:15:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.783+0000 7fc0ce1e1640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.784+0000 7fc0ce1e1640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 0x7fc0b406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00113e0 tx=0x7fc0c000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:27.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.788+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0c8153fa0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.973+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7fc0c80b4cd0 con 0x7fc0c8163aa0 2024-04-26T21:15:27.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.976+0000 7fc0c6ffd640 1 -- 172.21.15.40:0/807280365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v21)=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7fc0b8058b90 con 0x7fc0c8163aa0 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 msgr2=0x7fc0b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 0x7fc0b406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00113e0 tx=0x7fc0c000b040 comp rx=0 tx=0).stop 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 msgr2=0x7fc0c80b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800c480 tx=0x7fc0b800c940 comp rx=0 tx=0).stop 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 shutdown_connections 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8163aa0 0x7fc0c80b2720 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0c815f680 0x7fc0c80ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 --2- 172.21.15.40:0/807280365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0c80b60b0 0x7fc0c8162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 >> 172.21.15.40:0/807280365 conn(0x7fc0c808eca0 msgr2=0x7fc0c80b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 shutdown_connections 2024-04-26T21:15:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:27.979+0000 7fc0d0445640 1 -- 172.21.15.40:0/807280365 wait complete. 2024-04-26T21:15:27.991 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:27.991 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> n = 600 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-04-26T21:15:28.007 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-04-26T21:15:28.008 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-04-26T21:15:28.009 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:15:30.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:30.727 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:30.727 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:30.783 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:15:30.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:15:30.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:15:30.880 INFO:teuthology.orchestra.run.smithi040.stdout:client5477 2024-04-26T21:15:30.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.979+0000 7f7993f4e640 1 Processor -- start 2024-04-26T21:15:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 -- start start 2024-04-26T21:15:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f798c0d1740 con 0x7f798c0d11b0 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f798c0d1a20 con 0x7f798c0cf2e0 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.980+0000 7f7993f4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f798c0d1d00 con 0x7f798c06e490 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f7991cea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53454/0 (socket says 172.21.15.40:53454) 2024-04-26T21:15:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f7991cea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:32922/0 (socket says 172.21.15.40:32922) 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 -- 172.21.15.40:0/2471417789 learned_addr learned my addr 172.21.15.40:0/2471417789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f7990ce8640 1 -- 172.21.15.40:0/2471417789 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331001081 0 0) 0x7f798c0d1740 con 0x7f798c0d11b0 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f7990ce8640 1 -- 172.21.15.40:0/2471417789 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7978003520 con 0x7f798c0d11b0 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 -- 172.21.15.40:0/2471417789 >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 msgr2=0x7f798c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 --2- 172.21.15.40:0/2471417789 >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 -- 172.21.15.40:0/2471417789 >> v1:172.21.15.184:6789/0 conn(0x7f798c0d11b0 legacy=0x7f798c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f79924eb640 1 -- 172.21.15.40:0/2471417789 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c0d1fe0 con 0x7f798c06e490 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.981+0000 7f7991cea640 1 --2- 172.21.15.40:0/2471417789 >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.982+0000 7f79924eb640 1 --2- 172.21.15.40:0/2471417789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7988009e80 tx=0x7f798802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eadf21b012c7ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.982+0000 7f7990ce8640 1 -- 172.21.15.40:0/2471417789 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988036070 con 0x7f798c06e490 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.982+0000 7f7990ce8640 1 -- 172.21.15.40:0/2471417789 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7988031600 con 0x7f798c06e490 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.982+0000 7f7990ce8640 1 -- 172.21.15.40:0/2471417789 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880319b0 con 0x7f798c06e490 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 -- 172.21.15.40:0/2471417789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 msgr2=0x7f798c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 --2- 172.21.15.40:0/2471417789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7988009e80 tx=0x7f798802a750 comp rx=0 tx=0).stop 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 -- 172.21.15.40:0/2471417789 shutdown_connections 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 --2- 172.21.15.40:0/2471417789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c06e490 0x7f798c15ebe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 --2- 172.21.15.40:0/2471417789 >> v2:172.21.15.184:3300/0 conn(0x7f798c0cf2e0 0x7f798c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 -- 172.21.15.40:0/2471417789 >> 172.21.15.40:0/2471417789 conn(0x7f798c0b4a60 msgr2=0x7f798c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 -- 172.21.15.40:0/2471417789 shutdown_connections 2024-04-26T21:15:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.983+0000 7f7993f4e640 1 -- 172.21.15.40:0/2471417789 wait complete. 2024-04-26T21:15:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.984+0000 7f7993f4e640 1 Processor -- start 2024-04-26T21:15:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.984+0000 7f7993f4e640 1 -- start start 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 0x7f798c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c0cf2e0 0x7f798c161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f798c0cf6c0 con 0x7f798c0d11b0 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f798c06da10 con 0x7f798c06e490 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7993f4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f798c0cf840 con 0x7f798c0cf2e0 2024-04-26T21:15:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7991cea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 0x7f798c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79914e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c0cf2e0 0x7f798c161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58926/0 (socket says 172.21.15.40:58926) 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 -- 172.21.15.40:0/434327962 learned_addr learned my addr 172.21.15.40:0/434327962 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 -- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c0cf2e0 msgr2=0x7f798c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c0cf2e0 0x7f798c161010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 -- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 msgr2=0x7f798c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 0x7f798c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f79924eb640 1 -- 172.21.15.40:0/434327962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c0cdad0 con 0x7f798c0d11b0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.985+0000 7f7991cea640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 0x7f798c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f79924eb640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7988034040 tx=0x7f7988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988036050 con 0x7f798c0d11b0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7988042070 con 0x7f798c0d11b0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c0cd510 con 0x7f798c0d11b0 2024-04-26T21:15:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988009510 con 0x7f798c0d11b0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.986+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c09ef50 con 0x7f798c0d11b0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.987+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c06a290 con 0x7f798c0d11b0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.987+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7988004450 con 0x7f798c0d11b0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.988+0000 7f79827fc640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.988+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f79880ac6d0 con 0x7f798c0d11b0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.988+0000 7f7991cea640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.989+0000 7f7991cea640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7984009560 tx=0x7f7984009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:30.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:30.992+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f798c0cdad0 con 0x7f798c0d11b0 2024-04-26T21:15:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.217+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798c0b8ef0 con 0x7f798c0d11b0 2024-04-26T21:15:31.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.217+0000 7f79827fc640 1 -- 172.21.15.40:0/434327962 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7f798807a600 con 0x7f798c0d11b0 2024-04-26T21:15:31.219 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:31.219 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:31.219 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 msgr2=0x7f797006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7984009560 tx=0x7f7984009190 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 msgr2=0x7f798c0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7988034040 tx=0x7f7988004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 shutdown_connections 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f798c0d11b0 0x7f798c0ca930 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f798c0cf2e0 0x7f798c161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 --2- 172.21.15.40:0/434327962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f798c06e490 0x7f798c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 >> 172.21.15.40:0/434327962 conn(0x7f798c0b4a60 msgr2=0x7f798c162490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 shutdown_connections 2024-04-26T21:15:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:31.220+0000 7f7993f4e640 1 -- 172.21.15.40:0/434327962 wait complete. 2024-04-26T21:15:31.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-26T21:15:31.351 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":6645,"entity":{"name":{"type":"client","num":6645},"addr":{"type":"v1","addr":"172.21.15.184:0","nonce":2646666576}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":4.177949763,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9755645627246681,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6645 v1:172.21.15.184:0/2646666576","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":5477,"entity":{"name":{"type":"client","num":5477},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1313721528}},"state":"open","num_leases":1,"num_caps":602,"request_load_avg":615,"uptime":5.3159360790000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":822.86642471010987,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":601.89173975214453,"halflife":1.5},"recall_caps_throttle2o":{"value":357.93548592048467,"halflife":0.5},"session_cache_liveness":{"value":1199.7136093547192,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5477 v1:172.21.15.40:0/1313721528","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:15:31.352 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 2646666576, "type": "v1" }, "name": { "num": 6645, "type": "client" } }, "id": 6645, "inst": "client.6645 v1:172.21.15.184:0/2646666576", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.975564562724668 }, "state": "open", "uptime": 4.177949763 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1313721528, "type": "v1" }, "name": { "num": 5477, "type": "client" } }, "id": 5477, "inst": "client.5477 v1:172.21.15.40:0/1313721528", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 602, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 822.8664247101099 }, "recall_caps_throttle": { "halflife": 1.5, "value": 601.8917397521445 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 357.93548592048467 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 615, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1199.7136093547192 }, "state": "open", "uptime": 5.315936079 } ] 2024-04-26T21:15:31.352 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-26T21:15:31.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:15:31.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:15:36.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:36.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.467+0000 7fdc5b2d1640 1 Processor -- start 2024-04-26T21:15:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 -- start start 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc5415f370 0x7fdc540cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 0x7fdc540b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc540d03a0 con 0x7fdc540d37d0 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdc540b4d10 con 0x7fdc5415f370 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5b2d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc540d0680 con 0x7fdc5415f750 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.468+0000 7fdc5906d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc5415f370 0x7fdc540cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc5415f370 0x7fdc540cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58940/0 (socket says 172.21.15.40:58940) 2024-04-26T21:15:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 -- 172.21.15.40:0/1907331989 learned_addr learned my addr 172.21.15.40:0/1907331989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5886c640 1 --2- 172.21.15.40:0/1907331989 >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 0x7fdc540b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 -- 172.21.15.40:0/1907331989 >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 msgr2=0x7fdc540b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 --2- 172.21.15.40:0/1907331989 >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 0x7fdc540b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 -- 172.21.15.40:0/1907331989 >> v1:172.21.15.184:6789/0 conn(0x7fdc540d37d0 legacy=0x7fdc540d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5906d640 1 -- 172.21.15.40:0/1907331989 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540d0960 con 0x7fdc5415f370 2024-04-26T21:15:36.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.469+0000 7fdc5886c640 1 --2- 172.21.15.40:0/1907331989 >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 0x7fdc540b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5906d640 1 --2- 172.21.15.40:0/1907331989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f370 0x7fdc540cfd70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdc540d02b0 tx=0x7fdc4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b81cdbe82b7db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc53fff640 1 -- 172.21.15.40:0/1907331989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44036070 con 0x7fdc5415f370 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc53fff640 1 -- 172.21.15.40:0/1907331989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44031600 con 0x7fdc5415f370 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc53fff640 1 -- 172.21.15.40:0/1907331989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc440319d0 con 0x7fdc5415f370 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/1907331989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f370 msgr2=0x7fdc540cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/1907331989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f370 0x7fdc540cfd70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdc540d02b0 tx=0x7fdc4402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/1907331989 shutdown_connections 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/1907331989 >> v2:172.21.15.184:3300/0 conn(0x7fdc5415f750 0x7fdc540b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/1907331989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f370 0x7fdc540cfd70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.470+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/1907331989 >> 172.21.15.40:0/1907331989 conn(0x7fdc5408ec10 msgr2=0x7fdc5409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.471+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/1907331989 shutdown_connections 2024-04-26T21:15:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.471+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/1907331989 wait complete. 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.472+0000 7fdc5b2d1640 1 Processor -- start 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.472+0000 7fdc5b2d1640 1 -- start start 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.472+0000 7fdc5b2d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc540d37d0 0x7fdc54158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5415f370 0x7fdc5415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540b61d0 con 0x7fdc5415f750 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540d0f20 con 0x7fdc5415f370 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540b6350 con 0x7fdc540d37d0 2024-04-26T21:15:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58954/0 (socket says 172.21.15.40:58954) 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 -- 172.21.15.40:0/4033011380 learned_addr learned my addr 172.21.15.40:0/4033011380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5886c640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5415f370 0x7fdc5415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5906d640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc540d37d0 0x7fdc54158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 -- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc540d37d0 msgr2=0x7fdc54158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc540d37d0 0x7fdc54158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 -- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5415f370 msgr2=0x7fdc5415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5415f370 0x7fdc5415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 -- 172.21.15.40:0/4033011380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540bad40 con 0x7fdc5415f750 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5986e640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b410 tx=0x7fdc4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c015030 con 0x7fdc5415f750 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4c004020 con 0x7fdc5415f750 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c004410 con 0x7fdc5415f750 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.473+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc540c9cc0 con 0x7fdc5415f750 2024-04-26T21:15:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.474+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc540ba260 con 0x7fdc5415f750 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.475+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5406a290 con 0x7fdc5415f750 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.475+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc4c018070 con 0x7fdc5415f750 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.476+0000 7fdc51ffb640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.476+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fdc4c08b470 con 0x7fdc5415f750 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.476+0000 7fdc5906d640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.476+0000 7fdc5906d640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 0x7fdc4006edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc44033000 tx=0x7fdc44009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.480+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc540bad40 con 0x7fdc5415f750 2024-04-26T21:15:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.705+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc54075a00 con 0x7fdc5415f750 2024-04-26T21:15:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.706+0000 7fdc51ffb640 1 -- 172.21.15.40:0/4033011380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7fdc4c0592c0 con 0x7fdc5415f750 2024-04-26T21:15:36.707 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:36.707 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 msgr2=0x7fdc4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 0x7fdc4006edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc44033000 tx=0x7fdc44009000 comp rx=0 tx=0).stop 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 msgr2=0x7fdc540b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b410 tx=0x7fdc4c00b8d0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 shutdown_connections 2024-04-26T21:15:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5415f750 0x7fdc540b3a50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc5415f370 0x7fdc5415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 --2- 172.21.15.40:0/4033011380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc540d37d0 0x7fdc54158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 >> 172.21.15.40:0/4033011380 conn(0x7fdc5408ec10 msgr2=0x7fdc5415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 shutdown_connections 2024-04-26T21:15:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:36.708+0000 7fdc5b2d1640 1 -- 172.21.15.40:0/4033011380 wait complete. 2024-04-26T21:15:36.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-26T21:15:36.840 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":6645,"entity":{"name":{"type":"client","num":6645},"addr":{"type":"v1","addr":"172.21.15.184:0","nonce":2646666576}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":9.6668837629999995,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9380663460888692,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6645 v1:172.21.15.184:0/2646666576","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":5477,"entity":{"name":{"type":"client","num":5477},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1313721528}},"state":"open","num_leases":1,"num_caps":175,"request_load_avg":577,"uptime":10.804870079000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":2100.3334408990895,"halflife":60},"release_caps":{"value":416.1400656035832,"halflife":60},"recall_caps_throttle":{"value":453.74212780857744,"halflife":1.5},"recall_caps_throttle2o":{"value":37.161211834486167,"halflife":0.5},"session_cache_liveness":{"value":1184.5947571583797,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5477 v1:172.21.15.40:0/1313721528","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:15:36.841 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 2646666576, "type": "v1" }, "name": { "num": 6645, "type": "client" } }, "id": 6645, "inst": "client.6645 v1:172.21.15.184:0/2646666576", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.938066346088869 }, "state": "open", "uptime": 9.666883763 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1313721528, "type": "v1" }, "name": { "num": 5477, "type": "client" } }, "id": 5477, "inst": "client.5477 v1:172.21.15.40:0/1313721528", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 175, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 2100.3334408990895 }, "recall_caps_throttle": { "halflife": 1.5, "value": 453.74212780857744 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 37.16121183448617 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 416.1400656035832 }, "request_load_avg": 577, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1184.5947571583797 }, "state": "open", "uptime": 10.804870079 } ] 2024-04-26T21:15:36.841 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:15:36.841 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:15:36.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:36.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:36.917 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:15:36.917 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:15:36.918 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:36.918 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:36.933 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:36.933 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:36.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.129+0000 7f5a2b409640 1 Processor -- start 2024-04-26T21:15:37.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.129+0000 7f5a2b409640 1 -- start start 2024-04-26T21:15:37.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a2b409640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a2406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a2b409640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a240d1740 con 0x7f5a240d11b0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a2b409640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a240d1a20 con 0x7f5a2406e490 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a2b409640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5a240d1d00 con 0x7f5a240cf2e0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a289a4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5a240d11b0 0x7f5a2406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39966/0 (socket says 172.21.15.40:39966) 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a289a4640 1 -- 172.21.15.40:0/264987608 learned_addr learned my addr 172.21.15.40:0/264987608 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a291a5640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a2406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933122940 0 0) 0x7f5a240d1740 con 0x7f5a240d11b0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.130+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a10003520 con 0x7f5a240d11b0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143107635 0 0) 0x7f5a10003520 con 0x7f5a240d11b0 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 msgr2=0x7f5a2406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a2406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 >> v1:172.21.15.184:6789/0 conn(0x7f5a2406e490 legacy=0x7f5a2415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a240d1fe0 con 0x7f5a240d11b0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5a14001670 con 0x7f5a240d11b0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 >> v1:172.21.15.40:6789/0 conn(0x7f5a240d11b0 legacy=0x7f5a2406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100039e0 0x7f5a10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10004390 0x7f5a10004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a100050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5a240d1d00 con 0x7f5a100039e0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1000b9c0 con 0x7f5a10004390 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1000bbc0 con 0x7f5a240cf2e0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100039e0 0x7f5a10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a289a4640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10004390 0x7f5a10004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a299a6640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a100050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 -- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 msgr2=0x7f5a100050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a240cf2e0 0x7f5a100050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 -- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10004390 msgr2=0x7f5a10004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10004390 0x7f5a10004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.131+0000 7f5a291a5640 1 -- 172.21.15.40:0/264987608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a10003520 con 0x7f5a100039e0 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.132+0000 7f5a291a5640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100039e0 0x7f5a10003dc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ac0 tx=0x7f5a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d287057df321c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.132+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20036070 con 0x7f5a100039e0 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.132+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a20031400 con 0x7f5a100039e0 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.132+0000 7f5a1bfff640 1 -- 172.21.15.40:0/264987608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20009690 con 0x7f5a100039e0 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 -- 172.21.15.40:0/264987608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100039e0 msgr2=0x7f5a10003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 --2- 172.21.15.40:0/264987608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100039e0 0x7f5a10003dc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ac0 tx=0x7f5a10005830 comp rx=0 tx=0).stop 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a299a6640 1 -- 172.21.15.40:0/264987608 reap_dead start 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 -- 172.21.15.40:0/264987608 shutdown_connections 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 -- 172.21.15.40:0/264987608 >> 172.21.15.40:0/264987608 conn(0x7f5a240b4a60 msgr2=0x7f5a240b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 -- 172.21.15.40:0/264987608 shutdown_connections 2024-04-26T21:15:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.133+0000 7f5a2b409640 1 -- 172.21.15.40:0/264987608 wait complete. 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.134+0000 7f5a2b409640 1 Processor -- start 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.134+0000 7f5a2b409640 1 -- start start 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a100039e0 0x7f5a241f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a240b75c0 0x7f5a24152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5a240cf840 con 0x7f5a240b7a40 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a240cf6c0 con 0x7f5a240b75c0 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5a240ce660 con 0x7f5a100039e0 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58970/0 (socket says 172.21.15.40:58970) 2024-04-26T21:15:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 -- 172.21.15.40:0/2978343429 learned_addr learned my addr 172.21.15.40:0/2978343429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a291a5640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a100039e0 0x7f5a241f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a289a4640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a240b75c0 0x7f5a24152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 -- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a100039e0 msgr2=0x7f5a241f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a100039e0 0x7f5a241f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 -- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a240b75c0 msgr2=0x7f5a24152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a240b75c0 0x7f5a24152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 -- 172.21.15.40:0/2978343429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a241f9a20 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a299a6640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5a2406e8a0 tx=0x7f5a1c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c019070 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a1c006d60 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a24153970 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.135+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c003b90 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.136+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2409e330 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.137+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2406a290 con 0x7f5a240b7a40 2024-04-26T21:15:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.137+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a1c01c070 con 0x7f5a240b7a40 2024-04-26T21:15:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.137+0000 7f5a19ffb640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 0x7f5a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.138+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f5a1c08aef0 con 0x7f5a240b7a40 2024-04-26T21:15:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.138+0000 7f5a291a5640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 0x7f5a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.138+0000 7f5a291a5640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 0x7f5a0806edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ac0 tx=0x7f5a20008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.142+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a241f9a20 con 0x7f5a240b7a40 2024-04-26T21:15:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.330+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a240b8ef0 con 0x7f5a240b7a40 2024-04-26T21:15:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.332+0000 7f5a19ffb640 1 -- 172.21.15.40:0/2978343429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16668 (secure 0 0 0) 0x7f5a1c058c30 con 0x7f5a240b7a40 2024-04-26T21:15:37.333 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:37.333 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:15.986368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 msgr2=0x7f5a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 0x7f5a0806edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ac0 tx=0x7f5a20008030 comp rx=0 tx=0).stop 2024-04-26T21:15:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 msgr2=0x7f5a24153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5a2406e8a0 tx=0x7f5a1c00ff70 comp rx=0 tx=0).stop 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 shutdown_connections 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5a0806c8f0 0x7f5a0806edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a240b7a40 0x7f5a24153060 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a240b75c0 0x7f5a24152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 --2- 172.21.15.40:0/2978343429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a100039e0 0x7f5a241f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 >> 172.21.15.40:0/2978343429 conn(0x7f5a240b4a60 msgr2=0x7f5a2409ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 shutdown_connections 2024-04-26T21:15:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.334+0000 7f5a2b409640 1 -- 172.21.15.40:0/2978343429 wait complete. 2024-04-26T21:15:37.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:12.615950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:13.297387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:37.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 Processor -- start 2024-04-26T21:15:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 -- start start 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff34415e7e0 0x7ff3440ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 0x7ff3440d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3440d1d90 0x7ff344161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3440d2280 con 0x7ff34415e7e0 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff3440cdd40 con 0x7ff344161cb0 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.449+0000 7ff3497b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3440cdfe0 con 0x7ff3440d1d90 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff34415e7e0 0x7ff3440ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff34415e7e0 0x7ff3440ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58986/0 (socket says 172.21.15.40:58986) 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 -- 172.21.15.40:0/3573347622 learned_addr learned my addr 172.21.15.40:0/3573347622 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff3427fc640 1 --2- 172.21.15.40:0/3573347622 >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 0x7ff3440d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff3437fe640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3440d1d90 0x7ff344161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 -- 172.21.15.40:0/3573347622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3440d1d90 msgr2=0x7ff344161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3440d1d90 0x7ff344161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 -- 172.21.15.40:0/3573347622 >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 msgr2=0x7ff3440d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 --2- 172.21.15.40:0/3573347622 >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 0x7ff3440d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff342ffd640 1 -- 172.21.15.40:0/3573347622 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3440ce2c0 con 0x7ff34415e7e0 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.450+0000 7ff3427fc640 1 --2- 172.21.15.40:0/3573347622 >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 0x7ff3440d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff342ffd640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440ceda0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff344160fc0 tx=0x7ff33402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb5fd6a0d51e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff341ffb640 1 -- 172.21.15.40:0/3573347622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334036070 con 0x7ff34415e7e0 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff341ffb640 1 -- 172.21.15.40:0/3573347622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff334031600 con 0x7ff34415e7e0 2024-04-26T21:15:37.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff341ffb640 1 -- 172.21.15.40:0/3573347622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3340319d0 con 0x7ff34415e7e0 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 -- 172.21.15.40:0/3573347622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 msgr2=0x7ff3440ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440ceda0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff344160fc0 tx=0x7ff33402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 -- 172.21.15.40:0/3573347622 shutdown_connections 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3440d1d90 0x7ff344161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 --2- 172.21.15.40:0/3573347622 >> v2:172.21.15.184:3300/0 conn(0x7ff344161cb0 0x7ff3440d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 --2- 172.21.15.40:0/3573347622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440ceda0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 -- 172.21.15.40:0/3573347622 >> 172.21.15.40:0/3573347622 conn(0x7ff34409ba40 msgr2=0x7ff34409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.451+0000 7ff3497b4640 1 -- 172.21.15.40:0/3573347622 shutdown_connections 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.452+0000 7ff3497b4640 1 -- 172.21.15.40:0/3573347622 wait complete. 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 Processor -- start 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 -- start start 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3440d1d90 0x7ff344155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff344161cb0 0x7ff344160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440cf2e0 con 0x7ff34415e7e0 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440ce880 con 0x7ff3440d1d90 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.453+0000 7ff3497b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3441615c0 con 0x7ff344161cb0 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59002/0 (socket says 172.21.15.40:59002) 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 -- 172.21.15.40:0/2033513585 learned_addr learned my addr 172.21.15.40:0/2033513585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:37.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 -- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff344161cb0 msgr2=0x7ff344160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff344161cb0 0x7ff344160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff342ffd640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3440d1d90 0x7ff344155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 -- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3440d1d90 msgr2=0x7ff344155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3440d1d90 0x7ff344155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 -- 172.21.15.40:0/2033513585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34415f490 con 0x7ff34415e7e0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3427fc640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d800 tx=0x7ff32c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004030 con 0x7ff34415e7e0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c0041d0 con 0x7ff34415e7e0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34415f7d0 con 0x7ff34415e7e0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0096a0 con 0x7ff34415e7e0 2024-04-26T21:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.454+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff344160010 con 0x7ff34415e7e0 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.455+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34406a290 con 0x7ff34415e7e0 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.456+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff32c009840 con 0x7ff34415e7e0 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.456+0000 7ff33b7fe640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 0x7ff32406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.456+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff32c08bdc0 con 0x7ff34415e7e0 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.456+0000 7ff342ffd640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 0x7ff32406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.457+0000 7ff342ffd640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 0x7ff32406edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff334033000 tx=0x7ff334009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.460+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff34415f7d0 con 0x7ff34415e7e0 2024-04-26T21:15:37.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.645+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3440b8f10 con 0x7ff34415e7e0 2024-04-26T21:15:37.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.646+0000 7ff33b7fe640 1 -- 172.21.15.40:0/2033513585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7ff32c05a280 con 0x7ff34415e7e0 2024-04-26T21:15:37.647 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:37.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 msgr2=0x7ff32406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 0x7ff32406edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff334033000 tx=0x7ff334009000 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 msgr2=0x7ff3440c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d800 tx=0x7ff32c00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 shutdown_connections 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff32406c8f0 0x7ff32406edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff344161cb0 0x7ff344160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff34415e7e0 0x7ff3440c65a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 --2- 172.21.15.40:0/2033513585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3440d1d90 0x7ff344155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 >> 172.21.15.40:0/2033513585 conn(0x7ff34409ba40 msgr2=0x7ff3440c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 shutdown_connections 2024-04-26T21:15:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.648+0000 7ff3497b4640 1 -- 172.21.15.40:0/2033513585 wait complete. 2024-04-26T21:15:37.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:15:37.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.763+0000 7fd30ee93640 1 Processor -- start 2024-04-26T21:15:37.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.763+0000 7fd30ee93640 1 -- start start 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30ee93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3081631f0 0x7fd3080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30ee93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd30806ea30 0x7fd3080b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30ee93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd30809c790 con 0x7fd3080d1170 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30ee93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd30806eec0 con 0x7fd30806ea30 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30ee93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3080d15f0 con 0x7fd3081631f0 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd30806ea30 0x7fd3080b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd30806ea30 0x7fd3080b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59016/0 (socket says 172.21.15.40:59016) 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 -- 172.21.15.40:0/2223123236 learned_addr learned my addr 172.21.15.40:0/2223123236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:37.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30cc2f640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3081631f0 0x7fd3080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 -- 172.21.15.40:0/2223123236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3081631f0 msgr2=0x7fd3080b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3081631f0 0x7fd3080b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 -- 172.21.15.40:0/2223123236 >> v1:172.21.15.184:6789/0 conn(0x7fd3080d1170 legacy=0x7fd3081635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.764+0000 7fd30d430640 1 -- 172.21.15.40:0/2223123236 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd308163ec0 con 0x7fd30806ea30 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd30d430640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd30806ea30 0x7fd3080b3300 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e80 tx=0x7fd30002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af60a5b526d4543 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd2ff7fe640 1 -- 172.21.15.40:0/2223123236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300036070 con 0x7fd30806ea30 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd2ff7fe640 1 -- 172.21.15.40:0/2223123236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300031600 con 0x7fd30806ea30 2024-04-26T21:15:37.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd2ff7fe640 1 -- 172.21.15.40:0/2223123236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000319d0 con 0x7fd30806ea30 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd30ee93640 1 -- 172.21.15.40:0/2223123236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd30806ea30 msgr2=0x7fd3080b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.765+0000 7fd30ee93640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd30806ea30 0x7fd3080b3300 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e80 tx=0x7fd30002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 -- 172.21.15.40:0/2223123236 shutdown_connections 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd30806ea30 0x7fd3080b3300 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 --2- 172.21.15.40:0/2223123236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3081631f0 0x7fd3080b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 -- 172.21.15.40:0/2223123236 >> 172.21.15.40:0/2223123236 conn(0x7fd30809ba40 msgr2=0x7fd30809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 -- 172.21.15.40:0/2223123236 shutdown_connections 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.766+0000 7fd30ee93640 1 -- 172.21.15.40:0/2223123236 wait complete. 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.767+0000 7fd30ee93640 1 Processor -- start 2024-04-26T21:15:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.767+0000 7fd30ee93640 1 -- start start 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30806ea30 0x7fd308161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3080d1170 0x7fd3080c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3080d2000 con 0x7fd3081631f0 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3080d1d20 con 0x7fd30806ea30 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3080d0230 con 0x7fd3080d1170 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59030/0 (socket says 172.21.15.40:59030) 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 -- 172.21.15.40:0/3815008664 learned_addr learned my addr 172.21.15.40:0/3815008664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30cc2f640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30806ea30 0x7fd308161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd2fffff640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3080d1170 0x7fd3080c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 -- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3080d1170 msgr2=0x7fd3080c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3080d1170 0x7fd3080c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 -- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30806ea30 msgr2=0x7fd308161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30806ea30 0x7fd308161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 -- 172.21.15.40:0/3815008664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd308161db0 con 0x7fd3081631f0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30d430640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e20 tx=0x7fd300004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30003f020 con 0x7fd3081631f0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300037040 con 0x7fd3081631f0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd308162090 con 0x7fd3081631f0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.768+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300036070 con 0x7fd3081631f0 2024-04-26T21:15:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.769+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd308162650 con 0x7fd3081631f0 2024-04-26T21:15:37.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.770+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3081495b0 con 0x7fd3081631f0 2024-04-26T21:15:37.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.770+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd300042020 con 0x7fd3081631f0 2024-04-26T21:15:37.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.770+0000 7fd2fd7fa640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 0x7fd2f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:37.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.771+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fd3000abfa0 con 0x7fd3081631f0 2024-04-26T21:15:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.771+0000 7fd30cc2f640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 0x7fd2f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.771+0000 7fd30cc2f640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 0x7fd2f406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.775+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd300049030 con 0x7fd3081631f0 2024-04-26T21:15:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.958+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd30815e630 con 0x7fd3081631f0 2024-04-26T21:15:37.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.959+0000 7fd2fd7fa640 1 -- 172.21.15.40:0/3815008664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7fd30007a340 con 0x7fd3081631f0 2024-04-26T21:15:37.960 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.960+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 msgr2=0x7fd2f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.960+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 0x7fd2f406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304008040 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.960+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 msgr2=0x7fd3080c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.960+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e20 tx=0x7fd300004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 shutdown_connections 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd2f406c8f0 0x7fd2f406edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3081631f0 0x7fd3080c97f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3080d1170 0x7fd3080c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 --2- 172.21.15.40:0/3815008664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30806ea30 0x7fd308161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 >> 172.21.15.40:0/3815008664 conn(0x7fd30809ba40 msgr2=0x7fd30815eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:37.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 shutdown_connections 2024-04-26T21:15:37.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:37.961+0000 7fd30ee93640 1 -- 172.21.15.40:0/3815008664 wait complete. 2024-04-26T21:15:37.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:15:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.075+0000 7ff3ca4bd640 1 Processor -- start 2024-04-26T21:15:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.075+0000 7ff3ca4bd640 1 -- start start 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3ca4bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 0x7ff3c40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3ca4bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3ca4bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c409c790 con 0x7ff3c40b3300 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3ca4bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c40d15f0 con 0x7ff3c40d1170 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3ca4bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c4163ec0 con 0x7ff3c41631f0 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3c37fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.076+0000 7ff3c3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 0x7ff3c40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c37fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33012/0 (socket says 172.21.15.40:33012) 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c37fe640 1 -- 172.21.15.40:0/206482869 learned_addr learned my addr 172.21.15.40:0/206482869 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c3fff640 1 -- 172.21.15.40:0/206482869 >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 msgr2=0x7ff3c41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c3fff640 1 --2- 172.21.15.40:0/206482869 >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c3fff640 1 -- 172.21.15.40:0/206482869 >> v1:172.21.15.184:6789/0 conn(0x7ff3c40b3300 legacy=0x7ff3c40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c3fff640 1 -- 172.21.15.40:0/206482869 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40d1a40 con 0x7ff3c41631f0 2024-04-26T21:15:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.077+0000 7ff3c37fe640 1 --2- 172.21.15.40:0/206482869 >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3c3fff640 1 --2- 172.21.15.40:0/206482869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 0x7ff3c40b2000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49c1689f4a9d7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3c2ffd640 1 -- 172.21.15.40:0/206482869 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4037070 con 0x7ff3c41631f0 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3c2ffd640 1 -- 172.21.15.40:0/206482869 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b402ae50 con 0x7ff3c41631f0 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3c2ffd640 1 -- 172.21.15.40:0/206482869 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4032660 con 0x7ff3c41631f0 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/206482869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 msgr2=0x7ff3c40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/206482869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 0x7ff3c40b2000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402a100 comp rx=0 tx=0).stop 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/206482869 shutdown_connections 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/206482869 >> v2:172.21.15.184:3300/0 conn(0x7ff3c40d1170 0x7ff3c41635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/206482869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c41631f0 0x7ff3c40b2000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/206482869 >> 172.21.15.40:0/206482869 conn(0x7ff3c409ba40 msgr2=0x7ff3c409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/206482869 shutdown_connections 2024-04-26T21:15:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.078+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/206482869 wait complete. 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 Processor -- start 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 -- start start 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 0x7ff3c4151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3c40d1170 0x7ff3c40b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 0x7ff3c40b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40d0230 con 0x7ff3c41631f0 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40d2000 con 0x7ff3c40d1170 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.080+0000 7ff3ca4bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c41f5b90 con 0x7ff3c40b3300 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 0x7ff3c4151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 0x7ff3c40b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 0x7ff3c4151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:53566/0 (socket says 172.21.15.40:53566) 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c3fff640 1 -- 172.21.15.40:0/3155217374 learned_addr learned my addr 172.21.15.40:0/3155217374 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c37fe640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3c40d1170 0x7ff3c40b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 -- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 msgr2=0x7ff3c4151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 0x7ff3c4151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 -- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3c40d1170 msgr2=0x7ff3c40b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3c40d1170 0x7ff3c40b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 -- 172.21.15.40:0/3155217374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40b7c50 con 0x7ff3c41631f0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c8a5a640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 0x7ff3c40b4fc0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c690 tx=0x7ff3bc00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc003b70 con 0x7ff3c41631f0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc003d10 con 0x7ff3c41631f0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c40b7f30 con 0x7ff3c41631f0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc005310 con 0x7ff3c41631f0 2024-04-26T21:15:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.081+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41f54a0 con 0x7ff3c41631f0 2024-04-26T21:15:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.082+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c406a290 con 0x7ff3c41631f0 2024-04-26T21:15:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.083+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3bc0054b0 con 0x7ff3c41631f0 2024-04-26T21:15:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.083+0000 7ff3c0ff9640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.083+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff3bc08b850 con 0x7ff3c41631f0 2024-04-26T21:15:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.083+0000 7ff3c3fff640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.084+0000 7ff3c3fff640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 0x7ff3a406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.088+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3bc059630 con 0x7ff3c41631f0 2024-04-26T21:15:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.274+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3c40d2560 con 0x7ff3c41631f0 2024-04-26T21:15:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.274+0000 7ff3c0ff9640 1 -- 172.21.15.40:0/3155217374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7ff3bc05d930 con 0x7ff3c41631f0 2024-04-26T21:15:38.275 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 msgr2=0x7ff3a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 0x7ff3a406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 msgr2=0x7ff3c40b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 0x7ff3c40b4fc0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c690 tx=0x7ff3bc00cb50 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 shutdown_connections 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c41631f0 0x7ff3c40b4fc0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3c40d1170 0x7ff3c40b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 --2- 172.21.15.40:0/3155217374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3c40b3300 0x7ff3c4151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.276+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 >> 172.21.15.40:0/3155217374 conn(0x7ff3c409ba40 msgr2=0x7ff3c415e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.277+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 shutdown_connections 2024-04-26T21:15:38.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.277+0000 7ff3ca4bd640 1 -- 172.21.15.40:0/3155217374 wait complete. 2024-04-26T21:15:38.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:15:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.388+0000 7f0bac71b640 1 Processor -- start 2024-04-26T21:15:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.388+0000 7f0bac71b640 1 -- start start 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.389+0000 7f0bac71b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ba40d3ee0 0x7f0ba40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.389+0000 7f0bac71b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ba415f750 0x7f0ba40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.389+0000 7f0bac71b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba40d03a0 con 0x7f0ba415f370 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.389+0000 7f0bac71b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba40b4d10 con 0x7f0ba415f750 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.389+0000 7f0bac71b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba40d0680 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ba415f750 0x7f0ba40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ba415f750 0x7f0ba40b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59052/0 (socket says 172.21.15.40:59052) 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 -- 172.21.15.40:0/1864764110 learned_addr learned my addr 172.21.15.40:0/1864764110 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0ba9cb6640 1 --2- 172.21.15.40:0/1864764110 >> v2:172.21.15.184:3300/0 conn(0x7f0ba40d3ee0 0x7f0ba40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 -- 172.21.15.40:0/1864764110 >> v2:172.21.15.184:3300/0 conn(0x7f0ba40d3ee0 msgr2=0x7f0ba40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:38.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 --2- 172.21.15.40:0/1864764110 >> v2:172.21.15.184:3300/0 conn(0x7f0ba40d3ee0 0x7f0ba40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 -- 172.21.15.40:0/1864764110 >> v1:172.21.15.184:6789/0 conn(0x7f0ba415f370 legacy=0x7f0ba40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:38.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.390+0000 7f0baacb8640 1 -- 172.21.15.40:0/1864764110 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40d0960 con 0x7f0ba415f750 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0baacb8640 1 --2- 172.21.15.40:0/1864764110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba415f750 0x7f0ba40b47d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009e50 tx=0x7f0b9c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0f345e6e344def server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0ba94b5640 1 -- 172.21.15.40:0/1864764110 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c036070 con 0x7f0ba415f750 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0ba94b5640 1 -- 172.21.15.40:0/1864764110 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c031600 con 0x7f0ba415f750 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0ba94b5640 1 -- 172.21.15.40:0/1864764110 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0319d0 con 0x7f0ba415f750 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 -- 172.21.15.40:0/1864764110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba415f750 msgr2=0x7f0ba40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1864764110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba415f750 0x7f0ba40b47d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009e50 tx=0x7f0b9c02a710 comp rx=0 tx=0).stop 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 -- 172.21.15.40:0/1864764110 shutdown_connections 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1864764110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba415f750 0x7f0ba40b47d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1864764110 >> v2:172.21.15.184:3300/0 conn(0x7f0ba40d3ee0 0x7f0ba40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 -- 172.21.15.40:0/1864764110 >> 172.21.15.40:0/1864764110 conn(0x7f0ba408ec10 msgr2=0x7f0ba409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 -- 172.21.15.40:0/1864764110 shutdown_connections 2024-04-26T21:15:38.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.391+0000 7f0bac71b640 1 -- 172.21.15.40:0/1864764110 wait complete. 2024-04-26T21:15:38.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.392+0000 7f0bac71b640 1 Processor -- start 2024-04-26T21:15:38.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 -- start start 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 0x7f0ba415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba415f750 0x7f0ba40cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40b51c0 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40d0f20 con 0x7f0ba415f370 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0bac71b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba40b5340 con 0x7f0ba415f750 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0baa4b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0baa4b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59054/0 (socket says 172.21.15.40:59054) 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.393+0000 7f0baa4b7640 1 -- 172.21.15.40:0/1284691644 learned_addr learned my addr 172.21.15.40:0/1284691644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0ba9cb6640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 0x7f0ba415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baacb8640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba415f750 0x7f0ba40cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 -- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba415f750 msgr2=0x7f0ba40cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba415f750 0x7f0ba40cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 -- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 msgr2=0x7f0ba415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 0x7f0ba415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 -- 172.21.15.40:0/1284691644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40cd630 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0ba9cb6640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 0x7f0ba415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0baa4b7640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c430 tx=0x7f0ba000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0011790 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0017070 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba40cd910 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0011bb0 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.394+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba415e5e0 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.395+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba4162ea0 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.396+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ba001c460 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.396+0000 7f0b9affd640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 0x7f0b8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.396+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f0ba008b9f0 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.396+0000 7f0ba9cb6640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 0x7f0b8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.397+0000 7f0ba9cb6640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 0x7f0b8806ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406e250 tx=0x7f0b94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.401+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ba0059e20 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.599+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ba406a290 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.599+0000 7f0b9affd640 1 -- 172.21.15.40:0/1284691644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f0ba0059720 con 0x7f0ba40d3ee0 2024-04-26T21:15:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:15:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 msgr2=0x7f0b8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 0x7f0b8806ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406e250 tx=0x7f0b94008040 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 msgr2=0x7f0ba40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c430 tx=0x7f0ba000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 shutdown_connections 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b8806c920 0x7f0b8806ede0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba415f750 0x7f0ba40cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba415f370 0x7f0ba415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 --2- 172.21.15.40:0/1284691644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ba40d3ee0 0x7f0ba40ce3f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 >> 172.21.15.40:0/1284691644 conn(0x7f0ba408ec10 msgr2=0x7f0ba409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 shutdown_connections 2024-04-26T21:15:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.601+0000 7f0bac71b640 1 -- 172.21.15.40:0/1284691644 wait complete. 2024-04-26T21:15:38.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:15:38.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:38.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:15:38.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:15:38.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:38.717 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:15:38.733 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:15:38.733 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:38.734 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:38.753 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:38.753 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:38.811 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.910+0000 7fec706da640 1 Processor -- start 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.910+0000 7fec706da640 1 -- start start 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.911+0000 7fec706da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec6815f750 0x7fec680cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.911+0000 7fec706da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec680d03a0 con 0x7fec6815f370 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.911+0000 7fec706da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec680d0680 con 0x7fec680d3ee0 2024-04-26T21:15:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.911+0000 7fec706da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fec680d0960 con 0x7fec6815f750 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6ec77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec6815f750 0x7fec680cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6e476640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec6815f370 0x7fec680d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39978/0 (socket says 172.21.15.40:39978) 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6e476640 1 -- 172.21.15.40:0/3643855936 learned_addr learned my addr 172.21.15.40:0/3643855936 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6ec77640 1 -- 172.21.15.40:0/3643855936 >> v1:172.21.15.184:6789/0 conn(0x7fec680d3ee0 legacy=0x7fec680b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6ec77640 1 -- 172.21.15.40:0/3643855936 >> v1:172.21.15.40:6789/0 conn(0x7fec6815f370 legacy=0x7fec680d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6d474640 1 -- 172.21.15.40:0/3643855936 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552748770 0 0) 0x7fec680d03a0 con 0x7fec6815f370 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.912+0000 7fec6ec77640 1 -- 172.21.15.40:0/3643855936 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec680d0c40 con 0x7fec6815f750 2024-04-26T21:15:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec6ec77640 1 --2- 172.21.15.40:0/3643855936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680cfd70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fec58009d90 tx=0x7fec5802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f89a5bdcec70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec6d474640 1 -- 172.21.15.40:0/3643855936 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58007740 con 0x7fec6815f750 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec6d474640 1 -- 172.21.15.40:0/3643855936 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec580078e0 con 0x7fec6815f750 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec6d474640 1 -- 172.21.15.40:0/3643855936 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58007cb0 con 0x7fec6815f750 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 -- 172.21.15.40:0/3643855936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 msgr2=0x7fec680cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 --2- 172.21.15.40:0/3643855936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680cfd70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fec58009d90 tx=0x7fec5802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 -- 172.21.15.40:0/3643855936 shutdown_connections 2024-04-26T21:15:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 --2- 172.21.15.40:0/3643855936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680cfd70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 -- 172.21.15.40:0/3643855936 >> 172.21.15.40:0/3643855936 conn(0x7fec6808ec10 msgr2=0x7fec6809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 -- 172.21.15.40:0/3643855936 shutdown_connections 2024-04-26T21:15:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.913+0000 7fec706da640 1 -- 172.21.15.40:0/3643855936 wait complete. 2024-04-26T21:15:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 Processor -- start 2024-04-26T21:15:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 -- start start 2024-04-26T21:15:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec680d3ee0 0x7fec680c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 0x7fec680b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec680b4950 con 0x7fec6815f750 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec680b47d0 con 0x7fec680d3ee0 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec706da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec680b4ad0 con 0x7fec6815f370 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec6ec77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec6ec77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59084/0 (socket says 172.21.15.40:59084) 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.915+0000 7fec6ec77640 1 -- 172.21.15.40:0/1140621097 learned_addr learned my addr 172.21.15.40:0/1140621097 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6e476640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec680d3ee0 0x7fec680c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6dc75640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 0x7fec680b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 -- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 msgr2=0x7fec680b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 0x7fec680b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 -- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec680d3ee0 msgr2=0x7fec680c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec680d3ee0 0x7fec680c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 -- 172.21.15.40:0/1140621097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec680c9620 con 0x7fec6815f750 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6dc75640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 0x7fec680b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec6ec77640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fec58009eb0 tx=0x7fec58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58040070 con 0x7fec6815f750 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58031c40 con 0x7fec6815f750 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec680c9900 con 0x7fec6815f750 2024-04-26T21:15:38.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58003990 con 0x7fec6815f750 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.916+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec680ca1a0 con 0x7fec6815f750 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.917+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec680c9ad0 con 0x7fec6815f750 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.918+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec58039040 con 0x7fec6815f750 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.918+0000 7fec66ffd640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 0x7fec5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.918+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fec580acd30 con 0x7fec6815f750 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.918+0000 7fec6e476640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 0x7fec5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:38.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.919+0000 7fec6e476640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 0x7fec5406edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec60007f70 tx=0x7fec60007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:38.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:38.922+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec5807aba0 con 0x7fec6815f750 2024-04-26T21:15:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.111+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec6806a290 con 0x7fec6815f750 2024-04-26T21:15:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.112+0000 7fec66ffd640 1 -- 172.21.15.40:0/1140621097 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16668 (secure 0 0 0) 0x7fec5807eea0 con 0x7fec6815f750 2024-04-26T21:15:39.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:39.113 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:15.986368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 msgr2=0x7fec5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 0x7fec5406edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec60007f70 tx=0x7fec60007f00 comp rx=0 tx=0).stop 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 msgr2=0x7fec680b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fec58009eb0 tx=0x7fec58004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 shutdown_connections 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5406c8f0 0x7fec5406edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6815f750 0x7fec680b7df0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6815f370 0x7fec680b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 --2- 172.21.15.40:0/1140621097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec680d3ee0 0x7fec680c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 >> 172.21.15.40:0/1140621097 conn(0x7fec6808ec10 msgr2=0x7fec6809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 shutdown_connections 2024-04-26T21:15:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.114+0000 7fec706da640 1 -- 172.21.15.40:0/1140621097 wait complete. 2024-04-26T21:15:39.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:12.615950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:13.297387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:39.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.228+0000 7ff16922a640 1 Processor -- start 2024-04-26T21:15:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.228+0000 7ff16922a640 1 -- start start 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16922a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff16415f680 0x7ff1640d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16922a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 0x7ff1640d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16922a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1640d58f0 con 0x7ff164163aa0 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16922a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff16415fa60 con 0x7ff16415f680 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16922a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1640d3f50 con 0x7ff1640d4f20 2024-04-26T21:15:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff162d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff16415f680 0x7ff1640d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff164163aa0 0x7ff1640b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39982/0 (socket says 172.21.15.40:39982) 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16259c640 1 -- 172.21.15.40:0/2861727667 learned_addr learned my addr 172.21.15.40:0/2861727667 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.229+0000 7ff16359e640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 0x7ff1640d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff162d9d640 1 -- 172.21.15.40:0/2861727667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 msgr2=0x7ff1640d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff162d9d640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 0x7ff1640d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff162d9d640 1 -- 172.21.15.40:0/2861727667 >> v1:172.21.15.40:6789/0 conn(0x7ff164163aa0 legacy=0x7ff1640b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff162d9d640 1 -- 172.21.15.40:0/2861727667 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1640b5090 con 0x7ff16415f680 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff161d9b640 1 -- 172.21.15.40:0/2861727667 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251187441 0 0) 0x7ff1640d58f0 con 0x7ff164163aa0 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff16359e640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 0x7ff1640d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff162d9d640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16415f680 0x7ff1640d3820 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097a0 tx=0x7ff15c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab1070fedcbf6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff161d9b640 1 -- 172.21.15.40:0/2861727667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c037070 con 0x7ff16415f680 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff161d9b640 1 -- 172.21.15.40:0/2861727667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c032440 con 0x7ff16415f680 2024-04-26T21:15:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.230+0000 7ff161d9b640 1 -- 172.21.15.40:0/2861727667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c032810 con 0x7ff16415f680 2024-04-26T21:15:39.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 -- 172.21.15.40:0/2861727667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16415f680 msgr2=0x7ff1640d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16415f680 0x7ff1640d3820 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097a0 tx=0x7ff15c02a180 comp rx=0 tx=0).stop 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 -- 172.21.15.40:0/2861727667 shutdown_connections 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1640d4f20 0x7ff1640d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 --2- 172.21.15.40:0/2861727667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16415f680 0x7ff1640d3820 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 -- 172.21.15.40:0/2861727667 >> 172.21.15.40:0/2861727667 conn(0x7ff16408eca0 msgr2=0x7ff16409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 -- 172.21.15.40:0/2861727667 shutdown_connections 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.231+0000 7ff16922a640 1 -- 172.21.15.40:0/2861727667 wait complete. 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.232+0000 7ff16922a640 1 Processor -- start 2024-04-26T21:15:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.232+0000 7ff16922a640 1 -- start start 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 0x7ff16415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 0x7ff16415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff164163f30 con 0x7ff1640d4f20 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640b6280 con 0x7ff16415f680 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16922a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1640cfd80 con 0x7ff164163aa0 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff162d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59106/0 (socket says 172.21.15.40:59106) 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff162d9d640 1 -- 172.21.15.40:0/1221356635 learned_addr learned my addr 172.21.15.40:0/1221356635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16359e640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 0x7ff16415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff162d9d640 1 -- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 msgr2=0x7ff16415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.233+0000 7ff16259c640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 0x7ff16415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff162d9d640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 0x7ff16415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff162d9d640 1 -- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 msgr2=0x7ff16415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff162d9d640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 0x7ff16415fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff162d9d640 1 -- 172.21.15.40:0/1221356635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164160380 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff16359e640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 0x7ff16415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff16259c640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 0x7ff16415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff162d9d640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0098c0 tx=0x7ff15c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c037040 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c030770 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff164160660 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c030b40 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.234+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff164160c20 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.235+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16406a290 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.235+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff15c044020 con 0x7ff1640d4f20 2024-04-26T21:15:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.236+0000 7ff15b7fe640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 0x7ff14406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.236+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7ff15c0ac130 con 0x7ff1640d4f20 2024-04-26T21:15:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.236+0000 7ff16259c640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 0x7ff14406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.237+0000 7ff16259c640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 0x7ff14406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff1640b6a00 tx=0x7ff14c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.240+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff15c032e20 con 0x7ff1640d4f20 2024-04-26T21:15:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.422+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1640b8f90 con 0x7ff1640d4f20 2024-04-26T21:15:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.423+0000 7ff15b7fe640 1 -- 172.21.15.40:0/1221356635 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7ff15c07a560 con 0x7ff1640d4f20 2024-04-26T21:15:39.424 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 msgr2=0x7ff14406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 0x7ff14406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff1640b6a00 tx=0x7ff14c009040 comp rx=0 tx=0).stop 2024-04-26T21:15:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 msgr2=0x7ff1640cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0098c0 tx=0x7ff15c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 shutdown_connections 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff14406c8f0 0x7ff14406edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff164163aa0 0x7ff16415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16415f680 0x7ff16415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 --2- 172.21.15.40:0/1221356635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1640d4f20 0x7ff1640cedb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 >> 172.21.15.40:0/1221356635 conn(0x7ff16408eca0 msgr2=0x7ff1640b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 shutdown_connections 2024-04-26T21:15:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.425+0000 7ff16922a640 1 -- 172.21.15.40:0/1221356635 wait complete. 2024-04-26T21:15:39.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:15:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.539+0000 7f89c2c9c640 1 Processor -- start 2024-04-26T21:15:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.539+0000 7f89c2c9c640 1 -- start start 2024-04-26T21:15:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0d0010 0x7f89bc06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89bc0b3c20 0x7f89bc0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89bc0b3210 0x7f89bc0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f89bc0b4480 con 0x7f89bc0b3210 2024-04-26T21:15:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f89bc06e050 con 0x7f89bc0b3c20 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c2c9c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89bc09c700 con 0x7f89bc0d0010 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89bc0b3210 0x7f89bc0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89bc0b3210 0x7f89bc0b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59116/0 (socket says 172.21.15.40:59116) 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 -- 172.21.15.40:0/2318754641 learned_addr learned my addr 172.21.15.40:0/2318754641 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c0a38640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0d0010 0x7f89bc06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 -- 172.21.15.40:0/2318754641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0d0010 msgr2=0x7f89bc06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89bbfff640 1 --2- 172.21.15.40:0/2318754641 >> v2:172.21.15.184:3300/0 conn(0x7f89bc0b3c20 0x7f89bc0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0d0010 0x7f89bc06db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 -- 172.21.15.40:0/2318754641 >> v2:172.21.15.184:3300/0 conn(0x7f89bc0b3c20 msgr2=0x7f89bc0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 --2- 172.21.15.40:0/2318754641 >> v2:172.21.15.184:3300/0 conn(0x7f89bc0b3c20 0x7f89bc0b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.540+0000 7f89c1239640 1 -- 172.21.15.40:0/2318754641 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc0d1c50 con 0x7f89bc0b3210 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.541+0000 7f89c1239640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc0b36a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f89bc073f10 tx=0x7f89b402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482c328efaa71af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.541+0000 7f89bb7fe640 1 -- 172.21.15.40:0/2318754641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4036070 con 0x7f89bc0b3210 2024-04-26T21:15:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.541+0000 7f89bb7fe640 1 -- 172.21.15.40:0/2318754641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b4031600 con 0x7f89bc0b3210 2024-04-26T21:15:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.541+0000 7f89bb7fe640 1 -- 172.21.15.40:0/2318754641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40319d0 con 0x7f89bc0b3210 2024-04-26T21:15:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 -- 172.21.15.40:0/2318754641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 msgr2=0x7f89bc0b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc0b36a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f89bc073f10 tx=0x7f89b402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 -- 172.21.15.40:0/2318754641 shutdown_connections 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc0b36a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/2318754641 >> v2:172.21.15.184:3300/0 conn(0x7f89bc0b3c20 0x7f89bc0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/2318754641 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0d0010 0x7f89bc06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 -- 172.21.15.40:0/2318754641 >> 172.21.15.40:0/2318754641 conn(0x7f89bc09ba40 msgr2=0x7f89bc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 -- 172.21.15.40:0/2318754641 shutdown_connections 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.542+0000 7f89c2c9c640 1 -- 172.21.15.40:0/2318754641 wait complete. 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.543+0000 7f89c2c9c640 1 Processor -- start 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.543+0000 7f89c2c9c640 1 -- start start 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0b3c20 0x7f89bc0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89bc0d0010 0x7f89bc0b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc15fe90 con 0x7f89bc0b3210 2024-04-26T21:15:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc15fd10 con 0x7f89bc0d0010 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c2c9c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89bc160010 con 0x7f89bc0b3c20 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59122/0 (socket says 172.21.15.40:59122) 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 -- 172.21.15.40:0/627870671 learned_addr learned my addr 172.21.15.40:0/627870671 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89bbfff640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0b3c20 0x7f89bc0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 -- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0b3c20 msgr2=0x7f89bc0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0b3c20 0x7f89bc0b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 -- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89bc0d0010 msgr2=0x7f89bc0b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c1239640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89bc0d0010 0x7f89bc0b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89bc0d0010 0x7f89bc0b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.544+0000 7f89c0a38640 1 -- 172.21.15.40:0/627870671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc154910 con 0x7f89bc0b3210 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89c0a38640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c7f0 tx=0x7f89ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac015070 con 0x7f89bc0b3210 2024-04-26T21:15:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac004220 con 0x7f89bc0b3210 2024-04-26T21:15:39.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc0b52b0 con 0x7f89bc0b3210 2024-04-26T21:15:39.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac007620 con 0x7f89bc0b3210 2024-04-26T21:15:39.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.545+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc0b5870 con 0x7f89bc0b3210 2024-04-26T21:15:39.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.546+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89bc06a290 con 0x7f89bc0b3210 2024-04-26T21:15:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.546+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89ac0079d0 con 0x7f89bc0b3210 2024-04-26T21:15:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.547+0000 7f89b97fa640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 0x7f89a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.547+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f89ac08ba60 con 0x7f89bc0b3210 2024-04-26T21:15:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.547+0000 7f89bbfff640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 0x7f89a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.547+0000 7f89bbfff640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 0x7f89a806ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89b0006fd0 tx=0x7f89b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.551+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89ac059e90 con 0x7f89bc0b3210 2024-04-26T21:15:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.736+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89bc068e00 con 0x7f89bc0b3210 2024-04-26T21:15:39.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.736+0000 7f89b97fa640 1 -- 172.21.15.40:0/627870671 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f89ac059790 con 0x7f89bc0b3210 2024-04-26T21:15:39.738 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:15:39.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 msgr2=0x7f89a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 0x7f89a806ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89b0006fd0 tx=0x7f89b0008040 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 msgr2=0x7f89bc1543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c7f0 tx=0x7f89ac00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 shutdown_connections 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a806c920 0x7f89a806ede0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89bc0d0010 0x7f89bc0b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89bc0b3c20 0x7f89bc0b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 --2- 172.21.15.40:0/627870671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89bc0b3210 0x7f89bc1543d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 >> 172.21.15.40:0/627870671 conn(0x7f89bc09ba40 msgr2=0x7f89bc070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 shutdown_connections 2024-04-26T21:15:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.738+0000 7f89c2c9c640 1 -- 172.21.15.40:0/627870671 wait complete. 2024-04-26T21:15:39.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:15:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 Processor -- start 2024-04-26T21:15:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 -- start start 2024-04-26T21:15:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 0x7f41f80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41f80d4f20 0x7f41f80d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f80d58f0 con 0x7f41f8163aa0 2024-04-26T21:15:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f41f815fa60 con 0x7f41f80d4f20 2024-04-26T21:15:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.854+0000 7f41fd8f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80d3f50 con 0x7f41f815f680 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f6ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 0x7f41f80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41effff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f41f8163aa0 0x7f41f80b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48002/0 (socket says 172.21.15.40:48002) 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41effff640 1 -- 172.21.15.40:0/67560161 learned_addr learned my addr 172.21.15.40:0/67560161 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f77fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41f80d4f20 0x7f41f80d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f67fc640 1 -- 172.21.15.40:0/67560161 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803538413 0 0) 0x7f41f80d58f0 con 0x7f41f8163aa0 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f67fc640 1 -- 172.21.15.40:0/67560161 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e4003520 con 0x7f41f8163aa0 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f77fe640 1 -- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 msgr2=0x7f41f80d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f77fe640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 0x7f41f80d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f77fe640 1 -- 172.21.15.40:0/67560161 >> v1:172.21.15.184:6789/0 conn(0x7f41f8163aa0 legacy=0x7f41f80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f77fe640 1 -- 172.21.15.40:0/67560161 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80b5090 con 0x7f41f80d4f20 2024-04-26T21:15:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.855+0000 7f41f6ffd640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 0x7f41f80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.856+0000 7f41f77fe640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f80d4f20 0x7f41f80d53b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69cad7a760fec1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.856+0000 7f41f67fc640 1 -- 172.21.15.40:0/67560161 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007740 con 0x7f41f80d4f20 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.856+0000 7f41f67fc640 1 -- 172.21.15.40:0/67560161 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f00078e0 con 0x7f41f80d4f20 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.856+0000 7f41f67fc640 1 -- 172.21.15.40:0/67560161 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007c90 con 0x7f41f80d4f20 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 -- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f80d4f20 msgr2=0x7f41f80d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f80d4f20 0x7f41f80d53b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f002a660 comp rx=0 tx=0).stop 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 -- 172.21.15.40:0/67560161 shutdown_connections 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f80d4f20 0x7f41f80d53b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/67560161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f815f680 0x7f41f80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 -- 172.21.15.40:0/67560161 >> 172.21.15.40:0/67560161 conn(0x7f41f808eca0 msgr2=0x7f41f809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 -- 172.21.15.40:0/67560161 shutdown_connections 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.857+0000 7f41fd8f8640 1 -- 172.21.15.40:0/67560161 wait complete. 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.858+0000 7f41fd8f8640 1 Processor -- start 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.858+0000 7f41fd8f8640 1 -- start start 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f80d4f20 0x7f41f815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f815f680 0x7f41f815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8163f00 con 0x7f41f8163aa0 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80b6280 con 0x7f41f815f680 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41fd8f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80cfd80 con 0x7f41f80d4f20 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59130/0 (socket says 172.21.15.40:59130) 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 -- 172.21.15.40:0/1160041117 learned_addr learned my addr 172.21.15.40:0/1160041117 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f6ffd640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f80d4f20 0x7f41f815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41effff640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f815f680 0x7f41f815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 -- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f80d4f20 msgr2=0x7f41f815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f80d4f20 0x7f41f815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 -- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f815f680 msgr2=0x7f41f815fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f815f680 0x7f41f815fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 -- 172.21.15.40:0/1160041117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81f5e90 con 0x7f41f8163aa0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.859+0000 7f41f77fe640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009eb0 tx=0x7f41f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.860+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0009450 con 0x7f41f8163aa0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.860+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0004240 con 0x7f41f8163aa0 2024-04-26T21:15:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.860+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81f58d0 con 0x7f41f8163aa0 2024-04-26T21:15:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.860+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004630 con 0x7f41f8163aa0 2024-04-26T21:15:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.860+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f80b8b80 con 0x7f41f8163aa0 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.861+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f806a290 con 0x7f41f8163aa0 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.861+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41f0039040 con 0x7f41f8163aa0 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.861+0000 7f41f4ff9640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 0x7f41d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.862+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f41f00ac9a0 con 0x7f41f8163aa0 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.862+0000 7f41f6ffd640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 0x7f41d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.862+0000 7f41f6ffd640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 0x7f41d806ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41f80b4aa0 tx=0x7f41e8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:39.866+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41f0040030 con 0x7f41f8163aa0 2024-04-26T21:15:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.050+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41f80b8950 con 0x7f41f8163aa0 2024-04-26T21:15:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.051+0000 7f41f4ff9640 1 -- 172.21.15.40:0/1160041117 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f41f007a8d0 con 0x7f41f8163aa0 2024-04-26T21:15:40.052 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 msgr2=0x7f41d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 0x7f41d806ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41f80b4aa0 tx=0x7f41e8009190 comp rx=0 tx=0).stop 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 msgr2=0x7f41f81f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009eb0 tx=0x7f41f0004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 shutdown_connections 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d806c920 0x7f41d806ede0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41f8163aa0 0x7f41f81f4ec0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41f815f680 0x7f41f815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 --2- 172.21.15.40:0/1160041117 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41f80d4f20 0x7f41f815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 >> 172.21.15.40:0/1160041117 conn(0x7f41f808eca0 msgr2=0x7f41f815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 shutdown_connections 2024-04-26T21:15:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.053+0000 7f41fd8f8640 1 -- 172.21.15.40:0/1160041117 wait complete. 2024-04-26T21:15:40.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:15:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.164+0000 7f9467474640 1 Processor -- start 2024-04-26T21:15:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 -- start start 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f370 0x7f94600cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f946015f750 0x7f94600b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94600d37d0 0x7f94600d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94600b4d10 con 0x7f946015f750 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f94600d4120 con 0x7f94600d37d0 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.165+0000 7f9467474640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94600d03a0 con 0x7f946015f370 2024-04-26T21:15:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f946015f750 0x7f94600b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f946015f750 0x7f94600b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59134/0 (socket says 172.21.15.40:59134) 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 -- 172.21.15.40:0/592596513 learned_addr learned my addr 172.21.15.40:0/592596513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9465a11640 1 --2- 172.21.15.40:0/592596513 >> v2:172.21.15.184:3300/0 conn(0x7f94600d37d0 0x7f94600d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9465210640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f370 0x7f94600cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 -- 172.21.15.40:0/592596513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f370 msgr2=0x7f94600cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f370 0x7f94600cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 -- 172.21.15.40:0/592596513 >> v2:172.21.15.184:3300/0 conn(0x7f94600d37d0 msgr2=0x7f94600d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 --2- 172.21.15.40:0/592596513 >> v2:172.21.15.184:3300/0 conn(0x7f94600d37d0 0x7f94600d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.166+0000 7f9464a0f640 1 -- 172.21.15.40:0/592596513 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94600d0680 con 0x7f946015f750 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9464a0f640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f750 0x7f94600b47d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f94600d02b0 tx=0x7f945002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2555c8f3b311c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9457fff640 1 -- 172.21.15.40:0/592596513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450037070 con 0x7f946015f750 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9457fff640 1 -- 172.21.15.40:0/592596513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450032440 con 0x7f946015f750 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9457fff640 1 -- 172.21.15.40:0/592596513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450032810 con 0x7f946015f750 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 -- 172.21.15.40:0/592596513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f750 msgr2=0x7f94600b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f750 0x7f94600b47d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f94600d02b0 tx=0x7f945002a140 comp rx=0 tx=0).stop 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 -- 172.21.15.40:0/592596513 shutdown_connections 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 --2- 172.21.15.40:0/592596513 >> v2:172.21.15.184:3300/0 conn(0x7f94600d37d0 0x7f94600d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f750 0x7f94600b47d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 --2- 172.21.15.40:0/592596513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f370 0x7f94600cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 -- 172.21.15.40:0/592596513 >> 172.21.15.40:0/592596513 conn(0x7f946008ec10 msgr2=0x7f946009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 -- 172.21.15.40:0/592596513 shutdown_connections 2024-04-26T21:15:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.167+0000 7f9467474640 1 -- 172.21.15.40:0/592596513 wait complete. 2024-04-26T21:15:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 Processor -- start 2024-04-26T21:15:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 -- start start 2024-04-26T21:15:40.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 0x7f9460154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 0x7f94600b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94600d0f20 con 0x7f946015f370 2024-04-26T21:15:40.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94600d0c40 con 0x7f94600d37d0 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9467474640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f94600d1bd0 con 0x7f946015f750 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.169+0000 7f9464a0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59138/0 (socket says 172.21.15.40:59138) 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 -- 172.21.15.40:0/1868745060 learned_addr learned my addr 172.21.15.40:0/1868745060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9465a11640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 0x7f94600b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9465210640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 0x7f9460154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 -- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 msgr2=0x7f94600b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 0x7f94600b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 -- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 msgr2=0x7f9460154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 0x7f9460154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 -- 172.21.15.40:0/1868745060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601554c0 con 0x7f946015f370 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9465a11640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 0x7f94600b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9465210640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 0x7f9460154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9464a0f640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f94500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450037040 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450042070 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450030bb0 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601622b0 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.170+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9460162870 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.171+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946006a290 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.172+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9450047460 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.172+0000 7f9455ffb640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 0x7f944406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.172+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f94500ac770 con 0x7f946015f370 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.173+0000 7f9465210640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 0x7f944406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.173+0000 7f9465210640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 0x7f944406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f945c008890 tx=0x7f945c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.177+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f945007aba0 con 0x7f946015f370 2024-04-26T21:15:40.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.374+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94600c8820 con 0x7f946015f370 2024-04-26T21:15:40.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.375+0000 7f9455ffb640 1 -- 172.21.15.40:0/1868745060 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f945007a4a0 con 0x7f946015f370 2024-04-26T21:15:40.376 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:15:40.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 msgr2=0x7f944406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 0x7f944406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f945c008890 tx=0x7f945c008040 comp rx=0 tx=0).stop 2024-04-26T21:15:40.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 msgr2=0x7f94600b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f94500046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 shutdown_connections 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f944406c8f0 0x7f944406edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f946015f750 0x7f94600b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f946015f370 0x7f94600b4060 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 --2- 172.21.15.40:0/1868745060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94600d37d0 0x7f9460154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 >> 172.21.15.40:0/1868745060 conn(0x7f946008ec10 msgr2=0x7f946015e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 shutdown_connections 2024-04-26T21:15:40.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.377+0000 7f9467474640 1 -- 172.21.15.40:0/1868745060 wait complete. 2024-04-26T21:15:40.388 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:15:40.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:40.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:15:40.420 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:15:40.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:15:40.421 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:15:40.478 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:15:40.478 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:40.478 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:40.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:15:40.494 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:15:40.494 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:40.494 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:40.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:15:40.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.635+0000 7f4abb47e640 1 Processor -- start 2024-04-26T21:15:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.635+0000 7f4abb47e640 1 -- start start 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4abb47e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4abb47e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ab40b4200 0x7f4ab4163950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4abb47e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab40aed10 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4abb47e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ab4145d50 con 0x7f4ab40b4200 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4abb47e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab40cfaf0 con 0x7f4ab4145970 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4ab9a1b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ab40b4200 0x7f4ab4163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4ab9a1b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ab40b4200 0x7f4ab4163950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33876/0 (socket says 172.21.15.40:33876) 2024-04-26T21:15:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.636+0000 7f4ab9a1b640 1 -- 172.21.15.40:0/866296331 learned_addr learned my addr 172.21.15.40:0/866296331 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab921a640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab9a1b640 1 -- 172.21.15.40:0/866296331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 msgr2=0x7f4ab4073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab9a1b640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab4073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab9a1b640 1 -- 172.21.15.40:0/866296331 >> v1:172.21.15.184:6789/0 conn(0x7f4ab40ae8b0 legacy=0x7f4ab40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab9a1b640 1 -- 172.21.15.40:0/866296331 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab40cfdd0 con 0x7f4ab40b4200 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab921a640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.637+0000 7f4ab9a1b640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40b4200 0x7f4ab4163950 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009e50 tx=0x7f4aa402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8e2810ab9a0ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4ab3fff640 1 -- 172.21.15.40:0/866296331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4036070 con 0x7f4ab40b4200 2024-04-26T21:15:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4ab3fff640 1 -- 172.21.15.40:0/866296331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa4031600 con 0x7f4ab40b4200 2024-04-26T21:15:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4ab3fff640 1 -- 172.21.15.40:0/866296331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40319d0 con 0x7f4ab40b4200 2024-04-26T21:15:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 -- 172.21.15.40:0/866296331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40b4200 msgr2=0x7f4ab4163950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40b4200 0x7f4ab4163950 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009e50 tx=0x7f4aa402a6d0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 -- 172.21.15.40:0/866296331 shutdown_connections 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40b4200 0x7f4ab4163950 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 --2- 172.21.15.40:0/866296331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 -- 172.21.15.40:0/866296331 >> 172.21.15.40:0/866296331 conn(0x7f4ab40ce8b0 msgr2=0x7f4ab40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 -- 172.21.15.40:0/866296331 shutdown_connections 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.638+0000 7f4abb47e640 1 -- 172.21.15.40:0/866296331 wait complete. 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.639+0000 7f4abb47e640 1 Processor -- start 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.639+0000 7f4abb47e640 1 -- start start 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 0x7f4ab40c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab406f8f0 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab406f3d0 con 0x7f4ab40b4200 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4abb47e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab406fa70 con 0x7f4ab4145970 2024-04-26T21:15:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4ab921a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4ab921a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33888/0 (socket says 172.21.15.40:33888) 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4ab921a640 1 -- 172.21.15.40:0/3271913698 learned_addr learned my addr 172.21.15.40:0/3271913698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.640+0000 7f4ab9a1b640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab8a19640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 0x7f4ab40c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 -- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 msgr2=0x7f4ab415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 -- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 msgr2=0x7f4ab40c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 0x7f4ab40c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 -- 172.21.15.40:0/3271913698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab41f8b90 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab9a1b640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab8a19640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 0x7f4ab40c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab921a640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c7f0 tx=0x7f4aac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac015070 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aac007820 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab41f8e70 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac007bd0 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.641+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41f9950 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.642+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab406a290 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.643+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4aac018070 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.643+0000 7f4ab1ffb640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.643+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f4aac08bc30 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.643+0000 7f4ab8a19640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.644+0000 7f4ab8a19640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 0x7f4aa006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40b0fc0 tx=0x7f4aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.647+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4aac05a0f0 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.870+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ab4073980 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.871+0000 7f4ab1ffb640 1 -- 172.21.15.40:0/3271913698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7f4aac0599f0 con 0x7f4ab40ae8b0 2024-04-26T21:15:40.872 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:40.872 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:40.872 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 msgr2=0x7f4aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 0x7f4aa006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40b0fc0 tx=0x7f4aa8009040 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 msgr2=0x7f4ab40afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c7f0 tx=0x7f4aac00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 shutdown_connections 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4aa006c8f0 0x7f4aa006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ab4145970 0x7f4ab415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ab40b4200 0x7f4ab40c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 --2- 172.21.15.40:0/3271913698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab40ae8b0 0x7f4ab40afcc0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 >> 172.21.15.40:0/3271913698 conn(0x7f4ab40ce8b0 msgr2=0x7f4ab4144530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 shutdown_connections 2024-04-26T21:15:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.873+0000 7f4abb47e640 1 -- 172.21.15.40:0/3271913698 wait complete. 2024-04-26T21:15:40.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.984+0000 7fc9a7e93640 1 Processor -- start 2024-04-26T21:15:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.984+0000 7fc9a7e93640 1 -- start start 2024-04-26T21:15:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a7e93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9a015f340 0x7fc9a00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a7e93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9a00d5610 0x7fc9a00d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a7e93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a00b2510 con 0x7fc9a015f720 2024-04-26T21:15:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a7e93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a00b27f0 con 0x7fc9a00d5610 2024-04-26T21:15:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a7e93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a00b2ad0 con 0x7fc9a015f340 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a6430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9a00d5610 0x7fc9a00d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a6430640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9a00d5610 0x7fc9a00d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33898/0 (socket says 172.21.15.40:33898) 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a542e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc9a015f720 0x7fc9a00d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51328/0 (socket says 172.21.15.40:51328) 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a6430640 1 -- 172.21.15.40:0/819954387 learned_addr learned my addr 172.21.15.40:0/819954387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.985+0000 7fc9a5c2f640 1 --2- 172.21.15.40:0/819954387 >> v2:172.21.15.184:3300/0 conn(0x7fc9a015f340 0x7fc9a00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a6430640 1 -- 172.21.15.40:0/819954387 >> v2:172.21.15.184:3300/0 conn(0x7fc9a015f340 msgr2=0x7fc9a00b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a6430640 1 --2- 172.21.15.40:0/819954387 >> v2:172.21.15.184:3300/0 conn(0x7fc9a015f340 0x7fc9a00b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a6430640 1 -- 172.21.15.40:0/819954387 >> v1:172.21.15.40:6789/0 conn(0x7fc9a015f720 legacy=0x7fc9a00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a6430640 1 -- 172.21.15.40:0/819954387 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a00b2db0 con 0x7fc9a00d5610 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a4c2d640 1 -- 172.21.15.40:0/819954387 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404187871 0 0) 0x7fc9a00b2510 con 0x7fc9a015f720 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a6430640 1 --2- 172.21.15.40:0/819954387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a00d5610 0x7fc9a00d3d40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00d6000 tx=0x7fc99802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90addbb5c9b2048f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a4c2d640 1 -- 172.21.15.40:0/819954387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998036070 con 0x7fc9a00d5610 2024-04-26T21:15:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a4c2d640 1 -- 172.21.15.40:0/819954387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc998031600 con 0x7fc9a00d5610 2024-04-26T21:15:40.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.986+0000 7fc9a4c2d640 1 -- 172.21.15.40:0/819954387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980319d0 con 0x7fc9a00d5610 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 -- 172.21.15.40:0/819954387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a00d5610 msgr2=0x7fc9a00d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/819954387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a00d5610 0x7fc9a00d3d40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00d6000 tx=0x7fc99802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 -- 172.21.15.40:0/819954387 shutdown_connections 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/819954387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a00d5610 0x7fc9a00d3d40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/819954387 >> v2:172.21.15.184:3300/0 conn(0x7fc9a015f340 0x7fc9a00b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 -- 172.21.15.40:0/819954387 >> 172.21.15.40:0/819954387 conn(0x7fc9a008ec70 msgr2=0x7fc9a009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 -- 172.21.15.40:0/819954387 shutdown_connections 2024-04-26T21:15:40.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.987+0000 7fc9a7e93640 1 -- 172.21.15.40:0/819954387 wait complete. 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.988+0000 7fc9a7e93640 1 Processor -- start 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.988+0000 7fc9a7e93640 1 -- start start 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a00d5610 0x7fc9a015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 0x7fc9a00b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a00b3940 con 0x7fc9a015f720 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a00b37c0 con 0x7fc9a00d5610 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a7e93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a00b3ac0 con 0x7fc9a015f340 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33900/0 (socket says 172.21.15.40:33900) 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 -- 172.21.15.40:0/2218804679 learned_addr learned my addr 172.21.15.40:0/2218804679 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a542e640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 0x7fc9a00b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a5c2f640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a00d5610 0x7fc9a015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 -- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 msgr2=0x7fc9a00b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 0x7fc9a00b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 -- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a00d5610 msgr2=0x7fc9a015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a00d5610 0x7fc9a015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.989+0000 7fc9a6430640 1 -- 172.21.15.40:0/2218804679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a00b89b0 con 0x7fc9a015f720 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9a542e640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 0x7fc9a00b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9a6430640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc998034040 tx=0x7fc998004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998041020 con 0x7fc9a015f720 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc998037040 con 0x7fc9a015f720 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a00c95a0 con 0x7fc9a015f720 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998036050 con 0x7fc9a015f720 2024-04-26T21:15:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.990+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a00c9b60 con 0x7fc9a015f720 2024-04-26T21:15:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.991+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a00ca050 con 0x7fc9a015f720 2024-04-26T21:15:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.991+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc998007c30 con 0x7fc9a015f720 2024-04-26T21:15:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.992+0000 7fc9967fc640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.992+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc9980ac2c0 con 0x7fc9a015f720 2024-04-26T21:15:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.992+0000 7fc9a5c2f640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.993+0000 7fc9a5c2f640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:40.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:40.996+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc99804b060 con 0x7fc9a015f720 2024-04-26T21:15:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.182+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9a008f320 con 0x7fc9a015f720 2024-04-26T21:15:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.183+0000 7fc9967fc640 1 -- 172.21.15.40:0/2218804679 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16668 (secure 0 0 0) 0x7fc9a008f320 con 0x7fc9a015f720 2024-04-26T21:15:41.184 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":70,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:15.986368+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.185+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 msgr2=0x7fc98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.185+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c008040 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.185+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 msgr2=0x7fc9a00b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.185+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc998034040 tx=0x7fc998004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 shutdown_connections 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9a015f720 0x7fc9a00b8380 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9a015f340 0x7fc9a00b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 --2- 172.21.15.40:0/2218804679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a00d5610 0x7fc9a015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 >> 172.21.15.40:0/2218804679 conn(0x7fc9a008ec70 msgr2=0x7fc9a00d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 shutdown_connections 2024-04-26T21:15:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.186+0000 7fc9a7e93640 1 -- 172.21.15.40:0/2218804679 wait complete. 2024-04-26T21:15:41.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:12.615950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:13.297387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:41.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.304+0000 7f926b0f1640 1 Processor -- start 2024-04-26T21:15:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.304+0000 7f926b0f1640 1 -- start start 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926b0f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92640d0090 0x7f92640ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926b0f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92640d07e0 con 0x7f92640b1ef0 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926b0f1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92640d2090 con 0x7f9264160950 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926b0f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f926409c700 con 0x7f92640d0090 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f9263fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92640d0090 0x7f92640ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926968e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f92640b1ef0 0x7f92640ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51330/0 (socket says 172.21.15.40:51330) 2024-04-26T21:15:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f926968e640 1 -- 172.21.15.40:0/2564001719 learned_addr learned my addr 172.21.15.40:0/2564001719 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:41.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f9263fff640 1 -- 172.21.15.40:0/2564001719 >> v1:172.21.15.184:6789/0 conn(0x7f9264160950 legacy=0x7f92640cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f9263fff640 1 -- 172.21.15.40:0/2564001719 >> v1:172.21.15.40:6789/0 conn(0x7f92640b1ef0 legacy=0x7f92640ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f9263fff640 1 -- 172.21.15.40:0/2564001719 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92640d1a40 con 0x7f92640d0090 2024-04-26T21:15:41.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.305+0000 7f92637fe640 1 -- 172.21.15.40:0/2564001719 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711769536 0 0) 0x7f92640d07e0 con 0x7f92640b1ef0 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f9263fff640 1 --2- 172.21.15.40:0/2564001719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640ce030 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f925002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff498b76701689 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f92637fe640 1 -- 172.21.15.40:0/2564001719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250037070 con 0x7f92640d0090 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f92637fe640 1 -- 172.21.15.40:0/2564001719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9250032440 con 0x7f92640d0090 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f92637fe640 1 -- 172.21.15.40:0/2564001719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250032810 con 0x7f92640d0090 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f926b0f1640 1 -- 172.21.15.40:0/2564001719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 msgr2=0x7f92640ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f926b0f1640 1 --2- 172.21.15.40:0/2564001719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640ce030 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f925002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.306+0000 7f926b0f1640 1 -- 172.21.15.40:0/2564001719 shutdown_connections 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.307+0000 7f926b0f1640 1 --2- 172.21.15.40:0/2564001719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640ce030 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.307+0000 7f926b0f1640 1 -- 172.21.15.40:0/2564001719 >> 172.21.15.40:0/2564001719 conn(0x7f926409ba40 msgr2=0x7f926409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.307+0000 7f926b0f1640 1 -- 172.21.15.40:0/2564001719 shutdown_connections 2024-04-26T21:15:41.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.307+0000 7f926b0f1640 1 -- 172.21.15.40:0/2564001719 wait complete. 2024-04-26T21:15:41.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.308+0000 7f926b0f1640 1 Processor -- start 2024-04-26T21:15:41.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.308+0000 7f926b0f1640 1 -- start start 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 0x7f92640ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 0x7f92640c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92640b28f0 con 0x7f92640d0090 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9264160d30 con 0x7f92640b1ef0 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92640b2a70 con 0x7f9264160950 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33924/0 (socket says 172.21.15.40:33924) 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 -- 172.21.15.40:0/3708270652 learned_addr learned my addr 172.21.15.40:0/3708270652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926968e640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 0x7f92640c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9268e8d640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 0x7f92640ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 -- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 msgr2=0x7f92640c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 0x7f92640c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 -- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 msgr2=0x7f92640ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 0x7f92640ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 -- 172.21.15.40:0/3708270652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92640c80d0 con 0x7f92640d0090 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926968e640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 0x7f92640c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9268e8d640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 0x7f92640ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f9263fff640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f92500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250041070 con 0x7f92640d0090 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925003c870 con 0x7f92640d0090 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250037040 con 0x7f92640d0090 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92640c83b0 con 0x7f92640d0090 2024-04-26T21:15:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.309+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926406da10 con 0x7f92640d0090 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.311+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926406d560 con 0x7f92640d0090 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.311+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9250044070 con 0x7f92640d0090 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.311+0000 7f92617fa640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.311+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f92500ac640 con 0x7f92640d0090 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.312+0000 7f9268e8d640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.312+0000 7f9268e8d640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 0x7f924806ede0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9258001750 tx=0x7f9258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.316+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f925007abe0 con 0x7f92640d0090 2024-04-26T21:15:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.500+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f926406a290 con 0x7f92640d0090 2024-04-26T21:15:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.500+0000 7f92617fa640 1 -- 172.21.15.40:0/3708270652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f925007a4e0 con 0x7f92640d0090 2024-04-26T21:15:41.501 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 msgr2=0x7f924806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 0x7f924806ede0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9258001750 tx=0x7f9258009040 comp rx=0 tx=0).stop 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 msgr2=0x7f92640c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f92500046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 shutdown_connections 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9264160950 0x7f92640c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92640d0090 0x7f92640c7560 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 --2- 172.21.15.40:0/3708270652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92640b1ef0 0x7f92640ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 >> 172.21.15.40:0/3708270652 conn(0x7f926409ba40 msgr2=0x7f92640d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 shutdown_connections 2024-04-26T21:15:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.502+0000 7f926b0f1640 1 -- 172.21.15.40:0/3708270652 wait complete. 2024-04-26T21:15:41.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:15:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.616+0000 7f2d2f652640 1 Processor -- start 2024-04-26T21:15:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.616+0000 7f2d2f652640 1 -- start start 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2f652640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d280b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2f652640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d280b55c0 0x7f2d280ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2f652640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2815dca0 con 0x7f2d280b4520 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2f652640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2d28143c20 con 0x7f2d280b55c0 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2f652640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2815df80 con 0x7f2d28143840 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2cbed640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d280b4520 0x7f2d280b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51332/0 (socket says 172.21.15.40:51332) 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.617+0000 7f2d2cbed640 1 -- 172.21.15.40:0/3889018558 learned_addr learned my addr 172.21.15.40:0/3889018558 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2dbef640 1 --2- 172.21.15.40:0/3889018558 >> v2:172.21.15.184:3300/0 conn(0x7f2d280b55c0 0x7f2d280ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2539907514 0 0) 0x7f2d2815dca0 con 0x7f2d280b4520 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d280b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d14003520 con 0x7f2d280b4520 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858674723 0 0) 0x7f2d14003520 con 0x7f2d280b4520 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 msgr2=0x7f2d280b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d280b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 >> v2:172.21.15.184:3300/0 conn(0x7f2d280b55c0 msgr2=0x7f2d280ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 --2- 172.21.15.40:0/3889018558 >> v2:172.21.15.184:3300/0 conn(0x7f2d280b55c0 0x7f2d280ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2815e260 con 0x7f2d280b4520 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f2d18001670 con 0x7f2d280b4520 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 >> v1:172.21.15.40:6789/0 conn(0x7f2d280b4520 legacy=0x7f2d280b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d14004800 0x7f2d14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28143c20 con 0x7f2d140039b0 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2815df80 con 0x7f2d14004800 2024-04-26T21:15:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1400b9c0 con 0x7f2d28143840 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2cbed640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d14004800 0x7f2d14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2dbef640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 msgr2=0x7f2d140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d28143840 0x7f2d140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d14004800 msgr2=0x7f2d14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d14004800 0x7f2d14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.618+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/3889018558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d14003520 con 0x7f2d140039b0 2024-04-26T21:15:41.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.619+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d14003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009ac0 tx=0x7f2d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f654a7f5066ebaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.619+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24036070 con 0x7f2d140039b0 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.619+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24031400 con 0x7f2d140039b0 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.619+0000 7f2d1ffff640 1 -- 172.21.15.40:0/3889018558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24009690 con 0x7f2d140039b0 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 -- 172.21.15.40:0/3889018558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 msgr2=0x7f2d14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 --2- 172.21.15.40:0/3889018558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d14003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009ac0 tx=0x7f2d14005830 comp rx=0 tx=0).stop 2024-04-26T21:15:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2dbef640 1 -- 172.21.15.40:0/3889018558 reap_dead start 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 -- 172.21.15.40:0/3889018558 shutdown_connections 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 -- 172.21.15.40:0/3889018558 >> 172.21.15.40:0/3889018558 conn(0x7f2d280cd8a0 msgr2=0x7f2d280cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 -- 172.21.15.40:0/3889018558 shutdown_connections 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.620+0000 7f2d2f652640 1 -- 172.21.15.40:0/3889018558 wait complete. 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.621+0000 7f2d2f652640 1 Processor -- start 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.621+0000 7f2d2f652640 1 -- start start 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d280cb6e0 0x7f2d281f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 0x7f2d28145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2815da40 con 0x7f2d140039b0 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2815d8c0 con 0x7f2d280cb6e0 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2f652640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280b3fe0 con 0x7f2d280c6b10 2024-04-26T21:15:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33932/0 (socket says 172.21.15.40:33932) 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/4283442774 learned_addr learned my addr 172.21.15.40:0/4283442774 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2dbef640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 0x7f2d28145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2cbed640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d280cb6e0 0x7f2d281f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 msgr2=0x7f2d28145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 0x7f2d28145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d280cb6e0 msgr2=0x7f2d281f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d280cb6e0 0x7f2d281f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2d3ee640 1 -- 172.21.15.40:0/4283442774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2815b020 con 0x7f2d140039b0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.622+0000 7f2d2dbef640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 0x7f2d28145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d2d3ee640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009ac0 tx=0x7f2d24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24040020 con 0x7f2d140039b0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24003bc0 con 0x7f2d140039b0 2024-04-26T21:15:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d28133690 con 0x7f2d140039b0 2024-04-26T21:15:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24036040 con 0x7f2d140039b0 2024-04-26T21:15:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.623+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d28133c50 con 0x7f2d140039b0 2024-04-26T21:15:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.624+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2806a290 con 0x7f2d140039b0 2024-04-26T21:15:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.624+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d24043020 con 0x7f2d140039b0 2024-04-26T21:15:41.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.625+0000 7f2d1dffb640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 0x7f2d0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.625+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f2d240abc10 con 0x7f2d140039b0 2024-04-26T21:15:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.625+0000 7f2d2cbed640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 0x7f2d0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.626+0000 7f2d2cbed640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 0x7f2d0c06ede0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cbae0 tx=0x7f2d18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.629+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d2404a070 con 0x7f2d140039b0 2024-04-26T21:15:41.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.812+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d280ce420 con 0x7f2d140039b0 2024-04-26T21:15:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.813+0000 7f2d1dffb640 1 -- 172.21.15.40:0/4283442774 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f2d2815b020 con 0x7f2d140039b0 2024-04-26T21:15:41.814 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.814+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 msgr2=0x7f2d0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.814+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 0x7f2d0c06ede0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cbae0 tx=0x7f2d18009040 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.814+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 msgr2=0x7f2d2815aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.814+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009ac0 tx=0x7f2d24004730 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 shutdown_connections 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d0c06c920 0x7f2d0c06ede0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d280c6b10 0x7f2d28145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d280cb6e0 0x7f2d281f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 --2- 172.21.15.40:0/4283442774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d140039b0 0x7f2d2815aae0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 >> 172.21.15.40:0/4283442774 conn(0x7f2d280cd8a0 msgr2=0x7f2d2815fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 shutdown_connections 2024-04-26T21:15:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.815+0000 7f2d2f652640 1 -- 172.21.15.40:0/4283442774 wait complete. 2024-04-26T21:15:41.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:15:41.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.929+0000 7f33b7c08640 1 Processor -- start 2024-04-26T21:15:41.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.929+0000 7f33b7c08640 1 -- start start 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b015f260 0x7f33b006d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f33b0070560 0x7f33b00b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b006ebe0 0x7f33b00b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f33b015f640 con 0x7f33b006ebe0 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f33b006c2a0 con 0x7f33b0070560 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b7c08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f33b006c580 con 0x7f33b015f260 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b006ebe0 0x7f33b00b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b006ebe0 0x7f33b00b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33942/0 (socket says 172.21.15.40:33942) 2024-04-26T21:15:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 -- 172.21.15.40:0/2291502764 learned_addr learned my addr 172.21.15.40:0/2291502764 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b59a4640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b015f260 0x7f33b006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b51a3640 1 --2- 172.21.15.40:0/2291502764 >> v2:172.21.15.184:3300/0 conn(0x7f33b0070560 0x7f33b00b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 -- 172.21.15.40:0/2291502764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b015f260 msgr2=0x7f33b006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b015f260 0x7f33b006d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 -- 172.21.15.40:0/2291502764 >> v2:172.21.15.184:3300/0 conn(0x7f33b0070560 msgr2=0x7f33b00b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 --2- 172.21.15.40:0/2291502764 >> v2:172.21.15.184:3300/0 conn(0x7f33b0070560 0x7f33b00b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.930+0000 7f33b61a5640 1 -- 172.21.15.40:0/2291502764 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b006c860 con 0x7f33b006ebe0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b61a5640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b00b4c30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f33b0072590 tx=0x7f33a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6280235c0b23f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b49a2640 1 -- 172.21.15.40:0/2291502764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8036070 con 0x7f33b006ebe0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b49a2640 1 -- 172.21.15.40:0/2291502764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a8031600 con 0x7f33b006ebe0 2024-04-26T21:15:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b49a2640 1 -- 172.21.15.40:0/2291502764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80319d0 con 0x7f33b006ebe0 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b7c08640 1 -- 172.21.15.40:0/2291502764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 msgr2=0x7f33b00b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.931+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b00b4c30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f33b0072590 tx=0x7f33a802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 -- 172.21.15.40:0/2291502764 shutdown_connections 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b00b4c30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2291502764 >> v2:172.21.15.184:3300/0 conn(0x7f33b0070560 0x7f33b00b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2291502764 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b015f260 0x7f33b006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 -- 172.21.15.40:0/2291502764 >> 172.21.15.40:0/2291502764 conn(0x7f33b00d1660 msgr2=0x7f33b00d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:41.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 -- 172.21.15.40:0/2291502764 shutdown_connections 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.932+0000 7f33b7c08640 1 -- 172.21.15.40:0/2291502764 wait complete. 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.933+0000 7f33b7c08640 1 Processor -- start 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.933+0000 7f33b7c08640 1 -- start start 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.933+0000 7f33b7c08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 0x7f33b01f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 0x7f33b01f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f33b006d100 con 0x7f33b006ebe0 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33b006ce20 con 0x7f33b015f260 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f33b006db60 con 0x7f33b0070560 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33950/0 (socket says 172.21.15.40:33950) 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 -- 172.21.15.40:0/2128952258 learned_addr learned my addr 172.21.15.40:0/2128952258 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b51a3640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 0x7f33b01f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b61a5640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 0x7f33b01f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 -- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 msgr2=0x7f33b01f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 0x7f33b01f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 -- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 msgr2=0x7f33b01f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 0x7f33b01f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 -- 172.21.15.40:0/2128952258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b01e9a70 con 0x7f33b006ebe0 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b51a3640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 0x7f33b01f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b61a5640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 0x7f33b01f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b59a4640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d800 tx=0x7f33ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00b6c0 con 0x7f33b006ebe0 2024-04-26T21:15:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac019070 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b00d0a70 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00ba80 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.934+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b01e96a0 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.935+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b01e98a0 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.936+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33ac00bc20 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.936+0000 7f33a67fc640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.936+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f33ac08bfe0 con 0x7f33b006ebe0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.936+0000 7f33b51a3640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.937+0000 7f33b51a3640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 0x7f339406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f33a0006fd0 tx=0x7f33a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:41.940+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33ac05a3d0 con 0x7f33b006ebe0 2024-04-26T21:15:42.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.125+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33b006a290 con 0x7f33b006ebe0 2024-04-26T21:15:42.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.126+0000 7f33a67fc640 1 -- 172.21.15.40:0/2128952258 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f33ac059cd0 con 0x7f33b006ebe0 2024-04-26T21:15:42.127 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 msgr2=0x7f339406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 0x7f339406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f33a0006fd0 tx=0x7f33a0008040 comp rx=0 tx=0).stop 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 msgr2=0x7f33b01ea250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d800 tx=0x7f33ac00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 shutdown_connections 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b015f260 0x7f33b01f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b0070560 0x7f33b01f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 --2- 172.21.15.40:0/2128952258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b006ebe0 0x7f33b01ea250 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 >> 172.21.15.40:0/2128952258 conn(0x7f33b00d1660 msgr2=0x7f33b006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 shutdown_connections 2024-04-26T21:15:42.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.128+0000 7f33b7c08640 1 -- 172.21.15.40:0/2128952258 wait complete. 2024-04-26T21:15:42.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.239+0000 7f2e889f0640 1 Processor -- start 2024-04-26T21:15:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.239+0000 7f2e889f0640 1 -- start start 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e889f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e8413ff70 0x7f2e84146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e889f0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e840cff20 0x7f2e841465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e889f0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840d05b0 con 0x7f2e84140350 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e889f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e840d0890 con 0x7f2e8413ff70 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e889f0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8406e450 con 0x7f2e840cff20 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e8413ff70 0x7f2e84146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e8413ff70 0x7f2e84146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33960/0 (socket says 172.21.15.40:33960) 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 -- 172.21.15.40:0/442347621 learned_addr learned my addr 172.21.15.40:0/442347621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e79d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2e84140350 0x7f2e840cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51334/0 (socket says 172.21.15.40:51334) 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 -- 172.21.15.40:0/442347621 >> v2:172.21.15.184:3300/0 conn(0x7f2e840cff20 msgr2=0x7f2e841465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e82d9d640 1 --2- 172.21.15.40:0/442347621 >> v2:172.21.15.184:3300/0 conn(0x7f2e840cff20 0x7f2e841465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 --2- 172.21.15.40:0/442347621 >> v2:172.21.15.184:3300/0 conn(0x7f2e840cff20 0x7f2e841465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 -- 172.21.15.40:0/442347621 >> v1:172.21.15.40:6789/0 conn(0x7f2e84140350 legacy=0x7f2e840cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e81d9b640 1 -- 172.21.15.40:0/442347621 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1171810175 0 0) 0x7f2e840d05b0 con 0x7f2e84140350 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.240+0000 7f2e8259c640 1 -- 172.21.15.40:0/442347621 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8406e730 con 0x7f2e8413ff70 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e8259c640 1 --2- 172.21.15.40:0/442347621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e84146080 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2e840cec30 tx=0x7f2e7402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae05c926439d56ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e81d9b640 1 -- 172.21.15.40:0/442347621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74037070 con 0x7f2e8413ff70 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e81d9b640 1 -- 172.21.15.40:0/442347621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74032440 con 0x7f2e8413ff70 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e81d9b640 1 -- 172.21.15.40:0/442347621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74032810 con 0x7f2e8413ff70 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 -- 172.21.15.40:0/442347621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 msgr2=0x7f2e84146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 --2- 172.21.15.40:0/442347621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e84146080 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2e840cec30 tx=0x7f2e7402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 -- 172.21.15.40:0/442347621 shutdown_connections 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 --2- 172.21.15.40:0/442347621 >> v2:172.21.15.184:3300/0 conn(0x7f2e840cff20 0x7f2e841465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 --2- 172.21.15.40:0/442347621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e84146080 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 -- 172.21.15.40:0/442347621 >> 172.21.15.40:0/442347621 conn(0x7f2e840d0d30 msgr2=0x7f2e840d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 -- 172.21.15.40:0/442347621 shutdown_connections 2024-04-26T21:15:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.241+0000 7f2e889f0640 1 -- 172.21.15.40:0/442347621 wait complete. 2024-04-26T21:15:42.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.243+0000 7f2e889f0640 1 Processor -- start 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.243+0000 7f2e889f0640 1 -- start start 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 0x7f2e84144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 0x7f2e84142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8406efe0 con 0x7f2e8413ff70 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8406ee60 con 0x7f2e840cff20 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e889f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8406f160 con 0x7f2e84140350 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33964/0 (socket says 172.21.15.40:33964) 2024-04-26T21:15:42.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 -- 172.21.15.40:0/759847606 learned_addr learned my addr 172.21.15.40:0/759847606 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e8259c640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 0x7f2e84144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e82d9d640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 0x7f2e84142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 -- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 msgr2=0x7f2e84142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 0x7f2e84142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 -- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 msgr2=0x7f2e84144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 0x7f2e84144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 -- 172.21.15.40:0/759847606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e841f4f20 con 0x7f2e8413ff70 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e8259c640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 0x7f2e84144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e82d9d640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 0x7f2e84142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e79d9b640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c460 tx=0x7f2e6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018070 con 0x7f2e8413ff70 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.244+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6c007500 con 0x7f2e8413ff70 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.245+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c004a20 con 0x7f2e8413ff70 2024-04-26T21:15:42.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.245+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e841f5200 con 0x7f2e8413ff70 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.245+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e841f57c0 con 0x7f2e8413ff70 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.245+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8406a290 con 0x7f2e8413ff70 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.246+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e6c01b070 con 0x7f2e8413ff70 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.246+0000 7f2e7b7fe640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 0x7f2e6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.246+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f2e6c08aee0 con 0x7f2e8413ff70 2024-04-26T21:15:42.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.247+0000 7f2e8259c640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 0x7f2e6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.247+0000 7f2e8259c640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 0x7f2e6406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e74034000 tx=0x7f2e7402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.250+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e6c059230 con 0x7f2e8413ff70 2024-04-26T21:15:42.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.477+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e8413c1a0 con 0x7f2e8413ff70 2024-04-26T21:15:42.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.478+0000 7f2e7b7fe640 1 -- 172.21.15.40:0/759847606 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7f2e6c058b30 con 0x7f2e8413ff70 2024-04-26T21:15:42.479 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:42.479 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:42.480 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 msgr2=0x7f2e6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 0x7f2e6406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e74034000 tx=0x7f2e7402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 msgr2=0x7f2e841453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c460 tx=0x7f2e6c00c920 comp rx=0 tx=0).stop 2024-04-26T21:15:42.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 shutdown_connections 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e6406c8f0 0x7f2e6406edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e84140350 0x7f2e84142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e8413ff70 0x7f2e841453a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 --2- 172.21.15.40:0/759847606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e840cff20 0x7f2e84144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 >> 172.21.15.40:0/759847606 conn(0x7f2e840d0d30 msgr2=0x7f2e840b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 shutdown_connections 2024-04-26T21:15:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.480+0000 7f2e889f0640 1 -- 172.21.15.40:0/759847606 wait complete. 2024-04-26T21:15:42.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.591+0000 7f78c7f4e640 1 Processor -- start 2024-04-26T21:15:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.592+0000 7f78c7f4e640 1 -- start start 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.592+0000 7f78c7f4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78c00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.592+0000 7f78c7f4e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c00d03a0 con 0x7f78c00d3ee0 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.592+0000 7f78c7f4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c00d0680 con 0x7f78c015f370 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.592+0000 7f78c7f4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d0960 con 0x7f78c015f750 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c54e9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78c00d3ee0 0x7f78c00b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51338/0 (socket says 172.21.15.40:51338) 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c54e9640 1 -- 172.21.15.40:0/4177805097 learned_addr learned my addr 172.21.15.40:0/4177805097 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c64eb640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78c00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224609998 0 0) 0x7f78c00d03a0 con 0x7f78c00d3ee0 2024-04-26T21:15:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78ac003520 con 0x7f78c00d3ee0 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3002914821 0 0) 0x7f78ac003520 con 0x7f78c00d3ee0 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 msgr2=0x7f78c00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78c00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 >> v1:172.21.15.184:6789/0 conn(0x7f78c015f370 legacy=0x7f78c00d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00d0c40 con 0x7f78c00d3ee0 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f78b0001670 con 0x7f78c00d3ee0 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 >> v1:172.21.15.40:6789/0 conn(0x7f78c00d3ee0 legacy=0x7f78c00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ac003f60 0x7f78ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00d0960 con 0x7f78ac003f60 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac00baa0 con 0x7f78ac004d40 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac00bca0 con 0x7f78c015f750 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c54e9640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ac003f60 0x7f78ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c5cea640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.593+0000 7f78c64eb640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 -- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 msgr2=0x7f78ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c015f750 0x7f78ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 -- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac004d40 msgr2=0x7f78ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 -- 172.21.15.40:0/4177805097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac003520 con 0x7f78ac003f60 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c64eb640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c54e9640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ac003f60 0x7f78ac004800 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f78c008e680 tx=0x7f78ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5da2a4d34b8aaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007cf0 con 0x7f78ac003f60 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b0005af0 con 0x7f78ac003f60 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.594+0000 7f78c4ce8640 1 -- 172.21.15.40:0/4177805097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007070 con 0x7f78ac003f60 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 -- 172.21.15.40:0/4177805097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ac003f60 msgr2=0x7f78ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/4177805097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ac003f60 0x7f78ac004800 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f78c008e680 tx=0x7f78ac005830 comp rx=0 tx=0).stop 2024-04-26T21:15:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c64eb640 1 -- 172.21.15.40:0/4177805097 reap_dead start 2024-04-26T21:15:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 -- 172.21.15.40:0/4177805097 shutdown_connections 2024-04-26T21:15:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 -- 172.21.15.40:0/4177805097 >> 172.21.15.40:0/4177805097 conn(0x7f78c008ec10 msgr2=0x7f78c009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 -- 172.21.15.40:0/4177805097 shutdown_connections 2024-04-26T21:15:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.595+0000 7f78c7f4e640 1 -- 172.21.15.40:0/4177805097 wait complete. 2024-04-26T21:15:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.596+0000 7f78c7f4e640 1 Processor -- start 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.596+0000 7f78c7f4e640 1 -- start start 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac003f60 0x7f78c00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00cb950 0x7f78c014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00cfef0 con 0x7f78c014ba10 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00cfd70 con 0x7f78ac003f60 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c7f4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78c00d0070 con 0x7f78c00cb950 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33974/0 (socket says 172.21.15.40:33974) 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 -- 172.21.15.40:0/3678707695 learned_addr learned my addr 172.21.15.40:0/3678707695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c5cea640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac003f60 0x7f78c00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c64eb640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00cb950 0x7f78c014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 -- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00cb950 msgr2=0x7f78c014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00cb950 0x7f78c014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 -- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac003f60 msgr2=0x7f78c00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac003f60 0x7f78c00cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.597+0000 7f78c54e9640 1 -- 172.21.15.40:0/3678707695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c00cebe0 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78c54e9640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f78b000c300 tx=0x7f78c00d0e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0041020 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b0001430 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c00ceec0 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007070 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.598+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c00cc800 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.599+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c006a290 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.599+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78b0044020 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.600+0000 7f78b67fc640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 0x7f78a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.600+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f78b00abaa0 con 0x7f78c014ba10 2024-04-26T21:15:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.600+0000 7f78c5cea640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 0x7f78a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.601+0000 7f78c5cea640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 0x7f78a406edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0017e0 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.604+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78b004b030 con 0x7f78c014ba10 2024-04-26T21:15:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.789+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78c00c8a90 con 0x7f78c014ba10 2024-04-26T21:15:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.790+0000 7f78b67fc640 1 -- 172.21.15.40:0/3678707695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16765 (secure 0 0 0) 0x7f78b00798d0 con 0x7f78c014ba10 2024-04-26T21:15:42.792 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:42.792 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:42.258455+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 msgr2=0x7f78a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 0x7f78a406edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0017e0 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 msgr2=0x7f78c00cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f78b000c300 tx=0x7f78c00d0e10 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 shutdown_connections 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f78a406c8f0 0x7f78a406edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78c00cb950 0x7f78c014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c014ba10 0x7f78c00cb410 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.792+0000 7f78c7f4e640 1 --2- 172.21.15.40:0/3678707695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac003f60 0x7f78c00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.793+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 >> 172.21.15.40:0/3678707695 conn(0x7f78c008ec10 msgr2=0x7f78c015e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.793+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 shutdown_connections 2024-04-26T21:15:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.793+0000 7f78c7f4e640 1 -- 172.21.15.40:0/3678707695 wait complete. 2024-04-26T21:15:42.804 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:15:42.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.904+0000 7fe505989640 1 Processor -- start 2024-04-26T21:15:42.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.904+0000 7fe505989640 1 -- start start 2024-04-26T21:15:42.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe505989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 0x7fe50015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe505989640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5000d1740 con 0x7fe5000d11b0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe505989640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5000d1a20 con 0x7fe5000cf2e0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe505989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5000d1d00 con 0x7fe50006e490 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4f7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5000d11b0 0x7fe50006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51348/0 (socket says 172.21.15.40:51348) 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4f7fff640 1 -- 172.21.15.40:0/1580587738 learned_addr learned my addr 172.21.15.40:0/1580587738 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4ff7fe640 1 --2- 172.21.15.40:0/1580587738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 0x7fe50015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3305615286 0 0) 0x7fe5000d1740 con 0x7fe5000d11b0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4ec003520 con 0x7fe5000d11b0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4ff7fe640 1 -- 172.21.15.40:0/1580587738 >> v1:172.21.15.184:6789/0 conn(0x7fe5000cf2e0 legacy=0x7fe50006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 267042540 0 0) 0x7fe4ec003520 con 0x7fe5000d11b0 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4ff7fe640 1 -- 172.21.15.40:0/1580587738 >> v1:172.21.15.40:6789/0 conn(0x7fe5000d11b0 legacy=0x7fe50006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.905+0000 7fe4ff7fe640 1 -- 172.21.15.40:0/1580587738 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5000d1fe0 con 0x7fe50006e490 2024-04-26T21:15:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.906+0000 7fe4ff7fe640 1 --2- 172.21.15.40:0/1580587738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 0x7fe50015ebe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009df0 tx=0x7fe4f802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e2b7922b7bd1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.906+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8035020 con 0x7fe50006e490 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.906+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f8007540 con 0x7fe50006e490 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.906+0000 7fe4fe7fc640 1 -- 172.21.15.40:0/1580587738 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8001670 con 0x7fe50006e490 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 -- 172.21.15.40:0/1580587738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 msgr2=0x7fe50015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 --2- 172.21.15.40:0/1580587738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 0x7fe50015ebe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009df0 tx=0x7fe4f802a5b0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 -- 172.21.15.40:0/1580587738 shutdown_connections 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 --2- 172.21.15.40:0/1580587738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe50006e490 0x7fe50015ebe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 -- 172.21.15.40:0/1580587738 >> 172.21.15.40:0/1580587738 conn(0x7fe5000b4a60 msgr2=0x7fe5000b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 -- 172.21.15.40:0/1580587738 shutdown_connections 2024-04-26T21:15:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.907+0000 7fe505989640 1 -- 172.21.15.40:0/1580587738 wait complete. 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.908+0000 7fe505989640 1 Processor -- start 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.908+0000 7fe505989640 1 -- start start 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 0x7fe500154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5000d11b0 0x7fe5000715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5000cf840 con 0x7fe5000cf2e0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5000cf6c0 con 0x7fe50006e490 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe505989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5000ce660 con 0x7fe5000d11b0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33986/0 (socket says 172.21.15.40:33986) 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 -- 172.21.15.40:0/1063813644 learned_addr learned my addr 172.21.15.40:0/1063813644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4feffd640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 0x7fe500154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4ff7fe640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5000d11b0 0x7fe5000715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 -- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5000d11b0 msgr2=0x7fe5000715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5000d11b0 0x7fe5000715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 -- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 msgr2=0x7fe500154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 0x7fe500154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 -- 172.21.15.40:0/1063813644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5000ca040 con 0x7fe5000cf2e0 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4feffd640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 0x7fe500154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.909+0000 7fe4f7fff640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c520 tx=0x7fe4e800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.910+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8018070 con 0x7fe5000cf2e0 2024-04-26T21:15:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.910+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e8007500 con 0x7fe5000cf2e0 2024-04-26T21:15:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.910+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5001f4d80 con 0x7fe5000cf2e0 2024-04-26T21:15:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.910+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8004a20 con 0x7fe5000cf2e0 2024-04-26T21:15:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.910+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001f5340 con 0x7fe5000cf2e0 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.911+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50006a290 con 0x7fe5000cf2e0 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.911+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4e801b070 con 0x7fe5000cf2e0 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.912+0000 7fe4fcff9640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.912+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fe4e808aeb0 con 0x7fe5000cf2e0 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.912+0000 7fe4feffd640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.912+0000 7fe4feffd640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0001860 tx=0x7fe4f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:42.916+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4e8059330 con 0x7fe5000cf2e0 2024-04-26T21:15:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.137+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5000b8ef0 con 0x7fe5000cf2e0 2024-04-26T21:15:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.138+0000 7fe4fcff9640 1 -- 172.21.15.40:0/1063813644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7fe4e8058c30 con 0x7fe5000cf2e0 2024-04-26T21:15:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:43.139 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 msgr2=0x7fe4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0001860 tx=0x7fe4f0009040 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 msgr2=0x7fe500071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c520 tx=0x7fe4e800c9e0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 shutdown_connections 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5000d11b0 0x7fe5000715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5000cf2e0 0x7fe500071060 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 --2- 172.21.15.40:0/1063813644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe50006e490 0x7fe500154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 >> 172.21.15.40:0/1063813644 conn(0x7fe5000b4a60 msgr2=0x7fe5000b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 shutdown_connections 2024-04-26T21:15:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.139+0000 7fe505989640 1 -- 172.21.15.40:0/1063813644 wait complete. 2024-04-26T21:15:43.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.250+0000 7fb035252640 1 Processor -- start 2024-04-26T21:15:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.250+0000 7fb035252640 1 -- start start 2024-04-26T21:15:43.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb035252640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb03015b4e0 0x7fb030071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb035252640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0300733e0 con 0x7fb03015b8c0 2024-04-26T21:15:43.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb035252640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb030072f00 con 0x7fb03006f2c0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb035252640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb030072660 con 0x7fb03015b4e0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb02e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb03015b8c0 0x7fb03006ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51364/0 (socket says 172.21.15.40:51364) 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.251+0000 7fb02e59c640 1 -- 172.21.15.40:0/2633720180 learned_addr learned my addr 172.21.15.40:0/2633720180 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2633720180 >> v2:172.21.15.184:3300/0 conn(0x7fb03015b4e0 0x7fb030071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612458951 0 0) 0x7fb0300733e0 con 0x7fb03015b8c0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb01c003520 con 0x7fb03015b8c0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376678796 0 0) 0x7fb030072f00 con 0x7fb03006f2c0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0300733e0 con 0x7fb03006f2c0 2024-04-26T21:15:43.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2360248479 0 0) 0x7fb01c003520 con 0x7fb03015b8c0 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 >> v2:172.21.15.184:3300/0 conn(0x7fb03015b4e0 msgr2=0x7fb030071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2633720180 >> v2:172.21.15.184:3300/0 conn(0x7fb03015b4e0 0x7fb030071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 >> v1:172.21.15.184:6789/0 conn(0x7fb03006f2c0 legacy=0x7fb0300b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb030072940 con 0x7fb03015b8c0 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fb018001670 con 0x7fb03015b8c0 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 >> v1:172.21.15.40:6789/0 conn(0x7fb03015b8c0 legacy=0x7fb03006ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c003b40 0x7fb01c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c004800 0x7fb01c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 0x7fb01c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb030072660 con 0x7fb01c0051c0 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c00c0f0 con 0x7fb01c004800 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb01c00c2f0 con 0x7fb01c003b40 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 0x7fb01c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c003b40 0x7fb01c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.252+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c004800 0x7fb01c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 -- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c003b40 msgr2=0x7fb01c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c003b40 0x7fb01c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 -- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c004800 msgr2=0x7fb01c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb01c004800 0x7fb01c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 -- 172.21.15.40:0/2633720180 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c003520 con 0x7fb01c0051c0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 -- 172.21.15.40:0/2633720180 reap_dead start 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 0x7fb01c00b9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb020001bb0 tx=0x7fb01c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124733b8380cb13c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020001bf0 con 0x7fb01c0051c0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020003b70 con 0x7fb01c0051c0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.253+0000 7fb02dd9b640 1 -- 172.21.15.40:0/2633720180 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200050d0 con 0x7fb01c0051c0 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 -- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 msgr2=0x7fb01c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 0x7fb01c00b9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb020001bb0 tx=0x7fb01c005830 comp rx=0 tx=0).stop 2024-04-26T21:15:43.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 -- 172.21.15.40:0/2633720180 shutdown_connections 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 --2- 172.21.15.40:0/2633720180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0051c0 0x7fb01c00b9c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 -- 172.21.15.40:0/2633720180 >> 172.21.15.40:0/2633720180 conn(0x7fb0300d0d30 msgr2=0x7fb0300d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 -- 172.21.15.40:0/2633720180 shutdown_connections 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.254+0000 7fb035252640 1 -- 172.21.15.40:0/2633720180 wait complete. 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.255+0000 7fb035252640 1 Processor -- start 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.255+0000 7fb035252640 1 -- start start 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c0051c0 0x7fb0300cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 0x7fb0300b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb030073820 con 0x7fb03014f1d0 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300b3e80 con 0x7fb0300cb010 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb035252640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300739a0 con 0x7fb01c0051c0 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb02f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb02f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34000/0 (socket says 172.21.15.40:34000) 2024-04-26T21:15:43.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.256+0000 7fb02f59e640 1 -- 172.21.15.40:0/2342423102 learned_addr learned my addr 172.21.15.40:0/2342423102 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c0051c0 0x7fb0300cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 0x7fb0300b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 -- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c0051c0 msgr2=0x7fb0300cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c0051c0 0x7fb0300cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 -- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 msgr2=0x7fb0300b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 0x7fb0300b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 -- 172.21.15.40:0/2342423102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0301f4ca0 con 0x7fb03014f1d0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02e59c640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 0x7fb0300b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb02f59e640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0200020c0 tx=0x7fb020001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020001bf0 con 0x7fb03014f1d0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020003970 con 0x7fb03014f1d0 2024-04-26T21:15:43.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020005710 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0301f4f80 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.257+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0300cbce0 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.258+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0301f5a40 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.259+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb020042030 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.259+0000 7fb0277fe640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 0x7fb01006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.259+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7fb020038040 con 0x7fb03014f1d0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.259+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 0x7fb01006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.260+0000 7fb02ed9d640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 0x7fb01006edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb0301f5ea0 tx=0x7fb028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.263+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb020049020 con 0x7fb03014f1d0 2024-04-26T21:15:43.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.487+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03006a290 con 0x7fb03014f1d0 2024-04-26T21:15:43.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.488+0000 7fb0277fe640 1 -- 172.21.15.40:0/2342423102 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5177 (secure 0 0 0) 0x7fb0200323a0 con 0x7fb03014f1d0 2024-04-26T21:15:43.489 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:43.489 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:14.354319+0000","modified":"2024-04-26T21:15:17.358700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2174268870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2174268870},{"type":"v1","addr":"172.21.15.40:6835","nonce":2174268870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":5}]} 2024-04-26T21:15:43.489 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-04-26T21:15:43.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 msgr2=0x7fb01006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 0x7fb01006edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb0301f5ea0 tx=0x7fb028008040 comp rx=0 tx=0).stop 2024-04-26T21:15:43.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 msgr2=0x7fb0300b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0200020c0 tx=0x7fb020001650 comp rx=0 tx=0).stop 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 shutdown_connections 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01006c8f0 0x7fb01006edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb03014f1d0 0x7fb0300b47e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0300cb010 0x7fb0300b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 --2- 172.21.15.40:0/2342423102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb01c0051c0 0x7fb0300cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 >> 172.21.15.40:0/2342423102 conn(0x7fb0300d0d30 msgr2=0x7fb03014b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 shutdown_connections 2024-04-26T21:15:43.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.490+0000 7fb035252640 1 -- 172.21.15.40:0/2342423102 wait complete. 2024-04-26T21:15:43.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:43.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 Processor -- start 2024-04-26T21:15:43.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 -- start start 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dc8140350 0x7f4dc80cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc8146790 0x7f4dc80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc80d05b0 con 0x7f4dc813ff70 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4dc80d0890 con 0x7f4dc8140350 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.602+0000 7f4dcd738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc806e450 con 0x7f4dc8146790 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dc8140350 0x7f4dc80cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dc8140350 0x7f4dc80cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34008/0 (socket says 172.21.15.40:34008) 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 -- 172.21.15.40:0/1175019493 learned_addr learned my addr 172.21.15.40:0/1175019493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:43.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dc77fe640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc8146790 0x7f4dc80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 -- 172.21.15.40:0/1175019493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc8146790 msgr2=0x7f4dc80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc8146790 0x7f4dc80cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 -- 172.21.15.40:0/1175019493 >> v1:172.21.15.184:6789/0 conn(0x7f4dc813ff70 legacy=0x7f4dc8146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.603+0000 7f4dbe7fc640 1 -- 172.21.15.40:0/1175019493 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc806e730 con 0x7f4dc8140350 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dbe7fc640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8140350 0x7f4dc80cf810 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009870 tx=0x7f4db002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c283873f88b197 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dc67fc640 1 -- 172.21.15.40:0/1175019493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0037070 con 0x7f4dc8140350 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dc67fc640 1 -- 172.21.15.40:0/1175019493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db0032440 con 0x7f4dc8140350 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dc67fc640 1 -- 172.21.15.40:0/1175019493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0032810 con 0x7f4dc8140350 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 -- 172.21.15.40:0/1175019493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8140350 msgr2=0x7f4dc80cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8140350 0x7f4dc80cf810 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009870 tx=0x7f4db002a180 comp rx=0 tx=0).stop 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 -- 172.21.15.40:0/1175019493 shutdown_connections 2024-04-26T21:15:43.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc8146790 0x7f4dc80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 --2- 172.21.15.40:0/1175019493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8140350 0x7f4dc80cf810 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.604+0000 7f4dcd738640 1 -- 172.21.15.40:0/1175019493 >> 172.21.15.40:0/1175019493 conn(0x7f4dc80d0d30 msgr2=0x7f4dc80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.605+0000 7f4dcd738640 1 -- 172.21.15.40:0/1175019493 shutdown_connections 2024-04-26T21:15:43.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.605+0000 7f4dcd738640 1 -- 172.21.15.40:0/1175019493 wait complete. 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.606+0000 7f4dcd738640 1 Processor -- start 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.606+0000 7f4dcd738640 1 -- start start 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc813ff70 0x7f4dc80b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8140350 0x7f4dc813e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc806edd0 con 0x7f4dc8146790 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc806ec50 con 0x7f4dc8140350 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc806ef50 con 0x7f4dc813ff70 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34010/0 (socket says 172.21.15.40:34010) 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 -- 172.21.15.40:0/177165149 learned_addr learned my addr 172.21.15.40:0/177165149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:43.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc6ffd640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc813ff70 0x7f4dc80b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dbe7fc640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8140350 0x7f4dc813e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 -- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc813ff70 msgr2=0x7f4dc80b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc813ff70 0x7f4dc80b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 -- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8140350 msgr2=0x7f4dc813e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8140350 0x7f4dc813e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 -- 172.21.15.40:0/177165149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81f3b20 con 0x7f4dc8146790 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dc77fe640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0007d40 tx=0x7f4dc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0014420 con 0x7f4dc8146790 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0019070 con 0x7f4dc8146790 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc81f3e00 con 0x7f4dc8146790 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.607+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0014820 con 0x7f4dc8146790 2024-04-26T21:15:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.608+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81f43c0 con 0x7f4dc8146790 2024-04-26T21:15:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.608+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc806a290 con 0x7f4dc8146790 2024-04-26T21:15:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.609+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dc00149c0 con 0x7f4dc8146790 2024-04-26T21:15:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.609+0000 7f4dbffff640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 0x7f4da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.609+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f4dc008c180 con 0x7f4dc8146790 2024-04-26T21:15:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.610+0000 7f4dc6ffd640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 0x7f4da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.610+0000 7f4dc6ffd640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 0x7f4da806edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004f60 tx=0x7f4db8007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.614+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4dc0059fb0 con 0x7f4dc8146790 2024-04-26T21:15:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.798+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dc815d450 con 0x7f4dc8146790 2024-04-26T21:15:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.799+0000 7f4dbffff640 1 -- 172.21.15.40:0/177165149 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16765 (secure 0 0 0) 0x7f4dc005e2b0 con 0x7f4dc8146790 2024-04-26T21:15:43.801 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:43.801 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:42.258455+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:12.615950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:13.297387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 msgr2=0x7f4da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 0x7f4da806edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004f60 tx=0x7f4db8007e30 comp rx=0 tx=0).stop 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 msgr2=0x7f4dc81f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0007d40 tx=0x7f4dc000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 shutdown_connections 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4da806c8f0 0x7f4da806edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dc8146790 0x7f4dc81f70a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8140350 0x7f4dc813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 --2- 172.21.15.40:0/177165149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4dc813ff70 0x7f4dc80b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.801+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 >> 172.21.15.40:0/177165149 conn(0x7f4dc80d0d30 msgr2=0x7f4dc806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.802+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 shutdown_connections 2024-04-26T21:15:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.802+0000 7f4dcd738640 1 -- 172.21.15.40:0/177165149 wait complete. 2024-04-26T21:15:43.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:15:43.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.921+0000 7f55ed10a640 1 Processor -- start 2024-04-26T21:15:43.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.921+0000 7f55ed10a640 1 -- start start 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55ed10a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55ed10a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80d03a0 con 0x7f55e815f370 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55ed10a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80d0680 con 0x7f55e80d3ee0 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55ed10a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80d0960 con 0x7f55e815f750 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55e6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f55e815f370 0x7f55e80d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51376/0 (socket says 172.21.15.40:51376) 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.922+0000 7f55e6d9d640 1 -- 172.21.15.40:0/4247029397 learned_addr learned my addr 172.21.15.40:0/4247029397 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e759e640 1 --2- 172.21.15.40:0/4247029397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721782971 0 0) 0x7f55e80d03a0 con 0x7f55e815f370 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55d4003520 con 0x7f55e815f370 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742080832 0 0) 0x7f55e80d0680 con 0x7f55e80d3ee0 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e80d03a0 con 0x7f55e80d3ee0 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934360074 0 0) 0x7f55d4003520 con 0x7f55e815f370 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e759e640 1 -- 172.21.15.40:0/4247029397 >> v1:172.21.15.184:6789/0 conn(0x7f55e80d3ee0 legacy=0x7f55e80b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e759e640 1 -- 172.21.15.40:0/4247029397 >> v1:172.21.15.40:6789/0 conn(0x7f55e815f370 legacy=0x7f55e80d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.923+0000 7f55e759e640 1 -- 172.21.15.40:0/4247029397 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80d0c40 con 0x7f55e815f750 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55e759e640 1 --2- 172.21.15.40:0/4247029397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e80cfd70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097d0 tx=0x7f55e002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c0a05e2fbf734e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0037070 con 0x7f55e815f750 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e002ac30 con 0x7f55e815f750 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55e5d9b640 1 -- 172.21.15.40:0/4247029397 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d80019f0 con 0x7f55e815f750 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 -- 172.21.15.40:0/4247029397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 msgr2=0x7f55e80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 --2- 172.21.15.40:0/4247029397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e80cfd70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097d0 tx=0x7f55e002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 -- 172.21.15.40:0/4247029397 shutdown_connections 2024-04-26T21:15:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 --2- 172.21.15.40:0/4247029397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e80cfd70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 -- 172.21.15.40:0/4247029397 >> 172.21.15.40:0/4247029397 conn(0x7f55e808ec10 msgr2=0x7f55e809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 -- 172.21.15.40:0/4247029397 shutdown_connections 2024-04-26T21:15:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.924+0000 7f55ed10a640 1 -- 172.21.15.40:0/4247029397 wait complete. 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.926+0000 7f55ed10a640 1 Processor -- start 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.926+0000 7f55ed10a640 1 -- start start 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55e815f370 0x7f55e80b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e81f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80b4950 con 0x7f55e80d3ee0 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80b47d0 con 0x7f55e815f370 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80b4ad0 con 0x7f55e815f750 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34014/0 (socket says 172.21.15.40:34014) 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 -- 172.21.15.40:0/1246193659 learned_addr learned my addr 172.21.15.40:0/1246193659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e659c640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55e815f370 0x7f55e80b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e759e640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e81f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 -- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 msgr2=0x7f55e81f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e81f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 -- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55e815f370 msgr2=0x7f55e80b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55e815f370 0x7f55e80b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 -- 172.21.15.40:0/1246193659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80b8640 con 0x7f55e80d3ee0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55e6d9d640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f55d8001f60 tx=0x7f55d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8017070 con 0x7f55e80d3ee0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d8012400 con 0x7f55e80d3ee0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e80b8920 con 0x7f55e80d3ee0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8012830 con 0x7f55e80d3ee0 2024-04-26T21:15:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.927+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e80b8330 con 0x7f55e80d3ee0 2024-04-26T21:15:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.929+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e80b8d10 con 0x7f55e80d3ee0 2024-04-26T21:15:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.929+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55d801a070 con 0x7f55e80d3ee0 2024-04-26T21:15:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.929+0000 7f55df7fe640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 0x7f55c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.929+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5767+0+0 (secure 0 0 0) 0x7f55d808bb60 con 0x7f55e80d3ee0 2024-04-26T21:15:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.930+0000 7f55e659c640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 0x7f55c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.930+0000 7f55e659c640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 0x7f55c806edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f55e815dfc0 tx=0x7f55d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:43.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:43.933+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55d8059f50 con 0x7f55e80d3ee0 2024-04-26T21:15:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.156+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55e806a290 con 0x7f55e80d3ee0 2024-04-26T21:15:44.157 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:15:44.156+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:15:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.158+0000 7f55df7fe640 1 -- 172.21.15.40:0/1246193659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f55d8059850 con 0x7f55e80d3ee0 2024-04-26T21:15:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 msgr2=0x7f55c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 0x7f55c806edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f55e815dfc0 tx=0x7f55d0009040 comp rx=0 tx=0).stop 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 msgr2=0x7f55e81f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f55d8001f60 tx=0x7f55d800cf90 comp rx=0 tx=0).stop 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 shutdown_connections 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f55c806c8f0 0x7f55c806edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55e815f750 0x7f55e81f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55e815f370 0x7f55e80b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 --2- 172.21.15.40:0/1246193659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55e80d3ee0 0x7f55e81f4ca0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 >> 172.21.15.40:0/1246193659 conn(0x7f55e808ec10 msgr2=0x7f55e80b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 shutdown_connections 2024-04-26T21:15:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.161+0000 7f55ed10a640 1 -- 172.21.15.40:0/1246193659 wait complete. 2024-04-26T21:15:44.173 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:15:44.198 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:15:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.276+0000 7fac58b96640 1 Processor -- start 2024-04-26T21:15:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.276+0000 7fac58b96640 1 -- start start 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac58b96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac540d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac58b96640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac540d0bc0 0x7fac5406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac58b96640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac540d1070 con 0x7fac54146830 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac58b96640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fac54071970 con 0x7fac540d0bc0 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac58b96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac540d12f0 con 0x7fac5406c9b0 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac52d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac540d0bc0 0x7fac5406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac52d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac540d0bc0 0x7fac5406fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34050/0 (socket says 172.21.15.40:34050) 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac52d9d640 1 -- 172.21.15.40:0/2972776131 learned_addr learned my addr 172.21.15.40:0/2972776131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.277+0000 7fac51d9b640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac540d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac52d9d640 1 -- 172.21.15.40:0/2972776131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 msgr2=0x7fac540d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac52d9d640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac540d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac52d9d640 1 -- 172.21.15.40:0/2972776131 >> v1:172.21.15.184:6789/0 conn(0x7fac54146830 legacy=0x7fac5406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac52d9d640 1 -- 172.21.15.40:0/2972776131 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac540d15d0 con 0x7fac540d0bc0 2024-04-26T21:15:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac51d9b640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac540d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac52d9d640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac540d0bc0 0x7fac5406fa20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009870 tx=0x7fac4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9112307441e7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac5159a640 1 -- 172.21.15.40:0/2972776131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c037070 con 0x7fac540d0bc0 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac5159a640 1 -- 172.21.15.40:0/2972776131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c032440 con 0x7fac540d0bc0 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.278+0000 7fac5159a640 1 -- 172.21.15.40:0/2972776131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c032810 con 0x7fac540d0bc0 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 -- 172.21.15.40:0/2972776131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac540d0bc0 msgr2=0x7fac5406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac540d0bc0 0x7fac5406fa20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009870 tx=0x7fac4c02a270 comp rx=0 tx=0).stop 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 -- 172.21.15.40:0/2972776131 shutdown_connections 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac540d0bc0 0x7fac5406fa20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 --2- 172.21.15.40:0/2972776131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac540d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 -- 172.21.15.40:0/2972776131 >> 172.21.15.40:0/2972776131 conn(0x7fac540b3b60 msgr2=0x7fac540b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:44.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 -- 172.21.15.40:0/2972776131 shutdown_connections 2024-04-26T21:15:44.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.279+0000 7fac58b96640 1 -- 172.21.15.40:0/2972776131 wait complete. 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.280+0000 7fac58b96640 1 Processor -- start 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 -- start start 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac54140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac540d0bc0 0x7fac5413e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac5406f500 con 0x7fac54146830 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac540d1b90 con 0x7fac540d0bc0 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac58b96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fac54146c10 con 0x7fac5406c9b0 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac52d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac52d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34060/0 (socket says 172.21.15.40:34060) 2024-04-26T21:15:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.281+0000 7fac52d9d640 1 -- 172.21.15.40:0/4174449384 learned_addr learned my addr 172.21.15.40:0/4174449384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac51d9b640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac540d0bc0 0x7fac5413e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac5259c640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac54140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 -- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 msgr2=0x7fac54140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac54140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 -- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac540d0bc0 msgr2=0x7fac5413e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac540d0bc0 0x7fac5413e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 -- 172.21.15.40:0/4174449384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541603d0 con 0x7fac54146830 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac52d9d640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009870 tx=0x7fac4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c037040 con 0x7fac54146830 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c042070 con 0x7fac54146830 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5413ef10 con 0x7fac54146830 2024-04-26T21:15:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c030920 con 0x7fac54146830 2024-04-26T21:15:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.282+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5413f4d0 con 0x7fac54146830 2024-04-26T21:15:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.283+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac540693d0 con 0x7fac54146830 2024-04-26T21:15:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.284+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac4c030ac0 con 0x7fac54146830 2024-04-26T21:15:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.284+0000 7fac42ffd640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 0x7fac3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.284+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7fac4c0ac590 con 0x7fac54146830 2024-04-26T21:15:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.284+0000 7fac5259c640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 0x7fac3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.285+0000 7fac5259c640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 0x7fac3406edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac44001ca0 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.288+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac541603d0 con 0x7fac54146830 2024-04-26T21:15:44.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:44.516+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fac54070610 con 0x7fac54146830 2024-04-26T21:15:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.159+0000 7fac42ffd640 1 -- 172.21.15.40:0/4174449384 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7fac4c07a2e0 con 0x7fac54146830 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 msgr2=0x7fac3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 0x7fac3406edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac44001ca0 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 msgr2=0x7fac5413e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009870 tx=0x7fac4c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 shutdown_connections 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fac3406c8f0 0x7fac3406edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac54146830 0x7fac5413e870 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac540d0bc0 0x7fac5413e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 --2- 172.21.15.40:0/4174449384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5406c9b0 0x7fac54140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 >> 172.21.15.40:0/4174449384 conn(0x7fac540b3b60 msgr2=0x7fac540b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 shutdown_connections 2024-04-26T21:15:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.162+0000 7fac58b96640 1 -- 172.21.15.40:0/4174449384 wait complete. 2024-04-26T21:15:45.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:15:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.330+0000 7f30c5df2640 1 Processor -- start 2024-04-26T21:15:45.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.330+0000 7f30c5df2640 1 -- start start 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30c5df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c00d2140 0x7f30c00d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30c5df2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30c00b6020 0x7f30c00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30c5df2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c00d33a0 con 0x7f30c0162a10 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30c5df2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00b4970 con 0x7f30c00b6020 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30c5df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00d3680 con 0x7f30c00d2140 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30bffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30c00b6020 0x7f30c00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30bffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30c00b6020 0x7f30c00cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34072/0 (socket says 172.21.15.40:34072) 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.331+0000 7f30bffff640 1 -- 172.21.15.40:0/3138226380 learned_addr learned my addr 172.21.15.40:0/3138226380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30beffd640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c00d2140 0x7f30c00d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30bffff640 1 -- 172.21.15.40:0/3138226380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c00d2140 msgr2=0x7f30c00d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30bffff640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c00d2140 0x7f30c00d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30bffff640 1 -- 172.21.15.40:0/3138226380 >> v1:172.21.15.184:6789/0 conn(0x7f30c0162a10 legacy=0x7f30c00d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30bffff640 1 -- 172.21.15.40:0/3138226380 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00d3960 con 0x7f30c00b6020 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.332+0000 7f30bffff640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00b6020 0x7f30c00cffb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009e50 tx=0x7f30b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fa55d66c49f802 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30be7fc640 1 -- 172.21.15.40:0/3138226380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8036070 con 0x7f30c00b6020 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30be7fc640 1 -- 172.21.15.40:0/3138226380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8031600 con 0x7f30c00b6020 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30be7fc640 1 -- 172.21.15.40:0/3138226380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80319d0 con 0x7f30c00b6020 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 -- 172.21.15.40:0/3138226380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00b6020 msgr2=0x7f30c00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00b6020 0x7f30c00cffb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009e50 tx=0x7f30b802a710 comp rx=0 tx=0).stop 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 -- 172.21.15.40:0/3138226380 shutdown_connections 2024-04-26T21:15:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00b6020 0x7f30c00cffb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 --2- 172.21.15.40:0/3138226380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c00d2140 0x7f30c00d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 -- 172.21.15.40:0/3138226380 >> 172.21.15.40:0/3138226380 conn(0x7f30c009e530 msgr2=0x7f30c00d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 -- 172.21.15.40:0/3138226380 shutdown_connections 2024-04-26T21:15:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.333+0000 7f30c5df2640 1 -- 172.21.15.40:0/3138226380 wait complete. 2024-04-26T21:15:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.334+0000 7f30c5df2640 1 Processor -- start 2024-04-26T21:15:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 -- start start 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 0x7f30c015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 0x7f30c01537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f30c0162df0 con 0x7f30c00d2140 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d3f20 con 0x7f30c00b6020 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30c5df2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00b6600 con 0x7f30c0162a10 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34080/0 (socket says 172.21.15.40:34080) 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.335+0000 7f30beffd640 1 -- 172.21.15.40:0/1533382384 learned_addr learned my addr 172.21.15.40:0/1533382384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30bf7fe640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 0x7f30c015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30bffff640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 0x7f30c01537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 -- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 msgr2=0x7f30c01537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 0x7f30c01537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 -- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 msgr2=0x7f30c015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 0x7f30c015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 -- 172.21.15.40:0/1533382384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c015a9a0 con 0x7f30c00d2140 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30bffff640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 0x7f30c01537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30bf7fe640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 0x7f30c015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30beffd640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f30b400d620 tx=0x7f30b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b400b620 con 0x7f30c00d2140 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b4019070 con 0x7f30c00d2140 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c00b29d0 con 0x7f30c00d2140 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b400ba00 con 0x7f30c00d2140 2024-04-26T21:15:45.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.336+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c00b2f90 con 0x7f30c00d2140 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.337+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c006a290 con 0x7f30c00d2140 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.338+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30b400bba0 con 0x7f30c00d2140 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.338+0000 7f30a7fff640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.338+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f30b408bfd0 con 0x7f30c00d2140 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.339+0000 7f30bf7fe640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:45.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.339+0000 7f30bf7fe640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 0x7f30ac06edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30b0001660 tx=0x7f30b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:45.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.342+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30b405a370 con 0x7f30c00d2140 2024-04-26T21:15:45.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:45.525+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30c00d27a0 con 0x7f30c00d2140 2024-04-26T21:15:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.290+0000 7f30a7fff640 1 -- 172.21.15.40:0/1533382384 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7f30b4060030 con 0x7f30c00d2140 2024-04-26T21:15:46.291 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 msgr2=0x7f30ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 0x7f30ac06edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30b0001660 tx=0x7f30b0009040 comp rx=0 tx=0).stop 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 msgr2=0x7f30c015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f30b400d620 tx=0x7f30b400dae0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 shutdown_connections 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30c0162a10 0x7f30c01537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30c00d2140 0x7f30c015c3e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 --2- 172.21.15.40:0/1533382384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30c00b6020 0x7f30c015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 >> 172.21.15.40:0/1533382384 conn(0x7f30c009e530 msgr2=0x7f30c015f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 shutdown_connections 2024-04-26T21:15:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.293+0000 7f30c5df2640 1 -- 172.21.15.40:0/1533382384 wait complete. 2024-04-26T21:15:46.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:15:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.421+0000 7f3857903640 1 Processor -- start 2024-04-26T21:15:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.421+0000 7f3857903640 1 -- start start 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3857903640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3857903640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38500d3a30 0x7f385009ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3857903640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38500d1240 con 0x7f385009bf80 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3857903640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f38500d1e10 con 0x7f38500d3a30 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3857903640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38500d14e0 con 0x7f38500cee20 2024-04-26T21:15:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38500d3a30 0x7f385009ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38500d3a30 0x7f385009ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34090/0 (socket says 172.21.15.40:34090) 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 -- 172.21.15.40:0/2990679742 learned_addr learned my addr 172.21.15.40:0/2990679742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f385569f640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 -- 172.21.15.40:0/2990679742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 msgr2=0x7f385006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385006d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 -- 172.21.15.40:0/2990679742 >> v1:172.21.15.184:6789/0 conn(0x7f385009bf80 legacy=0x7f38501630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.422+0000 7f3854e9e640 1 -- 172.21.15.40:0/2990679742 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38500d17c0 con 0x7f38500d3a30 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3854e9e640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f385009ba40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3840009870 tx=0x7f384002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f040fbf6b3bdce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3847fff640 1 -- 172.21.15.40:0/2990679742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840037070 con 0x7f38500d3a30 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3847fff640 1 -- 172.21.15.40:0/2990679742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3840032440 con 0x7f38500d3a30 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3847fff640 1 -- 172.21.15.40:0/2990679742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840032810 con 0x7f38500d3a30 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3857903640 1 -- 172.21.15.40:0/2990679742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 msgr2=0x7f385009ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.423+0000 7f3857903640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f385009ba40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3840009870 tx=0x7f384002a2a0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 -- 172.21.15.40:0/2990679742 shutdown_connections 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f385009ba40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 --2- 172.21.15.40:0/2990679742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 -- 172.21.15.40:0/2990679742 >> 172.21.15.40:0/2990679742 conn(0x7f38500d2f10 msgr2=0x7f38500d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 -- 172.21.15.40:0/2990679742 shutdown_connections 2024-04-26T21:15:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.424+0000 7f3857903640 1 -- 172.21.15.40:0/2990679742 wait complete. 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.425+0000 7f3857903640 1 Processor -- start 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.425+0000 7f3857903640 1 -- start start 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f385009bf80 0x7f385015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38500cf200 con 0x7f38500d3a30 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f385009c7a0 con 0x7f385009bf80 2024-04-26T21:15:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38500cf380 con 0x7f38500cee20 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34100/0 (socket says 172.21.15.40:34100) 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 -- 172.21.15.40:0/3274608059 learned_addr learned my addr 172.21.15.40:0/3274608059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3854e9e640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f385569f640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f385009bf80 0x7f385015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 -- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 msgr2=0x7f385015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 -- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f385009bf80 msgr2=0x7f385015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f385009bf80 0x7f385015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 -- 172.21.15.40:0/3274608059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501f55d0 con 0x7f38500d3a30 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3855ea0640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c780 tx=0x7f384c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c017070 con 0x7f38500d3a30 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c003da0 con 0x7f38500d3a30 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0052b0 con 0x7f38500d3a30 2024-04-26T21:15:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.426+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38501f58b0 con 0x7f38500d3a30 2024-04-26T21:15:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.427+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501f5e70 con 0x7f38500d3a30 2024-04-26T21:15:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.428+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385006a290 con 0x7f38500d3a30 2024-04-26T21:15:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.428+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f384c005450 con 0x7f38500d3a30 2024-04-26T21:15:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.428+0000 7f3845ffb640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 0x7f383406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.428+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5415+0+0 (secure 0 0 0) 0x7f384c08aec0 con 0x7f38500d3a30 2024-04-26T21:15:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.429+0000 7f385569f640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 0x7f383406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:46.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.429+0000 7f385569f640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 0x7f383406edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f384800aac0 tx=0x7f3848008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.433+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f384c058e50 con 0x7f38500d3a30 2024-04-26T21:15:46.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:46.619+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f38500b4820 con 0x7f38500d3a30 2024-04-26T21:15:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.295+0000 7f3845ffb640 1 -- 172.21.15.40:0/3274608059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f384c05d150 con 0x7f38500d3a30 2024-04-26T21:15:47.297 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:15:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.299+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 msgr2=0x7f383406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.300+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 0x7f383406edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f384800aac0 tx=0x7f3848008040 comp rx=0 tx=0).stop 2024-04-26T21:15:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.300+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 msgr2=0x7f38500c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.300+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c780 tx=0x7f384c00cc40 comp rx=0 tx=0).stop 2024-04-26T21:15:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 shutdown_connections 2024-04-26T21:15:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f383406c8f0 0x7f383406edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38500d3a30 0x7f38500c8980 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38500cee20 0x7f385015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 --2- 172.21.15.40:0/3274608059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f385009bf80 0x7f385015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 >> 172.21.15.40:0/3274608059 conn(0x7f38500d2f10 msgr2=0x7f38500b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 shutdown_connections 2024-04-26T21:15:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.301+0000 7f3857903640 1 -- 172.21.15.40:0/3274608059 wait complete. 2024-04-26T21:15:47.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:15:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.426+0000 7f858be74640 1 Processor -- start 2024-04-26T21:15:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.426+0000 7f858be74640 1 -- start start 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858be74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f858416e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858be74640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8584073440 con 0x7f8584163b40 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858be74640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85840723a0 con 0x7f85840b65f0 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858be74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8584072680 con 0x7f858406fa50 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858940f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8584163b40 0x7f858406f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51382/0 (socket says 172.21.15.40:51382) 2024-04-26T21:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.427+0000 7f858940f640 1 -- 172.21.15.40:0/1860123602 learned_addr learned my addr 172.21.15.40:0/1860123602 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930953151 0 0) 0x7f8584073440 con 0x7f8584163b40 2024-04-26T21:15:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8570003520 con 0x7f8584163b40 2024-04-26T21:15:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f858a411640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f858416e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3756987935 0 0) 0x7f8570003520 con 0x7f8584163b40 2024-04-26T21:15:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 msgr2=0x7f858416e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f858416e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 >> v1:172.21.15.184:6789/0 conn(0x7f85840b65f0 legacy=0x7f85840b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584072960 con 0x7f8584163b40 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f8574001670 con 0x7f8584163b40 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 >> v1:172.21.15.40:6789/0 conn(0x7f8584163b40 legacy=0x7f858406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85700039e0 0x7f8570003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f8570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570004d40 0x7f85700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8584072680 con 0x7f85700039e0 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f857000bb20 con 0x7f8570004d40 2024-04-26T21:15:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f857000bd20 con 0x7f858406fa50 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85700039e0 0x7f8570003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f858a411640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570004d40 0x7f85700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f858940f640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f8570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 -- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 msgr2=0x7f8570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f8570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 -- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570004d40 msgr2=0x7f85700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570004d40 0x7f85700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f8589c10640 1 -- 172.21.15.40:0/1860123602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8570003520 con 0x7f85700039e0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.428+0000 7f858940f640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f858406fa50 0x7f8570004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.429+0000 7f8589c10640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85700039e0 0x7f8570003e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f857c001750 tx=0x7f8570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3f0501bfc8ed2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.429+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c001370 con 0x7f85700039e0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.429+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c0052d0 con 0x7f85700039e0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.429+0000 7f8588c0e640 1 -- 172.21.15.40:0/1860123602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c02ab20 con 0x7f85700039e0 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 -- 172.21.15.40:0/1860123602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85700039e0 msgr2=0x7f8570003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 --2- 172.21.15.40:0/1860123602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85700039e0 0x7f8570003e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f857c001750 tx=0x7f8570005830 comp rx=0 tx=0).stop 2024-04-26T21:15:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858a411640 1 -- 172.21.15.40:0/1860123602 reap_dead start 2024-04-26T21:15:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 -- 172.21.15.40:0/1860123602 shutdown_connections 2024-04-26T21:15:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 -- 172.21.15.40:0/1860123602 >> 172.21.15.40:0/1860123602 conn(0x7f85840cd8a0 msgr2=0x7f85840cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 -- 172.21.15.40:0/1860123602 shutdown_connections 2024-04-26T21:15:47.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.430+0000 7f858be74640 1 -- 172.21.15.40:0/1860123602 wait complete. 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.431+0000 7f858be74640 1 Processor -- start 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.431+0000 7f858be74640 1 -- start start 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85700039e0 0x7f8584166b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584153d20 0x7f8584163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85840cfe30 con 0x7f85840b1580 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85840cfcb0 con 0x7f8584153d20 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858be74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f858416c4c0 con 0x7f85700039e0 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34118/0 (socket says 172.21.15.40:34118) 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 -- 172.21.15.40:0/1742356262 learned_addr learned my addr 172.21.15.40:0/1742356262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 -- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85700039e0 msgr2=0x7f8584166b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85700039e0 0x7f8584166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 -- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584153d20 msgr2=0x7f8584163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584153d20 0x7f8584163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 -- 172.21.15.40:0/1742356262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858415ca50 con 0x7f85840b1580 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.432+0000 7f858a411640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f85840b6120 tx=0x7f858000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.433+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858000fe10 con 0x7f85840b1580 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.433+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8580015430 con 0x7f85840b1580 2024-04-26T21:15:47.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.433+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8584154860 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.433+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580015820 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.433+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858415c6e0 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.434+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858406a290 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.434+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85800159c0 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.434+0000 7f857a7fc640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 0x7f856806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.435+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f858008c520 con 0x7f85840b1580 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.435+0000 7f8589c10640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 0x7f856806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.435+0000 7f8589c10640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 0x7f856806ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f857c032000 tx=0x7f857c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.439+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8580019030 con 0x7f85840b1580 2024-04-26T21:15:47.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.627+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f85840b0df0 con 0x7f85840b1580 2024-04-26T21:15:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.631+0000 7f857a7fc640 1 -- 172.21.15.40:0/1742356262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f8580010040 con 0x7f85840b1580 2024-04-26T21:15:47.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 msgr2=0x7f856806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 0x7f856806ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f857c032000 tx=0x7f857c02c000 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 msgr2=0x7f8584153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f85840b6120 tx=0x7f858000f580 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 shutdown_connections 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f856806c920 0x7f856806ede0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85840b1580 0x7f8584153070 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584153d20 0x7f8584163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 --2- 172.21.15.40:0/1742356262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85700039e0 0x7f8584166b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 >> 172.21.15.40:0/1742356262 conn(0x7f85840cd8a0 msgr2=0x7f85840b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 shutdown_connections 2024-04-26T21:15:47.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.633+0000 7f858be74640 1 -- 172.21.15.40:0/1742356262 wait complete. 2024-04-26T21:15:47.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-04-26T21:15:47.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.746+0000 7f03fc237640 1 Processor -- start 2024-04-26T21:15:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 -- start start 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03f415f680 0x7f03f40d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 0x7f03f40d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f40d58f0 con 0x7f03f4163aa0 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f03f415fa60 con 0x7f03f415f680 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.747+0000 7f03fc237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03f40d3f50 con 0x7f03f40d4f20 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f9fd3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03f415f680 0x7f03f40d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f97d2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f03f4163aa0 0x7f03f40b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51396/0 (socket says 172.21.15.40:51396) 2024-04-26T21:15:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f97d2640 1 -- 172.21.15.40:0/833593852 learned_addr learned my addr 172.21.15.40:0/833593852 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03fa7d4640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 0x7f03f40d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f9fd3640 1 -- 172.21.15.40:0/833593852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 msgr2=0x7f03f40d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f9fd3640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 0x7f03f40d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f9fd3640 1 -- 172.21.15.40:0/833593852 >> v1:172.21.15.40:6789/0 conn(0x7f03f4163aa0 legacy=0x7f03f40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f9fd3640 1 -- 172.21.15.40:0/833593852 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f40b5090 con 0x7f03f415f680 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03f8fd1640 1 -- 172.21.15.40:0/833593852 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759967342 0 0) 0x7f03f40d58f0 con 0x7f03f4163aa0 2024-04-26T21:15:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.748+0000 7f03fa7d4640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 0x7f03f40d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03f9fd3640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f415f680 0x7f03f40d3820 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009d90 tx=0x7f03e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcea048e5c26d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03f8fd1640 1 -- 172.21.15.40:0/833593852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4007740 con 0x7f03f415f680 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03f8fd1640 1 -- 172.21.15.40:0/833593852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e40078e0 con 0x7f03f415f680 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03f8fd1640 1 -- 172.21.15.40:0/833593852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4007cb0 con 0x7f03f415f680 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 -- 172.21.15.40:0/833593852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f415f680 msgr2=0x7f03f40d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f415f680 0x7f03f40d3820 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009d90 tx=0x7f03e402a660 comp rx=0 tx=0).stop 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 -- 172.21.15.40:0/833593852 shutdown_connections 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f40d4f20 0x7f03f40d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 --2- 172.21.15.40:0/833593852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f415f680 0x7f03f40d3820 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 -- 172.21.15.40:0/833593852 >> 172.21.15.40:0/833593852 conn(0x7f03f408eca0 msgr2=0x7f03f409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 -- 172.21.15.40:0/833593852 shutdown_connections 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.749+0000 7f03fc237640 1 -- 172.21.15.40:0/833593852 wait complete. 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.750+0000 7f03fc237640 1 Processor -- start 2024-04-26T21:15:47.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 -- start start 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f40d4f20 0x7f03f41f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f415f680 0x7f03f40b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4163f30 con 0x7f03f4163aa0 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40b6280 con 0x7f03f40d4f20 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fc237640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03f40cfd80 con 0x7f03f415f680 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fa7d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fa7d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34132/0 (socket says 172.21.15.40:34132) 2024-04-26T21:15:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03fa7d4640 1 -- 172.21.15.40:0/1163473007 learned_addr learned my addr 172.21.15.40:0/1163473007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.751+0000 7f03f97d2640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f415f680 0x7f03f40b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03f9fd3640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f40d4f20 0x7f03f41f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 -- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f415f680 msgr2=0x7f03f40b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f415f680 0x7f03f40b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 -- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f40d4f20 msgr2=0x7f03f41f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f40d4f20 0x7f03f41f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 -- 172.21.15.40:0/1163473007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f4160940 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fa7d4640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c7f0 tx=0x7f03ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec015070 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec007820 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41f5ce0 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec007c50 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.752+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41600a0 con 0x7f03f4163aa0 2024-04-26T21:15:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.753+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f406a290 con 0x7f03f4163aa0 2024-04-26T21:15:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.753+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03ec018070 con 0x7f03f4163aa0 2024-04-26T21:15:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.754+0000 7f03f27fc640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 0x7f03e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.754+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f03ec08b880 con 0x7f03f4163aa0 2024-04-26T21:15:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.754+0000 7f03f9fd3640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 0x7f03e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.755+0000 7f03f9fd3640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 0x7f03e006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.758+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03ec0599f0 con 0x7f03f4163aa0 2024-04-26T21:15:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.955+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7f03f40b4cd0 con 0x7f03f4163aa0 2024-04-26T21:15:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.958+0000 7f03f27fc640 1 -- 172.21.15.40:0/1163473007 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v23) ==== 106+0+0 (secure 0 0 0) 0x7f03ec05dcf0 con 0x7f03f4163aa0 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 msgr2=0x7f03e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 0x7f03e006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402f000 comp rx=0 tx=0).stop 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 msgr2=0x7f03f41f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c7f0 tx=0x7f03ec00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 shutdown_connections 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f03e006c8f0 0x7f03e006edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f4163aa0 0x7f03f41f57a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03f415f680 0x7f03f40b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 --2- 172.21.15.40:0/1163473007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03f40d4f20 0x7f03f41f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:47.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 >> 172.21.15.40:0/1163473007 conn(0x7f03f408eca0 msgr2=0x7f03f40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:47.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 shutdown_connections 2024-04-26T21:15:47.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:47.960+0000 7f03fc237640 1 -- 172.21.15.40:0/1163473007 wait complete. 2024-04-26T21:15:47.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-04-26T21:15:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.072+0000 7f335409c640 1 Processor -- start 2024-04-26T21:15:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.072+0000 7f335409c640 1 -- start start 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f335409c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f335409c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f334c160640 0x7f334c1636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f335409c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c0cfa80 con 0x7f334c163be0 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f335409c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f334c0cfd60 con 0x7f334c160640 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f335409c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0d0040 con 0x7f334c148a30 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351637640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f334c160640 0x7f334c1636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3352639640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f334c163be0 0x7f334c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51412/0 (socket says 172.21.15.40:51412) 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3352639640 1 -- 172.21.15.40:0/2456772532 learned_addr learned my addr 172.21.15.40:0/2456772532 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351e38640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351637640 1 -- 172.21.15.40:0/2456772532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 msgr2=0x7f334c073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351637640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351637640 1 -- 172.21.15.40:0/2456772532 >> v1:172.21.15.40:6789/0 conn(0x7f334c163be0 legacy=0x7f334c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3350e36640 1 -- 172.21.15.40:0/2456772532 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247265790 0 0) 0x7f334c0cfa80 con 0x7f334c163be0 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.073+0000 7f3351637640 1 -- 172.21.15.40:0/2456772532 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c162ea0 con 0x7f334c160640 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f3351637640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c160640 0x7f334c1636a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f333c0097a0 tx=0x7f333c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5818defa365c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f3350e36640 1 -- 172.21.15.40:0/2456772532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c037070 con 0x7f334c160640 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f3350e36640 1 -- 172.21.15.40:0/2456772532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c02ae50 con 0x7f334c160640 2024-04-26T21:15:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f3350e36640 1 -- 172.21.15.40:0/2456772532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c032660 con 0x7f334c160640 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f335409c640 1 -- 172.21.15.40:0/2456772532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c160640 msgr2=0x7f334c1636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.074+0000 7f335409c640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c160640 0x7f334c1636a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f333c0097a0 tx=0x7f333c02a100 comp rx=0 tx=0).stop 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 -- 172.21.15.40:0/2456772532 shutdown_connections 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c160640 0x7f334c1636a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 --2- 172.21.15.40:0/2456772532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 -- 172.21.15.40:0/2456772532 >> 172.21.15.40:0/2456772532 conn(0x7f334c0ce8b0 msgr2=0x7f334c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 -- 172.21.15.40:0/2456772532 shutdown_connections 2024-04-26T21:15:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.075+0000 7f335409c640 1 -- 172.21.15.40:0/2456772532 wait complete. 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.076+0000 7f335409c640 1 Processor -- start 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.076+0000 7f335409c640 1 -- start start 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 0x7f334c15ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f334c074590 con 0x7f334c163be0 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f334c074410 con 0x7f334c160640 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f334c148e10 con 0x7f334c148a30 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34154/0 (socket says 172.21.15.40:34154) 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 -- 172.21.15.40:0/623004411 learned_addr learned my addr 172.21.15.40:0/623004411 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3351e38640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3351637640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 0x7f334c15ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 -- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 msgr2=0x7f334c142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 -- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 msgr2=0x7f334c15ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 0x7f334c15ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 -- 172.21.15.40:0/623004411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c0b7d40 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3351637640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 0x7f334c15ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f3352639640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f334400c730 tx=0x7f334400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344018070 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334400ce40 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c0b8020 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334400a860 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.077+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c0affe0 con 0x7f334c163be0 2024-04-26T21:15:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.078+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c06a290 con 0x7f334c163be0 2024-04-26T21:15:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.079+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f334401b070 con 0x7f334c163be0 2024-04-26T21:15:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.079+0000 7f33427fc640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 0x7f333006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.079+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f334408ada0 con 0x7f334c163be0 2024-04-26T21:15:48.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.080+0000 7f3351e38640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 0x7f333006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:48.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.080+0000 7f3351e38640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 0x7f333006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f334800aac0 tx=0x7f3348008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.084+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3344058e80 con 0x7f334c163be0 2024-04-26T21:15:48.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:48.340+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f334c161bd0 con 0x7f334c163be0 2024-04-26T21:15:49.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.303+0000 7f33427fc640 1 -- 172.21.15.40:0/623004411 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v218) ==== 148+0+0 (secure 0 0 0) 0x7f334405d180 con 0x7f334c163be0 2024-04-26T21:15:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 msgr2=0x7f333006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 0x7f333006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f334800aac0 tx=0x7f3348008040 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 msgr2=0x7f334c144530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f334400c730 tx=0x7f334400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 shutdown_connections 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f333006c8f0 0x7f333006edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f334c163be0 0x7f334c144530 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f334c160640 0x7f334c15ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 --2- 172.21.15.40:0/623004411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f334c148a30 0x7f334c142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 >> 172.21.15.40:0/623004411 conn(0x7f334c0ce8b0 msgr2=0x7f334c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 shutdown_connections 2024-04-26T21:15:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.306+0000 7f335409c640 1 -- 172.21.15.40:0/623004411 wait complete. 2024-04-26T21:15:49.321 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2024-04-26T21:15:49.321 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2024-04-26T21:15:49.321 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:15:49.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.428+0000 7fd8fa9b4640 1 Processor -- start 2024-04-26T21:15:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.428+0000 7fd8fa9b4640 1 -- start start 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8f4146980 0x7fd8f4073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8f4161850 0x7fd8f40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 0x7fd8f415db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f415e5c0 con 0x7fd8f4146980 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f41613c0 con 0x7fd8f4161850 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8fa9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f415e080 con 0x7fd8f415d6b0 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8f4146980 0x7fd8f4073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8f4146980 0x7fd8f4073bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34158/0 (socket says 172.21.15.40:34158) 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1159529505 learned_addr learned my addr 172.21.15.40:0/1159529505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f8f51640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 0x7fd8f415db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1159529505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 msgr2=0x7fd8f415db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 0x7fd8f415db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f37fe640 1 --2- 172.21.15.40:0/1159529505 >> v2:172.21.15.184:3300/0 conn(0x7fd8f4161850 0x7fd8f40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1159529505 >> v2:172.21.15.184:3300/0 conn(0x7fd8f4161850 msgr2=0x7fd8f40740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/1159529505 >> v2:172.21.15.184:3300/0 conn(0x7fd8f4161850 0x7fd8f40740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f3fff640 1 -- 172.21.15.40:0/1159529505 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f415e2e0 con 0x7fd8f4146980 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.429+0000 7fd8f8f51640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 0x7fd8f415db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4146980 0x7fd8f4073bb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40710a0 tx=0x7fd8e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8c068a13d65506 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8f2ffd640 1 -- 172.21.15.40:0/1159529505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4037070 con 0x7fd8f4146980 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8f2ffd640 1 -- 172.21.15.40:0/1159529505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4032440 con 0x7fd8f4146980 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8f2ffd640 1 -- 172.21.15.40:0/1159529505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4032810 con 0x7fd8f4146980 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/1159529505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4146980 msgr2=0x7fd8f4073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4146980 0x7fd8f4073bb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40710a0 tx=0x7fd8e402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/1159529505 shutdown_connections 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f415d6b0 0x7fd8f415db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.430+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/1159529505 >> v2:172.21.15.184:3300/0 conn(0x7fd8f4161850 0x7fd8f40740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.431+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/1159529505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4146980 0x7fd8f4073bb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.431+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/1159529505 >> 172.21.15.40:0/1159529505 conn(0x7fd8f40ce8b0 msgr2=0x7fd8f40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.431+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/1159529505 shutdown_connections 2024-04-26T21:15:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.431+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/1159529505 wait complete. 2024-04-26T21:15:49.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.432+0000 7fd8fa9b4640 1 Processor -- start 2024-04-26T21:15:49.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.432+0000 7fd8fa9b4640 1 -- start start 2024-04-26T21:15:49.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f4146980 0x7fd8f41f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f415d6b0 0x7fd8f40cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f40b5120 con 0x7fd8f4161850 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4161050 con 0x7fd8f415d6b0 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f4146d60 con 0x7fd8f4146980 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34170/0 (socket says 172.21.15.40:34170) 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 -- 172.21.15.40:0/4037584163 learned_addr learned my addr 172.21.15.40:0/4037584163 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f4146980 0x7fd8f41f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f37fe640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f415d6b0 0x7fd8f40cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 -- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f4146980 msgr2=0x7fd8f41f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f4146980 0x7fd8f41f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 -- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f415d6b0 msgr2=0x7fd8f40cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f415d6b0 0x7fd8f40cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 -- 172.21.15.40:0/4037584163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f41f9310 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f8f51640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00cd10 tx=0x7fd8ec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00f680 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00f840 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f41f95f0 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00fc30 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.433+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41f9bb0 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.434+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f40cec90 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.435+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8ec0213f0 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.435+0000 7fd8f0ff9640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.435+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7fd8ec08c460 con 0x7fd8f4161850 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.436+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.436+0000 7fd8f3fff640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 0x7fd8e006edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034000 tx=0x7fd8e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.440+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8f40cec90 con 0x7fd8f4161850 2024-04-26T21:15:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.625+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8f4069be0 con 0x7fd8f4161850 2024-04-26T21:15:49.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.626+0000 7fd8f0ff9640 1 -- 172.21.15.40:0/4037584163 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13021 (secure 0 0 0) 0x7fd8f4069be0 con 0x7fd8f4161850 2024-04-26T21:15:49.627 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:49.627 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":75,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:47.291412+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/2174268870":"2024-04-27T21:15:44.157462+0000","172.21.15.40:6833/2174268870":"2024-04-27T21:15:44.157462+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 msgr2=0x7fd8e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 0x7fd8e006edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034000 tx=0x7fd8e402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 msgr2=0x7fd8f4141b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00cd10 tx=0x7fd8ec00a650 comp rx=0 tx=0).stop 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 shutdown_connections 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd8e006c8f0 0x7fd8e006edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8f4161850 0x7fd8f4141b60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f415d6b0 0x7fd8f40cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 --2- 172.21.15.40:0/4037584163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f4146980 0x7fd8f41f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 >> 172.21.15.40:0/4037584163 conn(0x7fd8f40ce8b0 msgr2=0x7fd8f40b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 shutdown_connections 2024-04-26T21:15:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.628+0000 7fd8fa9b4640 1 -- 172.21.15.40:0/4037584163 wait complete. 2024-04-26T21:15:49.640 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:49.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.743+0000 7f566752b640 1 Processor -- start 2024-04-26T21:15:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.743+0000 7f566752b640 1 -- start start 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f566752b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56600d0010 0x7f566006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f566752b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 0x7f5660163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f566752b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566009c700 con 0x7f56600b3c20 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f566752b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f56600b4480 con 0x7f56600d0010 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f566752b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f566006e050 con 0x7f56600b33e0 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56600d0010 0x7f566006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56600d0010 0x7f566006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34180/0 (socket says 172.21.15.40:34180) 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 -- 172.21.15.40:0/314047672 learned_addr learned my addr 172.21.15.40:0/314047672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f5665ac8640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 0x7f5660163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 -- 172.21.15.40:0/314047672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 msgr2=0x7f5660163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 0x7f5660163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 -- 172.21.15.40:0/314047672 >> v1:172.21.15.184:6789/0 conn(0x7f56600b3c20 legacy=0x7f56600b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f56652c7640 1 -- 172.21.15.40:0/314047672 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56600d1c50 con 0x7f56600d0010 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.744+0000 7f5665ac8640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 0x7f5660163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.745+0000 7f56652c7640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f566006db10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5658009870 tx=0x7f565802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1564f78e59ddd24a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.745+0000 7f565ffff640 1 -- 172.21.15.40:0/314047672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658037070 con 0x7f56600d0010 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.745+0000 7f565ffff640 1 -- 172.21.15.40:0/314047672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5658032440 con 0x7f56600d0010 2024-04-26T21:15:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.745+0000 7f565ffff640 1 -- 172.21.15.40:0/314047672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658032810 con 0x7f56600d0010 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 -- 172.21.15.40:0/314047672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 msgr2=0x7f566006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f566006db10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5658009870 tx=0x7f565802a200 comp rx=0 tx=0).stop 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 -- 172.21.15.40:0/314047672 shutdown_connections 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b33e0 0x7f5660163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 --2- 172.21.15.40:0/314047672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f566006db10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 -- 172.21.15.40:0/314047672 >> 172.21.15.40:0/314047672 conn(0x7f566009ba40 msgr2=0x7f566009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 -- 172.21.15.40:0/314047672 shutdown_connections 2024-04-26T21:15:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.746+0000 7f566752b640 1 -- 172.21.15.40:0/314047672 wait complete. 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.747+0000 7f566752b640 1 Processor -- start 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.747+0000 7f566752b640 1 -- start start 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 0x7f5660155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b3c20 0x7f56601595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56600b39d0 con 0x7f56600d0010 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56600b3850 con 0x7f56600b33e0 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f566752b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56601637c0 con 0x7f56600b3c20 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34190/0 (socket says 172.21.15.40:34190) 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 -- 172.21.15.40:0/3501589763 learned_addr learned my addr 172.21.15.40:0/3501589763 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f56652c7640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 0x7f5660155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5664ac6640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b3c20 0x7f56601595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 -- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b3c20 msgr2=0x7f56601595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b3c20 0x7f56601595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 -- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 msgr2=0x7f5660155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 0x7f5660155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 -- 172.21.15.40:0/3501589763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5660163350 con 0x7f56600d0010 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f56652c7640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 0x7f5660155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f5665ac8640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f565000cef0 tx=0x7f565000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.748+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565001a070 con 0x7f56600d0010 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.749+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565000d860 con 0x7f56600d0010 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.749+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5660160f10 con 0x7f56600d0010 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.749+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000dc50 con 0x7f56600d0010 2024-04-26T21:15:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.749+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5660161490 con 0x7f56600d0010 2024-04-26T21:15:49.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.750+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566006a290 con 0x7f56600d0010 2024-04-26T21:15:49.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.750+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f565001d070 con 0x7f56600d0010 2024-04-26T21:15:49.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.750+0000 7f565dffb640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.750+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f565008c090 con 0x7f56600d0010 2024-04-26T21:15:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.751+0000 7f56652c7640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.751+0000 7f56652c7640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 0x7f564c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5658034000 tx=0x7f565802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.755+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f565005a910 con 0x7f56600d0010 2024-04-26T21:15:49.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.939+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5660068e00 con 0x7f56600d0010 2024-04-26T21:15:49.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.939+0000 7f565dffb640 1 -- 172.21.15.40:0/3501589763 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f565005a210 con 0x7f56600d0010 2024-04-26T21:15:49.940 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 msgr2=0x7f564c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 0x7f564c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5658034000 tx=0x7f565802e000 comp rx=0 tx=0).stop 2024-04-26T21:15:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 msgr2=0x7f56601f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f565000cef0 tx=0x7f565000a710 comp rx=0 tx=0).stop 2024-04-26T21:15:49.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 shutdown_connections 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56600d0010 0x7f56601f4d80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56600b3c20 0x7f56601595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 --2- 172.21.15.40:0/3501589763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56600b33e0 0x7f5660155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 >> 172.21.15.40:0/3501589763 conn(0x7f566009ba40 msgr2=0x7f566015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 shutdown_connections 2024-04-26T21:15:49.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:49.941+0000 7f566752b640 1 -- 172.21.15.40:0/3501589763 wait complete. 2024-04-26T21:15:49.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-04-26T21:15:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 Processor -- start 2024-04-26T21:15:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 -- start start 2024-04-26T21:15:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 0x7f15300cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15300d3ee0 0x7f15300b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15300d03a0 con 0x7f153015f750 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f15300b4d10 con 0x7f15300d3ee0 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.054+0000 7f15370c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15300d0680 con 0x7f153015f370 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15300d3ee0 0x7f15300b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15300d3ee0 0x7f15300b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34196/0 (socket says 172.21.15.40:34196) 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 -- 172.21.15.40:0/335217823 learned_addr learned my addr 172.21.15.40:0/335217823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1534e60640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 0x7f15300cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 -- 172.21.15.40:0/335217823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 msgr2=0x7f15300cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 0x7f15300cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 -- 172.21.15.40:0/335217823 >> v1:172.21.15.184:6789/0 conn(0x7f153015f750 legacy=0x7f15300d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1535661640 1 -- 172.21.15.40:0/335217823 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15300d0960 con 0x7f15300d3ee0 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.055+0000 7f1534e60640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 0x7f15300cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f1535661640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15300d3ee0 0x7f15300b47d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1520009870 tx=0x7f152002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f75c9fcde8b990 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f152f7fe640 1 -- 172.21.15.40:0/335217823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520037070 con 0x7f15300d3ee0 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f152f7fe640 1 -- 172.21.15.40:0/335217823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1520032440 con 0x7f15300d3ee0 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f152f7fe640 1 -- 172.21.15.40:0/335217823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520032810 con 0x7f15300d3ee0 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 -- 172.21.15.40:0/335217823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15300d3ee0 msgr2=0x7f15300b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15300d3ee0 0x7f15300b47d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1520009870 tx=0x7f152002a180 comp rx=0 tx=0).stop 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 -- 172.21.15.40:0/335217823 shutdown_connections 2024-04-26T21:15:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15300d3ee0 0x7f15300b47d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 --2- 172.21.15.40:0/335217823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f370 0x7f15300cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 -- 172.21.15.40:0/335217823 >> 172.21.15.40:0/335217823 conn(0x7f153008ec10 msgr2=0x7f153009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 -- 172.21.15.40:0/335217823 shutdown_connections 2024-04-26T21:15:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.056+0000 7f15370c4640 1 -- 172.21.15.40:0/335217823 wait complete. 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.057+0000 7f15370c4640 1 Processor -- start 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 -- start start 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300d3ee0 0x7f15300ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f750 0x7f15300cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15300b51c0 con 0x7f153015f370 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15300d0f20 con 0x7f15300d3ee0 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.058+0000 7f15370c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f15300b5340 con 0x7f153015f750 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34204/0 (socket says 172.21.15.40:34204) 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 -- 172.21.15.40:0/3382196792 learned_addr learned my addr 172.21.15.40:0/3382196792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f1534e60640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300d3ee0 0x7f15300ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f1535661640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f750 0x7f15300cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 -- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f750 msgr2=0x7f15300cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f750 0x7f15300cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 -- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300d3ee0 msgr2=0x7f15300ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300d3ee0 0x7f15300ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 -- 172.21.15.40:0/3382196792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15300c9fc0 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152ffff640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f151800b3f0 tx=0x7f151800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518015020 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1518011830 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15300ca2a0 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518011c20 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.059+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15300ca860 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.060+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153006a290 con 0x7f153015f370 2024-04-26T21:15:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.061+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1518018020 con 0x7f153015f370 2024-04-26T21:15:50.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.061+0000 7f152d7fa640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 0x7f151006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:50.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.061+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f151808b3a0 con 0x7f153015f370 2024-04-26T21:15:50.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.061+0000 7f1534e60640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 0x7f151006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:50.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.062+0000 7f1534e60640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 0x7f151006edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f152800d620 tx=0x7f1528007810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:50.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.066+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f151801f030 con 0x7f153015f370 2024-04-26T21:15:50.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:50.326+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f15300cafe0 con 0x7f153015f370 2024-04-26T21:15:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.312+0000 7f152d7fa640 1 -- 172.21.15.40:0/3382196792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v220) ==== 163+0+0 (secure 0 0 0) 0x7f1518059480 con 0x7f153015f370 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.314+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 msgr2=0x7f151006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.314+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 0x7f151006edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f152800d620 tx=0x7f1528007810 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.314+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 msgr2=0x7f15300b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.314+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f151800b3f0 tx=0x7f151800b8b0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 shutdown_connections 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f151006c8f0 0x7f151006edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f153015f750 0x7f15300cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f153015f370 0x7f15300b2d70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 --2- 172.21.15.40:0/3382196792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15300d3ee0 0x7f15300ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 >> 172.21.15.40:0/3382196792 conn(0x7f153008ec10 msgr2=0x7f153006db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 shutdown_connections 2024-04-26T21:15:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.315+0000 7f15370c4640 1 -- 172.21.15.40:0/3382196792 wait complete. 2024-04-26T21:15:51.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:15:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.438+0000 7f6d0efaf640 1 Processor -- start 2024-04-26T21:15:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 -- start start 2024-04-26T21:15:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0815f4f0 0x7f6d08073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0815c530 con 0x7f6d08074030 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0815c810 con 0x7f6d0815f4f0 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.439+0000 7f6d0efaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0815caf0 con 0x7f6d0815b930 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d0cd4b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0815f4f0 0x7f6d08073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d0cd4b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:58028/0 (socket says 172.21.15.40:58028) 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0815f4f0 0x7f6d08073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:39548/0 (socket says 172.21.15.40:39548) 2024-04-26T21:15:51.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d0cd4b640 1 -- 172.21.15.40:0/2553635247 learned_addr learned my addr 172.21.15.40:0/2553635247 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 -- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 msgr2=0x7f6d0815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 -- 172.21.15.40:0/2553635247 >> v1:172.21.15.184:6789/0 conn(0x7f6d08074030 legacy=0x7f6d080720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d07fff640 1 -- 172.21.15.40:0/2553635247 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0815cdd0 con 0x7f6d0815f4f0 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.440+0000 7f6d0cd4b640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.441+0000 7f6d07fff640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815f4f0 0x7f6d08073af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097a0 tx=0x7f6cf402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70fef7dfa28cc141 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.441+0000 7f6d077fe640 1 -- 172.21.15.40:0/2553635247 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4037070 con 0x7f6d0815f4f0 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.441+0000 7f6d077fe640 1 -- 172.21.15.40:0/2553635247 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf402ad10 con 0x7f6d0815f4f0 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.441+0000 7f6d077fe640 1 -- 172.21.15.40:0/2553635247 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4032660 con 0x7f6d0815f4f0 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 -- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815f4f0 msgr2=0x7f6d08073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815f4f0 0x7f6d08073af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097a0 tx=0x7f6cf402a080 comp rx=0 tx=0).stop 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 -- 172.21.15.40:0/2553635247 shutdown_connections 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815f4f0 0x7f6d08073af0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/2553635247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815b930 0x7f6d0815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 -- 172.21.15.40:0/2553635247 >> 172.21.15.40:0/2553635247 conn(0x7f6d080cd8a0 msgr2=0x7f6d080cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 -- 172.21.15.40:0/2553635247 shutdown_connections 2024-04-26T21:15:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.442+0000 7f6d0efaf640 1 -- 172.21.15.40:0/2553635247 wait complete. 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.443+0000 7f6d0efaf640 1 Processor -- start 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.443+0000 7f6d0efaf640 1 -- start start 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815b930 0x7f6d081f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815f4f0 0x7f6d0814aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0815c250 con 0x7f6d08074030 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08074480 con 0x7f6d0815b930 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0efaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0806fdd0 con 0x7f6d0815f4f0 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40102/0 (socket says 172.21.15.40:40102) 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 -- 172.21.15.40:0/572024605 learned_addr learned my addr 172.21.15.40:0/572024605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d07fff640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815b930 0x7f6d081f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0d54c640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815f4f0 0x7f6d0814aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 -- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815f4f0 msgr2=0x7f6d0814aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815f4f0 0x7f6d0814aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 -- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815b930 msgr2=0x7f6d081f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815b930 0x7f6d081f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.444+0000 7f6d0cd4b640 1 -- 172.21.15.40:0/572024605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d081fa390 con 0x7f6d08074030 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d0cd4b640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9d0 tx=0x7f6cfc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc017070 con 0x7f6d08074030 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00ca60 con 0x7f6d08074030 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0814b3f0 con 0x7f6d08074030 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00ce10 con 0x7f6d08074030 2024-04-26T21:15:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.445+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d080b13f0 con 0x7f6d08074030 2024-04-26T21:15:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.446+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0806a290 con 0x7f6d08074030 2024-04-26T21:15:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.446+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cfc01a070 con 0x7f6d08074030 2024-04-26T21:15:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.447+0000 7f6d057fa640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 0x7f6cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.447+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f6cfc08b7e0 con 0x7f6d08074030 2024-04-26T21:15:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.447+0000 7f6d07fff640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 0x7f6cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.448+0000 7f6d07fff640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 0x7f6cec06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4034000 tx=0x7f6cf402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.451+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d0814b3f0 con 0x7f6d08074030 2024-04-26T21:15:51.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.637+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6d08159d10 con 0x7f6d08074030 2024-04-26T21:15:51.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.642+0000 7f6d057fa640 1 -- 172.21.15.40:0/572024605 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f6d081fa390 con 0x7f6d08074030 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 msgr2=0x7f6cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 0x7f6cec06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4034000 tx=0x7f6cf402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 msgr2=0x7f6d080b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9d0 tx=0x7f6cfc00ee90 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 shutdown_connections 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cec06c8f0 0x7f6cec06edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0815f4f0 0x7f6d0814aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0815b930 0x7f6d081f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 --2- 172.21.15.40:0/572024605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08074030 0x7f6d080b0ac0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 >> 172.21.15.40:0/572024605 conn(0x7f6d080cd8a0 msgr2=0x7f6d080b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 shutdown_connections 2024-04-26T21:15:51.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.644+0000 7f6d0efaf640 1 -- 172.21.15.40:0/572024605 wait complete. 2024-04-26T21:15:51.656 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:15:51.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:51.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:15:51.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:15:51.744 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:51.744 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:51.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:15:51.760 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:51.760 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:51.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:15:51.778 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:15:51.778 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:51.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:15:51.816 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:15:51.816 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:15:51.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:15:51.835 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:15:51.835 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:15:51.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.955+0000 7ffb30bda640 1 Processor -- start 2024-04-26T21:15:51.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.955+0000 7ffb30bda640 1 -- start start 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb30bda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb2c0d3ee0 0x7ffb2c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb30bda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb2c15f750 0x7ffb2c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb30bda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2c0d03a0 con 0x7ffb2c15f370 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb30bda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2c0b4d10 con 0x7ffb2c15f750 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb30bda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2c0d0680 con 0x7ffb2c0d3ee0 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb2ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb2c15f750 0x7ffb2c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb2a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb2c15f370 0x7ffb2c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55544/0 (socket says 172.21.15.40:55544) 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.956+0000 7ffb2a59c640 1 -- 172.21.15.40:0/4011156563 learned_addr learned my addr 172.21.15.40:0/4011156563 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb29d9b640 1 --2- 172.21.15.40:0/4011156563 >> v2:172.21.15.184:3300/0 conn(0x7ffb2c0d3ee0 0x7ffb2c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2ad9d640 1 -- 172.21.15.40:0/4011156563 >> v2:172.21.15.184:3300/0 conn(0x7ffb2c0d3ee0 msgr2=0x7ffb2c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2ad9d640 1 --2- 172.21.15.40:0/4011156563 >> v2:172.21.15.184:3300/0 conn(0x7ffb2c0d3ee0 0x7ffb2c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2ad9d640 1 -- 172.21.15.40:0/4011156563 >> v1:172.21.15.40:6789/0 conn(0x7ffb2c15f370 legacy=0x7ffb2c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2ad9d640 1 -- 172.21.15.40:0/4011156563 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c0d0960 con 0x7ffb2c15f750 2024-04-26T21:15:51.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2959a640 1 -- 172.21.15.40:0/4011156563 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291579538 0 0) 0x7ffb2c0d03a0 con 0x7ffb2c15f370 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.957+0000 7ffb2ad9d640 1 --2- 172.21.15.40:0/4011156563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b47d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d90 tx=0x7ffb1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6856d28596d4de5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb2959a640 1 -- 172.21.15.40:0/4011156563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c007740 con 0x7ffb2c15f750 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb2959a640 1 -- 172.21.15.40:0/4011156563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0078e0 con 0x7ffb2c15f750 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb2959a640 1 -- 172.21.15.40:0/4011156563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c007cb0 con 0x7ffb2c15f750 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 -- 172.21.15.40:0/4011156563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f750 msgr2=0x7ffb2c0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 --2- 172.21.15.40:0/4011156563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b47d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d90 tx=0x7ffb1c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 -- 172.21.15.40:0/4011156563 shutdown_connections 2024-04-26T21:15:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 --2- 172.21.15.40:0/4011156563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b47d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 --2- 172.21.15.40:0/4011156563 >> v2:172.21.15.184:3300/0 conn(0x7ffb2c0d3ee0 0x7ffb2c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 -- 172.21.15.40:0/4011156563 >> 172.21.15.40:0/4011156563 conn(0x7ffb2c08ec10 msgr2=0x7ffb2c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 -- 172.21.15.40:0/4011156563 shutdown_connections 2024-04-26T21:15:51.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.958+0000 7ffb30bda640 1 -- 172.21.15.40:0/4011156563 wait complete. 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 Processor -- start 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 -- start start 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 0x7ffb2c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0b51c0 con 0x7ffb2c15f370 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0d0f20 con 0x7ffb2c15f750 2024-04-26T21:15:51.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.960+0000 7ffb30bda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c0b5340 con 0x7ffb2c0d3ee0 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40112/0 (socket says 172.21.15.40:40112) 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 -- 172.21.15.40:0/1870672671 learned_addr learned my addr 172.21.15.40:0/1870672671 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb2a59c640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 0x7ffb2c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb2ad9d640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 -- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 msgr2=0x7ffb2c0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 0x7ffb2c0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 -- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 msgr2=0x7ffb2c0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 -- 172.21.15.40:0/1870672671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c1f50a0 con 0x7ffb2c15f370 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb2a59c640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 0x7ffb2c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb2ad9d640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb29d9b640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400d800 tx=0x7ffb1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1400b6c0 con 0x7ffb2c15f370 2024-04-26T21:15:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb14019070 con 0x7ffb2c15f370 2024-04-26T21:15:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1400baa0 con 0x7ffb2c15f370 2024-04-26T21:15:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c161e50 con 0x7ffb2c15f370 2024-04-26T21:15:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.961+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c162410 con 0x7ffb2c15f370 2024-04-26T21:15:51.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.962+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2c06a290 con 0x7ffb2c15f370 2024-04-26T21:15:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.963+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb1400bc40 con 0x7ffb2c15f370 2024-04-26T21:15:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.963+0000 7ffb22ffd640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 0x7ffb0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.963+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7ffb1408bc50 con 0x7ffb2c15f370 2024-04-26T21:15:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.964+0000 7ffb2a59c640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 0x7ffb0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:51.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.964+0000 7ffb2a59c640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 0x7ffb0c06ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffb24000f60 tx=0x7ffb24007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:51.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:51.968+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb14091020 con 0x7ffb2c15f370 2024-04-26T21:15:52.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.194+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb2c0d4cc0 con 0x7ffb2c15f370 2024-04-26T21:15:52.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.194+0000 7ffb22ffd640 1 -- 172.21.15.40:0/1870672671 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3860 (secure 0 0 0) 0x7ffb1405a380 con 0x7ffb2c15f370 2024-04-26T21:15:52.196 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:52.196 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/672533202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":672533202},{"type":"v1","addr":"172.21.15.184:6835","nonce":672533202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/798898708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":798898708},{"type":"v1","addr":"172.21.15.184:6837","nonce":798898708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6450,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/12641439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":12641439},{"type":"v1","addr":"172.21.15.40:6834","nonce":12641439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":6810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/109822021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":109822021},{"type":"v1","addr":"172.21.15.40:6835","nonce":109822021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[]} 2024-04-26T21:15:52.196 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 40 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 msgr2=0x7ffb0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 0x7ffb0c06ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffb24000f60 tx=0x7ffb24007c10 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 msgr2=0x7ffb2c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400d800 tx=0x7ffb1400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 shutdown_connections 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffb0c06c920 0x7ffb0c06ede0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2c15f750 0x7ffb2c0b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb2c15f370 0x7ffb2c0b60a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 --2- 172.21.15.40:0/1870672671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb2c0d3ee0 0x7ffb2c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 >> 172.21.15.40:0/1870672671 conn(0x7ffb2c08ec10 msgr2=0x7ffb2c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 shutdown_connections 2024-04-26T21:15:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.196+0000 7ffb30bda640 1 -- 172.21.15.40:0/1870672671 wait complete. 2024-04-26T21:15:52.209 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:15:52.209 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:15:52.209 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:15:52.209 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:52.209 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:15:52.209 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:15:52.210 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:15:52.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:52.210 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:15:52.210 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:15:52.210 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:15:52.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:52.210 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:15:52.210 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:15:52.211 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:15:52.211 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:15:52.218 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:15:52.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:15:52.219 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:15:52.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:15:52.222 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:15:52.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:15:52.223 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:15:52.223 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:15:52.261 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:15:52.261 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:52.262 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:15:52.262 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:52.265 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:15:52.265 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:52.266 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:15:52.266 DEBUG:teuthology.parallel:result is None 2024-04-26T21:15:52.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:15:52.339 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:15:52.339 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:15:52.346 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:15:52.346 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:15:52.347 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-04-26T21:15:52.346+0000 7f1c3c9b1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:15:52.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.370+0000 7f060927e640 1 Processor -- start 2024-04-26T21:15:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.370+0000 7f060927e640 1 -- start start 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f060927e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f06040cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f060927e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06040d05b0 con 0x7f060413ff70 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f060927e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06040d0890 con 0x7f0604140350 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f060927e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f060406e450 con 0x7f0604146790 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f0602ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f060413ff70 0x7f0604146080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55572/0 (socket says 172.21.15.40:55572) 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.371+0000 7f0602ffd640 1 -- 172.21.15.40:0/2493433601 learned_addr learned my addr 172.21.15.40:0/2493433601 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06037fe640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f06040cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437120698 0 0) 0x7f06040d05b0 con 0x7f060413ff70 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05ec003520 con 0x7f060413ff70 2024-04-26T21:15:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455307436 0 0) 0x7f05ec003520 con 0x7f060413ff70 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 msgr2=0x7f06040cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f06040cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 >> v1:172.21.15.184:6789/0 conn(0x7f0604140350 legacy=0x7f06040cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060406e730 con 0x7f060413ff70 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f05f8001670 con 0x7f060413ff70 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06037fe640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f06040cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 >> v1:172.21.15.40:6789/0 conn(0x7f060413ff70 legacy=0x7f0604146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f05ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05ec003f20 0x7f05ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ec004d40 0x7f05ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f060406e450 con 0x7f05ec003f20 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00bb00 con 0x7f05ec004d40 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05ec00bd00 con 0x7f0604146790 2024-04-26T21:15:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06027fc640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05ec003f20 0x7f05ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f0602ffd640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f05ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06037fe640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ec004d40 0x7f05ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06027fc640 1 -- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 msgr2=0x7f05ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.372+0000 7f06027fc640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f05ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f06027fc640 1 -- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ec004d40 msgr2=0x7f05ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f06027fc640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05ec004d40 0x7f05ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f06027fc640 1 -- 172.21.15.40:0/2493433601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec003520 con 0x7f05ec003f20 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f0602ffd640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0604146790 0x7f05ec0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f06027fc640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05ec003f20 0x7f05ec004800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001860 tx=0x7f05ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec69ed5d0b18f885 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4037070 con 0x7f05ec003f20 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f402abf0 con 0x7f05ec003f20 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.373+0000 7f0601ffb640 1 -- 172.21.15.40:0/2493433601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4001430 con 0x7f05ec003f20 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 -- 172.21.15.40:0/2493433601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05ec003f20 msgr2=0x7f05ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 --2- 172.21.15.40:0/2493433601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05ec003f20 0x7f05ec004800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001860 tx=0x7f05ec005830 comp rx=0 tx=0).stop 2024-04-26T21:15:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f06037fe640 1 -- 172.21.15.40:0/2493433601 reap_dead start 2024-04-26T21:15:52.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 -- 172.21.15.40:0/2493433601 shutdown_connections 2024-04-26T21:15:52.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 -- 172.21.15.40:0/2493433601 >> 172.21.15.40:0/2493433601 conn(0x7f06040d0d30 msgr2=0x7f06040d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:52.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 -- 172.21.15.40:0/2493433601 shutdown_connections 2024-04-26T21:15:52.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.374+0000 7f060927e640 1 -- 172.21.15.40:0/2493433601 wait complete. 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.375+0000 7f060927e640 1 Processor -- start 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.375+0000 7f060927e640 1 -- start start 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ec003f20 0x7f0604142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 0x7f0604144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06040cff20 con 0x7f0604160220 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06040d00a0 con 0x7f0604143cf0 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f060927e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06040d0220 con 0x7f05ec003f20 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40168/0 (socket says 172.21.15.40:40168) 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 -- 172.21.15.40:0/405253704 learned_addr learned my addr 172.21.15.40:0/405253704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f0602ffd640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ec003f20 0x7f0604142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06037fe640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 0x7f0604144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 -- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ec003f20 msgr2=0x7f0604142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ec003f20 0x7f0604142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 -- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 msgr2=0x7f0604144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 0x7f0604144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06027fc640 1 -- 172.21.15.40:0/405253704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06041606a0 con 0x7f0604160220 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.376+0000 7f06037fe640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 0x7f0604144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f06027fc640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001980 tx=0x7f05f4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:52.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f403f020 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f40067d0 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4037040 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0604142c50 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.377+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0604143210 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.378+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060406a290 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.379+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05f4042020 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.379+0000 7f05eb7fe640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 0x7f05f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.379+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4943+0+0 (secure 0 0 0) 0x7f05f40ab620 con 0x7f0604160220 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.379+0000 7f0602ffd640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 0x7f05f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:52.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.380+0000 7f0602ffd640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 0x7f05f006ede0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05f80058b0 tx=0x7f05f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:52.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.383+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05f4049030 con 0x7f0604160220 2024-04-26T21:15:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:52.582+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f06040b5760 con 0x7f0604160220 2024-04-26T21:15:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.325+0000 7f05eb7fe640 1 -- 172.21.15.40:0/405253704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f05f4079d50 con 0x7f0604160220 2024-04-26T21:15:53.326 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:15:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 msgr2=0x7f05f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 0x7f05f006ede0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f05f80058b0 tx=0x7f05f8009040 comp rx=0 tx=0).stop 2024-04-26T21:15:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 msgr2=0x7f060413b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001980 tx=0x7f05f4001230 comp rx=0 tx=0).stop 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 shutdown_connections 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f05f006c920 0x7f05f006ede0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0604143cf0 0x7f0604144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0604160220 0x7f060413b750 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 --2- 172.21.15.40:0/405253704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05ec003f20 0x7f0604142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 >> 172.21.15.40:0/405253704 conn(0x7f06040d0d30 msgr2=0x7f060413c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.328+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 shutdown_connections 2024-04-26T21:15:53.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.329+0000 7f060927e640 1 -- 172.21.15.40:0/405253704 wait complete. 2024-04-26T21:15:53.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:15:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.450+0000 7fc15b79d640 1 Processor -- start 2024-04-26T21:15:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.450+0000 7fc15b79d640 1 -- start start 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc15b79d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1541485b0 0x7fc1540afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc15b79d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc154073900 0x7fc15406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc15b79d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1540ae8b0 con 0x7fc15406fa50 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc15b79d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1540aeb90 con 0x7fc154073900 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc15b79d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1540aee70 con 0x7fc1541485b0 2024-04-26T21:15:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc154073900 0x7fc15406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc154073900 0x7fc15406fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40172/0 (socket says 172.21.15.40:40172) 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 -- 172.21.15.40:0/2791644881 learned_addr learned my addr 172.21.15.40:0/2791644881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 -- 172.21.15.40:0/2791644881 >> v2:172.21.15.184:3300/0 conn(0x7fc1541485b0 msgr2=0x7fc1540afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 --2- 172.21.15.40:0/2791644881 >> v2:172.21.15.184:3300/0 conn(0x7fc1541485b0 0x7fc1540afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 -- 172.21.15.40:0/2791644881 >> v1:172.21.15.184:6789/0 conn(0x7fc15406fa50 legacy=0x7fc1540731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.451+0000 7fc159d3a640 1 -- 172.21.15.40:0/2791644881 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540af150 con 0x7fc154073900 2024-04-26T21:15:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.452+0000 7fc159d3a640 1 --2- 172.21.15.40:0/2791644881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc154073900 0x7fc15406fee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc15406e640 tx=0x7fc14c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2db1ebf34165b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.452+0000 7fc14bfff640 1 -- 172.21.15.40:0/2791644881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c036070 con 0x7fc154073900 2024-04-26T21:15:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.452+0000 7fc14bfff640 1 -- 172.21.15.40:0/2791644881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c031600 con 0x7fc154073900 2024-04-26T21:15:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.452+0000 7fc14bfff640 1 -- 172.21.15.40:0/2791644881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0319d0 con 0x7fc154073900 2024-04-26T21:15:53.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 -- 172.21.15.40:0/2791644881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc154073900 msgr2=0x7fc15406fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 --2- 172.21.15.40:0/2791644881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc154073900 0x7fc15406fee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc15406e640 tx=0x7fc14c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 -- 172.21.15.40:0/2791644881 shutdown_connections 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 --2- 172.21.15.40:0/2791644881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc154073900 0x7fc15406fee0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 --2- 172.21.15.40:0/2791644881 >> v2:172.21.15.184:3300/0 conn(0x7fc1541485b0 0x7fc1540afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 -- 172.21.15.40:0/2791644881 >> 172.21.15.40:0/2791644881 conn(0x7fc1540d0d30 msgr2=0x7fc1540d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 -- 172.21.15.40:0/2791644881 shutdown_connections 2024-04-26T21:15:53.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.453+0000 7fc15b79d640 1 -- 172.21.15.40:0/2791644881 wait complete. 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.454+0000 7fc15b79d640 1 Processor -- start 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.454+0000 7fc15b79d640 1 -- start start 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc15406fa50 0x7fc1540b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc154073900 0x7fc15415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540b0170 con 0x7fc1541485b0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540af710 con 0x7fc154073900 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc15b79d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1540b4cd0 con 0x7fc15406fa50 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40176/0 (socket says 172.21.15.40:40176) 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 -- 172.21.15.40:0/1015508376 learned_addr learned my addr 172.21.15.40:0/1015508376 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 -- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc15406fa50 msgr2=0x7fc1540b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc15406fa50 0x7fc1540b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 -- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc154073900 msgr2=0x7fc15415f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc154073900 0x7fc15415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 -- 172.21.15.40:0/1015508376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540b1f10 con 0x7fc1541485b0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.455+0000 7fc159d3a640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc14c034040 tx=0x7fc14c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.456+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c041020 con 0x7fc1541485b0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.456+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c007780 con 0x7fc1541485b0 2024-04-26T21:15:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.456+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c036040 con 0x7fc1541485b0 2024-04-26T21:15:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.456+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1540b21f0 con 0x7fc1541485b0 2024-04-26T21:15:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.456+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1540b27b0 con 0x7fc1541485b0 2024-04-26T21:15:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.457+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15406a290 con 0x7fc1541485b0 2024-04-26T21:15:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.457+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc14c044020 con 0x7fc1541485b0 2024-04-26T21:15:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.458+0000 7fc149ffb640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 0x7fc13806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.458+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fc14c0ac6a0 con 0x7fc1541485b0 2024-04-26T21:15:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.458+0000 7fc159539640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 0x7fc13806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.459+0000 7fc159539640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 0x7fc13806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc150009670 tx=0x7fc150006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.462+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc14c04b030 con 0x7fc1541485b0 2024-04-26T21:15:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.718+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc15406d2b0 con 0x7fc1541485b0 2024-04-26T21:15:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.718+0000 7fc149ffb640 1 -- 172.21.15.40:0/1015508376 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fc14c07aee0 con 0x7fc1541485b0 2024-04-26T21:15:53.719 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:53.719 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.720 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:53.721 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.722 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:15:53.724 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:15:53.726 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:15:53.727 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.728 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:15:53.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:15:53.731 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:15:53.732 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:53.733 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:15:53.733 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:15:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 msgr2=0x7fc13806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 0x7fc13806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc150009670 tx=0x7fc150006bf0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 msgr2=0x7fc15415f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc14c034040 tx=0x7fc14c004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 shutdown_connections 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc13806c8f0 0x7fc13806edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1541485b0 0x7fc15415f560 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc154073900 0x7fc15415f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 --2- 172.21.15.40:0/1015508376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc15406fa50 0x7fc1540b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 >> 172.21.15.40:0/1015508376 conn(0x7fc1540d0d30 msgr2=0x7fc1540b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 shutdown_connections 2024-04-26T21:15:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.720+0000 7fc15b79d640 1 -- 172.21.15.40:0/1015508376 wait complete. 2024-04-26T21:15:53.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.835+0000 7f1df3452640 1 Processor -- start 2024-04-26T21:15:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.835+0000 7f1df3452640 1 -- start start 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df3452640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1dec0d11b0 0x7f1dec06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df3452640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dec0d1740 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df3452640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dec0d1a20 con 0x7f1dec06e2c0 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df3452640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1dec06e710 con 0x7f1dec0d11b0 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df11ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1dec0cf2e0 0x7f1dec06d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55586/0 (socket says 172.21.15.40:55586) 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df11ee640 1 -- 172.21.15.40:0/4132832924 learned_addr learned my addr 172.21.15.40:0/4132832924 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1df09ed640 1 --2- 172.21.15.40:0/4132832924 >> v2:172.21.15.184:3300/0 conn(0x7f1dec0d11b0 0x7f1dec06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541703748 0 0) 0x7f1dec0d1740 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.836+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dd8003520 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4245982104 0 0) 0x7f1dd8003520 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 >> v2:172.21.15.184:3300/0 conn(0x7f1dec0d11b0 msgr2=0x7f1dec06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 --2- 172.21.15.40:0/4132832924 >> v2:172.21.15.184:3300/0 conn(0x7f1dec0d11b0 0x7f1dec06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 >> v1:172.21.15.184:6789/0 conn(0x7f1dec06e2c0 legacy=0x7f1dec15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec0d1d00 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f1de8001670 con 0x7f1dec0cf2e0 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 >> v1:172.21.15.40:6789/0 conn(0x7f1dec0cf2e0 legacy=0x7f1dec06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec06e710 con 0x7f1dd80051c0 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd800c090 con 0x7f1dd8004800 2024-04-26T21:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd800c290 con 0x7f1dd80039e0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df09ed640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df11ee640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 -- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80039e0 msgr2=0x7f1dd8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 -- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd8004800 msgr2=0x7f1dd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.837+0000 7f1df19ef640 1 -- 172.21.15.40:0/4132832924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd8003520 con 0x7f1dd80051c0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.838+0000 7f1df19ef640 1 -- 172.21.15.40:0/4132832924 reap_dead start 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.838+0000 7f1df19ef640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dd800b9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001e50 tx=0x7f1dd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c190b809020c52d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.838+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4001a70 con 0x7f1dd80051c0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.838+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de4003970 con 0x7f1dd80051c0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.838+0000 7f1de3fff640 1 -- 172.21.15.40:0/4132832924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003d60 con 0x7f1dd80051c0 2024-04-26T21:15:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 -- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 msgr2=0x7f1dd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dd800b9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001e50 tx=0x7f1dd8005830 comp rx=0 tx=0).stop 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 -- 172.21.15.40:0/4132832924 shutdown_connections 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 --2- 172.21.15.40:0/4132832924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dd800b9c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 -- 172.21.15.40:0/4132832924 >> 172.21.15.40:0/4132832924 conn(0x7f1dec0b4a60 msgr2=0x7f1dec0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 -- 172.21.15.40:0/4132832924 shutdown_connections 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.839+0000 7f1df3452640 1 -- 172.21.15.40:0/4132832924 wait complete. 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.840+0000 7f1df3452640 1 Processor -- start 2024-04-26T21:15:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.840+0000 7f1df3452640 1 -- start start 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dec15c510 0x7f1dec15f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 0x7f1dec14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0cf6c0 con 0x7f1dd80051c0 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0d22c0 con 0x7f1dec14df20 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1dec0cf840 con 0x7f1dec15c510 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40196/0 (socket says 172.21.15.40:40196) 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 -- 172.21.15.40:0/1778168035 learned_addr learned my addr 172.21.15.40:0/1778168035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df09ed640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dec15c510 0x7f1dec15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df19ef640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 0x7f1dec14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 -- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dec15c510 msgr2=0x7f1dec15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dec15c510 0x7f1dec15f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 -- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 msgr2=0x7f1dec14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 0x7f1dec14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 -- 172.21.15.40:0/1778168035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec14e8e0 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df19ef640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 0x7f1dec14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df11ee640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007340 tx=0x7f1de8007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8007e20 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de8018070 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de800fb60 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.841+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec1f5a20 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.842+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec1f5fe0 con 0x7f1dd80051c0 2024-04-26T21:15:53.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.843+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec1f66f0 con 0x7f1dd80051c0 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.843+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1de800fd00 con 0x7f1dd80051c0 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.843+0000 7f1de1ffb640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.844+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f1de808b8b0 con 0x7f1dd80051c0 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.844+0000 7f1df09ed640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.844+0000 7f1df09ed640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 0x7f1dd006ede0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1dec06d980 tx=0x7f1ddc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:53.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:53.848+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1de8059f90 con 0x7f1dd80051c0 2024-04-26T21:15:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.033+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dec0b7030 con 0x7f1dd80051c0 2024-04-26T21:15:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.034+0000 7f1de1ffb640 1 -- 172.21.15.40:0/1778168035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+12892 (secure 0 0 0) 0x7f1de8059890 con 0x7f1dd80051c0 2024-04-26T21:15:54.255 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:54.255 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:53.333199+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 msgr2=0x7f1dd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 0x7f1dd006ede0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1dec06d980 tx=0x7f1ddc009190 comp rx=0 tx=0).stop 2024-04-26T21:15:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 msgr2=0x7f1dec1f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007340 tx=0x7f1de8007740 comp rx=0 tx=0).stop 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 shutdown_connections 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dec14df20 0x7f1dec14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1dec15c510 0x7f1dec15f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 --2- 172.21.15.40:0/1778168035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dd80051c0 0x7f1dec1f5120 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 >> 172.21.15.40:0/1778168035 conn(0x7f1dec0b4a60 msgr2=0x7f1dec0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 shutdown_connections 2024-04-26T21:15:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.036+0000 7f1df3452640 1 -- 172.21.15.40:0/1778168035 wait complete. 2024-04-26T21:15:54.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:15:54.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:15:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.364+0000 7fe8f9ed1640 1 Processor -- start 2024-04-26T21:15:54.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.364+0000 7fe8f9ed1640 1 -- start start 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f9ed1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8f415e7e0 0x7fe8f40ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f9ed1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f4161cb0 0x7fe8f40d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f9ed1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f40cdd40 con 0x7fe8f40d1d90 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f9ed1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f40d2280 con 0x7fe8f415e7e0 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f9ed1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f40cdfe0 con 0x7fe8f4161cb0 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8f415e7e0 0x7fe8f40ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8f415e7e0 0x7fe8f40ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40202/0 (socket says 172.21.15.40:40202) 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 -- 172.21.15.40:0/625692214 learned_addr learned my addr 172.21.15.40:0/625692214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 -- 172.21.15.40:0/625692214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f4161cb0 msgr2=0x7fe8f40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 --2- 172.21.15.40:0/625692214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f4161cb0 0x7fe8f40d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 -- 172.21.15.40:0/625692214 >> v1:172.21.15.184:6789/0 conn(0x7fe8f40d1d90 legacy=0x7fe8f4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.365+0000 7fe8f37fe640 1 -- 172.21.15.40:0/625692214 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f40ce2c0 con 0x7fe8f415e7e0 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.366+0000 7fe8f37fe640 1 --2- 172.21.15.40:0/625692214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f40ceda0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009870 tx=0x7fe8e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42baad64d0d9972 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.366+0000 7fe8f27fc640 1 -- 172.21.15.40:0/625692214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4037070 con 0x7fe8f415e7e0 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.366+0000 7fe8f27fc640 1 -- 172.21.15.40:0/625692214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e4032440 con 0x7fe8f415e7e0 2024-04-26T21:15:54.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.366+0000 7fe8f27fc640 1 -- 172.21.15.40:0/625692214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4032810 con 0x7fe8f415e7e0 2024-04-26T21:15:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/625692214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 msgr2=0x7fe8f40ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/625692214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f40ceda0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009870 tx=0x7fe8e402a210 comp rx=0 tx=0).stop 2024-04-26T21:15:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/625692214 shutdown_connections 2024-04-26T21:15:54.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/625692214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f4161cb0 0x7fe8f40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/625692214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f40ceda0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/625692214 >> 172.21.15.40:0/625692214 conn(0x7fe8f409ba40 msgr2=0x7fe8f409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/625692214 shutdown_connections 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.367+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/625692214 wait complete. 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.368+0000 7fe8f9ed1640 1 Processor -- start 2024-04-26T21:15:54.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.368+0000 7fe8f9ed1640 1 -- start start 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f40d1d90 0x7fe8f40cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 0x7fe8f41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40cf2e0 con 0x7fe8f415e7e0 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40ce880 con 0x7fe8f4161cb0 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f9ed1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f4161790 con 0x7fe8f40d1d90 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40214/0 (socket says 172.21.15.40:40214) 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 -- 172.21.15.40:0/997381848 learned_addr learned my addr 172.21.15.40:0/997381848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f37fe640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f40d1d90 0x7fe8f40cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 0x7fe8f41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 -- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f40d1d90 msgr2=0x7fe8f40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f40d1d90 0x7fe8f40cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 -- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 msgr2=0x7fe8f41581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 0x7fe8f41581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.369+0000 7fe8f2ffd640 1 -- 172.21.15.40:0/997381848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f40cad20 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 0x7fe8f41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8f2ffd640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800b4a0 tx=0x7fe8e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8014070 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8010610 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f406d9a0 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8010a00 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.370+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f40cb270 con 0x7fe8f415e7e0 2024-04-26T21:15:54.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.371+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f406a290 con 0x7fe8f415e7e0 2024-04-26T21:15:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.371+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8e8017070 con 0x7fe8f415e7e0 2024-04-26T21:15:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.372+0000 7fe8dbfff640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.372+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fe8e808b150 con 0x7fe8f415e7e0 2024-04-26T21:15:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.372+0000 7fe8f37fe640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.372+0000 7fe8f37fe640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 0x7fe8e006edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4034000 tx=0x7fe8e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.376+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8e8059960 con 0x7fe8f415e7e0 2024-04-26T21:15:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.562+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8f4068e00 con 0x7fe8f415e7e0 2024-04-26T21:15:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.562+0000 7fe8dbfff640 1 -- 172.21.15.40:0/997381848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fe8e8059260 con 0x7fe8f415e7e0 2024-04-26T21:15:54.563 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:15:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 msgr2=0x7fe8e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 0x7fe8e006edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4034000 tx=0x7fe8e402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 msgr2=0x7fe8f406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800b4a0 tx=0x7fe8e800b960 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 shutdown_connections 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f4161cb0 0x7fe8f41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8f415e7e0 0x7fe8f406d460 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 --2- 172.21.15.40:0/997381848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f40d1d90 0x7fe8f40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 >> 172.21.15.40:0/997381848 conn(0x7fe8f409ba40 msgr2=0x7fe8f4159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 shutdown_connections 2024-04-26T21:15:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.564+0000 7fe8f9ed1640 1 -- 172.21.15.40:0/997381848 wait complete. 2024-04-26T21:15:54.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:15:54.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.676+0000 7f1be376a640 1 Processor -- start 2024-04-26T21:15:54.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.676+0000 7f1be376a640 1 -- start start 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be376a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bdc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be376a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bdc0d03a0 con 0x7f1bdc15f370 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be376a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bdc0d0680 con 0x7f1bdc15f750 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be376a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc0d0960 con 0x7f1bdc0d3ee0 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be1506640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1bdc15f370 0x7f1bdc0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55598/0 (socket says 172.21.15.40:55598) 2024-04-26T21:15:54.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.677+0000 7f1be1506640 1 -- 172.21.15.40:0/3578821264 learned_addr learned my addr 172.21.15.40:0/3578821264 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168543141 0 0) 0x7f1bdc0d03a0 con 0x7f1bdc15f370 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be0d05640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bdc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bc4003520 con 0x7f1bdc15f370 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359083605 0 0) 0x7f1bc4003520 con 0x7f1bdc15f370 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 msgr2=0x7f1bdc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bdc0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 >> v1:172.21.15.184:6789/0 conn(0x7f1bdc15f750 legacy=0x7f1bdc0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc0d0c40 con 0x7f1bdc15f370 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f1bd4001670 con 0x7f1bdc15f370 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 >> v1:172.21.15.40:6789/0 conn(0x7f1bdc15f370 legacy=0x7f1bdc0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc40039e0 0x7f1bc4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc4004d40 0x7f1bc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0d0960 con 0x7f1bc4004d40 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc400bb50 con 0x7f1bc40039e0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc400bd20 con 0x7f1bdc0d3ee0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc4004d40 0x7f1bc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1506640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc40039e0 0x7f1bc4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be0d05640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 -- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 msgr2=0x7f1bc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc0d3ee0 0x7f1bc4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 -- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc40039e0 msgr2=0x7f1bc4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bc40039e0 0x7f1bc4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.678+0000 7f1be1d07640 1 -- 172.21.15.40:0/3578821264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc4003520 con 0x7f1bc4004d40 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.679+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc4004d40 0x7f1bc40051c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc001e80 tx=0x7f1bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e3ef80d573a011 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.679+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc001a70 con 0x7f1bc4004d40 2024-04-26T21:15:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.679+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc02ae50 con 0x7f1bc4004d40 2024-04-26T21:15:54.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.679+0000 7f1bdbfff640 1 -- 172.21.15.40:0/3578821264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc003bb0 con 0x7f1bc4004d40 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 -- 172.21.15.40:0/3578821264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc4004d40 msgr2=0x7f1bc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 --2- 172.21.15.40:0/3578821264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bc4004d40 0x7f1bc40051c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc001e80 tx=0x7f1bc4005830 comp rx=0 tx=0).stop 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be1d07640 1 -- 172.21.15.40:0/3578821264 reap_dead start 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 -- 172.21.15.40:0/3578821264 shutdown_connections 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 -- 172.21.15.40:0/3578821264 >> 172.21.15.40:0/3578821264 conn(0x7f1bdc08ec10 msgr2=0x7f1bdc09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 -- 172.21.15.40:0/3578821264 shutdown_connections 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.680+0000 7f1be376a640 1 -- 172.21.15.40:0/3578821264 wait complete. 2024-04-26T21:15:54.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.681+0000 7f1be376a640 1 Processor -- start 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.681+0000 7f1be376a640 1 -- start start 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 0x7f1bdc1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc1f2ba0 0x7f1bdc0ba210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0cfef0 con 0x7f1bdc0ba750 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0cfd70 con 0x7f1bdc1eefd0 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be376a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc0d0070 con 0x7f1bdc1f2ba0 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40236/0 (socket says 172.21.15.40:40236) 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 -- 172.21.15.40:0/3762274152 learned_addr learned my addr 172.21.15.40:0/3762274152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be0d05640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc1f2ba0 0x7f1bdc0ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1506640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 0x7f1bdc1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 -- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc1f2ba0 msgr2=0x7f1bdc0ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc1f2ba0 0x7f1bdc0ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 -- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 msgr2=0x7f1bdc1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 0x7f1bdc1f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 -- 172.21.15.40:0/3762274152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc0bac00 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1506640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 0x7f1bdc1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.682+0000 7f1be1d07640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc005040 tx=0x7f1bcc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.683+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc001a70 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.683+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc042070 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.683+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc007db0 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.683+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc0a2bb0 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.683+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc0b29e0 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.684+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc06a290 con 0x7f1bdc0ba750 2024-04-26T21:15:54.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.684+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bcc005400 con 0x7f1bdc0ba750 2024-04-26T21:15:54.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.685+0000 7f1bd9ffb640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 0x7f1bc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.685+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f1bcc0abb50 con 0x7f1bdc0ba750 2024-04-26T21:15:54.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.685+0000 7f1be1506640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 0x7f1bc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.685+0000 7f1be1506640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 0x7f1bc806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4004840 tx=0x7f1bd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.689+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bcc07a300 con 0x7f1bdc0ba750 2024-04-26T21:15:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.874+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1bdc15de10 con 0x7f1bdc0ba750 2024-04-26T21:15:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.874+0000 7f1bd9ffb640 1 -- 172.21.15.40:0/3762274152 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f1bcc079c00 con 0x7f1bdc0ba750 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:15:54.876 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:15:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 msgr2=0x7f1bc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 0x7f1bc806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4004840 tx=0x7f1bd4009040 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 msgr2=0x7f1bdc1f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc005040 tx=0x7f1bcc0014a0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 shutdown_connections 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1bc806c920 0x7f1bc806ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bdc0ba750 0x7f1bdc1f4000 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1bdc1f2ba0 0x7f1bdc0ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 --2- 172.21.15.40:0/3762274152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bdc1eefd0 0x7f1bdc1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 >> 172.21.15.40:0/3762274152 conn(0x7f1bdc08ec10 msgr2=0x7f1bdc0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 shutdown_connections 2024-04-26T21:15:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.876+0000 7f1be376a640 1 -- 172.21.15.40:0/3762274152 wait complete. 2024-04-26T21:15:54.888 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:15:54.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:15:54.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.992+0000 7fa1e69f9640 1 Processor -- start 2024-04-26T21:15:54.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.992+0000 7fa1e69f9640 1 -- start start 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e69f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e00d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e69f9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1e00d2140 0x7fa1e00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e69f9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e00d33a0 con 0x7fa1e00b6020 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e69f9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e00b4970 con 0x7fa1e00d2140 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e69f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00d3680 con 0x7fa1e0162a10 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1e00d2140 0x7fa1e00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1e00d2140 0x7fa1e00cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40242/0 (socket says 172.21.15.40:40242) 2024-04-26T21:15:54.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 -- 172.21.15.40:0/2122665431 learned_addr learned my addr 172.21.15.40:0/2122665431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1dffff640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e00d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 -- 172.21.15.40:0/2122665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 msgr2=0x7fa1e00d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e00d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 -- 172.21.15.40:0/2122665431 >> v1:172.21.15.184:6789/0 conn(0x7fa1e00b6020 legacy=0x7fa1e00d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1e4f96640 1 -- 172.21.15.40:0/2122665431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00d3960 con 0x7fa1e00d2140 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.993+0000 7fa1dffff640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e00d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e4f96640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00cffb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009e50 tx=0x7fa1d802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e916c84a09282fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1deffd640 1 -- 172.21.15.40:0/2122665431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8036070 con 0x7fa1e00d2140 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1deffd640 1 -- 172.21.15.40:0/2122665431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8031600 con 0x7fa1e00d2140 2024-04-26T21:15:54.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1deffd640 1 -- 172.21.15.40:0/2122665431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80319d0 con 0x7fa1e00d2140 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 -- 172.21.15.40:0/2122665431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 msgr2=0x7fa1e00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00cffb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009e50 tx=0x7fa1d802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 -- 172.21.15.40:0/2122665431 shutdown_connections 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00cffb0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/2122665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e00d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.994+0000 7fa1e69f9640 1 -- 172.21.15.40:0/2122665431 >> 172.21.15.40:0/2122665431 conn(0x7fa1e009e530 msgr2=0x7fa1e00d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:54.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.995+0000 7fa1e69f9640 1 -- 172.21.15.40:0/2122665431 shutdown_connections 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.995+0000 7fa1e69f9640 1 -- 172.21.15.40:0/2122665431 wait complete. 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.996+0000 7fa1e69f9640 1 Processor -- start 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.996+0000 7fa1e69f9640 1 -- start start 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00b6020 0x7fa1e015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e0161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0162df0 con 0x7fa1e00d2140 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d3f20 con 0x7fa1e00b6020 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e69f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00b6600 con 0x7fa1e0162a10 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40256/0 (socket says 172.21.15.40:40256) 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 -- 172.21.15.40:0/1221776979 learned_addr learned my addr 172.21.15.40:0/1221776979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1dffff640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00b6020 0x7fa1e015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1e4f96640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e0161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 -- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 msgr2=0x7fa1e0161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e0161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 -- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00b6020 msgr2=0x7fa1e015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00b6020 0x7fa1e015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 -- 172.21.15.40:0/1221776979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00c9df0 con 0x7fa1e00d2140 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1df7fe640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c460 tx=0x7fa1d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4016020 con 0x7fa1e00d2140 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d4012870 con 0x7fa1e00d2140 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.997+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012c60 con 0x7fa1e00d2140 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.998+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e00c91e0 con 0x7fa1e00d2140 2024-04-26T21:15:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.998+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e00c97a0 con 0x7fa1e00d2140 2024-04-26T21:15:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.999+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e006a290 con 0x7fa1e00d2140 2024-04-26T21:15:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.999+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1d4019020 con 0x7fa1e00d2140 2024-04-26T21:15:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.999+0000 7fa1dcff9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 0x7fa1cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:54.999+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fa1d408b130 con 0x7fa1e00d2140 2024-04-26T21:15:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.000+0000 7fa1dffff640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 0x7fa1cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.000+0000 7fa1dffff640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 0x7fa1cc06ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000aac0 tx=0x7fa1d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.004+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1d4020070 con 0x7fa1e00d2140 2024-04-26T21:15:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.187+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa1e00d1f20 con 0x7fa1e00d2140 2024-04-26T21:15:55.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.331+0000 7fa1dcff9640 1 -- 172.21.15.40:0/1221776979 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7fa1d4059260 con 0x7fa1e00d2140 2024-04-26T21:15:55.333 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:15:55.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.334+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 msgr2=0x7fa1cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.334+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 0x7fa1cc06ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000aac0 tx=0x7fa1d0008040 comp rx=0 tx=0).stop 2024-04-26T21:15:55.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.334+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 msgr2=0x7fa1e00c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.334+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c460 tx=0x7fa1d400c920 comp rx=0 tx=0).stop 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 shutdown_connections 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa1cc06c920 0x7fa1cc06ede0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e0162a10 0x7fa1e0161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e00d2140 0x7fa1e00c8a70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 --2- 172.21.15.40:0/1221776979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00b6020 0x7fa1e015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 >> 172.21.15.40:0/1221776979 conn(0x7fa1e009e530 msgr2=0x7fa1e006e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.335+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 shutdown_connections 2024-04-26T21:15:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.336+0000 7fa1e69f9640 1 -- 172.21.15.40:0/1221776979 wait complete. 2024-04-26T21:15:55.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:15:55.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.456+0000 7fd6844b0640 1 Processor -- start 2024-04-26T21:15:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.457+0000 7fd6844b0640 1 -- start start 2024-04-26T21:15:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.457+0000 7fd6844b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd67c0b1ea0 0x7fd67c0d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.457+0000 7fd6844b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c0b2580 con 0x7fd67c0d56a0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd6844b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c0b2860 con 0x7fd67c15f6e0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd6844b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c0b2330 con 0x7fd67c0b1ea0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd682a4d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd67c0d56a0 0x7fd67c0b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55606/0 (socket says 172.21.15.40:55606) 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd682a4d640 1 -- 172.21.15.40:0/1507761114 learned_addr learned my addr 172.21.15.40:0/1507761114 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156325805 0 0) 0x7fd67c0b2580 con 0x7fd67c0d56a0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd681a4b640 1 --2- 172.21.15.40:0/1507761114 >> v2:172.21.15.184:3300/0 conn(0x7fd67c0b1ea0 0x7fd67c0d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd668003520 con 0x7fd67c0d56a0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2118069418 0 0) 0x7fd668003520 con 0x7fd67c0d56a0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 >> v2:172.21.15.184:3300/0 conn(0x7fd67c0b1ea0 msgr2=0x7fd67c0d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 --2- 172.21.15.40:0/1507761114 >> v2:172.21.15.184:3300/0 conn(0x7fd67c0b1ea0 0x7fd67c0d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 >> v1:172.21.15.184:6789/0 conn(0x7fd67c15f6e0 legacy=0x7fd67c0d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c0b2b40 con 0x7fd67c0d56a0 2024-04-26T21:15:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd6740019f0 con 0x7fd67c0d56a0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 >> v1:172.21.15.40:6789/0 conn(0x7fd67c0d56a0 legacy=0x7fd67c0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680039b0 0x7fd668003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd668004800 0x7fd668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 0x7fd66800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0b2330 con 0x7fd6680051c0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd66800c060 con 0x7fd6680039b0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.458+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd66800c260 con 0x7fd668004800 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 0x7fd66800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd68224c640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680039b0 0x7fd668003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd681a4b640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd668004800 0x7fd668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 -- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd668004800 msgr2=0x7fd668004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd668004800 0x7fd668004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 -- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680039b0 msgr2=0x7fd668003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680039b0 0x7fd668003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 -- 172.21.15.40:0/1507761114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd668003520 con 0x7fd6680051c0 2024-04-26T21:15:55.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd681a4b640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd668004800 0x7fd668004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd68224c640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680039b0 0x7fd668003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 -- 172.21.15.40:0/1507761114 reap_dead start 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.459+0000 7fd682a4d640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 0x7fd66800b9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd67400a850 tx=0x7fd668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc75ba55b865b3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67400a4a0 con 0x7fd6680051c0 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67402f070 con 0x7fd6680051c0 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd68124a640 1 -- 172.21.15.40:0/1507761114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67402bce0 con 0x7fd6680051c0 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 -- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 msgr2=0x7fd66800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 0x7fd66800b9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd67400a850 tx=0x7fd668005830 comp rx=0 tx=0).stop 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 -- 172.21.15.40:0/1507761114 shutdown_connections 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 --2- 172.21.15.40:0/1507761114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680051c0 0x7fd66800b9c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 -- 172.21.15.40:0/1507761114 >> 172.21.15.40:0/1507761114 conn(0x7fd67c08ec20 msgr2=0x7fd67c09e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 -- 172.21.15.40:0/1507761114 shutdown_connections 2024-04-26T21:15:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.460+0000 7fd6844b0640 1 -- 172.21.15.40:0/1507761114 wait complete. 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.461+0000 7fd6844b0640 1 Processor -- start 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.461+0000 7fd6844b0640 1 -- start start 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680051c0 0x7fd67c1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c15b070 0x7fd67c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0d5160 con 0x7fd67c154020 2024-04-26T21:15:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0d5ad0 con 0x7fd6680051c0 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd6844b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c0d52e0 con 0x7fd67c15b070 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd681a4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd681a4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40264/0 (socket says 172.21.15.40:40264) 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.462+0000 7fd681a4b640 1 -- 172.21.15.40:0/3824795088 learned_addr learned my addr 172.21.15.40:0/3824795088 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd682a4d640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c15b070 0x7fd67c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd68224c640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680051c0 0x7fd67c1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 -- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c15b070 msgr2=0x7fd67c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c15b070 0x7fd67c15b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 -- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680051c0 msgr2=0x7fd67c1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680051c0 0x7fd67c1f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 -- 172.21.15.40:0/3824795088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c1f5e10 con 0x7fd67c154020 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd681a4b640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd66c011540 tx=0x7fd66c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c015870 con 0x7fd67c154020 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c015a10 con 0x7fd67c154020 2024-04-26T21:15:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c1f60f0 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c015dc0 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.463+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c0ca520 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.464+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c06a290 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.465+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd66c00f460 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.465+0000 7fd672ffd640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 0x7fd66006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.465+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7fd66c08c1d0 con 0x7fd67c154020 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.465+0000 7fd68224c640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 0x7fd66006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.466+0000 7fd68224c640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 0x7fd66006edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd678001530 tx=0x7fd678009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.470+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd66c019030 con 0x7fd67c154020 2024-04-26T21:15:55.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:55.655+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd67c0badf0 con 0x7fd67c154020 2024-04-26T21:15:56.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.334+0000 7fd672ffd640 1 -- 172.21.15.40:0/3824795088 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7fd66c05a870 con 0x7fd67c154020 2024-04-26T21:15:56.336 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.338+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 msgr2=0x7fd66006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.338+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 0x7fd66006edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd678001530 tx=0x7fd678009040 comp rx=0 tx=0).stop 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.338+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 msgr2=0x7fd67c1f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.338+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fd66c011540 tx=0x7fd66c011a00 comp rx=0 tx=0).stop 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 shutdown_connections 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd66006c8f0 0x7fd66006edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c15b070 0x7fd67c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd67c154020 0x7fd67c1f5690 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 --2- 172.21.15.40:0/3824795088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6680051c0 0x7fd67c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.339+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 >> 172.21.15.40:0/3824795088 conn(0x7fd67c08ec20 msgr2=0x7fd67c0ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:56.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.345+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 shutdown_connections 2024-04-26T21:15:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.349+0000 7fd6844b0640 1 -- 172.21.15.40:0/3824795088 wait complete. 2024-04-26T21:15:56.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:15:56.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.477+0000 7fe04a6a9640 1 Processor -- start 2024-04-26T21:15:56.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.477+0000 7fe04a6a9640 1 -- start start 2024-04-26T21:15:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe04a6a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1be0 0x7fe0440d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe04a6a9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0440ba3a0 0x7fe0440ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe04a6a9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0440d4240 con 0x7fe04415f490 2024-04-26T21:15:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe04a6a9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440d2580 con 0x7fe0440ba3a0 2024-04-26T21:15:56.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe04a6a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0440d31a0 con 0x7fe0440d1be0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe048c46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0440ba3a0 0x7fe0440ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe043fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe04415f490 0x7fe0440d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55620/0 (socket says 172.21.15.40:55620) 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe043fff640 1 -- 172.21.15.40:0/1154722080 learned_addr learned my addr 172.21.15.40:0/1154722080 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe0437fe640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1be0 0x7fe0440d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe048c46640 1 -- 172.21.15.40:0/1154722080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1be0 msgr2=0x7fe0440d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe048c46640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1be0 0x7fe0440d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe048c46640 1 -- 172.21.15.40:0/1154722080 >> v1:172.21.15.40:6789/0 conn(0x7fe04415f490 legacy=0x7fe0440d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.478+0000 7fe048c46640 1 -- 172.21.15.40:0/1154722080 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440d3480 con 0x7fe0440ba3a0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.479+0000 7fe048c46640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440ba3a0 0x7fe0440ba7f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009d90 tx=0x7fe03c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e934668339c41e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.479+0000 7fe042ffd640 1 -- 172.21.15.40:0/1154722080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c007740 con 0x7fe0440ba3a0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.479+0000 7fe042ffd640 1 -- 172.21.15.40:0/1154722080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03c0078e0 con 0x7fe0440ba3a0 2024-04-26T21:15:56.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.479+0000 7fe042ffd640 1 -- 172.21.15.40:0/1154722080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c007cb0 con 0x7fe0440ba3a0 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 -- 172.21.15.40:0/1154722080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440ba3a0 msgr2=0x7fe0440ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440ba3a0 0x7fe0440ba7f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009d90 tx=0x7fe03c02a660 comp rx=0 tx=0).stop 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 -- 172.21.15.40:0/1154722080 shutdown_connections 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0440ba3a0 0x7fe0440ba7f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/1154722080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440d1be0 0x7fe0440d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 -- 172.21.15.40:0/1154722080 >> 172.21.15.40:0/1154722080 conn(0x7fe04409e530 msgr2=0x7fe0440d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 -- 172.21.15.40:0/1154722080 shutdown_connections 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.480+0000 7fe04a6a9640 1 -- 172.21.15.40:0/1154722080 wait complete. 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.481+0000 7fe04a6a9640 1 Processor -- start 2024-04-26T21:15:56.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.481+0000 7fe04a6a9640 1 -- start start 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440ba3a0 0x7fe0441e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0440d1be0 0x7fe0441e9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440d3d20 con 0x7fe04415f490 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440d3a40 con 0x7fe0440d1be0 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe04a6a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0440d11c0 con 0x7fe0440ba3a0 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40278/0 (socket says 172.21.15.40:40278) 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 -- 172.21.15.40:0/2227591173 learned_addr learned my addr 172.21.15.40:0/2227591173 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe043fff640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440ba3a0 0x7fe0441e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe0437fe640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0440d1be0 0x7fe0441e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 -- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440ba3a0 msgr2=0x7fe0441e8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440ba3a0 0x7fe0441e8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 -- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0440d1be0 msgr2=0x7fe0441e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0440d1be0 0x7fe0441e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.482+0000 7fe048c46640 1 -- 172.21.15.40:0/2227591173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe044161370 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe048c46640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009eb0 tx=0x7fe03c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c03f070 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03c031c60 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c043650 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe044161650 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.483+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe044161c10 con 0x7fe04415f490 2024-04-26T21:15:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.484+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04406a290 con 0x7fe04415f490 2024-04-26T21:15:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.484+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe03c03b410 con 0x7fe04415f490 2024-04-26T21:15:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.485+0000 7fe040ff9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.485+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7fe03c0acdd0 con 0x7fe04415f490 2024-04-26T21:15:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.485+0000 7fe043fff640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.486+0000 7fe043fff640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 0x7fe02406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe0440cce70 tx=0x7fe02c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:56.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.490+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe03c07aca0 con 0x7fe04415f490 2024-04-26T21:15:56.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:56.719+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe0440d0e30 con 0x7fe04415f490 2024-04-26T21:15:57.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.353+0000 7fe040ff9640 1 -- 172.21.15.40:0/2227591173 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v43) ==== 174+0+0 (secure 0 0 0) 0x7fe03c07efa0 con 0x7fe04415f490 2024-04-26T21:15:57.355 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 12 and data pool 13 2024-04-26T21:15:57.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.356+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 msgr2=0x7fe02406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.356+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 0x7fe02406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe0440cce70 tx=0x7fe02c007e80 comp rx=0 tx=0).stop 2024-04-26T21:15:57.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.356+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 msgr2=0x7fe044154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.356+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009eb0 tx=0x7fe03c004a40 comp rx=0 tx=0).stop 2024-04-26T21:15:57.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 shutdown_connections 2024-04-26T21:15:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe02406c8f0 0x7fe02406edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe04415f490 0x7fe044154fd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0440d1be0 0x7fe0441e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 --2- 172.21.15.40:0/2227591173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0440ba3a0 0x7fe0441e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.361+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 >> 172.21.15.40:0/2227591173 conn(0x7fe04409e530 msgr2=0x7fe044075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.373+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 shutdown_connections 2024-04-26T21:15:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.373+0000 7fe04a6a9640 1 -- 172.21.15.40:0/2227591173 wait complete. 2024-04-26T21:15:57.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:57.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 Processor -- start 2024-04-26T21:15:57.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 -- start start 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f522415f750 0x7f52240cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52240d03a0 con 0x7f52240d3ee0 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52240d0680 con 0x7f522415f370 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.500+0000 7f5229807640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f52240d0960 con 0x7f522415f750 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52227fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52240d3ee0 0x7f52240b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55628/0 (socket says 172.21.15.40:55628) 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52227fc640 1 -- 172.21.15.40:0/2798386982 learned_addr learned my addr 172.21.15.40:0/2798386982 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52237fe640 1 --2- 172.21.15.40:0/2798386982 >> v2:172.21.15.40:3300/0 conn(0x7f522415f750 0x7f52240cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f5221ffb640 1 -- 172.21.15.40:0/2798386982 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728374943 0 0) 0x7f52240d03a0 con 0x7f52240d3ee0 2024-04-26T21:15:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f5221ffb640 1 -- 172.21.15.40:0/2798386982 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f520c003520 con 0x7f52240d3ee0 2024-04-26T21:15:57.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52237fe640 1 -- 172.21.15.40:0/2798386982 >> v1:172.21.15.184:6789/0 conn(0x7f522415f370 legacy=0x7f52240d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52237fe640 1 -- 172.21.15.40:0/2798386982 >> v1:172.21.15.40:6789/0 conn(0x7f52240d3ee0 legacy=0x7f52240b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.501+0000 7f52237fe640 1 -- 172.21.15.40:0/2798386982 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52240d0c40 con 0x7f522415f750 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f52237fe640 1 --2- 172.21.15.40:0/2798386982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f522415f750 0x7f52240cfd70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5214009d90 tx=0x7f521402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5112724adfd2c470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5221ffb640 1 -- 172.21.15.40:0/2798386982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214007740 con 0x7f522415f750 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5221ffb640 1 -- 172.21.15.40:0/2798386982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52140078e0 con 0x7f522415f750 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5221ffb640 1 -- 172.21.15.40:0/2798386982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214007cb0 con 0x7f522415f750 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 -- 172.21.15.40:0/2798386982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f522415f750 msgr2=0x7f52240cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 --2- 172.21.15.40:0/2798386982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f522415f750 0x7f52240cfd70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5214009d90 tx=0x7f521402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 -- 172.21.15.40:0/2798386982 shutdown_connections 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 --2- 172.21.15.40:0/2798386982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f522415f750 0x7f52240cfd70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 -- 172.21.15.40:0/2798386982 >> 172.21.15.40:0/2798386982 conn(0x7f522408ec10 msgr2=0x7f522409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 -- 172.21.15.40:0/2798386982 shutdown_connections 2024-04-26T21:15:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.502+0000 7f5229807640 1 -- 172.21.15.40:0/2798386982 wait complete. 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.503+0000 7f5229807640 1 Processor -- start 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 -- start start 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 0x7f52240cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f522415f750 0x7f52240caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52240b4950 con 0x7f52240d3ee0 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52240b47d0 con 0x7f522415f750 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5229807640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f52240b4ad0 con 0x7f522415f370 2024-04-26T21:15:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.504+0000 7f5222ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40300/0 (socket says 172.21.15.40:40300) 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 -- 172.21.15.40:0/606976261 learned_addr learned my addr 172.21.15.40:0/606976261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f52237fe640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f522415f750 0x7f52240caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f52227fc640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 0x7f52240cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 -- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 msgr2=0x7f52240cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 0x7f52240cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 -- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f522415f750 msgr2=0x7f52240caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f522415f750 0x7f52240caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 -- 172.21.15.40:0/606976261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52240b82f0 con 0x7f52240d3ee0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f52227fc640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 0x7f52240cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5222ffd640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c400 tx=0x7f521c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0052d0 con 0x7f52240d3ee0 2024-04-26T21:15:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c005490 con 0x7f52240d3ee0 2024-04-26T21:15:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52240b85d0 con 0x7f52240d3ee0 2024-04-26T21:15:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c007260 con 0x7f52240d3ee0 2024-04-26T21:15:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.505+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52240b8b90 con 0x7f52240d3ee0 2024-04-26T21:15:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.506+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522406a290 con 0x7f52240d3ee0 2024-04-26T21:15:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.507+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f521c007400 con 0x7f52240d3ee0 2024-04-26T21:15:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.507+0000 7f520bfff640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.507+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f521c08b050 con 0x7f52240d3ee0 2024-04-26T21:15:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.507+0000 7f52227fc640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.508+0000 7f52227fc640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 0x7f521006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f522415d670 tx=0x7f5218009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.512+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f521c059510 con 0x7f52240d3ee0 2024-04-26T21:15:57.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.696+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52240b6350 con 0x7f52240d3ee0 2024-04-26T21:15:57.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.697+0000 7f520bfff640 1 -- 172.21.15.40:0/606976261 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16673 (secure 0 0 0) 0x7f521c058e10 con 0x7f52240d3ee0 2024-04-26T21:15:57.699 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:57.699 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:56.722134+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 msgr2=0x7f521006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 0x7f521006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f522415d670 tx=0x7f5218009040 comp rx=0 tx=0).stop 2024-04-26T21:15:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 msgr2=0x7f52241f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c400 tx=0x7f521c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 shutdown_connections 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f521006c8f0 0x7f521006edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f522415f750 0x7f52240caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f522415f370 0x7f52240cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 --2- 172.21.15.40:0/606976261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52240d3ee0 0x7f52241f47c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 >> 172.21.15.40:0/606976261 conn(0x7f522408ec10 msgr2=0x7f522409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 shutdown_connections 2024-04-26T21:15:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.699+0000 7f5229807640 1 -- 172.21.15.40:0/606976261 wait complete. 2024-04-26T21:15:57.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:15:57.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.811+0000 7f48bf5c1640 1 Processor -- start 2024-04-26T21:15:57.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.811+0000 7f48bf5c1640 1 -- start start 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bf5c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48b80cf2e0 0x7f48b806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bf5c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f48b80d11b0 0x7f48b806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bf5c1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b80d1740 con 0x7f48b806e2c0 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bf5c1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f48b806e710 con 0x7f48b80cf2e0 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bf5c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f48b80d1a20 con 0x7f48b80d11b0 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48b80cf2e0 0x7f48b806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48b80cf2e0 0x7f48b806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40316/0 (socket says 172.21.15.40:40316) 2024-04-26T21:15:57.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 -- 172.21.15.40:0/833685565 learned_addr learned my addr 172.21.15.40:0/833685565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:57.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 -- 172.21.15.40:0/833685565 >> v2:172.21.15.184:3300/0 conn(0x7f48b80d11b0 msgr2=0x7f48b806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:57.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 --2- 172.21.15.40:0/833685565 >> v2:172.21.15.184:3300/0 conn(0x7f48b80d11b0 0x7f48b806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 -- 172.21.15.40:0/833685565 >> v1:172.21.15.184:6789/0 conn(0x7f48b806e2c0 legacy=0x7f48b815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:57.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.812+0000 7f48bd35d640 1 -- 172.21.15.40:0/833685565 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b80d1d00 con 0x7f48b80cf2e0 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bd35d640 1 --2- 172.21.15.40:0/833685565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b806d4d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea2de5332db6ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48affff640 1 -- 172.21.15.40:0/833685565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4037070 con 0x7f48b80cf2e0 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48affff640 1 -- 172.21.15.40:0/833685565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b4032440 con 0x7f48b80cf2e0 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48affff640 1 -- 172.21.15.40:0/833685565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4032810 con 0x7f48b80cf2e0 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 -- 172.21.15.40:0/833685565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 msgr2=0x7f48b806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/833685565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b806d4d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402a270 comp rx=0 tx=0).stop 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 -- 172.21.15.40:0/833685565 shutdown_connections 2024-04-26T21:15:57.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/833685565 >> v2:172.21.15.184:3300/0 conn(0x7f48b80d11b0 0x7f48b806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/833685565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b806d4d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.813+0000 7f48bf5c1640 1 -- 172.21.15.40:0/833685565 >> 172.21.15.40:0/833685565 conn(0x7f48b80b4a60 msgr2=0x7f48b80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.814+0000 7f48bf5c1640 1 -- 172.21.15.40:0/833685565 shutdown_connections 2024-04-26T21:15:57.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.814+0000 7f48bf5c1640 1 -- 172.21.15.40:0/833685565 wait complete. 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 Processor -- start 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 -- start start 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b806e2c0 0x7f48b81f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b80d11b0 0x7f48b809fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48b806da10 con 0x7f48b80cf2e0 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48b80d22c0 con 0x7f48b806e2c0 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bf5c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f48b80cf6c0 con 0x7f48b80d11b0 2024-04-26T21:15:57.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.815+0000 7f48bcb5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40322/0 (socket says 172.21.15.40:40322) 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 -- 172.21.15.40:0/2995997595 learned_addr learned my addr 172.21.15.40:0/2995997595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bd35d640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b806e2c0 0x7f48b81f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bdb5e640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b80d11b0 0x7f48b809fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 -- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b80d11b0 msgr2=0x7f48b809fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b80d11b0 0x7f48b809fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 -- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b806e2c0 msgr2=0x7f48b81f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b806e2c0 0x7f48b81f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 -- 172.21.15.40:0/2995997595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b80b5ce0 con 0x7f48b80cf2e0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bcb5c640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f48a800b410 tx=0x7f48a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8015020 con 0x7f48b80cf2e0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8011850 con 0x7f48b80cf2e0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8011c40 con 0x7f48b80cf2e0 2024-04-26T21:15:57.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b809d040 con 0x7f48b80cf2e0 2024-04-26T21:15:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.816+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b809d600 con 0x7f48b80cf2e0 2024-04-26T21:15:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.817+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b806a290 con 0x7f48b80cf2e0 2024-04-26T21:15:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.818+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48a8018020 con 0x7f48b80cf2e0 2024-04-26T21:15:57.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.818+0000 7f48adffb640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 0x7f489c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:57.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.818+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f48a808ba70 con 0x7f48b80cf2e0 2024-04-26T21:15:57.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.818+0000 7f48bd35d640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 0x7f489c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:57.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.819+0000 7f48bd35d640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 0x7f489c06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:57.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:57.822+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48a801f020 con 0x7f48b80cf2e0 2024-04-26T21:15:58.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.008+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48b80b8ef0 con 0x7f48b80cf2e0 2024-04-26T21:15:58.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.009+0000 7f48adffb640 1 -- 172.21.15.40:0/2995997595 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16673 (secure 0 0 0) 0x7f48a80172c0 con 0x7f48b80cf2e0 2024-04-26T21:15:58.011 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:15:58.011 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:56.722134+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:15:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 msgr2=0x7f489c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 0x7f489c06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 msgr2=0x7f48b809f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f48a800b410 tx=0x7f48a800b8d0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 shutdown_connections 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f489c06c8f0 0x7f489c06edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48b80d11b0 0x7f48b809fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48b80cf2e0 0x7f48b809f550 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 --2- 172.21.15.40:0/2995997595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48b806e2c0 0x7f48b81f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 >> 172.21.15.40:0/2995997595 conn(0x7f48b80b4a60 msgr2=0x7f48b8162420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 shutdown_connections 2024-04-26T21:15:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.011+0000 7f48bf5c1640 1 -- 172.21.15.40:0/2995997595 wait complete. 2024-04-26T21:15:58.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:15:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.126+0000 7f70d01b4640 1 Processor -- start 2024-04-26T21:15:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.126+0000 7f70d01b4640 1 -- start start 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70d01b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70c815f680 0x7f70c80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70d01b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70d01b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c80d3f50 con 0x7f70c80b60b0 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70d01b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f70c815fa60 con 0x7f70c815f680 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70d01b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70c80b5090 con 0x7f70c8163aa0 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70ce751640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f70c80b60b0 0x7f70c80d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55636/0 (socket says 172.21.15.40:55636) 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.127+0000 7f70ce751640 1 -- 172.21.15.40:0/2100729395 learned_addr learned my addr 172.21.15.40:0/2100729395 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cd74f640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cdf50640 1 --2- 172.21.15.40:0/2100729395 >> v2:172.21.15.184:3300/0 conn(0x7f70c815f680 0x7f70c80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869070120 0 0) 0x7f70c80d3f50 con 0x7f70c80b60b0 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70b4003520 con 0x7f70c80b60b0 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1077615425 0 0) 0x7f70b4003520 con 0x7f70c80b60b0 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cdf50640 1 -- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 msgr2=0x7f70c80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cdf50640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cdf50640 1 -- 172.21.15.40:0/2100729395 >> v1:172.21.15.40:6789/0 conn(0x7f70c80b60b0 legacy=0x7f70c80d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.128+0000 7f70cdf50640 1 -- 172.21.15.40:0/2100729395 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c80b5370 con 0x7f70c815f680 2024-04-26T21:15:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70cdf50640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d3820 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70c40097a0 tx=0x7f70c402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc68c85214d4d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4037070 con 0x7f70c815f680 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c402aba0 con 0x7f70c815f680 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70ccf4e640 1 -- 172.21.15.40:0/2100729395 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00019f0 con 0x7f70c815f680 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 -- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 msgr2=0x7f70c80d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d3820 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70c40097a0 tx=0x7f70c402a110 comp rx=0 tx=0).stop 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 -- 172.21.15.40:0/2100729395 shutdown_connections 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 --2- 172.21.15.40:0/2100729395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d3820 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.129+0000 7f70d01b4640 1 -- 172.21.15.40:0/2100729395 >> 172.21.15.40:0/2100729395 conn(0x7f70c808eca0 msgr2=0x7f70c809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.130+0000 7f70d01b4640 1 -- 172.21.15.40:0/2100729395 shutdown_connections 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.130+0000 7f70d01b4640 1 -- 172.21.15.40:0/2100729395 wait complete. 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 Processor -- start 2024-04-26T21:15:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 -- start start 2024-04-26T21:15:58.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70c80d57b0 con 0x7f70c80b60b0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70c80d5630 con 0x7f70c815f680 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.131+0000 7f70d01b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70c80d5930 con 0x7f70c8163aa0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40324/0 (socket says 172.21.15.40:40324) 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 -- 172.21.15.40:0/138115356 learned_addr learned my addr 172.21.15.40:0/138115356 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cd74f640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70ce751640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 -- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 msgr2=0x7f70c80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 -- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 msgr2=0x7f70c80d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 -- 172.21.15.40:0/138115356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c80cd110 con 0x7f70c80b60b0 2024-04-26T21:15:58.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70ce751640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70cdf50640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098c0 tx=0x7f70c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4037050 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c4043070 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81f4c20 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4006e10 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.132+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c80d12c0 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.133+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c806a290 con 0x7f70c80b60b0 2024-04-26T21:15:58.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.134+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70c4048460 con 0x7f70c80b60b0 2024-04-26T21:15:58.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.134+0000 7f70be7fc640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 0x7f70ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:58.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.134+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f70c40ac2b0 con 0x7f70c80b60b0 2024-04-26T21:15:58.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.134+0000 7f70cd74f640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 0x7f70ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:58.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.135+0000 7f70cd74f640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 0x7f70ac06ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f70c815cea0 tx=0x7f70b8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:58.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.138+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70c407a6e0 con 0x7f70c80b60b0 2024-04-26T21:15:58.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:58.362+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f70c80b4cd0 con 0x7f70c80b60b0 2024-04-26T21:15:59.332 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-04-26T21:15:59.331+0000 7f1c369a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:15:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.381+0000 7f70be7fc640 1 -- 172.21.15.40:0/138115356 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f70c4079fe0 con 0x7f70c80b60b0 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 msgr2=0x7f70ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 0x7f70ac06ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f70c815cea0 tx=0x7f70b8009190 comp rx=0 tx=0).stop 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 msgr2=0x7f70c81f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098c0 tx=0x7f70c40046c0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 shutdown_connections 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f70ac06c920 0x7f70ac06ede0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8163aa0 0x7f70c80ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815f680 0x7f70c80d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 --2- 172.21.15.40:0/138115356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70c80b60b0 0x7f70c81f46e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 >> 172.21.15.40:0/138115356 conn(0x7f70c808eca0 msgr2=0x7f70c80b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 shutdown_connections 2024-04-26T21:15:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.384+0000 7f70d01b4640 1 -- 172.21.15.40:0/138115356 wait complete. 2024-04-26T21:15:59.399 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:15:59.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:15:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.507+0000 7fe17359e640 1 Processor -- start 2024-04-26T21:15:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.507+0000 7fe17359e640 1 -- start start 2024-04-26T21:15:59.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe17359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe1740d3830 0x7fe1740d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe17359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1740d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe17359e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1740cfb80 con 0x7fe1740d3c10 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe17359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe1740d2ee0 con 0x7fe1740d3830 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe17359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740d31c0 con 0x7fe1740d0600 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe171d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe1740d3c10 0x7fe1740b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55652/0 (socket says 172.21.15.40:55652) 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.508+0000 7fe171d9b640 1 -- 172.21.15.40:0/3457239149 learned_addr learned my addr 172.21.15.40:0/3457239149 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 --2- 172.21.15.40:0/3457239149 >> v2:172.21.15.184:3300/0 conn(0x7fe1740d3830 0x7fe1740d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe172d9d640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1740d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285767193 0 0) 0x7fe15c001110 con 0x7fe1740d3c10 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe160003520 con 0x7fe1740d3c10 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 754841109 0 0) 0x7fe160003520 con 0x7fe1740d3c10 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 msgr2=0x7fe1740d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1740d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 >> v2:172.21.15.184:3300/0 conn(0x7fe1740d3830 msgr2=0x7fe1740d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:59.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 --2- 172.21.15.40:0/3457239149 >> v2:172.21.15.184:3300/0 conn(0x7fe1740d3830 0x7fe1740d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1740cfe00 con 0x7fe1740d3c10 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe15c001840 con 0x7fe1740d3c10 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 >> v1:172.21.15.40:6789/0 conn(0x7fe1740d3c10 legacy=0x7fe1740b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600039b0 0x7fe160003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe160004800 0x7fe160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740d2ee0 con 0x7fe1600039b0 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740d31c0 con 0x7fe160004800 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe16000b9c0 con 0x7fe1740d0600 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600039b0 0x7fe160003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe171d9b640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe160004800 0x7fe160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe172d9d640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 -- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 msgr2=0x7fe1600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1740d0600 0x7fe1600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 -- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe160004800 msgr2=0x7fe160004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe160004800 0x7fe160004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.509+0000 7fe17259c640 1 -- 172.21.15.40:0/3457239149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe160003520 con 0x7fe1600039b0 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.510+0000 7fe17259c640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600039b0 0x7fe160003dd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009ac0 tx=0x7fe160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ce559f61760bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.510+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c036070 con 0x7fe1600039b0 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.510+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c031400 con 0x7fe1600039b0 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.510+0000 7fe17159a640 1 -- 172.21.15.40:0/3457239149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c009690 con 0x7fe1600039b0 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 -- 172.21.15.40:0/3457239149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600039b0 msgr2=0x7fe160003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 --2- 172.21.15.40:0/3457239149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1600039b0 0x7fe160003dd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009ac0 tx=0x7fe160005830 comp rx=0 tx=0).stop 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe172d9d640 1 -- 172.21.15.40:0/3457239149 reap_dead start 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 -- 172.21.15.40:0/3457239149 shutdown_connections 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 -- 172.21.15.40:0/3457239149 >> 172.21.15.40:0/3457239149 conn(0x7fe1740d4fe0 msgr2=0x7fe1740d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 -- 172.21.15.40:0/3457239149 shutdown_connections 2024-04-26T21:15:59.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.511+0000 7fe17359e640 1 -- 172.21.15.40:0/3457239149 wait complete. 2024-04-26T21:15:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.512+0000 7fe17359e640 1 Processor -- start 2024-04-26T21:15:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.512+0000 7fe17359e640 1 -- start start 2024-04-26T21:15:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600039b0 0x7fe1741567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174159210 0x7fe17415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740d2940 con 0x7fe174159690 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740b8540 con 0x7fe174159210 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe17359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740d5d90 con 0x7fe1600039b0 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40336/0 (socket says 172.21.15.40:40336) 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.513+0000 7fe172d9d640 1 -- 172.21.15.40:0/1661656164 learned_addr learned my addr 172.21.15.40:0/1661656164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe171d9b640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174159210 0x7fe17415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe17259c640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600039b0 0x7fe1741567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 -- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600039b0 msgr2=0x7fe1741567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600039b0 0x7fe1741567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 -- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174159210 msgr2=0x7fe17415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174159210 0x7fe17415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 -- 172.21.15.40:0/1661656164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174156cf0 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe172d9d640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe164011bf0 tx=0x7fe16400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16400fe10 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe164015430 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1741581b0 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164015820 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.514+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe174158730 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.515+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe174050730 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.516+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1640159c0 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.516+0000 7fe16affd640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 0x7fe15406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.516+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe16408c8d0 con 0x7fe174159690 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.516+0000 7fe17259c640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 0x7fe15406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.517+0000 7fe17259c640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 0x7fe15406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe16c033000 tx=0x7fe16c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.520+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe164019030 con 0x7fe174159690 2024-04-26T21:15:59.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:15:59.742+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe17415cb40 con 0x7fe174159690 2024-04-26T21:16:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.384+0000 7fe16affd640 1 -- 172.21.15.40:0/1661656164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fe164010040 con 0x7fe174159690 2024-04-26T21:16:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 msgr2=0x7fe15406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 0x7fe15406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe16c033000 tx=0x7fe16c008030 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 msgr2=0x7fe17415d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe164011bf0 tx=0x7fe16400f580 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 shutdown_connections 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe15406c920 0x7fe15406ede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe174159690 0x7fe17415d5a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe174159210 0x7fe17415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 --2- 172.21.15.40:0/1661656164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1600039b0 0x7fe1741567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 >> 172.21.15.40:0/1661656164 conn(0x7fe1740d4fe0 msgr2=0x7fe1740b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 shutdown_connections 2024-04-26T21:16:00.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.387+0000 7fe17359e640 1 -- 172.21.15.40:0/1661656164 wait complete. 2024-04-26T21:16:00.402 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:16:00.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:16:00.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.507+0000 7f3a69483640 1 Processor -- start 2024-04-26T21:16:00.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 -- start start 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a6415f370 0x7f3a640cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a640d3ee0 0x7f3a640b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a640d03a0 con 0x7f3a6415f750 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a640b4d10 con 0x7f3a640d3ee0 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.508+0000 7f3a69483640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3a640d0680 con 0x7f3a6415f370 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a640d3ee0 0x7f3a640b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a640d3ee0 0x7f3a640b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40348/0 (socket says 172.21.15.40:40348) 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 -- 172.21.15.40:0/717411953 learned_addr learned my addr 172.21.15.40:0/717411953 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:00.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a62ffd640 1 --2- 172.21.15.40:0/717411953 >> v2:172.21.15.184:3300/0 conn(0x7f3a6415f370 0x7f3a640cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 -- 172.21.15.40:0/717411953 >> v2:172.21.15.184:3300/0 conn(0x7f3a6415f370 msgr2=0x7f3a640cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:00.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 --2- 172.21.15.40:0/717411953 >> v2:172.21.15.184:3300/0 conn(0x7f3a6415f370 0x7f3a640cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 -- 172.21.15.40:0/717411953 >> v1:172.21.15.184:6789/0 conn(0x7f3a6415f750 legacy=0x7f3a640d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.509+0000 7f3a637fe640 1 -- 172.21.15.40:0/717411953 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a640d0960 con 0x7f3a640d3ee0 2024-04-26T21:16:00.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a637fe640 1 --2- 172.21.15.40:0/717411953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a640b47d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3a640d2d10 tx=0x7f3a5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4852b03a3815ef8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a61ffb640 1 -- 172.21.15.40:0/717411953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54036070 con 0x7f3a640d3ee0 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a61ffb640 1 -- 172.21.15.40:0/717411953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54031600 con 0x7f3a640d3ee0 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a61ffb640 1 -- 172.21.15.40:0/717411953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a540319d0 con 0x7f3a640d3ee0 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 -- 172.21.15.40:0/717411953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 msgr2=0x7f3a640b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 --2- 172.21.15.40:0/717411953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a640b47d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3a640d2d10 tx=0x7f3a5402a730 comp rx=0 tx=0).stop 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 -- 172.21.15.40:0/717411953 shutdown_connections 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 --2- 172.21.15.40:0/717411953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a640b47d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 --2- 172.21.15.40:0/717411953 >> v2:172.21.15.184:3300/0 conn(0x7f3a6415f370 0x7f3a640cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.510+0000 7f3a69483640 1 -- 172.21.15.40:0/717411953 >> 172.21.15.40:0/717411953 conn(0x7f3a6408ec10 msgr2=0x7f3a6409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.511+0000 7f3a69483640 1 -- 172.21.15.40:0/717411953 shutdown_connections 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.511+0000 7f3a69483640 1 -- 172.21.15.40:0/717411953 wait complete. 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.512+0000 7f3a69483640 1 Processor -- start 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.512+0000 7f3a69483640 1 -- start start 2024-04-26T21:16:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 0x7f3a640ba750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6415f750 0x7f3a6414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a640b51c0 con 0x7f3a640d3ee0 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a640d0f20 con 0x7f3a6415f370 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a640b5340 con 0x7f3a6415f750 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40356/0 (socket says 172.21.15.40:40356) 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 -- 172.21.15.40:0/723873399 learned_addr learned my addr 172.21.15.40:0/723873399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 -- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6415f750 msgr2=0x7f3a6414bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6415f750 0x7f3a6414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 -- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 msgr2=0x7f3a640ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a627fc640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 0x7f3a640ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 0x7f3a640ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 -- 172.21.15.40:0/723873399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6414c250 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a627fc640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 0x7f3a640ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a62ffd640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d830 tx=0x7f3a5c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00b6c0 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c019070 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00ba80 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a641f9290 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.513+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641f8cd0 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.514+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6406a290 con 0x7f3a640d3ee0 2024-04-26T21:16:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.515+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a5c01e460 con 0x7f3a640d3ee0 2024-04-26T21:16:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.515+0000 7f3a5b7fe640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 0x7f3a4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.515+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3a5c08c230 con 0x7f3a640d3ee0 2024-04-26T21:16:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.517+0000 7f3a627fc640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 0x7f3a4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.520+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a5c05a660 con 0x7f3a640d3ee0 2024-04-26T21:16:00.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.520+0000 7f3a627fc640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 0x7f3a4406edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ba4a0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.744+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a640d3920 con 0x7f3a640d3ee0 2024-04-26T21:16:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.745+0000 7f3a5b7fe640 1 -- 172.21.15.40:0/723873399 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5186 (secure 0 0 0) 0x7f3a5c059f60 con 0x7f3a640d3ee0 2024-04-26T21:16:00.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:00.747 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:15:59.744328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:16:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 46 2024-04-26T21:16:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 msgr2=0x7f3a4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 0x7f3a4406edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ba4a0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 msgr2=0x7f3a641f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00d830 tx=0x7f3a5c00dcf0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 shutdown_connections 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a4406c8f0 0x7f3a4406edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6415f750 0x7f3a6414bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6415f370 0x7f3a640ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 --2- 172.21.15.40:0/723873399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a640d3ee0 0x7f3a641f8360 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.746+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 >> 172.21.15.40:0/723873399 conn(0x7f3a6408ec10 msgr2=0x7f3a6406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.747+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 shutdown_connections 2024-04-26T21:16:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.747+0000 7f3a69483640 1 -- 172.21.15.40:0/723873399 wait complete. 2024-04-26T21:16:00.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:16:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.858+0000 7f645342a640 1 Processor -- start 2024-04-26T21:16:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.858+0000 7f645342a640 1 -- start start 2024-04-26T21:16:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f645342a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 0x7f644c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f645342a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f644c06e490 0x7f644c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f645342a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c0d1740 con 0x7f644c0d11b0 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f645342a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f644c0d1a20 con 0x7f644c06e490 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f645342a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f644c0d1d00 con 0x7f644c0cf2e0 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f644c06e490 0x7f644c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f644c06e490 0x7f644c15ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46884/0 (socket says 172.21.15.40:46884) 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 -- 172.21.15.40:0/1361575112 learned_addr learned my addr 172.21.15.40:0/1361575112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64511c6640 1 --2- 172.21.15.40:0/1361575112 >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 0x7f644c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 -- 172.21.15.40:0/1361575112 >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 msgr2=0x7f644c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 --2- 172.21.15.40:0/1361575112 >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 0x7f644c06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 -- 172.21.15.40:0/1361575112 >> v1:172.21.15.184:6789/0 conn(0x7f644c0d11b0 legacy=0x7f644c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64519c7640 1 -- 172.21.15.40:0/1361575112 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0d1fe0 con 0x7f644c06e490 2024-04-26T21:16:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.859+0000 7f64511c6640 1 --2- 172.21.15.40:0/1361575112 >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 0x7f644c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.860+0000 7f64519c7640 1 --2- 172.21.15.40:0/1361575112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c06e490 0x7f644c15ebe0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f643c009e50 tx=0x7f643c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe0a65f8c15553b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.860+0000 7f644bfff640 1 -- 172.21.15.40:0/1361575112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c036070 con 0x7f644c06e490 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.860+0000 7f644bfff640 1 -- 172.21.15.40:0/1361575112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c031600 con 0x7f644c06e490 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.860+0000 7f644bfff640 1 -- 172.21.15.40:0/1361575112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0319d0 con 0x7f644c06e490 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 -- 172.21.15.40:0/1361575112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c06e490 msgr2=0x7f644c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 --2- 172.21.15.40:0/1361575112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c06e490 0x7f644c15ebe0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f643c009e50 tx=0x7f643c02a710 comp rx=0 tx=0).stop 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 -- 172.21.15.40:0/1361575112 shutdown_connections 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 --2- 172.21.15.40:0/1361575112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c06e490 0x7f644c15ebe0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 --2- 172.21.15.40:0/1361575112 >> v2:172.21.15.184:3300/0 conn(0x7f644c0cf2e0 0x7f644c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 -- 172.21.15.40:0/1361575112 >> 172.21.15.40:0/1361575112 conn(0x7f644c0b4a60 msgr2=0x7f644c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 -- 172.21.15.40:0/1361575112 shutdown_connections 2024-04-26T21:16:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.861+0000 7f645342a640 1 -- 172.21.15.40:0/1361575112 wait complete. 2024-04-26T21:16:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.862+0000 7f645342a640 1 Processor -- start 2024-04-26T21:16:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.862+0000 7f645342a640 1 -- start start 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f644c06e490 0x7f644c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 0x7f644c0b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f644c0cf6c0 con 0x7f644c0cf2e0 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06da10 con 0x7f644c06e490 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f645342a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f644c0cf840 con 0x7f644c0d11b0 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46888/0 (socket says 172.21.15.40:46888) 2024-04-26T21:16:00.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 -- 172.21.15.40:0/4141814114 learned_addr learned my addr 172.21.15.40:0/4141814114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64511c6640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f644c06e490 0x7f644c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64519c7640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 0x7f644c0b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 -- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 msgr2=0x7f644c0b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 0x7f644c0b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 -- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f644c06e490 msgr2=0x7f644c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f644c06e490 0x7f644c154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64509c5640 1 -- 172.21.15.40:0/4141814114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0b7dd0 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.863+0000 7f64519c7640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 0x7f644c0b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f64509c5640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f644000b3f0 tx=0x7f644000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440015020 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6440011830 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c0b80b0 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440011c20 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.864+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c14f540 con 0x7f644c0cf2e0 2024-04-26T21:16:00.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.865+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c06a290 con 0x7f644c0cf2e0 2024-04-26T21:16:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.865+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6440018020 con 0x7f644c0cf2e0 2024-04-26T21:16:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.866+0000 7f6449ffb640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 0x7f643806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.866+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f644008b540 con 0x7f644c0cf2e0 2024-04-26T21:16:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.866+0000 7f64511c6640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 0x7f643806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.867+0000 7f64511c6640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 0x7f643806ee00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f644400aac0 tx=0x7f6444008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:00.870+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f644001f030 con 0x7f644c0cf2e0 2024-04-26T21:16:01.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.055+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f644c0b8ef0 con 0x7f644c0cf2e0 2024-04-26T21:16:01.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.056+0000 7f6449ffb640 1 -- 172.21.15.40:0/4141814114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16660 (secure 0 0 0) 0x7f6440059ae0 con 0x7f644c0cf2e0 2024-04-26T21:16:01.059 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:01.059 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:15:58.363928+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 msgr2=0x7f643806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 0x7f643806ee00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f644400aac0 tx=0x7f6444008040 comp rx=0 tx=0).stop 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 msgr2=0x7f644c0b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f644000b3f0 tx=0x7f644000b8b0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 shutdown_connections 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f643806c940 0x7f643806ee00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f644c0d11b0 0x7f644c0b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f644c0cf2e0 0x7f644c0b71f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 --2- 172.21.15.40:0/4141814114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f644c06e490 0x7f644c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 >> 172.21.15.40:0/4141814114 conn(0x7f644c0b4a60 msgr2=0x7f644c071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 shutdown_connections 2024-04-26T21:16:01.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.059+0000 7f645342a640 1 -- 172.21.15.40:0/4141814114 wait complete. 2024-04-26T21:16:01.072 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:16:01.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:16:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.171+0000 7f22e1e25640 1 Processor -- start 2024-04-26T21:16:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.171+0000 7f22e1e25640 1 -- start start 2024-04-26T21:16:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22e1e25640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22dc15f340 0x7f22dc0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22e1e25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22e1e25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc0b2510 con 0x7f22dc15f720 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22e1e25640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0b27f0 con 0x7f22dc15f340 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22e1e25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b2ad0 con 0x7f22dc0d5610 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22dc15f340 0x7f22dc0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22dc15f340 0x7f22dc0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:58080/0 (socket says 172.21.15.40:58080) 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22daffd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f22dc15f720 0x7f22dc0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:42202/0 (socket says 172.21.15.40:42202) 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 -- 172.21.15.40:0/923696757 learned_addr learned my addr 172.21.15.40:0/923696757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 -- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 msgr2=0x7f22dc0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 --2- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.172+0000 7f22db7fe640 1 -- 172.21.15.40:0/923696757 >> v1:172.21.15.184:6789/0 conn(0x7f22dc15f720 legacy=0x7f22dc0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.173+0000 7f22db7fe640 1 -- 172.21.15.40:0/923696757 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0b2db0 con 0x7f22dc15f340 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.173+0000 7f22db7fe640 1 --2- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f340 0x7f22dc0b1e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009870 tx=0x7f22cc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83d5664a4f2ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.173+0000 7f22da7fc640 1 -- 172.21.15.40:0/923696757 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc037070 con 0x7f22dc15f340 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22da7fc640 1 -- 172.21.15.40:0/923696757 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc032440 con 0x7f22dc15f340 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22da7fc640 1 -- 172.21.15.40:0/923696757 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc032810 con 0x7f22dc15f340 2024-04-26T21:16:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 -- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f340 msgr2=0x7f22dc0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 --2- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f340 0x7f22dc0b1e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009870 tx=0x7f22cc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 -- 172.21.15.40:0/923696757 shutdown_connections 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 --2- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 --2- 172.21.15.40:0/923696757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f340 0x7f22dc0b1e80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 -- 172.21.15.40:0/923696757 >> 172.21.15.40:0/923696757 conn(0x7f22dc08ec70 msgr2=0x7f22dc09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 -- 172.21.15.40:0/923696757 shutdown_connections 2024-04-26T21:16:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.174+0000 7f22e1e25640 1 -- 172.21.15.40:0/923696757 wait complete. 2024-04-26T21:16:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.175+0000 7f22e1e25640 1 Processor -- start 2024-04-26T21:16:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.175+0000 7f22e1e25640 1 -- start start 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f720 0x7f22dc06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0b3940 con 0x7f22dc15f340 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0b37c0 con 0x7f22dc15f720 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22e1e25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b3ac0 con 0x7f22dc0d5610 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46900/0 (socket says 172.21.15.40:46900) 2024-04-26T21:16:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 -- 172.21.15.40:0/1089838716 learned_addr learned my addr 172.21.15.40:0/1089838716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22dbfff640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f720 0x7f22dc06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22db7fe640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 -- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 msgr2=0x7f22dc150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 -- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f720 msgr2=0x7f22dc06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f720 0x7f22dc06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.176+0000 7f22daffd640 1 -- 172.21.15.40:0/1089838716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0ce390 con 0x7f22dc15f340 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22daffd640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f22d000c440 tx=0x7f22d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00074e0 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d0007680 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc0cd2f0 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0004a20 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.177+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc0cd8b0 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.178+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc06a290 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.178+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22d001a020 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.179+0000 7f22c3fff640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.179+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f22d008b2e0 con 0x7f22dc15f340 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.179+0000 7f22db7fe640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.179+0000 7f22db7fe640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 0x7f22c806edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.183+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22d0010300 con 0x7f22dc15f340 2024-04-26T21:16:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.363+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22dc0d3020 con 0x7f22c806c8f0 2024-04-26T21:16:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.370+0000 7f22c3fff640 1 -- 172.21.15.40:0/1089838716 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252018 (secure 0 0 0) 0x7f22dc0d3020 con 0x7f22c806c8f0 2024-04-26T21:16:01.375 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:01.376 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 msgr2=0x7f22c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 0x7f22c806edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 msgr2=0x7f22dc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f22d000c440 tx=0x7f22d000c900 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 shutdown_connections 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22dc15f720 0x7f22dc06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc15f340 0x7f22dc06e060 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 --2- 172.21.15.40:0/1089838716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22dc0d5610 0x7f22dc150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 >> 172.21.15.40:0/1089838716 conn(0x7f22dc08ec70 msgr2=0x7f22dc0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 shutdown_connections 2024-04-26T21:16:01.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.373+0000 7f22e1e25640 1 -- 172.21.15.40:0/1089838716 wait complete. 2024-04-26T21:16:01.386 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-04-26T21:16:00.973855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258448,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716272,"statfs":{"total":773094113280,"available":772829462528,"internally_reserved":0,"allocated":9199616,"data_stored":4537569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.639078"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381033+0000","last_change":"2024-04-26T21:15:57.374988+0000","last_active":"2024-04-26T21:15:57.381033+0000","last_peered":"2024-04-26T21:15:57.381033+0000","last_clean":"2024-04-26T21:15:57.381033+0000","last_became_active":"2024-04-26T21:15:57.374449+0000","last_became_peered":"2024-04-26T21:15:57.374449+0000","last_unstale":"2024-04-26T21:15:57.381033+0000","last_undegraded":"2024-04-26T21:15:57.381033+0000","last_fullsized":"2024-04-26T21:15:57.381033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:54.654984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387641+0000","last_change":"2024-04-26T21:15:57.373180+0000","last_active":"2024-04-26T21:15:58.387641+0000","last_peered":"2024-04-26T21:15:58.387641+0000","last_clean":"2024-04-26T21:15:58.387641+0000","last_became_active":"2024-04-26T21:15:57.372902+0000","last_became_peered":"2024-04-26T21:15:57.372902+0000","last_unstale":"2024-04-26T21:15:58.387641+0000","last_undegraded":"2024-04-26T21:15:58.387641+0000","last_fullsized":"2024-04-26T21:15:58.387641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:58.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.376598+0000","last_change":"2024-04-26T21:15:57.370058+0000","last_active":"2024-04-26T21:15:57.376598+0000","last_peered":"2024-04-26T21:15:57.376598+0000","last_clean":"2024-04-26T21:15:57.376598+0000","last_became_active":"2024-04-26T21:15:57.369735+0000","last_became_peered":"2024-04-26T21:15:57.369735+0000","last_unstale":"2024-04-26T21:15:57.376598+0000","last_undegraded":"2024-04-26T21:15:57.376598+0000","last_fullsized":"2024-04-26T21:15:57.376598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:47.265334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386296+0000","last_change":"2024-04-26T21:15:57.383762+0000","last_active":"2024-04-26T21:15:57.386296+0000","last_peered":"2024-04-26T21:15:57.386296+0000","last_clean":"2024-04-26T21:15:57.386296+0000","last_became_active":"2024-04-26T21:15:57.383417+0000","last_became_peered":"2024-04-26T21:15:57.383417+0000","last_unstale":"2024-04-26T21:15:57.386296+0000","last_undegraded":"2024-04-26T21:15:57.386296+0000","last_fullsized":"2024-04-26T21:15:57.386296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.272339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378382+0000","last_change":"2024-04-26T21:15:57.370650+0000","last_active":"2024-04-26T21:15:57.378382+0000","last_peered":"2024-04-26T21:15:57.378382+0000","last_clean":"2024-04-26T21:15:57.378382+0000","last_became_active":"2024-04-26T21:15:57.370396+0000","last_became_peered":"2024-04-26T21:15:57.370396+0000","last_unstale":"2024-04-26T21:15:57.378382+0000","last_undegraded":"2024-04-26T21:15:57.378382+0000","last_fullsized":"2024-04-26T21:15:57.378382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:38.711912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383057+0000","last_change":"2024-04-26T21:15:57.379773+0000","last_active":"2024-04-26T21:15:57.383057+0000","last_peered":"2024-04-26T21:15:57.383057+0000","last_clean":"2024-04-26T21:15:57.383057+0000","last_became_active":"2024-04-26T21:15:57.378939+0000","last_became_peered":"2024-04-26T21:15:57.378939+0000","last_unstale":"2024-04-26T21:15:57.383057+0000","last_undegraded":"2024-04-26T21:15:57.383057+0000","last_fullsized":"2024-04-26T21:15:57.383057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:57.529990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.373849+0000","last_change":"2024-04-26T21:15:57.369242+0000","last_active":"2024-04-26T21:15:57.373849+0000","last_peered":"2024-04-26T21:15:57.373849+0000","last_clean":"2024-04-26T21:15:57.373849+0000","last_became_active":"2024-04-26T21:15:57.368958+0000","last_became_peered":"2024-04-26T21:15:57.368958+0000","last_unstale":"2024-04-26T21:15:57.373849+0000","last_undegraded":"2024-04-26T21:15:57.373849+0000","last_fullsized":"2024-04-26T21:15:57.373849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:28.174941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382950+0000","last_change":"2024-04-26T21:15:57.378614+0000","last_active":"2024-04-26T21:15:57.382950+0000","last_peered":"2024-04-26T21:15:57.382950+0000","last_clean":"2024-04-26T21:15:57.382950+0000","last_became_active":"2024-04-26T21:15:57.378214+0000","last_became_peered":"2024-04-26T21:15:57.378214+0000","last_unstale":"2024-04-26T21:15:57.382950+0000","last_undegraded":"2024-04-26T21:15:57.382950+0000","last_fullsized":"2024-04-26T21:15:57.382950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386033+0000","last_change":"2024-04-26T21:15:57.381855+0000","last_active":"2024-04-26T21:15:57.386033+0000","last_peered":"2024-04-26T21:15:57.386033+0000","last_clean":"2024-04-26T21:15:57.386033+0000","last_became_active":"2024-04-26T21:15:57.381527+0000","last_became_peered":"2024-04-26T21:15:57.381527+0000","last_unstale":"2024-04-26T21:15:57.386033+0000","last_undegraded":"2024-04-26T21:15:57.386033+0000","last_fullsized":"2024-04-26T21:15:57.386033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.488433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380910+0000","last_change":"2024-04-26T21:15:57.373900+0000","last_active":"2024-04-26T21:15:57.380910+0000","last_peered":"2024-04-26T21:15:57.380910+0000","last_clean":"2024-04-26T21:15:57.380910+0000","last_became_active":"2024-04-26T21:15:57.372599+0000","last_became_peered":"2024-04-26T21:15:57.372599+0000","last_unstale":"2024-04-26T21:15:57.380910+0000","last_undegraded":"2024-04-26T21:15:57.380910+0000","last_fullsized":"2024-04-26T21:15:57.380910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:05.548167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388424+0000","last_change":"2024-04-26T21:15:57.367655+0000","last_active":"2024-04-26T21:15:58.388424+0000","last_peered":"2024-04-26T21:15:58.388424+0000","last_clean":"2024-04-26T21:15:58.388424+0000","last_became_active":"2024-04-26T21:15:57.367371+0000","last_became_peered":"2024-04-26T21:15:57.367371+0000","last_unstale":"2024-04-26T21:15:58.388424+0000","last_undegraded":"2024-04-26T21:15:58.388424+0000","last_fullsized":"2024-04-26T21:15:58.388424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:05.406933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381144+0000","last_change":"2024-04-26T21:15:57.376317+0000","last_active":"2024-04-26T21:15:57.381144+0000","last_peered":"2024-04-26T21:15:57.381144+0000","last_clean":"2024-04-26T21:15:57.381144+0000","last_became_active":"2024-04-26T21:15:57.372619+0000","last_became_peered":"2024-04-26T21:15:57.372619+0000","last_unstale":"2024-04-26T21:15:57.381144+0000","last_undegraded":"2024-04-26T21:15:57.381144+0000","last_fullsized":"2024-04-26T21:15:57.381144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383160+0000","last_change":"2024-04-26T21:15:57.378270+0000","last_active":"2024-04-26T21:15:57.383160+0000","last_peered":"2024-04-26T21:15:57.383160+0000","last_clean":"2024-04-26T21:15:57.383160+0000","last_became_active":"2024-04-26T21:15:57.377986+0000","last_became_peered":"2024-04-26T21:15:57.377986+0000","last_unstale":"2024-04-26T21:15:57.383160+0000","last_undegraded":"2024-04-26T21:15:57.383160+0000","last_fullsized":"2024-04-26T21:15:57.383160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:13.496498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386259+0000","last_change":"2024-04-26T21:15:57.383791+0000","last_active":"2024-04-26T21:15:57.386259+0000","last_peered":"2024-04-26T21:15:57.386259+0000","last_clean":"2024-04-26T21:15:57.386259+0000","last_became_active":"2024-04-26T21:15:57.383290+0000","last_became_peered":"2024-04-26T21:15:57.383290+0000","last_unstale":"2024-04-26T21:15:57.386259+0000","last_undegraded":"2024-04-26T21:15:57.386259+0000","last_fullsized":"2024-04-26T21:15:57.386259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:41.365811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381814+0000","last_change":"2024-04-26T21:15:57.373298+0000","last_active":"2024-04-26T21:15:57.381814+0000","last_peered":"2024-04-26T21:15:57.381814+0000","last_clean":"2024-04-26T21:15:57.381814+0000","last_became_active":"2024-04-26T21:15:57.373010+0000","last_became_peered":"2024-04-26T21:15:57.373010+0000","last_unstale":"2024-04-26T21:15:57.381814+0000","last_undegraded":"2024-04-26T21:15:57.381814+0000","last_fullsized":"2024-04-26T21:15:57.381814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:02.548714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385427+0000","last_change":"2024-04-26T21:15:57.377659+0000","last_active":"2024-04-26T21:15:57.385427+0000","last_peered":"2024-04-26T21:15:57.385427+0000","last_clean":"2024-04-26T21:15:57.385427+0000","last_became_active":"2024-04-26T21:15:57.376545+0000","last_became_peered":"2024-04-26T21:15:57.376545+0000","last_unstale":"2024-04-26T21:15:57.385427+0000","last_undegraded":"2024-04-26T21:15:57.385427+0000","last_fullsized":"2024-04-26T21:15:57.385427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377268+0000","last_change":"2024-04-26T21:15:57.372989+0000","last_active":"2024-04-26T21:15:57.377268+0000","last_peered":"2024-04-26T21:15:57.377268+0000","last_clean":"2024-04-26T21:15:57.377268+0000","last_became_active":"2024-04-26T21:15:57.371270+0000","last_became_peered":"2024-04-26T21:15:57.371270+0000","last_unstale":"2024-04-26T21:15:57.377268+0000","last_undegraded":"2024-04-26T21:15:57.377268+0000","last_fullsized":"2024-04-26T21:15:57.377268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:34.777058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384241+0000","last_change":"2024-04-26T21:15:57.382178+0000","last_active":"2024-04-26T21:15:57.384241+0000","last_peered":"2024-04-26T21:15:57.384241+0000","last_clean":"2024-04-26T21:15:57.384241+0000","last_became_active":"2024-04-26T21:15:57.381866+0000","last_became_peered":"2024-04-26T21:15:57.381866+0000","last_unstale":"2024-04-26T21:15:57.384241+0000","last_undegraded":"2024-04-26T21:15:57.384241+0000","last_fullsized":"2024-04-26T21:15:57.384241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.278080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.375834+0000","last_change":"2024-04-26T21:15:57.367346+0000","last_active":"2024-04-26T21:15:57.375834+0000","last_peered":"2024-04-26T21:15:57.375834+0000","last_clean":"2024-04-26T21:15:57.375834+0000","last_became_active":"2024-04-26T21:15:57.366946+0000","last_became_peered":"2024-04-26T21:15:57.366946+0000","last_unstale":"2024-04-26T21:15:57.375834+0000","last_undegraded":"2024-04-26T21:15:57.375834+0000","last_fullsized":"2024-04-26T21:15:57.375834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:33.795425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384200+0000","last_change":"2024-04-26T21:15:57.380752+0000","last_active":"2024-04-26T21:15:57.384200+0000","last_peered":"2024-04-26T21:15:57.384200+0000","last_clean":"2024-04-26T21:15:57.384200+0000","last_became_active":"2024-04-26T21:15:57.380451+0000","last_became_peered":"2024-04-26T21:15:57.380451+0000","last_unstale":"2024-04-26T21:15:57.384200+0000","last_undegraded":"2024-04-26T21:15:57.384200+0000","last_fullsized":"2024-04-26T21:15:57.384200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:46.768008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384875+0000","last_change":"2024-04-26T21:15:57.381142+0000","last_active":"2024-04-26T21:15:57.384875+0000","last_peered":"2024-04-26T21:15:57.384875+0000","last_clean":"2024-04-26T21:15:57.384875+0000","last_became_active":"2024-04-26T21:15:57.379965+0000","last_became_peered":"2024-04-26T21:15:57.379965+0000","last_unstale":"2024-04-26T21:15:57.384875+0000","last_undegraded":"2024-04-26T21:15:57.384875+0000","last_fullsized":"2024-04-26T21:15:57.384875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:58.514375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388061+0000","last_change":"2024-04-26T21:15:57.372444+0000","last_active":"2024-04-26T21:15:58.388061+0000","last_peered":"2024-04-26T21:15:58.388061+0000","last_clean":"2024-04-26T21:15:58.388061+0000","last_became_active":"2024-04-26T21:15:57.372051+0000","last_became_peered":"2024-04-26T21:15:57.372051+0000","last_unstale":"2024-04-26T21:15:58.388061+0000","last_undegraded":"2024-04-26T21:15:58.388061+0000","last_fullsized":"2024-04-26T21:15:58.388061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:49.987885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380831+0000","last_change":"2024-04-26T21:15:57.373462+0000","last_active":"2024-04-26T21:15:57.380831+0000","last_peered":"2024-04-26T21:15:57.380831+0000","last_clean":"2024-04-26T21:15:57.380831+0000","last_became_active":"2024-04-26T21:15:57.373093+0000","last_became_peered":"2024-04-26T21:15:57.373093+0000","last_unstale":"2024-04-26T21:15:57.380831+0000","last_undegraded":"2024-04-26T21:15:57.380831+0000","last_fullsized":"2024-04-26T21:15:57.380831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:34.777058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384272+0000","last_change":"2024-04-26T21:15:57.380796+0000","last_active":"2024-04-26T21:15:57.384272+0000","last_peered":"2024-04-26T21:15:57.384272+0000","last_clean":"2024-04-26T21:15:57.384272+0000","last_became_active":"2024-04-26T21:15:57.380473+0000","last_became_peered":"2024-04-26T21:15:57.380473+0000","last_unstale":"2024-04-26T21:15:57.384272+0000","last_undegraded":"2024-04-26T21:15:57.384272+0000","last_fullsized":"2024-04-26T21:15:57.384272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:16.577476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387428+0000","last_change":"2024-04-26T21:15:57.372118+0000","last_active":"2024-04-26T21:15:58.387428+0000","last_peered":"2024-04-26T21:15:58.387428+0000","last_clean":"2024-04-26T21:15:58.387428+0000","last_became_active":"2024-04-26T21:15:57.371792+0000","last_became_peered":"2024-04-26T21:15:57.371792+0000","last_unstale":"2024-04-26T21:15:58.387428+0000","last_undegraded":"2024-04-26T21:15:58.387428+0000","last_fullsized":"2024-04-26T21:15:58.387428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:48.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388514+0000","last_change":"2024-04-26T21:15:57.383565+0000","last_active":"2024-04-26T21:15:57.388514+0000","last_peered":"2024-04-26T21:15:57.388514+0000","last_clean":"2024-04-26T21:15:57.388514+0000","last_became_active":"2024-04-26T21:15:57.383300+0000","last_became_peered":"2024-04-26T21:15:57.383300+0000","last_unstale":"2024-04-26T21:15:57.388514+0000","last_undegraded":"2024-04-26T21:15:57.388514+0000","last_fullsized":"2024-04-26T21:15:57.388514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:09.302484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377959+0000","last_change":"2024-04-26T21:15:57.367724+0000","last_active":"2024-04-26T21:15:57.377959+0000","last_peered":"2024-04-26T21:15:57.377959+0000","last_clean":"2024-04-26T21:15:57.377959+0000","last_became_active":"2024-04-26T21:15:57.367435+0000","last_became_peered":"2024-04-26T21:15:57.367435+0000","last_unstale":"2024-04-26T21:15:57.377959+0000","last_undegraded":"2024-04-26T21:15:57.377959+0000","last_fullsized":"2024-04-26T21:15:57.377959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:28.662470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383274+0000","last_change":"2024-04-26T21:15:57.376702+0000","last_active":"2024-04-26T21:15:57.383274+0000","last_peered":"2024-04-26T21:15:57.383274+0000","last_clean":"2024-04-26T21:15:57.383274+0000","last_became_active":"2024-04-26T21:15:57.374238+0000","last_became_peered":"2024-04-26T21:15:57.374238+0000","last_unstale":"2024-04-26T21:15:57.383274+0000","last_undegraded":"2024-04-26T21:15:57.383274+0000","last_fullsized":"2024-04-26T21:15:57.383274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387786+0000","last_change":"2024-04-26T21:15:57.373192+0000","last_active":"2024-04-26T21:15:58.387786+0000","last_peered":"2024-04-26T21:15:58.387786+0000","last_clean":"2024-04-26T21:15:58.387786+0000","last_became_active":"2024-04-26T21:15:57.369312+0000","last_became_peered":"2024-04-26T21:15:57.369312+0000","last_unstale":"2024-04-26T21:15:58.387786+0000","last_undegraded":"2024-04-26T21:15:58.387786+0000","last_fullsized":"2024-04-26T21:15:58.387786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:35.191238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388204+0000","last_change":"2024-04-26T21:15:57.383773+0000","last_active":"2024-04-26T21:15:58.388204+0000","last_peered":"2024-04-26T21:15:58.388204+0000","last_clean":"2024-04-26T21:15:58.388204+0000","last_became_active":"2024-04-26T21:15:57.383269+0000","last_became_peered":"2024-04-26T21:15:57.383269+0000","last_unstale":"2024-04-26T21:15:58.388204+0000","last_undegraded":"2024-04-26T21:15:58.388204+0000","last_fullsized":"2024-04-26T21:15:58.388204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:37.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377954+0000","last_change":"2024-04-26T21:15:57.374353+0000","last_active":"2024-04-26T21:15:57.377954+0000","last_peered":"2024-04-26T21:15:57.377954+0000","last_clean":"2024-04-26T21:15:57.377954+0000","last_became_active":"2024-04-26T21:15:57.371452+0000","last_became_peered":"2024-04-26T21:15:57.371452+0000","last_unstale":"2024-04-26T21:15:57.377954+0000","last_undegraded":"2024-04-26T21:15:57.377954+0000","last_fullsized":"2024-04-26T21:15:57.377954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:05.548167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388388+0000","last_change":"2024-04-26T21:15:57.381158+0000","last_active":"2024-04-26T21:15:57.388388+0000","last_peered":"2024-04-26T21:15:57.388388+0000","last_clean":"2024-04-26T21:15:57.388388+0000","last_became_active":"2024-04-26T21:15:57.380866+0000","last_became_peered":"2024-04-26T21:15:57.380866+0000","last_unstale":"2024-04-26T21:15:57.388388+0000","last_undegraded":"2024-04-26T21:15:57.388388+0000","last_fullsized":"2024-04-26T21:15:57.388388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.488433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378972+0000","last_change":"2024-04-26T21:15:57.376008+0000","last_active":"2024-04-26T21:15:57.378972+0000","last_peered":"2024-04-26T21:15:57.378972+0000","last_clean":"2024-04-26T21:15:57.378972+0000","last_became_active":"2024-04-26T21:15:57.375716+0000","last_became_peered":"2024-04-26T21:15:57.375716+0000","last_unstale":"2024-04-26T21:15:57.378972+0000","last_undegraded":"2024-04-26T21:15:57.378972+0000","last_fullsized":"2024-04-26T21:15:57.378972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384665+0000","last_change":"2024-04-26T21:15:56.360331+0000","last_active":"2024-04-26T21:15:57.384665+0000","last_peered":"2024-04-26T21:15:57.384665+0000","last_clean":"2024-04-26T21:15:57.384665+0000","last_became_active":"2024-04-26T21:15:56.360043+0000","last_became_peered":"2024-04-26T21:15:56.360043+0000","last_unstale":"2024-04-26T21:15:57.384665+0000","last_undegraded":"2024-04-26T21:15:57.384665+0000","last_fullsized":"2024-04-26T21:15:57.384665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:44.618403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383561+0000","last_change":"2024-04-26T21:15:57.378594+0000","last_active":"2024-04-26T21:15:57.383561+0000","last_peered":"2024-04-26T21:15:57.383561+0000","last_clean":"2024-04-26T21:15:57.383561+0000","last_became_active":"2024-04-26T21:15:57.378345+0000","last_became_peered":"2024-04-26T21:15:57.378345+0000","last_unstale":"2024-04-26T21:15:57.383561+0000","last_undegraded":"2024-04-26T21:15:57.383561+0000","last_fullsized":"2024-04-26T21:15:57.383561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.857200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378758+0000","last_change":"2024-04-26T21:15:56.353313+0000","last_active":"2024-04-26T21:15:57.378758+0000","last_peered":"2024-04-26T21:15:57.378758+0000","last_clean":"2024-04-26T21:15:57.378758+0000","last_became_active":"2024-04-26T21:15:56.353053+0000","last_became_peered":"2024-04-26T21:15:56.353053+0000","last_unstale":"2024-04-26T21:15:57.378758+0000","last_undegraded":"2024-04-26T21:15:57.378758+0000","last_fullsized":"2024-04-26T21:15:57.378758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:44.618403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383297+0000","last_change":"2024-04-26T21:15:57.377394+0000","last_active":"2024-04-26T21:15:57.383297+0000","last_peered":"2024-04-26T21:15:57.383297+0000","last_clean":"2024-04-26T21:15:57.383297+0000","last_became_active":"2024-04-26T21:15:57.377133+0000","last_became_peered":"2024-04-26T21:15:57.377133+0000","last_unstale":"2024-04-26T21:15:57.383297+0000","last_undegraded":"2024-04-26T21:15:57.383297+0000","last_fullsized":"2024-04-26T21:15:57.383297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385722+0000","last_change":"2024-04-26T21:15:56.357994+0000","last_active":"2024-04-26T21:15:57.385722+0000","last_peered":"2024-04-26T21:15:57.385722+0000","last_clean":"2024-04-26T21:15:57.385722+0000","last_became_active":"2024-04-26T21:15:56.357702+0000","last_became_peered":"2024-04-26T21:15:56.357702+0000","last_unstale":"2024-04-26T21:15:57.385722+0000","last_undegraded":"2024-04-26T21:15:57.385722+0000","last_fullsized":"2024-04-26T21:15:57.385722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:48.481603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.376952+0000","last_change":"2024-04-26T21:15:57.372261+0000","last_active":"2024-04-26T21:15:57.376952+0000","last_peered":"2024-04-26T21:15:57.376952+0000","last_clean":"2024-04-26T21:15:57.376952+0000","last_became_active":"2024-04-26T21:15:57.371828+0000","last_became_peered":"2024-04-26T21:15:57.371828+0000","last_unstale":"2024-04-26T21:15:57.376952+0000","last_undegraded":"2024-04-26T21:15:57.376952+0000","last_fullsized":"2024-04-26T21:15:57.376952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:22.551744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385534+0000","last_change":"2024-04-26T21:15:56.358806+0000","last_active":"2024-04-26T21:15:57.385534+0000","last_peered":"2024-04-26T21:15:57.385534+0000","last_clean":"2024-04-26T21:15:57.385534+0000","last_became_active":"2024-04-26T21:15:56.358392+0000","last_became_peered":"2024-04-26T21:15:56.358392+0000","last_unstale":"2024-04-26T21:15:57.385534+0000","last_undegraded":"2024-04-26T21:15:57.385534+0000","last_fullsized":"2024-04-26T21:15:57.385534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:11.757910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388349+0000","last_change":"2024-04-26T21:15:57.383558+0000","last_active":"2024-04-26T21:15:58.388349+0000","last_peered":"2024-04-26T21:15:58.388349+0000","last_clean":"2024-04-26T21:15:58.388349+0000","last_became_active":"2024-04-26T21:15:57.382886+0000","last_became_peered":"2024-04-26T21:15:57.382886+0000","last_unstale":"2024-04-26T21:15:58.388349+0000","last_undegraded":"2024-04-26T21:15:58.388349+0000","last_fullsized":"2024-04-26T21:15:58.388349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:01.368912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384957+0000","last_change":"2024-04-26T21:15:56.354973+0000","last_active":"2024-04-26T21:15:57.384957+0000","last_peered":"2024-04-26T21:15:57.384957+0000","last_clean":"2024-04-26T21:15:57.384957+0000","last_became_active":"2024-04-26T21:15:56.354692+0000","last_became_peered":"2024-04-26T21:15:56.354692+0000","last_unstale":"2024-04-26T21:15:57.384957+0000","last_undegraded":"2024-04-26T21:15:57.384957+0000","last_fullsized":"2024-04-26T21:15:57.384957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:42.437324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378361+0000","last_change":"2024-04-26T21:15:57.369379+0000","last_active":"2024-04-26T21:15:57.378361+0000","last_peered":"2024-04-26T21:15:57.378361+0000","last_clean":"2024-04-26T21:15:57.378361+0000","last_became_active":"2024-04-26T21:15:57.369092+0000","last_became_peered":"2024-04-26T21:15:57.369092+0000","last_unstale":"2024-04-26T21:15:57.378361+0000","last_undegraded":"2024-04-26T21:15:57.378361+0000","last_fullsized":"2024-04-26T21:15:57.378361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:29.762730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382148+0000","last_change":"2024-04-26T21:15:56.353583+0000","last_active":"2024-04-26T21:15:57.382148+0000","last_peered":"2024-04-26T21:15:57.382148+0000","last_clean":"2024-04-26T21:15:57.382148+0000","last_became_active":"2024-04-26T21:15:56.353407+0000","last_became_peered":"2024-04-26T21:15:56.353407+0000","last_unstale":"2024-04-26T21:15:57.382148+0000","last_undegraded":"2024-04-26T21:15:57.382148+0000","last_fullsized":"2024-04-26T21:15:57.382148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:08.816857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377116+0000","last_change":"2024-04-26T21:15:57.372627+0000","last_active":"2024-04-26T21:15:57.377116+0000","last_peered":"2024-04-26T21:15:57.377116+0000","last_clean":"2024-04-26T21:15:57.377116+0000","last_became_active":"2024-04-26T21:15:57.372346+0000","last_became_peered":"2024-04-26T21:15:57.372346+0000","last_unstale":"2024-04-26T21:15:57.377116+0000","last_undegraded":"2024-04-26T21:15:57.377116+0000","last_fullsized":"2024-04-26T21:15:57.377116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:33.795425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.360374+0000","last_change":"2024-04-26T21:15:56.352391+0000","last_active":"2024-04-26T21:15:57.360374+0000","last_peered":"2024-04-26T21:15:57.360374+0000","last_clean":"2024-04-26T21:15:57.360374+0000","last_became_active":"2024-04-26T21:15:56.352127+0000","last_became_peered":"2024-04-26T21:15:56.352127+0000","last_unstale":"2024-04-26T21:15:57.360374+0000","last_undegraded":"2024-04-26T21:15:57.360374+0000","last_fullsized":"2024-04-26T21:15:57.360374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:56.523160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.371083+0000","last_change":"2024-04-26T21:15:57.368471+0000","last_active":"2024-04-26T21:15:57.371083+0000","last_peered":"2024-04-26T21:15:57.371083+0000","last_clean":"2024-04-26T21:15:57.371083+0000","last_became_active":"2024-04-26T21:15:57.368175+0000","last_became_peered":"2024-04-26T21:15:57.368175+0000","last_unstale":"2024-04-26T21:15:57.371083+0000","last_undegraded":"2024-04-26T21:15:57.371083+0000","last_fullsized":"2024-04-26T21:15:57.371083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.301494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387768+0000","last_change":"2024-04-26T21:15:56.353692+0000","last_active":"2024-04-26T21:15:58.387768+0000","last_peered":"2024-04-26T21:15:58.387768+0000","last_clean":"2024-04-26T21:15:58.387768+0000","last_became_active":"2024-04-26T21:15:56.353046+0000","last_became_peered":"2024-04-26T21:15:56.353046+0000","last_unstale":"2024-04-26T21:15:58.387768+0000","last_undegraded":"2024-04-26T21:15:58.387768+0000","last_fullsized":"2024-04-26T21:15:58.387768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:03.824987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382784+0000","last_change":"2024-04-26T21:15:57.374226+0000","last_active":"2024-04-26T21:15:57.382784+0000","last_peered":"2024-04-26T21:15:57.382784+0000","last_clean":"2024-04-26T21:15:57.382784+0000","last_became_active":"2024-04-26T21:15:57.373964+0000","last_became_peered":"2024-04-26T21:15:57.373964+0000","last_unstale":"2024-04-26T21:15:57.382784+0000","last_undegraded":"2024-04-26T21:15:57.382784+0000","last_fullsized":"2024-04-26T21:15:57.382784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:25.987879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366661+0000","last_change":"2024-04-26T21:15:56.357077+0000","last_active":"2024-04-26T21:15:57.366661+0000","last_peered":"2024-04-26T21:15:57.366661+0000","last_clean":"2024-04-26T21:15:57.366661+0000","last_became_active":"2024-04-26T21:15:56.356770+0000","last_became_peered":"2024-04-26T21:15:56.356770+0000","last_unstale":"2024-04-26T21:15:57.366661+0000","last_undegraded":"2024-04-26T21:15:57.366661+0000","last_fullsized":"2024-04-26T21:15:57.366661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:27.655640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380503+0000","last_change":"2024-04-26T21:15:57.372414+0000","last_active":"2024-04-26T21:15:57.380503+0000","last_peered":"2024-04-26T21:15:57.380503+0000","last_clean":"2024-04-26T21:15:57.380503+0000","last_became_active":"2024-04-26T21:15:57.372144+0000","last_became_peered":"2024-04-26T21:15:57.372144+0000","last_unstale":"2024-04-26T21:15:57.380503+0000","last_undegraded":"2024-04-26T21:15:57.380503+0000","last_fullsized":"2024-04-26T21:15:57.380503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.367123+0000","last_change":"2024-04-26T21:15:56.353538+0000","last_active":"2024-04-26T21:15:57.367123+0000","last_peered":"2024-04-26T21:15:57.367123+0000","last_clean":"2024-04-26T21:15:57.367123+0000","last_became_active":"2024-04-26T21:15:56.353177+0000","last_became_peered":"2024-04-26T21:15:56.353177+0000","last_unstale":"2024-04-26T21:15:57.367123+0000","last_undegraded":"2024-04-26T21:15:57.367123+0000","last_fullsized":"2024-04-26T21:15:57.367123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:29.650899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378221+0000","last_change":"2024-04-26T21:15:57.375536+0000","last_active":"2024-04-26T21:15:57.378221+0000","last_peered":"2024-04-26T21:15:57.378221+0000","last_clean":"2024-04-26T21:15:57.378221+0000","last_became_active":"2024-04-26T21:15:57.374047+0000","last_became_peered":"2024-04-26T21:15:57.374047+0000","last_unstale":"2024-04-26T21:15:57.378221+0000","last_undegraded":"2024-04-26T21:15:57.378221+0000","last_fullsized":"2024-04-26T21:15:57.378221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:13.496498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384105+0000","last_change":"2024-04-26T21:15:56.353205+0000","last_active":"2024-04-26T21:15:57.384105+0000","last_peered":"2024-04-26T21:15:57.384105+0000","last_clean":"2024-04-26T21:15:57.384105+0000","last_became_active":"2024-04-26T21:15:56.352929+0000","last_became_peered":"2024-04-26T21:15:56.352929+0000","last_unstale":"2024-04-26T21:15:57.384105+0000","last_undegraded":"2024-04-26T21:15:57.384105+0000","last_fullsized":"2024-04-26T21:15:57.384105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:42.437324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378030+0000","last_change":"2024-04-26T21:15:57.371482+0000","last_active":"2024-04-26T21:15:57.378030+0000","last_peered":"2024-04-26T21:15:57.378030+0000","last_clean":"2024-04-26T21:15:57.378030+0000","last_became_active":"2024-04-26T21:15:57.371212+0000","last_became_peered":"2024-04-26T21:15:57.371212+0000","last_unstale":"2024-04-26T21:15:57.378030+0000","last_undegraded":"2024-04-26T21:15:57.378030+0000","last_fullsized":"2024-04-26T21:15:57.378030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:32.190009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388955+0000","last_change":"2024-04-26T21:15:56.343814+0000","last_active":"2024-04-26T21:15:57.388955+0000","last_peered":"2024-04-26T21:15:57.388955+0000","last_clean":"2024-04-26T21:15:57.388955+0000","last_became_active":"2024-04-26T21:15:56.343509+0000","last_became_peered":"2024-04-26T21:15:56.343509+0000","last_unstale":"2024-04-26T21:15:57.388955+0000","last_undegraded":"2024-04-26T21:15:57.388955+0000","last_fullsized":"2024-04-26T21:15:57.388955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:53.648155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384154+0000","last_change":"2024-04-26T21:15:57.377950+0000","last_active":"2024-04-26T21:15:57.384154+0000","last_peered":"2024-04-26T21:15:57.384154+0000","last_clean":"2024-04-26T21:15:57.384154+0000","last_became_active":"2024-04-26T21:15:57.377622+0000","last_became_peered":"2024-04-26T21:15:57.377622+0000","last_unstale":"2024-04-26T21:15:57.384154+0000","last_undegraded":"2024-04-26T21:15:57.384154+0000","last_fullsized":"2024-04-26T21:15:57.384154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:47.897899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387893+0000","last_change":"2024-04-26T21:15:56.353009+0000","last_active":"2024-04-26T21:15:58.387893+0000","last_peered":"2024-04-26T21:15:58.387893+0000","last_clean":"2024-04-26T21:15:58.387893+0000","last_became_active":"2024-04-26T21:15:56.352553+0000","last_became_peered":"2024-04-26T21:15:56.352553+0000","last_unstale":"2024-04-26T21:15:58.387893+0000","last_undegraded":"2024-04-26T21:15:58.387893+0000","last_fullsized":"2024-04-26T21:15:58.387893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:26.213797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.370622+0000","last_change":"2024-04-26T21:15:57.368703+0000","last_active":"2024-04-26T21:15:57.370622+0000","last_peered":"2024-04-26T21:15:57.370622+0000","last_clean":"2024-04-26T21:15:57.370622+0000","last_became_active":"2024-04-26T21:15:57.368413+0000","last_became_peered":"2024-04-26T21:15:57.368413+0000","last_unstale":"2024-04-26T21:15:57.370622+0000","last_undegraded":"2024-04-26T21:15:57.370622+0000","last_fullsized":"2024-04-26T21:15:57.370622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:12.764740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364189+0000","last_change":"2024-04-26T21:15:56.351391+0000","last_active":"2024-04-26T21:15:57.364189+0000","last_peered":"2024-04-26T21:15:57.364189+0000","last_clean":"2024-04-26T21:15:57.364189+0000","last_became_active":"2024-04-26T21:15:56.351121+0000","last_became_peered":"2024-04-26T21:15:56.351121+0000","last_unstale":"2024-04-26T21:15:57.364189+0000","last_undegraded":"2024-04-26T21:15:57.364189+0000","last_fullsized":"2024-04-26T21:15:57.364189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:08.295654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387916+0000","last_change":"2024-04-26T21:15:57.368742+0000","last_active":"2024-04-26T21:15:58.387916+0000","last_peered":"2024-04-26T21:15:58.387916+0000","last_clean":"2024-04-26T21:15:58.387916+0000","last_became_active":"2024-04-26T21:15:57.368448+0000","last_became_peered":"2024-04-26T21:15:57.368448+0000","last_unstale":"2024-04-26T21:15:58.387916+0000","last_undegraded":"2024-04-26T21:15:58.387916+0000","last_fullsized":"2024-04-26T21:15:58.387916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:09.581989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364423+0000","last_change":"2024-04-26T21:15:56.352175+0000","last_active":"2024-04-26T21:15:57.364423+0000","last_peered":"2024-04-26T21:15:57.364423+0000","last_clean":"2024-04-26T21:15:57.364423+0000","last_became_active":"2024-04-26T21:15:56.351870+0000","last_became_peered":"2024-04-26T21:15:56.351870+0000","last_unstale":"2024-04-26T21:15:57.364423+0000","last_undegraded":"2024-04-26T21:15:57.364423+0000","last_fullsized":"2024-04-26T21:15:57.364423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:04.541337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387976+0000","last_change":"2024-04-26T21:15:57.373045+0000","last_active":"2024-04-26T21:15:58.387976+0000","last_peered":"2024-04-26T21:15:58.387976+0000","last_clean":"2024-04-26T21:15:58.387976+0000","last_became_active":"2024-04-26T21:15:57.372520+0000","last_became_peered":"2024-04-26T21:15:57.372520+0000","last_unstale":"2024-04-26T21:15:58.387976+0000","last_undegraded":"2024-04-26T21:15:58.387976+0000","last_fullsized":"2024-04-26T21:15:58.387976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:56.685228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386521+0000","last_change":"2024-04-26T21:15:56.356138+0000","last_active":"2024-04-26T21:15:57.386521+0000","last_peered":"2024-04-26T21:15:57.386521+0000","last_clean":"2024-04-26T21:15:57.386521+0000","last_became_active":"2024-04-26T21:15:56.355852+0000","last_became_peered":"2024-04-26T21:15:56.355852+0000","last_unstale":"2024-04-26T21:15:57.386521+0000","last_undegraded":"2024-04-26T21:15:57.386521+0000","last_fullsized":"2024-04-26T21:15:57.386521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:08.850370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385602+0000","last_change":"2024-04-26T21:15:57.380191+0000","last_active":"2024-04-26T21:15:57.385602+0000","last_peered":"2024-04-26T21:15:57.385602+0000","last_clean":"2024-04-26T21:15:57.385602+0000","last_became_active":"2024-04-26T21:15:57.379612+0000","last_became_peered":"2024-04-26T21:15:57.379612+0000","last_unstale":"2024-04-26T21:15:57.385602+0000","last_undegraded":"2024-04-26T21:15:57.385602+0000","last_fullsized":"2024-04-26T21:15:57.385602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:33.513570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383498+0000","last_change":"2024-04-26T21:15:56.354493+0000","last_active":"2024-04-26T21:15:57.383498+0000","last_peered":"2024-04-26T21:15:57.383498+0000","last_clean":"2024-04-26T21:15:57.383498+0000","last_became_active":"2024-04-26T21:15:56.354169+0000","last_became_peered":"2024-04-26T21:15:56.354169+0000","last_unstale":"2024-04-26T21:15:57.383498+0000","last_undegraded":"2024-04-26T21:15:57.383498+0000","last_fullsized":"2024-04-26T21:15:57.383498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:28.755901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377241+0000","last_change":"2024-04-26T21:15:57.369918+0000","last_active":"2024-04-26T21:15:57.377241+0000","last_peered":"2024-04-26T21:15:57.377241+0000","last_clean":"2024-04-26T21:15:57.377241+0000","last_became_active":"2024-04-26T21:15:57.369480+0000","last_became_peered":"2024-04-26T21:15:57.369480+0000","last_unstale":"2024-04-26T21:15:57.377241+0000","last_undegraded":"2024-04-26T21:15:57.377241+0000","last_fullsized":"2024-04-26T21:15:57.377241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:38.711912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386303+0000","last_change":"2024-04-26T21:15:56.355055+0000","last_active":"2024-04-26T21:15:57.386303+0000","last_peered":"2024-04-26T21:15:57.386303+0000","last_clean":"2024-04-26T21:15:57.386303+0000","last_became_active":"2024-04-26T21:15:56.354745+0000","last_became_peered":"2024-04-26T21:15:56.354745+0000","last_unstale":"2024-04-26T21:15:57.386303+0000","last_undegraded":"2024-04-26T21:15:57.386303+0000","last_fullsized":"2024-04-26T21:15:57.386303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377993+0000","last_change":"2024-04-26T21:15:57.370004+0000","last_active":"2024-04-26T21:15:57.377993+0000","last_peered":"2024-04-26T21:15:57.377993+0000","last_clean":"2024-04-26T21:15:57.377993+0000","last_became_active":"2024-04-26T21:15:57.369679+0000","last_became_peered":"2024-04-26T21:15:57.369679+0000","last_unstale":"2024-04-26T21:15:57.377993+0000","last_undegraded":"2024-04-26T21:15:57.377993+0000","last_fullsized":"2024-04-26T21:15:57.377993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:30.657729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.361100+0000","last_change":"2024-04-26T21:15:56.351578+0000","last_active":"2024-04-26T21:15:57.361100+0000","last_peered":"2024-04-26T21:15:57.361100+0000","last_clean":"2024-04-26T21:15:57.361100+0000","last_became_active":"2024-04-26T21:15:56.351257+0000","last_became_peered":"2024-04-26T21:15:56.351257+0000","last_unstale":"2024-04-26T21:15:57.361100+0000","last_undegraded":"2024-04-26T21:15:57.361100+0000","last_fullsized":"2024-04-26T21:15:57.361100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:46.258504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388860+0000","last_change":"2024-04-26T21:15:57.384296+0000","last_active":"2024-04-26T21:15:57.388860+0000","last_peered":"2024-04-26T21:15:57.388860+0000","last_clean":"2024-04-26T21:15:57.388860+0000","last_became_active":"2024-04-26T21:15:57.383991+0000","last_became_peered":"2024-04-26T21:15:57.383991+0000","last_unstale":"2024-04-26T21:15:57.388860+0000","last_undegraded":"2024-04-26T21:15:57.388860+0000","last_fullsized":"2024-04-26T21:15:57.388860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:06.382737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382386+0000","last_change":"2024-04-26T21:15:56.344164+0000","last_active":"2024-04-26T21:15:57.382386+0000","last_peered":"2024-04-26T21:15:57.382386+0000","last_clean":"2024-04-26T21:15:57.382386+0000","last_became_active":"2024-04-26T21:15:56.343513+0000","last_became_peered":"2024-04-26T21:15:56.343513+0000","last_unstale":"2024-04-26T21:15:57.382386+0000","last_undegraded":"2024-04-26T21:15:57.382386+0000","last_fullsized":"2024-04-26T21:15:57.382386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:37.870191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385924+0000","last_change":"2024-04-26T21:15:57.380390+0000","last_active":"2024-04-26T21:15:57.385924+0000","last_peered":"2024-04-26T21:15:57.385924+0000","last_clean":"2024-04-26T21:15:57.385924+0000","last_became_active":"2024-04-26T21:15:57.378377+0000","last_became_peered":"2024-04-26T21:15:57.378377+0000","last_unstale":"2024-04-26T21:15:57.385924+0000","last_undegraded":"2024-04-26T21:15:57.385924+0000","last_fullsized":"2024-04-26T21:15:57.385924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.857200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.365186+0000","last_change":"2024-04-26T21:15:56.350179+0000","last_active":"2024-04-26T21:15:57.365186+0000","last_peered":"2024-04-26T21:15:57.365186+0000","last_clean":"2024-04-26T21:15:57.365186+0000","last_became_active":"2024-04-26T21:15:56.349306+0000","last_became_peered":"2024-04-26T21:15:56.349306+0000","last_unstale":"2024-04-26T21:15:57.365186+0000","last_undegraded":"2024-04-26T21:15:57.365186+0000","last_fullsized":"2024-04-26T21:15:57.365186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:51.420883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382980+0000","last_change":"2024-04-26T21:15:57.380472+0000","last_active":"2024-04-26T21:15:57.382980+0000","last_peered":"2024-04-26T21:15:57.382980+0000","last_clean":"2024-04-26T21:15:57.382980+0000","last_became_active":"2024-04-26T21:15:57.380058+0000","last_became_peered":"2024-04-26T21:15:57.380058+0000","last_unstale":"2024-04-26T21:15:57.382980+0000","last_undegraded":"2024-04-26T21:15:57.382980+0000","last_fullsized":"2024-04-26T21:15:57.382980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:38.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364004+0000","last_change":"2024-04-26T21:15:56.354336+0000","last_active":"2024-04-26T21:15:57.364004+0000","last_peered":"2024-04-26T21:15:57.364004+0000","last_clean":"2024-04-26T21:15:57.364004+0000","last_became_active":"2024-04-26T21:15:56.353959+0000","last_became_peered":"2024-04-26T21:15:56.353959+0000","last_unstale":"2024-04-26T21:15:57.364004+0000","last_undegraded":"2024-04-26T21:15:57.364004+0000","last_fullsized":"2024-04-26T21:15:57.364004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:00.971813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386034+0000","last_change":"2024-04-26T21:15:57.381920+0000","last_active":"2024-04-26T21:15:57.386034+0000","last_peered":"2024-04-26T21:15:57.386034+0000","last_clean":"2024-04-26T21:15:57.386034+0000","last_became_active":"2024-04-26T21:15:57.381547+0000","last_became_peered":"2024-04-26T21:15:57.381547+0000","last_unstale":"2024-04-26T21:15:57.386034+0000","last_undegraded":"2024-04-26T21:15:57.386034+0000","last_fullsized":"2024-04-26T21:15:57.386034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:55.348773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.362261+0000","last_change":"2024-04-26T21:15:56.347210+0000","last_active":"2024-04-26T21:15:57.362261+0000","last_peered":"2024-04-26T21:15:57.362261+0000","last_clean":"2024-04-26T21:15:57.362261+0000","last_became_active":"2024-04-26T21:15:56.346921+0000","last_became_peered":"2024-04-26T21:15:56.346921+0000","last_unstale":"2024-04-26T21:15:57.362261+0000","last_undegraded":"2024-04-26T21:15:57.362261+0000","last_fullsized":"2024-04-26T21:15:57.362261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:35.344968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385607+0000","last_change":"2024-04-26T21:15:57.381473+0000","last_active":"2024-04-26T21:15:57.385607+0000","last_peered":"2024-04-26T21:15:57.385607+0000","last_clean":"2024-04-26T21:15:57.385607+0000","last_became_active":"2024-04-26T21:15:57.381097+0000","last_became_peered":"2024-04-26T21:15:57.381097+0000","last_unstale":"2024-04-26T21:15:57.385607+0000","last_undegraded":"2024-04-26T21:15:57.385607+0000","last_fullsized":"2024-04-26T21:15:57.385607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387540+0000","last_change":"2024-04-26T21:15:56.356463+0000","last_active":"2024-04-26T21:15:58.387540+0000","last_peered":"2024-04-26T21:15:58.387540+0000","last_clean":"2024-04-26T21:15:58.387540+0000","last_became_active":"2024-04-26T21:15:56.356102+0000","last_became_peered":"2024-04-26T21:15:56.356102+0000","last_unstale":"2024-04-26T21:15:58.387540+0000","last_undegraded":"2024-04-26T21:15:58.387540+0000","last_fullsized":"2024-04-26T21:15:58.387540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.577161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383284+0000","last_change":"2024-04-26T21:15:57.379372+0000","last_active":"2024-04-26T21:15:57.383284+0000","last_peered":"2024-04-26T21:15:57.383284+0000","last_clean":"2024-04-26T21:15:57.383284+0000","last_became_active":"2024-04-26T21:15:57.379111+0000","last_became_peered":"2024-04-26T21:15:57.379111+0000","last_unstale":"2024-04-26T21:15:57.383284+0000","last_undegraded":"2024-04-26T21:15:57.383284+0000","last_fullsized":"2024-04-26T21:15:57.383284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:47.897899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:59.362560+0000","last_change":"2024-04-26T21:15:56.357277+0000","last_active":"2024-04-26T21:15:59.362560+0000","last_peered":"2024-04-26T21:15:59.362560+0000","last_clean":"2024-04-26T21:15:59.362560+0000","last_became_active":"2024-04-26T21:15:56.356980+0000","last_became_peered":"2024-04-26T21:15:56.356980+0000","last_unstale":"2024-04-26T21:15:59.362560+0000","last_undegraded":"2024-04-26T21:15:59.362560+0000","last_fullsized":"2024-04-26T21:15:59.362560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:14.174870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383317+0000","last_change":"2024-04-26T21:15:57.377814+0000","last_active":"2024-04-26T21:15:57.383317+0000","last_peered":"2024-04-26T21:15:57.383317+0000","last_clean":"2024-04-26T21:15:57.383317+0000","last_became_active":"2024-04-26T21:15:57.377383+0000","last_became_peered":"2024-04-26T21:15:57.377383+0000","last_unstale":"2024-04-26T21:15:57.383317+0000","last_undegraded":"2024-04-26T21:15:57.383317+0000","last_fullsized":"2024-04-26T21:15:57.383317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:57.529990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.365588+0000","last_change":"2024-04-26T21:15:56.360214+0000","last_active":"2024-04-26T21:15:57.365588+0000","last_peered":"2024-04-26T21:15:57.365588+0000","last_clean":"2024-04-26T21:15:57.365588+0000","last_became_active":"2024-04-26T21:15:56.359940+0000","last_became_peered":"2024-04-26T21:15:56.359940+0000","last_unstale":"2024-04-26T21:15:57.365588+0000","last_undegraded":"2024-04-26T21:15:57.365588+0000","last_fullsized":"2024-04-26T21:15:57.365588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:48.572696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378049+0000","last_change":"2024-04-26T21:15:57.374814+0000","last_active":"2024-04-26T21:15:57.378049+0000","last_peered":"2024-04-26T21:15:57.378049+0000","last_clean":"2024-04-26T21:15:57.378049+0000","last_became_active":"2024-04-26T21:15:57.374510+0000","last_became_peered":"2024-04-26T21:15:57.374510+0000","last_unstale":"2024-04-26T21:15:57.378049+0000","last_undegraded":"2024-04-26T21:15:57.378049+0000","last_fullsized":"2024-04-26T21:15:57.378049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:54.654984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366227+0000","last_change":"2024-04-26T21:15:56.357679+0000","last_active":"2024-04-26T21:15:57.366227+0000","last_peered":"2024-04-26T21:15:57.366227+0000","last_clean":"2024-04-26T21:15:57.366227+0000","last_became_active":"2024-04-26T21:15:56.357334+0000","last_became_peered":"2024-04-26T21:15:56.357334+0000","last_unstale":"2024-04-26T21:15:57.366227+0000","last_undegraded":"2024-04-26T21:15:57.366227+0000","last_fullsized":"2024-04-26T21:15:57.366227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:08.816857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383039+0000","last_change":"2024-04-26T21:15:57.379322+0000","last_active":"2024-04-26T21:15:57.383039+0000","last_peered":"2024-04-26T21:15:57.383039+0000","last_clean":"2024-04-26T21:15:57.383039+0000","last_became_active":"2024-04-26T21:15:57.378975+0000","last_became_peered":"2024-04-26T21:15:57.378975+0000","last_unstale":"2024-04-26T21:15:57.383039+0000","last_undegraded":"2024-04-26T21:15:57.383039+0000","last_fullsized":"2024-04-26T21:15:57.383039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.361289+0000","last_change":"2024-04-26T21:15:56.355797+0000","last_active":"2024-04-26T21:15:57.361289+0000","last_peered":"2024-04-26T21:15:57.361289+0000","last_clean":"2024-04-26T21:15:57.361289+0000","last_became_active":"2024-04-26T21:15:56.355460+0000","last_became_peered":"2024-04-26T21:15:56.355460+0000","last_unstale":"2024-04-26T21:15:57.361289+0000","last_undegraded":"2024-04-26T21:15:57.361289+0000","last_fullsized":"2024-04-26T21:15:57.361289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:21.544914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384199+0000","last_change":"2024-04-26T21:15:57.378563+0000","last_active":"2024-04-26T21:15:57.384199+0000","last_peered":"2024-04-26T21:15:57.384199+0000","last_clean":"2024-04-26T21:15:57.384199+0000","last_became_active":"2024-04-26T21:15:57.377998+0000","last_became_peered":"2024-04-26T21:15:57.377998+0000","last_unstale":"2024-04-26T21:15:57.384199+0000","last_undegraded":"2024-04-26T21:15:57.384199+0000","last_fullsized":"2024-04-26T21:15:57.384199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:33.513570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.359440+0000","last_change":"2024-04-26T21:15:56.353413+0000","last_active":"2024-04-26T21:15:57.359440+0000","last_peered":"2024-04-26T21:15:57.359440+0000","last_clean":"2024-04-26T21:15:57.359440+0000","last_became_active":"2024-04-26T21:15:56.353087+0000","last_became_peered":"2024-04-26T21:15:56.353087+0000","last_unstale":"2024-04-26T21:15:57.359440+0000","last_undegraded":"2024-04-26T21:15:57.359440+0000","last_fullsized":"2024-04-26T21:15:57.359440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.710875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388505+0000","last_change":"2024-04-26T21:15:57.383524+0000","last_active":"2024-04-26T21:15:57.388505+0000","last_peered":"2024-04-26T21:15:57.388505+0000","last_clean":"2024-04-26T21:15:57.388505+0000","last_became_active":"2024-04-26T21:15:57.383249+0000","last_became_peered":"2024-04-26T21:15:57.383249+0000","last_unstale":"2024-04-26T21:15:57.388505+0000","last_undegraded":"2024-04-26T21:15:57.388505+0000","last_fullsized":"2024-04-26T21:15:57.388505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:36.351797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366089+0000","last_change":"2024-04-26T21:15:56.357085+0000","last_active":"2024-04-26T21:15:57.366089+0000","last_peered":"2024-04-26T21:15:57.366089+0000","last_clean":"2024-04-26T21:15:57.366089+0000","last_became_active":"2024-04-26T21:15:56.356768+0000","last_became_peered":"2024-04-26T21:15:56.356768+0000","last_unstale":"2024-04-26T21:15:57.366089+0000","last_undegraded":"2024-04-26T21:15:57.366089+0000","last_fullsized":"2024-04-26T21:15:57.366089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.710875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380808+0000","last_change":"2024-04-26T21:15:57.372228+0000","last_active":"2024-04-26T21:15:57.380808+0000","last_peered":"2024-04-26T21:15:57.380808+0000","last_clean":"2024-04-26T21:15:57.380808+0000","last_became_active":"2024-04-26T21:15:57.371956+0000","last_became_peered":"2024-04-26T21:15:57.371956+0000","last_unstale":"2024-04-26T21:15:57.380808+0000","last_undegraded":"2024-04-26T21:15:57.380808+0000","last_fullsized":"2024-04-26T21:15:57.380808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:32.190009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.358129+0000","last_change":"2024-04-26T21:15:56.352990+0000","last_active":"2024-04-26T21:15:57.358129+0000","last_peered":"2024-04-26T21:15:57.358129+0000","last_clean":"2024-04-26T21:15:57.358129+0000","last_became_active":"2024-04-26T21:15:56.352667+0000","last_became_peered":"2024-04-26T21:15:56.352667+0000","last_unstale":"2024-04-26T21:15:57.358129+0000","last_undegraded":"2024-04-26T21:15:57.358129+0000","last_fullsized":"2024-04-26T21:15:57.358129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:48.572696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388252+0000","last_change":"2024-04-26T21:15:57.380461+0000","last_active":"2024-04-26T21:15:58.388252+0000","last_peered":"2024-04-26T21:15:58.388252+0000","last_clean":"2024-04-26T21:15:58.388252+0000","last_became_active":"2024-04-26T21:15:57.379448+0000","last_became_peered":"2024-04-26T21:15:57.379448+0000","last_unstale":"2024-04-26T21:15:58.388252+0000","last_undegraded":"2024-04-26T21:15:58.388252+0000","last_fullsized":"2024-04-26T21:15:58.388252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:57.681128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384995+0000","last_change":"2024-04-26T21:15:56.351248+0000","last_active":"2024-04-26T21:15:57.384995+0000","last_peered":"2024-04-26T21:15:57.384995+0000","last_clean":"2024-04-26T21:15:57.384995+0000","last_became_active":"2024-04-26T21:15:56.350939+0000","last_became_peered":"2024-04-26T21:15:56.350939+0000","last_unstale":"2024-04-26T21:15:57.384995+0000","last_undegraded":"2024-04-26T21:15:57.384995+0000","last_fullsized":"2024-04-26T21:15:57.384995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":277,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.362727+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:57.362727+0000","last_peered":"2024-04-26T21:15:57.362727+0000","last_clean":"2024-04-26T21:15:57.362727+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:57.362727+0000","last_undegraded":"2024-04-26T21:15:57.362727+0000","last_fullsized":"2024-04-26T21:15:57.362727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1597440,"data_stored":1011474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":421730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":421036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":422648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":1004629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":427530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:16:01.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:16:01.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.531+0000 7fec737b4640 1 Processor -- start 2024-04-26T21:16:01.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.531+0000 7fec737b4640 1 -- start start 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec737b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fec6c06e490 0x7fec6c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec737b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6c0d1740 con 0x7fec6c0d11b0 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec737b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6c0d1a20 con 0x7fec6c0cf2e0 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec737b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fec6c0d1d00 con 0x7fec6c06e490 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec70d4f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec6c0d11b0 0x7fec6c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59402/0 (socket says 172.21.15.40:59402) 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.532+0000 7fec70d4f640 1 -- 172.21.15.40:0/352932278 learned_addr learned my addr 172.21.15.40:0/352932278 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088112043 0 0) 0x7fec6c0d1740 con 0x7fec6c0d11b0 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec58003520 con 0x7fec6c0d11b0 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 --2- 172.21.15.40:0/352932278 >> v2:172.21.15.184:3300/0 conn(0x7fec6c06e490 0x7fec6c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 373435157 0 0) 0x7fec58003520 con 0x7fec6c0d11b0 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 >> v2:172.21.15.184:3300/0 conn(0x7fec6c06e490 msgr2=0x7fec6c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 --2- 172.21.15.40:0/352932278 >> v2:172.21.15.184:3300/0 conn(0x7fec6c06e490 0x7fec6c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 >> v1:172.21.15.184:6789/0 conn(0x7fec6c0cf2e0 legacy=0x7fec6c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c0d1fe0 con 0x7fec6c0d11b0 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fec5c001670 con 0x7fec6c0d11b0 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 >> v1:172.21.15.40:6789/0 conn(0x7fec6c0d11b0 legacy=0x7fec6c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec580039b0 0x7fec58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec58004800 0x7fec58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 0x7fec5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c0d1d00 con 0x7fec580051c0 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800c060 con 0x7fec580039b0 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec5800c260 con 0x7fec58004800 2024-04-26T21:16:01.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 0x7fec5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec70d4f640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec58004800 0x7fec58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71550640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec580039b0 0x7fec58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 -- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec58004800 msgr2=0x7fec58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec58004800 0x7fec58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 -- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec580039b0 msgr2=0x7fec58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec580039b0 0x7fec58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.533+0000 7fec71d51640 1 -- 172.21.15.40:0/352932278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58003520 con 0x7fec580051c0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.534+0000 7fec71d51640 1 -- 172.21.15.40:0/352932278 reap_dead start 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.534+0000 7fec71d51640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 0x7fec5800b9c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fec6c074240 tx=0x7fec58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642943301150aadc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.534+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64009c10 con 0x7fec580051c0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.534+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64031410 con 0x7fec580051c0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.534+0000 7fec63fff640 1 -- 172.21.15.40:0/352932278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec640317e0 con 0x7fec580051c0 2024-04-26T21:16:01.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 -- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 msgr2=0x7fec5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 0x7fec5800b9c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fec6c074240 tx=0x7fec58005830 comp rx=0 tx=0).stop 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 -- 172.21.15.40:0/352932278 shutdown_connections 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 --2- 172.21.15.40:0/352932278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec580051c0 0x7fec5800b9c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 -- 172.21.15.40:0/352932278 >> 172.21.15.40:0/352932278 conn(0x7fec6c0b4a60 msgr2=0x7fec6c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 -- 172.21.15.40:0/352932278 shutdown_connections 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.535+0000 7fec737b4640 1 -- 172.21.15.40:0/352932278 wait complete. 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.536+0000 7fec737b4640 1 Processor -- start 2024-04-26T21:16:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.536+0000 7fec737b4640 1 -- start start 2024-04-26T21:16:01.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec580051c0 0x7fec6c09df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c1f4f20 0x7fec6c1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c0cf840 con 0x7fec6c09ed60 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c0cf6c0 con 0x7fec6c1f4f20 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec737b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fec6c06dd80 con 0x7fec580051c0 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46918/0 (socket says 172.21.15.40:46918) 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 -- 172.21.15.40:0/1832212491 learned_addr learned my addr 172.21.15.40:0/1832212491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec71d51640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c1f4f20 0x7fec6c1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 -- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec580051c0 msgr2=0x7fec6c09df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec580051c0 0x7fec6c09df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 -- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c1f4f20 msgr2=0x7fec6c1f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c1f4f20 0x7fec6c1f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 -- 172.21.15.40:0/1832212491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1f5e80 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.537+0000 7fec70d4f640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007290 tx=0x7fec5c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.538+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00f960 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.538+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5c018070 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.538+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00fd40 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.538+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1f58e0 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.538+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c09d4a0 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.539+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6c06a290 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.539+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec5c01d460 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.539+0000 7fec61ffb640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.540+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fec5c08bb40 con 0x7fec6c09ed60 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.540+0000 7fec71550640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.540+0000 7fec71550640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 0x7fec5006edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.544+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec5c05a000 con 0x7fec6c09ed60 2024-04-26T21:16:01.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:16:01.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:16:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.727+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec6c0cda60 con 0x7fec5006c8f0 2024-04-26T21:16:01.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.734+0000 7fec61ffb640 1 -- 172.21.15.40:0/1832212491 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252018 (secure 0 0 0) 0x7fec68001c60 con 0x7fec5006c8f0 2024-04-26T21:16:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:01.741 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 msgr2=0x7fec5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 0x7fec5006edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec68009040 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 msgr2=0x7fec6c09f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007290 tx=0x7fec5c007750 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 shutdown_connections 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c1f4f20 0x7fec6c1f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6c09ed60 0x7fec6c09f1e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 --2- 172.21.15.40:0/1832212491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec580051c0 0x7fec6c09df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.736+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 >> 172.21.15.40:0/1832212491 conn(0x7fec6c0b4a60 msgr2=0x7fec6c162ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.737+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 shutdown_connections 2024-04-26T21:16:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.737+0000 7fec737b4640 1 -- 172.21.15.40:0/1832212491 wait complete. 2024-04-26T21:16:01.749 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-04-26T21:16:00.973855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258448,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716272,"statfs":{"total":773094113280,"available":772829462528,"internally_reserved":0,"allocated":9199616,"data_stored":4537569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.639078"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381033+0000","last_change":"2024-04-26T21:15:57.374988+0000","last_active":"2024-04-26T21:15:57.381033+0000","last_peered":"2024-04-26T21:15:57.381033+0000","last_clean":"2024-04-26T21:15:57.381033+0000","last_became_active":"2024-04-26T21:15:57.374449+0000","last_became_peered":"2024-04-26T21:15:57.374449+0000","last_unstale":"2024-04-26T21:15:57.381033+0000","last_undegraded":"2024-04-26T21:15:57.381033+0000","last_fullsized":"2024-04-26T21:15:57.381033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:54.654984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387641+0000","last_change":"2024-04-26T21:15:57.373180+0000","last_active":"2024-04-26T21:15:58.387641+0000","last_peered":"2024-04-26T21:15:58.387641+0000","last_clean":"2024-04-26T21:15:58.387641+0000","last_became_active":"2024-04-26T21:15:57.372902+0000","last_became_peered":"2024-04-26T21:15:57.372902+0000","last_unstale":"2024-04-26T21:15:58.387641+0000","last_undegraded":"2024-04-26T21:15:58.387641+0000","last_fullsized":"2024-04-26T21:15:58.387641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:58.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.376598+0000","last_change":"2024-04-26T21:15:57.370058+0000","last_active":"2024-04-26T21:15:57.376598+0000","last_peered":"2024-04-26T21:15:57.376598+0000","last_clean":"2024-04-26T21:15:57.376598+0000","last_became_active":"2024-04-26T21:15:57.369735+0000","last_became_peered":"2024-04-26T21:15:57.369735+0000","last_unstale":"2024-04-26T21:15:57.376598+0000","last_undegraded":"2024-04-26T21:15:57.376598+0000","last_fullsized":"2024-04-26T21:15:57.376598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:47.265334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386296+0000","last_change":"2024-04-26T21:15:57.383762+0000","last_active":"2024-04-26T21:15:57.386296+0000","last_peered":"2024-04-26T21:15:57.386296+0000","last_clean":"2024-04-26T21:15:57.386296+0000","last_became_active":"2024-04-26T21:15:57.383417+0000","last_became_peered":"2024-04-26T21:15:57.383417+0000","last_unstale":"2024-04-26T21:15:57.386296+0000","last_undegraded":"2024-04-26T21:15:57.386296+0000","last_fullsized":"2024-04-26T21:15:57.386296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:16.272339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378382+0000","last_change":"2024-04-26T21:15:57.370650+0000","last_active":"2024-04-26T21:15:57.378382+0000","last_peered":"2024-04-26T21:15:57.378382+0000","last_clean":"2024-04-26T21:15:57.378382+0000","last_became_active":"2024-04-26T21:15:57.370396+0000","last_became_peered":"2024-04-26T21:15:57.370396+0000","last_unstale":"2024-04-26T21:15:57.378382+0000","last_undegraded":"2024-04-26T21:15:57.378382+0000","last_fullsized":"2024-04-26T21:15:57.378382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:38.711912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383057+0000","last_change":"2024-04-26T21:15:57.379773+0000","last_active":"2024-04-26T21:15:57.383057+0000","last_peered":"2024-04-26T21:15:57.383057+0000","last_clean":"2024-04-26T21:15:57.383057+0000","last_became_active":"2024-04-26T21:15:57.378939+0000","last_became_peered":"2024-04-26T21:15:57.378939+0000","last_unstale":"2024-04-26T21:15:57.383057+0000","last_undegraded":"2024-04-26T21:15:57.383057+0000","last_fullsized":"2024-04-26T21:15:57.383057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:57.529990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.373849+0000","last_change":"2024-04-26T21:15:57.369242+0000","last_active":"2024-04-26T21:15:57.373849+0000","last_peered":"2024-04-26T21:15:57.373849+0000","last_clean":"2024-04-26T21:15:57.373849+0000","last_became_active":"2024-04-26T21:15:57.368958+0000","last_became_peered":"2024-04-26T21:15:57.368958+0000","last_unstale":"2024-04-26T21:15:57.373849+0000","last_undegraded":"2024-04-26T21:15:57.373849+0000","last_fullsized":"2024-04-26T21:15:57.373849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:28.174941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382950+0000","last_change":"2024-04-26T21:15:57.378614+0000","last_active":"2024-04-26T21:15:57.382950+0000","last_peered":"2024-04-26T21:15:57.382950+0000","last_clean":"2024-04-26T21:15:57.382950+0000","last_became_active":"2024-04-26T21:15:57.378214+0000","last_became_peered":"2024-04-26T21:15:57.378214+0000","last_unstale":"2024-04-26T21:15:57.382950+0000","last_undegraded":"2024-04-26T21:15:57.382950+0000","last_fullsized":"2024-04-26T21:15:57.382950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386033+0000","last_change":"2024-04-26T21:15:57.381855+0000","last_active":"2024-04-26T21:15:57.386033+0000","last_peered":"2024-04-26T21:15:57.386033+0000","last_clean":"2024-04-26T21:15:57.386033+0000","last_became_active":"2024-04-26T21:15:57.381527+0000","last_became_peered":"2024-04-26T21:15:57.381527+0000","last_unstale":"2024-04-26T21:15:57.386033+0000","last_undegraded":"2024-04-26T21:15:57.386033+0000","last_fullsized":"2024-04-26T21:15:57.386033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.488433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380910+0000","last_change":"2024-04-26T21:15:57.373900+0000","last_active":"2024-04-26T21:15:57.380910+0000","last_peered":"2024-04-26T21:15:57.380910+0000","last_clean":"2024-04-26T21:15:57.380910+0000","last_became_active":"2024-04-26T21:15:57.372599+0000","last_became_peered":"2024-04-26T21:15:57.372599+0000","last_unstale":"2024-04-26T21:15:57.380910+0000","last_undegraded":"2024-04-26T21:15:57.380910+0000","last_fullsized":"2024-04-26T21:15:57.380910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:05.548167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388424+0000","last_change":"2024-04-26T21:15:57.367655+0000","last_active":"2024-04-26T21:15:58.388424+0000","last_peered":"2024-04-26T21:15:58.388424+0000","last_clean":"2024-04-26T21:15:58.388424+0000","last_became_active":"2024-04-26T21:15:57.367371+0000","last_became_peered":"2024-04-26T21:15:57.367371+0000","last_unstale":"2024-04-26T21:15:58.388424+0000","last_undegraded":"2024-04-26T21:15:58.388424+0000","last_fullsized":"2024-04-26T21:15:58.388424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:05.406933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381144+0000","last_change":"2024-04-26T21:15:57.376317+0000","last_active":"2024-04-26T21:15:57.381144+0000","last_peered":"2024-04-26T21:15:57.381144+0000","last_clean":"2024-04-26T21:15:57.381144+0000","last_became_active":"2024-04-26T21:15:57.372619+0000","last_became_peered":"2024-04-26T21:15:57.372619+0000","last_unstale":"2024-04-26T21:15:57.381144+0000","last_undegraded":"2024-04-26T21:15:57.381144+0000","last_fullsized":"2024-04-26T21:15:57.381144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383160+0000","last_change":"2024-04-26T21:15:57.378270+0000","last_active":"2024-04-26T21:15:57.383160+0000","last_peered":"2024-04-26T21:15:57.383160+0000","last_clean":"2024-04-26T21:15:57.383160+0000","last_became_active":"2024-04-26T21:15:57.377986+0000","last_became_peered":"2024-04-26T21:15:57.377986+0000","last_unstale":"2024-04-26T21:15:57.383160+0000","last_undegraded":"2024-04-26T21:15:57.383160+0000","last_fullsized":"2024-04-26T21:15:57.383160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:13.496498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386259+0000","last_change":"2024-04-26T21:15:57.383791+0000","last_active":"2024-04-26T21:15:57.386259+0000","last_peered":"2024-04-26T21:15:57.386259+0000","last_clean":"2024-04-26T21:15:57.386259+0000","last_became_active":"2024-04-26T21:15:57.383290+0000","last_became_peered":"2024-04-26T21:15:57.383290+0000","last_unstale":"2024-04-26T21:15:57.386259+0000","last_undegraded":"2024-04-26T21:15:57.386259+0000","last_fullsized":"2024-04-26T21:15:57.386259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:41.365811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.381814+0000","last_change":"2024-04-26T21:15:57.373298+0000","last_active":"2024-04-26T21:15:57.381814+0000","last_peered":"2024-04-26T21:15:57.381814+0000","last_clean":"2024-04-26T21:15:57.381814+0000","last_became_active":"2024-04-26T21:15:57.373010+0000","last_became_peered":"2024-04-26T21:15:57.373010+0000","last_unstale":"2024-04-26T21:15:57.381814+0000","last_undegraded":"2024-04-26T21:15:57.381814+0000","last_fullsized":"2024-04-26T21:15:57.381814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:02.548714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385427+0000","last_change":"2024-04-26T21:15:57.377659+0000","last_active":"2024-04-26T21:15:57.385427+0000","last_peered":"2024-04-26T21:15:57.385427+0000","last_clean":"2024-04-26T21:15:57.385427+0000","last_became_active":"2024-04-26T21:15:57.376545+0000","last_became_peered":"2024-04-26T21:15:57.376545+0000","last_unstale":"2024-04-26T21:15:57.385427+0000","last_undegraded":"2024-04-26T21:15:57.385427+0000","last_fullsized":"2024-04-26T21:15:57.385427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377268+0000","last_change":"2024-04-26T21:15:57.372989+0000","last_active":"2024-04-26T21:15:57.377268+0000","last_peered":"2024-04-26T21:15:57.377268+0000","last_clean":"2024-04-26T21:15:57.377268+0000","last_became_active":"2024-04-26T21:15:57.371270+0000","last_became_peered":"2024-04-26T21:15:57.371270+0000","last_unstale":"2024-04-26T21:15:57.377268+0000","last_undegraded":"2024-04-26T21:15:57.377268+0000","last_fullsized":"2024-04-26T21:15:57.377268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:34.777058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384241+0000","last_change":"2024-04-26T21:15:57.382178+0000","last_active":"2024-04-26T21:15:57.384241+0000","last_peered":"2024-04-26T21:15:57.384241+0000","last_clean":"2024-04-26T21:15:57.384241+0000","last_became_active":"2024-04-26T21:15:57.381866+0000","last_became_peered":"2024-04-26T21:15:57.381866+0000","last_unstale":"2024-04-26T21:15:57.384241+0000","last_undegraded":"2024-04-26T21:15:57.384241+0000","last_fullsized":"2024-04-26T21:15:57.384241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.278080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.375834+0000","last_change":"2024-04-26T21:15:57.367346+0000","last_active":"2024-04-26T21:15:57.375834+0000","last_peered":"2024-04-26T21:15:57.375834+0000","last_clean":"2024-04-26T21:15:57.375834+0000","last_became_active":"2024-04-26T21:15:57.366946+0000","last_became_peered":"2024-04-26T21:15:57.366946+0000","last_unstale":"2024-04-26T21:15:57.375834+0000","last_undegraded":"2024-04-26T21:15:57.375834+0000","last_fullsized":"2024-04-26T21:15:57.375834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:33.795425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384200+0000","last_change":"2024-04-26T21:15:57.380752+0000","last_active":"2024-04-26T21:15:57.384200+0000","last_peered":"2024-04-26T21:15:57.384200+0000","last_clean":"2024-04-26T21:15:57.384200+0000","last_became_active":"2024-04-26T21:15:57.380451+0000","last_became_peered":"2024-04-26T21:15:57.380451+0000","last_unstale":"2024-04-26T21:15:57.384200+0000","last_undegraded":"2024-04-26T21:15:57.384200+0000","last_fullsized":"2024-04-26T21:15:57.384200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:46.768008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384875+0000","last_change":"2024-04-26T21:15:57.381142+0000","last_active":"2024-04-26T21:15:57.384875+0000","last_peered":"2024-04-26T21:15:57.384875+0000","last_clean":"2024-04-26T21:15:57.384875+0000","last_became_active":"2024-04-26T21:15:57.379965+0000","last_became_peered":"2024-04-26T21:15:57.379965+0000","last_unstale":"2024-04-26T21:15:57.384875+0000","last_undegraded":"2024-04-26T21:15:57.384875+0000","last_fullsized":"2024-04-26T21:15:57.384875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:58.514375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388061+0000","last_change":"2024-04-26T21:15:57.372444+0000","last_active":"2024-04-26T21:15:58.388061+0000","last_peered":"2024-04-26T21:15:58.388061+0000","last_clean":"2024-04-26T21:15:58.388061+0000","last_became_active":"2024-04-26T21:15:57.372051+0000","last_became_peered":"2024-04-26T21:15:57.372051+0000","last_unstale":"2024-04-26T21:15:58.388061+0000","last_undegraded":"2024-04-26T21:15:58.388061+0000","last_fullsized":"2024-04-26T21:15:58.388061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:49.987885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380831+0000","last_change":"2024-04-26T21:15:57.373462+0000","last_active":"2024-04-26T21:15:57.380831+0000","last_peered":"2024-04-26T21:15:57.380831+0000","last_clean":"2024-04-26T21:15:57.380831+0000","last_became_active":"2024-04-26T21:15:57.373093+0000","last_became_peered":"2024-04-26T21:15:57.373093+0000","last_unstale":"2024-04-26T21:15:57.380831+0000","last_undegraded":"2024-04-26T21:15:57.380831+0000","last_fullsized":"2024-04-26T21:15:57.380831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:34.777058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384272+0000","last_change":"2024-04-26T21:15:57.380796+0000","last_active":"2024-04-26T21:15:57.384272+0000","last_peered":"2024-04-26T21:15:57.384272+0000","last_clean":"2024-04-26T21:15:57.384272+0000","last_became_active":"2024-04-26T21:15:57.380473+0000","last_became_peered":"2024-04-26T21:15:57.380473+0000","last_unstale":"2024-04-26T21:15:57.384272+0000","last_undegraded":"2024-04-26T21:15:57.384272+0000","last_fullsized":"2024-04-26T21:15:57.384272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:16.577476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387428+0000","last_change":"2024-04-26T21:15:57.372118+0000","last_active":"2024-04-26T21:15:58.387428+0000","last_peered":"2024-04-26T21:15:58.387428+0000","last_clean":"2024-04-26T21:15:58.387428+0000","last_became_active":"2024-04-26T21:15:57.371792+0000","last_became_peered":"2024-04-26T21:15:57.371792+0000","last_unstale":"2024-04-26T21:15:58.387428+0000","last_undegraded":"2024-04-26T21:15:58.387428+0000","last_fullsized":"2024-04-26T21:15:58.387428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:48.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388514+0000","last_change":"2024-04-26T21:15:57.383565+0000","last_active":"2024-04-26T21:15:57.388514+0000","last_peered":"2024-04-26T21:15:57.388514+0000","last_clean":"2024-04-26T21:15:57.388514+0000","last_became_active":"2024-04-26T21:15:57.383300+0000","last_became_peered":"2024-04-26T21:15:57.383300+0000","last_unstale":"2024-04-26T21:15:57.388514+0000","last_undegraded":"2024-04-26T21:15:57.388514+0000","last_fullsized":"2024-04-26T21:15:57.388514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:09.302484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377959+0000","last_change":"2024-04-26T21:15:57.367724+0000","last_active":"2024-04-26T21:15:57.377959+0000","last_peered":"2024-04-26T21:15:57.377959+0000","last_clean":"2024-04-26T21:15:57.377959+0000","last_became_active":"2024-04-26T21:15:57.367435+0000","last_became_peered":"2024-04-26T21:15:57.367435+0000","last_unstale":"2024-04-26T21:15:57.377959+0000","last_undegraded":"2024-04-26T21:15:57.377959+0000","last_fullsized":"2024-04-26T21:15:57.377959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:28.662470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383274+0000","last_change":"2024-04-26T21:15:57.376702+0000","last_active":"2024-04-26T21:15:57.383274+0000","last_peered":"2024-04-26T21:15:57.383274+0000","last_clean":"2024-04-26T21:15:57.383274+0000","last_became_active":"2024-04-26T21:15:57.374238+0000","last_became_peered":"2024-04-26T21:15:57.374238+0000","last_unstale":"2024-04-26T21:15:57.383274+0000","last_undegraded":"2024-04-26T21:15:57.383274+0000","last_fullsized":"2024-04-26T21:15:57.383274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:26.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387786+0000","last_change":"2024-04-26T21:15:57.373192+0000","last_active":"2024-04-26T21:15:58.387786+0000","last_peered":"2024-04-26T21:15:58.387786+0000","last_clean":"2024-04-26T21:15:58.387786+0000","last_became_active":"2024-04-26T21:15:57.369312+0000","last_became_peered":"2024-04-26T21:15:57.369312+0000","last_unstale":"2024-04-26T21:15:58.387786+0000","last_undegraded":"2024-04-26T21:15:58.387786+0000","last_fullsized":"2024-04-26T21:15:58.387786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:35.191238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388204+0000","last_change":"2024-04-26T21:15:57.383773+0000","last_active":"2024-04-26T21:15:58.388204+0000","last_peered":"2024-04-26T21:15:58.388204+0000","last_clean":"2024-04-26T21:15:58.388204+0000","last_became_active":"2024-04-26T21:15:57.383269+0000","last_became_peered":"2024-04-26T21:15:57.383269+0000","last_unstale":"2024-04-26T21:15:58.388204+0000","last_undegraded":"2024-04-26T21:15:58.388204+0000","last_fullsized":"2024-04-26T21:15:58.388204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:37.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377954+0000","last_change":"2024-04-26T21:15:57.374353+0000","last_active":"2024-04-26T21:15:57.377954+0000","last_peered":"2024-04-26T21:15:57.377954+0000","last_clean":"2024-04-26T21:15:57.377954+0000","last_became_active":"2024-04-26T21:15:57.371452+0000","last_became_peered":"2024-04-26T21:15:57.371452+0000","last_unstale":"2024-04-26T21:15:57.377954+0000","last_undegraded":"2024-04-26T21:15:57.377954+0000","last_fullsized":"2024-04-26T21:15:57.377954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:05.548167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388388+0000","last_change":"2024-04-26T21:15:57.381158+0000","last_active":"2024-04-26T21:15:57.388388+0000","last_peered":"2024-04-26T21:15:57.388388+0000","last_clean":"2024-04-26T21:15:57.388388+0000","last_became_active":"2024-04-26T21:15:57.380866+0000","last_became_peered":"2024-04-26T21:15:57.380866+0000","last_unstale":"2024-04-26T21:15:57.388388+0000","last_undegraded":"2024-04-26T21:15:57.388388+0000","last_fullsized":"2024-04-26T21:15:57.388388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.488433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378972+0000","last_change":"2024-04-26T21:15:57.376008+0000","last_active":"2024-04-26T21:15:57.378972+0000","last_peered":"2024-04-26T21:15:57.378972+0000","last_clean":"2024-04-26T21:15:57.378972+0000","last_became_active":"2024-04-26T21:15:57.375716+0000","last_became_peered":"2024-04-26T21:15:57.375716+0000","last_unstale":"2024-04-26T21:15:57.378972+0000","last_undegraded":"2024-04-26T21:15:57.378972+0000","last_fullsized":"2024-04-26T21:15:57.378972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384665+0000","last_change":"2024-04-26T21:15:56.360331+0000","last_active":"2024-04-26T21:15:57.384665+0000","last_peered":"2024-04-26T21:15:57.384665+0000","last_clean":"2024-04-26T21:15:57.384665+0000","last_became_active":"2024-04-26T21:15:56.360043+0000","last_became_peered":"2024-04-26T21:15:56.360043+0000","last_unstale":"2024-04-26T21:15:57.384665+0000","last_undegraded":"2024-04-26T21:15:57.384665+0000","last_fullsized":"2024-04-26T21:15:57.384665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:44.618403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383561+0000","last_change":"2024-04-26T21:15:57.378594+0000","last_active":"2024-04-26T21:15:57.383561+0000","last_peered":"2024-04-26T21:15:57.383561+0000","last_clean":"2024-04-26T21:15:57.383561+0000","last_became_active":"2024-04-26T21:15:57.378345+0000","last_became_peered":"2024-04-26T21:15:57.378345+0000","last_unstale":"2024-04-26T21:15:57.383561+0000","last_undegraded":"2024-04-26T21:15:57.383561+0000","last_fullsized":"2024-04-26T21:15:57.383561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.857200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378758+0000","last_change":"2024-04-26T21:15:56.353313+0000","last_active":"2024-04-26T21:15:57.378758+0000","last_peered":"2024-04-26T21:15:57.378758+0000","last_clean":"2024-04-26T21:15:57.378758+0000","last_became_active":"2024-04-26T21:15:56.353053+0000","last_became_peered":"2024-04-26T21:15:56.353053+0000","last_unstale":"2024-04-26T21:15:57.378758+0000","last_undegraded":"2024-04-26T21:15:57.378758+0000","last_fullsized":"2024-04-26T21:15:57.378758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:44.618403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383297+0000","last_change":"2024-04-26T21:15:57.377394+0000","last_active":"2024-04-26T21:15:57.383297+0000","last_peered":"2024-04-26T21:15:57.383297+0000","last_clean":"2024-04-26T21:15:57.383297+0000","last_became_active":"2024-04-26T21:15:57.377133+0000","last_became_peered":"2024-04-26T21:15:57.377133+0000","last_unstale":"2024-04-26T21:15:57.383297+0000","last_undegraded":"2024-04-26T21:15:57.383297+0000","last_fullsized":"2024-04-26T21:15:57.383297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385722+0000","last_change":"2024-04-26T21:15:56.357994+0000","last_active":"2024-04-26T21:15:57.385722+0000","last_peered":"2024-04-26T21:15:57.385722+0000","last_clean":"2024-04-26T21:15:57.385722+0000","last_became_active":"2024-04-26T21:15:56.357702+0000","last_became_peered":"2024-04-26T21:15:56.357702+0000","last_unstale":"2024-04-26T21:15:57.385722+0000","last_undegraded":"2024-04-26T21:15:57.385722+0000","last_fullsized":"2024-04-26T21:15:57.385722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:48.481603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.376952+0000","last_change":"2024-04-26T21:15:57.372261+0000","last_active":"2024-04-26T21:15:57.376952+0000","last_peered":"2024-04-26T21:15:57.376952+0000","last_clean":"2024-04-26T21:15:57.376952+0000","last_became_active":"2024-04-26T21:15:57.371828+0000","last_became_peered":"2024-04-26T21:15:57.371828+0000","last_unstale":"2024-04-26T21:15:57.376952+0000","last_undegraded":"2024-04-26T21:15:57.376952+0000","last_fullsized":"2024-04-26T21:15:57.376952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:22.551744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385534+0000","last_change":"2024-04-26T21:15:56.358806+0000","last_active":"2024-04-26T21:15:57.385534+0000","last_peered":"2024-04-26T21:15:57.385534+0000","last_clean":"2024-04-26T21:15:57.385534+0000","last_became_active":"2024-04-26T21:15:56.358392+0000","last_became_peered":"2024-04-26T21:15:56.358392+0000","last_unstale":"2024-04-26T21:15:57.385534+0000","last_undegraded":"2024-04-26T21:15:57.385534+0000","last_fullsized":"2024-04-26T21:15:57.385534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:11.757910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388349+0000","last_change":"2024-04-26T21:15:57.383558+0000","last_active":"2024-04-26T21:15:58.388349+0000","last_peered":"2024-04-26T21:15:58.388349+0000","last_clean":"2024-04-26T21:15:58.388349+0000","last_became_active":"2024-04-26T21:15:57.382886+0000","last_became_peered":"2024-04-26T21:15:57.382886+0000","last_unstale":"2024-04-26T21:15:58.388349+0000","last_undegraded":"2024-04-26T21:15:58.388349+0000","last_fullsized":"2024-04-26T21:15:58.388349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:01.368912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384957+0000","last_change":"2024-04-26T21:15:56.354973+0000","last_active":"2024-04-26T21:15:57.384957+0000","last_peered":"2024-04-26T21:15:57.384957+0000","last_clean":"2024-04-26T21:15:57.384957+0000","last_became_active":"2024-04-26T21:15:56.354692+0000","last_became_peered":"2024-04-26T21:15:56.354692+0000","last_unstale":"2024-04-26T21:15:57.384957+0000","last_undegraded":"2024-04-26T21:15:57.384957+0000","last_fullsized":"2024-04-26T21:15:57.384957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:42.437324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378361+0000","last_change":"2024-04-26T21:15:57.369379+0000","last_active":"2024-04-26T21:15:57.378361+0000","last_peered":"2024-04-26T21:15:57.378361+0000","last_clean":"2024-04-26T21:15:57.378361+0000","last_became_active":"2024-04-26T21:15:57.369092+0000","last_became_peered":"2024-04-26T21:15:57.369092+0000","last_unstale":"2024-04-26T21:15:57.378361+0000","last_undegraded":"2024-04-26T21:15:57.378361+0000","last_fullsized":"2024-04-26T21:15:57.378361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:29.762730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382148+0000","last_change":"2024-04-26T21:15:56.353583+0000","last_active":"2024-04-26T21:15:57.382148+0000","last_peered":"2024-04-26T21:15:57.382148+0000","last_clean":"2024-04-26T21:15:57.382148+0000","last_became_active":"2024-04-26T21:15:56.353407+0000","last_became_peered":"2024-04-26T21:15:56.353407+0000","last_unstale":"2024-04-26T21:15:57.382148+0000","last_undegraded":"2024-04-26T21:15:57.382148+0000","last_fullsized":"2024-04-26T21:15:57.382148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:08.816857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377116+0000","last_change":"2024-04-26T21:15:57.372627+0000","last_active":"2024-04-26T21:15:57.377116+0000","last_peered":"2024-04-26T21:15:57.377116+0000","last_clean":"2024-04-26T21:15:57.377116+0000","last_became_active":"2024-04-26T21:15:57.372346+0000","last_became_peered":"2024-04-26T21:15:57.372346+0000","last_unstale":"2024-04-26T21:15:57.377116+0000","last_undegraded":"2024-04-26T21:15:57.377116+0000","last_fullsized":"2024-04-26T21:15:57.377116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:33.795425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.360374+0000","last_change":"2024-04-26T21:15:56.352391+0000","last_active":"2024-04-26T21:15:57.360374+0000","last_peered":"2024-04-26T21:15:57.360374+0000","last_clean":"2024-04-26T21:15:57.360374+0000","last_became_active":"2024-04-26T21:15:56.352127+0000","last_became_peered":"2024-04-26T21:15:56.352127+0000","last_unstale":"2024-04-26T21:15:57.360374+0000","last_undegraded":"2024-04-26T21:15:57.360374+0000","last_fullsized":"2024-04-26T21:15:57.360374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:56.523160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.371083+0000","last_change":"2024-04-26T21:15:57.368471+0000","last_active":"2024-04-26T21:15:57.371083+0000","last_peered":"2024-04-26T21:15:57.371083+0000","last_clean":"2024-04-26T21:15:57.371083+0000","last_became_active":"2024-04-26T21:15:57.368175+0000","last_became_peered":"2024-04-26T21:15:57.368175+0000","last_unstale":"2024-04-26T21:15:57.371083+0000","last_undegraded":"2024-04-26T21:15:57.371083+0000","last_fullsized":"2024-04-26T21:15:57.371083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:55.301494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387768+0000","last_change":"2024-04-26T21:15:56.353692+0000","last_active":"2024-04-26T21:15:58.387768+0000","last_peered":"2024-04-26T21:15:58.387768+0000","last_clean":"2024-04-26T21:15:58.387768+0000","last_became_active":"2024-04-26T21:15:56.353046+0000","last_became_peered":"2024-04-26T21:15:56.353046+0000","last_unstale":"2024-04-26T21:15:58.387768+0000","last_undegraded":"2024-04-26T21:15:58.387768+0000","last_fullsized":"2024-04-26T21:15:58.387768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:03.824987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382784+0000","last_change":"2024-04-26T21:15:57.374226+0000","last_active":"2024-04-26T21:15:57.382784+0000","last_peered":"2024-04-26T21:15:57.382784+0000","last_clean":"2024-04-26T21:15:57.382784+0000","last_became_active":"2024-04-26T21:15:57.373964+0000","last_became_peered":"2024-04-26T21:15:57.373964+0000","last_unstale":"2024-04-26T21:15:57.382784+0000","last_undegraded":"2024-04-26T21:15:57.382784+0000","last_fullsized":"2024-04-26T21:15:57.382784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:25.987879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366661+0000","last_change":"2024-04-26T21:15:56.357077+0000","last_active":"2024-04-26T21:15:57.366661+0000","last_peered":"2024-04-26T21:15:57.366661+0000","last_clean":"2024-04-26T21:15:57.366661+0000","last_became_active":"2024-04-26T21:15:56.356770+0000","last_became_peered":"2024-04-26T21:15:56.356770+0000","last_unstale":"2024-04-26T21:15:57.366661+0000","last_undegraded":"2024-04-26T21:15:57.366661+0000","last_fullsized":"2024-04-26T21:15:57.366661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:27.655640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380503+0000","last_change":"2024-04-26T21:15:57.372414+0000","last_active":"2024-04-26T21:15:57.380503+0000","last_peered":"2024-04-26T21:15:57.380503+0000","last_clean":"2024-04-26T21:15:57.380503+0000","last_became_active":"2024-04-26T21:15:57.372144+0000","last_became_peered":"2024-04-26T21:15:57.372144+0000","last_unstale":"2024-04-26T21:15:57.380503+0000","last_undegraded":"2024-04-26T21:15:57.380503+0000","last_fullsized":"2024-04-26T21:15:57.380503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:00.973689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.367123+0000","last_change":"2024-04-26T21:15:56.353538+0000","last_active":"2024-04-26T21:15:57.367123+0000","last_peered":"2024-04-26T21:15:57.367123+0000","last_clean":"2024-04-26T21:15:57.367123+0000","last_became_active":"2024-04-26T21:15:56.353177+0000","last_became_peered":"2024-04-26T21:15:56.353177+0000","last_unstale":"2024-04-26T21:15:57.367123+0000","last_undegraded":"2024-04-26T21:15:57.367123+0000","last_fullsized":"2024-04-26T21:15:57.367123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:29.650899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378221+0000","last_change":"2024-04-26T21:15:57.375536+0000","last_active":"2024-04-26T21:15:57.378221+0000","last_peered":"2024-04-26T21:15:57.378221+0000","last_clean":"2024-04-26T21:15:57.378221+0000","last_became_active":"2024-04-26T21:15:57.374047+0000","last_became_peered":"2024-04-26T21:15:57.374047+0000","last_unstale":"2024-04-26T21:15:57.378221+0000","last_undegraded":"2024-04-26T21:15:57.378221+0000","last_fullsized":"2024-04-26T21:15:57.378221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:13.496498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384105+0000","last_change":"2024-04-26T21:15:56.353205+0000","last_active":"2024-04-26T21:15:57.384105+0000","last_peered":"2024-04-26T21:15:57.384105+0000","last_clean":"2024-04-26T21:15:57.384105+0000","last_became_active":"2024-04-26T21:15:56.352929+0000","last_became_peered":"2024-04-26T21:15:56.352929+0000","last_unstale":"2024-04-26T21:15:57.384105+0000","last_undegraded":"2024-04-26T21:15:57.384105+0000","last_fullsized":"2024-04-26T21:15:57.384105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:42.437324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378030+0000","last_change":"2024-04-26T21:15:57.371482+0000","last_active":"2024-04-26T21:15:57.378030+0000","last_peered":"2024-04-26T21:15:57.378030+0000","last_clean":"2024-04-26T21:15:57.378030+0000","last_became_active":"2024-04-26T21:15:57.371212+0000","last_became_peered":"2024-04-26T21:15:57.371212+0000","last_unstale":"2024-04-26T21:15:57.378030+0000","last_undegraded":"2024-04-26T21:15:57.378030+0000","last_fullsized":"2024-04-26T21:15:57.378030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:32.190009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388955+0000","last_change":"2024-04-26T21:15:56.343814+0000","last_active":"2024-04-26T21:15:57.388955+0000","last_peered":"2024-04-26T21:15:57.388955+0000","last_clean":"2024-04-26T21:15:57.388955+0000","last_became_active":"2024-04-26T21:15:56.343509+0000","last_became_peered":"2024-04-26T21:15:56.343509+0000","last_unstale":"2024-04-26T21:15:57.388955+0000","last_undegraded":"2024-04-26T21:15:57.388955+0000","last_fullsized":"2024-04-26T21:15:57.388955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:53.648155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384154+0000","last_change":"2024-04-26T21:15:57.377950+0000","last_active":"2024-04-26T21:15:57.384154+0000","last_peered":"2024-04-26T21:15:57.384154+0000","last_clean":"2024-04-26T21:15:57.384154+0000","last_became_active":"2024-04-26T21:15:57.377622+0000","last_became_peered":"2024-04-26T21:15:57.377622+0000","last_unstale":"2024-04-26T21:15:57.384154+0000","last_undegraded":"2024-04-26T21:15:57.384154+0000","last_fullsized":"2024-04-26T21:15:57.384154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:47.897899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387893+0000","last_change":"2024-04-26T21:15:56.353009+0000","last_active":"2024-04-26T21:15:58.387893+0000","last_peered":"2024-04-26T21:15:58.387893+0000","last_clean":"2024-04-26T21:15:58.387893+0000","last_became_active":"2024-04-26T21:15:56.352553+0000","last_became_peered":"2024-04-26T21:15:56.352553+0000","last_unstale":"2024-04-26T21:15:58.387893+0000","last_undegraded":"2024-04-26T21:15:58.387893+0000","last_fullsized":"2024-04-26T21:15:58.387893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:26.213797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.370622+0000","last_change":"2024-04-26T21:15:57.368703+0000","last_active":"2024-04-26T21:15:57.370622+0000","last_peered":"2024-04-26T21:15:57.370622+0000","last_clean":"2024-04-26T21:15:57.370622+0000","last_became_active":"2024-04-26T21:15:57.368413+0000","last_became_peered":"2024-04-26T21:15:57.368413+0000","last_unstale":"2024-04-26T21:15:57.370622+0000","last_undegraded":"2024-04-26T21:15:57.370622+0000","last_fullsized":"2024-04-26T21:15:57.370622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:12.764740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364189+0000","last_change":"2024-04-26T21:15:56.351391+0000","last_active":"2024-04-26T21:15:57.364189+0000","last_peered":"2024-04-26T21:15:57.364189+0000","last_clean":"2024-04-26T21:15:57.364189+0000","last_became_active":"2024-04-26T21:15:56.351121+0000","last_became_peered":"2024-04-26T21:15:56.351121+0000","last_unstale":"2024-04-26T21:15:57.364189+0000","last_undegraded":"2024-04-26T21:15:57.364189+0000","last_fullsized":"2024-04-26T21:15:57.364189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:08.295654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387916+0000","last_change":"2024-04-26T21:15:57.368742+0000","last_active":"2024-04-26T21:15:58.387916+0000","last_peered":"2024-04-26T21:15:58.387916+0000","last_clean":"2024-04-26T21:15:58.387916+0000","last_became_active":"2024-04-26T21:15:57.368448+0000","last_became_peered":"2024-04-26T21:15:57.368448+0000","last_unstale":"2024-04-26T21:15:58.387916+0000","last_undegraded":"2024-04-26T21:15:58.387916+0000","last_fullsized":"2024-04-26T21:15:58.387916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:09.581989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364423+0000","last_change":"2024-04-26T21:15:56.352175+0000","last_active":"2024-04-26T21:15:57.364423+0000","last_peered":"2024-04-26T21:15:57.364423+0000","last_clean":"2024-04-26T21:15:57.364423+0000","last_became_active":"2024-04-26T21:15:56.351870+0000","last_became_peered":"2024-04-26T21:15:56.351870+0000","last_unstale":"2024-04-26T21:15:57.364423+0000","last_undegraded":"2024-04-26T21:15:57.364423+0000","last_fullsized":"2024-04-26T21:15:57.364423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:04.541337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387976+0000","last_change":"2024-04-26T21:15:57.373045+0000","last_active":"2024-04-26T21:15:58.387976+0000","last_peered":"2024-04-26T21:15:58.387976+0000","last_clean":"2024-04-26T21:15:58.387976+0000","last_became_active":"2024-04-26T21:15:57.372520+0000","last_became_peered":"2024-04-26T21:15:57.372520+0000","last_unstale":"2024-04-26T21:15:58.387976+0000","last_undegraded":"2024-04-26T21:15:58.387976+0000","last_fullsized":"2024-04-26T21:15:58.387976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:56.685228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386521+0000","last_change":"2024-04-26T21:15:56.356138+0000","last_active":"2024-04-26T21:15:57.386521+0000","last_peered":"2024-04-26T21:15:57.386521+0000","last_clean":"2024-04-26T21:15:57.386521+0000","last_became_active":"2024-04-26T21:15:56.355852+0000","last_became_peered":"2024-04-26T21:15:56.355852+0000","last_unstale":"2024-04-26T21:15:57.386521+0000","last_undegraded":"2024-04-26T21:15:57.386521+0000","last_fullsized":"2024-04-26T21:15:57.386521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:08.850370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385602+0000","last_change":"2024-04-26T21:15:57.380191+0000","last_active":"2024-04-26T21:15:57.385602+0000","last_peered":"2024-04-26T21:15:57.385602+0000","last_clean":"2024-04-26T21:15:57.385602+0000","last_became_active":"2024-04-26T21:15:57.379612+0000","last_became_peered":"2024-04-26T21:15:57.379612+0000","last_unstale":"2024-04-26T21:15:57.385602+0000","last_undegraded":"2024-04-26T21:15:57.385602+0000","last_fullsized":"2024-04-26T21:15:57.385602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:33.513570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383498+0000","last_change":"2024-04-26T21:15:56.354493+0000","last_active":"2024-04-26T21:15:57.383498+0000","last_peered":"2024-04-26T21:15:57.383498+0000","last_clean":"2024-04-26T21:15:57.383498+0000","last_became_active":"2024-04-26T21:15:56.354169+0000","last_became_peered":"2024-04-26T21:15:56.354169+0000","last_unstale":"2024-04-26T21:15:57.383498+0000","last_undegraded":"2024-04-26T21:15:57.383498+0000","last_fullsized":"2024-04-26T21:15:57.383498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:28.755901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377241+0000","last_change":"2024-04-26T21:15:57.369918+0000","last_active":"2024-04-26T21:15:57.377241+0000","last_peered":"2024-04-26T21:15:57.377241+0000","last_clean":"2024-04-26T21:15:57.377241+0000","last_became_active":"2024-04-26T21:15:57.369480+0000","last_became_peered":"2024-04-26T21:15:57.369480+0000","last_unstale":"2024-04-26T21:15:57.377241+0000","last_undegraded":"2024-04-26T21:15:57.377241+0000","last_fullsized":"2024-04-26T21:15:57.377241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:38.711912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386303+0000","last_change":"2024-04-26T21:15:56.355055+0000","last_active":"2024-04-26T21:15:57.386303+0000","last_peered":"2024-04-26T21:15:57.386303+0000","last_clean":"2024-04-26T21:15:57.386303+0000","last_became_active":"2024-04-26T21:15:56.354745+0000","last_became_peered":"2024-04-26T21:15:56.354745+0000","last_unstale":"2024-04-26T21:15:57.386303+0000","last_undegraded":"2024-04-26T21:15:57.386303+0000","last_fullsized":"2024-04-26T21:15:57.386303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.377993+0000","last_change":"2024-04-26T21:15:57.370004+0000","last_active":"2024-04-26T21:15:57.377993+0000","last_peered":"2024-04-26T21:15:57.377993+0000","last_clean":"2024-04-26T21:15:57.377993+0000","last_became_active":"2024-04-26T21:15:57.369679+0000","last_became_peered":"2024-04-26T21:15:57.369679+0000","last_unstale":"2024-04-26T21:15:57.377993+0000","last_undegraded":"2024-04-26T21:15:57.377993+0000","last_fullsized":"2024-04-26T21:15:57.377993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:30.657729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.361100+0000","last_change":"2024-04-26T21:15:56.351578+0000","last_active":"2024-04-26T21:15:57.361100+0000","last_peered":"2024-04-26T21:15:57.361100+0000","last_clean":"2024-04-26T21:15:57.361100+0000","last_became_active":"2024-04-26T21:15:56.351257+0000","last_became_peered":"2024-04-26T21:15:56.351257+0000","last_unstale":"2024-04-26T21:15:57.361100+0000","last_undegraded":"2024-04-26T21:15:57.361100+0000","last_fullsized":"2024-04-26T21:15:57.361100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:46.258504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388860+0000","last_change":"2024-04-26T21:15:57.384296+0000","last_active":"2024-04-26T21:15:57.388860+0000","last_peered":"2024-04-26T21:15:57.388860+0000","last_clean":"2024-04-26T21:15:57.388860+0000","last_became_active":"2024-04-26T21:15:57.383991+0000","last_became_peered":"2024-04-26T21:15:57.383991+0000","last_unstale":"2024-04-26T21:15:57.388860+0000","last_undegraded":"2024-04-26T21:15:57.388860+0000","last_fullsized":"2024-04-26T21:15:57.388860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:06.382737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382386+0000","last_change":"2024-04-26T21:15:56.344164+0000","last_active":"2024-04-26T21:15:57.382386+0000","last_peered":"2024-04-26T21:15:57.382386+0000","last_clean":"2024-04-26T21:15:57.382386+0000","last_became_active":"2024-04-26T21:15:56.343513+0000","last_became_peered":"2024-04-26T21:15:56.343513+0000","last_unstale":"2024-04-26T21:15:57.382386+0000","last_undegraded":"2024-04-26T21:15:57.382386+0000","last_fullsized":"2024-04-26T21:15:57.382386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:37.870191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385924+0000","last_change":"2024-04-26T21:15:57.380390+0000","last_active":"2024-04-26T21:15:57.385924+0000","last_peered":"2024-04-26T21:15:57.385924+0000","last_clean":"2024-04-26T21:15:57.385924+0000","last_became_active":"2024-04-26T21:15:57.378377+0000","last_became_peered":"2024-04-26T21:15:57.378377+0000","last_unstale":"2024-04-26T21:15:57.385924+0000","last_undegraded":"2024-04-26T21:15:57.385924+0000","last_fullsized":"2024-04-26T21:15:57.385924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:09.857200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.365186+0000","last_change":"2024-04-26T21:15:56.350179+0000","last_active":"2024-04-26T21:15:57.365186+0000","last_peered":"2024-04-26T21:15:57.365186+0000","last_clean":"2024-04-26T21:15:57.365186+0000","last_became_active":"2024-04-26T21:15:56.349306+0000","last_became_peered":"2024-04-26T21:15:56.349306+0000","last_unstale":"2024-04-26T21:15:57.365186+0000","last_undegraded":"2024-04-26T21:15:57.365186+0000","last_fullsized":"2024-04-26T21:15:57.365186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:51.420883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.382980+0000","last_change":"2024-04-26T21:15:57.380472+0000","last_active":"2024-04-26T21:15:57.382980+0000","last_peered":"2024-04-26T21:15:57.382980+0000","last_clean":"2024-04-26T21:15:57.382980+0000","last_became_active":"2024-04-26T21:15:57.380058+0000","last_became_peered":"2024-04-26T21:15:57.380058+0000","last_unstale":"2024-04-26T21:15:57.382980+0000","last_undegraded":"2024-04-26T21:15:57.382980+0000","last_fullsized":"2024-04-26T21:15:57.382980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:38.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.364004+0000","last_change":"2024-04-26T21:15:56.354336+0000","last_active":"2024-04-26T21:15:57.364004+0000","last_peered":"2024-04-26T21:15:57.364004+0000","last_clean":"2024-04-26T21:15:57.364004+0000","last_became_active":"2024-04-26T21:15:56.353959+0000","last_became_peered":"2024-04-26T21:15:56.353959+0000","last_unstale":"2024-04-26T21:15:57.364004+0000","last_undegraded":"2024-04-26T21:15:57.364004+0000","last_fullsized":"2024-04-26T21:15:57.364004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:00.971813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.386034+0000","last_change":"2024-04-26T21:15:57.381920+0000","last_active":"2024-04-26T21:15:57.386034+0000","last_peered":"2024-04-26T21:15:57.386034+0000","last_clean":"2024-04-26T21:15:57.386034+0000","last_became_active":"2024-04-26T21:15:57.381547+0000","last_became_peered":"2024-04-26T21:15:57.381547+0000","last_unstale":"2024-04-26T21:15:57.386034+0000","last_undegraded":"2024-04-26T21:15:57.386034+0000","last_fullsized":"2024-04-26T21:15:57.386034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:55.348773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.362261+0000","last_change":"2024-04-26T21:15:56.347210+0000","last_active":"2024-04-26T21:15:57.362261+0000","last_peered":"2024-04-26T21:15:57.362261+0000","last_clean":"2024-04-26T21:15:57.362261+0000","last_became_active":"2024-04-26T21:15:56.346921+0000","last_became_peered":"2024-04-26T21:15:56.346921+0000","last_unstale":"2024-04-26T21:15:57.362261+0000","last_undegraded":"2024-04-26T21:15:57.362261+0000","last_fullsized":"2024-04-26T21:15:57.362261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:35.344968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.385607+0000","last_change":"2024-04-26T21:15:57.381473+0000","last_active":"2024-04-26T21:15:57.385607+0000","last_peered":"2024-04-26T21:15:57.385607+0000","last_clean":"2024-04-26T21:15:57.385607+0000","last_became_active":"2024-04-26T21:15:57.381097+0000","last_became_peered":"2024-04-26T21:15:57.381097+0000","last_unstale":"2024-04-26T21:15:57.385607+0000","last_undegraded":"2024-04-26T21:15:57.385607+0000","last_fullsized":"2024-04-26T21:15:57.385607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.387540+0000","last_change":"2024-04-26T21:15:56.356463+0000","last_active":"2024-04-26T21:15:58.387540+0000","last_peered":"2024-04-26T21:15:58.387540+0000","last_clean":"2024-04-26T21:15:58.387540+0000","last_became_active":"2024-04-26T21:15:56.356102+0000","last_became_peered":"2024-04-26T21:15:56.356102+0000","last_unstale":"2024-04-26T21:15:58.387540+0000","last_undegraded":"2024-04-26T21:15:58.387540+0000","last_fullsized":"2024-04-26T21:15:58.387540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.577161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383284+0000","last_change":"2024-04-26T21:15:57.379372+0000","last_active":"2024-04-26T21:15:57.383284+0000","last_peered":"2024-04-26T21:15:57.383284+0000","last_clean":"2024-04-26T21:15:57.383284+0000","last_became_active":"2024-04-26T21:15:57.379111+0000","last_became_peered":"2024-04-26T21:15:57.379111+0000","last_unstale":"2024-04-26T21:15:57.383284+0000","last_undegraded":"2024-04-26T21:15:57.383284+0000","last_fullsized":"2024-04-26T21:15:57.383284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:47.897899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:59.362560+0000","last_change":"2024-04-26T21:15:56.357277+0000","last_active":"2024-04-26T21:15:59.362560+0000","last_peered":"2024-04-26T21:15:59.362560+0000","last_clean":"2024-04-26T21:15:59.362560+0000","last_became_active":"2024-04-26T21:15:56.356980+0000","last_became_peered":"2024-04-26T21:15:56.356980+0000","last_unstale":"2024-04-26T21:15:59.362560+0000","last_undegraded":"2024-04-26T21:15:59.362560+0000","last_fullsized":"2024-04-26T21:15:59.362560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:14.174870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383317+0000","last_change":"2024-04-26T21:15:57.377814+0000","last_active":"2024-04-26T21:15:57.383317+0000","last_peered":"2024-04-26T21:15:57.383317+0000","last_clean":"2024-04-26T21:15:57.383317+0000","last_became_active":"2024-04-26T21:15:57.377383+0000","last_became_peered":"2024-04-26T21:15:57.377383+0000","last_unstale":"2024-04-26T21:15:57.383317+0000","last_undegraded":"2024-04-26T21:15:57.383317+0000","last_fullsized":"2024-04-26T21:15:57.383317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:57.529990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.365588+0000","last_change":"2024-04-26T21:15:56.360214+0000","last_active":"2024-04-26T21:15:57.365588+0000","last_peered":"2024-04-26T21:15:57.365588+0000","last_clean":"2024-04-26T21:15:57.365588+0000","last_became_active":"2024-04-26T21:15:56.359940+0000","last_became_peered":"2024-04-26T21:15:56.359940+0000","last_unstale":"2024-04-26T21:15:57.365588+0000","last_undegraded":"2024-04-26T21:15:57.365588+0000","last_fullsized":"2024-04-26T21:15:57.365588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:48.572696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.378049+0000","last_change":"2024-04-26T21:15:57.374814+0000","last_active":"2024-04-26T21:15:57.378049+0000","last_peered":"2024-04-26T21:15:57.378049+0000","last_clean":"2024-04-26T21:15:57.378049+0000","last_became_active":"2024-04-26T21:15:57.374510+0000","last_became_peered":"2024-04-26T21:15:57.374510+0000","last_unstale":"2024-04-26T21:15:57.378049+0000","last_undegraded":"2024-04-26T21:15:57.378049+0000","last_fullsized":"2024-04-26T21:15:57.378049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:54.654984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366227+0000","last_change":"2024-04-26T21:15:56.357679+0000","last_active":"2024-04-26T21:15:57.366227+0000","last_peered":"2024-04-26T21:15:57.366227+0000","last_clean":"2024-04-26T21:15:57.366227+0000","last_became_active":"2024-04-26T21:15:56.357334+0000","last_became_peered":"2024-04-26T21:15:56.357334+0000","last_unstale":"2024-04-26T21:15:57.366227+0000","last_undegraded":"2024-04-26T21:15:57.366227+0000","last_fullsized":"2024-04-26T21:15:57.366227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:08.816857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.383039+0000","last_change":"2024-04-26T21:15:57.379322+0000","last_active":"2024-04-26T21:15:57.383039+0000","last_peered":"2024-04-26T21:15:57.383039+0000","last_clean":"2024-04-26T21:15:57.383039+0000","last_became_active":"2024-04-26T21:15:57.378975+0000","last_became_peered":"2024-04-26T21:15:57.378975+0000","last_unstale":"2024-04-26T21:15:57.383039+0000","last_undegraded":"2024-04-26T21:15:57.383039+0000","last_fullsized":"2024-04-26T21:15:57.383039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:06.510169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.361289+0000","last_change":"2024-04-26T21:15:56.355797+0000","last_active":"2024-04-26T21:15:57.361289+0000","last_peered":"2024-04-26T21:15:57.361289+0000","last_clean":"2024-04-26T21:15:57.361289+0000","last_became_active":"2024-04-26T21:15:56.355460+0000","last_became_peered":"2024-04-26T21:15:56.355460+0000","last_unstale":"2024-04-26T21:15:57.361289+0000","last_undegraded":"2024-04-26T21:15:57.361289+0000","last_fullsized":"2024-04-26T21:15:57.361289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:21.544914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384199+0000","last_change":"2024-04-26T21:15:57.378563+0000","last_active":"2024-04-26T21:15:57.384199+0000","last_peered":"2024-04-26T21:15:57.384199+0000","last_clean":"2024-04-26T21:15:57.384199+0000","last_became_active":"2024-04-26T21:15:57.377998+0000","last_became_peered":"2024-04-26T21:15:57.377998+0000","last_unstale":"2024-04-26T21:15:57.384199+0000","last_undegraded":"2024-04-26T21:15:57.384199+0000","last_fullsized":"2024-04-26T21:15:57.384199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:33.513570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.359440+0000","last_change":"2024-04-26T21:15:56.353413+0000","last_active":"2024-04-26T21:15:57.359440+0000","last_peered":"2024-04-26T21:15:57.359440+0000","last_clean":"2024-04-26T21:15:57.359440+0000","last_became_active":"2024-04-26T21:15:56.353087+0000","last_became_peered":"2024-04-26T21:15:56.353087+0000","last_unstale":"2024-04-26T21:15:57.359440+0000","last_undegraded":"2024-04-26T21:15:57.359440+0000","last_fullsized":"2024-04-26T21:15:57.359440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.710875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.388505+0000","last_change":"2024-04-26T21:15:57.383524+0000","last_active":"2024-04-26T21:15:57.388505+0000","last_peered":"2024-04-26T21:15:57.388505+0000","last_clean":"2024-04-26T21:15:57.388505+0000","last_became_active":"2024-04-26T21:15:57.383249+0000","last_became_peered":"2024-04-26T21:15:57.383249+0000","last_unstale":"2024-04-26T21:15:57.388505+0000","last_undegraded":"2024-04-26T21:15:57.388505+0000","last_fullsized":"2024-04-26T21:15:57.388505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:36.351797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.366089+0000","last_change":"2024-04-26T21:15:56.357085+0000","last_active":"2024-04-26T21:15:57.366089+0000","last_peered":"2024-04-26T21:15:57.366089+0000","last_clean":"2024-04-26T21:15:57.366089+0000","last_became_active":"2024-04-26T21:15:56.356768+0000","last_became_peered":"2024-04-26T21:15:56.356768+0000","last_unstale":"2024-04-26T21:15:57.366089+0000","last_undegraded":"2024-04-26T21:15:57.366089+0000","last_fullsized":"2024-04-26T21:15:57.366089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:04.710875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.380808+0000","last_change":"2024-04-26T21:15:57.372228+0000","last_active":"2024-04-26T21:15:57.380808+0000","last_peered":"2024-04-26T21:15:57.380808+0000","last_clean":"2024-04-26T21:15:57.380808+0000","last_became_active":"2024-04-26T21:15:57.371956+0000","last_became_peered":"2024-04-26T21:15:57.371956+0000","last_unstale":"2024-04-26T21:15:57.380808+0000","last_undegraded":"2024-04-26T21:15:57.380808+0000","last_fullsized":"2024-04-26T21:15:57.380808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:32.190009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.358129+0000","last_change":"2024-04-26T21:15:56.352990+0000","last_active":"2024-04-26T21:15:57.358129+0000","last_peered":"2024-04-26T21:15:57.358129+0000","last_clean":"2024-04-26T21:15:57.358129+0000","last_became_active":"2024-04-26T21:15:56.352667+0000","last_became_peered":"2024-04-26T21:15:56.352667+0000","last_unstale":"2024-04-26T21:15:57.358129+0000","last_undegraded":"2024-04-26T21:15:57.358129+0000","last_fullsized":"2024-04-26T21:15:57.358129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:48.572696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T21:15:58.388252+0000","last_change":"2024-04-26T21:15:57.380461+0000","last_active":"2024-04-26T21:15:58.388252+0000","last_peered":"2024-04-26T21:15:58.388252+0000","last_clean":"2024-04-26T21:15:58.388252+0000","last_became_active":"2024-04-26T21:15:57.379448+0000","last_became_peered":"2024-04-26T21:15:57.379448+0000","last_unstale":"2024-04-26T21:15:58.388252+0000","last_undegraded":"2024-04-26T21:15:58.388252+0000","last_fullsized":"2024-04-26T21:15:58.388252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:56.333295+0000","last_clean_scrub_stamp":"2024-04-26T21:15:56.333295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:57.681128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.384995+0000","last_change":"2024-04-26T21:15:56.351248+0000","last_active":"2024-04-26T21:15:57.384995+0000","last_peered":"2024-04-26T21:15:57.384995+0000","last_clean":"2024-04-26T21:15:57.384995+0000","last_became_active":"2024-04-26T21:15:56.350939+0000","last_became_peered":"2024-04-26T21:15:56.350939+0000","last_unstale":"2024-04-26T21:15:57.384995+0000","last_undegraded":"2024-04-26T21:15:57.384995+0000","last_fullsized":"2024-04-26T21:15:57.384995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:15:55.326465+0000","last_clean_scrub_stamp":"2024-04-26T21:15:55.326465+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:54.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":277,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-26T21:15:57.362727+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:15:57.362727+0000","last_peered":"2024-04-26T21:15:57.362727+0000","last_clean":"2024-04-26T21:15:57.362727+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:15:57.362727+0000","last_undegraded":"2024-04-26T21:15:57.362727+0000","last_fullsized":"2024-04-26T21:15:57.362727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1597440,"data_stored":1011474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":421730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":421036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":422648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":1004629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":427530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:16:01.753 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:16:01.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:16:01.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.895+0000 7f66767e9640 1 Processor -- start 2024-04-26T21:16:01.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.895+0000 7f66767e9640 1 -- start start 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 0x7f66700b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 0x7f66700d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d56a0 0x7f66700b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f66700d5b10 con 0x7f66700d56a0 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f66700b2580 con 0x7f667015f6e0 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f66767e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66700b2860 con 0x7f66700d3d10 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f6674d86640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d56a0 0x7f66700b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f6674d86640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d56a0 0x7f66700b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46930/0 (socket says 172.21.15.40:46930) 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.896+0000 7f6674d86640 1 -- 172.21.15.40:0/1806413049 learned_addr learned my addr 172.21.15.40:0/1806413049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f666ffff640 1 --2- 172.21.15.40:0/1806413049 >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 0x7f66700b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f666f7fe640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 0x7f66700d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 -- 172.21.15.40:0/1806413049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 msgr2=0x7f66700d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 0x7f66700d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 -- 172.21.15.40:0/1806413049 >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 msgr2=0x7f66700b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 --2- 172.21.15.40:0/1806413049 >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 0x7f66700b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 -- 172.21.15.40:0/1806413049 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66700b2b40 con 0x7f66700d56a0 2024-04-26T21:16:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f666ffff640 1 --2- 172.21.15.40:0/1806413049 >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 0x7f66700b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f666f7fe640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 0x7f66700d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.897+0000 7f6674d86640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d56a0 0x7f66700b4b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f66700b6cd0 tx=0x7f666802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454364c9820f8c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f666effd640 1 -- 172.21.15.40:0/1806413049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668036070 con 0x7f66700d56a0 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f666effd640 1 -- 172.21.15.40:0/1806413049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6668031600 con 0x7f66700d56a0 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f666effd640 1 -- 172.21.15.40:0/1806413049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66680319d0 con 0x7f66700d56a0 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 -- 172.21.15.40:0/1806413049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d56a0 msgr2=0x7f66700b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d56a0 0x7f66700b4b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f66700b6cd0 tx=0x7f666802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 -- 172.21.15.40:0/1806413049 shutdown_connections 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d56a0 0x7f66700b4b10 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 --2- 172.21.15.40:0/1806413049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d3d10 0x7f66700d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 --2- 172.21.15.40:0/1806413049 >> v2:172.21.15.184:3300/0 conn(0x7f667015f6e0 0x7f66700b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 -- 172.21.15.40:0/1806413049 >> 172.21.15.40:0/1806413049 conn(0x7f667008ec20 msgr2=0x7f667009e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 -- 172.21.15.40:0/1806413049 shutdown_connections 2024-04-26T21:16:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.898+0000 7f66767e9640 1 -- 172.21.15.40:0/1806413049 wait complete. 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.899+0000 7f66767e9640 1 Processor -- start 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 -- start start 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 0x7f66700cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 0x7f667015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66700d4170 con 0x7f66700d3d10 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66700b5050 con 0x7f667015f6e0 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f66767e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66701612c0 con 0x7f66700d56a0 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f666ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f666ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46940/0 (socket says 172.21.15.40:46940) 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.900+0000 7f666ffff640 1 -- 172.21.15.40:0/3522176799 learned_addr learned my addr 172.21.15.40:0/3522176799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666f7fe640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 0x7f66700cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f6674d86640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 0x7f667015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 -- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 msgr2=0x7f66700cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 0x7f66700cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 -- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 msgr2=0x7f667015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 0x7f667015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 -- 172.21.15.40:0/3522176799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701615a0 con 0x7f66700d3d10 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f6674d86640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 0x7f667015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666f7fe640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 0x7f66700cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666ffff640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f666000c7f0 tx=0x7f666000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660015070 con 0x7f66700d3d10 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6660007820 con 0x7f66700d3d10 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6670161880 con 0x7f66700d3d10 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660007bd0 con 0x7f66700d3d10 2024-04-26T21:16:01.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.901+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6670161e40 con 0x7f66700d3d10 2024-04-26T21:16:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.902+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66700cb3b0 con 0x7f66700d3d10 2024-04-26T21:16:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.903+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6660018070 con 0x7f66700d3d10 2024-04-26T21:16:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.903+0000 7f666cff9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.903+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f666008b980 con 0x7f66700d3d10 2024-04-26T21:16:01.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.903+0000 7f666f7fe640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:01.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.904+0000 7f666f7fe640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 0x7f665c06ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f66700c9990 tx=0x7f6664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:01.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:01.907+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66700cb3b0 con 0x7f66700d3d10 2024-04-26T21:16:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.166+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f667006a290 con 0x7f66700d3d10 2024-04-26T21:16:02.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.171+0000 7f666cff9640 1 -- 172.21.15.40:0/3522176799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f6660059e40 con 0x7f66700d3d10 2024-04-26T21:16:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:16:02.172 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:16:02.172 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:16:02.172 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:16:02.173 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:16:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 msgr2=0x7f665c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 0x7f665c06ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f66700c9990 tx=0x7f6664008040 comp rx=0 tx=0).stop 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 msgr2=0x7f6670160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f666000c7f0 tx=0x7f666000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 shutdown_connections 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f667015f6e0 0x7f667015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66700d56a0 0x7f66700cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 --2- 172.21.15.40:0/3522176799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d3d10 0x7f6670160ce0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.172+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 >> 172.21.15.40:0/3522176799 conn(0x7f667008ec20 msgr2=0x7f667006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.173+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 shutdown_connections 2024-04-26T21:16:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.173+0000 7f66767e9640 1 -- 172.21.15.40:0/3522176799 wait complete. 2024-04-26T21:16:02.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:16:02.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.286+0000 7f8d23338640 1 Processor -- start 2024-04-26T21:16:02.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.286+0000 7f8d23338640 1 -- start start 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d23338640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 0x7f8d1c0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d23338640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d1c146790 0x7f8d1c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d23338640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1c0d05b0 con 0x7f8d1c13ff70 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d23338640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8d1c0d0890 con 0x7f8d1c146790 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d23338640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8d1c06e450 con 0x7f8d1c140350 2024-04-26T21:16:02.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d218d5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d1c146790 0x7f8d1c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d210d4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8d1c13ff70 0x7f8d1c146080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59406/0 (socket says 172.21.15.40:59406) 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d210d4640 1 -- 172.21.15.40:0/2473951092 learned_addr learned my addr 172.21.15.40:0/2473951092 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d218d5640 1 -- 172.21.15.40:0/2473951092 >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 msgr2=0x7f8d1c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d208d3640 1 --2- 172.21.15.40:0/2473951092 >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 0x7f8d1c0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d13fff640 1 -- 172.21.15.40:0/2473951092 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1721083849 0 0) 0x7f8d1c0d05b0 con 0x7f8d1c13ff70 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d218d5640 1 --2- 172.21.15.40:0/2473951092 >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 0x7f8d1c0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d218d5640 1 -- 172.21.15.40:0/2473951092 >> v1:172.21.15.40:6789/0 conn(0x7f8d1c13ff70 legacy=0x7f8d1c146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d218d5640 1 -- 172.21.15.40:0/2473951092 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c06e730 con 0x7f8d1c146790 2024-04-26T21:16:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.287+0000 7f8d208d3640 1 --2- 172.21.15.40:0/2473951092 >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 0x7f8d1c0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d218d5640 1 --2- 172.21.15.40:0/2473951092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c0cfd50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d90 tx=0x7f8d1402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5eb997825be2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d13fff640 1 -- 172.21.15.40:0/2473951092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14031600 con 0x7f8d1c146790 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d13fff640 1 -- 172.21.15.40:0/2473951092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d140317a0 con 0x7f8d1c146790 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d13fff640 1 -- 172.21.15.40:0/2473951092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14031b70 con 0x7f8d1c146790 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d23338640 1 -- 172.21.15.40:0/2473951092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 msgr2=0x7f8d1c0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.288+0000 7f8d23338640 1 --2- 172.21.15.40:0/2473951092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c0cfd50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d90 tx=0x7f8d1402a610 comp rx=0 tx=0).stop 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 -- 172.21.15.40:0/2473951092 shutdown_connections 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 --2- 172.21.15.40:0/2473951092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c0cfd50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 --2- 172.21.15.40:0/2473951092 >> v2:172.21.15.184:3300/0 conn(0x7f8d1c140350 0x7f8d1c0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 -- 172.21.15.40:0/2473951092 >> 172.21.15.40:0/2473951092 conn(0x7f8d1c0d0d30 msgr2=0x7f8d1c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 -- 172.21.15.40:0/2473951092 shutdown_connections 2024-04-26T21:16:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.289+0000 7f8d23338640 1 -- 172.21.15.40:0/2473951092 wait complete. 2024-04-26T21:16:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 Processor -- start 2024-04-26T21:16:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 -- start start 2024-04-26T21:16:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c13ff70 0x7f8d1c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c140350 0x7f8d1c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c06edd0 con 0x7f8d1c146790 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c06ec50 con 0x7f8d1c13ff70 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.290+0000 7f8d23338640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1c06ef50 con 0x7f8d1c140350 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46950/0 (socket says 172.21.15.40:46950) 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 -- 172.21.15.40:0/1976912700 learned_addr learned my addr 172.21.15.40:0/1976912700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d210d4640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c13ff70 0x7f8d1c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d208d3640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c140350 0x7f8d1c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 -- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c140350 msgr2=0x7f8d1c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c140350 0x7f8d1c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 -- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c13ff70 msgr2=0x7f8d1c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c13ff70 0x7f8d1c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 -- 172.21.15.40:0/1976912700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c15fd60 con 0x7f8d1c146790 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d218d5640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d90 tx=0x7f8d14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1403f070 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d14030be0 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c0aefb0 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14009450 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.291+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c0af570 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.292+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c06a290 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.293+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d14003990 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.293+0000 7f8d11ffb640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.293+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f8d140acc50 con 0x7f8d1c146790 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.294+0000 7f8d210d4640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.294+0000 7f8d210d4640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 0x7f8d0006edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f60 tx=0x7f8d18007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.298+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d1407afe0 con 0x7f8d1c146790 2024-04-26T21:16:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.521+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d1c0b45e0 con 0x7f8d1c146790 2024-04-26T21:16:02.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.521+0000 7f8d11ffb640 1 -- 172.21.15.40:0/1976912700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5186 (secure 0 0 0) 0x7f8d1407fe20 con 0x7f8d1c146790 2024-04-26T21:16:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:02.523 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:15:59.744328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:16:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 46 2024-04-26T21:16:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 msgr2=0x7f8d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 0x7f8d0006edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f60 tx=0x7f8d18007b50 comp rx=0 tx=0).stop 2024-04-26T21:16:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 msgr2=0x7f8d1c15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009d90 tx=0x7f8d14004a40 comp rx=0 tx=0).stop 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 shutdown_connections 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d1c146790 0x7f8d1c15eef0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1c140350 0x7f8d1c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 --2- 172.21.15.40:0/1976912700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1c13ff70 0x7f8d1c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 >> 172.21.15.40:0/1976912700 conn(0x7f8d1c0d0d30 msgr2=0x7f8d1c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 shutdown_connections 2024-04-26T21:16:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:02.523+0000 7f8d23338640 1 -- 172.21.15.40:0/1976912700 wait complete. 2024-04-26T21:16:02.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:15:56.722100+0000', 'modified': '2024-04-26T21:15:59.744328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6912}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6912': {'gid': 6912, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6836/1963635253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 1963635253}, {'type': 'v1', 'addr': '172.21.15.184:6836', 'nonce': 1963635253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6912, 'qdb_cluster': [6912]} 2024-04-26T21:16:02.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:16:02.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T21:16:03.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.043+0000 7f7b95650640 1 Processor -- start 2024-04-26T21:16:03.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.043+0000 7f7b95650640 1 -- start start 2024-04-26T21:16:03.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b95650640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b9013ff70 0x7f7b90146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b95650640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 0x7f7b901465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b95650640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b900d05b0 con 0x7f7b90140350 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b95650640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7b900d0890 con 0x7f7b9013ff70 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b95650640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9006e450 con 0x7f7b900cff20 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b9013ff70 0x7f7b90146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b9013ff70 0x7f7b90146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46964/0 (socket says 172.21.15.40:46964) 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 -- 172.21.15.40:0/3443288525 learned_addr learned my addr 172.21.15.40:0/3443288525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 -- 172.21.15.40:0/3443288525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 msgr2=0x7f7b901465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 0x7f7b901465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 0x7f7b901465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 -- 172.21.15.40:0/3443288525 >> v1:172.21.15.184:6789/0 conn(0x7f7b90140350 legacy=0x7f7b900cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8effd640 1 -- 172.21.15.40:0/3443288525 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9006e730 con 0x7f7b9013ff70 2024-04-26T21:16:03.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.044+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 0x7f7b901465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.045+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b9013ff70 0x7f7b90146080 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a6463cb004aa86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.045+0000 7f7b8e7fc640 1 -- 172.21.15.40:0/3443288525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80037070 con 0x7f7b9013ff70 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.045+0000 7f7b8e7fc640 1 -- 172.21.15.40:0/3443288525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80032440 con 0x7f7b9013ff70 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.045+0000 7f7b8e7fc640 1 -- 172.21.15.40:0/3443288525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80032810 con 0x7f7b9013ff70 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 -- 172.21.15.40:0/3443288525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b9013ff70 msgr2=0x7f7b90146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b9013ff70 0x7f7b90146080 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b8002a180 comp rx=0 tx=0).stop 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 -- 172.21.15.40:0/3443288525 shutdown_connections 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b900cff20 0x7f7b901465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 --2- 172.21.15.40:0/3443288525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b9013ff70 0x7f7b90146080 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 -- 172.21.15.40:0/3443288525 >> 172.21.15.40:0/3443288525 conn(0x7f7b900d0d30 msgr2=0x7f7b900d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 -- 172.21.15.40:0/3443288525 shutdown_connections 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.046+0000 7f7b95650640 1 -- 172.21.15.40:0/3443288525 wait complete. 2024-04-26T21:16:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.047+0000 7f7b95650640 1 Processor -- start 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.047+0000 7f7b95650640 1 -- start start 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9013ff70 0x7f7b9015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b90140350 0x7f7b9015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006efe0 con 0x7f7b900cff20 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006ee60 con 0x7f7b9013ff70 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b95650640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9006f160 con 0x7f7b90140350 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46972/0 (socket says 172.21.15.40:46972) 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 -- 172.21.15.40:0/19340004 learned_addr learned my addr 172.21.15.40:0/19340004 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b90140350 0x7f7b9015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b87fff640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9013ff70 0x7f7b9015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 -- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b90140350 msgr2=0x7f7b9015fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b90140350 0x7f7b9015fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 -- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9013ff70 msgr2=0x7f7b9015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9013ff70 0x7f7b9015a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 -- 172.21.15.40:0/19340004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9015edb0 con 0x7f7b900cff20 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.048+0000 7f7b8effd640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.049+0000 7f7b877fe640 1 -- 172.21.15.40:0/19340004 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80041020 con 0x7f7b900cff20 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.049+0000 7f7b877fe640 1 -- 172.21.15.40:0/19340004 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80030970 con 0x7f7b900cff20 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.049+0000 7f7b877fe640 1 -- 172.21.15.40:0/19340004 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80037040 con 0x7f7b900cff20 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.049+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9015f090 con 0x7f7b900cff20 2024-04-26T21:16:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.049+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9015f650 con 0x7f7b900cff20 2024-04-26T21:16:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.050+0000 7f7b877fe640 1 -- 172.21.15.40:0/19340004 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b80044020 con 0x7f7b900cff20 2024-04-26T21:16:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.051+0000 7f7b877fe640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 0x7f7b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.051+0000 7f7b877fe640 1 -- 172.21.15.40:0/19340004 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7b800ac540 con 0x7f7b900cff20 2024-04-26T21:16:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.051+0000 7f7b87fff640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 0x7f7b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.051+0000 7f7b87fff640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 0x7f7b7006edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006fd0 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.052+0000 7f7b98368740 1 Processor -- start 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.052+0000 7f7b98368740 1 -- start start 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.052+0000 7f7b98368740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b27120 0x5651a1b6d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b98368740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5651a1a8b740 con 0x5651a1b51440 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651a1a273a0 con 0x5651a1b27120 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5651a1a87700 con 0x5651a1b6ece0 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46982/0 (socket says 172.21.15.40:46982) 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 -- 172.21.15.40:0/3417885792 learned_addr learned my addr 172.21.15.40:0/3417885792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b87fff640 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 -- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 msgr2=0x5651a1b6f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 -- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b27120 msgr2=0x5651a1b6d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b27120 0x5651a1b6d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b27120 0x5651a1b6d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 -- 172.21.15.40:0/3417885792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a1a43750 con 0x5651a1b51440 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7b80082910 tx=0x7f7b80079da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f8444778b6ba98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b84ff9640 1 -- 172.21.15.40:0/3417885792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8007a260 con 0x5651a1b51440 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b84ff9640 1 -- 172.21.15.40:0/3417885792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8007e610 con 0x5651a1b51440 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.053+0000 7f7b84ff9640 1 -- 172.21.15.40:0/3417885792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80082e40 con 0x5651a1b51440 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- 172.21.15.40:0/3417885792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 msgr2=0x5651a1b51820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7b80082910 tx=0x7f7b80079da0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- 172.21.15.40:0/3417885792 shutdown_connections 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b27120 0x5651a1b6d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 --2- 172.21.15.40:0/3417885792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b51820 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- 172.21.15.40:0/3417885792 >> 172.21.15.40:0/3417885792 conn(0x5651a1b23a50 msgr2=0x5651a1b6c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- 172.21.15.40:0/3417885792 shutdown_connections 2024-04-26T21:16:03.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- 172.21.15.40:0/3417885792 wait complete. 2024-04-26T21:16:03.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.054+0000 7f7b98368740 1 -- start start 2024-04-26T21:16:03.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 Processor -- start 2024-04-26T21:16:03.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b4be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5651a1a8b740 con 0x5651a1b4d6f0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651a1a12fe0 con 0x5651a1b51440 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5651a1b27ec0 con 0x5651a1b6ece0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b98368740 10 client.0 _ll_register_callbacks cb 0x5651a1a29150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46996/0 (socket says 172.21.15.40:46996) 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.055+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/3915228532 learned_addr learned my addr 172.21.15.40:0/3915228532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b87fff640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b4be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 msgr2=0x5651a1b6b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 msgr2=0x5651a1b4be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b4be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/3915228532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a179d2d0 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800fbf0 tx=0x7f7b8800a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a5253d63ccab23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800ae40 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7b88003990 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8007e610 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7b88021050 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b98368740 10 client.7041 fetch_fsmap learned FSMap version 46 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5651a18676b0 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8007e610 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.056+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f7b88024020 con 0x5651a1b4d6f0 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 10 client.7041 fetch_fsmap finished waiting for FSMap version 46 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 10 client.7041 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 20 client.7041 populate_metadata read hostname 'smithi040' 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1b4a450 0x5651a1b4a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 4 client.7041 mds_command: new command op to 6912 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 --> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a1a99350 con 0x5651a1b4a450 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1b4a450 0x5651a1b4a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b6dffb640 10 client.7041 ms_handle_connect on v2:172.21.15.184:6835/1963635253 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.057+0000 7f7b8effd640 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1b4a450 0x5651a1b4a830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fc2150c1efd29b server_cookie=9e55cd67acec0605 in_seq=0 out_seq=0 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b6dffb640 1 -- 172.21.15.40:0/3915228532 <== mds.0 v2:172.21.15.184:6835/1963635253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651a1a99350 con 0x5651a1b4a450 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b6dffb640 10 client.7041 handle_command_reply: tid=0 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 client.7041 shutdown 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 msgr2=0x5651a1b6bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800fbf0 tx=0x7f7b8800a400 comp rx=0 tx=0).stop 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 shutdown_connections 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1b4a450 0x5651a1b4a830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b4d6f0 0x5651a1b6bbf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b6dffb640 0 client.7041 ms_handle_reset on v2:172.21.15.184:6835/1963635253 2024-04-26T21:16:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1b6b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 --2- 172.21.15.40:0/3915228532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b4be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 >> 172.21.15.40:0/3915228532 conn(0x5651a1b27120 msgr2=0x5651a1b70060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 shutdown_connections 2024-04-26T21:16:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 1 -- 172.21.15.40:0/3915228532 wait complete. 2024-04-26T21:16:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.058+0000 7f7b98368740 20 client.7041 trim_cache size 0 max 16384 2024-04-26T21:16:03.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 Processor -- start 2024-04-26T21:16:03.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 -- start start 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b4d6f0 0x5651a1b452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b45830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9015e180 con 0x5651a1b6ece0 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90144460 con 0x5651a1b51440 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9015db30 con 0x5651a1b4d6f0 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47004/0 (socket says 172.21.15.40:47004) 2024-04-26T21:16:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.077+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/2838507581 learned_addr learned my addr 172.21.15.40:0/2838507581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8effd640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b4d6f0 0x5651a1b452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b87fff640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b45830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b4d6f0 msgr2=0x5651a1b452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b4d6f0 0x5651a1b452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 msgr2=0x5651a1b45830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b45830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 -- 172.21.15.40:0/2838507581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a1b20fc0 con 0x5651a1b6ece0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b87fff640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b45830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7b88000f80 tx=0x7f7b88012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27bbe366a1da386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b6ffff640 1 -- 172.21.15.40:0/2838507581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8802b070 con 0x5651a1b6ece0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b6ffff640 1 -- 172.21.15.40:0/2838507581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8800ae40 con 0x5651a1b6ece0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b6ffff640 1 -- 172.21.15.40:0/2838507581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800e680 con 0x5651a1b6ece0 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b98368740 1 -- 172.21.15.40:0/2838507581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 msgr2=0x5651a1b61be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.078+0000 7f7b98368740 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7b88000f80 tx=0x7f7b88012000 comp rx=0 tx=0).stop 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 -- 172.21.15.40:0/2838507581 shutdown_connections 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b6ece0 0x5651a1b61be0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b51440 0x5651a1b45830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 --2- 172.21.15.40:0/2838507581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b4d6f0 0x5651a1b452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 -- 172.21.15.40:0/2838507581 >> 172.21.15.40:0/2838507581 conn(0x5651a1b27120 msgr2=0x5651a1b54110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 -- 172.21.15.40:0/2838507581 shutdown_connections 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 -- 172.21.15.40:0/2838507581 wait complete. 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 -- start start 2024-04-26T21:16:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.079+0000 7f7b98368740 1 Processor -- start 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b4d6f0 0x5651a1b86350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1bd3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5651a1b4ef10 con 0x5651a1b51440 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651a1b4e690 con 0x5651a1b4d6f0 2024-04-26T21:16:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5651a1b474f0 con 0x5651a1b6ece0 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b98368740 10 client.0 _ll_register_callbacks cb 0x7f7b88007750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47020/0 (socket says 172.21.15.40:47020) 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 -- 172.21.15.40:0/2207040698 learned_addr learned my addr 172.21.15.40:0/2207040698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b8effd640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b4d6f0 0x5651a1b86350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1bd3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 -- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 msgr2=0x5651a1bd3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1bd3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 -- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b4d6f0 msgr2=0x5651a1b86350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b4d6f0 0x5651a1b86350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b87fff640 1 -- 172.21.15.40:0/2207040698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a1b86890 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.080+0000 7f7b8f7fe640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1bd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b87fff640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7b900b3c30 tx=0x7f7b78004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44738feaec0fb1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78026070 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7b78004020 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8800ae40 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7b78030050 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 10 client.7047 fetch_fsmap learned FSMap version 46 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5651a1bd6af0 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8800ae40 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f7b78033020 con 0x5651a1b51440 2024-04-26T21:16:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 10 client.7047 fetch_fsmap finished waiting for FSMap version 46 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 10 client.7047 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 20 client.7047 populate_metadata read hostname 'smithi040' 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1bd36c0 0x5651a1b48140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 4 client.7047 mds_command: new command op to 6912 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 --> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] -- command(tid 0: {"prefix": "status"}) -- 0x5651a1a931c0 con 0x5651a1bd36c0 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.081+0000 7f7b8effd640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1bd36c0 0x5651a1b48140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b8effd640 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1bd36c0 0x5651a1b48140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7d73da4e8faee8 server_cookie=3c3b68aacfbc8383 in_seq=0 out_seq=0 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b6dffb640 10 client.7047 ms_handle_connect on v2:172.21.15.184:6835/1963635253 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b6dffb640 1 -- 172.21.15.40:0/2207040698 <== mds.0 v2:172.21.15.184:6835/1963635253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5651a1a931c0 con 0x5651a1bd36c0 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b6dffb640 10 client.7047 handle_command_reply: tid=0 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 client.7047 shutdown 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 msgr2=0x5651a1b452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7b900b3c30 tx=0x7f7b78004680 comp rx=0 tx=0).stop 2024-04-26T21:16:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 shutdown_connections 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:6835/1963635253,v1:172.21.15.184:6836/1963635253] conn(0x5651a1bd36c0 0x5651a1b48140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5651a1b6ece0 0x5651a1bd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5651a1b51440 0x5651a1b452f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b6dffb640 0 client.7047 ms_handle_reset on v2:172.21.15.184:6835/1963635253 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 --2- 172.21.15.40:0/2207040698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a1b4d6f0 0x5651a1b86350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 >> 172.21.15.40:0/2207040698 conn(0x5651a1b27120 msgr2=0x5651a1b8edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 shutdown_connections 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 1 -- 172.21.15.40:0/2207040698 wait complete. 2024-04-26T21:16:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.082+0000 7f7b98368740 20 client.7047 trim_cache size 0 max 16384 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6912, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 5.7287092079999997, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 46, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 83, 2024-04-26T21:16:03.092 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.736422953 2024-04-26T21:16:03.093 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:16:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 msgr2=0x7f7b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 0x7f7b7006edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006fd0 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 msgr2=0x7f7b90158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b800046c0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 shutdown_connections 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b7006c8f0 0x7f7b7006edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b90140350 0x7f7b9015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b9013ff70 0x7f7b9015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 --2- 172.21.15.40:0/19340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b900cff20 0x7f7b90158400 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 >> 172.21.15.40:0/19340004 conn(0x7f7b900d0d30 msgr2=0x7f7b900b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 shutdown_connections 2024-04-26T21:16:03.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.083+0000 7f7b95650640 1 -- 172.21.15.40:0/19340004 wait complete. 2024-04-26T21:16:03.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:16:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.194+0000 7fd6141f0640 1 Processor -- start 2024-04-26T21:16:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.194+0000 7fd6141f0640 1 -- start start 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd6141f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd6141f0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd60c0d51a0 0x7fd60c0d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd6141f0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd60c0d2440 con 0x7fd60c0d3320 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd6141f0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd60c0d0030 con 0x7fd60c0d51a0 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd6141f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c0d2720 con 0x7fd60c15f750 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd60c0d51a0 0x7fd60c0d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd60c0d51a0 0x7fd60c0d5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47028/0 (socket says 172.21.15.40:47028) 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 -- 172.21.15.40:0/2743952223 learned_addr learned my addr 172.21.15.40:0/2743952223 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd611f8c640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 -- 172.21.15.40:0/2743952223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 msgr2=0x7fd60c0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 -- 172.21.15.40:0/2743952223 >> v1:172.21.15.184:6789/0 conn(0x7fd60c0d3320 legacy=0x7fd60c0d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.195+0000 7fd61278d640 1 -- 172.21.15.40:0/2743952223 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c0d2a00 con 0x7fd60c0d51a0 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.196+0000 7fd61278d640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0d5650 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e50 tx=0x7fd60402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdffe1ed5cbc17f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.196+0000 7fd610f8a640 1 -- 172.21.15.40:0/2743952223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604036070 con 0x7fd60c0d51a0 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.196+0000 7fd610f8a640 1 -- 172.21.15.40:0/2743952223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604031600 con 0x7fd60c0d51a0 2024-04-26T21:16:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.196+0000 7fd610f8a640 1 -- 172.21.15.40:0/2743952223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6040319d0 con 0x7fd60c0d51a0 2024-04-26T21:16:03.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 -- 172.21.15.40:0/2743952223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d51a0 msgr2=0x7fd60c0d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0d5650 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e50 tx=0x7fd60402a710 comp rx=0 tx=0).stop 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 -- 172.21.15.40:0/2743952223 shutdown_connections 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0d5650 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 --2- 172.21.15.40:0/2743952223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 -- 172.21.15.40:0/2743952223 >> 172.21.15.40:0/2743952223 conn(0x7fd60c08ec20 msgr2=0x7fd60c09e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 -- 172.21.15.40:0/2743952223 shutdown_connections 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.197+0000 7fd6141f0640 1 -- 172.21.15.40:0/2743952223 wait complete. 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.198+0000 7fd6141f0640 1 Processor -- start 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.198+0000 7fd6141f0640 1 -- start start 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c0d0850 con 0x7fd60c0d3320 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c0d06d0 con 0x7fd60c0d51a0 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd6141f0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c0d09d0 con 0x7fd60c15f750 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47036/0 (socket says 172.21.15.40:47036) 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 -- 172.21.15.40:0/30195197 learned_addr learned my addr 172.21.15.40:0/30195197 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd61278d640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd61178b640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 -- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 msgr2=0x7fd60c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 -- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60c0d51a0 msgr2=0x7fd60c0ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 -- 172.21.15.40:0/30195197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c161820 con 0x7fd60c0d3320 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.199+0000 7fd611f8c640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d620 tx=0x7fd60800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.200+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60800b620 con 0x7fd60c0d3320 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.200+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd608019070 con 0x7fd60c0d3320 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.200+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c161b00 con 0x7fd60c0d3320 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.200+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60800ba00 con 0x7fd60c0d3320 2024-04-26T21:16:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.200+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1620c0 con 0x7fd60c0d3320 2024-04-26T21:16:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.201+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c0693d0 con 0x7fd60c0d3320 2024-04-26T21:16:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.201+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd60800bba0 con 0x7fd60c0d3320 2024-04-26T21:16:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.202+0000 7fd6027fc640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 0x7fd5f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.202+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fd60808bfd0 con 0x7fd60c0d3320 2024-04-26T21:16:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.202+0000 7fd61178b640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 0x7fd5f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.202+0000 7fd61178b640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 0x7fd5f006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd60c1f8660 tx=0x7fd5fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:03.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.206+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd60805a400 con 0x7fd60c0d3320 2024-04-26T21:16:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.431+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd60c0d3f40 con 0x7fd60c0d3320 2024-04-26T21:16:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.431+0000 7fd6027fc640 1 -- 172.21.15.40:0/30195197 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7fd608059d00 con 0x7fd60c0d3320 2024-04-26T21:16:03.434 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 msgr2=0x7fd5f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 0x7fd5f006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd60c1f8660 tx=0x7fd5fc009040 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 msgr2=0x7fd60c154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d620 tx=0x7fd60800dae0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 shutdown_connections 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd5f006c8f0 0x7fd5f006edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd60c15f750 0x7fd60c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd60c0d51a0 0x7fd60c0ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 --2- 172.21.15.40:0/30195197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd60c0d3320 0x7fd60c154bf0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 >> 172.21.15.40:0/30195197 conn(0x7fd60c08ec20 msgr2=0x7fd60c1630f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 shutdown_connections 2024-04-26T21:16:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03.434+0000 7fd6141f0640 1 -- 172.21.15.40:0/30195197 wait complete. 2024-04-26T21:16:03.446 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:16:03.447 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:16:03.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.447 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:16:03.463 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42506sec preferred_lft 42506sec 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:16:03.464 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:16:03.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.466 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:16:03.466 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:16:03.466 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:16:03.466 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:03.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.466 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:16:03.526 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:16:03.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.528 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:16:03.585 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:16:03.585 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:16:03.585 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:16:03.586 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:16:03.586 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:16:03.586 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:16:03.586 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:16:03.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:16:03.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:16:03.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:16:03.587 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:16:03.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:16:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:16:03.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.712 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:16:03.727 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:16:03.727 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:16:03.727 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:16:03.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:16:03.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:16:03.792 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:16:03.847 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:16:03.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:16:03.847 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:16:03.901 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:16:03.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:03.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:16:03.984 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:16:03.984 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:16:03.984 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:16:03.992 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:16:03.992 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:16:03.992 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:16:03.993 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:16:03.993 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:16:03.998 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:16:03.999 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:16:03.999 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:16:03.999 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:16:03.999 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:16:03.999 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:16:04.000 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:16:04.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:16:04.065 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:16:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:16:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:16:04.169 INFO:teuthology.orchestra.run.smithi040.stdout:client5711 2024-04-26T21:16:04.169 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:16:04.169 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:16:04.185 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:16:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 Processor -- start 2024-04-26T21:16:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 -- start start 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f133c160640 0x7f133c1636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0cfa80 con 0x7f133c163be0 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0cfd60 con 0x7f133c148a30 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.323+0000 7f1342340640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f133c0745e0 con 0x7f133c160640 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f133c160640 0x7f133c1636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f13408dd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f133c163be0 0x7f133c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59416/0 (socket says 172.21.15.40:59416) 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f133c160640 0x7f133c1636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47046/0 (socket says 172.21.15.40:47046) 2024-04-26T21:16:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f13408dd640 1 -- 172.21.15.40:0/3039587743 learned_addr learned my addr 172.21.15.40:0/3039587743 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133b7fe640 1 -- 172.21.15.40:0/3039587743 >> v1:172.21.15.184:6789/0 conn(0x7f133c148a30 legacy=0x7f133c073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133b7fe640 1 -- 172.21.15.40:0/3039587743 >> v1:172.21.15.40:6789/0 conn(0x7f133c163be0 legacy=0x7f133c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133affd640 1 -- 172.21.15.40:0/3039587743 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930175506 0 0) 0x7f133c0cfa80 con 0x7f133c163be0 2024-04-26T21:16:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.324+0000 7f133b7fe640 1 -- 172.21.15.40:0/3039587743 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c0d0040 con 0x7f133c160640 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f133b7fe640 1 --2- 172.21.15.40:0/3039587743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c160640 0x7f133c1636a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f13300097a0 tx=0x7f133002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198de7518063edec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f133affd640 1 -- 172.21.15.40:0/3039587743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330037070 con 0x7f133c160640 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f133affd640 1 -- 172.21.15.40:0/3039587743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133002ae50 con 0x7f133c160640 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f133affd640 1 -- 172.21.15.40:0/3039587743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330032660 con 0x7f133c160640 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 -- 172.21.15.40:0/3039587743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c160640 msgr2=0x7f133c1636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 --2- 172.21.15.40:0/3039587743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c160640 0x7f133c1636a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f13300097a0 tx=0x7f133002a140 comp rx=0 tx=0).stop 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 -- 172.21.15.40:0/3039587743 shutdown_connections 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 --2- 172.21.15.40:0/3039587743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c160640 0x7f133c1636a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 -- 172.21.15.40:0/3039587743 >> 172.21.15.40:0/3039587743 conn(0x7f133c0ce8b0 msgr2=0x7f133c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 -- 172.21.15.40:0/3039587743 shutdown_connections 2024-04-26T21:16:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.325+0000 7f1342340640 1 -- 172.21.15.40:0/3039587743 wait complete. 2024-04-26T21:16:04.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.326+0000 7f1342340640 1 Processor -- start 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 -- start start 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c148a30 0x7f133c15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c160640 0x7f133c161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f133c148e10 con 0x7f133c163be0 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f133c163180 con 0x7f133c148a30 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f1342340640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f133c148f90 con 0x7f133c160640 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.327+0000 7f13408dd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47060/0 (socket says 172.21.15.40:47060) 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 -- 172.21.15.40:0/1356163032 learned_addr learned my addr 172.21.15.40:0/1356163032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f133bfff640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c148a30 0x7f133c15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f133b7fe640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c160640 0x7f133c161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 -- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c160640 msgr2=0x7f133c161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c160640 0x7f133c161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 -- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c148a30 msgr2=0x7f133c15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c148a30 0x7f133c15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 -- 172.21.15.40:0/1356163032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c0b13e0 con 0x7f133c163be0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f13408dd640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f133400c730 tx=0x7f133400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334018070 con 0x7f133c163be0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133400ce40 con 0x7f133c163be0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133400a860 con 0x7f133c163be0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c0b4d80 con 0x7f133c163be0 2024-04-26T21:16:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.328+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c0b5020 con 0x7f133c163be0 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.329+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c06a290 con 0x7f133c163be0 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.330+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133401b070 con 0x7f133c163be0 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.330+0000 7f1338ff9640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 0x7f132806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.330+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f133408b0c0 con 0x7f133c163be0 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.330+0000 7f133bfff640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 0x7f132806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.331+0000 7f133bfff640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 0x7f132806edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f132c004ea0 tx=0x7f132c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.335+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1334059580 con 0x7f133c163be0 2024-04-26T21:16:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.561+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f133c0739f0 con 0x7f133c163be0 2024-04-26T21:16:04.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.562+0000 7f1338ff9640 1 -- 172.21.15.40:0/1356163032 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f1334058e80 con 0x7f133c163be0 2024-04-26T21:16:04.564 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 msgr2=0x7f132806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 0x7f132806edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f132c004ea0 tx=0x7f132c009040 comp rx=0 tx=0).stop 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 msgr2=0x7f133c1620e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f133400c730 tx=0x7f133400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 shutdown_connections 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f132806c8f0 0x7f132806edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f133c163be0 0x7f133c1620e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c160640 0x7f133c161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 --2- 172.21.15.40:0/1356163032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c148a30 0x7f133c15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:04.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 >> 172.21.15.40:0/1356163032 conn(0x7f133c0ce8b0 msgr2=0x7f133c147d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:04.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 shutdown_connections 2024-04-26T21:16:04.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:04.564+0000 7f1342340640 1 -- 172.21.15.40:0/1356163032 wait complete. 2024-04-26T21:16:04.576 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:16:04.576 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:16:04.577 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:16:04.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.577 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:16:04.596 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:16:04.597 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42509sec preferred_lft 42509sec 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:16:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.599 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:16:04.599 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:16:04.599 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:16:04.599 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:16:04.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.599 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:16:04.663 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:16:04.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.665 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:16:04.722 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:16:04.722 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:16:04.722 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:16:04.723 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:16:04.723 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:16:04.723 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:16:04.815 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:16:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:16:04.846 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:16:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:16:04.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.853 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:16:04.868 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:16:04.868 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:16:04.868 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:16:04.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:04.868 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:16:04.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:16:04.932 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:16:04.987 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:16:04.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:16:04.987 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:16:05.042 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:16:05.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:05.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:16:05.123 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:16:05.132 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:16:05.132 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:16:05.132 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:16:05.132 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:16:05.132 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:16:05.136 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:16:05.137 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:16:05.138 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:16:05.138 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:16:05.138 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:16:05.138 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:16:05.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:16:05.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:16:05.139 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:16:05.202 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:16:05.289 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:16:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:16:05.291 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:16:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:16:05.296 INFO:teuthology.orchestra.run.smithi184.stdout:client6271 2024-04-26T21:16:05.296 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:16:05.297 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:16:05.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-26T21:16:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 Processor -- start 2024-04-26T21:16:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 -- start start 2024-04-26T21:16:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a180d22f0 0x7f9a180ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a180d4240 con 0x7f9a180d0ff0 2024-04-26T21:16:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9a180d31a0 con 0x7f9a180d22f0 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.427+0000 7f9a1c9da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180d3480 con 0x7f9a1815f490 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a180d22f0 0x7f9a180ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a180d22f0 0x7f9a180ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47066/0 (socket says 172.21.15.40:47066) 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 -- 172.21.15.40:0/130705091 learned_addr learned my addr 172.21.15.40:0/130705091 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a1659c640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 -- 172.21.15.40:0/130705091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 msgr2=0x7f9a180d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 -- 172.21.15.40:0/130705091 >> v1:172.21.15.184:6789/0 conn(0x7f9a180d0ff0 legacy=0x7f9a180d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a16d9d640 1 -- 172.21.15.40:0/130705091 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a180d3760 con 0x7f9a180d22f0 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.428+0000 7f9a1659c640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a16d9d640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a180ba3a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009e50 tx=0x7f9a1002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e62743e2a0da87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a15d9b640 1 -- 172.21.15.40:0/130705091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10036070 con 0x7f9a180d22f0 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a15d9b640 1 -- 172.21.15.40:0/130705091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10031600 con 0x7f9a180d22f0 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a15d9b640 1 -- 172.21.15.40:0/130705091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100319d0 con 0x7f9a180d22f0 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a1c9da640 1 -- 172.21.15.40:0/130705091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 msgr2=0x7f9a180ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.429+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a180ba3a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009e50 tx=0x7f9a1002a710 comp rx=0 tx=0).stop 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 -- 172.21.15.40:0/130705091 shutdown_connections 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a180ba3a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/130705091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 -- 172.21.15.40:0/130705091 >> 172.21.15.40:0/130705091 conn(0x7f9a1809e530 msgr2=0x7f9a180d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 -- 172.21.15.40:0/130705091 shutdown_connections 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.430+0000 7f9a1c9da640 1 -- 172.21.15.40:0/130705091 wait complete. 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.431+0000 7f9a1c9da640 1 Processor -- start 2024-04-26T21:16:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.431+0000 7f9a1c9da640 1 -- start start 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 0x7f9a180cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1815f870 con 0x7f9a180d22f0 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180d3d20 con 0x7f9a180d0ff0 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1c9da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180d2dc0 con 0x7f9a1815f490 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47080/0 (socket says 172.21.15.40:47080) 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 -- 172.21.15.40:0/3906512486 learned_addr learned my addr 172.21.15.40:0/3906512486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1659c640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 0x7f9a180cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a16d9d640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 -- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 msgr2=0x7f9a180caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 -- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 msgr2=0x7f9a180cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 0x7f9a180cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 -- 172.21.15.40:0/3906512486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a180c94e0 con 0x7f9a180d22f0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a16d9d640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a1659c640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 0x7f9a180cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.432+0000 7f9a0ffff640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c440 tx=0x7f9a0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.433+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000052d0 con 0x7f9a180d22f0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.433+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00005470 con 0x7f9a180d22f0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.433+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a180c97c0 con 0x7f9a180d22f0 2024-04-26T21:16:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.433+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00007260 con 0x7f9a180d22f0 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.433+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181584b0 con 0x7f9a180d22f0 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.434+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1806a290 con 0x7f9a180d22f0 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.434+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a000074c0 con 0x7f9a180d22f0 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.435+0000 7f9a0effd640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.435+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f9a0008afd0 con 0x7f9a180d22f0 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.435+0000 7f9a1659c640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.435+0000 7f9a1659c640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9a08006fd0 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.439+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a00059400 con 0x7f9a180d22f0 2024-04-26T21:16:05.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.630+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f9a180d5900 con 0x7f9a180d22f0 2024-04-26T21:16:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.633+0000 7f9a0effd640 1 -- 172.21.15.40:0/3906512486 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f9a00058d00 con 0x7f9a180d22f0 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 msgr2=0x7f99f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9a08006fd0 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 msgr2=0x7f9a18157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c440 tx=0x7f9a0000c900 comp rx=0 tx=0).stop 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 shutdown_connections 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a1815f490 0x7f9a180caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a180d22f0 0x7f9a18157d90 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 --2- 172.21.15.40:0/3906512486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a180d0ff0 0x7f9a180cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 >> 172.21.15.40:0/3906512486 conn(0x7f9a1809e530 msgr2=0x7f9a18162ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 shutdown_connections 2024-04-26T21:16:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.636+0000 7f9a1c9da640 1 -- 172.21.15.40:0/3906512486 wait complete. 2024-04-26T21:16:05.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-04-26T21:16:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.750+0000 7f6654bac640 1 Processor -- start 2024-04-26T21:16:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 -- start start 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f665015f680 0x7f66500d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6650163aa0 0x7f66500b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f66500b60b0 0x7f66500d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f665015fa60 con 0x7f6650163aa0 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f66500d3f50 con 0x7f66500b60b0 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6654bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66500b5090 con 0x7f665015f680 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.751+0000 7f6647fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6650163aa0 0x7f66500b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6650163aa0 0x7f66500b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47090/0 (socket says 172.21.15.40:47090) 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 -- 172.21.15.40:0/3097205181 learned_addr learned my addr 172.21.15.40:0/3097205181 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f664ed9d640 1 --2- 172.21.15.40:0/3097205181 >> v2:172.21.15.184:3300/0 conn(0x7f66500b60b0 0x7f66500d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f664e59c640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f665015f680 0x7f66500d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 -- 172.21.15.40:0/3097205181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f665015f680 msgr2=0x7f66500d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f665015f680 0x7f66500d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 -- 172.21.15.40:0/3097205181 >> v2:172.21.15.184:3300/0 conn(0x7f66500b60b0 msgr2=0x7f66500d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 --2- 172.21.15.40:0/3097205181 >> v2:172.21.15.184:3300/0 conn(0x7f66500b60b0 0x7f66500d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.752+0000 7f6647fff640 1 -- 172.21.15.40:0/3097205181 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500b5370 con 0x7f6650163aa0 2024-04-26T21:16:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6647fff640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f66500b5b70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f66500b4b10 tx=0x7f663802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dffd4095574ea24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f664dd9b640 1 -- 172.21.15.40:0/3097205181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638037070 con 0x7f6650163aa0 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f664dd9b640 1 -- 172.21.15.40:0/3097205181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638032440 con 0x7f6650163aa0 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f664dd9b640 1 -- 172.21.15.40:0/3097205181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638032810 con 0x7f6650163aa0 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 -- 172.21.15.40:0/3097205181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 msgr2=0x7f66500b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f66500b5b70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f66500b4b10 tx=0x7f663802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 -- 172.21.15.40:0/3097205181 shutdown_connections 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 --2- 172.21.15.40:0/3097205181 >> v2:172.21.15.184:3300/0 conn(0x7f66500b60b0 0x7f66500d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f66500b5b70 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 --2- 172.21.15.40:0/3097205181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f665015f680 0x7f66500d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 -- 172.21.15.40:0/3097205181 >> 172.21.15.40:0/3097205181 conn(0x7f665008eca0 msgr2=0x7f665009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 -- 172.21.15.40:0/3097205181 shutdown_connections 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.753+0000 7f6654bac640 1 -- 172.21.15.40:0/3097205181 wait complete. 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 Processor -- start 2024-04-26T21:16:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 -- start start 2024-04-26T21:16:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66500b60b0 0x7f6650149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f665015f680 0x7f66500cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66500d55e0 con 0x7f6650163aa0 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66500d5460 con 0x7f665015f680 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.755+0000 7f6654bac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66500d5760 con 0x7f66500b60b0 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47096/0 (socket says 172.21.15.40:47096) 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 -- 172.21.15.40:0/2491079467 learned_addr learned my addr 172.21.15.40:0/2491079467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6647fff640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f665015f680 0x7f66500cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664e59c640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66500b60b0 0x7f6650149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 -- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66500b60b0 msgr2=0x7f6650149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66500b60b0 0x7f6650149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 -- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f665015f680 msgr2=0x7f66500cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f665015f680 0x7f66500cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 -- 172.21.15.40:0/2491079467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501490f0 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f664ed9d640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f664800eed0 tx=0x7f664800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648018020 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6648014400 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6650160150 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480147f0 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.756+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6650162540 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.757+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665006a290 con 0x7f6650163aa0 2024-04-26T21:16:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.758+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6648005d10 con 0x7f6650163aa0 2024-04-26T21:16:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.758+0000 7f6646ffd640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 0x7f663006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.758+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f664808bff0 con 0x7f6650163aa0 2024-04-26T21:16:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.759+0000 7f664e59c640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 0x7f663006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.759+0000 7f664e59c640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 0x7f663006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6640009b40 tx=0x7f6640009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.763+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6648022070 con 0x7f6650163aa0 2024-04-26T21:16:05.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.950+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f66500b4cd0 con 0x7f6650163aa0 2024-04-26T21:16:05.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.953+0000 7f6646ffd640 1 -- 172.21.15.40:0/2491079467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v26)=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7f66501490f0 con 0x7f6650163aa0 2024-04-26T21:16:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 msgr2=0x7f663006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 0x7f663006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6640009b40 tx=0x7f6640009190 comp rx=0 tx=0).stop 2024-04-26T21:16:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 msgr2=0x7f6650145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f664800eed0 tx=0x7f664800b480 comp rx=0 tx=0).stop 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 shutdown_connections 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f663006c8f0 0x7f663006edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6650163aa0 0x7f6650145300 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f665015f680 0x7f66500cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 --2- 172.21.15.40:0/2491079467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66500b60b0 0x7f6650149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:05.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 >> 172.21.15.40:0/2491079467 conn(0x7f665008eca0 msgr2=0x7f66500b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:05.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 shutdown_connections 2024-04-26T21:16:05.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:05.955+0000 7f6654bac640 1 -- 172.21.15.40:0/2491079467 wait complete. 2024-04-26T21:16:05.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-04-26T21:16:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.071+0000 7fb2dba47640 1 Processor -- start 2024-04-26T21:16:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.071+0000 7fb2dba47640 1 -- start start 2024-04-26T21:16:06.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2dba47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2d406db10 0x7fb2d416a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2dba47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d409c700 con 0x7fb2d40b3c20 2024-04-26T21:16:06.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2dba47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d40d1c50 con 0x7fb2d40d0010 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2dba47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d40b4480 con 0x7fb2d406db10 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d9fe4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2d406db10 0x7fb2d416a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d8fe2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb2d40b3c20 0x7fb2d40b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59426/0 (socket says 172.21.15.40:59426) 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d8fe2640 1 -- 172.21.15.40:0/432923343 learned_addr learned my addr 172.21.15.40:0/432923343 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/432923343 >> v1:172.21.15.184:6789/0 conn(0x7fb2d40d0010 legacy=0x7fb2d40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/432923343 >> v1:172.21.15.40:6789/0 conn(0x7fb2d40b3c20 legacy=0x7fb2d40b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2cbfff640 1 -- 172.21.15.40:0/432923343 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983345284 0 0) 0x7fb2d409c700 con 0x7fb2d40b3c20 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.072+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/432923343 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d40d1f30 con 0x7fb2d406db10 2024-04-26T21:16:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.073+0000 7fb2d9fe4640 1 --2- 172.21.15.40:0/432923343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d406db10 0x7fb2d416a5b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d90 tx=0x7fb2cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce080fd7ae8ca1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.073+0000 7fb2cbfff640 1 -- 172.21.15.40:0/432923343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc007740 con 0x7fb2d406db10 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.073+0000 7fb2cbfff640 1 -- 172.21.15.40:0/432923343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc0078e0 con 0x7fb2d406db10 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.073+0000 7fb2cbfff640 1 -- 172.21.15.40:0/432923343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc007cb0 con 0x7fb2d406db10 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 -- 172.21.15.40:0/432923343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d406db10 msgr2=0x7fb2d416a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 --2- 172.21.15.40:0/432923343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d406db10 0x7fb2d416a5b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d90 tx=0x7fb2cc02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 -- 172.21.15.40:0/432923343 shutdown_connections 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 --2- 172.21.15.40:0/432923343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d406db10 0x7fb2d416a5b0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 -- 172.21.15.40:0/432923343 >> 172.21.15.40:0/432923343 conn(0x7fb2d409ba40 msgr2=0x7fb2d409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 -- 172.21.15.40:0/432923343 shutdown_connections 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.074+0000 7fb2dba47640 1 -- 172.21.15.40:0/432923343 wait complete. 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.075+0000 7fb2dba47640 1 Processor -- start 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.075+0000 7fb2dba47640 1 -- start start 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 0x7fb2d415f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 0x7fb2d4163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d416ac70 con 0x7fb2d40d0010 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d416aaf0 con 0x7fb2d406db10 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2dba47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d416adf0 con 0x7fb2d40b3c20 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47118/0 (socket says 172.21.15.40:47118) 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/1731838018 learned_addr learned my addr 172.21.15.40:0/1731838018 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d8fe2640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 0x7fb2d4163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d97e3640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 0x7fb2d415f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 msgr2=0x7fb2d4163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 0x7fb2d4163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 msgr2=0x7fb2d415f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 0x7fb2d415f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 -- 172.21.15.40:0/1731838018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d40b36a0 con 0x7fb2d40d0010 2024-04-26T21:16:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d97e3640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 0x7fb2d415f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d8fe2640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 0x7fb2d4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.076+0000 7fb2d9fe4640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009eb0 tx=0x7fb2cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.077+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc03f070 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.077+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc031c90 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.077+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d415e810 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.077+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc043630 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.077+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d4158530 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.078+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d40d2420 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.078+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2cc03b410 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.079+0000 7fb2c9ffb640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.079+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fb2cc0ad170 con 0x7fb2d40d0010 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.079+0000 7fb2d97e3640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.080+0000 7fb2d97e3640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 0x7fb2b806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0007fd0 tx=0x7fb2d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.083+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2d40d2420 con 0x7fb2d40d0010 2024-04-26T21:16:06.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.267+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fb2d406a290 con 0x7fb2d40d0010 2024-04-26T21:16:06.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.270+0000 7fb2c9ffb640 1 -- 172.21.15.40:0/1731838018 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v27)=0 v27) ==== 128+0+0 (secure 0 0 0) 0x7fb2cc07b5c0 con 0x7fb2d40d0010 2024-04-26T21:16:06.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 msgr2=0x7fb2b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 0x7fb2b806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0007fd0 tx=0x7fb2d0009040 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 msgr2=0x7fb2d4155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009eb0 tx=0x7fb2cc004a40 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 shutdown_connections 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2b806c8f0 0x7fb2b806edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2d40d0010 0x7fb2d4155550 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2d40b3c20 0x7fb2d4163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 --2- 172.21.15.40:0/1731838018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2d406db10 0x7fb2d415f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 >> 172.21.15.40:0/1731838018 conn(0x7fb2d409ba40 msgr2=0x7fb2d40b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 shutdown_connections 2024-04-26T21:16:06.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.273+0000 7fb2dba47640 1 -- 172.21.15.40:0/1731838018 wait complete. 2024-04-26T21:16:06.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2024-04-26T21:16:06.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.386+0000 7f0b7b76a640 1 Processor -- start 2024-04-26T21:16:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.387+0000 7f0b7b76a640 1 -- start start 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.387+0000 7f0b7b76a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7406f340 0x7f0b740b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.387+0000 7f0b7b76a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0b740b3e50 0x7f0b740720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b7b76a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b740b46a0 con 0x7f0b740d0300 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b7b76a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0b740b5db0 con 0x7f0b7406f340 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b7b76a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0b740b6090 con 0x7f0b740b3e50 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7406f340 0x7f0b740b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b78d05640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0b740d0300 0x7f0b7406f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59438/0 (socket says 172.21.15.40:59438) 2024-04-26T21:16:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7406f340 0x7f0b740b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47122/0 (socket says 172.21.15.40:47122) 2024-04-26T21:16:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b78d05640 1 -- 172.21.15.40:0/4080952514 learned_addr learned my addr 172.21.15.40:0/4080952514 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 -- 172.21.15.40:0/4080952514 >> v2:172.21.15.184:3300/0 conn(0x7f0b740b3e50 msgr2=0x7f0b740720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 --2- 172.21.15.40:0/4080952514 >> v2:172.21.15.184:3300/0 conn(0x7f0b740b3e50 0x7f0b740720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 -- 172.21.15.40:0/4080952514 >> v1:172.21.15.40:6789/0 conn(0x7f0b740d0300 legacy=0x7f0b7406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.388+0000 7f0b79506640 1 -- 172.21.15.40:0/4080952514 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b740b6370 con 0x7f0b7406f340 2024-04-26T21:16:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b79506640 1 --2- 172.21.15.40:0/4080952514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b740b6b70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0b74070020 tx=0x7f0b7002c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c79ed325fa13006 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b6bfff640 1 -- 172.21.15.40:0/4080952514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70039070 con 0x7f0b7406f340 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b6bfff640 1 -- 172.21.15.40:0/4080952514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70034440 con 0x7f0b7406f340 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b6bfff640 1 -- 172.21.15.40:0/4080952514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b700347f0 con 0x7f0b7406f340 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 -- 172.21.15.40:0/4080952514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 msgr2=0x7f0b740b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/4080952514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b740b6b70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0b74070020 tx=0x7f0b7002c290 comp rx=0 tx=0).stop 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 -- 172.21.15.40:0/4080952514 shutdown_connections 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/4080952514 >> v2:172.21.15.184:3300/0 conn(0x7f0b740b3e50 0x7f0b740720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/4080952514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b740b6b70 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.389+0000 7f0b7b76a640 1 -- 172.21.15.40:0/4080952514 >> 172.21.15.40:0/4080952514 conn(0x7f0b740cd8a0 msgr2=0x7f0b740cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.390+0000 7f0b7b76a640 1 -- 172.21.15.40:0/4080952514 shutdown_connections 2024-04-26T21:16:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.390+0000 7f0b7b76a640 1 -- 172.21.15.40:0/4080952514 wait complete. 2024-04-26T21:16:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 Processor -- start 2024-04-26T21:16:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 -- start start 2024-04-26T21:16:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 0x7f0b7413ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 0x7f0b7413b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b4460 con 0x7f0b7406f340 2024-04-26T21:16:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b42e0 con 0x7f0b740b3e50 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b7b76a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b74142a30 con 0x7f0b740d0300 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b79506640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.391+0000 7f0b79506640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47132/0 (socket says 172.21.15.40:47132) 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 -- 172.21.15.40:0/2488414449 learned_addr learned my addr 172.21.15.40:0/2488414449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b78d05640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 0x7f0b7413ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79d07640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 0x7f0b7413b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 -- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 msgr2=0x7f0b7413b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 0x7f0b7413b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 -- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 msgr2=0x7f0b7413ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 0x7f0b7413ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 -- 172.21.15.40:0/2488414449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b741419d0 con 0x7f0b7406f340 2024-04-26T21:16:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b78d05640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 0x7f0b7413ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79d07640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 0x7f0b7413b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b79506640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009890 tx=0x7f0b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70039040 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70044070 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70032bb0 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b74151c60 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.392+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b74152220 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.393+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7406a290 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.394+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b70049460 con 0x7f0b7406f340 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.394+0000 7f0b69ffb640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.394+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0b700ac9b0 con 0x7f0b7406f340 2024-04-26T21:16:06.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.395+0000 7f0b78d05640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.395+0000 7f0b78d05640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 0x7f0b5806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0b741581f0 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.400+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b70034e20 con 0x7f0b7406f340 2024-04-26T21:16:06.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.583+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f0b7415c120 con 0x7f0b7406f340 2024-04-26T21:16:06.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.586+0000 7f0b69ffb640 1 -- 172.21.15.40:0/2488414449 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v28)=0 v28) ==== 124+0+0 (secure 0 0 0) 0x7f0b7007acb0 con 0x7f0b7406f340 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 msgr2=0x7f0b5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 0x7f0b5806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0b741581f0 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 msgr2=0x7f0b74154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009890 tx=0x7f0b700046c0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 shutdown_connections 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5806c8f0 0x7f0b5806edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b740d0300 0x7f0b7413b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b740b3e50 0x7f0b7413ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 --2- 172.21.15.40:0/2488414449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7406f340 0x7f0b74154520 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 >> 172.21.15.40:0/2488414449 conn(0x7f0b740cd8a0 msgr2=0x7f0b7415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 shutdown_connections 2024-04-26T21:16:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.589+0000 7f0b7b76a640 1 -- 172.21.15.40:0/2488414449 wait complete. 2024-04-26T21:16:06.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-26T21:16:06.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.703+0000 7f66a48cc640 1 Processor -- start 2024-04-26T21:16:06.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.703+0000 7f66a48cc640 1 -- start start 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f66a48cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66a00d1be0 0x7f66a00d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f66a48cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a00ba3a0 0x7f66a00ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f66a48cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a00d4240 con 0x7f66a015f490 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f66a48cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f66a00d31a0 con 0x7f66a00d1be0 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f66a48cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66a00d3480 con 0x7f66a00ba3a0 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66a00d1be0 0x7f66a00d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f66a015f490 0x7f66a00d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59450/0 (socket says 172.21.15.40:59450) 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669e59c640 1 -- 172.21.15.40:0/2941591520 learned_addr learned my addr 172.21.15.40:0/2941591520 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669ed9d640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a00ba3a0 0x7f66a00ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669dd9b640 1 -- 172.21.15.40:0/2941591520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a00ba3a0 msgr2=0x7f66a00ba7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669dd9b640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a00ba3a0 0x7f66a00ba7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669dd9b640 1 -- 172.21.15.40:0/2941591520 >> v1:172.21.15.40:6789/0 conn(0x7f66a015f490 legacy=0x7f66a00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669d59a640 1 -- 172.21.15.40:0/2941591520 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679456290 0 0) 0x7f66a00d4240 con 0x7f66a015f490 2024-04-26T21:16:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.704+0000 7f669dd9b640 1 -- 172.21.15.40:0/2941591520 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a00d3760 con 0x7f66a00d1be0 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f669dd9b640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00d1be0 0x7f66a00d2070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f66940097a0 tx=0x7f669402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52020c2a40d29e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f669d59a640 1 -- 172.21.15.40:0/2941591520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694037070 con 0x7f66a00d1be0 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f669d59a640 1 -- 172.21.15.40:0/2941591520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694032440 con 0x7f66a00d1be0 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f669d59a640 1 -- 172.21.15.40:0/2941591520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694032810 con 0x7f66a00d1be0 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f66a48cc640 1 -- 172.21.15.40:0/2941591520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00d1be0 msgr2=0x7f66a00d2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.705+0000 7f66a48cc640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00d1be0 0x7f66a00d2070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f66940097a0 tx=0x7f669402a180 comp rx=0 tx=0).stop 2024-04-26T21:16:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 -- 172.21.15.40:0/2941591520 shutdown_connections 2024-04-26T21:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a00ba3a0 0x7f66a00ba7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 --2- 172.21.15.40:0/2941591520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00d1be0 0x7f66a00d2070 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 -- 172.21.15.40:0/2941591520 >> 172.21.15.40:0/2941591520 conn(0x7f66a009e530 msgr2=0x7f66a00d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 -- 172.21.15.40:0/2941591520 shutdown_connections 2024-04-26T21:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.706+0000 7f66a48cc640 1 -- 172.21.15.40:0/2941591520 wait complete. 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.707+0000 7f66a48cc640 1 Processor -- start 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.707+0000 7f66a48cc640 1 -- start start 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66a00d1be0 0x7f66a0162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a015f490 0x7f66a014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f66a00d11c0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66a00d3d20 con 0x7f66a00d1be0 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f66a015f870 con 0x7f66a015f490 2024-04-26T21:16:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47150/0 (socket says 172.21.15.40:47150) 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 -- 172.21.15.40:0/313730989 learned_addr learned my addr 172.21.15.40:0/313730989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669dd9b640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66a00d1be0 0x7f66a0162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 -- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a015f490 msgr2=0x7f66a014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a015f490 0x7f66a014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 -- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66a00d1be0 msgr2=0x7f66a0162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66a00d1be0 0x7f66a0162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 -- 172.21.15.40:0/313730989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a00cf8f0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f669e59c640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f669000c460 tx=0x7f669000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690018070 con 0x7f66a00ba3a0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690007500 con 0x7f66a00ba3a0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690004a20 con 0x7f66a00ba3a0 2024-04-26T21:16:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.708+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a014b4b0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.709+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a00ce5e0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.709+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a006a290 con 0x7f66a00ba3a0 2024-04-26T21:16:06.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.710+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f669001b070 con 0x7f66a00ba3a0 2024-04-26T21:16:06.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.710+0000 7f668effd640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 0x7f668006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:06.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.710+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f669008ad90 con 0x7f66a00ba3a0 2024-04-26T21:16:06.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.711+0000 7f669dd9b640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 0x7f668006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:06.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.711+0000 7f669dd9b640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 0x7f668006ede0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6694034000 tx=0x7f669402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:06.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.715+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66900591c0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.902+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f66a00d5900 con 0x7f66a00ba3a0 2024-04-26T21:16:06.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.902+0000 7f668effd640 1 -- 172.21.15.40:0/313730989 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v28) ==== 106+0+4 (secure 0 0 0) 0x7f6690058ac0 con 0x7f66a00ba3a0 2024-04-26T21:16:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:100 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 msgr2=0x7f668006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 0x7f668006ede0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6694034000 tx=0x7f669402e000 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 msgr2=0x7f66a00ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f669000c460 tx=0x7f669000c920 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 shutdown_connections 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f668006c920 0x7f668006ede0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66a015f490 0x7f66a014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66a00d1be0 0x7f66a0162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 --2- 172.21.15.40:0/313730989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66a00ba3a0 0x7f66a00ccd40 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 >> 172.21.15.40:0/313730989 conn(0x7f66a009e530 msgr2=0x7f66a006e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 shutdown_connections 2024-04-26T21:16:06.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:06.905+0000 7f66a48cc640 1 -- 172.21.15.40:0/313730989 wait complete. 2024-04-26T21:16:06.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-26T21:16:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 Processor -- start 2024-04-26T21:16:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 -- start start 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca81631f0 0x7f0ca80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca81635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca809c790 con 0x7f0ca80b3300 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca80d15f0 con 0x7f0ca81631f0 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0cad6f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca8163ec0 con 0x7f0ca80d1170 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca81631f0 0x7f0ca80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0ca80b3300 0x7f0ca80b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59464/0 (socket says 172.21.15.40:59464) 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca81631f0 0x7f0ca80b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47164/0 (socket says 172.21.15.40:47164) 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca77fe640 1 -- 172.21.15.40:0/2817280283 learned_addr learned my addr 172.21.15.40:0/2817280283 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0c9ffff640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca81635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 -- 172.21.15.40:0/2817280283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 msgr2=0x7f0ca81635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca81635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 -- 172.21.15.40:0/2817280283 >> v1:172.21.15.40:6789/0 conn(0x7f0ca80b3300 legacy=0x7f0ca80b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca6ffd640 1 -- 172.21.15.40:0/2817280283 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80d1a40 con 0x7f0ca81631f0 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0ca67fc640 1 -- 172.21.15.40:0/2817280283 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845543275 0 0) 0x7f0ca809c790 con 0x7f0ca80b3300 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.019+0000 7f0c9ffff640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca81635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.020+0000 7f0ca6ffd640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca80b2000 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0c980097a0 tx=0x7f0c9802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f429ee500916e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.020+0000 7f0ca67fc640 1 -- 172.21.15.40:0/2817280283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98037070 con 0x7f0ca81631f0 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.020+0000 7f0ca67fc640 1 -- 172.21.15.40:0/2817280283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9802ae50 con 0x7f0ca81631f0 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.020+0000 7f0ca67fc640 1 -- 172.21.15.40:0/2817280283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98032660 con 0x7f0ca81631f0 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 -- 172.21.15.40:0/2817280283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 msgr2=0x7f0ca80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca80b2000 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0c980097a0 tx=0x7f0c9802a100 comp rx=0 tx=0).stop 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 -- 172.21.15.40:0/2817280283 shutdown_connections 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca81635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/2817280283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca80b2000 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 -- 172.21.15.40:0/2817280283 >> 172.21.15.40:0/2817280283 conn(0x7f0ca809ba40 msgr2=0x7f0ca809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 -- 172.21.15.40:0/2817280283 shutdown_connections 2024-04-26T21:16:07.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.021+0000 7f0cad6f1640 1 -- 172.21.15.40:0/2817280283 wait complete. 2024-04-26T21:16:07.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.022+0000 7f0cad6f1640 1 Processor -- start 2024-04-26T21:16:07.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.022+0000 7f0cad6f1640 1 -- start start 2024-04-26T21:16:07.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca80b3300 0x7f0ca81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca80c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d0230 con 0x7f0ca81631f0 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d2000 con 0x7f0ca80b3300 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80c74f0 con 0x7f0ca80d1170 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47178/0 (socket says 172.21.15.40:47178) 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 -- 172.21.15.40:0/1418825564 learned_addr learned my addr 172.21.15.40:0/1418825564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0c9ffff640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca80c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca6ffd640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca80b3300 0x7f0ca81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 -- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 msgr2=0x7f0ca80c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca80c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 -- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca80b3300 msgr2=0x7f0ca81502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca80b3300 0x7f0ca81502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 -- 172.21.15.40:0/1418825564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80c77d0 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0ca77fe640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c7d0 tx=0x7f0ca000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0018070 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca000a640 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca8160690 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.023+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000aa70 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.024+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca8150810 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.024+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca806a290 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.025+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ca001b070 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.025+0000 7f0c9f7fe640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 0x7f0c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.026+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0ca008b080 con 0x7f0ca81631f0 2024-04-26T21:16:07.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.026+0000 7f0ca6ffd640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 0x7f0c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.026+0000 7f0ca6ffd640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 0x7f0c8806edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c98034000 tx=0x7f0c9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.030+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ca0058e50 con 0x7f0ca81631f0 2024-04-26T21:16:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.217+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f0ca80d2560 con 0x7f0ca81631f0 2024-04-26T21:16:07.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.217+0000 7f0c9f7fe640 1 -- 172.21.15.40:0/1418825564 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v28) ==== 112+0+10 (secure 0 0 0) 0x7f0ca005d150 con 0x7f0ca81631f0 2024-04-26T21:16:07.219 INFO:teuthology.orchestra.run.smithi040.stdout:60.000000 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 msgr2=0x7f0c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 0x7f0c8806edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c98034000 tx=0x7f0c9802e000 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 msgr2=0x7f0ca806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c7d0 tx=0x7f0ca000cc90 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 shutdown_connections 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c8806c8f0 0x7f0c8806edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca81631f0 0x7f0ca806d9e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ca80d1170 0x7f0ca80c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 --2- 172.21.15.40:0/1418825564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ca80b3300 0x7f0ca81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.219+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 >> 172.21.15.40:0/1418825564 conn(0x7f0ca809ba40 msgr2=0x7f0ca80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.220+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 shutdown_connections 2024-04-26T21:16:07.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.220+0000 7f0cad6f1640 1 -- 172.21.15.40:0/1418825564 wait complete. 2024-04-26T21:16:07.232 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:16:07.233 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:16:07.248 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:16:07.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:16:07.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:16:07.345 INFO:teuthology.orchestra.run.smithi040.stdout:client5711 2024-04-26T21:16:07.345 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:16:07.345 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:16:07.361 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:16:07.361 DEBUG:teuthology.orchestra.run.smithi040:> import fcntl 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> import time 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> n = 400 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> if not os.path.exists(abs_path): 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(abs_path) 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:16:07.362 DEBUG:teuthology.orchestra.run.smithi040:> handles = [] 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(0, n): 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> fname = "file_"+str(i) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> path = os.path.join(abs_path, fname) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> handles.append(open(path, '"'"'w'"'"')) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> print("waiting with handles open", file=sys.stderr) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> while True: 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> if os.read(0, 4096) == b"": 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> break 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> except BlockingIOError: 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> pass 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(2) 2024-04-26T21:16:07.363 DEBUG:teuthology.orchestra.run.smithi040:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-26T21:16:07.364 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:16:07.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:16:07.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.504+0000 7f1d2b538640 1 Processor -- start 2024-04-26T21:16:07.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.504+0000 7f1d2b538640 1 -- start start 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d2b538640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d24070d20 0x7f1d24071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d2b538640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d24070520 0x7f1d24163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d2b538640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d240d14c0 con 0x7f1d2409c400 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d2b538640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d240d1290 con 0x7f1d24070520 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d2b538640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d24071640 con 0x7f1d24070d20 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d28ad3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d2409c400 0x7f1d24162f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59466/0 (socket says 172.21.15.40:59466) 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d29ad5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d24070520 0x7f1d24163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d28ad3640 1 -- 172.21.15.40:0/540739179 learned_addr learned my addr 172.21.15.40:0/540739179 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d292d4640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d24070d20 0x7f1d24071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d29ad5640 1 -- 172.21.15.40:0/540739179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d24070d20 msgr2=0x7f1d24071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d29ad5640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d24070d20 0x7f1d24071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d29ad5640 1 -- 172.21.15.40:0/540739179 >> v1:172.21.15.40:6789/0 conn(0x7f1d2409c400 legacy=0x7f1d24162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.505+0000 7f1d29ad5640 1 -- 172.21.15.40:0/540739179 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240ce5b0 con 0x7f1d24070520 2024-04-26T21:16:07.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.506+0000 7f1d29ad5640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070520 0x7f1d24163640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009d90 tx=0x7f1d1c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a41c3113e165b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.506+0000 7f1d17fff640 1 -- 172.21.15.40:0/540739179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c007740 con 0x7f1d24070520 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.506+0000 7f1d17fff640 1 -- 172.21.15.40:0/540739179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0078e0 con 0x7f1d24070520 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.506+0000 7f1d17fff640 1 -- 172.21.15.40:0/540739179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c007cb0 con 0x7f1d24070520 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 -- 172.21.15.40:0/540739179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070520 msgr2=0x7f1d24163640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070520 0x7f1d24163640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009d90 tx=0x7f1d1c02a660 comp rx=0 tx=0).stop 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 -- 172.21.15.40:0/540739179 shutdown_connections 2024-04-26T21:16:07.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070520 0x7f1d24163640 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 --2- 172.21.15.40:0/540739179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d24070d20 0x7f1d24071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 -- 172.21.15.40:0/540739179 >> 172.21.15.40:0/540739179 conn(0x7f1d240b4a40 msgr2=0x7f1d240b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 -- 172.21.15.40:0/540739179 shutdown_connections 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.507+0000 7f1d2b538640 1 -- 172.21.15.40:0/540739179 wait complete. 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.508+0000 7f1d2b538640 1 Processor -- start 2024-04-26T21:16:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.508+0000 7f1d2b538640 1 -- start start 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d24070520 0x7f1d240b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d2409c400 0x7f1d240b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d240d1d20 con 0x7f1d24070d20 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d240d1a40 con 0x7f1d24070520 2024-04-26T21:16:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d2b538640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d240d2000 con 0x7f1d2409c400 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47200/0 (socket says 172.21.15.40:47200) 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 -- 172.21.15.40:0/3782683648 learned_addr learned my addr 172.21.15.40:0/3782683648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d292d4640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d24070520 0x7f1d240b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d29ad5640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d2409c400 0x7f1d240b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 -- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d2409c400 msgr2=0x7f1d240b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d2409c400 0x7f1d240b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 -- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d24070520 msgr2=0x7f1d240b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d24070520 0x7f1d240b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.509+0000 7f1d28ad3640 1 -- 172.21.15.40:0/3782683648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240b6470 con 0x7f1d24070d20 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d28ad3640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c3f0 tx=0x7f1d2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20017070 con 0x7f1d24070d20 2024-04-26T21:16:07.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d20005ce0 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d240b6750 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200072f0 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.510+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d240b6d10 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.511+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d240693d0 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.511+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d2001a070 con 0x7f1d24070d20 2024-04-26T21:16:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.512+0000 7f1d15ffb640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.512+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f1d2008ae20 con 0x7f1d24070d20 2024-04-26T21:16:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.512+0000 7f1d292d4640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.512+0000 7f1d292d4640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 0x7f1d0806edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009aa0 tx=0x7f1d18009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.516+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d20058c00 con 0x7f1d24070d20 2024-04-26T21:16:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.740+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d24074880 con 0x7f1d24070d20 2024-04-26T21:16:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.741+0000 7f1d15ffb640 1 -- 172.21.15.40:0/3782683648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5186 (secure 0 0 0) 0x7f1d2005cf00 con 0x7f1d24070d20 2024-04-26T21:16:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:07.742 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:15:59.744328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:16:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 46 2024-04-26T21:16:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 msgr2=0x7f1d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 0x7f1d0806edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009aa0 tx=0x7f1d18009190 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 msgr2=0x7f1d240c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c3f0 tx=0x7f1d2000c8b0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 shutdown_connections 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d2409c400 0x7f1d240b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d24070d20 0x7f1d240c8fe0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 --2- 172.21.15.40:0/3782683648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d24070520 0x7f1d240b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 >> 172.21.15.40:0/3782683648 conn(0x7f1d240b4a40 msgr2=0x7f1d240b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 shutdown_connections 2024-04-26T21:16:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.743+0000 7f1d2b538640 1 -- 172.21.15.40:0/3782683648 wait complete. 2024-04-26T21:16:07.755 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-26T21:16:07.780 INFO:teuthology.orchestra.run.smithi040.stderr:waiting with handles open 2024-04-26T21:16:07.780 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:07.873 INFO:teuthology.orchestra.run.smithi184.stdout:[{"id":6271,"entity":{"name":{"type":"client","num":6271},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2217415069}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":2.72994469,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9918971296048094,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6271 v1:192.168.0.1:0/2217415069","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":5711,"entity":{"name":{"type":"client","num":5711},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":2382857355}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":425,"uptime":3.8699215929999999,"requests_in_flight":400,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":804.53806496146967,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5711 v1:172.21.15.40:0/2382857355","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:16:07.874 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2217415069, "type": "v1" }, "name": { "num": 6271, "type": "client" } }, "id": 6271, "inst": "client.6271 v1:192.168.0.1:0/2217415069", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9918971296048094 }, "state": "open", "uptime": 2.72994469 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 2382857355, "type": "v1" }, "name": { "num": 5711, "type": "client" } }, "id": 5711, "inst": "client.5711 v1:172.21.15.40:0/2382857355", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 425, "requests_in_flight": 400, "session_cache_liveness": { "halflife": 300, "value": 804.5380649614697 }, "state": "open", "uptime": 3.869921593 } ] 2024-04-26T21:16:07.874 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-26T21:16:07.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:07.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 Processor -- start 2024-04-26T21:16:07.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 -- start start 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf81468f0 0x7f9cf80d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cf80b43c0 0x7f9cf806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf806d060 con 0x7f9cf806f070 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf80d1510 con 0x7f9cf80b43c0 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.974+0000 7f9cfe19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf80d0940 con 0x7f9cf81468f0 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cf80b43c0 0x7f9cf806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cf80b43c0 0x7f9cf806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47206/0 (socket says 172.21.15.40:47206) 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 -- 172.21.15.40:0/2846436351 learned_addr learned my addr 172.21.15.40:0/2846436351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf77fe640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf81468f0 0x7f9cf80d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 -- 172.21.15.40:0/2846436351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf81468f0 msgr2=0x7f9cf80d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf81468f0 0x7f9cf80d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 -- 172.21.15.40:0/2846436351 >> v1:172.21.15.184:6789/0 conn(0x7f9cf806f070 legacy=0x7f9cf80b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf7fff640 1 -- 172.21.15.40:0/2846436351 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf80d0be0 con 0x7f9cf80b43c0 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.975+0000 7f9cf67fc640 1 -- 172.21.15.40:0/2846436351 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851875559 0 0) 0x7f9cf806d060 con 0x7f9cf806f070 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cf7fff640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf80b43c0 0x7f9cf806f4d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009e50 tx=0x7f9cf002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c3ab4ff93bd529 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cf67fc640 1 -- 172.21.15.40:0/2846436351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0036070 con 0x7f9cf80b43c0 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cf67fc640 1 -- 172.21.15.40:0/2846436351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf0031600 con 0x7f9cf80b43c0 2024-04-26T21:16:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cf67fc640 1 -- 172.21.15.40:0/2846436351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00319d0 con 0x7f9cf80b43c0 2024-04-26T21:16:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 -- 172.21.15.40:0/2846436351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf80b43c0 msgr2=0x7f9cf806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf80b43c0 0x7f9cf806f4d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009e50 tx=0x7f9cf002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 -- 172.21.15.40:0/2846436351 shutdown_connections 2024-04-26T21:16:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf80b43c0 0x7f9cf806f4d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/2846436351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf81468f0 0x7f9cf80d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 -- 172.21.15.40:0/2846436351 >> 172.21.15.40:0/2846436351 conn(0x7f9cf806d3b0 msgr2=0x7f9cf806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 -- 172.21.15.40:0/2846436351 shutdown_connections 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.976+0000 7f9cfe19d640 1 -- 172.21.15.40:0/2846436351 wait complete. 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.978+0000 7f9cfe19d640 1 Processor -- start 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.978+0000 7f9cfe19d640 1 -- start start 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.978+0000 7f9cfe19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf806f070 0x7f9cf813e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cf80b43c0 0x7f9cf813ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8146cd0 con 0x7f9cf81468f0 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf80d11a0 con 0x7f9cf80b43c0 2024-04-26T21:16:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf8141280 con 0x7f9cf806f070 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47208/0 (socket says 172.21.15.40:47208) 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 -- 172.21.15.40:0/3851192269 learned_addr learned my addr 172.21.15.40:0/3851192269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf77fe640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf806f070 0x7f9cf813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf6ffd640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cf80b43c0 0x7f9cf813ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 -- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf806f070 msgr2=0x7f9cf813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf806f070 0x7f9cf813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 -- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cf80b43c0 msgr2=0x7f9cf813ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cf80b43c0 0x7f9cf813ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 -- 172.21.15.40:0/3851192269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf8141520 con 0x7f9cf81468f0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cf7fff640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0034040 tx=0x7f9cf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0036050 con 0x7f9cf81468f0 2024-04-26T21:16:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf0042070 con 0x7f9cf81468f0 2024-04-26T21:16:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0007bf0 con 0x7f9cf81468f0 2024-04-26T21:16:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf8141800 con 0x7f9cf81468f0 2024-04-26T21:16:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.979+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf80aed10 con 0x7f9cf81468f0 2024-04-26T21:16:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.980+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf80693d0 con 0x7f9cf81468f0 2024-04-26T21:16:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.981+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cf0047460 con 0x7f9cf81468f0 2024-04-26T21:16:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.981+0000 7f9cdffff640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 0x7f9ce406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.982+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f9cf00ac7a0 con 0x7f9cf81468f0 2024-04-26T21:16:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.982+0000 7f9cf77fe640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 0x7f9ce406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.982+0000 7f9cf77fe640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 0x7f9ce406ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009b40 tx=0x7f9ce8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:07.986+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9cf007a610 con 0x7f9cf81468f0 2024-04-26T21:16:08.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.212+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9cf81419d0 con 0x7f9cf81468f0 2024-04-26T21:16:08.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.212+0000 7f9cdffff640 1 -- 172.21.15.40:0/3851192269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9cf007e910 con 0x7f9cf81468f0 2024-04-26T21:16:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:08.213 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:16:08.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 msgr2=0x7f9ce406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 0x7f9ce406ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009b40 tx=0x7f9ce8009190 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 msgr2=0x7f9cf8140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0034040 tx=0x7f9cf0004a40 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 shutdown_connections 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9ce406c920 0x7f9ce406ede0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cf81468f0 0x7f9cf8140d40 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cf80b43c0 0x7f9cf813ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 --2- 172.21.15.40:0/3851192269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9cf806f070 0x7f9cf813e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 >> 172.21.15.40:0/3851192269 conn(0x7f9cf806d3b0 msgr2=0x7f9cf8143840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 shutdown_connections 2024-04-26T21:16:08.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:08.214+0000 7f9cfe19d640 1 -- 172.21.15.40:0/3851192269 wait complete. 2024-04-26T21:16:08.226 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-26T21:16:08.226 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:13.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.339+0000 7f3b04a99640 1 Processor -- start 2024-04-26T21:16:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.339+0000 7f3b04a99640 1 -- start start 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3b04a99640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3b0015f4c0 0x7f3b00163a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3b04a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3b000d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3b04a99640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0006f2b0 con 0x7f3b0006d910 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3b04a99640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3b00068300 con 0x7f3b0015f4c0 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3b04a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006f060 con 0x7f3b00073010 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3af7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b0006d910 0x7f3b00072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59114/0 (socket says 172.21.15.40:59114) 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3af7fff640 1 -- 172.21.15.40:0/437654165 learned_addr learned my addr 172.21.15.40:0/437654165 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afe59c640 1 --2- 172.21.15.40:0/437654165 >> v2:172.21.15.184:3300/0 conn(0x7f3b0015f4c0 0x7f3b00163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afed9d640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3b000d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934375181 0 0) 0x7f3b0006f2b0 con 0x7f3b0006d910 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aec003520 con 0x7f3b0006d910 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4176672819 0 0) 0x7f3aec003520 con 0x7f3b0006d910 2024-04-26T21:16:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 msgr2=0x7f3b000d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3b000d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 >> v2:172.21.15.184:3300/0 conn(0x7f3b0015f4c0 msgr2=0x7f3b00163a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 --2- 172.21.15.40:0/437654165 >> v2:172.21.15.184:3300/0 conn(0x7f3b0015f4c0 0x7f3b00163a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00162fa0 con 0x7f3b0006d910 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3ae8001670 con 0x7f3b0006d910 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 >> v1:172.21.15.40:6789/0 conn(0x7f3b0006d910 legacy=0x7f3b00072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.340+0000 7f3afdd9b640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aec0039b0 0x7f3aec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afdd9b640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3aec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afdd9b640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aec004d40 0x7f3aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00068300 con 0x7f3aec0039b0 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0006f060 con 0x7f3aec004d40 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec00b9f0 con 0x7f3b00073010 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aec0039b0 0x7f3aec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3af7fff640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3aec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afed9d640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aec004d40 0x7f3aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 -- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 msgr2=0x7f3aec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b00073010 0x7f3aec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 -- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aec004d40 msgr2=0x7f3aec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aec004d40 0x7f3aec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.341+0000 7f3afe59c640 1 -- 172.21.15.40:0/437654165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec003520 con 0x7f3aec0039b0 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3afe59c640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aec0039b0 0x7f3aec003dd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009ac0 tx=0x7f3aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6103f02a8f0ad78b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0036070 con 0x7f3aec0039b0 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af0031400 con 0x7f3aec0039b0 2024-04-26T21:16:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3afdd9b640 1 -- 172.21.15.40:0/437654165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0009690 con 0x7f3aec0039b0 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3b04a99640 1 -- 172.21.15.40:0/437654165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aec0039b0 msgr2=0x7f3aec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3b04a99640 1 --2- 172.21.15.40:0/437654165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aec0039b0 0x7f3aec003dd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009ac0 tx=0x7f3aec005830 comp rx=0 tx=0).stop 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3afed9d640 1 -- 172.21.15.40:0/437654165 reap_dead start 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3b04a99640 1 -- 172.21.15.40:0/437654165 shutdown_connections 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.342+0000 7f3b04a99640 1 -- 172.21.15.40:0/437654165 >> 172.21.15.40:0/437654165 conn(0x7f3b000ce140 msgr2=0x7f3b000ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.343+0000 7f3b04a99640 1 -- 172.21.15.40:0/437654165 shutdown_connections 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.343+0000 7f3b04a99640 1 -- 172.21.15.40:0/437654165 wait complete. 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.344+0000 7f3b04a99640 1 Processor -- start 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.344+0000 7f3b04a99640 1 -- start start 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3aec0039b0 0x7f3b00152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00155a50 0x7f3b0015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0006f590 con 0x7f3b000b6580 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00163560 con 0x7f3b00155a50 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3b04a99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006e960 con 0x7f3aec0039b0 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41102/0 (socket says 172.21.15.40:41102) 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 -- 172.21.15.40:0/3862829190 learned_addr learned my addr 172.21.15.40:0/3862829190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3afe59c640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3aec0039b0 0x7f3b00152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3afed9d640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00155a50 0x7f3b0015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 -- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3aec0039b0 msgr2=0x7f3b00152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3aec0039b0 0x7f3b00152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 -- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00155a50 msgr2=0x7f3b0015c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00155a50 0x7f3b0015c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 -- 172.21.15.40:0/3862829190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00153480 con 0x7f3b000b6580 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af7fff640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80072f0 tx=0x7f3ae80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.345+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae801a070 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.346+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae800f960 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.346+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b00155360 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.346+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800fd10 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.346+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001545a0 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.347+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000b0b00 con 0x7f3b000b6580 2024-04-26T21:16:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.347+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ae801d070 con 0x7f3b000b6580 2024-04-26T21:16:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.347+0000 7f3af6ffd640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 0x7f3ae006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.348+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae808b5e0 con 0x7f3b000b6580 2024-04-26T21:16:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.348+0000 7f3afe59c640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 0x7f3ae006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.348+0000 7f3afe59c640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 0x7f3ae006ede0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3af0033000 tx=0x7f3af0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.352+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b000b0b00 con 0x7f3b000b6580 2024-04-26T21:16:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.579+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3b0006a290 con 0x7f3b000b6580 2024-04-26T21:16:13.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.580+0000 7f3af6ffd640 1 -- 172.21.15.40:0/3862829190 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3ae8059a10 con 0x7f3b000b6580 2024-04-26T21:16:13.581 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:13.581 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.581+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 msgr2=0x7f3ae006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.581+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 0x7f3ae006ede0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3af0033000 tx=0x7f3af0008030 comp rx=0 tx=0).stop 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 msgr2=0x7f3b0015be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80072f0 tx=0x7f3ae80077b0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 shutdown_connections 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ae006c920 0x7f3ae006ede0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b00155a50 0x7f3b0015c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b000b6580 0x7f3b0015be60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 --2- 172.21.15.40:0/3862829190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3aec0039b0 0x7f3b00152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 >> 172.21.15.40:0/3862829190 conn(0x7f3b000ce140 msgr2=0x7f3b0006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 shutdown_connections 2024-04-26T21:16:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.582+0000 7f3b04a99640 1 -- 172.21.15.40:0/3862829190 wait complete. 2024-04-26T21:16:13.593 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:16:13.593 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-26T21:16:13.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:13.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.691+0000 7fe38330a640 1 Processor -- start 2024-04-26T21:16:13.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.692+0000 7fe38330a640 1 -- start start 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.692+0000 7fe38330a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe37c15f6a0 0x7fe37c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.692+0000 7fe38330a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c15fce0 con 0x7fe37c0b5530 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.692+0000 7fe38330a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c15ffc0 con 0x7fe37c09ba40 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.692+0000 7fe38330a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c0b59c0 con 0x7fe37c15f6a0 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3818a7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe37c0b5530 0x7fe37c0d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59120/0 (socket says 172.21.15.40:59120) 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3818a7640 1 -- 172.21.15.40:0/3831014133 learned_addr learned my addr 172.21.15.40:0/3831014133 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3808a5640 1 --2- 172.21.15.40:0/3831014133 >> v2:172.21.15.184:3300/0 conn(0x7fe37c15f6a0 0x7fe37c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883786730 0 0) 0x7fe37c15fce0 con 0x7fe37c0b5530 2024-04-26T21:16:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe364003520 con 0x7fe37c0b5530 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 750223523 0 0) 0x7fe364003520 con 0x7fe37c0b5530 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 >> v2:172.21.15.184:3300/0 conn(0x7fe37c15f6a0 msgr2=0x7fe37c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 --2- 172.21.15.40:0/3831014133 >> v2:172.21.15.184:3300/0 conn(0x7fe37c15f6a0 0x7fe37c070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 >> v1:172.21.15.184:6789/0 conn(0x7fe37c09ba40 legacy=0x7fe37c09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1602a0 con 0x7fe37c0b5530 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe36c0019f0 con 0x7fe37c0b5530 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 >> v1:172.21.15.40:6789/0 conn(0x7fe37c0b5530 legacy=0x7fe37c0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3640039b0 0x7fe364003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3640043b0 0x7fe364004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 0x7fe36400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0b59c0 con 0x7fe364005080 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe36400bfe0 con 0x7fe3640043b0 2024-04-26T21:16:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe36400c1e0 con 0x7fe3640039b0 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3818a7640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 0x7fe36400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3808a5640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3640043b0 0x7fe364004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.693+0000 7fe3810a6640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3640039b0 0x7fe364003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 -- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3640039b0 msgr2=0x7fe364003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3640039b0 0x7fe364003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 -- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3640043b0 msgr2=0x7fe364004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3640043b0 0x7fe364004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 -- 172.21.15.40:0/3831014133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe364003520 con 0x7fe364005080 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3810a6640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3640039b0 0x7fe364003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 -- 172.21.15.40:0/3831014133 reap_dead start 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe3818a7640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 0x7fe36400b9c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00a920 tx=0x7fe364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d331af30f9062049 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00a4c0 con 0x7fe364005080 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c02f070 con 0x7fe364005080 2024-04-26T21:16:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.694+0000 7fe37bfff640 1 -- 172.21.15.40:0/3831014133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c02bce0 con 0x7fe364005080 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 -- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 msgr2=0x7fe36400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 0x7fe36400b9c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00a920 tx=0x7fe364005830 comp rx=0 tx=0).stop 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 -- 172.21.15.40:0/3831014133 shutdown_connections 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 --2- 172.21.15.40:0/3831014133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe364005080 0x7fe36400b9c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 -- 172.21.15.40:0/3831014133 >> 172.21.15.40:0/3831014133 conn(0x7fe37c0b4a10 msgr2=0x7fe37c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 -- 172.21.15.40:0/3831014133 shutdown_connections 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.695+0000 7fe38330a640 1 -- 172.21.15.40:0/3831014133 wait complete. 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.696+0000 7fe38330a640 1 Processor -- start 2024-04-26T21:16:13.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.696+0000 7fe38330a640 1 -- start start 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 0x7fe37c1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 0x7fe37c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c070900 con 0x7fe37c0b1110 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c070780 con 0x7fe37c0ce4d0 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe38330a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c160610 con 0x7fe364005080 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41128/0 (socket says 172.21.15.40:41128) 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 -- 172.21.15.40:0/2645543121 learned_addr learned my addr 172.21.15.40:0/2645543121 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3810a6640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 0x7fe37c1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3818a7640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 0x7fe37c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 -- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 msgr2=0x7fe37c1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 0x7fe37c1534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 -- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 msgr2=0x7fe37c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 0x7fe37c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3808a5640 1 -- 172.21.15.40:0/2645543121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0c6b40 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3810a6640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 0x7fe37c1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.697+0000 7fe3818a7640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 0x7fe37c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe3808a5640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe370011940 tx=0x7fe370011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fa90 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37000fc30 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c0c6e20 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370014440 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.698+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c0c73e0 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.699+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c06a290 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.700+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe370012040 con 0x7fe37c0b1110 2024-04-26T21:16:13.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.700+0000 7fe379ffb640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 0x7fe36806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:13.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.700+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe37008c020 con 0x7fe37c0b1110 2024-04-26T21:16:13.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.700+0000 7fe3810a6640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 0x7fe36806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:13.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.701+0000 7fe3810a6640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 0x7fe36806ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe374005580 tx=0x7fe374009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.706+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe370019030 con 0x7fe37c0b1110 2024-04-26T21:16:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:13.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.930+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe37c06db60 con 0x7fe37c0b1110 2024-04-26T21:16:13.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.930+0000 7fe379ffb640 1 -- 172.21.15.40:0/2645543121 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fe37005a4e0 con 0x7fe37c0b1110 2024-04-26T21:16:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:13.931 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 msgr2=0x7fe36806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 0x7fe36806ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe374005580 tx=0x7fe374009040 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 msgr2=0x7fe37c0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe370011940 tx=0x7fe370011e00 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 shutdown_connections 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe36806c920 0x7fe36806ede0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe37c0ce4d0 0x7fe37c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b1110 0x7fe37c0b1590 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 --2- 172.21.15.40:0/2645543121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe364005080 0x7fe37c1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 >> 172.21.15.40:0/2645543121 conn(0x7fe37c0b4a10 msgr2=0x7fe37c0d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 shutdown_connections 2024-04-26T21:16:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:13.932+0000 7fe38330a640 1 -- 172.21.15.40:0/2645543121 wait complete. 2024-04-26T21:16:13.944 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:13.944 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:15.788 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:17.790 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:18.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.057+0000 7f83e6fda640 1 Processor -- start 2024-04-26T21:16:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.057+0000 7f83e6fda640 1 -- start start 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.057+0000 7f83e6fda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83e0145870 0x7f83e006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83e6fda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83e006f980 0x7f83e0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83e6fda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e0073a40 con 0x7f83e0073250 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83e6fda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f83e006fde0 con 0x7f83e006f980 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83e6fda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f83e015d8d0 con 0x7f83e0145870 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83e006f980 0x7f83e0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83e006f980 0x7f83e0072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41134/0 (socket says 172.21.15.40:41134) 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 -- 172.21.15.40:0/2073193482 learned_addr learned my addr 172.21.15.40:0/2073193482 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83e4d76640 1 --2- 172.21.15.40:0/2073193482 >> v2:172.21.15.184:3300/0 conn(0x7f83e0145870 0x7f83e006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 -- 172.21.15.40:0/2073193482 >> v2:172.21.15.184:3300/0 conn(0x7f83e0145870 msgr2=0x7f83e006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 --2- 172.21.15.40:0/2073193482 >> v2:172.21.15.184:3300/0 conn(0x7f83e0145870 0x7f83e006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 -- 172.21.15.40:0/2073193482 >> v1:172.21.15.184:6789/0 conn(0x7f83e0073250 legacy=0x7f83e015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.058+0000 7f83dffff640 1 -- 172.21.15.40:0/2073193482 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e015dbb0 con 0x7f83e006f980 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.059+0000 7f83dffff640 1 --2- 172.21.15.40:0/2073193482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e0072d10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f83e00cf1a0 tx=0x7f83cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac655889b1cf371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.059+0000 7f83df7fe640 1 -- 172.21.15.40:0/2073193482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc037070 con 0x7f83e006f980 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.059+0000 7f83df7fe640 1 -- 172.21.15.40:0/2073193482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc032440 con 0x7f83e006f980 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.059+0000 7f83df7fe640 1 -- 172.21.15.40:0/2073193482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032810 con 0x7f83e006f980 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 -- 172.21.15.40:0/2073193482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 msgr2=0x7f83e0072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 --2- 172.21.15.40:0/2073193482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e0072d10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f83e00cf1a0 tx=0x7f83cc02a230 comp rx=0 tx=0).stop 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 -- 172.21.15.40:0/2073193482 shutdown_connections 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 --2- 172.21.15.40:0/2073193482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e0072d10 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 --2- 172.21.15.40:0/2073193482 >> v2:172.21.15.184:3300/0 conn(0x7f83e0145870 0x7f83e006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 -- 172.21.15.40:0/2073193482 >> 172.21.15.40:0/2073193482 conn(0x7f83e00cd830 msgr2=0x7f83e00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 -- 172.21.15.40:0/2073193482 shutdown_connections 2024-04-26T21:16:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.060+0000 7f83e6fda640 1 -- 172.21.15.40:0/2073193482 wait complete. 2024-04-26T21:16:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.061+0000 7f83e6fda640 1 Processor -- start 2024-04-26T21:16:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.061+0000 7f83e6fda640 1 -- start start 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 0x7f83e015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 0x7f83e013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0145c50 con 0x7f83e006f980 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83e015d5d0 con 0x7f83e0145870 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e6fda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83e0073690 con 0x7f83e0073250 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41144/0 (socket says 172.21.15.40:41144) 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 -- 172.21.15.40:0/1499766944 learned_addr learned my addr 172.21.15.40:0/1499766944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e5577640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 0x7f83e013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83dffff640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 0x7f83e015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 -- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 msgr2=0x7f83e015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 0x7f83e015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 -- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 msgr2=0x7f83e013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 0x7f83e013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e4d76640 1 -- 172.21.15.40:0/1499766944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e01403d0 con 0x7f83e006f980 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83dffff640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 0x7f83e015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.062+0000 7f83e5577640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 0x7f83e013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83e4d76640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f83d400d800 tx=0x7f83d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d400b6c0 con 0x7f83e006f980 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d4019070 con 0x7f83e006f980 2024-04-26T21:16:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d400bac0 con 0x7f83e006f980 2024-04-26T21:16:19.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e01f84c0 con 0x7f83e006f980 2024-04-26T21:16:19.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.063+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01f8a80 con 0x7f83e006f980 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.064+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e006a290 con 0x7f83e006f980 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.064+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83d400bc60 con 0x7f83e006f980 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.065+0000 7f83dd7fa640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.065+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f83d408bfc0 con 0x7f83e006f980 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.065+0000 7f83dffff640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.065+0000 7f83dffff640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f83cc034000 tx=0x7f83cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:19.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.069+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83e01403d0 con 0x7f83e006f980 2024-04-26T21:16:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.293+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f83e006d2b0 con 0x7f83e006f980 2024-04-26T21:16:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.293+0000 7f83dd7fa640 1 -- 172.21.15.40:0/1499766944 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f83d405a480 con 0x7f83e006f980 2024-04-26T21:16:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:19.294 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 msgr2=0x7f83c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f83cc034000 tx=0x7f83cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 msgr2=0x7f83e00b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f83d400d800 tx=0x7f83d400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 shutdown_connections 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e0145870 0x7f83e013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e0073250 0x7f83e015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 --2- 172.21.15.40:0/1499766944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e006f980 0x7f83e00b2180 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 >> 172.21.15.40:0/1499766944 conn(0x7f83e00cd830 msgr2=0x7f83e00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 shutdown_connections 2024-04-26T21:16:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:19.295+0000 7f83e6fda640 1 -- 172.21.15.40:0/1499766944 wait complete. 2024-04-26T21:16:19.307 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:19.307 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:19.792 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:21.794 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:24.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.420+0000 7fbecf093640 1 Processor -- start 2024-04-26T21:16:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.420+0000 7fbecf093640 1 -- start start 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbec815b6f0 0x7fbec806f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec806e330 0x7fbec8072930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbec8072e70 0x7fbec815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbec80b6bc0 con 0x7fbec8072e70 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbec815bad0 con 0x7fbec815b6f0 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecf093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbec80cefb0 con 0x7fbec806e330 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbec8072e70 0x7fbec815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbec8072e70 0x7fbec815f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59450/0 (socket says 172.21.15.40:59450) 2024-04-26T21:16:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 -- 172.21.15.40:0/2538942558 learned_addr learned my addr 172.21.15.40:0/2538942558 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbec7fff640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec806e330 0x7fbec8072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecce2f640 1 --2- 172.21.15.40:0/2538942558 >> v2:172.21.15.184:3300/0 conn(0x7fbec815b6f0 0x7fbec806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 -- 172.21.15.40:0/2538942558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec806e330 msgr2=0x7fbec8072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec806e330 0x7fbec8072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 -- 172.21.15.40:0/2538942558 >> v2:172.21.15.184:3300/0 conn(0x7fbec815b6f0 msgr2=0x7fbec806f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 --2- 172.21.15.40:0/2538942558 >> v2:172.21.15.184:3300/0 conn(0x7fbec815b6f0 0x7fbec806f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.421+0000 7fbecd630640 1 -- 172.21.15.40:0/2538942558 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec806f8e0 con 0x7fbec8072e70 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.422+0000 7fbecd630640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec8072e70 0x7fbec815f210 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbec80d1720 tx=0x7fbeb802c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4086c7c3a2c247 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.422+0000 7fbec77fe640 1 -- 172.21.15.40:0/2538942558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8038070 con 0x7fbec8072e70 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.422+0000 7fbec77fe640 1 -- 172.21.15.40:0/2538942558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb8031710 con 0x7fbec8072e70 2024-04-26T21:16:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.422+0000 7fbec77fe640 1 -- 172.21.15.40:0/2538942558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8031ae0 con 0x7fbec8072e70 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 -- 172.21.15.40:0/2538942558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec8072e70 msgr2=0x7fbec815f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec8072e70 0x7fbec815f210 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbec80d1720 tx=0x7fbeb802c420 comp rx=0 tx=0).stop 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 -- 172.21.15.40:0/2538942558 shutdown_connections 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec8072e70 0x7fbec815f210 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 --2- 172.21.15.40:0/2538942558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec806e330 0x7fbec8072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 --2- 172.21.15.40:0/2538942558 >> v2:172.21.15.184:3300/0 conn(0x7fbec815b6f0 0x7fbec806f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 -- 172.21.15.40:0/2538942558 >> 172.21.15.40:0/2538942558 conn(0x7fbec80cd830 msgr2=0x7fbec80cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 -- 172.21.15.40:0/2538942558 shutdown_connections 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.423+0000 7fbecf093640 1 -- 172.21.15.40:0/2538942558 wait complete. 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.424+0000 7fbecf093640 1 Processor -- start 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.424+0000 7fbecf093640 1 -- start start 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 0x7fbec8158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbec815b6f0 0x7fbec8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbec80cf550 con 0x7fbec806e330 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbec80cf270 con 0x7fbec815b6f0 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecf093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbec80b4020 con 0x7fbec8072e70 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59462/0 (socket says 172.21.15.40:59462) 2024-04-26T21:16:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 -- 172.21.15.40:0/511513447 learned_addr learned my addr 172.21.15.40:0/511513447 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecd630640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbec815b6f0 0x7fbec8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbec7fff640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 0x7fbec8158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 -- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 msgr2=0x7fbec8158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 0x7fbec8158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 -- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbec815b6f0 msgr2=0x7fbec8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbec815b6f0 0x7fbec8159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbecce2f640 1 -- 172.21.15.40:0/511513447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec8156ba0 con 0x7fbec806e330 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.425+0000 7fbec7fff640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 0x7fbec8158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbecce2f640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbec0007e40 tx=0x7fbec000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0014620 con 0x7fbec806e330 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec0019070 con 0x7fbec806e330 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0014a00 con 0x7fbec806e330 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec8156e40 con 0x7fbec806e330 2024-04-26T21:16:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.426+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec8157400 con 0x7fbec806e330 2024-04-26T21:16:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.427+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec806a290 con 0x7fbec806e330 2024-04-26T21:16:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.427+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbec0014ba0 con 0x7fbec806e330 2024-04-26T21:16:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.428+0000 7fbec57fa640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 0x7fbeb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.428+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbec008c370 con 0x7fbec806e330 2024-04-26T21:16:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.428+0000 7fbec7fff640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 0x7fbeb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.429+0000 7fbec7fff640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 0x7fbeb406ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b2d30 tx=0x7fbebc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:24.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.432+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbec005a710 con 0x7fbec806e330 2024-04-26T21:16:24.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.660+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbec8073ac0 con 0x7fbec806e330 2024-04-26T21:16:24.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.660+0000 7fbec57fa640 1 -- 172.21.15.40:0/511513447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fbec005a010 con 0x7fbec806e330 2024-04-26T21:16:24.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:24.661 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:24.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 msgr2=0x7fbeb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:24.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 0x7fbeb406ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b2d30 tx=0x7fbebc008040 comp rx=0 tx=0).stop 2024-04-26T21:16:24.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 msgr2=0x7fbec814a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbec0007e40 tx=0x7fbec000e4b0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 shutdown_connections 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbeb406c920 0x7fbeb406ede0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbec815b6f0 0x7fbec8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbec8072e70 0x7fbec8158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 --2- 172.21.15.40:0/511513447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbec806e330 0x7fbec814a930 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 >> 172.21.15.40:0/511513447 conn(0x7fbec80cd830 msgr2=0x7fbec80b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 shutdown_connections 2024-04-26T21:16:24.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:24.662+0000 7fbecf093640 1 -- 172.21.15.40:0/511513447 wait complete. 2024-04-26T21:16:24.674 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:24.674 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:25.798 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:27.801 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:29.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:29.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.787+0000 7f6147493640 1 Processor -- start 2024-04-26T21:16:29.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.788+0000 7f6147493640 1 -- start start 2024-04-26T21:16:29.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.788+0000 7f6147493640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f614015f4d0 0x7f61400b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:29.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.788+0000 7f6147493640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61400b2550 con 0x7f61400d5810 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.788+0000 7f6147493640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61400b2830 con 0x7f61400d2e20 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.788+0000 7f6147493640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f61400b6180 con 0x7f614015f4d0 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f6145a30640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61400d5810 0x7f61400b35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40736/0 (socket says 172.21.15.40:40736) 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f6145a30640 1 -- 172.21.15.40:0/160005081 learned_addr learned my addr 172.21.15.40:0/160005081 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f614522f640 1 --2- 172.21.15.40:0/160005081 >> v2:172.21.15.184:3300/0 conn(0x7f614015f4d0 0x7f61400b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3183489057 0 0) 0x7f61400b2550 con 0x7f61400d5810 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6128003520 con 0x7f61400d5810 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803832309 0 0) 0x7f61400b2830 con 0x7f61400d2e20 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61400b2550 con 0x7f61400d2e20 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1397290213 0 0) 0x7f6128003520 con 0x7f61400d5810 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f614522f640 1 -- 172.21.15.40:0/160005081 >> v1:172.21.15.184:6789/0 conn(0x7f61400d2e20 legacy=0x7f61400d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f614522f640 1 -- 172.21.15.40:0/160005081 >> v1:172.21.15.40:6789/0 conn(0x7f61400d5810 legacy=0x7f61400b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:29.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.789+0000 7f614522f640 1 -- 172.21.15.40:0/160005081 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61400b2b10 con 0x7f614015f4d0 2024-04-26T21:16:29.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.790+0000 7f614522f640 1 --2- 172.21.15.40:0/160005081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614015f4d0 0x7f61400b1ee0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61380097a0 tx=0x7f613802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f03b561dce602fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:29.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.790+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138037070 con 0x7f614015f4d0 2024-04-26T21:16:29.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.790+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613802ab30 con 0x7f614015f4d0 2024-04-26T21:16:29.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.790+0000 7f613ffff640 1 -- 172.21.15.40:0/160005081 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300019f0 con 0x7f614015f4d0 2024-04-26T21:16:29.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 -- 172.21.15.40:0/160005081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614015f4d0 msgr2=0x7f61400b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 --2- 172.21.15.40:0/160005081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614015f4d0 0x7f61400b1ee0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61380097a0 tx=0x7f613802a0a0 comp rx=0 tx=0).stop 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 -- 172.21.15.40:0/160005081 shutdown_connections 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 --2- 172.21.15.40:0/160005081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f614015f4d0 0x7f61400b1ee0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 -- 172.21.15.40:0/160005081 >> 172.21.15.40:0/160005081 conn(0x7f614008ebf0 msgr2=0x7f614009e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 -- 172.21.15.40:0/160005081 shutdown_connections 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.791+0000 7f6147493640 1 -- 172.21.15.40:0/160005081 wait complete. 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.792+0000 7f6147493640 1 Processor -- start 2024-04-26T21:16:29.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.792+0000 7f6147493640 1 -- start start 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61400d2e20 0x7f61401f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614015f4d0 0x7f6140161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61400b3d00 con 0x7f61400d5810 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61400b30d0 con 0x7f61400d2e20 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6147493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61400b3e80 con 0x7f614015f4d0 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59474/0 (socket says 172.21.15.40:59474) 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 -- 172.21.15.40:0/2107539365 learned_addr learned my addr 172.21.15.40:0/2107539365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6145a30640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614015f4d0 0x7f6140161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:29.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f614522f640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61400d2e20 0x7f61401f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 -- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614015f4d0 msgr2=0x7f6140161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614015f4d0 0x7f6140161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 -- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61400d2e20 msgr2=0x7f61401f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61400d2e20 0x7f61401f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 -- 172.21.15.40:0/2107539365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61400ce0c0 con 0x7f61400d5810 2024-04-26T21:16:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.793+0000 7f6144a2e640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f613400c490 tx=0x7f613400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.794+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340052d0 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.794+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6134005470 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.794+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61400bad40 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.794+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134012870 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.794+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61401f58d0 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.795+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614006a290 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.795+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6134012a10 con 0x7f61400d5810 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.796+0000 7f613dffb640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:29.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.796+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f613408b7f0 con 0x7f61400d5810 2024-04-26T21:16:29.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.796+0000 7f614522f640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:29.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.797+0000 7f614522f640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6138034000 tx=0x7f613802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:29.801+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6134059c10 con 0x7f61400d5810 2024-04-26T21:16:29.802 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:30.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.026+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f61400d0e60 con 0x7f61400d5810 2024-04-26T21:16:30.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.027+0000 7f613dffb640 1 -- 172.21.15.40:0/2107539365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6134059510 con 0x7f61400d5810 2024-04-26T21:16:30.028 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:30.028 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 msgr2=0x7f612c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 0x7f612c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6138034000 tx=0x7f613802e000 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 msgr2=0x7f61400cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f613400c490 tx=0x7f613400c950 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 shutdown_connections 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f612c06c8f0 0x7f612c06edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f614015f4d0 0x7f6140161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61400d5810 0x7f61400cdb80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 --2- 172.21.15.40:0/2107539365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61400d2e20 0x7f61401f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 >> 172.21.15.40:0/2107539365 conn(0x7f614008ebf0 msgr2=0x7f614006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 shutdown_connections 2024-04-26T21:16:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:30.029+0000 7f6147493640 1 -- 172.21.15.40:0/2107539365 wait complete. 2024-04-26T21:16:30.041 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:30.041 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:31.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:16:31.792 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:16:31.804 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:35.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.154+0000 7fa37de64640 1 Processor -- start 2024-04-26T21:16:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.154+0000 7fa37de64640 1 -- start start 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa37de64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 0x7fa378072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa37de64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa378073250 0x7fa37815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa37de64640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa378073a40 con 0x7fa378145870 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa37de64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa37815d8d0 con 0x7fa378073250 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa37de64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa37815dbb0 con 0x7fa37806fb50 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa377fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa378073250 0x7fa37815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa377fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa378073250 0x7fa37815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53284/0 (socket says 172.21.15.40:53284) 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.155+0000 7fa377fff640 1 -- 172.21.15.40:0/4016872481 learned_addr learned my addr 172.21.15.40:0/4016872481 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa376ffd640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 0x7fa378072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa377fff640 1 -- 172.21.15.40:0/4016872481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 msgr2=0x7fa378072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa377fff640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 0x7fa378072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa377fff640 1 -- 172.21.15.40:0/4016872481 >> v1:172.21.15.184:6789/0 conn(0x7fa378145870 legacy=0x7fa37806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa377fff640 1 -- 172.21.15.40:0/4016872481 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37815de90 con 0x7fa378073250 2024-04-26T21:16:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa376ffd640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 0x7fa378072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.156+0000 7fa377fff640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa378073250 0x7fa37815e3b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa370009e50 tx=0x7fa37002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574cccaa88efc8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa3767fc640 1 -- 172.21.15.40:0/4016872481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370036070 con 0x7fa378073250 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa3767fc640 1 -- 172.21.15.40:0/4016872481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa370031600 con 0x7fa378073250 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa3767fc640 1 -- 172.21.15.40:0/4016872481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700319d0 con 0x7fa378073250 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 -- 172.21.15.40:0/4016872481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa378073250 msgr2=0x7fa37815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa378073250 0x7fa37815e3b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa370009e50 tx=0x7fa37002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 -- 172.21.15.40:0/4016872481 shutdown_connections 2024-04-26T21:16:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa378073250 0x7fa37815e3b0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 --2- 172.21.15.40:0/4016872481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37806fb50 0x7fa378072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 -- 172.21.15.40:0/4016872481 >> 172.21.15.40:0/4016872481 conn(0x7fa3780cd830 msgr2=0x7fa3780cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 -- 172.21.15.40:0/4016872481 shutdown_connections 2024-04-26T21:16:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.157+0000 7fa37de64640 1 -- 172.21.15.40:0/4016872481 wait complete. 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.158+0000 7fa37de64640 1 Processor -- start 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 -- start start 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa378073250 0x7fa37813f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa378145870 0x7fa378144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780736e0 con 0x7fa37806fb50 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa378145c50 con 0x7fa378073250 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.159+0000 7fa37de64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa37815e8f0 con 0x7fa378145870 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53288/0 (socket says 172.21.15.40:53288) 2024-04-26T21:16:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 -- 172.21.15.40:0/461838701 learned_addr learned my addr 172.21.15.40:0/461838701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa377fff640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa378145870 0x7fa378144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa376ffd640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa378073250 0x7fa37813f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 -- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa378145870 msgr2=0x7fa378144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa378145870 0x7fa378144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 -- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa378073250 msgr2=0x7fa37813f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa378073250 0x7fa37813f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 -- 172.21.15.40:0/461838701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa378137c50 con 0x7fa37806fb50 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa3777fe640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa36800b3f0 tx=0x7fa36800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368015020 con 0x7fa37806fb50 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368011830 con 0x7fa37806fb50 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37813d770 con 0x7fa37806fb50 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368011c20 con 0x7fa37806fb50 2024-04-26T21:16:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.160+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37813be20 con 0x7fa37806fb50 2024-04-26T21:16:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.161+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37806a290 con 0x7fa37806fb50 2024-04-26T21:16:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.162+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa368018020 con 0x7fa37806fb50 2024-04-26T21:16:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.162+0000 7fa35ffff640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.163+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa36808b5c0 con 0x7fa37806fb50 2024-04-26T21:16:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.163+0000 7fa376ffd640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.163+0000 7fa376ffd640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3781402f0 tx=0x7fa36c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.167+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa36801f030 con 0x7fa37806fb50 2024-04-26T21:16:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.393+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa378153030 con 0x7fa37806fb50 2024-04-26T21:16:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.393+0000 7fa35ffff640 1 -- 172.21.15.40:0/461838701 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fa37813d770 con 0x7fa37806fb50 2024-04-26T21:16:35.394 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:35.394 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 msgr2=0x7fa36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3781402f0 tx=0x7fa36c008040 comp rx=0 tx=0).stop 2024-04-26T21:16:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 msgr2=0x7fa3780b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa36800b3f0 tx=0x7fa36800b8b0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 shutdown_connections 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa378145870 0x7fa378144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa378073250 0x7fa37813f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 --2- 172.21.15.40:0/461838701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37806fb50 0x7fa3780b4880 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 >> 172.21.15.40:0/461838701 conn(0x7fa3780cd830 msgr2=0x7fa3780ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 shutdown_connections 2024-04-26T21:16:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:35.395+0000 7fa37de64640 1 -- 172.21.15.40:0/461838701 wait complete. 2024-04-26T21:16:35.407 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:35.407 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:35.809 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:40.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.519+0000 7ff972b89640 1 Processor -- start 2024-04-26T21:16:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.519+0000 7ff972b89640 1 -- start start 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 0x7ff96c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff96c0b6790 0x7ff96c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 0x7ff96c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0b56f0 con 0x7ff96c0b6790 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0b59d0 con 0x7ff96c06d1f0 2024-04-26T21:16:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff972b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0b5cb0 con 0x7ff96c15d050 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff96c0b6790 0x7ff96c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff96c0b6790 0x7ff96c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53300/0 (socket says 172.21.15.40:53300) 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 -- 172.21.15.40:0/2515734277 learned_addr learned my addr 172.21.15.40:0/2515734277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff971126640 1 --2- 172.21.15.40:0/2515734277 >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 0x7ff96c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff970925640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 0x7ff96c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 -- 172.21.15.40:0/2515734277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 msgr2=0x7ff96c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 0x7ff96c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 -- 172.21.15.40:0/2515734277 >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 msgr2=0x7ff96c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 --2- 172.21.15.40:0/2515734277 >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 0x7ff96c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff96bfff640 1 -- 172.21.15.40:0/2515734277 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0b5f90 con 0x7ff96c0b6790 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff971126640 1 --2- 172.21.15.40:0/2515734277 >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 0x7ff96c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.520+0000 7ff970925640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 0x7ff96c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.521+0000 7ff96bfff640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c0b6790 0x7ff96c072900 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b4fe0 tx=0x7ff95802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4251a25cf1af8edc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.521+0000 7ff96b7fe640 1 -- 172.21.15.40:0/2515734277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958037070 con 0x7ff96c0b6790 2024-04-26T21:16:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.521+0000 7ff96b7fe640 1 -- 172.21.15.40:0/2515734277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff958032440 con 0x7ff96c0b6790 2024-04-26T21:16:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.521+0000 7ff96b7fe640 1 -- 172.21.15.40:0/2515734277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958032810 con 0x7ff96c0b6790 2024-04-26T21:16:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 -- 172.21.15.40:0/2515734277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c0b6790 msgr2=0x7ff96c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c0b6790 0x7ff96c072900 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b4fe0 tx=0x7ff95802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 -- 172.21.15.40:0/2515734277 shutdown_connections 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 --2- 172.21.15.40:0/2515734277 >> v2:172.21.15.184:3300/0 conn(0x7ff96c06d1f0 0x7ff96c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c0b6790 0x7ff96c072900 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 --2- 172.21.15.40:0/2515734277 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c15d050 0x7ff96c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 -- 172.21.15.40:0/2515734277 >> 172.21.15.40:0/2515734277 conn(0x7ff96c0ce140 msgr2=0x7ff96c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 -- 172.21.15.40:0/2515734277 shutdown_connections 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.522+0000 7ff972b89640 1 -- 172.21.15.40:0/2515734277 wait complete. 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.523+0000 7ff972b89640 1 Processor -- start 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.523+0000 7ff972b89640 1 -- start start 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c0b6790 0x7ff96c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff96c15d050 0x7ff96c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15d430 con 0x7ff96c06d1f0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c068320 con 0x7ff96c15d050 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff972b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c160c20 con 0x7ff96c0b6790 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53314/0 (socket says 172.21.15.40:53314) 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 -- 172.21.15.40:0/4081224178 learned_addr learned my addr 172.21.15.40:0/4081224178 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 -- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c0b6790 msgr2=0x7ff96c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c0b6790 0x7ff96c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 -- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff96c15d050 msgr2=0x7ff96c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff96c15d050 0x7ff96c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.524+0000 7ff970925640 1 -- 172.21.15.40:0/4081224178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0af810 con 0x7ff96c06d1f0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff970925640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c820 tx=0x7ff96000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960015070 con 0x7ff96c06d1f0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960007a30 con 0x7ff96c06d1f0 2024-04-26T21:16:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960007de0 con 0x7ff96c06d1f0 2024-04-26T21:16:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c0aea50 con 0x7ff96c06d1f0 2024-04-26T21:16:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.525+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c0af010 con 0x7ff96c06d1f0 2024-04-26T21:16:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.526+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c06a290 con 0x7ff96c06d1f0 2024-04-26T21:16:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.526+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff960018070 con 0x7ff96c06d1f0 2024-04-26T21:16:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.527+0000 7ff9697fa640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.527+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff96008ba00 con 0x7ff96c06d1f0 2024-04-26T21:16:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.527+0000 7ff96bfff640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:40.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.528+0000 7ff96bfff640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff958034000 tx=0x7ff95802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:40.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.531+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff960059e30 con 0x7ff96c06d1f0 2024-04-26T21:16:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.761+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff96c0737b0 con 0x7ff96c06d1f0 2024-04-26T21:16:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.761+0000 7ff9697fa640 1 -- 172.21.15.40:0/4081224178 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff960059730 con 0x7ff96c06d1f0 2024-04-26T21:16:40.762 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:40.763 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 msgr2=0x7ff95006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff958034000 tx=0x7ff95802c000 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 msgr2=0x7ff96c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c820 tx=0x7ff96000cce0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 shutdown_connections 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff96c15d050 0x7ff96c0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff96c0b6790 0x7ff96c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 --2- 172.21.15.40:0/4081224178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff96c06d1f0 0x7ff96c144db0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 >> 172.21.15.40:0/4081224178 conn(0x7ff96c0ce140 msgr2=0x7ff96c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.763+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 shutdown_connections 2024-04-26T21:16:40.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:40.764+0000 7ff972b89640 1 -- 172.21.15.40:0/4081224178 wait complete. 2024-04-26T21:16:40.775 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:40.775 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:41.815 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:43.817 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:45.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:45.818 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.888+0000 7f20c5bf0640 1 Processor -- start 2024-04-26T21:16:45.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 -- start start 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20c0145870 0x7f20c006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 0x7f20c0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c0073250 0x7f20c015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f20c006fde0 con 0x7f20c0145870 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f20c0073a40 con 0x7f20c006f980 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.889+0000 7f20c5bf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20c015d8d0 con 0x7f20c0073250 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20c0145870 0x7f20c006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20c0145870 0x7f20c006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52218/0 (socket says 172.21.15.40:52218) 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 -- 172.21.15.40:0/3458157311 learned_addr learned my addr 172.21.15.40:0/3458157311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20beffd640 1 --2- 172.21.15.40:0/3458157311 >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 0x7f20c0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bffff640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c0073250 0x7f20c015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 -- 172.21.15.40:0/3458157311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c0073250 msgr2=0x7f20c015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c0073250 0x7f20c015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 -- 172.21.15.40:0/3458157311 >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 msgr2=0x7f20c0072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3458157311 >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 0x7f20c0072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20bf7fe640 1 -- 172.21.15.40:0/3458157311 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c015dbb0 con 0x7f20c0145870 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.890+0000 7f20beffd640 1 --2- 172.21.15.40:0/3458157311 >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 0x7f20c0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0145870 0x7f20c006f440 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f20c0070530 tx=0x7f20b002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdf3d3b32def11c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20be7fc640 1 -- 172.21.15.40:0/3458157311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0032790 con 0x7f20c0145870 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20be7fc640 1 -- 172.21.15.40:0/3458157311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b0038070 con 0x7f20c0145870 2024-04-26T21:16:45.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20be7fc640 1 -- 172.21.15.40:0/3458157311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0032b50 con 0x7f20c0145870 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3458157311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0145870 msgr2=0x7f20c006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0145870 0x7f20c006f440 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f20c0070530 tx=0x7f20b002c890 comp rx=0 tx=0).stop 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3458157311 shutdown_connections 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c0073250 0x7f20c015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3458157311 >> v2:172.21.15.184:3300/0 conn(0x7f20c006f980 0x7f20c0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3458157311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0145870 0x7f20c006f440 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3458157311 >> 172.21.15.40:0/3458157311 conn(0x7f20c00cd830 msgr2=0x7f20c00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.891+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3458157311 shutdown_connections 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.892+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3458157311 wait complete. 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.893+0000 7f20c5bf0640 1 Processor -- start 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.893+0000 7f20c5bf0640 1 -- start start 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006f980 0x7f20c01f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c0145870 0x7f20c015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0145c50 con 0x7f20c0073250 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20c015d5d0 con 0x7f20c0145870 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00736e0 con 0x7f20c006f980 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52226/0 (socket says 172.21.15.40:52226) 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 -- 172.21.15.40:0/3962504378 learned_addr learned my addr 172.21.15.40:0/3962504378 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006f980 0x7f20c01f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20bffff640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c0145870 0x7f20c015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 -- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006f980 msgr2=0x7f20c01f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006f980 0x7f20c01f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 -- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c0145870 msgr2=0x7f20c015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c0145870 0x7f20c015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 -- 172.21.15.40:0/3962504378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c00b3940 con 0x7f20c0073250 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20beffd640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c7f0 tx=0x7f20b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4015070 con 0x7f20c0073250 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4007820 con 0x7f20c0073250 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4007bd0 con 0x7f20c0073250 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.894+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c0156f10 con 0x7f20c0073250 2024-04-26T21:16:45.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.895+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01571b0 con 0x7f20c0073250 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.896+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c006a290 con 0x7f20c0073250 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.896+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20b4018070 con 0x7f20c0073250 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.897+0000 7f20a7fff640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.897+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f20b408bb80 con 0x7f20c0073250 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.897+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.898+0000 7f20bf7fe640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 0x7f20ac06ede0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20b0037000 tx=0x7f20b0032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:45.901+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20b405a020 con 0x7f20c0073250 2024-04-26T21:16:46.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.126+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f20c00b70c0 con 0x7f20c0073250 2024-04-26T21:16:46.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.127+0000 7f20a7fff640 1 -- 172.21.15.40:0/3962504378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f20b4059920 con 0x7f20c0073250 2024-04-26T21:16:46.128 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:46.128 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 msgr2=0x7f20ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 0x7f20ac06ede0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20b0037000 tx=0x7f20b0032000 comp rx=0 tx=0).stop 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 msgr2=0x7f20c00b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c7f0 tx=0x7f20b400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 shutdown_connections 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f20ac06c920 0x7f20ac06ede0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c0145870 0x7f20c015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20c0073250 0x7f20c00b0ac0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 --2- 172.21.15.40:0/3962504378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006f980 0x7f20c01f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 >> 172.21.15.40:0/3962504378 conn(0x7f20c00cd830 msgr2=0x7f20c015a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 shutdown_connections 2024-04-26T21:16:46.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:46.129+0000 7f20c5bf0640 1 -- 172.21.15.40:0/3962504378 wait complete. 2024-04-26T21:16:46.140 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:46.141 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:51.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.253+0000 7f0e8a6a9640 1 Processor -- start 2024-04-26T21:16:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.253+0000 7f0e8a6a9640 1 -- start start 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e8a6a9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 0x7f0e8415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e8a6a9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e84070570 0x7f0e840b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e8a6a9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8415fce0 con 0x7f0e8409be20 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e8a6a9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0e84160610 con 0x7f0e84070570 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e8a6a9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0e840709b0 con 0x7f0e8409ba40 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e84070570 0x7f0e840b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e84070570 0x7f0e840b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33242/0 (socket says 172.21.15.40:33242) 2024-04-26T21:16:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 -- 172.21.15.40:0/3286706070 learned_addr learned my addr 172.21.15.40:0/3286706070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 -- 172.21.15.40:0/3286706070 >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 msgr2=0x7f0e8415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e83fff640 1 --2- 172.21.15.40:0/3286706070 >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 0x7f0e8415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 --2- 172.21.15.40:0/3286706070 >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 0x7f0e8415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 -- 172.21.15.40:0/3286706070 >> v1:172.21.15.184:6789/0 conn(0x7f0e8409be20 legacy=0x7f0e84162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e837fe640 1 -- 172.21.15.40:0/3286706070 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8415ffc0 con 0x7f0e84070570 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.254+0000 7f0e83fff640 1 --2- 172.21.15.40:0/3286706070 >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 0x7f0e8415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.255+0000 7f0e837fe640 1 --2- 172.21.15.40:0/3286706070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e84070570 0x7f0e840b5530 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e8406fca0 tx=0x7f0e7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b754a80e507ac8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.255+0000 7f0e82ffd640 1 -- 172.21.15.40:0/3286706070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78037070 con 0x7f0e84070570 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.255+0000 7f0e82ffd640 1 -- 172.21.15.40:0/3286706070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78032440 con 0x7f0e84070570 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.255+0000 7f0e82ffd640 1 -- 172.21.15.40:0/3286706070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78032810 con 0x7f0e84070570 2024-04-26T21:16:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3286706070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e84070570 msgr2=0x7f0e840b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3286706070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e84070570 0x7f0e840b5530 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e8406fca0 tx=0x7f0e7802a230 comp rx=0 tx=0).stop 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3286706070 shutdown_connections 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3286706070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e84070570 0x7f0e840b5530 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3286706070 >> v2:172.21.15.184:3300/0 conn(0x7f0e8409ba40 0x7f0e8415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3286706070 >> 172.21.15.40:0/3286706070 conn(0x7f0e840b4a10 msgr2=0x7f0e840b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3286706070 shutdown_connections 2024-04-26T21:16:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.256+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3286706070 wait complete. 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.257+0000 7f0e8a6a9640 1 Processor -- start 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.257+0000 7f0e8a6a9640 1 -- start start 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e84070570 0x7f0e840c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e8409be20 0x7f0e841f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e840d2050 con 0x7f0e8409ba40 2024-04-26T21:16:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8409c2b0 con 0x7f0e8409be20 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e8a6a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e840d21d0 con 0x7f0e84070570 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33256/0 (socket says 172.21.15.40:33256) 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 -- 172.21.15.40:0/3015597306 learned_addr learned my addr 172.21.15.40:0/3015597306 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e88c46640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e8409be20 0x7f0e841f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e83fff640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e84070570 0x7f0e840c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 -- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e84070570 msgr2=0x7f0e840c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e84070570 0x7f0e840c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 -- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e8409be20 msgr2=0x7f0e841f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e8409be20 0x7f0e841f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.258+0000 7f0e837fe640 1 -- 172.21.15.40:0/3015597306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841f5800 con 0x7f0e8409ba40 2024-04-26T21:16:51.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e837fe640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78041070 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7803c870 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e840c9a40 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78037040 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.259+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e840c8c80 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.260+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8406a290 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.260+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e78044070 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.261+0000 7f0e80ff9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.261+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f0e780ac8a0 con 0x7f0e8409ba40 2024-04-26T21:16:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.261+0000 7f0e83fff640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.262+0000 7f0e83fff640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 0x7f0e7006edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400a990 tx=0x7f0e74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:51.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.266+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e841f5800 con 0x7f0e8409ba40 2024-04-26T21:16:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.489+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0e84153950 con 0x7f0e8409ba40 2024-04-26T21:16:51.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.490+0000 7f0e80ff9640 1 -- 172.21.15.40:0/3015597306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f0e7807adc0 con 0x7f0e8409ba40 2024-04-26T21:16:51.491 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:51.491 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 msgr2=0x7f0e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 0x7f0e7006edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400a990 tx=0x7f0e74008040 comp rx=0 tx=0).stop 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 msgr2=0x7f0e840cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e780046c0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 shutdown_connections 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e8409be20 0x7f0e841f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8409ba40 0x7f0e840cc4f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 --2- 172.21.15.40:0/3015597306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e84070570 0x7f0e840c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 >> 172.21.15.40:0/3015597306 conn(0x7f0e840b4a10 msgr2=0x7f0e84071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 shutdown_connections 2024-04-26T21:16:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:51.492+0000 7f0e8a6a9640 1 -- 172.21.15.40:0/3015597306 wait complete. 2024-04-26T21:16:51.504 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:51.504 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:51.825 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:55.829 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:56.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:16:56.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.616+0000 7fc14d42a640 1 Processor -- start 2024-04-26T21:16:56.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.616+0000 7fc14d42a640 1 -- start start 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc14d42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1480d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc14d42a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1480d0220 con 0x7fc1480b60a0 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc14d42a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1480d0500 con 0x7fc1480cfda0 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc14d42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480d07e0 con 0x7fc1480d1b80 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc1477fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1480b60a0 0x7fc1480b6530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53056/0 (socket says 172.21.15.40:53056) 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc1477fe640 1 -- 172.21.15.40:0/3642509174 learned_addr learned my addr 172.21.15.40:0/3642509174 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:16:56.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc146ffd640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1480d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269407851 0 0) 0x7fc1480d0220 con 0x7fc1480b60a0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.617+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc134003520 con 0x7fc1480b60a0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1973893057 0 0) 0x7fc134003520 con 0x7fc1480b60a0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 msgr2=0x7fc1480d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1480d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 >> v1:172.21.15.184:6789/0 conn(0x7fc1480cfda0 legacy=0x7fc1480d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480d0ac0 con 0x7fc1480b60a0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc1400019f0 con 0x7fc1480b60a0 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 >> v1:172.21.15.40:6789/0 conn(0x7fc1480b60a0 legacy=0x7fc1480b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc134003f20 0x7fc134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc1340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d07e0 con 0x7fc134004d40 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc13400ba20 con 0x7fc134003f20 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc13400bc20 con 0x7fc1480d1b80 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc1340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc146ffd640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc13ffff640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc134003f20 0x7fc134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 -- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 msgr2=0x7fc1340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1480d1b80 0x7fc1340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 -- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc134003f20 msgr2=0x7fc134004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc134003f20 0x7fc134004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc1477fe640 1 -- 172.21.15.40:0/3642509174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134003520 con 0x7fc134004d40 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.618+0000 7fc13ffff640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc134003f20 0x7fc134004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.619+0000 7fc1477fe640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc1340051c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc14000a810 tx=0x7fc134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7484b41a96875eec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.619+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14000a430 con 0x7fc134004d40 2024-04-26T21:16:56.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.619+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14002f070 con 0x7fc134004d40 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.619+0000 7fc1467fc640 1 -- 172.21.15.40:0/3642509174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14002bc90 con 0x7fc134004d40 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 -- 172.21.15.40:0/3642509174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 msgr2=0x7fc1340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3642509174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc1340051c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc14000a810 tx=0x7fc134005830 comp rx=0 tx=0).stop 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc1477fe640 1 -- 172.21.15.40:0/3642509174 reap_dead start 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 -- 172.21.15.40:0/3642509174 shutdown_connections 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 -- 172.21.15.40:0/3642509174 >> 172.21.15.40:0/3642509174 conn(0x7fc14809e540 msgr2=0x7fc1480d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 -- 172.21.15.40:0/3642509174 shutdown_connections 2024-04-26T21:16:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.620+0000 7fc14d42a640 1 -- 172.21.15.40:0/3642509174 wait complete. 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.621+0000 7fc14d42a640 1 Processor -- start 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.621+0000 7fc14d42a640 1 -- start start 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14815f360 0x7fc1480ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1481533a0 0x7fc148153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b9060 con 0x7fc134004d40 2024-04-26T21:16:56.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d4170 con 0x7fc14815f360 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc14d42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480b91e0 con 0x7fc1481533a0 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33270/0 (socket says 172.21.15.40:33270) 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 -- 172.21.15.40:0/3092719706 learned_addr learned my addr 172.21.15.40:0/3092719706 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc13ffff640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14815f360 0x7fc1480ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc1477fe640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1481533a0 0x7fc148153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 -- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1481533a0 msgr2=0x7fc148153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1481533a0 0x7fc148153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 -- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14815f360 msgr2=0x7fc1480ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14815f360 0x7fc1480ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 -- 172.21.15.40:0/3092719706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14815db90 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.622+0000 7fc146ffd640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc1380119d0 tx=0x7fc138011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.623+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800fb50 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.623+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13800fcf0 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.623+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800fea0 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.623+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14815de70 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.623+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14815e430 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.624+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14806a290 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.624+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc138012040 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.625+0000 7fc13f7fe640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 0x7fc12806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.625+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fc13808c270 con 0x7fc134004d40 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.625+0000 7fc13ffff640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 0x7fc12806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:16:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.626+0000 7fc13ffff640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 0x7fc12806ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1480cc280 tx=0x7fc130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:16:56.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.629+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc138019030 con 0x7fc134004d40 2024-04-26T21:16:56.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.856+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc14806d960 con 0x7fc134004d40 2024-04-26T21:16:56.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.857+0000 7fc13f7fe640 1 -- 172.21.15.40:0/3092719706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc13805a5e0 con 0x7fc134004d40 2024-04-26T21:16:56.858 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:16:56.858 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 msgr2=0x7fc12806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 0x7fc12806ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1480cc280 tx=0x7fc130009040 comp rx=0 tx=0).stop 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 msgr2=0x7fc14815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc1380119d0 tx=0x7fc138011e90 comp rx=0 tx=0).stop 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 shutdown_connections 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc12806c920 0x7fc12806ede0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1481533a0 0x7fc148153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc14815f360 0x7fc1480ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 --2- 172.21.15.40:0/3092719706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc134004d40 0x7fc14815ff90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:16:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.858+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 >> 172.21.15.40:0/3092719706 conn(0x7fc14809e540 msgr2=0x7fc1480ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:16:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.859+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 shutdown_connections 2024-04-26T21:16:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:16:56.859+0000 7fc14d42a640 1 -- 172.21.15.40:0/3092719706 wait complete. 2024-04-26T21:16:56.870 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:16:56.870 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:16:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:16:59.833 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:01.835 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:01.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:01.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:17:01.917 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:17:01.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.983+0000 7ff2d45af640 1 Processor -- start 2024-04-26T21:17:01.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.983+0000 7ff2d45af640 1 -- start start 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d45af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2cc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d45af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2cc0b2550 con 0x7ff2cc0d5810 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d45af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2cc0b2830 con 0x7ff2cc15f4d0 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d45af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b6180 con 0x7ff2cc0d2e20 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d1b4a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2cc0d5810 0x7ff2cc0b35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50544/0 (socket says 172.21.15.40:50544) 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d1b4a640 1 -- 172.21.15.40:0/535029040 learned_addr learned my addr 172.21.15.40:0/535029040 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.984+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2cc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118739606 0 0) 0x7ff2cc0b2550 con 0x7ff2cc0d5810 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2b8003520 con 0x7ff2cc0d5810 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793123155 0 0) 0x7ff2cc0b2830 con 0x7ff2cc15f4d0 2024-04-26T21:17:01.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2cc0b2550 con 0x7ff2cc15f4d0 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879351458 0 0) 0x7ff2b8003520 con 0x7ff2cc0d5810 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 msgr2=0x7ff2cc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2cc0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 >> v1:172.21.15.184:6789/0 conn(0x7ff2cc15f4d0 legacy=0x7ff2cc0d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc0b2b10 con 0x7ff2cc0d5810 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff2bc001670 con 0x7ff2cc0d5810 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2cc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 >> v1:172.21.15.40:6789/0 conn(0x7ff2cc0d5810 legacy=0x7ff2cc0b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8003b70 0x7ff2b8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8004d40 0x7ff2b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b6180 con 0x7ff2b8003b70 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b800bad0 con 0x7ff2b8004d40 2024-04-26T21:17:01.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b800bcd0 con 0x7ff2cc0d2e20 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8003b70 0x7ff2b8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d1b4a640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8004d40 0x7ff2b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 -- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 msgr2=0x7ff2b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc0d2e20 0x7ff2b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 -- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8004d40 msgr2=0x7ff2b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8004d40 0x7ff2b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d234b640 1 -- 172.21.15.40:0/535029040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b8003520 con 0x7ff2b8003b70 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.985+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8004d40 0x7ff2b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.986+0000 7ff2d234b640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8003b70 0x7ff2b8003f70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80018d0 tx=0x7ff2b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b65cfc084bef37e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.986+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80014f0 con 0x7ff2b8003b70 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.986+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c80052d0 con 0x7ff2b8003b70 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.986+0000 7ff2d1349640 1 -- 172.21.15.40:0/535029040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c802aca0 con 0x7ff2b8003b70 2024-04-26T21:17:01.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 -- 172.21.15.40:0/535029040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8003b70 msgr2=0x7ff2b8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 --2- 172.21.15.40:0/535029040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8003b70 0x7ff2b8003f70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80018d0 tx=0x7ff2b8005830 comp rx=0 tx=0).stop 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d2b4c640 1 -- 172.21.15.40:0/535029040 reap_dead start 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 -- 172.21.15.40:0/535029040 shutdown_connections 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 -- 172.21.15.40:0/535029040 >> 172.21.15.40:0/535029040 conn(0x7ff2cc08ebf0 msgr2=0x7ff2cc09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 -- 172.21.15.40:0/535029040 shutdown_connections 2024-04-26T21:17:01.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.987+0000 7ff2d45af640 1 -- 172.21.15.40:0/535029040 wait complete. 2024-04-26T21:17:01.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.988+0000 7ff2d45af640 1 Processor -- start 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.988+0000 7ff2d45af640 1 -- start start 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8003b70 0x7ff2cc0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc14d650 0x7ff2cc0b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b1ee0 con 0x7ff2cc161a80 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b30d0 con 0x7ff2b8003b70 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d45af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b2060 con 0x7ff2cc14d650 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35110/0 (socket says 172.21.15.40:35110) 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 -- 172.21.15.40:0/853182378 learned_addr learned my addr 172.21.15.40:0/853182378 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d234b640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8003b70 0x7ff2cc0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 -- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc14d650 msgr2=0x7ff2cc0b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:01.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d1b4a640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc14d650 0x7ff2cc0b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc14d650 0x7ff2cc0b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 -- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8003b70 msgr2=0x7ff2cc0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8003b70 0x7ff2cc0c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.989+0000 7ff2d2b4c640 1 -- 172.21.15.40:0/853182378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc0b9870 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2d2b4c640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ed0 tx=0x7ff2c40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4011de0 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c401b070 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc0c9360 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003990 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.990+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc0c9920 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.991+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc06a290 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.991+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2c4005d10 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.992+0000 7ff2c2ffd640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 0x7ff2b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.992+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7ff2c408bef0 con 0x7ff2cc161a80 2024-04-26T21:17:01.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.992+0000 7ff2d234b640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 0x7ff2b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:01.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.993+0000 7ff2d234b640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 0x7ff2b006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8032000 tx=0x7ff2c8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:01.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:01.996+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2c4014350 con 0x7ff2cc161a80 2024-04-26T21:17:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.219+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff2cc09e7d0 con 0x7ff2cc161a80 2024-04-26T21:17:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.219+0000 7ff2c2ffd640 1 -- 172.21.15.40:0/853182378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff2c405a300 con 0x7ff2cc161a80 2024-04-26T21:17:02.220 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:02.220 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 msgr2=0x7ff2b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 0x7ff2b006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8032000 tx=0x7ff2c8009030 comp rx=0 tx=0).stop 2024-04-26T21:17:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 msgr2=0x7ff2cc161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ed0 tx=0x7ff2c40115a0 comp rx=0 tx=0).stop 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 shutdown_connections 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff2b006c8f0 0x7ff2b006edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2cc161a80 0x7ff2cc161f00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2cc14d650 0x7ff2cc0b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 --2- 172.21.15.40:0/853182378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b8003b70 0x7ff2cc0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 >> 172.21.15.40:0/853182378 conn(0x7ff2cc08ebf0 msgr2=0x7ff2cc0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 shutdown_connections 2024-04-26T21:17:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:02.221+0000 7ff2d45af640 1 -- 172.21.15.40:0/853182378 wait complete. 2024-04-26T21:17:02.233 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:02.233 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:03.838 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:07.233 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.345+0000 7f3b168bd640 1 Processor -- start 2024-04-26T21:17:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.345+0000 7f3b168bd640 1 -- start start 2024-04-26T21:17:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b168bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b1015d050 0x7f3b1006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b168bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 0x7f3b1006ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b168bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100b56f0 con 0x7f3b100b6790 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b168bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100b59d0 con 0x7f3b1015d050 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b168bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100b5cb0 con 0x7f3b10073010 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b1015d050 0x7f3b1006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b100b6790 0x7f3b10072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50550/0 (socket says 172.21.15.40:50550) 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0f7fe640 1 -- 172.21.15.40:0/1496425977 learned_addr learned my addr 172.21.15.40:0/1496425977 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b14e5a640 1 --2- 172.21.15.40:0/1496425977 >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 0x7f3b1006ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0ffff640 1 -- 172.21.15.40:0/1496425977 >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 msgr2=0x7f3b1006ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0ffff640 1 --2- 172.21.15.40:0/1496425977 >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 0x7f3b1006ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0ffff640 1 -- 172.21.15.40:0/1496425977 >> v1:172.21.15.40:6789/0 conn(0x7f3b100b6790 legacy=0x7f3b10072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0effd640 1 -- 172.21.15.40:0/1496425977 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809555674 0 0) 0x7f3b100b56f0 con 0x7f3b100b6790 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b0ffff640 1 -- 172.21.15.40:0/1496425977 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100b5f90 con 0x7f3b1015d050 2024-04-26T21:17:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.346+0000 7f3b14e5a640 1 --2- 172.21.15.40:0/1496425977 >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 0x7f3b1006ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.347+0000 7f3b0ffff640 1 --2- 172.21.15.40:0/1496425977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1015d050 0x7f3b1006ccb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3b1006e890 tx=0x7f3b0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe82cebf819f990e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.347+0000 7f3b0effd640 1 -- 172.21.15.40:0/1496425977 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04037070 con 0x7f3b1015d050 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.347+0000 7f3b0effd640 1 -- 172.21.15.40:0/1496425977 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04032440 con 0x7f3b1015d050 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.347+0000 7f3b0effd640 1 -- 172.21.15.40:0/1496425977 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04032810 con 0x7f3b1015d050 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 -- 172.21.15.40:0/1496425977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1015d050 msgr2=0x7f3b1006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 --2- 172.21.15.40:0/1496425977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1015d050 0x7f3b1006ccb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3b1006e890 tx=0x7f3b0402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 -- 172.21.15.40:0/1496425977 shutdown_connections 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 --2- 172.21.15.40:0/1496425977 >> v2:172.21.15.184:3300/0 conn(0x7f3b10073010 0x7f3b1006ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 --2- 172.21.15.40:0/1496425977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1015d050 0x7f3b1006ccb0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 -- 172.21.15.40:0/1496425977 >> 172.21.15.40:0/1496425977 conn(0x7f3b100ce140 msgr2=0x7f3b100ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 -- 172.21.15.40:0/1496425977 shutdown_connections 2024-04-26T21:17:07.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.348+0000 7f3b168bd640 1 -- 172.21.15.40:0/1496425977 wait complete. 2024-04-26T21:17:07.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.349+0000 7f3b168bd640 1 Processor -- start 2024-04-26T21:17:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.349+0000 7f3b168bd640 1 -- start start 2024-04-26T21:17:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100b6790 0x7f3b1006e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b1015d050 0x7f3b1006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1015d430 con 0x7f3b10073010 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b10068320 con 0x7f3b100b6790 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b168bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1006d1f0 con 0x7f3b1015d050 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35132/0 (socket says 172.21.15.40:35132) 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 -- 172.21.15.40:0/2534983843 learned_addr learned my addr 172.21.15.40:0/2534983843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 -- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b1015d050 msgr2=0x7f3b1006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b1015d050 0x7f3b1006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 -- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100b6790 msgr2=0x7f3b1006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100b6790 0x7f3b1006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.350+0000 7f3b0ffff640 1 -- 172.21.15.40:0/2534983843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100aa640 con 0x7f3b10073010 2024-04-26T21:17:07.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b0ffff640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:07.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04037040 con 0x7f3b10073010 2024-04-26T21:17:07.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04042070 con 0x7f3b10073010 2024-04-26T21:17:07.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b100aa920 con 0x7f3b10073010 2024-04-26T21:17:07.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04030bd0 con 0x7f3b10073010 2024-04-26T21:17:07.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.351+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b100aaee0 con 0x7f3b10073010 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.352+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1006a290 con 0x7f3b10073010 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.352+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b04047460 con 0x7f3b10073010 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.353+0000 7f3b0cff9640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 0x7f3afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.353+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f3b040ac860 con 0x7f3b10073010 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.353+0000 7f3b0f7fe640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 0x7f3afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:07.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.354+0000 7f3b0f7fe640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 0x7f3afc06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:07.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.357+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b04032e20 con 0x7f3b10073010 2024-04-26T21:17:07.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.583+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3b10073750 con 0x7f3b10073010 2024-04-26T21:17:07.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.583+0000 7f3b0cff9640 1 -- 172.21.15.40:0/2534983843 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3b0407ac70 con 0x7f3b10073010 2024-04-26T21:17:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:07.584 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 msgr2=0x7f3afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 0x7f3afc06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00007920 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 msgr2=0x7f3b10149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b040046c0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 shutdown_connections 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3afc06c8f0 0x7f3afc06edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b1015d050 0x7f3b1006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b100b6790 0x7f3b1006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 --2- 172.21.15.40:0/2534983843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10073010 0x7f3b10149090 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 >> 172.21.15.40:0/2534983843 conn(0x7f3b100ce140 msgr2=0x7f3b100ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.585+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 shutdown_connections 2024-04-26T21:17:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:07.586+0000 7f3b168bd640 1 -- 172.21.15.40:0/2534983843 wait complete. 2024-04-26T21:17:07.597 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:07.597 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:07.842 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:09.844 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:11.846 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:12.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.711+0000 7f2b7f9da640 1 Processor -- start 2024-04-26T21:17:12.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 -- start start 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b7815f4d0 0x7f2b780b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b780d5810 0x7f2b780b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b780b2550 con 0x7f2b780d2e20 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2b780b6180 con 0x7f2b780d5810 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.712+0000 7f2b7f9da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2b780b2830 con 0x7f2b7815f4d0 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b780d5810 0x7f2b780b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b780d5810 0x7f2b780b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38934/0 (socket says 172.21.15.40:38934) 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 -- 172.21.15.40:0/2029566736 learned_addr learned my addr 172.21.15.40:0/2029566736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7d776640 1 --2- 172.21.15.40:0/2029566736 >> v2:172.21.15.184:3300/0 conn(0x7f2b7815f4d0 0x7f2b780b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 -- 172.21.15.40:0/2029566736 >> v2:172.21.15.184:3300/0 conn(0x7f2b7815f4d0 msgr2=0x7f2b780b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 --2- 172.21.15.40:0/2029566736 >> v2:172.21.15.184:3300/0 conn(0x7f2b7815f4d0 0x7f2b780b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 -- 172.21.15.40:0/2029566736 >> v1:172.21.15.184:6789/0 conn(0x7f2b780d2e20 legacy=0x7f2b780d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:12.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.713+0000 7f2b7df77640 1 -- 172.21.15.40:0/2029566736 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780b2b10 con 0x7f2b780d5810 2024-04-26T21:17:12.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7df77640 1 --2- 172.21.15.40:0/2029566736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b780b35f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2b7808f070 tx=0x7f2b6802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b4420981e8ad7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:12.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b77fff640 1 -- 172.21.15.40:0/2029566736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68036070 con 0x7f2b780d5810 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b77fff640 1 -- 172.21.15.40:0/2029566736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68031600 con 0x7f2b780d5810 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b77fff640 1 -- 172.21.15.40:0/2029566736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680319d0 con 0x7f2b780d5810 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 -- 172.21.15.40:0/2029566736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 msgr2=0x7f2b780b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/2029566736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b780b35f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2b7808f070 tx=0x7f2b6802a610 comp rx=0 tx=0).stop 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 -- 172.21.15.40:0/2029566736 shutdown_connections 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/2029566736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b780b35f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/2029566736 >> v2:172.21.15.184:3300/0 conn(0x7f2b7815f4d0 0x7f2b780b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.714+0000 7f2b7f9da640 1 -- 172.21.15.40:0/2029566736 >> 172.21.15.40:0/2029566736 conn(0x7f2b7808ebf0 msgr2=0x7f2b7809e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.715+0000 7f2b7f9da640 1 -- 172.21.15.40:0/2029566736 shutdown_connections 2024-04-26T21:17:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.715+0000 7f2b7f9da640 1 -- 172.21.15.40:0/2029566736 wait complete. 2024-04-26T21:17:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 Processor -- start 2024-04-26T21:17:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 -- start start 2024-04-26T21:17:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b780d2e20 0x7f2b781582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b7815f4d0 0x7f2b780ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b780b3b30 con 0x7f2b780d5810 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b780b30d0 con 0x7f2b780d2e20 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.716+0000 7f2b7f9da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b780b3cb0 con 0x7f2b7815f4d0 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38942/0 (socket says 172.21.15.40:38942) 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 -- 172.21.15.40:0/1620665112 learned_addr learned my addr 172.21.15.40:0/1620665112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7d776640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b780d2e20 0x7f2b781582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7df77640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b7815f4d0 0x7f2b780ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 -- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b7815f4d0 msgr2=0x7f2b780ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b7815f4d0 0x7f2b780ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 -- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b780d2e20 msgr2=0x7f2b781582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b780d2e20 0x7f2b781582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 -- 172.21.15.40:0/1620665112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780a1ba0 con 0x7f2b780d5810 2024-04-26T21:17:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7cf75640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c430 tx=0x7f2b6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0052d0 con 0x7f2b780d5810 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c005490 con 0x7f2b780d5810 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b780a1e80 con 0x7f2b780d5810 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.717+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c007260 con 0x7f2b780d5810 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.718+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b780a2440 con 0x7f2b780d5810 2024-04-26T21:17:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.719+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7806a290 con 0x7f2b780d5810 2024-04-26T21:17:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.719+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b6c007400 con 0x7f2b780d5810 2024-04-26T21:17:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.719+0000 7f2b75ffb640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.719+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f2b6c08b070 con 0x7f2b780d5810 2024-04-26T21:17:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.720+0000 7f2b7d776640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.720+0000 7f2b7d776640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 0x7f2b6406edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2b700079a0 tx=0x7f2b70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.724+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b6c058da0 con 0x7f2b780d5810 2024-04-26T21:17:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.949+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2b780d5390 con 0x7f2b780d5810 2024-04-26T21:17:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.950+0000 7f2b75ffb640 1 -- 172.21.15.40:0/1620665112 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f2b6c05d0a0 con 0x7f2b780d5810 2024-04-26T21:17:12.951 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:12.951 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 msgr2=0x7f2b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 0x7f2b6406edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2b700079a0 tx=0x7f2b70008040 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 msgr2=0x7f2b7809f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c430 tx=0x7f2b6c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 shutdown_connections 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.951+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b7815f4d0 0x7f2b780ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b780d5810 0x7f2b7809f060 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 --2- 172.21.15.40:0/1620665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b780d2e20 0x7f2b781582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 >> 172.21.15.40:0/1620665112 conn(0x7f2b7808ebf0 msgr2=0x7f2b7806db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 shutdown_connections 2024-04-26T21:17:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:12.952+0000 7f2b7f9da640 1 -- 172.21.15.40:0/1620665112 wait complete. 2024-04-26T21:17:12.963 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:12.963 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:15.850 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:17.852 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:17.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.075+0000 7fa828cbd640 1 Processor -- start 2024-04-26T21:17:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.075+0000 7fa828cbd640 1 -- start start 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.075+0000 7fa828cbd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8240d5810 0x7fa8240b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.075+0000 7fa828cbd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8240d2e20 0x7fa8240b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.075+0000 7fa828cbd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8240b2550 con 0x7fa82415f4d0 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa828cbd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa8240b6180 con 0x7fa8240d5810 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa828cbd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa8240b2830 con 0x7fa8240d2e20 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8240d5810 0x7fa8240b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8240d5810 0x7fa8240b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38948/0 (socket says 172.21.15.40:38948) 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 -- 172.21.15.40:0/1599167195 learned_addr learned my addr 172.21.15.40:0/1599167195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa822d9d640 1 --2- 172.21.15.40:0/1599167195 >> v2:172.21.15.184:3300/0 conn(0x7fa8240d2e20 0x7fa8240b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 -- 172.21.15.40:0/1599167195 >> v2:172.21.15.184:3300/0 conn(0x7fa8240d2e20 msgr2=0x7fa8240b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 --2- 172.21.15.40:0/1599167195 >> v2:172.21.15.184:3300/0 conn(0x7fa8240d2e20 0x7fa8240b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 -- 172.21.15.40:0/1599167195 >> v1:172.21.15.184:6789/0 conn(0x7fa82415f4d0 legacy=0x7fa8240d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.076+0000 7fa81bfff640 1 -- 172.21.15.40:0/1599167195 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240b2b10 con 0x7fa8240d5810 2024-04-26T21:17:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa81bfff640 1 --2- 172.21.15.40:0/1599167195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8240d5810 0x7fa8240b1ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa824162e60 tx=0x7fa80c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618ce44b4b8e0afc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa821d9b640 1 -- 172.21.15.40:0/1599167195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c037070 con 0x7fa8240d5810 2024-04-26T21:17:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa821d9b640 1 -- 172.21.15.40:0/1599167195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80c032440 con 0x7fa8240d5810 2024-04-26T21:17:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa821d9b640 1 -- 172.21.15.40:0/1599167195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c032810 con 0x7fa8240d5810 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa828cbd640 1 -- 172.21.15.40:0/1599167195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8240d5810 msgr2=0x7fa8240b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.077+0000 7fa828cbd640 1 --2- 172.21.15.40:0/1599167195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8240d5810 0x7fa8240b1ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa824162e60 tx=0x7fa80c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 -- 172.21.15.40:0/1599167195 shutdown_connections 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 --2- 172.21.15.40:0/1599167195 >> v2:172.21.15.184:3300/0 conn(0x7fa8240d2e20 0x7fa8240b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 --2- 172.21.15.40:0/1599167195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8240d5810 0x7fa8240b1ee0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 -- 172.21.15.40:0/1599167195 >> 172.21.15.40:0/1599167195 conn(0x7fa82408ebf0 msgr2=0x7fa82409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 -- 172.21.15.40:0/1599167195 shutdown_connections 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.078+0000 7fa828cbd640 1 -- 172.21.15.40:0/1599167195 wait complete. 2024-04-26T21:17:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.079+0000 7fa828cbd640 1 Processor -- start 2024-04-26T21:17:18.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.079+0000 7fa828cbd640 1 -- start start 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 0x7fa824159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8240d5810 0x7fa8240cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240b3b30 con 0x7fa82415f4d0 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240b30d0 con 0x7fa8240d2e20 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa828cbd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa8240b3cb0 con 0x7fa8240d5810 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38960/0 (socket says 172.21.15.40:38960) 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 -- 172.21.15.40:0/2240315123 learned_addr learned my addr 172.21.15.40:0/2240315123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa81bfff640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8240d5810 0x7fa8240cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 -- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8240d5810 msgr2=0x7fa8240cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8240d5810 0x7fa8240cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 -- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 msgr2=0x7fa824159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa82259c640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 0x7fa824159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 0x7fa824159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 -- 172.21.15.40:0/2240315123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa824159d70 con 0x7fa82415f4d0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa82259c640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 0x7fa824159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa822d9d640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007e40 tx=0x7fa81c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.080+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c014620 con 0x7fa82415f4d0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.081+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c019070 con 0x7fa82415f4d0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.081+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8240cce20 con 0x7fa82415f4d0 2024-04-26T21:17:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.081+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c014a20 con 0x7fa82415f4d0 2024-04-26T21:17:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.081+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8240cd3e0 con 0x7fa82415f4d0 2024-04-26T21:17:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.082+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8240cdba0 con 0x7fa82415f4d0 2024-04-26T21:17:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.082+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa81c014bc0 con 0x7fa82415f4d0 2024-04-26T21:17:18.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.082+0000 7fa81affd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 0x7fa80406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:18.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.083+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fa81c08c430 con 0x7fa82415f4d0 2024-04-26T21:17:18.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.083+0000 7fa82259c640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 0x7fa80406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:18.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.083+0000 7fa82259c640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 0x7fa80406ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa8140015e0 tx=0x7fa814009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.087+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8240cdba0 con 0x7fa82415f4d0 2024-04-26T21:17:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.310+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa82406a290 con 0x7fa82415f4d0 2024-04-26T21:17:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.311+0000 7fa81affd640 1 -- 172.21.15.40:0/2240315123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fa81c05a840 con 0x7fa82415f4d0 2024-04-26T21:17:18.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:18.313 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 msgr2=0x7fa80406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 0x7fa80406ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa8140015e0 tx=0x7fa814009040 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 msgr2=0x7fa8240cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007e40 tx=0x7fa81c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 shutdown_connections 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa80406c920 0x7fa80406ede0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa82415f4d0 0x7fa8240cc320 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8240d5810 0x7fa8240cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 --2- 172.21.15.40:0/2240315123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8240d2e20 0x7fa824159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:18.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 >> 172.21.15.40:0/2240315123 conn(0x7fa82408ebf0 msgr2=0x7fa82406e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:18.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 shutdown_connections 2024-04-26T21:17:18.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:18.313+0000 7fa828cbd640 1 -- 172.21.15.40:0/2240315123 wait complete. 2024-04-26T21:17:18.325 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:18.325 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:19.853 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:23.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:23.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.436+0000 7ff4c5af1640 1 Processor -- start 2024-04-26T21:17:23.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.436+0000 7ff4c5af1640 1 -- start start 2024-04-26T21:17:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4c5af1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4c0070570 0x7ff4c00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4c5af1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c015fce0 con 0x7ff4c009ba40 2024-04-26T21:17:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4c5af1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c015ffc0 con 0x7ff4c015f6a0 2024-04-26T21:17:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4c5af1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c0160610 con 0x7ff4c0070570 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4bffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4c0070570 0x7ff4c00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4bf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff4c009ba40 0x7ff4c009be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56896/0 (socket says 172.21.15.40:56896) 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4bf7fe640 1 -- 172.21.15.40:0/221065777 learned_addr learned my addr 172.21.15.40:0/221065777 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4bffff640 1 -- 172.21.15.40:0/221065777 >> v1:172.21.15.184:6789/0 conn(0x7ff4c015f6a0 legacy=0x7ff4c00d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.437+0000 7ff4bffff640 1 -- 172.21.15.40:0/221065777 >> v1:172.21.15.40:6789/0 conn(0x7ff4c009ba40 legacy=0x7ff4c009be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4be7fc640 1 -- 172.21.15.40:0/221065777 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214042930 0 0) 0x7ff4c015fce0 con 0x7ff4c009ba40 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4bffff640 1 -- 172.21.15.40:0/221065777 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c01602a0 con 0x7ff4c0070570 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4bffff640 1 --2- 172.21.15.40:0/221065777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c0070570 0x7ff4c00b5530 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009d90 tx=0x7ff4b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cb64b5ed54a4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4be7fc640 1 -- 172.21.15.40:0/221065777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8007740 con 0x7ff4c0070570 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4be7fc640 1 -- 172.21.15.40:0/221065777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b80078e0 con 0x7ff4c0070570 2024-04-26T21:17:23.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.438+0000 7ff4be7fc640 1 -- 172.21.15.40:0/221065777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8007cb0 con 0x7ff4c0070570 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 -- 172.21.15.40:0/221065777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c0070570 msgr2=0x7ff4c00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/221065777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c0070570 0x7ff4c00b5530 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009d90 tx=0x7ff4b802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 -- 172.21.15.40:0/221065777 shutdown_connections 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/221065777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c0070570 0x7ff4c00b5530 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 -- 172.21.15.40:0/221065777 >> 172.21.15.40:0/221065777 conn(0x7ff4c00b4a10 msgr2=0x7ff4c00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 -- 172.21.15.40:0/221065777 shutdown_connections 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.439+0000 7ff4c5af1640 1 -- 172.21.15.40:0/221065777 wait complete. 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.440+0000 7ff4c5af1640 1 Processor -- start 2024-04-26T21:17:23.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 -- start start 2024-04-26T21:17:23.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c0070570 0x7ff4c01f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c015f6a0 0x7ff4c01f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00d1b20 con 0x7ff4c009ba40 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00d19a0 con 0x7ff4c0070570 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4c5af1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c00d1ca0 con 0x7ff4c015f6a0 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56656/0 (socket says 172.21.15.40:56656) 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.441+0000 7ff4beffd640 1 -- 172.21.15.40:0/219174831 learned_addr learned my addr 172.21.15.40:0/219174831 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4bf7fe640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c0070570 0x7ff4c01f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4bffff640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c015f6a0 0x7ff4c01f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 -- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c015f6a0 msgr2=0x7ff4c01f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c015f6a0 0x7ff4c01f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 -- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c0070570 msgr2=0x7ff4c01f99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c0070570 0x7ff4c01f99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 -- 172.21.15.40:0/219174831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0161000 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4beffd640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b3f0 tx=0x7ff4b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4015020 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4011830 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c0163730 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4011c20 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.442+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c00c8650 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.443+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c006a290 con 0x7ff4c009ba40 2024-04-26T21:17:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.444+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4b4018020 con 0x7ff4c009ba40 2024-04-26T21:17:23.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.444+0000 7ff4a7fff640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 0x7ff4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:23.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.444+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7ff4b408b870 con 0x7ff4c009ba40 2024-04-26T21:17:23.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.445+0000 7ff4bf7fe640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 0x7ff4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:23.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.445+0000 7ff4bf7fe640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 0x7ff4ac06edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000f60 tx=0x7ff4b0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:23.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.449+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4b401f030 con 0x7ff4c009ba40 2024-04-26T21:17:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.672+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4c00c8140 con 0x7ff4c009ba40 2024-04-26T21:17:23.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.673+0000 7ff4a7fff640 1 -- 172.21.15.40:0/219174831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff4b40172c0 con 0x7ff4c009ba40 2024-04-26T21:17:23.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:23.675 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 msgr2=0x7ff4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 0x7ff4ac06edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000f60 tx=0x7ff4b0007b50 comp rx=0 tx=0).stop 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 msgr2=0x7ff4c0160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b3f0 tx=0x7ff4b400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 shutdown_connections 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4ac06c8f0 0x7ff4ac06edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4c015f6a0 0x7ff4c01f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4c009ba40 0x7ff4c0160ac0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 --2- 172.21.15.40:0/219174831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4c0070570 0x7ff4c01f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:23.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 >> 172.21.15.40:0/219174831 conn(0x7ff4c00b4a10 msgr2=0x7ff4c0070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:23.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 shutdown_connections 2024-04-26T21:17:23.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:23.675+0000 7ff4c5af1640 1 -- 172.21.15.40:0/219174831 wait complete. 2024-04-26T21:17:23.688 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:23.688 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:23.857 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:28.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:28.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.799+0000 7fd454e83640 1 Processor -- start 2024-04-26T21:17:28.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.799+0000 7fd454e83640 1 -- start start 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd454e83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 0x7fd450163a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd454e83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd450073010 0x7fd4500d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd454e83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45006f2b0 con 0x7fd45006d910 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd454e83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd450068300 con 0x7fd450073010 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd454e83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd45006f060 con 0x7fd45015f4c0 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd450073010 0x7fd4500d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd450073010 0x7fd4500d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56668/0 (socket says 172.21.15.40:56668) 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 -- 172.21.15.40:0/1942037878 learned_addr learned my addr 172.21.15.40:0/1942037878 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44e59c640 1 --2- 172.21.15.40:0/1942037878 >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 0x7fd450163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 -- 172.21.15.40:0/1942037878 >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 msgr2=0x7fd450163a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 --2- 172.21.15.40:0/1942037878 >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 0x7fd450163a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 -- 172.21.15.40:0/1942037878 >> v1:172.21.15.184:6789/0 conn(0x7fd45006d910 legacy=0x7fd450072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44ed9d640 1 -- 172.21.15.40:0/1942037878 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd450162fa0 con 0x7fd450073010 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.800+0000 7fd44e59c640 1 --2- 172.21.15.40:0/1942037878 >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 0x7fd450163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.801+0000 7fd44ed9d640 1 --2- 172.21.15.40:0/1942037878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd450073010 0x7fd4500d02b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd448009e50 tx=0x7fd44802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a525f1dda7bef91e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.801+0000 7fd44dd9b640 1 -- 172.21.15.40:0/1942037878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448036070 con 0x7fd450073010 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.801+0000 7fd44dd9b640 1 -- 172.21.15.40:0/1942037878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd448031600 con 0x7fd450073010 2024-04-26T21:17:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.801+0000 7fd44dd9b640 1 -- 172.21.15.40:0/1942037878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44802ae60 con 0x7fd450073010 2024-04-26T21:17:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 -- 172.21.15.40:0/1942037878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd450073010 msgr2=0x7fd4500d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 --2- 172.21.15.40:0/1942037878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd450073010 0x7fd4500d02b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd448009e50 tx=0x7fd44802a680 comp rx=0 tx=0).stop 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 -- 172.21.15.40:0/1942037878 shutdown_connections 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 --2- 172.21.15.40:0/1942037878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd450073010 0x7fd4500d02b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 --2- 172.21.15.40:0/1942037878 >> v2:172.21.15.184:3300/0 conn(0x7fd45015f4c0 0x7fd450163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 -- 172.21.15.40:0/1942037878 >> 172.21.15.40:0/1942037878 conn(0x7fd4500ce140 msgr2=0x7fd4500ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 -- 172.21.15.40:0/1942037878 shutdown_connections 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.802+0000 7fd454e83640 1 -- 172.21.15.40:0/1942037878 wait complete. 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.803+0000 7fd454e83640 1 Processor -- start 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.803+0000 7fd454e83640 1 -- start start 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd450073010 0x7fd4501eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 0x7fd4500b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd45006f590 con 0x7fd45006d910 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd450163560 con 0x7fd450073010 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd454e83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd45006e960 con 0x7fd45015f4c0 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56680/0 (socket says 172.21.15.40:56680) 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 -- 172.21.15.40:0/2072526611 learned_addr learned my addr 172.21.15.40:0/2072526611 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd447fff640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd450073010 0x7fd4501eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44ed9d640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 0x7fd4500b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 -- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 msgr2=0x7fd4500b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 0x7fd4500b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 -- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd450073010 msgr2=0x7fd4501eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd450073010 0x7fd4501eab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44e59c640 1 -- 172.21.15.40:0/2072526611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4501e8f00 con 0x7fd45006d910 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.804+0000 7fd44ed9d640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 0x7fd4500b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd44e59c640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c7f0 tx=0x7fd44000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440015070 con 0x7fd45006d910 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440007820 con 0x7fd45006d910 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4501e91e0 con 0x7fd45006d910 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440007bd0 con 0x7fd45006d910 2024-04-26T21:17:28.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.805+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501e97a0 con 0x7fd45006d910 2024-04-26T21:17:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.806+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45006a290 con 0x7fd45006d910 2024-04-26T21:17:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.807+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd440018070 con 0x7fd45006d910 2024-04-26T21:17:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.807+0000 7fd446ffd640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 0x7fd43006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.807+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fd44008bbd0 con 0x7fd45006d910 2024-04-26T21:17:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.807+0000 7fd447fff640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 0x7fd43006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:28.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.808+0000 7fd447fff640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 0x7fd43006ede0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4500afde0 tx=0x7fd438007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:28.812+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd440059990 con 0x7fd45006d910 2024-04-26T21:17:29.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.045+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd450073a10 con 0x7fd45006d910 2024-04-26T21:17:29.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.045+0000 7fd446ffd640 1 -- 172.21.15.40:0/2072526611 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fd44005ee20 con 0x7fd45006d910 2024-04-26T21:17:29.047 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:29.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:29.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 msgr2=0x7fd43006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:29.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 0x7fd43006ede0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4500afde0 tx=0x7fd438007e30 comp rx=0 tx=0).stop 2024-04-26T21:17:29.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 msgr2=0x7fd450149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c7f0 tx=0x7fd44000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 shutdown_connections 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd43006c920 0x7fd43006ede0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd45015f4c0 0x7fd4500b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd450073010 0x7fd4501eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 --2- 172.21.15.40:0/2072526611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd45006d910 0x7fd450149240 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 >> 172.21.15.40:0/2072526611 conn(0x7fd4500ce140 msgr2=0x7fd45006cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 shutdown_connections 2024-04-26T21:17:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:29.047+0000 7fd454e83640 1 -- 172.21.15.40:0/2072526611 wait complete. 2024-04-26T21:17:29.060 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:29.060 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:31.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:17:31.978 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:17:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:34.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.172+0000 7f53815c1640 1 Processor -- start 2024-04-26T21:17:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.172+0000 7f53815c1640 1 -- start start 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f53815c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f537c070570 0x7f537c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f53815c1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537c15fce0 con 0x7f537c15f6a0 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f53815c1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537c15ffc0 con 0x7f537c09ba40 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f53815c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f537c160610 con 0x7f537c070570 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f537a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f537c15f6a0 0x7f537c0d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37632/0 (socket says 172.21.15.40:37632) 2024-04-26T21:17:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f537a7fc640 1 -- 172.21.15.40:0/1445152169 learned_addr learned my addr 172.21.15.40:0/1445152169 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732662924 0 0) 0x7f537c15fce0 con 0x7f537c15f6a0 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f537b7fe640 1 --2- 172.21.15.40:0/1445152169 >> v2:172.21.15.184:3300/0 conn(0x7f537c070570 0x7f537c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.173+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5364003520 con 0x7f537c15f6a0 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 560550222 0 0) 0x7f5364003520 con 0x7f537c15f6a0 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 >> v2:172.21.15.184:3300/0 conn(0x7f537c070570 msgr2=0x7f537c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 --2- 172.21.15.40:0/1445152169 >> v2:172.21.15.184:3300/0 conn(0x7f537c070570 0x7f537c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 >> v1:172.21.15.184:6789/0 conn(0x7f537c09ba40 legacy=0x7f537c09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1602a0 con 0x7f537c15f6a0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5370001670 con 0x7f537c15f6a0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 >> v1:172.21.15.40:6789/0 conn(0x7f537c15f6a0 legacy=0x7f537c0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53640039b0 0x7f5364003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5364004800 0x7f5364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 0x7f536400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f537c160610 con 0x7f53640051c0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f536400c060 con 0x7f53640039b0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f536400c260 con 0x7f5364004800 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 0x7f536400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537affd640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53640039b0 0x7f5364003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537a7fc640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5364004800 0x7f5364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 -- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5364004800 msgr2=0x7f5364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5364004800 0x7f5364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 -- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53640039b0 msgr2=0x7f5364003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53640039b0 0x7f5364003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537b7fe640 1 -- 172.21.15.40:0/1445152169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5364003520 con 0x7f53640051c0 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.174+0000 7f537affd640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53640039b0 0x7f5364003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.175+0000 7f537b7fe640 1 -- 172.21.15.40:0/1445152169 reap_dead start 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.175+0000 7f537b7fe640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 0x7f536400b9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fc0 tx=0x7f5364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef65855a269fcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.175+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374035020 con 0x7f53640051c0 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.175+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374003bd0 con 0x7f53640051c0 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.175+0000 7f5379ffb640 1 -- 172.21.15.40:0/1445152169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374009c10 con 0x7f53640051c0 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 -- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 msgr2=0x7f536400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 0x7f536400b9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fc0 tx=0x7f5364005830 comp rx=0 tx=0).stop 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 -- 172.21.15.40:0/1445152169 shutdown_connections 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 --2- 172.21.15.40:0/1445152169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53640051c0 0x7f536400b9c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 -- 172.21.15.40:0/1445152169 >> 172.21.15.40:0/1445152169 conn(0x7f537c0b4a10 msgr2=0x7f537c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 -- 172.21.15.40:0/1445152169 shutdown_connections 2024-04-26T21:17:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.176+0000 7f53815c1640 1 -- 172.21.15.40:0/1445152169 wait complete. 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.177+0000 7f53815c1640 1 Processor -- start 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.177+0000 7f53815c1640 1 -- start start 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53640051c0 0x7f537c0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c0b5c40 0x7f537c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f537c0b58c0 con 0x7f537c0cf2f0 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f537c0b5740 con 0x7f537c0b5c40 2024-04-26T21:17:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f53815c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f537c09be20 con 0x7f53640051c0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48876/0 (socket says 172.21.15.40:48876) 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 -- 172.21.15.40:0/398502229 learned_addr learned my addr 172.21.15.40:0/398502229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537affd640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53640051c0 0x7f537c0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537a7fc640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c0b5c40 0x7f537c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 -- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53640051c0 msgr2=0x7f537c0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53640051c0 0x7f537c0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 -- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c0b5c40 msgr2=0x7f537c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c0b5c40 0x7f537c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 -- 172.21.15.40:0/398502229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c0c8d70 con 0x7f537c0cf2f0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.178+0000 7f537b7fe640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f537400e230 tx=0x7f5374009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.179+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374040070 con 0x7f537c0cf2f0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.179+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374032e80 con 0x7f537c0cf2f0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.179+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c0c9050 con 0x7f537c0cf2f0 2024-04-26T21:17:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.179+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374035030 con 0x7f537c0cf2f0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.179+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c0b6480 con 0x7f537c0cf2f0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.180+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c06a290 con 0x7f537c0cf2f0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.180+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5374043070 con 0x7f537c0cf2f0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.181+0000 7f53637fe640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 0x7f536806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.181+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f53740acb10 con 0x7f537c0cf2f0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.181+0000 7f537affd640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 0x7f536806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.182+0000 7f537affd640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 0x7f536806edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:34.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.185+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f537407a960 con 0x7f537c0cf2f0 2024-04-26T21:17:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.409+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f537c0b6650 con 0x7f537c0cf2f0 2024-04-26T21:17:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.409+0000 7f53637fe640 1 -- 172.21.15.40:0/398502229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f537407ec60 con 0x7f537c0cf2f0 2024-04-26T21:17:34.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:34.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 msgr2=0x7f536806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 0x7f536806edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c009040 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 msgr2=0x7f537c0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f537400e230 tx=0x7f5374009750 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 shutdown_connections 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f536806c8f0 0x7f536806edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c0cf2f0 0x7f537c0cf780 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c0b5c40 0x7f537c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 --2- 172.21.15.40:0/398502229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53640051c0 0x7f537c0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 >> 172.21.15.40:0/398502229 conn(0x7f537c0b4a10 msgr2=0x7f537c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 shutdown_connections 2024-04-26T21:17:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:34.411+0000 7f53815c1640 1 -- 172.21.15.40:0/398502229 wait complete. 2024-04-26T21:17:34.423 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:34.423 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:35.870 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:37.872 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:39.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:39.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.535+0000 7f6a8da47640 1 Processor -- start 2024-04-26T21:17:39.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.535+0000 7f6a8da47640 1 -- start start 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a8da47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a8809ba40 0x7f6a8815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a8da47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a880b5530 0x7f6a880d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a8da47640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8815fce0 con 0x7f6a88070570 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a8da47640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6a880b59c0 con 0x7f6a880b5530 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a8da47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a88160610 con 0x7f6a8809ba40 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a877fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a880b5530 0x7f6a880d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a867fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6a88070570 0x7f6a8809be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37640/0 (socket says 172.21.15.40:37640) 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.536+0000 7f6a867fc640 1 -- 172.21.15.40:0/914057349 learned_addr learned my addr 172.21.15.40:0/914057349 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a86ffd640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a8809ba40 0x7f6a8815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a877fe640 1 -- 172.21.15.40:0/914057349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a8809ba40 msgr2=0x7f6a8815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a877fe640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a8809ba40 0x7f6a8815f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a877fe640 1 -- 172.21.15.40:0/914057349 >> v1:172.21.15.40:6789/0 conn(0x7f6a88070570 legacy=0x7f6a8809be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a85ffb640 1 -- 172.21.15.40:0/914057349 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595682737 0 0) 0x7f6a8815fce0 con 0x7f6a88070570 2024-04-26T21:17:39.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a877fe640 1 -- 172.21.15.40:0/914057349 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8815ffc0 con 0x7f6a880b5530 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.537+0000 7f6a877fe640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a880b5530 0x7f6a880d1040 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009dc0 tx=0x7f6a7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d84e8ecab9d0bda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a85ffb640 1 -- 172.21.15.40:0/914057349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7802ae60 con 0x7f6a880b5530 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a85ffb640 1 -- 172.21.15.40:0/914057349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a78007740 con 0x7f6a880b5530 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a85ffb640 1 -- 172.21.15.40:0/914057349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007b10 con 0x7f6a880b5530 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 -- 172.21.15.40:0/914057349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a880b5530 msgr2=0x7f6a880d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a880b5530 0x7f6a880d1040 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009dc0 tx=0x7f6a7802a680 comp rx=0 tx=0).stop 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 -- 172.21.15.40:0/914057349 shutdown_connections 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a880b5530 0x7f6a880d1040 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 --2- 172.21.15.40:0/914057349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a8809ba40 0x7f6a8815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 -- 172.21.15.40:0/914057349 >> 172.21.15.40:0/914057349 conn(0x7f6a880b4a10 msgr2=0x7f6a880b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 -- 172.21.15.40:0/914057349 shutdown_connections 2024-04-26T21:17:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.538+0000 7f6a8da47640 1 -- 172.21.15.40:0/914057349 wait complete. 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.540+0000 7f6a8da47640 1 Processor -- start 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.540+0000 7f6a8da47640 1 -- start start 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.540+0000 7f6a8da47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a88070570 0x7f6a8814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.540+0000 7f6a8da47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a880b5530 0x7f6a880c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880d1580 con 0x7f6a8809ba40 2024-04-26T21:17:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880709b0 con 0x7f6a880b5530 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880d1700 con 0x7f6a88070570 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48890/0 (socket says 172.21.15.40:48890) 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 -- 172.21.15.40:0/2427201514 learned_addr learned my addr 172.21.15.40:0/2427201514 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a86ffd640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a88070570 0x7f6a8814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a877fe640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a880b5530 0x7f6a880c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 -- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a88070570 msgr2=0x7f6a8814aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a88070570 0x7f6a8814aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 -- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a880b5530 msgr2=0x7f6a880c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a880b5530 0x7f6a880c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 -- 172.21.15.40:0/2427201514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880c9230 con 0x7f6a8809ba40 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a867fc640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c4d0 tx=0x7f6a7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70018070 con 0x7f6a8809ba40 2024-04-26T21:17:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70007500 con 0x7f6a8809ba40 2024-04-26T21:17:39.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a880c9510 con 0x7f6a8809ba40 2024-04-26T21:17:39.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004a20 con 0x7f6a8809ba40 2024-04-26T21:17:39.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.541+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a880ceca0 con 0x7f6a8809ba40 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.543+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8806a290 con 0x7f6a8809ba40 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.543+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a7001b070 con 0x7f6a8809ba40 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.543+0000 7f6a7ffff640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 0x7f6a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.544+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f6a7008ad90 con 0x7f6a8809ba40 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.544+0000 7f6a86ffd640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 0x7f6a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:39.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.544+0000 7f6a86ffd640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 0x7f6a6806edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:39.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.548+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a70058be0 con 0x7f6a8809ba40 2024-04-26T21:17:39.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.772+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a880c9a30 con 0x7f6a8809ba40 2024-04-26T21:17:39.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.772+0000 7f6a7ffff640 1 -- 172.21.15.40:0/2427201514 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6a7005cee0 con 0x7f6a8809ba40 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:39.774 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 msgr2=0x7f6a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 0x7f6a6806edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80008040 comp rx=0 tx=0).stop 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 msgr2=0x7f6a881f5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c4d0 tx=0x7f6a7000c990 comp rx=0 tx=0).stop 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 shutdown_connections 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a6806c8f0 0x7f6a6806edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a880b5530 0x7f6a880c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8809ba40 0x7f6a881f5890 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 --2- 172.21.15.40:0/2427201514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a88070570 0x7f6a8814aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.774+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 >> 172.21.15.40:0/2427201514 conn(0x7f6a880b4a10 msgr2=0x7f6a880b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.775+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 shutdown_connections 2024-04-26T21:17:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:39.775+0000 7f6a8da47640 1 -- 172.21.15.40:0/2427201514 wait complete. 2024-04-26T21:17:39.786 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:39.786 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:39.874 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:44.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 Processor -- start 2024-04-26T21:17:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 -- start start 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f879015f4d0 0x7f87900b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87900d2e20 0x7f87900d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87900d5ab0 con 0x7f87900d5640 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f87900b6180 con 0x7f87900d2e20 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.899+0000 7f8795e52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87900b2550 con 0x7f879015f4d0 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87900d2e20 0x7f87900d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87900d2e20 0x7f87900d5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58808/0 (socket says 172.21.15.40:58808) 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 -- 172.21.15.40:0/3567863485 learned_addr learned my addr 172.21.15.40:0/3567863485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878f7fe640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f879015f4d0 0x7f87900b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 -- 172.21.15.40:0/3567863485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f879015f4d0 msgr2=0x7f87900b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f879015f4d0 0x7f87900b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 -- 172.21.15.40:0/3567863485 >> v1:172.21.15.184:6789/0 conn(0x7f87900d5640 legacy=0x7f87900b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.900+0000 7f878effd640 1 -- 172.21.15.40:0/3567863485 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900b2830 con 0x7f87900d2e20 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f878effd640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87900d2e20 0x7f87900d5100 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8784009870 tx=0x7f878402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3962c58425651c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f878e7fc640 1 -- 172.21.15.40:0/3567863485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784037070 con 0x7f87900d2e20 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f878e7fc640 1 -- 172.21.15.40:0/3567863485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8784032440 con 0x7f87900d2e20 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f878e7fc640 1 -- 172.21.15.40:0/3567863485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784032810 con 0x7f87900d2e20 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f8795e52640 1 -- 172.21.15.40:0/3567863485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87900d2e20 msgr2=0x7f87900d5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f8795e52640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87900d2e20 0x7f87900d5100 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8784009870 tx=0x7f878402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f8795e52640 1 -- 172.21.15.40:0/3567863485 shutdown_connections 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f8795e52640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87900d2e20 0x7f87900d5100 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.901+0000 7f8795e52640 1 --2- 172.21.15.40:0/3567863485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f879015f4d0 0x7f87900b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.902+0000 7f8795e52640 1 -- 172.21.15.40:0/3567863485 >> 172.21.15.40:0/3567863485 conn(0x7f879008ebf0 msgr2=0x7f879009e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.902+0000 7f8795e52640 1 -- 172.21.15.40:0/3567863485 shutdown_connections 2024-04-26T21:17:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.902+0000 7f8795e52640 1 -- 172.21.15.40:0/3567863485 wait complete. 2024-04-26T21:17:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.903+0000 7f8795e52640 1 Processor -- start 2024-04-26T21:17:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.903+0000 7f8795e52640 1 -- start start 2024-04-26T21:17:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87900d2e20 0x7f87900c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87900d5640 0x7f87900cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f87900b30d0 con 0x7f879015f4d0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87900b2df0 con 0x7f87900d2e20 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f8795e52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87900b4600 con 0x7f87900d5640 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58812/0 (socket says 172.21.15.40:58812) 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 -- 172.21.15.40:0/1760207642 learned_addr learned my addr 172.21.15.40:0/1760207642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878effd640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87900d5640 0x7f87900cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878f7fe640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87900d2e20 0x7f87900c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 -- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87900d5640 msgr2=0x7f87900cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87900d5640 0x7f87900cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 -- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87900d2e20 msgr2=0x7f87900c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87900d2e20 0x7f87900c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 -- 172.21.15.40:0/1760207642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900c9380 con 0x7f879015f4d0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.904+0000 7f878ffff640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f878800c7a0 tx=0x7f878800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.905+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788003b70 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.905+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788003d30 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.905+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87900c9660 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.905+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800aa90 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.905+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87900c9f00 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.906+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87900c9830 con 0x7f879015f4d0 2024-04-26T21:17:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.906+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f878800ac30 con 0x7f879015f4d0 2024-04-26T21:17:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.906+0000 7f8777fff640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.907+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f878808b980 con 0x7f879015f4d0 2024-04-26T21:17:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.907+0000 7f878f7fe640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.907+0000 7f878f7fe640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 0x7f877c06edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87800079a0 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:44.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:44.911+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8788059740 con 0x7f879015f4d0 2024-04-26T21:17:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.135+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f87900ca100 con 0x7f879015f4d0 2024-04-26T21:17:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.135+0000 7f8777fff640 1 -- 172.21.15.40:0/1760207642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f87900ca100 con 0x7f879015f4d0 2024-04-26T21:17:45.137 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:45.137 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:45.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 msgr2=0x7f877c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 0x7f877c06edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87800079a0 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 msgr2=0x7f8790161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f878800c7a0 tx=0x7f878800cc60 comp rx=0 tx=0).stop 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 shutdown_connections 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015f4d0 0x7f8790161b30 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87900d5640 0x7f87900cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 --2- 172.21.15.40:0/1760207642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87900d2e20 0x7f87900c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 >> 172.21.15.40:0/1760207642 conn(0x7f879008ebf0 msgr2=0x7f879006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 shutdown_connections 2024-04-26T21:17:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:45.137+0000 7f8795e52640 1 -- 172.21.15.40:0/1760207642 wait complete. 2024-04-26T21:17:45.149 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:45.149 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:47.883 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:49.885 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:50.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.261+0000 7fbc8a4bd640 1 Processor -- start 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.261+0000 7fbc8a4bd640 1 -- start start 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.262+0000 7fbc8a4bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8415f4c0 0x7fbc84163a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.262+0000 7fbc8a4bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc84073010 0x7fbc840d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.262+0000 7fbc8a4bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8406f2b0 con 0x7fbc8406d910 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.262+0000 7fbc8a4bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbc84068300 con 0x7fbc84073010 2024-04-26T21:17:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.262+0000 7fbc8a4bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8406f060 con 0x7fbc8415f4c0 2024-04-26T21:17:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc84073010 0x7fbc840d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc837fe640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fbc8406d910 0x7fbc84072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37270/0 (socket says 172.21.15.40:37270) 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc84073010 0x7fbc840d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:32906/0 (socket says 172.21.15.40:32906) 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc837fe640 1 -- 172.21.15.40:0/3260059379 learned_addr learned my addr 172.21.15.40:0/3260059379 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc83fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8415f4c0 0x7fbc84163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 -- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8415f4c0 msgr2=0x7fbc84163a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 --2- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8415f4c0 0x7fbc84163a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 -- 172.21.15.40:0/3260059379 >> v1:172.21.15.184:6789/0 conn(0x7fbc8406d910 legacy=0x7fbc84072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.263+0000 7fbc88a5a640 1 -- 172.21.15.40:0/3260059379 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc84162fa0 con 0x7fbc84073010 2024-04-26T21:17:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.264+0000 7fbc88a5a640 1 --2- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc84073010 0x7fbc840d02b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009d90 tx=0x7fbc7c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6588dfee64293530 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.264+0000 7fbc82ffd640 1 -- 172.21.15.40:0/3260059379 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c031600 con 0x7fbc84073010 2024-04-26T21:17:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.264+0000 7fbc82ffd640 1 -- 172.21.15.40:0/3260059379 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0317a0 con 0x7fbc84073010 2024-04-26T21:17:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.264+0000 7fbc82ffd640 1 -- 172.21.15.40:0/3260059379 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c031b70 con 0x7fbc84073010 2024-04-26T21:17:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc84073010 msgr2=0x7fbc840d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc84073010 0x7fbc840d02b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009d90 tx=0x7fbc7c02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/3260059379 shutdown_connections 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc84073010 0x7fbc840d02b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/3260059379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8415f4c0 0x7fbc84163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/3260059379 >> 172.21.15.40:0/3260059379 conn(0x7fbc840ce140 msgr2=0x7fbc840ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/3260059379 shutdown_connections 2024-04-26T21:17:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.265+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/3260059379 wait complete. 2024-04-26T21:17:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.266+0000 7fbc8a4bd640 1 Processor -- start 2024-04-26T21:17:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.266+0000 7fbc8a4bd640 1 -- start start 2024-04-26T21:17:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 0x7fbc841f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 0x7fbc84149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8406f590 con 0x7fbc84073010 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84163560 con 0x7fbc8415f4c0 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc8a4bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8406e960 con 0x7fbc8406d910 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58828/0 (socket says 172.21.15.40:58828) 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 -- 172.21.15.40:0/4102988724 learned_addr learned my addr 172.21.15.40:0/4102988724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc83fff640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 0x7fbc841f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc88a5a640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 0x7fbc84149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 -- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 msgr2=0x7fbc841f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 0x7fbc841f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 -- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 msgr2=0x7fbc84149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 0x7fbc84149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc837fe640 1 -- 172.21.15.40:0/4102988724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8414a620 con 0x7fbc84073010 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc83fff640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 0x7fbc841f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.267+0000 7fbc88a5a640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 0x7fbc84149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc837fe640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c430 tx=0x7fbc7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78017070 con 0x7fbc84073010 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc78005ce0 con 0x7fbc84073010 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8414a900 con 0x7fbc84073010 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780072f0 con 0x7fbc84073010 2024-04-26T21:17:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.268+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8414aec0 con 0x7fbc84073010 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.269+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc841499d0 con 0x7fbc84073010 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.269+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc7801a070 con 0x7fbc84073010 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.270+0000 7fbc80ff9640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 0x7fbc7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.270+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fbc7808b040 con 0x7fbc84073010 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.270+0000 7fbc83fff640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 0x7fbc7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.271+0000 7fbc83fff640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 0x7fbc7006ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009a20 tx=0x7fbc74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.274+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc841499d0 con 0x7fbc84073010 2024-04-26T21:17:50.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.498+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbc840696f0 con 0x7fbc84073010 2024-04-26T21:17:50.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.498+0000 7fbc80ff9640 1 -- 172.21.15.40:0/4102988724 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fbc780593c0 con 0x7fbc84073010 2024-04-26T21:17:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:50.499 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 msgr2=0x7fbc7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 0x7fbc7006ede0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009a20 tx=0x7fbc74008040 comp rx=0 tx=0).stop 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 msgr2=0x7fbc8414b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800c430 tx=0x7fbc7800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 shutdown_connections 2024-04-26T21:17:50.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc7006c920 0x7fbc7006ede0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8415f4c0 0x7fbc84149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc84073010 0x7fbc8414b6c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 --2- 172.21.15.40:0/4102988724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8406d910 0x7fbc841f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 >> 172.21.15.40:0/4102988724 conn(0x7fbc840ce140 msgr2=0x7fbc8406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 shutdown_connections 2024-04-26T21:17:50.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:50.500+0000 7fbc8a4bd640 1 -- 172.21.15.40:0/4102988724 wait complete. 2024-04-26T21:17:50.512 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:50.512 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:51.887 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:55.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:17:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.624+0000 7fefe64f1640 1 Processor -- start 2024-04-26T21:17:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.624+0000 7fefe64f1640 1 -- start start 2024-04-26T21:17:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe64f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefe015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe64f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fefe0070570 0x7fefe00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe64f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe015fce0 con 0x7fefe009be20 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe64f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fefe0160610 con 0x7fefe0070570 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe64f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefe00709b0 con 0x7fefe009ba40 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe4a8e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefe009be20 0x7fefe0162f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43168/0 (socket says 172.21.15.40:43168) 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefe4a8e640 1 -- 172.21.15.40:0/3741218071 learned_addr learned my addr 172.21.15.40:0/3741218071 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdffff640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefe015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3741218071 >> v2:172.21.15.184:3300/0 conn(0x7fefe0070570 0x7fefe00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449868372 0 0) 0x7fefe015fce0 con 0x7fefe009be20 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc8003520 con 0x7fefe009be20 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4015925542 0 0) 0x7fefc8003520 con 0x7fefe009be20 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 msgr2=0x7fefe015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefe015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 >> v2:172.21.15.184:3300/0 conn(0x7fefe0070570 msgr2=0x7fefe00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 --2- 172.21.15.40:0/3741218071 >> v2:172.21.15.184:3300/0 conn(0x7fefe0070570 0x7fefe00b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe015ffc0 con 0x7fefe009be20 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fefd80019f0 con 0x7fefe009be20 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.625+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 >> v1:172.21.15.40:6789/0 conn(0x7fefe009be20 legacy=0x7fefe0162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc80039b0 0x7fefc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0160610 con 0x7fefc8004800 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe00709b0 con 0x7fefc80039b0 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefc800b9c0 con 0x7fefe009ba40 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdffff640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc80039b0 0x7fefc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefe4a8e640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 -- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 msgr2=0x7fefc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe009ba40 0x7fefc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 -- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc80039b0 msgr2=0x7fefc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefc80039b0 0x7fefc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.626+0000 7fefdf7fe640 1 -- 172.21.15.40:0/3741218071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc8003520 con 0x7fefc8004800 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefc8004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009ac0 tx=0x7fefc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65316e318be8502 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4036070 con 0x7fefc8004800 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd4031400 con 0x7fefc8004800 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefdeffd640 1 -- 172.21.15.40:0/3741218071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4009690 con 0x7fefc8004800 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefe64f1640 1 -- 172.21.15.40:0/3741218071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 msgr2=0x7fefc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3741218071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefc8004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009ac0 tx=0x7fefc8005830 comp rx=0 tx=0).stop 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.627+0000 7fefe4a8e640 1 -- 172.21.15.40:0/3741218071 reap_dead start 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.628+0000 7fefe64f1640 1 -- 172.21.15.40:0/3741218071 shutdown_connections 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.628+0000 7fefe64f1640 1 -- 172.21.15.40:0/3741218071 >> 172.21.15.40:0/3741218071 conn(0x7fefe00b4a10 msgr2=0x7fefe00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.628+0000 7fefe64f1640 1 -- 172.21.15.40:0/3741218071 shutdown_connections 2024-04-26T21:17:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.628+0000 7fefe64f1640 1 -- 172.21.15.40:0/3741218071 wait complete. 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.629+0000 7fefe64f1640 1 Processor -- start 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.629+0000 7fefe64f1640 1 -- start start 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe01609a0 0x7fefe00cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 0x7fefe0162d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefe015f6a0 con 0x7fefc8004800 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe009c2b0 con 0x7fefe01609a0 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe64f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fefe015f820 con 0x7fefe00cfa30 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57786/0 (socket says 172.21.15.40:57786) 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 -- 172.21.15.40:0/3544633162 learned_addr learned my addr 172.21.15.40:0/3544633162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe4a8e640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 0x7fefe0162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 -- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 msgr2=0x7fefe0162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 0x7fefe0162d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 -- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe01609a0 msgr2=0x7fefe00cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe01609a0 0x7fefe00cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 -- 172.21.15.40:0/3544633162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe00ca300 con 0x7fefc8004800 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefe4a8e640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 0x7fefe0162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.630+0000 7fefdffff640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fefd0011910 tx=0x7fefd0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.631+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd000faf0 con 0x7fefc8004800 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.631+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd000fc90 con 0x7fefc8004800 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.631+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe00ca5e0 con 0x7fefc8004800 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.631+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0014440 con 0x7fefc8004800 2024-04-26T21:17:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.631+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe0162950 con 0x7fefc8004800 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.632+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe006a290 con 0x7fefc8004800 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.632+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefd0012040 con 0x7fefc8004800 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.633+0000 7fefdcff9640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 0x7fefcc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.633+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fefd008bfd0 con 0x7fefc8004800 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.633+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 0x7fefcc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:17:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.634+0000 7fefdf7fe640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 0x7fefcc06ede0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fefd4033000 tx=0x7fefd402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:17:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.637+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefd0019030 con 0x7fefc8004800 2024-04-26T21:17:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.860+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fefe00d0790 con 0x7fefc8004800 2024-04-26T21:17:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.860+0000 7fefdcff9640 1 -- 172.21.15.40:0/3544633162 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fefd005a470 con 0x7fefc8004800 2024-04-26T21:17:55.861 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:17:55.862 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:17:55.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.862+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 msgr2=0x7fefcc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.862+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 0x7fefcc06ede0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fefd4033000 tx=0x7fefd402e000 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.862+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 msgr2=0x7fefe0153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.862+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fefd0011910 tx=0x7fefd0011dd0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 shutdown_connections 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefcc06c920 0x7fefcc06ede0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fefe00cfa30 0x7fefe0162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe01609a0 0x7fefe00cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 --2- 172.21.15.40:0/3544633162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc8004800 0x7fefe0153550 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 >> 172.21.15.40:0/3544633162 conn(0x7fefe00b4a10 msgr2=0x7fefe015d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 shutdown_connections 2024-04-26T21:17:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:17:55.863+0000 7fefe64f1640 1 -- 172.21.15.40:0/3544633162 wait complete. 2024-04-26T21:17:55.874 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:17:55.874 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:17:55.889 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:57.890 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:17:59.892 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:00.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:18:00.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.987+0000 7f1989445640 1 Processor -- start 2024-04-26T21:18:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.987+0000 7f1989445640 1 -- start start 2024-04-26T21:18:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1989445640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19840d5810 0x7f19840b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1989445640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 0x7f19840b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1989445640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19840b2550 con 0x7f198415f4d0 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1989445640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19840b6180 con 0x7f19840d5810 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1989445640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f19840b2830 con 0x7f19840d2e20 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f19827fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19840d5810 0x7f19840b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1982ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f198415f4d0 0x7f19840d5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47344/0 (socket says 172.21.15.40:47344) 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.988+0000 7f1982ffd640 1 -- 172.21.15.40:0/734249183 learned_addr learned my addr 172.21.15.40:0/734249183 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19837fe640 1 --2- 172.21.15.40:0/734249183 >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 0x7f19840b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19827fc640 1 -- 172.21.15.40:0/734249183 >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 msgr2=0x7f19840b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19827fc640 1 --2- 172.21.15.40:0/734249183 >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 0x7f19840b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19827fc640 1 -- 172.21.15.40:0/734249183 >> v1:172.21.15.40:6789/0 conn(0x7f198415f4d0 legacy=0x7f19840d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:00.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19827fc640 1 -- 172.21.15.40:0/734249183 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19840b2b10 con 0x7f19840d5810 2024-04-26T21:18:00.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19837fe640 1 --2- 172.21.15.40:0/734249183 >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 0x7f19840b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.989+0000 7f19827fc640 1 --2- 172.21.15.40:0/734249183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d5810 0x7f19840b1ee0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1984162e60 tx=0x7f196c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c8d71ec0257c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1981ffb640 1 -- 172.21.15.40:0/734249183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c037070 con 0x7f19840d5810 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1981ffb640 1 -- 172.21.15.40:0/734249183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c032440 con 0x7f19840d5810 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1981ffb640 1 -- 172.21.15.40:0/734249183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c02ae60 con 0x7f19840d5810 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 -- 172.21.15.40:0/734249183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d5810 msgr2=0x7f19840b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 --2- 172.21.15.40:0/734249183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d5810 0x7f19840b1ee0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1984162e60 tx=0x7f196c02a110 comp rx=0 tx=0).stop 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 -- 172.21.15.40:0/734249183 shutdown_connections 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 --2- 172.21.15.40:0/734249183 >> v2:172.21.15.184:3300/0 conn(0x7f19840d2e20 0x7f19840b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 --2- 172.21.15.40:0/734249183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d5810 0x7f19840b1ee0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 -- 172.21.15.40:0/734249183 >> 172.21.15.40:0/734249183 conn(0x7f198408ebf0 msgr2=0x7f198409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 -- 172.21.15.40:0/734249183 shutdown_connections 2024-04-26T21:18:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.990+0000 7f1989445640 1 -- 172.21.15.40:0/734249183 wait complete. 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 Processor -- start 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 -- start start 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 0x7f198409eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f198415f4d0 0x7f19840ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19840b3b30 con 0x7f19840d2e20 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19840b30d0 con 0x7f19840d5810 2024-04-26T21:18:00.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1989445640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19840b3cb0 con 0x7f198415f4d0 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.992+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38698/0 (socket says 172.21.15.40:38698) 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 -- 172.21.15.40:0/2583246014 learned_addr learned my addr 172.21.15.40:0/2583246014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f19837fe640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f198415f4d0 0x7f19840ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f19827fc640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 0x7f198409eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 -- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f198415f4d0 msgr2=0x7f19840ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f198415f4d0 0x7f19840ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 -- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 msgr2=0x7f198409eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 0x7f198409eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 -- 172.21.15.40:0/2583246014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198409f760 con 0x7f19840d2e20 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f19827fc640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 0x7f198409eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:00.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1982ffd640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c3c0 tx=0x7f197c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c017070 con 0x7f19840d2e20 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c005ce0 con 0x7f19840d2e20 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19840a0e90 con 0x7f19840d2e20 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0072f0 con 0x7f19840d2e20 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.993+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19840a1450 con 0x7f19840d2e20 2024-04-26T21:18:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.994+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198406a290 con 0x7f19840d2e20 2024-04-26T21:18:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.995+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f197c01a070 con 0x7f19840d2e20 2024-04-26T21:18:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.995+0000 7f197b7fe640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 0x7f196406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.995+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f197c08aec0 con 0x7f19840d2e20 2024-04-26T21:18:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.996+0000 7f19827fc640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 0x7f196406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:00.996+0000 7f19827fc640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 0x7f196406edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f196c034000 tx=0x7f196c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.000+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f197c0592d0 con 0x7f19840d2e20 2024-04-26T21:18:01.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.226+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f19840a2e80 con 0x7f19840d2e20 2024-04-26T21:18:01.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.226+0000 7f197b7fe640 1 -- 172.21.15.40:0/2583246014 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f197c058bd0 con 0x7f19840d2e20 2024-04-26T21:18:01.227 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:01.227 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 msgr2=0x7f196406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 0x7f196406edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f196c034000 tx=0x7f196c02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 msgr2=0x7f1984155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c3c0 tx=0x7f197c00c880 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 shutdown_connections 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f196406c8f0 0x7f196406edb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f198415f4d0 0x7f19840ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19840d5810 0x7f198409eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 --2- 172.21.15.40:0/2583246014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840d2e20 0x7f1984155f10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 >> 172.21.15.40:0/2583246014 conn(0x7f198408ebf0 msgr2=0x7f19840d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 shutdown_connections 2024-04-26T21:18:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:01.229+0000 7f1989445640 1 -- 172.21.15.40:0/2583246014 wait complete. 2024-04-26T21:18:01.240 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:18:01.240 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:18:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:02.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:18:02.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:18:03.896 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:05.898 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:06.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:18:06.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.352+0000 7facca91b640 1 Processor -- start 2024-04-26T21:18:06.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 -- start start 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facc415f4d0 0x7facc40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 0x7facc40b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc40b2550 con 0x7facc40d2e20 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7facc40b6180 con 0x7facc415f4d0 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.353+0000 7facca91b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7facc40b2830 con 0x7facc40d5810 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facc415f4d0 0x7facc40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7facc40d2e20 0x7facc40d5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47358/0 (socket says 172.21.15.40:47358) 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc37fe640 1 -- 172.21.15.40:0/2713015282 learned_addr learned my addr 172.21.15.40:0/2713015282 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc8eb8640 1 --2- 172.21.15.40:0/2713015282 >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 0x7facc40b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc3fff640 1 -- 172.21.15.40:0/2713015282 >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 msgr2=0x7facc40b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:06.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc3fff640 1 --2- 172.21.15.40:0/2713015282 >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 0x7facc40b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc3fff640 1 -- 172.21.15.40:0/2713015282 >> v1:172.21.15.40:6789/0 conn(0x7facc40d2e20 legacy=0x7facc40d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc3fff640 1 -- 172.21.15.40:0/2713015282 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40b2b10 con 0x7facc415f4d0 2024-04-26T21:18:06.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc2ffd640 1 -- 172.21.15.40:0/2713015282 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849426537 0 0) 0x7facc40b2550 con 0x7facc40d2e20 2024-04-26T21:18:06.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.354+0000 7facc8eb8640 1 --2- 172.21.15.40:0/2713015282 >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 0x7facc40b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facc3fff640 1 --2- 172.21.15.40:0/2713015282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc415f4d0 0x7facc40b1ee0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7facc4162e60 tx=0x7facb402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33f474d11c9b7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facc2ffd640 1 -- 172.21.15.40:0/2713015282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4037070 con 0x7facc415f4d0 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facc2ffd640 1 -- 172.21.15.40:0/2713015282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb402add0 con 0x7facc415f4d0 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facc2ffd640 1 -- 172.21.15.40:0/2713015282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4032660 con 0x7facc415f4d0 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 -- 172.21.15.40:0/2713015282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc415f4d0 msgr2=0x7facc40b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 --2- 172.21.15.40:0/2713015282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc415f4d0 0x7facc40b1ee0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7facc4162e60 tx=0x7facb402a080 comp rx=0 tx=0).stop 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 -- 172.21.15.40:0/2713015282 shutdown_connections 2024-04-26T21:18:06.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 --2- 172.21.15.40:0/2713015282 >> v2:172.21.15.184:3300/0 conn(0x7facc40d5810 0x7facc40b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 --2- 172.21.15.40:0/2713015282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc415f4d0 0x7facc40b1ee0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.355+0000 7facca91b640 1 -- 172.21.15.40:0/2713015282 >> 172.21.15.40:0/2713015282 conn(0x7facc408ebf0 msgr2=0x7facc409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.356+0000 7facca91b640 1 -- 172.21.15.40:0/2713015282 shutdown_connections 2024-04-26T21:18:06.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.356+0000 7facca91b640 1 -- 172.21.15.40:0/2713015282 wait complete. 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.357+0000 7facca91b640 1 Processor -- start 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.357+0000 7facca91b640 1 -- start start 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facc40d2e20 0x7facc4155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc415f4d0 0x7facc4162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b3b30 con 0x7facc40d5810 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b30d0 con 0x7facc415f4d0 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7facc40b3cb0 con 0x7facc40d2e20 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38714/0 (socket says 172.21.15.40:38714) 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 -- 172.21.15.40:0/18990128 learned_addr learned my addr 172.21.15.40:0/18990128 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc3fff640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facc40d2e20 0x7facc4155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc8eb8640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc415f4d0 0x7facc4162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 -- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facc40d2e20 msgr2=0x7facc4155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facc40d2e20 0x7facc4155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 -- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc415f4d0 msgr2=0x7facc4162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc415f4d0 0x7facc4162060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 -- 172.21.15.40:0/18990128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41555b0 con 0x7facc40d5810 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc37fe640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7facac00c4d0 tx=0x7facac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac018070 con 0x7facc40d5810 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facac007500 con 0x7facc40d5810 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc4160220 con 0x7facc40d5810 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.358+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac004a20 con 0x7facc40d5810 2024-04-26T21:18:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.359+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41607e0 con 0x7facc40d5810 2024-04-26T21:18:06.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.359+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc406a290 con 0x7facc40d5810 2024-04-26T21:18:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.360+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facac01b070 con 0x7facc40d5810 2024-04-26T21:18:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.360+0000 7facc0ff9640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.360+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7facac08ae70 con 0x7facc40d5810 2024-04-26T21:18:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.361+0000 7facc3fff640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.361+0000 7facc3fff640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 0x7faca406edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7facb4034000 tx=0x7facb402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.365+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7facac059280 con 0x7facc40d5810 2024-04-26T21:18:06.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.589+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7facc40b5cb0 con 0x7facc40d5810 2024-04-26T21:18:06.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.589+0000 7facc0ff9640 1 -- 172.21.15.40:0/18990128 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7facac058b80 con 0x7facc40d5810 2024-04-26T21:18:06.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:06.590 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 msgr2=0x7faca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 0x7faca406edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7facb4034000 tx=0x7facb402e000 comp rx=0 tx=0).stop 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 msgr2=0x7facc415f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7facac00c4d0 tx=0x7facac00c990 comp rx=0 tx=0).stop 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 shutdown_connections 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facc415f4d0 0x7facc4162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d5810 0x7facc415f8b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 --2- 172.21.15.40:0/18990128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7facc40d2e20 0x7facc4155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:06.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 >> 172.21.15.40:0/18990128 conn(0x7facc408ebf0 msgr2=0x7facc40d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:06.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 shutdown_connections 2024-04-26T21:18:06.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:06.591+0000 7facca91b640 1 -- 172.21.15.40:0/18990128 wait complete. 2024-04-26T21:18:06.603 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:18:06.603 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:18:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:09.903 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:11.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:18:11.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.715+0000 7f3f04b78640 1 Processor -- start 2024-04-26T21:18:11.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.715+0000 7f3f04b78640 1 -- start start 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3f04b78640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f00070570 0x7f3f000b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3f04b78640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f0015fce0 con 0x7f3f0015f6a0 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3f04b78640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f0015ffc0 con 0x7f3f0009ba40 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3f04b78640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3f00160610 con 0x7f3f00070570 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3efdd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f0015f6a0 0x7f3f000d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46142/0 (socket says 172.21.15.40:46142) 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3efdd9b640 1 -- 172.21.15.40:0/1533463854 learned_addr learned my addr 172.21.15.40:0/1533463854 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3efed9d640 1 --2- 172.21.15.40:0/1533463854 >> v2:172.21.15.184:3300/0 conn(0x7f3f00070570 0x7f3f000b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716801401 0 0) 0x7f3f0015fce0 con 0x7f3f0015f6a0 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.716+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee8003520 con 0x7f3f0015f6a0 2024-04-26T21:18:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 854090825 0 0) 0x7f3ee8003520 con 0x7f3f0015f6a0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 >> v2:172.21.15.184:3300/0 conn(0x7f3f00070570 msgr2=0x7f3f000b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 --2- 172.21.15.40:0/1533463854 >> v2:172.21.15.184:3300/0 conn(0x7f3f00070570 0x7f3f000b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 >> v1:172.21.15.184:6789/0 conn(0x7f3f0009ba40 legacy=0x7f3f0009be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001602a0 con 0x7f3f0015f6a0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3ef4001670 con 0x7f3f0015f6a0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 >> v1:172.21.15.40:6789/0 conn(0x7f3f0015f6a0 legacy=0x7f3f000d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee8004800 0x7f3ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 0x7f3ee800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00160610 con 0x7f3ee80051c0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee800c060 con 0x7f3ee80039b0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee800c260 con 0x7f3ee8004800 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 0x7f3ee800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efdd9b640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee8004800 0x7f3ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efe59c640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 -- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee8004800 msgr2=0x7f3ee8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee8004800 0x7f3ee8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 -- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ee80039b0 msgr2=0x7f3ee8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.717+0000 7f3efed9d640 1 -- 172.21.15.40:0/1533463854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee8003520 con 0x7f3ee80051c0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.718+0000 7f3efed9d640 1 -- 172.21.15.40:0/1533463854 reap_dead start 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.718+0000 7f3efed9d640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 0x7f3ee800b9c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fc0 tx=0x7f3ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10d5af46369d8eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.718+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8035020 con 0x7f3ee80051c0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.718+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef8003bd0 con 0x7f3ee80051c0 2024-04-26T21:18:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.718+0000 7f3efd59a640 1 -- 172.21.15.40:0/1533463854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8009c10 con 0x7f3ee80051c0 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 -- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 msgr2=0x7f3ee800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 0x7f3ee800b9c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fc0 tx=0x7f3ee8005830 comp rx=0 tx=0).stop 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 -- 172.21.15.40:0/1533463854 shutdown_connections 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 --2- 172.21.15.40:0/1533463854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee80051c0 0x7f3ee800b9c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 -- 172.21.15.40:0/1533463854 >> 172.21.15.40:0/1533463854 conn(0x7f3f000b4a10 msgr2=0x7f3f000b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 -- 172.21.15.40:0/1533463854 shutdown_connections 2024-04-26T21:18:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.719+0000 7f3f04b78640 1 -- 172.21.15.40:0/1533463854 wait complete. 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.720+0000 7f3f04b78640 1 Processor -- start 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.720+0000 7f3f04b78640 1 -- start start 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee80051c0 0x7f3f001f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 0x7f3f000b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b58c0 con 0x7f3f000cfbc0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b5740 con 0x7f3f000b7ca0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3f04b78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0009be20 con 0x7f3ee80051c0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40106/0 (socket says 172.21.15.40:40106) 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 -- 172.21.15.40:0/401300573 learned_addr learned my addr 172.21.15.40:0/401300573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efe59c640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee80051c0 0x7f3f001f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efed9d640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 0x7f3f000b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 -- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee80051c0 msgr2=0x7f3f001f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee80051c0 0x7f3f001f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 -- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 msgr2=0x7f3f000b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 0x7f3f000b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efdd9b640 1 -- 172.21.15.40:0/401300573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000b79e0 con 0x7f3f000cfbc0 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.721+0000 7f3efed9d640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 0x7f3f000b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3efdd9b640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40072c0 tx=0x7f3ef4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4001660 con 0x7f3f000cfbc0 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef4018070 con 0x7f3f000cfbc0 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f000cee00 con 0x7f3f000cfbc0 2024-04-26T21:18:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400fb80 con 0x7f3f000cfbc0 2024-04-26T21:18:11.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.722+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f000cf3c0 con 0x7f3f000cfbc0 2024-04-26T21:18:11.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.723+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0006a290 con 0x7f3f000cfbc0 2024-04-26T21:18:11.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.723+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ef400fd20 con 0x7f3f000cfbc0 2024-04-26T21:18:11.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.724+0000 7f3eeeffd640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 0x7f3ee006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:11.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.724+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f3ef408bb40 con 0x7f3f000cfbc0 2024-04-26T21:18:11.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.724+0000 7f3efe59c640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 0x7f3ee006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:11.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.725+0000 7f3efe59c640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 0x7f3ee006ef50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:11.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.729+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ef4059fe0 con 0x7f3f000cfbc0 2024-04-26T21:18:11.904 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.953+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3f0015afd0 con 0x7f3f000cfbc0 2024-04-26T21:18:11.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.954+0000 7f3eeeffd640 1 -- 172.21.15.40:0/401300573 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3ef405ee20 con 0x7f3f000cfbc0 2024-04-26T21:18:11.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:11.955 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 msgr2=0x7f3ee006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 0x7f3ee006ef50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef0009040 comp rx=0 tx=0).stop 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 msgr2=0x7f3f000b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40072c0 tx=0x7f3ef4007780 comp rx=0 tx=0).stop 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 shutdown_connections 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3ee006ca90 0x7f3ee006ef50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f000b7ca0 0x7f3f000b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f000cfbc0 0x7f3f000b6ef0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 --2- 172.21.15.40:0/401300573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ee80051c0 0x7f3f001f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:11.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 >> 172.21.15.40:0/401300573 conn(0x7f3f000b4a10 msgr2=0x7f3f0006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:11.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 shutdown_connections 2024-04-26T21:18:11.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:11.956+0000 7f3f04b78640 1 -- 172.21.15.40:0/401300573 wait complete. 2024-04-26T21:18:11.967 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:18:11.968 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:18:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:16.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:18:17.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.080+0000 7f0f8b9c6640 1 Processor -- start 2024-04-26T21:18:17.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 -- start start 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f8415f4d0 0x7f0f840b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f840d2e20 0x7f0f840d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f840d5ab0 con 0x7f0f840d5640 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0f840b6180 con 0x7f0f8415f4d0 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f8b9c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f840b2550 con 0x7f0f840d2e20 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.081+0000 7f0f89f63640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f840d5640 0x7f0f840b4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46146/0 (socket says 172.21.15.40:46146) 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f89f63640 1 -- 172.21.15.40:0/955771017 learned_addr learned my addr 172.21.15.40:0/955771017 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:17.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f89762640 1 --2- 172.21.15.40:0/955771017 >> v2:172.21.15.184:3300/0 conn(0x7f0f8415f4d0 0x7f0f840b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f840d2e20 0x7f0f840d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470071477 0 0) 0x7f0f840d5ab0 con 0x7f0f840d5640 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f6c003520 con 0x7f0f840d5640 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2596827280 0 0) 0x7f0f6c003520 con 0x7f0f840d5640 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f840d2e20 msgr2=0x7f0f840d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f840d2e20 0x7f0f840d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 >> v2:172.21.15.184:3300/0 conn(0x7f0f8415f4d0 msgr2=0x7f0f840b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 --2- 172.21.15.40:0/955771017 >> v2:172.21.15.184:3300/0 conn(0x7f0f8415f4d0 0x7f0f840b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840b2830 con 0x7f0f840d5640 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f0f740019f0 con 0x7f0f840d5640 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f89762640 1 --2- 172.21.15.40:0/955771017 >> v2:172.21.15.184:3300/0 conn(0x7f0f8415f4d0 0x7f0f840b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 >> v1:172.21.15.40:6789/0 conn(0x7f0f840d5640 legacy=0x7f0f840b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f6c0039b0 0x7f0f6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f6c004800 0x7f0f6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c0051c0 0x7f0f6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840b6180 con 0x7f0f6c004800 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840b2550 con 0x7f0f6c0039b0 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c00bf30 con 0x7f0f6c0051c0 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f840d2e20 0x7f0f840d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f89762640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f6c0039b0 0x7f0f6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f6c004800 0x7f0f6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.082+0000 7f0f89f63640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c0051c0 0x7f0f6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 -- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c0051c0 msgr2=0x7f0f6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c0051c0 0x7f0f6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 -- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f6c0039b0 msgr2=0x7f0f6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f6c0039b0 0x7f0f6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 -- 172.21.15.40:0/955771017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c003520 con 0x7f0f6c004800 2024-04-26T21:18:17.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f89762640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f6c0039b0 0x7f0f6c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.083+0000 7f0f88f61640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f6c004800 0x7f0f6c004c80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0f8408eb80 tx=0x7f0f6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b84d26b8c3a451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78036070 con 0x7f0f6c004800 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f78004030 con 0x7f0f6c004800 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f83fff640 1 -- 172.21.15.40:0/955771017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780097d0 con 0x7f0f6c004800 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/955771017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f6c004800 msgr2=0x7f0f6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/955771017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f6c004800 0x7f0f6c004c80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0f8408eb80 tx=0x7f0f6c005830 comp rx=0 tx=0).stop 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f89f63640 1 -- 172.21.15.40:0/955771017 reap_dead start 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/955771017 shutdown_connections 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/955771017 >> 172.21.15.40:0/955771017 conn(0x7f0f8408ebf0 msgr2=0x7f0f8409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/955771017 shutdown_connections 2024-04-26T21:18:17.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.084+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/955771017 wait complete. 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 Processor -- start 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 -- start start 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c004800 0x7f0f8415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 0x7f0f84160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840b30d0 con 0x7f0f841610e0 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840b2df0 con 0x7f0f8415d280 2024-04-26T21:18:17.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f8b9c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f840b4600 con 0x7f0f6c004800 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f89f63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.086+0000 7f0f89f63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40124/0 (socket says 172.21.15.40:40124) 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 -- 172.21.15.40:0/1050880276 learned_addr learned my addr 172.21.15.40:0/1050880276 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89762640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c004800 0x7f0f8415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f88f61640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 0x7f0f84160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 -- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c004800 msgr2=0x7f0f8415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c004800 0x7f0f8415caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 -- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 msgr2=0x7f0f84160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 0x7f0f84160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 -- 172.21.15.40:0/1050880276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8415fb00 con 0x7f0f841610e0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f88f61640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 0x7f0f84160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f89f63640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0f840cf830 tx=0x7f0f74007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c009860 con 0x7f0f841610e0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f74018070 con 0x7f0f841610e0 2024-04-26T21:18:17.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8415fde0 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400ae20 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.087+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841603a0 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.089+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8406a290 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.089+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f7401d460 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.089+0000 7f0f81ffb640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 0x7f0f7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.089+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f0f7408c010 con 0x7f0f841610e0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.089+0000 7f0f89762640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 0x7f0f7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.090+0000 7f0f89762640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 0x7f0f7006ede0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e30 tx=0x7f0f7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.094+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f7405a420 con 0x7f0f841610e0 2024-04-26T21:18:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.322+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0f840d2520 con 0x7f0f841610e0 2024-04-26T21:18:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.323+0000 7f0f81ffb640 1 -- 172.21.15.40:0/1050880276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f0f74059d20 con 0x7f0f841610e0 2024-04-26T21:18:17.324 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:17.324 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 msgr2=0x7f0f7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 0x7f0f7006ede0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e30 tx=0x7f0f7c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 msgr2=0x7f0f84161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0f840cf830 tx=0x7f0f74007b00 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 shutdown_connections 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0f7006c920 0x7f0f7006ede0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f841610e0 0x7f0f84161770 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f8415d280 0x7f0f84160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 --2- 172.21.15.40:0/1050880276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f6c004800 0x7f0f8415caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 >> 172.21.15.40:0/1050880276 conn(0x7f0f8408ebf0 msgr2=0x7f0f840d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 shutdown_connections 2024-04-26T21:18:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:17.325+0000 7f0f8b9c6640 1 -- 172.21.15.40:0/1050880276 wait complete. 2024-04-26T21:18:17.337 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:18:17.337 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:18:17.911 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:19.913 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:22.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:18:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 Processor -- start 2024-04-26T21:18:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 -- start start 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 0x7fb8b40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8b40d2e20 0x7fb8b40d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b40b2550 con 0x7fb8b40d5640 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b40b6180 con 0x7fb8b40d2e20 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.449+0000 7fb8ba3af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b40b2830 con 0x7fb8b415f4d0 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8b40d2e20 0x7fb8b40d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b894c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8b40d5640 0x7fb8b40b4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60842/0 (socket says 172.21.15.40:60842) 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b894c640 1 -- 172.21.15.40:0/95287220 learned_addr learned my addr 172.21.15.40:0/95287220 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b3fff640 1 --2- 172.21.15.40:0/95287220 >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 0x7fb8b40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b37fe640 1 -- 172.21.15.40:0/95287220 >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 msgr2=0x7fb8b40b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b37fe640 1 --2- 172.21.15.40:0/95287220 >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 0x7fb8b40b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b37fe640 1 -- 172.21.15.40:0/95287220 >> v1:172.21.15.40:6789/0 conn(0x7fb8b40d5640 legacy=0x7fb8b40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b37fe640 1 -- 172.21.15.40:0/95287220 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b40b2b10 con 0x7fb8b40d2e20 2024-04-26T21:18:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.450+0000 7fb8b3fff640 1 --2- 172.21.15.40:0/95287220 >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 0x7fb8b40b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8b37fe640 1 --2- 172.21.15.40:0/95287220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b40d5100 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb8b408f070 tx=0x7fb8a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60837794ac3339ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8b2ffd640 1 -- 172.21.15.40:0/95287220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8037070 con 0x7fb8b40d2e20 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8b2ffd640 1 -- 172.21.15.40:0/95287220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a8032440 con 0x7fb8b40d2e20 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8b2ffd640 1 -- 172.21.15.40:0/95287220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802ae60 con 0x7fb8b40d2e20 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 -- 172.21.15.40:0/95287220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 msgr2=0x7fb8b40d5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/95287220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b40d5100 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb8b408f070 tx=0x7fb8a802a110 comp rx=0 tx=0).stop 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 -- 172.21.15.40:0/95287220 shutdown_connections 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/95287220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b40d5100 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/95287220 >> v2:172.21.15.184:3300/0 conn(0x7fb8b415f4d0 0x7fb8b40b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.451+0000 7fb8ba3af640 1 -- 172.21.15.40:0/95287220 >> 172.21.15.40:0/95287220 conn(0x7fb8b408ebf0 msgr2=0x7fb8b409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.452+0000 7fb8ba3af640 1 -- 172.21.15.40:0/95287220 shutdown_connections 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.452+0000 7fb8ba3af640 1 -- 172.21.15.40:0/95287220 wait complete. 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.453+0000 7fb8ba3af640 1 Processor -- start 2024-04-26T21:18:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.453+0000 7fb8ba3af640 1 -- start start 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8b40d5640 0x7fb8b406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 0x7fb8b406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b40d5a70 con 0x7fb8b40d2e20 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b40b30d0 con 0x7fb8b415f4d0 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b40b4600 con 0x7fb8b40d5640 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43492/0 (socket says 172.21.15.40:43492) 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 -- 172.21.15.40:0/1747114735 learned_addr learned my addr 172.21.15.40:0/1747114735 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b37fe640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8b40d5640 0x7fb8b406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b894c640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 0x7fb8b406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 -- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8b40d5640 msgr2=0x7fb8b406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8b40d5640 0x7fb8b406e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 -- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 msgr2=0x7fb8b406e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 0x7fb8b406e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 -- 172.21.15.40:0/1747114735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b415c850 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b894c640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 0x7fb8b406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b3fff640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400c820 tx=0x7fb8a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4015070 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a4007a30 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007de0 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.454+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b40c8fd0 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.455+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b40c9590 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.455+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b406a290 con 0x7fb8b40d2e20 2024-04-26T21:18:22.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.456+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8a4018070 con 0x7fb8b40d2e20 2024-04-26T21:18:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.456+0000 7fb8b0ff9640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.456+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fb8a408b980 con 0x7fb8b40d2e20 2024-04-26T21:18:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.457+0000 7fb8b37fe640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.457+0000 7fb8b37fe640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8034000 tx=0x7fb8a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:22.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.461+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8a4059e20 con 0x7fb8b40d2e20 2024-04-26T21:18:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.684+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb8b40b63c0 con 0x7fb8b40d2e20 2024-04-26T21:18:22.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.685+0000 7fb8b0ff9640 1 -- 172.21.15.40:0/1747114735 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb8a4059720 con 0x7fb8b40d2e20 2024-04-26T21:18:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:22.686 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 msgr2=0x7fb8a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8034000 tx=0x7fb8a802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 msgr2=0x7fb8b415c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400c820 tx=0x7fb8a400cce0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 shutdown_connections 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8b415f4d0 0x7fb8b406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8b40d5640 0x7fb8b406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 --2- 172.21.15.40:0/1747114735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b40d2e20 0x7fb8b415c310 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 >> 172.21.15.40:0/1747114735 conn(0x7fb8b408ebf0 msgr2=0x7fb8b409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:22.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 shutdown_connections 2024-04-26T21:18:22.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22.687+0000 7fb8ba3af640 1 -- 172.21.15.40:0/1747114735 wait complete. 2024-04-26T21:18:22.698 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-26T21:18:22.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:22 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-26T21:18:23.917 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:18:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:stdin closed, goodbye! 2024-04-26T21:18:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:23.938 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:18:23.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:24.022 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:24.063 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:18:24.063 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:24.064 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi040') 2024-04-26T21:18:24.064 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:24.064 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:24.082 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:24.082 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:24.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.287+0000 7feed54dc640 1 Processor -- start 2024-04-26T21:18:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.287+0000 7feed54dc640 1 -- start start 2024-04-26T21:18:24.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feed54dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 0x7feed015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feed54dc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00d1740 con 0x7feed00cf2e0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feed54dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00d1a20 con 0x7feed00d11b0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feed54dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feed00d1d00 con 0x7feed006e490 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feeceffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feed00cf2e0 0x7feed006d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60848/0 (socket says 172.21.15.40:60848) 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feeceffd640 1 -- 172.21.15.40:0/4173688195 learned_addr learned my addr 172.21.15.40:0/4173688195 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feecf7fe640 1 --2- 172.21.15.40:0/4173688195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 0x7feed015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065149367 0 0) 0x7feed00d1740 con 0x7feed00cf2e0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8003520 con 0x7feed00cf2e0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238210292 0 0) 0x7feed00d1a20 con 0x7feed00d11b0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.288+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feed00d1740 con 0x7feed00d11b0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.289+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1649213130 0 0) 0x7feeb8003520 con 0x7feed00cf2e0 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.289+0000 7feecf7fe640 1 -- 172.21.15.40:0/4173688195 >> v1:172.21.15.184:6789/0 conn(0x7feed00d11b0 legacy=0x7feed006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.289+0000 7feecf7fe640 1 -- 172.21.15.40:0/4173688195 >> v1:172.21.15.40:6789/0 conn(0x7feed00cf2e0 legacy=0x7feed006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.289+0000 7feecf7fe640 1 -- 172.21.15.40:0/4173688195 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00d1fe0 con 0x7feed006e490 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.289+0000 7feecf7fe640 1 --2- 172.21.15.40:0/4173688195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 0x7feed015ebe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feec8009df0 tx=0x7feec802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230a84bd02f7bce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0001670 con 0x7feed006e490 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8007540 con 0x7feed006e490 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feecdffb640 1 -- 172.21.15.40:0/4173688195 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80076f0 con 0x7feed006e490 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 -- 172.21.15.40:0/4173688195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 msgr2=0x7feed015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 --2- 172.21.15.40:0/4173688195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 0x7feed015ebe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feec8009df0 tx=0x7feec802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 -- 172.21.15.40:0/4173688195 shutdown_connections 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 --2- 172.21.15.40:0/4173688195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006e490 0x7feed015ebe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 -- 172.21.15.40:0/4173688195 >> 172.21.15.40:0/4173688195 conn(0x7feed00b4a60 msgr2=0x7feed00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 -- 172.21.15.40:0/4173688195 shutdown_connections 2024-04-26T21:18:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.290+0000 7feed54dc640 1 -- 172.21.15.40:0/4173688195 wait complete. 2024-04-26T21:18:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.291+0000 7feed54dc640 1 Processor -- start 2024-04-26T21:18:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 -- start start 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed006e490 0x7feed0156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed00d11b0 0x7feed0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feed00cf840 con 0x7feed00cf2e0 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feed00cf6c0 con 0x7feed006e490 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feed54dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feed00ce660 con 0x7feed00d11b0 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feece7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feece7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43504/0 (socket says 172.21.15.40:43504) 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feece7fc640 1 -- 172.21.15.40:0/2700069425 learned_addr learned my addr 172.21.15.40:0/2700069425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.292+0000 7feecf7fe640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed00d11b0 0x7feed0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feeceffd640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed006e490 0x7feed0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 -- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed00d11b0 msgr2=0x7feed0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed00d11b0 0x7feed0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 -- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed006e490 msgr2=0x7feed0156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed006e490 0x7feed0156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 -- 172.21.15.40:0/2700069425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00b6b10 con 0x7feed00cf2e0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feece7fc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feec400c430 tx=0x7feec400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4017070 con 0x7feed00cf2e0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec40052f0 con 0x7feed00cf2e0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4003830 con 0x7feed00cf2e0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed00b6df0 con 0x7feed00cf2e0 2024-04-26T21:18:24.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.293+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed00b73b0 con 0x7feed00cf2e0 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.294+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed006a290 con 0x7feed00cf2e0 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.295+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feec401a070 con 0x7feed00cf2e0 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.295+0000 7feeb77fe640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 0x7feebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.295+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7feec408af00 con 0x7feed00cf2e0 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.295+0000 7feeceffd640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 0x7feebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.296+0000 7feeceffd640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 0x7feebc06edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feec00017e0 tx=0x7feec0001610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.299+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feec4058cd0 con 0x7feed00cf2e0 2024-04-26T21:18:24.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.486+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feed0070330 con 0x7feed00cf2e0 2024-04-26T21:18:24.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.487+0000 7feeb77fe640 1 -- 172.21.15.40:0/2700069425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16710 (secure 0 0 0) 0x7feec405cfd0 con 0x7feed00cf2e0 2024-04-26T21:18:24.488 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:24.488 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:16:42.760777+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:24.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 msgr2=0x7feebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 0x7feebc06edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feec00017e0 tx=0x7feec0001610 comp rx=0 tx=0).stop 2024-04-26T21:18:24.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 msgr2=0x7feed00b81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feec400c430 tx=0x7feec400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 shutdown_connections 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feebc06c8f0 0x7feebc06edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed00d11b0 0x7feed0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed00cf2e0 0x7feed00b81e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 --2- 172.21.15.40:0/2700069425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed006e490 0x7feed0156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 >> 172.21.15.40:0/2700069425 conn(0x7feed00b4a60 msgr2=0x7feed006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 shutdown_connections 2024-04-26T21:18:24.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.489+0000 7feed54dc640 1 -- 172.21.15.40:0/2700069425 wait complete. 2024-04-26T21:18:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:55.191765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:55.661024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:24.501 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.604+0000 7f5ff2e74640 1 Processor -- start 2024-04-26T21:18:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.604+0000 7f5ff2e74640 1 -- start start 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec143840 0x7f5fec0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fec0b4520 0x7f5fec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 0x7f5fec160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec143c20 con 0x7f5fec0b4520 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec15dca0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff2e74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec15df80 con 0x7f5fec143840 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fec0b4520 0x7f5fec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fec0b4520 0x7f5fec0b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43518/0 (socket says 172.21.15.40:43518) 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 -- 172.21.15.40:0/3933326816 learned_addr learned my addr 172.21.15.40:0/3933326816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff0c10640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec143840 0x7f5fec0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 -- 172.21.15.40:0/3933326816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec143840 msgr2=0x7f5fec0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec143840 0x7f5fec0b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff1411640 1 --2- 172.21.15.40:0/3933326816 >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 0x7f5fec160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 -- 172.21.15.40:0/3933326816 >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 msgr2=0x7f5fec160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 --2- 172.21.15.40:0/3933326816 >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 0x7f5fec160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5febfff640 1 -- 172.21.15.40:0/3933326816 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec15e260 con 0x7f5fec0b4520 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.605+0000 7f5ff1411640 1 --2- 172.21.15.40:0/3933326816 >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 0x7f5fec160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.606+0000 7f5febfff640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b4520 0x7f5fec0b4eb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5fec070480 tx=0x7f5fd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120444ec648145ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.606+0000 7f5feb7fe640 1 -- 172.21.15.40:0/3933326816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8037070 con 0x7f5fec0b4520 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.606+0000 7f5feb7fe640 1 -- 172.21.15.40:0/3933326816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd8032440 con 0x7f5fec0b4520 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.606+0000 7f5feb7fe640 1 -- 172.21.15.40:0/3933326816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8032810 con 0x7f5fec0b4520 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 -- 172.21.15.40:0/3933326816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b4520 msgr2=0x7f5fec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b4520 0x7f5fec0b4eb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5fec070480 tx=0x7f5fd802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 -- 172.21.15.40:0/3933326816 shutdown_connections 2024-04-26T21:18:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/3933326816 >> v2:172.21.15.184:3300/0 conn(0x7f5fec0b53f0 0x7f5fec160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b4520 0x7f5fec0b4eb0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/3933326816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec143840 0x7f5fec0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 -- 172.21.15.40:0/3933326816 >> 172.21.15.40:0/3933326816 conn(0x7f5fec0cd8a0 msgr2=0x7f5fec0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:24.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 -- 172.21.15.40:0/3933326816 shutdown_connections 2024-04-26T21:18:24.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.607+0000 7f5ff2e74640 1 -- 172.21.15.40:0/3933326816 wait complete. 2024-04-26T21:18:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.608+0000 7f5ff2e74640 1 Processor -- start 2024-04-26T21:18:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.608+0000 7f5ff2e74640 1 -- start start 2024-04-26T21:18:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b4520 0x7f5fec15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec143840 0x7f5fec141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec15da40 con 0x7f5fec0b53f0 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec15d8c0 con 0x7f5fec143840 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5ff2e74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec161380 con 0x7f5fec0b4520 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43532/0 (socket says 172.21.15.40:43532) 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 -- 172.21.15.40:0/1868639300 learned_addr learned my addr 172.21.15.40:0/1868639300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 -- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b4520 msgr2=0x7f5fec15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b4520 0x7f5fec15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 -- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec143840 msgr2=0x7f5fec141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec143840 0x7f5fec141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.609+0000 7f5febfff640 1 -- 172.21.15.40:0/1868639300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec15a8e0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5febfff640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5fd802a220 tx=0x7f5fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8037040 con 0x7f5fec0b53f0 2024-04-26T21:18:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd8042070 con 0x7f5fec0b53f0 2024-04-26T21:18:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec0ae8b0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8030bb0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.610+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec0aee70 con 0x7f5fec0b53f0 2024-04-26T21:18:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.611+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec06a290 con 0x7f5fec0b53f0 2024-04-26T21:18:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.611+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fd8047460 con 0x7f5fec0b53f0 2024-04-26T21:18:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.612+0000 7f5fe97fa640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.612+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f5fd80ac7c0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.612+0000 7f5ff0c10640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.613+0000 7f5ff0c10640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 0x7f5fd006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000a810 tx=0x7f5fe0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.616+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fec15a8e0 con 0x7f5fec0b53f0 2024-04-26T21:18:24.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.799+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fec0ce420 con 0x7f5fec0b53f0 2024-04-26T21:18:24.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.800+0000 7f5fe97fa640 1 -- 172.21.15.40:0/1868639300 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f5fd8045360 con 0x7f5fec0b53f0 2024-04-26T21:18:24.801 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.801+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 msgr2=0x7f5fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.801+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 0x7f5fd006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000a810 tx=0x7f5fe0008040 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.801+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 msgr2=0x7f5fec141530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.801+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5fd802a220 tx=0x7f5fd80046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 shutdown_connections 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec143840 0x7f5fec141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fec0b53f0 0x7f5fec141530 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 --2- 172.21.15.40:0/1868639300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b4520 0x7f5fec15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 >> 172.21.15.40:0/1868639300 conn(0x7f5fec0cd8a0 msgr2=0x7f5fec0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:24.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 shutdown_connections 2024-04-26T21:18:24.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.802+0000 7f5ff2e74640 1 -- 172.21.15.40:0/1868639300 wait complete. 2024-04-26T21:18:24.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.916+0000 7fbce392b640 1 Processor -- start 2024-04-26T21:18:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce392b640 1 -- start start 2024-04-26T21:18:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce392b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcdc0d3820 0x7fbcdc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce392b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcdc0d3c90 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce392b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcdc0d3f50 con 0x7fbcdc15f680 2024-04-26T21:18:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce392b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbcdc15fa60 con 0x7fbcdc0d3820 2024-04-26T21:18:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce1ec8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcdc0b60b0 0x7fbcdc0d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60854/0 (socket says 172.21.15.40:60854) 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.917+0000 7fbce1ec8640 1 -- 172.21.15.40:0/3246421075 learned_addr learned my addr 172.21.15.40:0/3246421075 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/3246421075 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc0d3820 0x7fbcdc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388013700 0 0) 0x7fbcdc0d3c90 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcc4003520 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174351485 0 0) 0x7fbcdc0d3f50 con 0x7fbcdc15f680 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcdc0d3c90 con 0x7fbcdc15f680 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664731890 0 0) 0x7fbcc4003520 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc0d3820 msgr2=0x7fbcdc0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 --2- 172.21.15.40:0/3246421075 >> v2:172.21.15.184:3300/0 conn(0x7fbcdc0d3820 0x7fbcdc0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 >> v1:172.21.15.184:6789/0 conn(0x7fbcdc15f680 legacy=0x7fbcdc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc0b5090 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fbcd4001670 con 0x7fbcdc0b60b0 2024-04-26T21:18:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 >> v1:172.21.15.40:6789/0 conn(0x7fbcdc0b60b0 legacy=0x7fbcdc0d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc4003b90 0x7fbcc4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 0x7fbcc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc40051c0 0x7fbcc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc15fa60 con 0x7fbcc4004800 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc400c090 con 0x7fbcc4003b90 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc400c290 con 0x7fbcc40051c0 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 0x7fbcc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce16c7640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc4003b90 0x7fbcc4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce1ec8640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc40051c0 0x7fbcc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce0ec6640 1 -- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc40051c0 msgr2=0x7fbcc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.918+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc40051c0 0x7fbcc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbce0ec6640 1 -- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc4003b90 msgr2=0x7fbcc4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcc4003b90 0x7fbcc4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbce0ec6640 1 -- 172.21.15.40:0/3246421075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc4003520 con 0x7fbcc4004800 2024-04-26T21:18:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbce1ec8640 1 -- 172.21.15.40:0/3246421075 reap_dead start 2024-04-26T21:18:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 0x7fbcc4004c80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009ac0 tx=0x7fbcc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94e786c98510fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0036070 con 0x7fbcc4004800 2024-04-26T21:18:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd0031400 con 0x7fbcc4004800 2024-04-26T21:18:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.919+0000 7fbcdbfff640 1 -- 172.21.15.40:0/3246421075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0009690 con 0x7fbcc4004800 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 -- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 msgr2=0x7fbcc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 0x7fbcc4004c80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009ac0 tx=0x7fbcc4005830 comp rx=0 tx=0).stop 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 -- 172.21.15.40:0/3246421075 shutdown_connections 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 --2- 172.21.15.40:0/3246421075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcc4004800 0x7fbcc4004c80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 -- 172.21.15.40:0/3246421075 >> 172.21.15.40:0/3246421075 conn(0x7fbcdc08eca0 msgr2=0x7fbcdc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 -- 172.21.15.40:0/3246421075 shutdown_connections 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.920+0000 7fbce392b640 1 -- 172.21.15.40:0/3246421075 wait complete. 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.921+0000 7fbce392b640 1 Processor -- start 2024-04-26T21:18:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.921+0000 7fbce392b640 1 -- start start 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc4004800 0x7fbcdc161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc153880 0x7fbcdc0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc0b5cf0 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc0b5b70 con 0x7fbcdc153880 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce392b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcdc0b5e70 con 0x7fbcc4004800 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43548/0 (socket says 172.21.15.40:43548) 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 -- 172.21.15.40:0/2194816976 learned_addr learned my addr 172.21.15.40:0/2194816976 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce16c7640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc4004800 0x7fbcdc161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce0ec6640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc153880 0x7fbcdc0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 -- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc4004800 msgr2=0x7fbcdc161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc4004800 0x7fbcdc161be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 -- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc153880 msgr2=0x7fbcdc0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc153880 0x7fbcdc0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 -- 172.21.15.40:0/2194816976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc161120 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.922+0000 7fbce1ec8640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0d60f0 tx=0x7fbcd400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.923+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4019070 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.923+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd4014480 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.923+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc0ced90 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.923+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4014850 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.923+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc1613e0 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.924+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc06a290 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.924+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd401c070 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.925+0000 7fbcd9ffb640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 0x7fbcc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.925+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fbcd408b000 con 0x7fbcdc0ce8b0 2024-04-26T21:18:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.925+0000 7fbce16c7640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 0x7fbcc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.926+0000 7fbce16c7640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 0x7fbcc806edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbccc000f80 tx=0x7fbccc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:24.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:24.929+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcd4059410 con 0x7fbcdc0ce8b0 2024-04-26T21:18:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.113+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbcdc0b48c0 con 0x7fbcdc0ce8b0 2024-04-26T21:18:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.114+0000 7fbcd9ffb640 1 -- 172.21.15.40:0/2194816976 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fbcd4058d10 con 0x7fbcdc0ce8b0 2024-04-26T21:18:25.115 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 msgr2=0x7fbcc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 0x7fbcc806edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbccc000f80 tx=0x7fbccc009040 comp rx=0 tx=0).stop 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 msgr2=0x7fbcdc0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0d60f0 tx=0x7fbcd400e890 comp rx=0 tx=0).stop 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 shutdown_connections 2024-04-26T21:18:25.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbcc806c8f0 0x7fbcc806edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcdc0ce8b0 0x7fbcdc0d1e80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcdc153880 0x7fbcdc0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 --2- 172.21.15.40:0/2194816976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcc4004800 0x7fbcdc161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 >> 172.21.15.40:0/2194816976 conn(0x7fbcdc08eca0 msgr2=0x7fbcdc0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 shutdown_connections 2024-04-26T21:18:25.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.116+0000 7fbce392b640 1 -- 172.21.15.40:0/2194816976 wait complete. 2024-04-26T21:18:25.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:25.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.231+0000 7fa7bb15f640 1 Processor -- start 2024-04-26T21:18:25.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.231+0000 7fa7bb15f640 1 -- start start 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7bb15f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa7b40d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7bb15f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b40d58f0 con 0x7fa7b40d3820 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7bb15f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b40d3c90 con 0x7fa7b415f680 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7bb15f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b415fa60 con 0x7fa7b40d4f20 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7b40d3820 0x7fa7b40b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60866/0 (socket says 172.21.15.40:60866) 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b3fff640 1 -- 172.21.15.40:0/2524766771 learned_addr learned my addr 172.21.15.40:0/2524766771 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454776677 0 0) 0x7fa7b40d58f0 con 0x7fa7b40d3820 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa79c003520 con 0x7fa7b40d3820 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa7b40d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4124554177 0 0) 0x7fa79c003520 con 0x7fa7b40d3820 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 msgr2=0x7fa7b40d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:25.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa7b40d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.232+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 >> v1:172.21.15.184:6789/0 conn(0x7fa7b415f680 legacy=0x7fa7b4163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40d3f50 con 0x7fa7b40d3820 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa7a8001670 con 0x7fa7b40d3820 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 >> v1:172.21.15.40:6789/0 conn(0x7fa7b40d3820 legacy=0x7fa7b40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79c0039e0 0x7fa79c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa79c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa79c004d40 0x7fa79c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b415fa60 con 0x7fa79c004d40 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00ba70 con 0x7fa79c0039e0 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c00bc70 con 0x7fa7b40d4f20 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa79c004d40 0x7fa79c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa79c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b8efb640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79c0039e0 0x7fa79c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 -- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 msgr2=0x7fa79c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b40d4f20 0x7fa79c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 -- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79c0039e0 msgr2=0x7fa79c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79c0039e0 0x7fa79c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.233+0000 7fa7b96fc640 1 -- 172.21.15.40:0/2524766771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c003520 con 0x7fa79c004d40 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa79c004d40 0x7fa79c0051c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009f90 tx=0x7fa79c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae051932e97835d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac009c10 con 0x7fa79c004d40 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ac036070 con 0x7fa79c004d40 2024-04-26T21:18:25.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7b37fe640 1 -- 172.21.15.40:0/2524766771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac031850 con 0x7fa79c004d40 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 -- 172.21.15.40:0/2524766771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa79c004d40 msgr2=0x7fa79c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/2524766771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa79c004d40 0x7fa79c0051c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009f90 tx=0x7fa79c005830 comp rx=0 tx=0).stop 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7b96fc640 1 -- 172.21.15.40:0/2524766771 reap_dead start 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 -- 172.21.15.40:0/2524766771 shutdown_connections 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 -- 172.21.15.40:0/2524766771 >> 172.21.15.40:0/2524766771 conn(0x7fa7b408eca0 msgr2=0x7fa7b409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 -- 172.21.15.40:0/2524766771 shutdown_connections 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.234+0000 7fa7bb15f640 1 -- 172.21.15.40:0/2524766771 wait complete. 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 Processor -- start 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 -- start start 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b414c550 0x7fa7b40cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 0x7fa7b40d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.236+0000 7fa7bb15f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b40b6280 con 0x7fa7b40d02c0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7bb15f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b40b5370 con 0x7fa7b414c550 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7bb15f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b40d53b0 con 0x7fa7b406e560 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43566/0 (socket says 172.21.15.40:43566) 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 -- 172.21.15.40:0/337391434 learned_addr learned my addr 172.21.15.40:0/337391434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b8efb640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b414c550 0x7fa7b40cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 0x7fa7b40d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 -- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 msgr2=0x7fa7b40d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 0x7fa7b40d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 -- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b414c550 msgr2=0x7fa7b40cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b414c550 0x7fa7b40cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 -- 172.21.15.40:0/337391434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40c8e40 con 0x7fa7b40d02c0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b96fc640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 0x7fa7b40d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80072c0 tx=0x7fa7a8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f960 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a8018070 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b40c90b0 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800fd60 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.237+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b40c9670 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.238+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b406a290 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.239+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7a801d460 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.239+0000 7fa7b17fa640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 0x7fa7a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.239+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fa7a808bc00 con 0x7fa7b40d02c0 2024-04-26T21:18:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.239+0000 7fa7b8efb640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 0x7fa7a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.240+0000 7fa7b8efb640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 0x7fa7a006ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40017e0 tx=0x7fa7a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.244+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7a8092080 con 0x7fa7b40d02c0 2024-04-26T21:18:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.428+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa7b40b4cd0 con 0x7fa7b40d02c0 2024-04-26T21:18:25.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.428+0000 7fa7b17fa640 1 -- 172.21.15.40:0/337391434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fa7a805a010 con 0x7fa7b40d02c0 2024-04-26T21:18:25.429 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:18:25.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 msgr2=0x7fa7a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 0x7fa7a006ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40017e0 tx=0x7fa7a4009040 comp rx=0 tx=0).stop 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 msgr2=0x7fa7b406e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80072c0 tx=0x7fa7a8007780 comp rx=0 tx=0).stop 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 shutdown_connections 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa7a006c920 0x7fa7a006ede0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7b406e560 0x7fa7b40d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7b40d02c0 0x7fa7b406e020 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 --2- 172.21.15.40:0/337391434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7b414c550 0x7fa7b40cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 >> 172.21.15.40:0/337391434 conn(0x7fa7b408eca0 msgr2=0x7fa7b40d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:25.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 shutdown_connections 2024-04-26T21:18:25.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.430+0000 7fa7bb15f640 1 -- 172.21.15.40:0/337391434 wait complete. 2024-04-26T21:18:25.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:18:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.542+0000 7fc82dde6640 1 Processor -- start 2024-04-26T21:18:25.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.542+0000 7fc82dde6640 1 -- start start 2024-04-26T21:18:25.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc82dde6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc828145980 0x7fc828160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc82dde6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b3de0 0x7fc8280b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc82dde6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8280b4260 con 0x7fc8280b5fd0 2024-04-26T21:18:25.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc82dde6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc8280b6440 con 0x7fc828145980 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc82dde6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8280b5140 con 0x7fc8280b3de0 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc828145980 0x7fc828160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc828145980 0x7fc828160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43582/0 (socket says 172.21.15.40:43582) 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 -- 172.21.15.40:0/1449592625 learned_addr learned my addr 172.21.15.40:0/1449592625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc826ffd640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b3de0 0x7fc8280b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 -- 172.21.15.40:0/1449592625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b3de0 msgr2=0x7fc8280b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b3de0 0x7fc8280b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 -- 172.21.15.40:0/1449592625 >> v1:172.21.15.184:6789/0 conn(0x7fc8280b5fd0 legacy=0x7fc82815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.543+0000 7fc8277fe640 1 -- 172.21.15.40:0/1449592625 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8280b4530 con 0x7fc828145980 2024-04-26T21:18:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.544+0000 7fc8277fe640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc828160630 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc818009870 tx=0x7fc81802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6941c7e91a1f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.544+0000 7fc8267fc640 1 -- 172.21.15.40:0/1449592625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818037070 con 0x7fc828145980 2024-04-26T21:18:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.544+0000 7fc8267fc640 1 -- 172.21.15.40:0/1449592625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc818032440 con 0x7fc828145980 2024-04-26T21:18:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.544+0000 7fc8267fc640 1 -- 172.21.15.40:0/1449592625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818032810 con 0x7fc828145980 2024-04-26T21:18:25.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 -- 172.21.15.40:0/1449592625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 msgr2=0x7fc828160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc828160630 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc818009870 tx=0x7fc81802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 -- 172.21.15.40:0/1449592625 shutdown_connections 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b3de0 0x7fc8280b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 --2- 172.21.15.40:0/1449592625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc828160630 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 -- 172.21.15.40:0/1449592625 >> 172.21.15.40:0/1449592625 conn(0x7fc8280d0d30 msgr2=0x7fc8280d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 -- 172.21.15.40:0/1449592625 shutdown_connections 2024-04-26T21:18:25.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.545+0000 7fc82dde6640 1 -- 172.21.15.40:0/1449592625 wait complete. 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.546+0000 7fc82dde6640 1 Processor -- start 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.546+0000 7fc82dde6640 1 -- start start 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8280b3de0 0x7fc82815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 0x7fc82815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8280b4dd0 con 0x7fc828145980 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8280b4af0 con 0x7fc8280b3de0 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc82dde6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8280730e0 con 0x7fc8280b5fd0 2024-04-26T21:18:25.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43596/0 (socket says 172.21.15.40:43596) 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 -- 172.21.15.40:0/4195019830 learned_addr learned my addr 172.21.15.40:0/4195019830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc8277fe640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8280b3de0 0x7fc82815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc826ffd640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 0x7fc82815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 -- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 msgr2=0x7fc82815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 0x7fc82815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 -- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8280b3de0 msgr2=0x7fc82815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8280b3de0 0x7fc82815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 -- 172.21.15.40:0/4195019830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82815cc80 con 0x7fc828145980 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc826ffd640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 0x7fc82815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.547+0000 7fc827fff640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c730 tx=0x7fc82000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.548+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820003b90 con 0x7fc828145980 2024-04-26T21:18:25.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.548+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820003d30 con 0x7fc828145980 2024-04-26T21:18:25.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.548+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82815cf60 con 0x7fc828145980 2024-04-26T21:18:25.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.548+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82000aa90 con 0x7fc828145980 2024-04-26T21:18:25.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.548+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82815d520 con 0x7fc828145980 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.549+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8280b1ff0 con 0x7fc828145980 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.549+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc82000ac30 con 0x7fc828145980 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.549+0000 7fc80ffff640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.550+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fc82008b9f0 con 0x7fc828145980 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.550+0000 7fc8277fe640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:25.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.550+0000 7fc8277fe640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc818034000 tx=0x7fc81802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:25.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.554+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8280b1ff0 con 0x7fc828145980 2024-04-26T21:18:25.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.750+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc82806a290 con 0x7fc828145980 2024-04-26T21:18:25.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.750+0000 7fc80ffff640 1 -- 172.21.15.40:0/4195019830 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7fc82008de20 con 0x7fc828145980 2024-04-26T21:18:25.752 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:18:25.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 msgr2=0x7fc81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc818034000 tx=0x7fc81802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 msgr2=0x7fc82815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c730 tx=0x7fc82000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 shutdown_connections 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc828145980 0x7fc82815b0d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8280b5fd0 0x7fc82815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 --2- 172.21.15.40:0/4195019830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8280b3de0 0x7fc82815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 >> 172.21.15.40:0/4195019830 conn(0x7fc8280d0d30 msgr2=0x7fc8280b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 shutdown_connections 2024-04-26T21:18:25.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:25.752+0000 7fc82dde6640 1 -- 172.21.15.40:0/4195019830 wait complete. 2024-04-26T21:18:25.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:18:25.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:25.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:25.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:18:25.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:25.818 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:18:25.838 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:25.838 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:25.838 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:25.855 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:25.855 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:25.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.012+0000 7f9bc6171640 1 Processor -- start 2024-04-26T21:18:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.012+0000 7f9bc6171640 1 -- start start 2024-04-26T21:18:26.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bc6171640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9bc00d11b0 0x7f9bc006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bc6171640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9bc015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bc6171640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00d1740 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bc6171640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc006e750 con 0x7f9bc00d11b0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bc6171640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc00d1a20 con 0x7f9bc006e2c0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9bc00cf2e0 0x7f9bc006d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60880/0 (socket says 172.21.15.40:60880) 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbf7fe640 1 -- 172.21.15.40:0/69185311 learned_addr learned my addr 172.21.15.40:0/69185311 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbffff640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9bc015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbeffd640 1 --2- 172.21.15.40:0/69185311 >> v2:172.21.15.184:3300/0 conn(0x7f9bc00d11b0 0x7f9bc006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 762226482 0 0) 0x7f9bc00d1740 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ba8003520 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 427655059 0 0) 0x7f9ba8003520 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 msgr2=0x7f9bc015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.013+0000 7f9bbe7fc640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9bc015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 >> v2:172.21.15.184:3300/0 conn(0x7f9bc00d11b0 msgr2=0x7f9bc006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 --2- 172.21.15.40:0/69185311 >> v2:172.21.15.184:3300/0 conn(0x7f9bc00d11b0 0x7f9bc006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00d1d00 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9bb0001670 con 0x7f9bc00cf2e0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 >> v1:172.21.15.40:6789/0 conn(0x7f9bc00cf2e0 legacy=0x7f9bc006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba80039b0 0x7f9ba8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9ba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8004d40 0x7f9ba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc006e750 con 0x7f9ba80039b0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d1a20 con 0x7f9ba8004d40 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba800b9c0 con 0x7f9bc006e2c0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba80039b0 0x7f9ba8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbeffd640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9ba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbffff640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8004d40 0x7f9ba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 -- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 msgr2=0x7f9ba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bc006e2c0 0x7f9ba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 -- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8004d40 msgr2=0x7f9ba80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ba8004d40 0x7f9ba80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.014+0000 7f9bbf7fe640 1 -- 172.21.15.40:0/69185311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba8003520 con 0x7f9ba80039b0 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba80039b0 0x7f9ba8003dd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000c210 tx=0x7f9ba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e630ac1572cf47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0007d30 con 0x7f9ba80039b0 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0005af0 con 0x7f9ba80039b0 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bbe7fc640 1 -- 172.21.15.40:0/69185311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0007070 con 0x7f9ba80039b0 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 -- 172.21.15.40:0/69185311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba80039b0 msgr2=0x7f9ba8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 --2- 172.21.15.40:0/69185311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba80039b0 0x7f9ba8003dd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000c210 tx=0x7f9ba8005830 comp rx=0 tx=0).stop 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bbffff640 1 -- 172.21.15.40:0/69185311 reap_dead start 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 -- 172.21.15.40:0/69185311 shutdown_connections 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 -- 172.21.15.40:0/69185311 >> 172.21.15.40:0/69185311 conn(0x7f9bc00b4a60 msgr2=0x7f9bc00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 -- 172.21.15.40:0/69185311 shutdown_connections 2024-04-26T21:18:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.015+0000 7f9bc6171640 1 -- 172.21.15.40:0/69185311 wait complete. 2024-04-26T21:18:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.017+0000 7f9bc6171640 1 Processor -- start 2024-04-26T21:18:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.017+0000 7f9bc6171640 1 -- start start 2024-04-26T21:18:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ba80039b0 0x7f9bc00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bc00b5b00 0x7f9bc00b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00cf6c0 con 0x7f9bc00afd20 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d22c0 con 0x7f9bc00b5b00 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc00cf840 con 0x7f9ba80039b0 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43606/0 (socket says 172.21.15.40:43606) 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 -- 172.21.15.40:0/1185905847 learned_addr learned my addr 172.21.15.40:0/1185905847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ba80039b0 0x7f9bc00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbffff640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bc00b5b00 0x7f9bc00b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 -- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ba80039b0 msgr2=0x7f9bc00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ba80039b0 0x7f9bc00b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 -- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bc00b5b00 msgr2=0x7f9bc00b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bc00b5b00 0x7f9bc00b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 -- 172.21.15.40:0/1185905847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00b3b90 con 0x7f9bc00afd20 2024-04-26T21:18:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bbeffd640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011990 tx=0x7f9bb4011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400fa10 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb400fbb0 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc00b1480 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4014440 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.018+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc00b0be0 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.019+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc006a290 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.020+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bb4012040 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.020+0000 7f9ba7fff640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 0x7f9bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.020+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f9bb408be60 con 0x7f9bc00afd20 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.021+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 0x7f9bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.021+0000 7f9bbf7fe640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 0x7f9bac06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0033000 tx=0x7f9bb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.025+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9bc00b1480 con 0x7f9bc00afd20 2024-04-26T21:18:26.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.214+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bc009c920 con 0x7f9bc00afd20 2024-04-26T21:18:26.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.215+0000 7f9ba7fff640 1 -- 172.21.15.40:0/1185905847 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16710 (secure 0 0 0) 0x7f9bb4019030 con 0x7f9bc00afd20 2024-04-26T21:18:26.216 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:26.216 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:16:42.760777+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 msgr2=0x7f9bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 0x7f9bac06edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0033000 tx=0x7f9bb002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 msgr2=0x7f9bc00b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011990 tx=0x7f9bb4011e50 comp rx=0 tx=0).stop 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 shutdown_connections 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9bac06c8f0 0x7f9bac06edb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bc00b5b00 0x7f9bc00b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc00afd20 0x7f9bc00b01a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 --2- 172.21.15.40:0/1185905847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ba80039b0 0x7f9bc00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 >> 172.21.15.40:0/1185905847 conn(0x7f9bc00b4a60 msgr2=0x7f9bc00c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 shutdown_connections 2024-04-26T21:18:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.217+0000 7f9bc6171640 1 -- 172.21.15.40:0/1185905847 wait complete. 2024-04-26T21:18:26.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:55.191765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:55.661024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:26.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 Processor -- start 2024-04-26T21:18:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 -- start start 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe89c0b3c20 0x7fe89c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe89c0b33e0 0x7fe89c163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe89c09c700 con 0x7fe89c0d0010 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe89c0b4480 con 0x7fe89c0b3c20 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.333+0000 7fe8a1259640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe89c06e050 con 0x7fe89c0b33e0 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe89c0b3c20 0x7fe89c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe89c0b3c20 0x7fe89c06db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43622/0 (socket says 172.21.15.40:43622) 2024-04-26T21:18:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 -- 172.21.15.40:0/3858877350 learned_addr learned my addr 172.21.15.40:0/3858877350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89b59e640 1 --2- 172.21.15.40:0/3858877350 >> v2:172.21.15.184:3300/0 conn(0x7fe89c0b33e0 0x7fe89c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 -- 172.21.15.40:0/3858877350 >> v2:172.21.15.184:3300/0 conn(0x7fe89c0b33e0 msgr2=0x7fe89c163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 --2- 172.21.15.40:0/3858877350 >> v2:172.21.15.184:3300/0 conn(0x7fe89c0b33e0 0x7fe89c163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 -- 172.21.15.40:0/3858877350 >> v1:172.21.15.184:6789/0 conn(0x7fe89c0d0010 legacy=0x7fe89c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.334+0000 7fe89a59c640 1 -- 172.21.15.40:0/3858877350 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0d1c50 con 0x7fe89c0b3c20 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe89a59c640 1 --2- 172.21.15.40:0/3858877350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c06db10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe89c073bb0 tx=0x7fe89002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacf1d4f3be1986a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe899d9b640 1 -- 172.21.15.40:0/3858877350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890037070 con 0x7fe89c0b3c20 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe899d9b640 1 -- 172.21.15.40:0/3858877350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890032440 con 0x7fe89c0b3c20 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe899d9b640 1 -- 172.21.15.40:0/3858877350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890032810 con 0x7fe89c0b3c20 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 -- 172.21.15.40:0/3858877350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 msgr2=0x7fe89c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3858877350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c06db10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe89c073bb0 tx=0x7fe89002a270 comp rx=0 tx=0).stop 2024-04-26T21:18:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 -- 172.21.15.40:0/3858877350 shutdown_connections 2024-04-26T21:18:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3858877350 >> v2:172.21.15.184:3300/0 conn(0x7fe89c0b33e0 0x7fe89c163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3858877350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c06db10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 -- 172.21.15.40:0/3858877350 >> 172.21.15.40:0/3858877350 conn(0x7fe89c09ba40 msgr2=0x7fe89c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 -- 172.21.15.40:0/3858877350 shutdown_connections 2024-04-26T21:18:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.335+0000 7fe8a1259640 1 -- 172.21.15.40:0/3858877350 wait complete. 2024-04-26T21:18:26.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.337+0000 7fe8a1259640 1 Processor -- start 2024-04-26T21:18:26.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.337+0000 7fe8a1259640 1 -- start start 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 0x7fe89c1f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe89c0d0010 0x7fe89c1e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0b3990 con 0x7fe89c0b3c20 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0b3810 con 0x7fe89c0d0010 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c1637c0 con 0x7fe89c0b33e0 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43634/0 (socket says 172.21.15.40:43634) 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 -- 172.21.15.40:0/3965662200 learned_addr learned my addr 172.21.15.40:0/3965662200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89ad9d640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 0x7fe89c1f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89b59e640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe89c0d0010 0x7fe89c1e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 -- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 msgr2=0x7fe89c1f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 0x7fe89c1f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 -- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe89c0d0010 msgr2=0x7fe89c1e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe89c0d0010 0x7fe89c1e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 -- 172.21.15.40:0/3965662200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c157440 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89ad9d640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 0x7fe89c1f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe89a59c640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe89002a2b0 tx=0x7fe8900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890037040 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890042070 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c146fc0 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890030880 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.338+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c147580 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.339+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c06a290 con 0x7fe89c0b3c20 2024-04-26T21:18:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.340+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe890030a20 con 0x7fe89c0b3c20 2024-04-26T21:18:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.340+0000 7fe8837fe640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 0x7fe88806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.340+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fe8900ac4a0 con 0x7fe89c0b3c20 2024-04-26T21:18:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.340+0000 7fe89ad9d640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 0x7fe88806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.341+0000 7fe89ad9d640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 0x7fe88806edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe89c14b4a0 tx=0x7fe88c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.344+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe89007a810 con 0x7fe89c0b3c20 2024-04-26T21:18:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.535+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe89c068e00 con 0x7fe89c0b3c20 2024-04-26T21:18:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.535+0000 7fe8837fe640 1 -- 172.21.15.40:0/3965662200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fe89007a110 con 0x7fe89c0b3c20 2024-04-26T21:18:26.536 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:26.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 msgr2=0x7fe88806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 0x7fe88806edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe89c14b4a0 tx=0x7fe88c009040 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 msgr2=0x7fe89c1e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe89002a2b0 tx=0x7fe8900046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 shutdown_connections 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe88806c8f0 0x7fe88806edb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe89c0d0010 0x7fe89c1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe89c0b3c20 0x7fe89c1e98e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 --2- 172.21.15.40:0/3965662200 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe89c0b33e0 0x7fe89c1f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 >> 172.21.15.40:0/3965662200 conn(0x7fe89c09ba40 msgr2=0x7fe89c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 shutdown_connections 2024-04-26T21:18:26.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.537+0000 7fe8a1259640 1 -- 172.21.15.40:0/3965662200 wait complete. 2024-04-26T21:18:26.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:26.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.652+0000 7f99cb378640 1 Processor -- start 2024-04-26T21:18:26.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.652+0000 7f99cb378640 1 -- start start 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99cb378640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f99c40d3340 0x7f99c40d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99cb378640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c40b6aa0 con 0x7f99c415f5b0 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99cb378640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c40d3eb0 con 0x7f99c415f990 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99cb378640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f99c40d4190 con 0x7f99c40d3340 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99c9114640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f99c415f5b0 0x7f99c40b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60886/0 (socket says 172.21.15.40:60886) 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99c9114640 1 -- 172.21.15.40:0/1433750647 learned_addr learned my addr 172.21.15.40:0/1433750647 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184590797 0 0) 0x7f99c40b6aa0 con 0x7f99c415f5b0 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.653+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ac003520 con 0x7f99c415f5b0 2024-04-26T21:18:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9915640 1 --2- 172.21.15.40:0/1433750647 >> v2:172.21.15.184:3300/0 conn(0x7f99c40d3340 0x7f99c40d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2488010224 0 0) 0x7f99ac003520 con 0x7f99c415f5b0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 >> v2:172.21.15.184:3300/0 conn(0x7f99c40d3340 msgr2=0x7f99c40d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 --2- 172.21.15.40:0/1433750647 >> v2:172.21.15.184:3300/0 conn(0x7f99c40d3340 0x7f99c40d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 >> v1:172.21.15.184:6789/0 conn(0x7f99c415f990 legacy=0x7f99c40d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40d0200 con 0x7f99c415f5b0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f99bc001670 con 0x7f99c415f5b0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 >> v1:172.21.15.40:6789/0 conn(0x7f99c415f5b0 legacy=0x7f99c40b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 0x7f99ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99ac004800 0x7f99ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0051c0 0x7f99ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f99c40d4190 con 0x7f99ac0039b0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac00c160 con 0x7f99ac0051c0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99ac00c360 con 0x7f99ac004800 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 0x7f99ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c8913640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99ac004800 0x7f99ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9915640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0051c0 0x7f99ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 -- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99ac004800 msgr2=0x7f99ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99ac004800 0x7f99ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 -- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0051c0 msgr2=0x7f99ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0051c0 0x7f99ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9114640 1 -- 172.21.15.40:0/1433750647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac003520 con 0x7f99ac0039b0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.654+0000 7f99c9915640 1 -- 172.21.15.40:0/1433750647 reap_dead start 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.655+0000 7f99c9114640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 0x7f99ac003dd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c240 tx=0x7f99ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a219b358023692e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.655+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007d10 con 0x7f99ac0039b0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.655+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc005ad0 con 0x7f99ac0039b0 2024-04-26T21:18:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.655+0000 7f99c3fff640 1 -- 172.21.15.40:0/1433750647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007070 con 0x7f99ac0039b0 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 -- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 msgr2=0x7f99ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 0x7f99ac003dd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c240 tx=0x7f99ac005830 comp rx=0 tx=0).stop 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 -- 172.21.15.40:0/1433750647 shutdown_connections 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 --2- 172.21.15.40:0/1433750647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0039b0 0x7f99ac003dd0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 -- 172.21.15.40:0/1433750647 >> 172.21.15.40:0/1433750647 conn(0x7f99c408ec40 msgr2=0x7f99c409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 -- 172.21.15.40:0/1433750647 shutdown_connections 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.656+0000 7f99cb378640 1 -- 172.21.15.40:0/1433750647 wait complete. 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.657+0000 7f99cb378640 1 Processor -- start 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.657+0000 7f99cb378640 1 -- start start 2024-04-26T21:18:26.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0039b0 0x7f99c40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99c41f89b0 0x7f99c41f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f99c40d3910 con 0x7f99c41540d0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99c40d3790 con 0x7f99ac0039b0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99c40d3a90 con 0x7f99c41f89b0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43642/0 (socket says 172.21.15.40:43642) 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 -- 172.21.15.40:0/2174172269 learned_addr learned my addr 172.21.15.40:0/2174172269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c9114640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0039b0 0x7f99c40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c9915640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99c41f89b0 0x7f99c41f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 -- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99c41f89b0 msgr2=0x7f99c41f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99c41f89b0 0x7f99c41f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 -- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0039b0 msgr2=0x7f99c40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0039b0 0x7f99c40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 -- 172.21.15.40:0/2174172269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c415b9c0 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c8913640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f99b800f620 tx=0x7f99b800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b80052e0 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b8018070 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.658+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c415bca0 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.659+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003810 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.659+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c415c260 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.660+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c406a290 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.660+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99b80039b0 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.660+0000 7f99c1ffb640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.660+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f99b808b8e0 con 0x7f99c41540d0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.661+0000 7f99c9114640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.661+0000 7f99c9114640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 0x7f99b006ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f99bc032000 tx=0x7f99bc034040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.665+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99b8059cf0 con 0x7f99c41540d0 2024-04-26T21:18:26.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.848+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99c415d020 con 0x7f99c41540d0 2024-04-26T21:18:26.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.848+0000 7f99c1ffb640 1 -- 172.21.15.40:0/2174172269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f99b80595f0 con 0x7f99c41540d0 2024-04-26T21:18:26.850 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:18:26.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 msgr2=0x7f99b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 0x7f99b006ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f99bc032000 tx=0x7f99bc034040 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 msgr2=0x7f99c4154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f99b800f620 tx=0x7f99b800fae0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 shutdown_connections 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99c41f89b0 0x7f99c41f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c41540d0 0x7f99c4154550 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 --2- 172.21.15.40:0/2174172269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ac0039b0 0x7f99c40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.850+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 >> 172.21.15.40:0/2174172269 conn(0x7f99c408ec40 msgr2=0x7f99c415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.851+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 shutdown_connections 2024-04-26T21:18:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.851+0000 7f99cb378640 1 -- 172.21.15.40:0/2174172269 wait complete. 2024-04-26T21:18:26.862 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.965+0000 7ff5a8de9640 1 Processor -- start 2024-04-26T21:18:26.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.965+0000 7ff5a8de9640 1 -- start start 2024-04-26T21:18:26.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5a415f490 0x7ff5a40d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d2120 0x7ff5a40ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a40d2570 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a40d4240 con 0x7ff5a415f490 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a8de9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a40d31a0 con 0x7ff5a40d2120 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43648/0 (socket says 172.21.15.40:43648) 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 -- 172.21.15.40:0/2204725448 learned_addr learned my addr 172.21.15.40:0/2204725448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a259c640 1 --2- 172.21.15.40:0/2204725448 >> v2:172.21.15.184:3300/0 conn(0x7ff5a415f490 0x7ff5a40d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff5a2d9d640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d2120 0x7ff5a40ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 -- 172.21.15.40:0/2204725448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d2120 msgr2=0x7ff5a40ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d2120 0x7ff5a40ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 -- 172.21.15.40:0/2204725448 >> v2:172.21.15.184:3300/0 conn(0x7ff5a415f490 msgr2=0x7ff5a40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 --2- 172.21.15.40:0/2204725448 >> v2:172.21.15.184:3300/0 conn(0x7ff5a415f490 0x7ff5a40d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.966+0000 7ff599d9b640 1 -- 172.21.15.40:0/2204725448 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a40d3480 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff599d9b640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40b7cf0 tx=0x7ff58c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd894efd2d1faf73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a1d9b640 1 -- 172.21.15.40:0/2204725448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c037070 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a1d9b640 1 -- 172.21.15.40:0/2204725448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c032440 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a1d9b640 1 -- 172.21.15.40:0/2204725448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c032810 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 -- 172.21.15.40:0/2204725448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 msgr2=0x7ff5a40d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40b7cf0 tx=0x7ff58c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 -- 172.21.15.40:0/2204725448 shutdown_connections 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40d2120 0x7ff5a40ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/2204725448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a40d1be0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/2204725448 >> v2:172.21.15.184:3300/0 conn(0x7ff5a415f490 0x7ff5a40d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.967+0000 7ff5a8de9640 1 -- 172.21.15.40:0/2204725448 >> 172.21.15.40:0/2204725448 conn(0x7ff5a409e530 msgr2=0x7ff5a40d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.968+0000 7ff5a8de9640 1 -- 172.21.15.40:0/2204725448 shutdown_connections 2024-04-26T21:18:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.968+0000 7ff5a8de9640 1 -- 172.21.15.40:0/2204725448 wait complete. 2024-04-26T21:18:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.969+0000 7ff5a8de9640 1 Processor -- start 2024-04-26T21:18:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.969+0000 7ff5a8de9640 1 -- start start 2024-04-26T21:18:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 0x7ff5a40b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 0x7ff5a4161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40d3d20 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40d3a40 con 0x7ff5a40d2120 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a415f870 con 0x7ff5a415f490 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43650/0 (socket says 172.21.15.40:43650) 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 -- 172.21.15.40:0/3942258059 learned_addr learned my addr 172.21.15.40:0/3942258059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a2d9d640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 0x7ff5a4161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff599d9b640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 0x7ff5a40b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 -- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 msgr2=0x7ff5a4161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 0x7ff5a4161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 -- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 msgr2=0x7ff5a40b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 0x7ff5a40b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 -- 172.21.15.40:0/3942258059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a414dfd0 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff599d9b640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 0x7ff5a40b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a2d9d640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 0x7ff5a4161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a259c640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c820 tx=0x7ff59400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594015070 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff594004240 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a414e2b0 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.970+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59400ce10 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.971+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41509e0 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.971+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a406a290 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.972+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff594007830 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.972+0000 7ff59b7fe640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.972+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7ff59408bdf0 con 0x7ff5a40d0ff0 2024-04-26T21:18:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.972+0000 7ff599d9b640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.973+0000 7ff599d9b640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 0x7ff58406edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff58c034000 tx=0x7ff58c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:26.977+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff59405a170 con 0x7ff5a40d0ff0 2024-04-26T21:18:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.161+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5a40d5900 con 0x7ff5a40d0ff0 2024-04-26T21:18:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.162+0000 7ff59b7fe640 1 -- 172.21.15.40:0/3942258059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7ff594059a70 con 0x7ff5a40d0ff0 2024-04-26T21:18:27.163 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 msgr2=0x7ff58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 0x7ff58406edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff58c034000 tx=0x7ff58c02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 msgr2=0x7ff5a414d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c820 tx=0x7ff59400cce0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 shutdown_connections 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff58406c8f0 0x7ff58406edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a415f490 0x7ff5a4161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a40d2120 0x7ff5a40b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 --2- 172.21.15.40:0/3942258059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a40d0ff0 0x7ff5a414d710 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 >> 172.21.15.40:0/3942258059 conn(0x7ff5a409e530 msgr2=0x7ff5a415ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 shutdown_connections 2024-04-26T21:18:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.164+0000 7ff5a8de9640 1 -- 172.21.15.40:0/3942258059 wait complete. 2024-04-26T21:18:27.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:18:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.276+0000 7fc08b71c640 1 Processor -- start 2024-04-26T21:18:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.276+0000 7fc08b71c640 1 -- start start 2024-04-26T21:18:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc08b71c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0840cf2e0 0x7fc08406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc08b71c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc08406e490 0x7fc08415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc08b71c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0840d1740 con 0x7fc0840d11b0 2024-04-26T21:18:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc08b71c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc0840d1a20 con 0x7fc0840cf2e0 2024-04-26T21:18:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc08b71c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0840d1d00 con 0x7fc08406e490 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0840cf2e0 0x7fc08406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0840cf2e0 0x7fc08406d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43664/0 (socket says 172.21.15.40:43664) 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 -- 172.21.15.40:0/2134296828 learned_addr learned my addr 172.21.15.40:0/2134296828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 -- 172.21.15.40:0/2134296828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc08406e490 msgr2=0x7fc08415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 --2- 172.21.15.40:0/2134296828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc08406e490 0x7fc08415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 -- 172.21.15.40:0/2134296828 >> v1:172.21.15.184:6789/0 conn(0x7fc0840d11b0 legacy=0x7fc08406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.277+0000 7fc0894b8640 1 -- 172.21.15.40:0/2134296828 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0840d1fe0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.278+0000 7fc0894b8640 1 --2- 172.21.15.40:0/2134296828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc08406d4d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc08002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640833619cab6d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.278+0000 7fc07bfff640 1 -- 172.21.15.40:0/2134296828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc0840cf2e0 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.278+0000 7fc07bfff640 1 -- 172.21.15.40:0/2134296828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc080032440 con 0x7fc0840cf2e0 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.278+0000 7fc07bfff640 1 -- 172.21.15.40:0/2134296828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080032810 con 0x7fc0840cf2e0 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 -- 172.21.15.40:0/2134296828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 msgr2=0x7fc08406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 --2- 172.21.15.40:0/2134296828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc08406d4d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc08002a270 comp rx=0 tx=0).stop 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 -- 172.21.15.40:0/2134296828 shutdown_connections 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 --2- 172.21.15.40:0/2134296828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc08406e490 0x7fc08415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 --2- 172.21.15.40:0/2134296828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc08406d4d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 -- 172.21.15.40:0/2134296828 >> 172.21.15.40:0/2134296828 conn(0x7fc0840b4a60 msgr2=0x7fc0840b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 -- 172.21.15.40:0/2134296828 shutdown_connections 2024-04-26T21:18:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.279+0000 7fc08b71c640 1 -- 172.21.15.40:0/2134296828 wait complete. 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.280+0000 7fc08b71c640 1 Processor -- start 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.280+0000 7fc08b71c640 1 -- start start 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 0x7fc0840b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 0x7fc0840b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0840cf6c0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc08406da10 con 0x7fc08406e490 2024-04-26T21:18:27.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc08b71c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc0840cf840 con 0x7fc0840d11b0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43672/0 (socket says 172.21.15.40:43672) 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 -- 172.21.15.40:0/4014864077 learned_addr learned my addr 172.21.15.40:0/4014864077 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc0894b8640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 0x7fc0840b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc089cb9640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 0x7fc0840b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 -- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 msgr2=0x7fc0840b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 0x7fc0840b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 -- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 msgr2=0x7fc0840b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 0x7fc0840b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 -- 172.21.15.40:0/4014864077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0840b68f0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc0894b8640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 0x7fc0840b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc089cb9640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 0x7fc0840b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc088cb7640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc07400b3e0 tx=0x7fc07400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.281+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074015020 con 0x7fc0840cf2e0 2024-04-26T21:18:27.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.282+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc074011850 con 0x7fc0840cf2e0 2024-04-26T21:18:27.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.282+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0840b6bd0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.282+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074011c40 con 0x7fc0840cf2e0 2024-04-26T21:18:27.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.282+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0840b7190 con 0x7fc0840cf2e0 2024-04-26T21:18:27.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.283+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08406a290 con 0x7fc0840cf2e0 2024-04-26T21:18:27.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.283+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc074018020 con 0x7fc0840cf2e0 2024-04-26T21:18:27.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.284+0000 7fc079ffb640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 0x7fc06806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.284+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fc07408bb40 con 0x7fc0840cf2e0 2024-04-26T21:18:27.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.284+0000 7fc0894b8640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 0x7fc06806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.284+0000 7fc0894b8640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 0x7fc06806ede0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.288+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc07401f020 con 0x7fc0840cf2e0 2024-04-26T21:18:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.489+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0840b8ef0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.490+0000 7fc079ffb640 1 -- 172.21.15.40:0/4014864077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7fc0740172c0 con 0x7fc0840cf2e0 2024-04-26T21:18:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:18:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 msgr2=0x7fc06806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 0x7fc06806ede0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 msgr2=0x7fc0840ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc07400b3e0 tx=0x7fc07400b8a0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 shutdown_connections 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc06806c920 0x7fc06806ede0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0840d11b0 0x7fc0840b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0840cf2e0 0x7fc0840ccdf0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 --2- 172.21.15.40:0/4014864077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc08406e490 0x7fc0840b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 >> 172.21.15.40:0/4014864077 conn(0x7fc0840b4a60 msgr2=0x7fc0840b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 shutdown_connections 2024-04-26T21:18:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.492+0000 7fc08b71c640 1 -- 172.21.15.40:0/4014864077 wait complete. 2024-04-26T21:18:27.503 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:18:27.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:27.504 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:27.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:18:27.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:27.535 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:18:27.594 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:27.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:27.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:27.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:18:27.610 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:27.610 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:27.610 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:27.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:18:27.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.752+0000 7f83a8652640 1 Processor -- start 2024-04-26T21:18:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.752+0000 7f83a8652640 1 -- start start 2024-04-26T21:18:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a8652640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f83a00d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a8652640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83a0160d50 0x7f83a01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a8652640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a00d0be0 con 0x7f83a015d380 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a8652640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f83a00d0ec0 con 0x7f83a0160d50 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a8652640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00d11a0 con 0x7f83a00b56d0 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a63ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f83a015d380 0x7f83a00ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60896/0 (socket says 172.21.15.40:60896) 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a63ee640 1 -- 172.21.15.40:0/2009682389 learned_addr learned my addr 172.21.15.40:0/2009682389 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a5bed640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f83a00d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a6bef640 1 --2- 172.21.15.40:0/2009682389 >> v2:172.21.15.184:3300/0 conn(0x7f83a0160d50 0x7f83a01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346689956 0 0) 0x7f83a00d0be0 con 0x7f83a015d380 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c003520 con 0x7f83a015d380 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 765285166 0 0) 0x7f838c003520 con 0x7f83a015d380 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 msgr2=0x7f83a00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f83a00d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 >> v2:172.21.15.184:3300/0 conn(0x7f83a0160d50 msgr2=0x7f83a01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 --2- 172.21.15.40:0/2009682389 >> v2:172.21.15.184:3300/0 conn(0x7f83a0160d50 0x7f83a01611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00d1480 con 0x7f83a015d380 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f839c001670 con 0x7f83a015d380 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.753+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 >> v1:172.21.15.40:6789/0 conn(0x7f83a015d380 legacy=0x7f83a00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f838c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f838c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f838c004d40 0x7f838c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d0ec0 con 0x7f838c0039b0 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d11a0 con 0x7f838c004d40 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f838c00b9c0 con 0x7f83a00b56d0 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f838c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a5bed640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f838c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a6bef640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f838c004d40 0x7f838c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 -- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 msgr2=0x7f838c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f838c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 -- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f838c004d40 msgr2=0x7f838c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f838c004d40 0x7f838c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a63ee640 1 -- 172.21.15.40:0/2009682389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c003520 con 0x7f838c0039b0 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a5bed640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b56d0 0x7f838c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.754+0000 7f83a6bef640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f838c004d40 0x7f838c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a63ee640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f838c003d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f839c005130 tx=0x7f838c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d343e522dd687 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c004d00 con 0x7f838c0039b0 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c02f030 con 0x7f838c0039b0 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a53ec640 1 -- 172.21.15.40:0/2009682389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c005d10 con 0x7f838c0039b0 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a8652640 1 -- 172.21.15.40:0/2009682389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 msgr2=0x7f838c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a8652640 1 --2- 172.21.15.40:0/2009682389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f838c003d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f839c005130 tx=0x7f838c005830 comp rx=0 tx=0).stop 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a6bef640 1 -- 172.21.15.40:0/2009682389 reap_dead start 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a8652640 1 -- 172.21.15.40:0/2009682389 shutdown_connections 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a8652640 1 -- 172.21.15.40:0/2009682389 >> 172.21.15.40:0/2009682389 conn(0x7f83a00b4a60 msgr2=0x7f83a00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.755+0000 7f83a8652640 1 -- 172.21.15.40:0/2009682389 shutdown_connections 2024-04-26T21:18:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.756+0000 7f83a8652640 1 -- 172.21.15.40:0/2009682389 wait complete. 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 Processor -- start 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 -- start start 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b5ce0 0x7f83a00b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a00b7d40 0x7f83a00b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.757+0000 7f83a8652640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d1b20 con 0x7f838c0039b0 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a8652640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d19a0 con 0x7f83a00b7d40 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a8652640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00d1ca0 con 0x7f83a00b5ce0 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43696/0 (socket says 172.21.15.40:43696) 2024-04-26T21:18:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 -- 172.21.15.40:0/854449268 learned_addr learned my addr 172.21.15.40:0/854449268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a5bed640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b5ce0 0x7f83a00b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a6bef640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a00b7d40 0x7f83a00b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 -- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b5ce0 msgr2=0x7f83a00b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b5ce0 0x7f83a00b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 -- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a00b7d40 msgr2=0x7f83a00b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a00b7d40 0x7f83a00b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 -- 172.21.15.40:0/854449268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00b8960 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a63ee640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f839c02b5a0 tx=0x7f839c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c000ed0 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c001c50 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c02bcc0 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a015eed0 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.758+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a00b6160 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.759+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a006cdd0 con 0x7f838c0039b0 2024-04-26T21:18:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.760+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f839c03f030 con 0x7f838c0039b0 2024-04-26T21:18:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.760+0000 7f8396ffd640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 0x7f838406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.760+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f839c0ab3f0 con 0x7f838c0039b0 2024-04-26T21:18:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.760+0000 7f83a5bed640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 0x7f838406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.761+0000 7f83a5bed640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 0x7f838406ef50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c6a60 tx=0x7f8390008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.764+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83a006cdd0 con 0x7f838c0039b0 2024-04-26T21:18:27.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.987+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a006a290 con 0x7f838c0039b0 2024-04-26T21:18:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.988+0000 7f8396ffd640 1 -- 172.21.15.40:0/854449268 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5187 (secure 0 0 0) 0x7f839c046020 con 0x7f838c0039b0 2024-04-26T21:18:27.989 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:27.989 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:17:17.395375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":23,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:18:27.989 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 50 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 msgr2=0x7f838406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 0x7f838406ef50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c6a60 tx=0x7f8390008040 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 msgr2=0x7f83a0148c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f839c02b5a0 tx=0x7f839c001e50 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 shutdown_connections 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f838406ca90 0x7f838406ef50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a00b7d40 0x7f83a00b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83a00b5ce0 0x7f83a00b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 --2- 172.21.15.40:0/854449268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f838c0039b0 0x7f83a0148c00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 >> 172.21.15.40:0/854449268 conn(0x7f83a00b4a60 msgr2=0x7f83a00709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:27.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 shutdown_connections 2024-04-26T21:18:27.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:27.990+0000 7f83a8652640 1 -- 172.21.15.40:0/854449268 wait complete. 2024-04-26T21:18:28.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.101+0000 7fcf472d3640 1 Processor -- start 2024-04-26T21:18:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.101+0000 7fcf472d3640 1 -- start start 2024-04-26T21:18:28.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf472d3640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf400cf2e0 0x7fcf4006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf472d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf472d3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf400d1740 con 0x7fcf4006e2c0 2024-04-26T21:18:28.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf472d3640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4006e750 con 0x7fcf400cf2e0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf472d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf400d1a20 con 0x7fcf400d11b0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4486e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4506f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf400cf2e0 0x7fcf4006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4486e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:44634/0 (socket says 172.21.15.40:44634) 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4486e640 1 -- 172.21.15.40:0/593685275 learned_addr learned my addr 172.21.15.40:0/593685275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4506f640 1 -- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 msgr2=0x7fcf4006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4506f640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4506f640 1 -- 172.21.15.40:0/593685275 >> v1:172.21.15.184:6789/0 conn(0x7fcf4006e2c0 legacy=0x7fcf4015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4506f640 1 -- 172.21.15.40:0/593685275 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf400d1d00 con 0x7fcf400cf2e0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf37fff640 1 -- 172.21.15.40:0/593685275 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2734168160 0 0) 0x7fcf400d1740 con 0x7fcf4006e2c0 2024-04-26T21:18:28.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.102+0000 7fcf4486e640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:28.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.103+0000 7fcf4506f640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf400cf2e0 0x7fcf4006d4d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf300097a0 tx=0x7fcf3002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac891131836e0afa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.103+0000 7fcf37fff640 1 -- 172.21.15.40:0/593685275 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30037070 con 0x7fcf400cf2e0 2024-04-26T21:18:28.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.103+0000 7fcf37fff640 1 -- 172.21.15.40:0/593685275 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30032440 con 0x7fcf400cf2e0 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.103+0000 7fcf37fff640 1 -- 172.21.15.40:0/593685275 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30032810 con 0x7fcf400cf2e0 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 -- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf400cf2e0 msgr2=0x7fcf4006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf400cf2e0 0x7fcf4006d4d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf300097a0 tx=0x7fcf3002a180 comp rx=0 tx=0).stop 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 -- 172.21.15.40:0/593685275 shutdown_connections 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf4006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 --2- 172.21.15.40:0/593685275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf400cf2e0 0x7fcf4006d4d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 -- 172.21.15.40:0/593685275 >> 172.21.15.40:0/593685275 conn(0x7fcf400b4a60 msgr2=0x7fcf400b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 -- 172.21.15.40:0/593685275 shutdown_connections 2024-04-26T21:18:28.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.104+0000 7fcf472d3640 1 -- 172.21.15.40:0/593685275 wait complete. 2024-04-26T21:18:28.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.105+0000 7fcf472d3640 1 Processor -- start 2024-04-26T21:18:28.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.105+0000 7fcf472d3640 1 -- start start 2024-04-26T21:18:28.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4006e2c0 0x7fcf400b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf400b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4006da10 con 0x7fcf400cf2e0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf400d22c0 con 0x7fcf4006e2c0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf472d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf400cf6c0 con 0x7fcf400d11b0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf4486e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf4486e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43710/0 (socket says 172.21.15.40:43710) 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf4486e640 1 -- 172.21.15.40:0/4268362683 learned_addr learned my addr 172.21.15.40:0/4268362683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf4506f640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4006e2c0 0x7fcf400b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.106+0000 7fcf45870640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf400b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 -- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 msgr2=0x7fcf400b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf400b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 -- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4006e2c0 msgr2=0x7fcf400b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4006e2c0 0x7fcf400b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 -- 172.21.15.40:0/4268362683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf401f5f60 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf4486e640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9b0 tx=0x7fcf3c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c017070 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00c9a0 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf400b33d0 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00cd50 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.107+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf401f61d0 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.108+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4006a290 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.109+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf3c01a070 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.109+0000 7fcf35ffb640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 0x7fcf2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.109+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fcf3c08b8b0 con 0x7fcf400cf2e0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.109+0000 7fcf4506f640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 0x7fcf2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.110+0000 7fcf4506f640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 0x7fcf2406ede0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcf30034000 tx=0x7fcf3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.114+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf3c059700 con 0x7fcf400cf2e0 2024-04-26T21:18:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.303+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf400b8ef0 con 0x7fcf400cf2e0 2024-04-26T21:18:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.304+0000 7fcf35ffb640 1 -- 172.21.15.40:0/4268362683 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16710 (secure 0 0 0) 0x7fcf3c05da00 con 0x7fcf400cf2e0 2024-04-26T21:18:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:28.306 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:16:42.760777+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.306+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 msgr2=0x7fcf2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.306+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 0x7fcf2406ede0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcf30034000 tx=0x7fcf3002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.306+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 msgr2=0x7fcf400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.306+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9b0 tx=0x7fcf3c00ee70 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 shutdown_connections 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf2406c920 0x7fcf2406ede0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf400d11b0 0x7fcf400b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf400cf2e0 0x7fcf400b77a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 --2- 172.21.15.40:0/4268362683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4006e2c0 0x7fcf400b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 >> 172.21.15.40:0/4268362683 conn(0x7fcf400b4a60 msgr2=0x7fcf40070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 shutdown_connections 2024-04-26T21:18:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.307+0000 7fcf472d3640 1 -- 172.21.15.40:0/4268362683 wait complete. 2024-04-26T21:18:28.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:15:55.191765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:15:55.661024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:28.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.423+0000 7f0ed603e640 1 Processor -- start 2024-04-26T21:18:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.423+0000 7f0ed603e640 1 -- start start 2024-04-26T21:18:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.423+0000 7f0ed603e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0ed006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ed603e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ed006c9b0 0x7f0ed00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ed603e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed0070b30 con 0x7f0ed00706a0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ed603e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00af610 con 0x7f0ed006c9b0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ed603e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00700b0 con 0x7f0ed015f2b0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0eceffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0ed00706a0 0x7f0ed006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60902/0 (socket says 172.21.15.40:60902) 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0eceffd640 1 -- 172.21.15.40:0/792264271 learned_addr learned my addr 172.21.15.40:0/792264271 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ecf7fe640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0ed006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ecffff640 1 --2- 172.21.15.40:0/792264271 >> v2:172.21.15.184:3300/0 conn(0x7f0ed006c9b0 0x7f0ed00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519789143 0 0) 0x7f0ed0070b30 con 0x7f0ed00706a0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb8003520 con 0x7f0ed00706a0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 633418592 0 0) 0x7f0eb8003520 con 0x7f0ed00706a0 2024-04-26T21:18:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 msgr2=0x7f0ed006fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0ed006fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 >> v2:172.21.15.184:3300/0 conn(0x7f0ed006c9b0 msgr2=0x7f0ed00afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 --2- 172.21.15.40:0/792264271 >> v2:172.21.15.184:3300/0 conn(0x7f0ed006c9b0 0x7f0ed00afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0070e10 con 0x7f0ed00706a0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f0ec4001670 con 0x7f0ed00706a0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 >> v1:172.21.15.40:6789/0 conn(0x7f0ed00706a0 legacy=0x7f0ed006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0eb80039b0 0x7f0eb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0eb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0eb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00700b0 con 0x7f0eb8004d40 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80141d0 con 0x7f0eb80039b0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.424+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb8014350 con 0x7f0ed015f2b0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0eb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecf7fe640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0eb80039b0 0x7f0eb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0eceffd640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0eb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 -- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 msgr2=0x7f0eb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed015f2b0 0x7f0eb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 -- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0eb80039b0 msgr2=0x7f0eb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0eb80039b0 0x7f0eb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.425+0000 7f0ecffff640 1 -- 172.21.15.40:0/792264271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8003520 con 0x7f0eb8004d40 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ecffff640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0eb80051c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009f90 tx=0x7f0eb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e96f8af98e86118 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8009c10 con 0x7f0eb8004d40 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec8036070 con 0x7f0eb8004d40 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ece7fc640 1 -- 172.21.15.40:0/792264271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8031880 con 0x7f0eb8004d40 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 -- 172.21.15.40:0/792264271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 msgr2=0x7f0eb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 --2- 172.21.15.40:0/792264271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0eb80051c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009f90 tx=0x7f0eb8005830 comp rx=0 tx=0).stop 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ecffff640 1 -- 172.21.15.40:0/792264271 reap_dead start 2024-04-26T21:18:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 -- 172.21.15.40:0/792264271 shutdown_connections 2024-04-26T21:18:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 -- 172.21.15.40:0/792264271 >> 172.21.15.40:0/792264271 conn(0x7f0ed006f050 msgr2=0x7f0ed006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 -- 172.21.15.40:0/792264271 shutdown_connections 2024-04-26T21:18:28.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.426+0000 7f0ed603e640 1 -- 172.21.15.40:0/792264271 wait complete. 2024-04-26T21:18:28.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.427+0000 7f0ed603e640 1 Processor -- start 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 -- start start 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 0x7f0ed014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed014a880 0x7f0ed015f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00b4f40 con 0x7f0eb8004d40 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00713d0 con 0x7f0ed014f2e0 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ed603e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00b1890 con 0x7f0ed014a880 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ecf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ecf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43728/0 (socket says 172.21.15.40:43728) 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.428+0000 7f0ecf7fe640 1 -- 172.21.15.40:0/778966339 learned_addr learned my addr 172.21.15.40:0/778966339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0eceffd640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 0x7f0ed014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecffff640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed014a880 0x7f0ed015f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 -- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed014a880 msgr2=0x7f0ed015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed014a880 0x7f0ed015f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 -- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 msgr2=0x7f0ed014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 0x7f0ed014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 -- 172.21.15.40:0/778966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed014f740 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0eceffd640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 0x7f0ed014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ecf7fe640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00119d0 tx=0x7f0ec0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000fb50 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec000fcf0 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed014e240 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000fea0 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.429+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed014e800 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.430+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed014ef10 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.431+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ec0012040 con 0x7f0eb8004d40 2024-04-26T21:18:28.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.431+0000 7f0eb7fff640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 0x7f0ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.431+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f0ec008c2e0 con 0x7f0eb8004d40 2024-04-26T21:18:28.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.432+0000 7f0eceffd640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 0x7f0ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.432+0000 7f0eceffd640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 0x7f0ebc06edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4000fb0 tx=0x7f0ec4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.436+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ec0019030 con 0x7f0eb8004d40 2024-04-26T21:18:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.619+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ed006a290 con 0x7f0eb8004d40 2024-04-26T21:18:28.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.619+0000 7f0eb7fff640 1 -- 172.21.15.40:0/778966339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f0ec005a660 con 0x7f0eb8004d40 2024-04-26T21:18:28.620 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:28.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 msgr2=0x7f0ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 0x7f0ebc06edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4000fb0 tx=0x7f0ec4009040 comp rx=0 tx=0).stop 2024-04-26T21:18:28.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 msgr2=0x7f0ed00b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00119d0 tx=0x7f0ec0011e90 comp rx=0 tx=0).stop 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 shutdown_connections 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0ebc06c8f0 0x7f0ebc06edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed014a880 0x7f0ed015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed014f2e0 0x7f0ed014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 --2- 172.21.15.40:0/778966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb8004d40 0x7f0ed00b04b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 >> 172.21.15.40:0/778966339 conn(0x7f0ed006f050 msgr2=0x7f0ed00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 shutdown_connections 2024-04-26T21:18:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.621+0000 7f0ed603e640 1 -- 172.21.15.40:0/778966339 wait complete. 2024-04-26T21:18:28.633 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.735+0000 7fbcbc5ac640 1 Processor -- start 2024-04-26T21:18:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.735+0000 7fbcbc5ac640 1 -- start start 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcbc5ac640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcb40d3340 0x7fbcb40d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcbc5ac640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb40b6aa0 con 0x7fbcb415f5b0 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcbc5ac640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb40d3eb0 con 0x7fbcb415f990 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcbc5ac640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb40d4190 con 0x7fbcb40d3340 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcba348640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcb415f5b0 0x7fbcb40b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60906/0 (socket says 172.21.15.40:60906) 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcba348640 1 -- 172.21.15.40:0/3441775885 learned_addr learned my addr 172.21.15.40:0/3441775885 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcbab49640 1 --2- 172.21.15.40:0/3441775885 >> v2:172.21.15.184:3300/0 conn(0x7fbcb40d3340 0x7fbcb40d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3165154418 0 0) 0x7fbcb40b6aa0 con 0x7fbcb415f5b0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.736+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc9c003520 con 0x7fbcb415f5b0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 67785605 0 0) 0x7fbc9c003520 con 0x7fbcb415f5b0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 >> v2:172.21.15.184:3300/0 conn(0x7fbcb40d3340 msgr2=0x7fbcb40d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 --2- 172.21.15.40:0/3441775885 >> v2:172.21.15.184:3300/0 conn(0x7fbcb40d3340 0x7fbcb40d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 >> v1:172.21.15.184:6789/0 conn(0x7fbcb415f990 legacy=0x7fbcb40d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb40d0200 con 0x7fbcb415f5b0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fbcac001670 con 0x7fbcb415f5b0 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 >> v1:172.21.15.40:6789/0 conn(0x7fbcb415f5b0 legacy=0x7fbcb40b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc9c0039b0 0x7fbc9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 0x7fbc9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c0051c0 0x7fbc9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40d4190 con 0x7fbc9c004800 2024-04-26T21:18:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c00c160 con 0x7fbc9c0051c0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9c00c360 con 0x7fbc9c0039b0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 0x7fbc9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcba348640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc9c0039b0 0x7fbc9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcbab49640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c0051c0 0x7fbc9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 -- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc9c0039b0 msgr2=0x7fbc9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc9c0039b0 0x7fbc9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 -- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c0051c0 msgr2=0x7fbc9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c0051c0 0x7fbc9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcbab49640 1 -- 172.21.15.40:0/3441775885 reap_dead start 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.737+0000 7fbcb9b47640 1 -- 172.21.15.40:0/3441775885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c003520 con 0x7fbc9c004800 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.738+0000 7fbcb9b47640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 0x7fbc9c004c80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbca8001750 tx=0x7fbc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0c4a280f834a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.738+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8001370 con 0x7fbc9c004800 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.738+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca80052d0 con 0x7fbc9c004800 2024-04-26T21:18:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.738+0000 7fbcb9346640 1 -- 172.21.15.40:0/3441775885 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca802abc0 con 0x7fbc9c004800 2024-04-26T21:18:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 msgr2=0x7fbc9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 0x7fbc9c004c80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbca8001750 tx=0x7fbc9c005830 comp rx=0 tx=0).stop 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/3441775885 shutdown_connections 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/3441775885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc9c004800 0x7fbc9c004c80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/3441775885 >> 172.21.15.40:0/3441775885 conn(0x7fbcb408ec40 msgr2=0x7fbcb409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/3441775885 shutdown_connections 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.739+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/3441775885 wait complete. 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.740+0000 7fbcbc5ac640 1 Processor -- start 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.740+0000 7fbcbc5ac640 1 -- start start 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c004800 0x7fbcb40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb41540d0 0x7fbcb4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40d3910 con 0x7fbcb41f89b0 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40d3790 con 0x7fbc9c004800 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbc5ac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb40d3a90 con 0x7fbcb41540d0 2024-04-26T21:18:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43750/0 (socket says 172.21.15.40:43750) 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 -- 172.21.15.40:0/2093557533 learned_addr learned my addr 172.21.15.40:0/2093557533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcba348640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c004800 0x7fbcb40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcb9b47640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb41540d0 0x7fbcb4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 -- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb41540d0 msgr2=0x7fbcb4154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb41540d0 0x7fbcb4154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 -- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c004800 msgr2=0x7fbcb40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c004800 0x7fbcb40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 -- 172.21.15.40:0/2093557533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb415b9c0 con 0x7fbcb41f89b0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.741+0000 7fbcbab49640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40cfc30 tx=0x7fbca400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.742+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400fdb0 con 0x7fbcb41f89b0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.742+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca401d070 con 0x7fbcb41f89b0 2024-04-26T21:18:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.742+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb415bca0 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.742+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4015670 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.742+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb415c260 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.743+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb406a290 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.743+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbca4015810 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.744+0000 7fbcb2ffd640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 0x7fbca006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.744+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fbca408c4a0 con 0x7fbcb41f89b0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.744+0000 7fbcba348640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 0x7fbca006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.744+0000 7fbcba348640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 0x7fbca006ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000fb0 tx=0x7fbcac0078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.748+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbca4010040 con 0x7fbcb41f89b0 2024-04-26T21:18:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.936+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbcb415d020 con 0x7fbcb41f89b0 2024-04-26T21:18:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.936+0000 7fbcb2ffd640 1 -- 172.21.15.40:0/2093557533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fbca405a1e0 con 0x7fbcb41f89b0 2024-04-26T21:18:28.937 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 msgr2=0x7fbca006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 0x7fbca006ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000fb0 tx=0x7fbcac0078f0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 msgr2=0x7fbcb41f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40cfc30 tx=0x7fbca400f580 comp rx=0 tx=0).stop 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 shutdown_connections 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbca006c920 0x7fbca006ede0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb41f89b0 0x7fbcb41f8e30 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcb41540d0 0x7fbcb4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 --2- 172.21.15.40:0/2093557533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc9c004800 0x7fbcb40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:28.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 >> 172.21.15.40:0/2093557533 conn(0x7fbcb408ec40 msgr2=0x7fbcb415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:28.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 shutdown_connections 2024-04-26T21:18:28.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:28.938+0000 7fbcbc5ac640 1 -- 172.21.15.40:0/2093557533 wait complete. 2024-04-26T21:18:28.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.053+0000 7f97873ac640 1 Processor -- start 2024-04-26T21:18:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.053+0000 7f97873ac640 1 -- start start 2024-04-26T21:18:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f97873ac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f978006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f97873ac640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97800b33e0 0x7f9780163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f97873ac640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f978009c700 con 0x7f97800b3c20 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f97873ac640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f97800b4480 con 0x7f97800b33e0 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f97873ac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f978006e050 con 0x7f97800d0010 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f9784947640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f97800b3c20 0x7f97800b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60920/0 (socket says 172.21.15.40:60920) 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f9784947640 1 -- 172.21.15.40:0/1089474899 learned_addr learned my addr 172.21.15.40:0/1089474899 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f9785148640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f978006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733791553 0 0) 0x7f978009c700 con 0x7f97800b3c20 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f9785949640 1 --2- 172.21.15.40:0/1089474899 >> v2:172.21.15.184:3300/0 conn(0x7f97800b33e0 0x7f9780163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.054+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9768003520 con 0x7f97800b3c20 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2595545164 0 0) 0x7f9768003520 con 0x7f97800b3c20 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 msgr2=0x7f978006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f978006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 >> v2:172.21.15.184:3300/0 conn(0x7f97800b33e0 msgr2=0x7f9780163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 --2- 172.21.15.40:0/1089474899 >> v2:172.21.15.184:3300/0 conn(0x7f97800b33e0 0x7f9780163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97800d1c50 con 0x7f97800b3c20 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9774001670 con 0x7f97800b3c20 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 >> v1:172.21.15.40:6789/0 conn(0x7f97800b3c20 legacy=0x7f97800b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f9768003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9768004800 0x7f9768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f97680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97800b4480 con 0x7f97680039b0 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f978006e050 con 0x7f9768004800 2024-04-26T21:18:29.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f976800b9c0 con 0x7f97800d0010 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f9768003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785949640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f97680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9784947640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9768004800 0x7f9768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 -- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 msgr2=0x7f97680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f97680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 -- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9768004800 msgr2=0x7f9768004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9768004800 0x7f9768004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785148640 1 -- 172.21.15.40:0/1089474899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9768003520 con 0x7f97680039b0 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9785949640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97800d0010 0x7f97680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.055+0000 7f9784947640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9768004800 0x7f9768004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f9785148640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f9768003dd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9778009ac0 tx=0x7f9768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd29d5515643d168 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778036070 con 0x7f97680039b0 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9778031400 con 0x7f97680039b0 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f977ffff640 1 -- 172.21.15.40:0/1089474899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778009690 con 0x7f97680039b0 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f97873ac640 1 -- 172.21.15.40:0/1089474899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 msgr2=0x7f9768003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f97873ac640 1 --2- 172.21.15.40:0/1089474899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f9768003dd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9778009ac0 tx=0x7f9768005830 comp rx=0 tx=0).stop 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.056+0000 7f9785949640 1 -- 172.21.15.40:0/1089474899 reap_dead start 2024-04-26T21:18:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.057+0000 7f97873ac640 1 -- 172.21.15.40:0/1089474899 shutdown_connections 2024-04-26T21:18:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.057+0000 7f97873ac640 1 -- 172.21.15.40:0/1089474899 >> 172.21.15.40:0/1089474899 conn(0x7f978009ba40 msgr2=0x7f978009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.057+0000 7f97873ac640 1 -- 172.21.15.40:0/1089474899 shutdown_connections 2024-04-26T21:18:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.057+0000 7f97873ac640 1 -- 172.21.15.40:0/1089474899 wait complete. 2024-04-26T21:18:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.058+0000 7f97873ac640 1 Processor -- start 2024-04-26T21:18:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.058+0000 7f97873ac640 1 -- start start 2024-04-26T21:18:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.058+0000 7f97873ac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.058+0000 7f97873ac640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978015b1b0 0x7f978009f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f97873ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 0x7f97801f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f97873ac640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97800b39f0 con 0x7f97680039b0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f97873ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97800b3870 con 0x7f97801f57f0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f97873ac640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f978006db10 con 0x7f978015b1b0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43776/0 (socket says 172.21.15.40:43776) 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 -- 172.21.15.40:0/2800897799 learned_addr learned my addr 172.21.15.40:0/2800897799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9784947640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978015b1b0 0x7f978009f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785949640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 0x7f97801f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 -- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978015b1b0 msgr2=0x7f978009f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978015b1b0 0x7f978009f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 -- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 msgr2=0x7f97801f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 0x7f97801f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 -- 172.21.15.40:0/2800897799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97801f8b40 con 0x7f97680039b0 2024-04-26T21:18:29.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785949640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 0x7f97801f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f9785148640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9778009ac0 tx=0x7f9778004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778040020 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9778003bc0 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97801f8e20 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.059+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778036040 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.060+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978009d060 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.060+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978006a290 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.061+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9778043020 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.061+0000 7f977dffb640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.061+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f97780abcc0 con 0x7f97680039b0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.062+0000 7f9784947640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.062+0000 7f9784947640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 0x7f976c06ede0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f97800b4800 tx=0x7f9774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.066+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97780b2080 con 0x7f97680039b0 2024-04-26T21:18:29.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.255+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f978009ef90 con 0x7f97680039b0 2024-04-26T21:18:29.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.256+0000 7f977dffb640 1 -- 172.21.15.40:0/2800897799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f977804a070 con 0x7f97680039b0 2024-04-26T21:18:29.257 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 msgr2=0x7f976c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 0x7f976c06ede0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f97800b4800 tx=0x7f9774009040 comp rx=0 tx=0).stop 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 msgr2=0x7f978009f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9778009ac0 tx=0x7f9778004730 comp rx=0 tx=0).stop 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 shutdown_connections 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97801f57f0 0x7f97801f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978015b1b0 0x7f978009f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 --2- 172.21.15.40:0/2800897799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97680039b0 0x7f978009f1b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 >> 172.21.15.40:0/2800897799 conn(0x7f978009ba40 msgr2=0x7f97800b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 shutdown_connections 2024-04-26T21:18:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.258+0000 7f97873ac640 1 -- 172.21.15.40:0/2800897799 wait complete. 2024-04-26T21:18:29.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.369+0000 7fcf6ed71640 1 Processor -- start 2024-04-26T21:18:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.369+0000 7fcf6ed71640 1 -- start start 2024-04-26T21:18:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6ed71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680d11b0 0x7fcf6806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6ed71640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf6806e2c0 0x7fcf6815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6ed71640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680d1740 con 0x7fcf680cf2e0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6ed71640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6806e750 con 0x7fcf6806e2c0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6ed71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680d1a20 con 0x7fcf680d11b0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6cb0d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcf680cf2e0 0x7fcf6806d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60932/0 (socket says 172.21.15.40:60932) 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6cb0d640 1 -- 172.21.15.40:0/208498488 learned_addr learned my addr 172.21.15.40:0/208498488 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf67fff640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680d11b0 0x7fcf6806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf6d30e640 1 --2- 172.21.15.40:0/208498488 >> v2:172.21.15.184:3300/0 conn(0x7fcf6806e2c0 0x7fcf6815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.370+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2216866758 0 0) 0x7fcf680d1740 con 0x7fcf680cf2e0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf54003520 con 0x7fcf680cf2e0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128060795 0 0) 0x7fcf54003520 con 0x7fcf680cf2e0 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680d11b0 msgr2=0x7fcf6806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680d11b0 0x7fcf6806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 >> v2:172.21.15.184:3300/0 conn(0x7fcf6806e2c0 msgr2=0x7fcf6815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 --2- 172.21.15.40:0/208498488 >> v2:172.21.15.184:3300/0 conn(0x7fcf6806e2c0 0x7fcf6815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680d1d00 con 0x7fcf680cf2e0 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fcf60001670 con 0x7fcf680cf2e0 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 >> v1:172.21.15.40:6789/0 conn(0x7fcf680cf2e0 legacy=0x7fcf6806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf540039b0 0x7fcf54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf54004800 0x7fcf54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540051c0 0x7fcf5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6806e750 con 0x7fcf540039b0 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680d1a20 con 0x7fcf540051c0 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5400bf00 con 0x7fcf54004800 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf67fff640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf680d11b0 0x7fcf6806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf540039b0 0x7fcf54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.371+0000 7fcf67fff640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf54004800 0x7fcf54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6d30e640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540051c0 0x7fcf5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6cb0d640 1 -- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf54004800 msgr2=0x7fcf54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf54004800 0x7fcf54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6cb0d640 1 -- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540051c0 msgr2=0x7fcf5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540051c0 0x7fcf5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.372+0000 7fcf6cb0d640 1 -- 172.21.15.40:0/208498488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf54003520 con 0x7fcf540039b0 2024-04-26T21:18:29.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf540039b0 0x7fcf54003dd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c190 tx=0x7fcf54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aca37d0f9082a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60007c80 con 0x7fcf540039b0 2024-04-26T21:18:29.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6002e070 con 0x7fcf540039b0 2024-04-26T21:18:29.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf677fe640 1 -- 172.21.15.40:0/208498488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60005d10 con 0x7fcf540039b0 2024-04-26T21:18:29.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6ed71640 1 -- 172.21.15.40:0/208498488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf540039b0 msgr2=0x7fcf54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/208498488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf540039b0 0x7fcf54003dd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c190 tx=0x7fcf54005830 comp rx=0 tx=0).stop 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6d30e640 1 -- 172.21.15.40:0/208498488 reap_dead start 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6ed71640 1 -- 172.21.15.40:0/208498488 shutdown_connections 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.373+0000 7fcf6ed71640 1 -- 172.21.15.40:0/208498488 >> 172.21.15.40:0/208498488 conn(0x7fcf680b4a60 msgr2=0x7fcf680b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.374+0000 7fcf6ed71640 1 -- 172.21.15.40:0/208498488 shutdown_connections 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.374+0000 7fcf6ed71640 1 -- 172.21.15.40:0/208498488 wait complete. 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.375+0000 7fcf6ed71640 1 Processor -- start 2024-04-26T21:18:29.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.375+0000 7fcf6ed71640 1 -- start start 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540039b0 0x7fcf680b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6815d990 0x7fcf680b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680cf6c0 con 0x7fcf6809cf80 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680d22c0 con 0x7fcf540039b0 2024-04-26T21:18:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6ed71640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680cf840 con 0x7fcf6815d990 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6cb0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540039b0 0x7fcf680b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43786/0 (socket says 172.21.15.40:43786) 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 -- 172.21.15.40:0/3107005000 learned_addr learned my addr 172.21.15.40:0/3107005000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf67fff640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6815d990 0x7fcf680b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 -- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6815d990 msgr2=0x7fcf680b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6815d990 0x7fcf680b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 -- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540039b0 msgr2=0x7fcf680b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540039b0 0x7fcf680b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.376+0000 7fcf6d30e640 1 -- 172.21.15.40:0/3107005000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680b1770 con 0x7fcf6809cf80 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf6d30e640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcf68073840 tx=0x7fcf50011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004240 con 0x7fcf6809cf80 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5001b070 con 0x7fcf6809cf80 2024-04-26T21:18:29.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000f770 con 0x7fcf6809cf80 2024-04-26T21:18:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf680b1a50 con 0x7fcf6809cf80 2024-04-26T21:18:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.377+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6815e200 con 0x7fcf6809cf80 2024-04-26T21:18:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.378+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6806a290 con 0x7fcf6809cf80 2024-04-26T21:18:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.378+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf50012040 con 0x7fcf6809cf80 2024-04-26T21:18:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.379+0000 7fcf657fa640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 0x7fcf4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.379+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fcf5008b810 con 0x7fcf6809cf80 2024-04-26T21:18:29.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.379+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 0x7fcf4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.380+0000 7fcf6cb0d640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 0x7fcf4806ede0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcf60032000 tx=0x7fcf60007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.384+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf500595d0 con 0x7fcf6809cf80 2024-04-26T21:18:29.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.605+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf68162960 con 0x7fcf6809cf80 2024-04-26T21:18:29.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.605+0000 7fcf657fa640 1 -- 172.21.15.40:0/3107005000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5187 (secure 0 0 0) 0x7fcf5005d8d0 con 0x7fcf6809cf80 2024-04-26T21:18:29.607 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:29.607 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:17:17.395375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":23,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:18:29.607 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 50 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 msgr2=0x7fcf4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 0x7fcf4806ede0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcf60032000 tx=0x7fcf60007000 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 msgr2=0x7fcf680b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcf68073840 tx=0x7fcf50011cf0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 shutdown_connections 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4806c920 0x7fcf4806ede0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6809cf80 0x7fcf680b1d10 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6815d990 0x7fcf680b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 --2- 172.21.15.40:0/3107005000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf540039b0 0x7fcf680b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.607+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 >> 172.21.15.40:0/3107005000 conn(0x7fcf680b4a60 msgr2=0x7fcf680c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.608+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 shutdown_connections 2024-04-26T21:18:29.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.608+0000 7fcf6ed71640 1 -- 172.21.15.40:0/3107005000 wait complete. 2024-04-26T21:18:29.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 Processor -- start 2024-04-26T21:18:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 -- start start 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 0x7f1b6406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b640d1740 con 0x7f1b6406e490 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b640d1a20 con 0x7f1b640d11b0 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.719+0000 7f1b6b03f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b640d1d00 con 0x7f1b640cf2e0 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b695dc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b6406e490 0x7f1b6415ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60944/0 (socket says 172.21.15.40:60944) 2024-04-26T21:18:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b695dc640 1 -- 172.21.15.40:0/1368936338 learned_addr learned my addr 172.21.15.40:0/1368936338 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b68ddb640 1 --2- 172.21.15.40:0/1368936338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 0x7f1b6406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560586923 0 0) 0x7f1b640d1740 con 0x7f1b6406e490 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b54003520 con 0x7f1b6406e490 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526972133 0 0) 0x7f1b640d1a20 con 0x7f1b640d11b0 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b68ddb640 1 -- 172.21.15.40:0/1368936338 >> v1:172.21.15.184:6789/0 conn(0x7f1b640d11b0 legacy=0x7f1b6406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b68ddb640 1 -- 172.21.15.40:0/1368936338 >> v1:172.21.15.40:6789/0 conn(0x7f1b6406e490 legacy=0x7f1b6415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.720+0000 7f1b68ddb640 1 -- 172.21.15.40:0/1368936338 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b640d1fe0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b68ddb640 1 --2- 172.21.15.40:0/1368936338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 0x7f1b6406d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b580097d0 tx=0x7f1b5802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c68aabba5a0a1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58037070 con 0x7f1b640cf2e0 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5802ac10 con 0x7f1b640cf2e0 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b637fe640 1 -- 172.21.15.40:0/1368936338 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0019f0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b6b03f640 1 -- 172.21.15.40:0/1368936338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 msgr2=0x7f1b6406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.721+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/1368936338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 0x7f1b6406d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b580097d0 tx=0x7f1b5802a180 comp rx=0 tx=0).stop 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.722+0000 7f1b6b03f640 1 -- 172.21.15.40:0/1368936338 shutdown_connections 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.722+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/1368936338 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640cf2e0 0x7f1b6406d4d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.722+0000 7f1b6b03f640 1 -- 172.21.15.40:0/1368936338 >> 172.21.15.40:0/1368936338 conn(0x7f1b640b4a60 msgr2=0x7f1b640b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.722+0000 7f1b6b03f640 1 -- 172.21.15.40:0/1368936338 shutdown_connections 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.722+0000 7f1b6b03f640 1 -- 172.21.15.40:0/1368936338 wait complete. 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.723+0000 7f1b6b03f640 1 Processor -- start 2024-04-26T21:18:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.723+0000 7f1b6b03f640 1 -- start start 2024-04-26T21:18:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6406e490 0x7f1b641f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640d11b0 0x7f1b640a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b640cf6c0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6406da10 con 0x7f1b6406e490 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1b640cf840 con 0x7f1b640d11b0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43800/0 (socket says 172.21.15.40:43800) 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 -- 172.21.15.40:0/2540313580 learned_addr learned my addr 172.21.15.40:0/2540313580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b68ddb640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6406e490 0x7f1b641f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b695dc640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640d11b0 0x7f1b640a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 -- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640d11b0 msgr2=0x7f1b640a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640d11b0 0x7f1b640a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 -- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6406e490 msgr2=0x7f1b641f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6406e490 0x7f1b641f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 -- 172.21.15.40:0/2540313580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6409f690 con 0x7f1b640cf2e0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b63fff640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c410 tx=0x7f1b5000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50017070 con 0x7f1b640cf2e0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b500035f0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6409f970 con 0x7f1b640cf2e0 2024-04-26T21:18:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b500039e0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.724+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6409ff30 con 0x7f1b640cf2e0 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.725+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b640a0420 con 0x7f1b640cf2e0 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.726+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b5001a070 con 0x7f1b640cf2e0 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.726+0000 7f1b617fa640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.726+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f1b5008aeb0 con 0x7f1b640cf2e0 2024-04-26T21:18:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.727+0000 7f1b68ddb640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.727+0000 7f1b68ddb640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 0x7f1b4806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b58034000 tx=0x7f1b5802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.731+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b50058d00 con 0x7f1b640cf2e0 2024-04-26T21:18:29.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.915+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b6406a290 con 0x7f1b640cf2e0 2024-04-26T21:18:29.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.916+0000 7f1b617fa640 1 -- 172.21.15.40:0/2540313580 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16710 (secure 0 0 0) 0x7f1b5005d000 con 0x7f1b640cf2e0 2024-04-26T21:18:29.917 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:29.917 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:16:42.760777+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 msgr2=0x7f1b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 0x7f1b4806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b58034000 tx=0x7f1b5802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 msgr2=0x7f1b6409efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c410 tx=0x7f1b5000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 shutdown_connections 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1b4806c8f0 0x7f1b4806edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b640d11b0 0x7f1b640a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b640cf2e0 0x7f1b6409efb0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 --2- 172.21.15.40:0/2540313580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6406e490 0x7f1b641f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 >> 172.21.15.40:0/2540313580 conn(0x7f1b640b4a60 msgr2=0x7f1b640b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:29.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 shutdown_connections 2024-04-26T21:18:29.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:29.918+0000 7f1b6b03f640 1 -- 172.21.15.40:0/2540313580 wait complete. 2024-04-26T21:18:29.930 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:18:29.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.029+0000 7f110c3e2640 1 Processor -- start 2024-04-26T21:18:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.029+0000 7f110c3e2640 1 -- start start 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.030+0000 7f110c3e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f110415d3b0 0x7f11040735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.030+0000 7f110c3e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f11040d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.030+0000 7f110c3e2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11040b6b80 con 0x7f1104073b10 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.030+0000 7f110c3e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f11040aedf0 con 0x7f110415d3b0 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.030+0000 7f110c3e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11040b6930 con 0x7f110406c9b0 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110997d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1104073b10 0x7f110406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60960/0 (socket says 172.21.15.40:60960) 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110997d640 1 -- 172.21.15.40:0/2638663380 learned_addr learned my addr 172.21.15.40:0/2638663380 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a97f640 1 --2- 172.21.15.40:0/2638663380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f11040d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a17e640 1 --2- 172.21.15.40:0/2638663380 >> v2:172.21.15.184:3300/0 conn(0x7f110415d3b0 0x7f11040735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520169488 0 0) 0x7f11040b6b80 con 0x7f1104073b10 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10f0003520 con 0x7f1104073b10 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1256070642 0 0) 0x7f10f0003520 con 0x7f1104073b10 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a97f640 1 -- 172.21.15.40:0/2638663380 >> v2:172.21.15.184:3300/0 conn(0x7f110415d3b0 msgr2=0x7f11040735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a97f640 1 --2- 172.21.15.40:0/2638663380 >> v2:172.21.15.184:3300/0 conn(0x7f110415d3b0 0x7f11040735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a97f640 1 -- 172.21.15.40:0/2638663380 >> v1:172.21.15.40:6789/0 conn(0x7f1104073b10 legacy=0x7f110406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.031+0000 7f110a97f640 1 -- 172.21.15.40:0/2638663380 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11040b6e60 con 0x7f110406c9b0 2024-04-26T21:18:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.032+0000 7f110a97f640 1 --2- 172.21.15.40:0/2638663380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f11040d0690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009d90 tx=0x7f10fc02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28f90ebb4abbea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.032+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100001670 con 0x7f110406c9b0 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.032+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc007540 con 0x7f110406c9b0 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.032+0000 7f110917c640 1 -- 172.21.15.40:0/2638663380 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc007910 con 0x7f110406c9b0 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 -- 172.21.15.40:0/2638663380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 msgr2=0x7f11040d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 --2- 172.21.15.40:0/2638663380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f11040d0690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009d90 tx=0x7f10fc02a560 comp rx=0 tx=0).stop 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 -- 172.21.15.40:0/2638663380 shutdown_connections 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 --2- 172.21.15.40:0/2638663380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f11040d0690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 --2- 172.21.15.40:0/2638663380 >> v2:172.21.15.184:3300/0 conn(0x7f110415d3b0 0x7f11040735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 -- 172.21.15.40:0/2638663380 >> 172.21.15.40:0/2638663380 conn(0x7f11040b3b10 msgr2=0x7f11040b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 -- 172.21.15.40:0/2638663380 shutdown_connections 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.033+0000 7f110c3e2640 1 -- 172.21.15.40:0/2638663380 wait complete. 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.034+0000 7f110c3e2640 1 Processor -- start 2024-04-26T21:18:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.034+0000 7f110c3e2640 1 -- start start 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f110406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 0x7f11041f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f110415d790 con 0x7f1104073b10 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11040d0170 con 0x7f110415d3b0 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f110415d910 con 0x7f110406c9b0 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43816/0 (socket says 172.21.15.40:43816) 2024-04-26T21:18:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 -- 172.21.15.40:0/1928278732 learned_addr learned my addr 172.21.15.40:0/1928278732 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110a17e640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f110406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110a97f640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 0x7f11041f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 -- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 msgr2=0x7f110406fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f110406fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 -- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 msgr2=0x7f11041f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 0x7f11041f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 -- 172.21.15.40:0/1928278732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110415e9e0 con 0x7f1104073b10 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110a97f640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 0x7f11041f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110a17e640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f110406fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110997d640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f110000c780 tx=0x7f110000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100017070 con 0x7f1104073b10 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100005ad0 con 0x7f1104073b10 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100007070 con 0x7f1104073b10 2024-04-26T21:18:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.035+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110415ec80 con 0x7f1104073b10 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.036+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110415f240 con 0x7f1104073b10 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.036+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1104154b10 con 0x7f1104073b10 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.037+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f110001a070 con 0x7f1104073b10 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.037+0000 7f10faffd640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 0x7f10e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.037+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f110008b0d0 con 0x7f1104073b10 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.038+0000 7f110a17e640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 0x7f10e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.038+0000 7f110a17e640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 0x7f10e806edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009af0 tx=0x7f10f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.042+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1104154b10 con 0x7f1104073b10 2024-04-26T21:18:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.266+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11040693d0 con 0x7f1104073b10 2024-04-26T21:18:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.267+0000 7f10faffd640 1 -- 172.21.15.40:0/1928278732 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5187 (secure 0 0 0) 0x7f11000594e0 con 0x7f1104073b10 2024-04-26T21:18:30.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:30.268 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:18:29.401222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":41,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:18:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 51 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 msgr2=0x7f10e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 0x7f10e806edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009af0 tx=0x7f10f4008040 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 msgr2=0x7f110415e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f110000c780 tx=0x7f110000cc40 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 shutdown_connections 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10e806c8f0 0x7f10e806edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f110415d3b0 0x7f11041f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1104073b10 0x7f110415e4a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 --2- 172.21.15.40:0/1928278732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f110406c9b0 0x7f110406fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 >> 172.21.15.40:0/1928278732 conn(0x7f11040b3b10 msgr2=0x7f11040d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 shutdown_connections 2024-04-26T21:18:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.269+0000 7f110c3e2640 1 -- 172.21.15.40:0/1928278732 wait complete. 2024-04-26T21:18:30.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.380+0000 7fe3e87c9640 1 Processor -- start 2024-04-26T21:18:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.380+0000 7fe3e87c9640 1 -- start start 2024-04-26T21:18:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e87c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3e015d380 0x7fe3e00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e87c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e0160d50 0x7fe3e01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e87c9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3e00d0be0 con 0x7fe3e00b56d0 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e87c9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe3e00d0ec0 con 0x7fe3e015d380 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e87c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e00d11a0 con 0x7fe3e0160d50 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e6565640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3e015d380 0x7fe3e00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e5d64640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe3e00b56d0 0x7fe3e00d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60972/0 (socket says 172.21.15.40:60972) 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.381+0000 7fe3e5d64640 1 -- 172.21.15.40:0/2820822656 learned_addr learned my addr 172.21.15.40:0/2820822656 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e6565640 1 -- 172.21.15.40:0/2820822656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e0160d50 msgr2=0x7fe3e01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2820822656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e0160d50 0x7fe3e01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e6565640 1 -- 172.21.15.40:0/2820822656 >> v1:172.21.15.40:6789/0 conn(0x7fe3e00b56d0 legacy=0x7fe3e00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e6565640 1 -- 172.21.15.40:0/2820822656 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e00d1480 con 0x7fe3e015d380 2024-04-26T21:18:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e5563640 1 -- 172.21.15.40:0/2820822656 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046860784 0 0) 0x7fe3e00d0be0 con 0x7fe3e00b56d0 2024-04-26T21:18:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2820822656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e015d380 0x7fe3e00ce590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0097a0 tx=0x7fe3dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bdd33875f7c078 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e5563640 1 -- 172.21.15.40:0/2820822656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc037070 con 0x7fe3e015d380 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e5563640 1 -- 172.21.15.40:0/2820822656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc02ae50 con 0x7fe3e015d380 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.382+0000 7fe3e5563640 1 -- 172.21.15.40:0/2820822656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc032660 con 0x7fe3e015d380 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2820822656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e015d380 msgr2=0x7fe3e00ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2820822656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e015d380 0x7fe3e00ce590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0097a0 tx=0x7fe3dc02a100 comp rx=0 tx=0).stop 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2820822656 shutdown_connections 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2820822656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e0160d50 0x7fe3e01611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2820822656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e015d380 0x7fe3e00ce590 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2820822656 >> 172.21.15.40:0/2820822656 conn(0x7fe3e00b4a60 msgr2=0x7fe3e00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2820822656 shutdown_connections 2024-04-26T21:18:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.383+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2820822656 wait complete. 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.384+0000 7fe3e87c9640 1 Processor -- start 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.384+0000 7fe3e87c9640 1 -- start start 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 0x7fe3e00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 0x7fe3e00a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e00b5dc0 con 0x7fe3e0160d50 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e00cedc0 con 0x7fe3e00b56d0 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e87c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e00b5f40 con 0x7fe3e015d380 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43830/0 (socket says 172.21.15.40:43830) 2024-04-26T21:18:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 -- 172.21.15.40:0/2294534815 learned_addr learned my addr 172.21.15.40:0/2294534815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 0x7fe3e00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e5d64640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 0x7fe3e00a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 -- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 msgr2=0x7fe3e00a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 0x7fe3e00a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 -- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 msgr2=0x7fe3e00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 0x7fe3e00ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6d66640 1 -- 172.21.15.40:0/2294534815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e00a0d50 con 0x7fe3e0160d50 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 0x7fe3e00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.385+0000 7fe3e5d64640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 0x7fe3e00a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3e6d66640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800b970 tx=0x7fe3d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8019070 con 0x7fe3e0160d50 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d800cb60 con 0x7fe3e0160d50 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8014440 con 0x7fe3e0160d50 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01f7360 con 0x7fe3e0160d50 2024-04-26T21:18:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.386+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01f7920 con 0x7fe3e0160d50 2024-04-26T21:18:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.387+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e006a290 con 0x7fe3e0160d50 2024-04-26T21:18:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.387+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3d801c070 con 0x7fe3e0160d50 2024-04-26T21:18:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.388+0000 7fe3d6ffd640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 0x7fe3c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.388+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7fe3d808b7b0 con 0x7fe3e0160d50 2024-04-26T21:18:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.388+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 0x7fe3c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.388+0000 7fe3e6565640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 0x7fe3c406ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc034000 tx=0x7fe3dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.392+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3d8059c50 con 0x7fe3e0160d50 2024-04-26T21:18:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.615+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3e0075660 con 0x7fe3e0160d50 2024-04-26T21:18:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.615+0000 7fe3d6ffd640 1 -- 172.21.15.40:0/2294534815 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5187 (secure 0 0 0) 0x7fe3d8059550 con 0x7fe3e0160d50 2024-04-26T21:18:30.616 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:15:56.722100+0000","modified":"2024-04-26T21:18:29.401222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6912":{"gid":6912,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":41,"addr":"172.21.15.184:6836/1963635253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1963635253},{"type":"v1","addr":"172.21.15.184:6836","nonce":1963635253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6912,"qdb_cluster":[6912]},"id":6}]} 2024-04-26T21:18:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 51 2024-04-26T21:18:30.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 msgr2=0x7fe3c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 0x7fe3c406ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc034000 tx=0x7fe3dc02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 msgr2=0x7fe3e0159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800b970 tx=0x7fe3d800be30 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 shutdown_connections 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe3c406c920 0x7fe3c406ede0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0160d50 0x7fe3e0159700 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e015d380 0x7fe3e00a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 --2- 172.21.15.40:0/2294534815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e00b56d0 0x7fe3e00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 >> 172.21.15.40:0/2294534815 conn(0x7fe3e00b4a60 msgr2=0x7fe3e00715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 shutdown_connections 2024-04-26T21:18:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.617+0000 7fe3e87c9640 1 -- 172.21.15.40:0/2294534815 wait complete. 2024-04-26T21:18:30.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:30.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.728+0000 7f8eb9f9e640 1 Processor -- start 2024-04-26T21:18:30.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 -- start start 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8eb40cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8eb40cdb80 0x7f8eb4072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb40ce070 con 0x7f8eb415c280 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40cfab0 con 0x7f8eb40cdb80 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.729+0000 7f8eb9f9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb40cfd90 con 0x7f8eb415bea0 2024-04-26T21:18:30.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8eb415c280 0x7f8eb40d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53018/0 (socket says 172.21.15.40:53018) 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb3fff640 1 -- 172.21.15.40:0/281482132 learned_addr learned my addr 172.21.15.40:0/281482132 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb8f9c640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8eb40cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb979d640 1 --2- 172.21.15.40:0/281482132 >> v2:172.21.15.184:3300/0 conn(0x7f8eb40cdb80 0x7f8eb4072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598775522 0 0) 0x7f8eb40ce070 con 0x7f8eb415c280 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea4003520 con 0x7f8eb415c280 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3365956346 0 0) 0x7f8ea4003520 con 0x7f8eb415c280 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 msgr2=0x7f8eb40cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8eb40cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 >> v2:172.21.15.184:3300/0 conn(0x7f8eb40cdb80 msgr2=0x7f8eb4072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 --2- 172.21.15.40:0/281482132 >> v2:172.21.15.184:3300/0 conn(0x7f8eb40cdb80 0x7f8eb4072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40d0070 con 0x7f8eb415c280 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f8ea0001670 con 0x7f8eb415c280 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 >> v1:172.21.15.40:6789/0 conn(0x7f8eb415c280 legacy=0x7f8eb40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40039b0 0x7f8ea4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8ea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40cfab0 con 0x7f8ea4004800 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40cfd90 con 0x7f8ea40039b0 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea400dbd0 con 0x7f8eb415bea0 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb8f9c640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40039b0 0x7f8ea4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.730+0000 7f8eb979d640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8ea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 -- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 msgr2=0x7f8ea40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb415bea0 0x7f8ea40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 -- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40039b0 msgr2=0x7f8ea4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40039b0 0x7f8ea4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 -- 172.21.15.40:0/281482132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea4003520 con 0x7f8ea4004800 2024-04-26T21:18:30.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37ad7d89df8e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0007d50 con 0x7f8ea4004800 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea0005ad0 con 0x7f8ea4004800 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.731+0000 7f8eb37fe640 1 -- 172.21.15.40:0/281482132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0007070 con 0x7f8ea4004800 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/281482132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 msgr2=0x7f8ea4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/281482132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea4005830 comp rx=0 tx=0).stop 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb979d640 1 -- 172.21.15.40:0/281482132 reap_dead start 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/281482132 shutdown_connections 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/281482132 >> 172.21.15.40:0/281482132 conn(0x7f8eb40d1d40 msgr2=0x7f8eb40d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/281482132 shutdown_connections 2024-04-26T21:18:30.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.732+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/281482132 wait complete. 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.733+0000 7f8eb9f9e640 1 Processor -- start 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.733+0000 7f8eb9f9e640 1 -- start start 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 0x7f8eb4155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb41f4cd0 0x7f8eb41f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4160b70 con 0x7f8ea4004800 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41609f0 con 0x7f8eb41f4cd0 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb9f9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb40cd640 con 0x7f8eb41559b0 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb8f9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb8f9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53940/0 (socket says 172.21.15.40:53940) 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb8f9c640 1 -- 172.21.15.40:0/470590307 learned_addr learned my addr 172.21.15.40:0/470590307 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:30.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb979d640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb41f4cd0 0x7f8eb41f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.734+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 0x7f8eb4155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 -- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 msgr2=0x7f8eb4155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 0x7f8eb4155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 -- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb41f4cd0 msgr2=0x7f8eb41f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb41f4cd0 0x7f8eb41f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 -- 172.21.15.40:0/470590307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb4156470 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 0x7f8eb4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb8f9c640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80118e0 tx=0x7f8ea8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800fa90 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea800fc30 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb4156750 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8014440 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.735+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb40c60f0 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.736+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb4074840 con 0x7f8ea4004800 2024-04-26T21:18:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.737+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ea8012040 con 0x7f8ea4004800 2024-04-26T21:18:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.737+0000 7f8eb17fa640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 0x7f8e9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.737+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f8ea808bfd0 con 0x7f8ea4004800 2024-04-26T21:18:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.738+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 0x7f8e9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.738+0000 7f8eb3fff640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 0x7f8e9806ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0033000 tx=0x7f8ea0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.742+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ea8019030 con 0x7f8ea4004800 2024-04-26T21:18:30.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.930+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8eb4074260 con 0x7f8ea4004800 2024-04-26T21:18:30.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.931+0000 7f8eb17fa640 1 -- 172.21.15.40:0/470590307 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16710 (secure 0 0 0) 0x7f8ea805a470 con 0x7f8ea4004800 2024-04-26T21:18:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:30.932 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":85,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:16:42.760777+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:15:55.191765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-26T21:15:55.661024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 msgr2=0x7f8e9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 0x7f8e9806ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0033000 tx=0x7f8ea0009030 comp rx=0 tx=0).stop 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 msgr2=0x7f8eb41f0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80118e0 tx=0x7f8ea8011da0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 shutdown_connections 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8e9806c920 0x7f8e9806ede0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8eb41f4cd0 0x7f8eb41f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8eb41559b0 0x7f8eb4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 --2- 172.21.15.40:0/470590307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ea4004800 0x7f8eb41f0fe0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 >> 172.21.15.40:0/470590307 conn(0x7f8eb40d1d40 msgr2=0x7f8eb41598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 shutdown_connections 2024-04-26T21:18:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:30.933+0000 7f8eb9f9e640 1 -- 172.21.15.40:0/470590307 wait complete. 2024-04-26T21:18:30.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:18:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.047+0000 7f9118971640 1 Processor -- start 2024-04-26T21:18:31.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.047+0000 7f9118971640 1 -- start start 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9118971640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f9114160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9118971640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91140b61a0 0x7f91140720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9118971640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91140b4260 con 0x7f91140b3de0 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9118971640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f91140b5140 con 0x7f91140b61a0 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9118971640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91140b4530 con 0x7f9114145980 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9111d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91140b3de0 0x7f91140b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53030/0 (socket says 172.21.15.40:53030) 2024-04-26T21:18:31.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9111d9b640 1 -- 172.21.15.40:0/2071752793 learned_addr learned my addr 172.21.15.40:0/2071752793 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2071752793 >> v2:172.21.15.184:3300/0 conn(0x7f91140b61a0 0x7f91140720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911259c640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f9114160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2514101143 0 0) 0x7f91140b4260 con 0x7f91140b3de0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90fc003520 con 0x7f91140b3de0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373346720 0 0) 0x7f90fc003520 con 0x7f91140b3de0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 msgr2=0x7f9114160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f9114160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 >> v2:172.21.15.184:3300/0 conn(0x7f91140b61a0 msgr2=0x7f91140720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 --2- 172.21.15.40:0/2071752793 >> v2:172.21.15.184:3300/0 conn(0x7f91140b61a0 0x7f91140720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91140b4810 con 0x7f91140b3de0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9108001690 con 0x7f91140b3de0 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 >> v1:172.21.15.40:6789/0 conn(0x7f91140b3de0 legacy=0x7f91140b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f90fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003f30 0x7f90fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc004d40 0x7f90fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91140b5140 con 0x7f90fc004d40 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91140b4530 con 0x7f90fc003f30 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.048+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc00dc20 con 0x7f9114145980 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc004d40 0x7f90fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9111d9b640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003f30 0x7f90fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f911259c640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f90fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 -- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 msgr2=0x7f90fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9114145980 0x7f90fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 -- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003f30 msgr2=0x7f90fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc003f30 0x7f90fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.049+0000 7f9112d9d640 1 -- 172.21.15.40:0/2071752793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc003520 con 0x7f90fc004d40 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc004d40 0x7f90fc0051c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f910c009af0 tx=0x7f90fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bc015d070ecb1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c036070 con 0x7f90fc004d40 2024-04-26T21:18:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910c031400 con 0x7f90fc004d40 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f911159a640 1 -- 172.21.15.40:0/2071752793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c009690 con 0x7f90fc004d40 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 -- 172.21.15.40:0/2071752793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc004d40 msgr2=0x7f90fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 --2- 172.21.15.40:0/2071752793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc004d40 0x7f90fc0051c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f910c009af0 tx=0x7f90fc005830 comp rx=0 tx=0).stop 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9112d9d640 1 -- 172.21.15.40:0/2071752793 reap_dead start 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 -- 172.21.15.40:0/2071752793 shutdown_connections 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 -- 172.21.15.40:0/2071752793 >> 172.21.15.40:0/2071752793 conn(0x7f91140d0d30 msgr2=0x7f91140d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 -- 172.21.15.40:0/2071752793 shutdown_connections 2024-04-26T21:18:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.050+0000 7f9118971640 1 -- 172.21.15.40:0/2071752793 wait complete. 2024-04-26T21:18:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.052+0000 7f9118971640 1 Processor -- start 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.052+0000 7f9118971640 1 -- start start 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.052+0000 7f9118971640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004d40 0x7f9114151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.052+0000 7f9118971640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140ae920 0x7f91140af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f911415d0b0 0x7f911415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9114160630 con 0x7f91140ae920 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91140b4dd0 con 0x7f90fc004d40 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91141607b0 con 0x7f911415d0b0 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140ae920 0x7f91140af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140ae920 0x7f91140af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53964/0 (socket says 172.21.15.40:53964) 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 -- 172.21.15.40:0/2280673171 learned_addr learned my addr 172.21.15.40:0/2280673171 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f911259c640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004d40 0x7f9114151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9112d9d640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f911415d0b0 0x7f911415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 -- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f911415d0b0 msgr2=0x7f911415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f911415d0b0 0x7f911415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 -- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004d40 msgr2=0x7f9114151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004d40 0x7f9114151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 -- 172.21.15.40:0/2280673171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91140b0020 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f911259c640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc004d40 0x7f9114151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9111d9b640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140ae920 0x7f91140af7d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9108007340 tx=0x7f9108007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800f940 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9108018070 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 -- 172.21.15.40:0/2280673171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911415c5d0 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800fd20 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.053+0000 7f9118971640 1 -- 172.21.15.40:0/2280673171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911415cb90 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.054+0000 7f9118971640 1 -- 172.21.15.40:0/2280673171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911415cd90 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.055+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f910801d460 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.055+0000 7f9102ffd640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90f406c920 0x7f90f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.055+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f910808bc90 con 0x7f91140ae920 2024-04-26T21:18:31.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.055+0000 7f911259c640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90f406c920 0x7f90f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.056+0000 7f911259c640 1 --2- 172.21.15.40:0/2280673171 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90f406c920 0x7f90f406ede0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9104007440 tx=0x7f910400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.059+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f911415cd90 con 0x7f91140ae920 2024-04-26T21:18:31.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.283+0000 7f9118971640 1 -- 172.21.15.40:0/2280673171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91140cddd0 con 0x7f91140ae920 2024-04-26T21:18:31.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.283+0000 7f9102ffd640 1 -- 172.21.15.40:0/2280673171 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v51) ==== 316+0+0 (secure 0 0 0) 0x7f910805a130 con 0x7f91140ae920 2024-04-26T21:18:31.284 INFO:teuthology.orchestra.run.smithi040.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-04-26T21:18:31.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:18:31.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-04-26T21:18:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.391+0000 7f163ce37640 1 Processor -- start 2024-04-26T21:18:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.391+0000 7f163ce37640 1 -- start start 2024-04-26T21:18:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163ce37640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1638143840 0x7f16380b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163ce37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163815dca0 con 0x7f16380b4520 2024-04-26T21:18:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163ce37640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163815df80 con 0x7f16380b55c0 2024-04-26T21:18:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163ce37640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1638143c20 con 0x7f1638143840 2024-04-26T21:18:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1638143840 0x7f16380b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1638143840 0x7f16380b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53970/0 (socket says 172.21.15.40:53970) 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f162ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f16380b4520 0x7f16380b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53044/0 (socket says 172.21.15.40:53044) 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 -- 172.21.15.40:0/3926255870 learned_addr learned my addr 172.21.15.40:0/3926255870 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 -- 172.21.15.40:0/3926255870 >> v1:172.21.15.184:6789/0 conn(0x7f16380b55c0 legacy=0x7f16380ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 -- 172.21.15.40:0/3926255870 >> v1:172.21.15.40:6789/0 conn(0x7f16380b4520 legacy=0x7f16380b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f163659c640 1 -- 172.21.15.40:0/3926255870 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163815e260 con 0x7f1638143840 2024-04-26T21:18:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.392+0000 7f1635d9b640 1 -- 172.21.15.40:0/3926255870 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144663433 0 0) 0x7f163815dca0 con 0x7f16380b4520 2024-04-26T21:18:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.393+0000 7f163659c640 1 --2- 172.21.15.40:0/3926255870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1638143840 0x7f16380b3fe0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f16280098a0 tx=0x7f162802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a655967de0c27095 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.393+0000 7f1635d9b640 1 -- 172.21.15.40:0/3926255870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628037070 con 0x7f1638143840 2024-04-26T21:18:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.393+0000 7f1635d9b640 1 -- 172.21.15.40:0/3926255870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628032440 con 0x7f1638143840 2024-04-26T21:18:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.393+0000 7f1635d9b640 1 -- 172.21.15.40:0/3926255870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628032810 con 0x7f1638143840 2024-04-26T21:18:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 -- 172.21.15.40:0/3926255870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1638143840 msgr2=0x7f16380b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 --2- 172.21.15.40:0/3926255870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1638143840 0x7f16380b3fe0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f16280098a0 tx=0x7f162802a2e0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 -- 172.21.15.40:0/3926255870 shutdown_connections 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 --2- 172.21.15.40:0/3926255870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1638143840 0x7f16380b3fe0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 -- 172.21.15.40:0/3926255870 >> 172.21.15.40:0/3926255870 conn(0x7f16380cd8a0 msgr2=0x7f16380cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 -- 172.21.15.40:0/3926255870 shutdown_connections 2024-04-26T21:18:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.394+0000 7f163ce37640 1 -- 172.21.15.40:0/3926255870 wait complete. 2024-04-26T21:18:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.395+0000 7f163ce37640 1 Processor -- start 2024-04-26T21:18:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.395+0000 7f163ce37640 1 -- start start 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 0x7f163813cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 0x7f163815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f163815da40 con 0x7f16380b4520 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f163815d8c0 con 0x7f16380b55c0 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163ce37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f16380cf700 con 0x7f1638143840 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53976/0 (socket says 172.21.15.40:53976) 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 -- 172.21.15.40:0/1363461640 learned_addr learned my addr 172.21.15.40:0/1363461640 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f162ffff640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 0x7f163813cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f1636d9d640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 0x7f163815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 -- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 msgr2=0x7f163815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 0x7f163815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 -- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 msgr2=0x7f163813cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 0x7f163813cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 -- 172.21.15.40:0/1363461640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16381f7b70 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f162ffff640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 0x7f163813cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f1636d9d640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 0x7f163815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.396+0000 7f163659c640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f16280090b0 tx=0x7f16280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.397+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628037040 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.397+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628004250 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.397+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1638145890 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.397+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628003610 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.397+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1638145e50 con 0x7f16380b4520 2024-04-26T21:18:31.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.398+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16380ceff0 con 0x7f16380b4520 2024-04-26T21:18:31.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.398+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f162803a070 con 0x7f16380b4520 2024-04-26T21:18:31.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.398+0000 7f162effd640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 0x7f161806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.399+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5738+0+0 (secure 0 0 0) 0x7f16280ac120 con 0x7f16380b4520 2024-04-26T21:18:31.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.399+0000 7f162ffff640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 0x7f161806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.400+0000 7f162ffff640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 0x7f161806edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f163815d1f0 tx=0x7f1620009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.403+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16380ceff0 con 0x7f16380b4520 2024-04-26T21:18:31.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.624+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f163806a290 con 0x7f16380b4520 2024-04-26T21:18:31.625 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:18:31.625+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:18:31.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.626+0000 7f162effd640 1 -- 172.21.15.40:0/1363461640 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 191+0+0 (secure 0 0 0) 0x7f1628041020 con 0x7f16380b4520 2024-04-26T21:18:31.627 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:18:31.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 msgr2=0x7f161806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 0x7f161806edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f163815d1f0 tx=0x7f1620009040 comp rx=0 tx=0).stop 2024-04-26T21:18:31.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 msgr2=0x7f16381f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f16280090b0 tx=0x7f16280046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 shutdown_connections 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f161806c8f0 0x7f161806edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1638143840 0x7f163815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16380b55c0 0x7f163813cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 --2- 172.21.15.40:0/1363461640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16380b4520 0x7f16381f7630 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 >> 172.21.15.40:0/1363461640 conn(0x7f16380cd8a0 msgr2=0x7f16380b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.628+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 shutdown_connections 2024-04-26T21:18:31.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.629+0000 7f163ce37640 1 -- 172.21.15.40:0/1363461640 wait complete. 2024-04-26T21:18:31.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:18:31.669 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:18:31.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.742+0000 7eff67a50640 1 Processor -- start 2024-04-26T21:18:31.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 -- start start 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff60162ca0 0x7eff60070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600d1c20 0x7eff60070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff60071160 0x7eff600cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff600715d0 con 0x7eff60162ca0 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7eff600cdc40 con 0x7eff60071160 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff67a50640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff600d2470 con 0x7eff600d1c20 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff657ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff60162ca0 0x7eff60070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.743+0000 7eff657ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff60162ca0 0x7eff60070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53980/0 (socket says 172.21.15.40:53980) 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 -- 172.21.15.40:0/4231533529 learned_addr learned my addr 172.21.15.40:0/4231533529 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:31.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff64feb640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600d1c20 0x7eff60070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff65fed640 1 --2- 172.21.15.40:0/4231533529 >> v2:172.21.15.184:3300/0 conn(0x7eff60071160 0x7eff600cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 -- 172.21.15.40:0/4231533529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600d1c20 msgr2=0x7eff60070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600d1c20 0x7eff60070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 -- 172.21.15.40:0/4231533529 >> v2:172.21.15.184:3300/0 conn(0x7eff60071160 msgr2=0x7eff600cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 --2- 172.21.15.40:0/4231533529 >> v2:172.21.15.184:3300/0 conn(0x7eff60071160 0x7eff600cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 -- 172.21.15.40:0/4231533529 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff600d2750 con 0x7eff60162ca0 2024-04-26T21:18:31.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.744+0000 7eff657ec640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60162ca0 0x7eff60070640 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7eff60073160 tx=0x7eff5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd01f85d24e76dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff5ffff640 1 -- 172.21.15.40:0/4231533529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58037070 con 0x7eff60162ca0 2024-04-26T21:18:31.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff5ffff640 1 -- 172.21.15.40:0/4231533529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff58032440 con 0x7eff60162ca0 2024-04-26T21:18:31.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff5ffff640 1 -- 172.21.15.40:0/4231533529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58032810 con 0x7eff60162ca0 2024-04-26T21:18:31.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 -- 172.21.15.40:0/4231533529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60162ca0 msgr2=0x7eff60070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60162ca0 0x7eff60070640 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7eff60073160 tx=0x7eff5802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 -- 172.21.15.40:0/4231533529 shutdown_connections 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 --2- 172.21.15.40:0/4231533529 >> v2:172.21.15.184:3300/0 conn(0x7eff60071160 0x7eff600cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff600d1c20 0x7eff60070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 --2- 172.21.15.40:0/4231533529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff60162ca0 0x7eff60070640 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 -- 172.21.15.40:0/4231533529 >> 172.21.15.40:0/4231533529 conn(0x7eff6009ba40 msgr2=0x7eff6009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 -- 172.21.15.40:0/4231533529 shutdown_connections 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.745+0000 7eff67a50640 1 -- 172.21.15.40:0/4231533529 wait complete. 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 Processor -- start 2024-04-26T21:18:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 -- start start 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff60071160 0x7eff600cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60162ca0 0x7eff6015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff600d2080 con 0x7eff600d1c20 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff600cd8d0 con 0x7eff60162ca0 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff67a50640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7eff600d2200 con 0x7eff60071160 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff64feb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff64feb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53986/0 (socket says 172.21.15.40:53986) 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.747+0000 7eff64feb640 1 -- 172.21.15.40:0/1008127241 learned_addr learned my addr 172.21.15.40:0/1008127241 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff657ec640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff60071160 0x7eff600cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff65fed640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60162ca0 0x7eff6015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 -- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff60071160 msgr2=0x7eff600cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff60071160 0x7eff600cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 -- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60162ca0 msgr2=0x7eff6015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60162ca0 0x7eff6015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 -- 172.21.15.40:0/1008127241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff600cc250 con 0x7eff600d1c20 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff64feb640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7eff5400e9e0 tx=0x7eff5400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54017070 con 0x7eff600d1c20 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5400c9d0 con 0x7eff600d1c20 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff600cd150 con 0x7eff600d1c20 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400cdc0 con 0x7eff600d1c20 2024-04-26T21:18:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.748+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff600caf70 con 0x7eff600d1c20 2024-04-26T21:18:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.749+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff600693d0 con 0x7eff600d1c20 2024-04-26T21:18:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.750+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff5401a070 con 0x7eff600d1c20 2024-04-26T21:18:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.750+0000 7eff5dffb640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 0x7eff4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.750+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7eff5408bba0 con 0x7eff600d1c20 2024-04-26T21:18:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.751+0000 7eff657ec640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 0x7eff4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.751+0000 7eff657ec640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 0x7eff4c06edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7eff58034000 tx=0x7eff5802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.755+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff54059880 con 0x7eff600d1c20 2024-04-26T21:18:31.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:31.977+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff600b26e0 con 0x7eff600d1c20 2024-04-26T21:18:32.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:18:32.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:18:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.628+0000 7eff5dffb640 1 -- 172.21.15.40:0/1008127241 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7eff5405db80 con 0x7eff600d1c20 2024-04-26T21:18:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 msgr2=0x7eff4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 0x7eff4c06edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7eff58034000 tx=0x7eff5802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 msgr2=0x7eff600ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7eff5400e9e0 tx=0x7eff5400eea0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 shutdown_connections 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7eff4c06c8f0 0x7eff4c06edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff60162ca0 0x7eff6015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff600d1c20 0x7eff600ccc10 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 --2- 172.21.15.40:0/1008127241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff60071160 0x7eff600cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 >> 172.21.15.40:0/1008127241 conn(0x7eff6009ba40 msgr2=0x7eff6009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 shutdown_connections 2024-04-26T21:18:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.630+0000 7eff67a50640 1 -- 172.21.15.40:0/1008127241 wait complete. 2024-04-26T21:18:32.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:18:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.750+0000 7f70bfb78640 1 Processor -- start 2024-04-26T21:18:32.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.750+0000 7f70bfb78640 1 -- start start 2024-04-26T21:18:32.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bfb78640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70b80cee20 0x7f70b806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bfb78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bfb78640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b80d1240 con 0x7f70b80d3a30 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bfb78640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f70b80d1e10 con 0x7f70b80cee20 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bfb78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70b80d14e0 con 0x7f70b809c150 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70b80cee20 0x7f70b806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70b80cee20 0x7f70b806d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53998/0 (socket says 172.21.15.40:53998) 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 -- 172.21.15.40:0/1683327921 learned_addr learned my addr 172.21.15.40:0/1683327921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70be115640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 -- 172.21.15.40:0/1683327921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 msgr2=0x7f70b815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 -- 172.21.15.40:0/1683327921 >> v1:172.21.15.40:6789/0 conn(0x7f70b80d3a30 legacy=0x7f70b809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.751+0000 7f70bd914640 1 -- 172.21.15.40:0/1683327921 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b80d17c0 con 0x7f70b80cee20 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.752+0000 7f70bd914640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80cee20 0x7f70b806d5c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009870 tx=0x7f70b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67b74f66c8dc1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:32.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.752+0000 7f70bc912640 1 -- 172.21.15.40:0/1683327921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0037070 con 0x7f70b80cee20 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.752+0000 7f70bc912640 1 -- 172.21.15.40:0/1683327921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b0032440 con 0x7f70b80cee20 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.752+0000 7f70bc912640 1 -- 172.21.15.40:0/1683327921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0032810 con 0x7f70b80cee20 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 -- 172.21.15.40:0/1683327921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80cee20 msgr2=0x7f70b806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80cee20 0x7f70b806d5c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009870 tx=0x7f70b002a270 comp rx=0 tx=0).stop 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 -- 172.21.15.40:0/1683327921 shutdown_connections 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 --2- 172.21.15.40:0/1683327921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80cee20 0x7f70b806d5c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 -- 172.21.15.40:0/1683327921 >> 172.21.15.40:0/1683327921 conn(0x7f70b80d2f10 msgr2=0x7f70b80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 -- 172.21.15.40:0/1683327921 shutdown_connections 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.753+0000 7f70bfb78640 1 -- 172.21.15.40:0/1683327921 wait complete. 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.754+0000 7f70bfb78640 1 Processor -- start 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.754+0000 7f70bfb78640 1 -- start start 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b81deab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 0x7f70b80c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70b80cf200 con 0x7f70b80d3a30 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70b809c7a0 con 0x7f70b80cee20 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bfb78640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f70b80cf380 con 0x7f70b809c150 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54004/0 (socket says 172.21.15.40:54004) 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 -- 172.21.15.40:0/3855682123 learned_addr learned my addr 172.21.15.40:0/3855682123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bd113640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 0x7f70b80c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bd914640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b81deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 -- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 msgr2=0x7f70b81deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b81deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 -- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 msgr2=0x7f70b80c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 0x7f70b80c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70be115640 1 -- 172.21.15.40:0/3855682123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b81df3e0 con 0x7f70b80d3a30 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bd113640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 0x7f70b80c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.755+0000 7f70bd914640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b81deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70be115640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f70b4007e30 tx=0x7f70b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4014620 con 0x7f70b80d3a30 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4019070 con 0x7f70b80d3a30 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b80c7590 con 0x7f70b80d3a30 2024-04-26T21:18:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4014a00 con 0x7f70b80d3a30 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.756+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b80c7b50 con 0x7f70b80d3a30 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.757+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b806a290 con 0x7f70b80d3a30 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.757+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70b4014ba0 con 0x7f70b80d3a30 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.758+0000 7f70ae7fc640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 0x7f709c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.758+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f70b408c480 con 0x7f70b80d3a30 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.758+0000 7f70bd914640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 0x7f709c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.758+0000 7f70bd914640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 0x7f709c06ede0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f70b0034000 tx=0x7f70b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.762+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70b81df3e0 con 0x7f70b80d3a30 2024-04-26T21:18:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:32.946+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f70b80b4820 con 0x7f70b80d3a30 2024-04-26T21:18:33.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.635+0000 7f70ae7fc640 1 -- 172.21.15.40:0/3855682123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7f70b405a840 con 0x7f70b80d3a30 2024-04-26T21:18:33.636 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:18:33.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 msgr2=0x7f709c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:33.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 0x7f709c06ede0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f70b0034000 tx=0x7f70b002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:33.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 msgr2=0x7f70b81e2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:33.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f70b4007e30 tx=0x7f70b400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 shutdown_connections 2024-04-26T21:18:33.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f709c06c920 0x7f709c06ede0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70b80d3a30 0x7f70b81e2650 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70b80cee20 0x7f70b80c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 --2- 172.21.15.40:0/3855682123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70b809c150 0x7f70b81deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 >> 172.21.15.40:0/3855682123 conn(0x7f70b80d2f10 msgr2=0x7f70b80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 shutdown_connections 2024-04-26T21:18:33.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.639+0000 7f70bfb78640 1 -- 172.21.15.40:0/3855682123 wait complete. 2024-04-26T21:18:33.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:18:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.762+0000 7fd1e03a4640 1 Processor -- start 2024-04-26T21:18:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.762+0000 7fd1e03a4640 1 -- start start 2024-04-26T21:18:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1e03a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd1d80cee20 0x7fd1d806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1e03a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1d809c150 0x7fd1d815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1e03a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d80d1240 con 0x7fd1d80d3a30 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1e03a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80d1e10 con 0x7fd1d80cee20 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1e03a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80d14e0 con 0x7fd1d809c150 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd1d80cee20 0x7fd1d806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd1d80cee20 0x7fd1d806d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54010/0 (socket says 172.21.15.40:54010) 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 -- 172.21.15.40:0/666457232 learned_addr learned my addr 172.21.15.40:0/666457232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 -- 172.21.15.40:0/666457232 >> v2:172.21.15.184:3300/0 conn(0x7fd1d809c150 msgr2=0x7fd1d815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 --2- 172.21.15.40:0/666457232 >> v2:172.21.15.184:3300/0 conn(0x7fd1d809c150 0x7fd1d815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 -- 172.21.15.40:0/666457232 >> v1:172.21.15.184:6789/0 conn(0x7fd1d80d3a30 legacy=0x7fd1d809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:33.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.763+0000 7fd1de140640 1 -- 172.21.15.40:0/666457232 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d80d17c0 con 0x7fd1d80cee20 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.764+0000 7fd1de140640 1 --2- 172.21.15.40:0/666457232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d80cee20 0x7fd1d806d5c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8073e90 tx=0x7fd1d002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e2834d14acbc71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.764+0000 7fd1dd13e640 1 -- 172.21.15.40:0/666457232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0037070 con 0x7fd1d80cee20 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.764+0000 7fd1dd13e640 1 -- 172.21.15.40:0/666457232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0032440 con 0x7fd1d80cee20 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.764+0000 7fd1dd13e640 1 -- 172.21.15.40:0/666457232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0032810 con 0x7fd1d80cee20 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 -- 172.21.15.40:0/666457232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d80cee20 msgr2=0x7fd1d806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/666457232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d80cee20 0x7fd1d806d5c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8073e90 tx=0x7fd1d002a250 comp rx=0 tx=0).stop 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 -- 172.21.15.40:0/666457232 shutdown_connections 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/666457232 >> v2:172.21.15.184:3300/0 conn(0x7fd1d809c150 0x7fd1d815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/666457232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d80cee20 0x7fd1d806d5c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 -- 172.21.15.40:0/666457232 >> 172.21.15.40:0/666457232 conn(0x7fd1d80d2f10 msgr2=0x7fd1d80d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 -- 172.21.15.40:0/666457232 shutdown_connections 2024-04-26T21:18:33.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.765+0000 7fd1e03a4640 1 -- 172.21.15.40:0/666457232 wait complete. 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.766+0000 7fd1e03a4640 1 Processor -- start 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.766+0000 7fd1e03a4640 1 -- start start 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1d80cee20 0x7fd1d80a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d80d3a30 0x7fd1d80cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d80cf200 con 0x7fd1d809c150 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d809c7a0 con 0x7fd1d80d3a30 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1e03a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d80cf380 con 0x7fd1d80cee20 2024-04-26T21:18:33.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54022/0 (socket says 172.21.15.40:54022) 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 -- 172.21.15.40:0/1464938095 learned_addr learned my addr 172.21.15.40:0/1464938095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1dd93f640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1d80cee20 0x7fd1d80a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de941640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d80d3a30 0x7fd1d80cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 -- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1d80cee20 msgr2=0x7fd1d80a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1d80cee20 0x7fd1d80a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 -- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d80d3a30 msgr2=0x7fd1d80cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d80d3a30 0x7fd1d80cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.767+0000 7fd1de140640 1 -- 172.21.15.40:0/1464938095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d80cc800 con 0x7fd1d809c150 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1de140640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00097c0 tx=0x7fd1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:33.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0041070 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d003c950 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d80ccae0 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0037040 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.768+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d80cd0a0 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.769+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d809e990 con 0x7fd1d809c150 2024-04-26T21:18:33.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.769+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1d0044070 con 0x7fd1d809c150 2024-04-26T21:18:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.770+0000 7fd1d6ffd640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 0x7fd1c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.770+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5386+0+0 (secure 0 0 0) 0x7fd1d00ac6f0 con 0x7fd1d809c150 2024-04-26T21:18:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.770+0000 7fd1dd93f640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 0x7fd1c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:33.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.770+0000 7fd1dd93f640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 0x7fd1c406ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:33.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.774+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1d007ae10 con 0x7fd1d809c150 2024-04-26T21:18:33.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:33.961+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1d806a290 con 0x7fd1d809c150 2024-04-26T21:18:34.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.637+0000 7fd1d6ffd640 1 -- 172.21.15.40:0/1464938095 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7fd1d007a710 con 0x7fd1d809c150 2024-04-26T21:18:34.638 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:18:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 msgr2=0x7fd1c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 0x7fd1c406ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc008040 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 msgr2=0x7fd1d80cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00097c0 tx=0x7fd1d00046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 shutdown_connections 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd1c406c920 0x7fd1c406ede0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d80d3a30 0x7fd1d80cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1d80cee20 0x7fd1d80a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 --2- 172.21.15.40:0/1464938095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d809c150 0x7fd1d80cdba0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 >> 172.21.15.40:0/1464938095 conn(0x7fd1d80d2f10 msgr2=0x7fd1d80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 shutdown_connections 2024-04-26T21:18:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.639+0000 7fd1e03a4640 1 -- 172.21.15.40:0/1464938095 wait complete. 2024-04-26T21:18:34.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-26T21:18:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.762+0000 7f1e91589640 1 Processor -- start 2024-04-26T21:18:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.762+0000 7f1e91589640 1 -- start start 2024-04-26T21:18:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e91589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1e8c06f340 0x7f1e8c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e91589640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e8c0d0300 0x7f1e8c06f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e91589640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8c06fc60 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e91589640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8c0b46a0 con 0x7f1e8c0d0300 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e91589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8c0b5db0 con 0x7f1e8c06f340 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e8c0d0300 0x7f1e8c06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e8c0d0300 0x7f1e8c06f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54024/0 (socket says 172.21.15.40:54024) 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/561952448 learned_addr learned my addr 172.21.15.40:0/561952448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8affd640 1 --2- 172.21.15.40:0/561952448 >> v2:172.21.15.184:3300/0 conn(0x7f1e8c06f340 0x7f1e8c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/561952448 >> v2:172.21.15.184:3300/0 conn(0x7f1e8c06f340 msgr2=0x7f1e8c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 --2- 172.21.15.40:0/561952448 >> v2:172.21.15.184:3300/0 conn(0x7f1e8c06f340 0x7f1e8c0b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/561952448 >> v1:172.21.15.184:6789/0 conn(0x7f1e8c0b3e50 legacy=0x7f1e8c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.763+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/561952448 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c0b6090 con 0x7f1e8c0d0300 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e8a7fc640 1 --2- 172.21.15.40:0/561952448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0d0300 0x7f1e8c06f720 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c076180 tx=0x7f1e8002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf9ef3e3baef193 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e89ffb640 1 -- 172.21.15.40:0/561952448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80037070 con 0x7f1e8c0d0300 2024-04-26T21:18:34.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e89ffb640 1 -- 172.21.15.40:0/561952448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80032440 con 0x7f1e8c0d0300 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e89ffb640 1 -- 172.21.15.40:0/561952448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80032810 con 0x7f1e8c0d0300 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 -- 172.21.15.40:0/561952448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0d0300 msgr2=0x7f1e8c06f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 --2- 172.21.15.40:0/561952448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0d0300 0x7f1e8c06f720 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c076180 tx=0x7f1e8002a150 comp rx=0 tx=0).stop 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 -- 172.21.15.40:0/561952448 shutdown_connections 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 --2- 172.21.15.40:0/561952448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0d0300 0x7f1e8c06f720 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 --2- 172.21.15.40:0/561952448 >> v2:172.21.15.184:3300/0 conn(0x7f1e8c06f340 0x7f1e8c0b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.764+0000 7f1e91589640 1 -- 172.21.15.40:0/561952448 >> 172.21.15.40:0/561952448 conn(0x7f1e8c0cd8a0 msgr2=0x7f1e8c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.765+0000 7f1e91589640 1 -- 172.21.15.40:0/561952448 shutdown_connections 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.765+0000 7f1e91589640 1 -- 172.21.15.40:0/561952448 wait complete. 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.766+0000 7f1e91589640 1 Processor -- start 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.766+0000 7f1e91589640 1 -- start start 2024-04-26T21:18:34.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.766+0000 7f1e91589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 0x7f1e8c13a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e8c0d0300 0x7f1e8c1e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b42e0 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b6650 con 0x7f1e8c06f340 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b4460 con 0x7f1e8c0d0300 2024-04-26T21:18:34.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54028/0 (socket says 172.21.15.40:54028) 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/2090090945 learned_addr learned my addr 172.21.15.40:0/2090090945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8affd640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 0x7f1e8c13a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8b7fe640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e8c0d0300 0x7f1e8c1e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e8c0d0300 msgr2=0x7f1e8c1e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e8c0d0300 0x7f1e8c1e8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 msgr2=0x7f1e8c13a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 0x7f1e8c13a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 -- 172.21.15.40:0/2090090945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c13b600 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8affd640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 0x7f1e8c13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:34.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e8a7fc640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000c00 tx=0x7f1e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80041070 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8003c870 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c13b8e0 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80037040 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.767+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c13bea0 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.769+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c06a290 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.769+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e80044070 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.769+0000 7f1e737fe640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 0x7f1e7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.769+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f1e800ac400 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.770+0000 7f1e8affd640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 0x7f1e7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.770+0000 7f1e8affd640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 0x7f1e7806edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a20 tx=0x7f1e7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:34.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.774+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e8007ac50 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.958+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f1e8c145930 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.961+0000 7f1e737fe640 1 -- 172.21.15.40:0/2090090945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v29) ==== 111+0+0 (secure 0 0 0) 0x7f1e8007a550 con 0x7f1e8c0b3e50 2024-04-26T21:18:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 msgr2=0x7f1e7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 0x7f1e7806edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a20 tx=0x7f1e7c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 msgr2=0x7f1e8c144560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1e80000c00 tx=0x7f1e800046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 shutdown_connections 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e7806c8f0 0x7f1e7806edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1e8c0d0300 0x7f1e8c1e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e8c0b3e50 0x7f1e8c144560 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 --2- 172.21.15.40:0/2090090945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e8c06f340 0x7f1e8c13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 >> 172.21.15.40:0/2090090945 conn(0x7f1e8c0cd8a0 msgr2=0x7f1e8c15d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 shutdown_connections 2024-04-26T21:18:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:34.963+0000 7f1e91589640 1 -- 172.21.15.40:0/2090090945 wait complete. 2024-04-26T21:18:34.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:18:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.081+0000 7fb58876a640 1 Processor -- start 2024-04-26T21:18:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.082+0000 7fb58876a640 1 -- start start 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.082+0000 7fb58876a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 0x7fb58015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.082+0000 7fb58876a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015f4f0 0x7fb580073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.082+0000 7fb58876a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58015c530 con 0x7fb580074030 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.082+0000 7fb58876a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb58015c810 con 0x7fb58015f4f0 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb58876a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb58015caf0 con 0x7fb58015b930 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015f4f0 0x7fb580073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015f4f0 0x7fb580073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 -- 172.21.15.40:0/2370297721 learned_addr learned my addr 172.21.15.40:0/2370297721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 -- 172.21.15.40:0/2370297721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 msgr2=0x7fb58015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb586506640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 0x7fb58015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 0x7fb58015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 -- 172.21.15.40:0/2370297721 >> v1:172.21.15.184:6789/0 conn(0x7fb580074030 legacy=0x7fb5800720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb585d05640 1 -- 172.21.15.40:0/2370297721 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58015cdd0 con 0x7fb58015f4f0 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.083+0000 7fb586506640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 0x7fb58015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb585d05640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015f4f0 0x7fb580073af0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb570009870 tx=0x7fb57002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1463b31ce97b3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb585504640 1 -- 172.21.15.40:0/2370297721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570037070 con 0x7fb58015f4f0 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb585504640 1 -- 172.21.15.40:0/2370297721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570032440 con 0x7fb58015f4f0 2024-04-26T21:18:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb585504640 1 -- 172.21.15.40:0/2370297721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570032810 con 0x7fb58015f4f0 2024-04-26T21:18:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb58876a640 1 -- 172.21.15.40:0/2370297721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015f4f0 msgr2=0x7fb580073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.084+0000 7fb58876a640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015f4f0 0x7fb580073af0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb570009870 tx=0x7fb57002a210 comp rx=0 tx=0).stop 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 -- 172.21.15.40:0/2370297721 shutdown_connections 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015f4f0 0x7fb580073af0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 --2- 172.21.15.40:0/2370297721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015b930 0x7fb58015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 -- 172.21.15.40:0/2370297721 >> 172.21.15.40:0/2370297721 conn(0x7fb5800cd8a0 msgr2=0x7fb5800cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 -- 172.21.15.40:0/2370297721 shutdown_connections 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.085+0000 7fb58876a640 1 -- 172.21.15.40:0/2370297721 wait complete. 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.086+0000 7fb58876a640 1 Processor -- start 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.086+0000 7fb58876a640 1 -- start start 2024-04-26T21:18:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.086+0000 7fb58876a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb580074030 0x7fb58014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 0x7fb5801f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb58006fdd0 con 0x7fb58015b930 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb58015c250 con 0x7fb580074030 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800d0270 con 0x7fb58015f4f0 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54042/0 (socket says 172.21.15.40:54042) 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 -- 172.21.15.40:0/1852632924 learned_addr learned my addr 172.21.15.40:0/1852632924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb586506640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb580074030 0x7fb58014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb586d07640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 0x7fb5801f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 -- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 msgr2=0x7fb5801f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 0x7fb5801f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 -- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb580074030 msgr2=0x7fb58014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb580074030 0x7fb58014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 -- 172.21.15.40:0/1852632924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58014a470 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb586d07640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 0x7fb5801f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb585d05640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb570009870 tx=0x7fb5700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570041070 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57003c870 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb580149bd0 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570037040 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.087+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58014a190 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.089+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58006fa30 con 0x7fb58015b930 2024-04-26T21:18:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.089+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb570044070 con 0x7fb58015b930 2024-04-26T21:18:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.089+0000 7fb576ffd640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.089+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7fb5700ac3b0 con 0x7fb58015b930 2024-04-26T21:18:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.090+0000 7fb586506640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.090+0000 7fb586506640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a990 tx=0x7fb57c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.094+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb58006fa30 con 0x7fb58015b930 2024-04-26T21:18:35.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.277+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb58006a290 con 0x7fb58015b930 2024-04-26T21:18:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.280+0000 7fb576ffd640 1 -- 172.21.15.40:0/1852632924 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fb57007aae0 con 0x7fb58015b930 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.282+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 msgr2=0x7fb56406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.282+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00a990 tx=0x7fb57c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.282+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 msgr2=0x7fb5800afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.282+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb570009870 tx=0x7fb5700046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 shutdown_connections 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb58015f4f0 0x7fb5801f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015b930 0x7fb5800afd70 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 --2- 172.21.15.40:0/1852632924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb580074030 0x7fb58014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 >> 172.21.15.40:0/1852632924 conn(0x7fb5800cd8a0 msgr2=0x7fb580150a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 shutdown_connections 2024-04-26T21:18:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.283+0000 7fb58876a640 1 -- 172.21.15.40:0/1852632924 wait complete. 2024-04-26T21:18:35.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-26T21:18:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.396+0000 7f685d693640 1 Processor -- start 2024-04-26T21:18:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.396+0000 7f685d693640 1 -- start start 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f685d693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f68580d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f685d693640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685809c790 con 0x7f68580b2000 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f685d693640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6858163ec0 con 0x7f68581631f0 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f685d693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f685806eec0 con 0x7f685806ea30 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f68567fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f68580b2000 0x7f68580b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53064/0 (socket says 172.21.15.40:53064) 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f68567fc640 1 -- 172.21.15.40:0/83615687 learned_addr learned my addr 172.21.15.40:0/83615687 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f68577fe640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f68580d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168606172 0 0) 0x7f685809c790 con 0x7f68580b2000 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6840003520 con 0x7f68580b2000 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.397+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611730890 0 0) 0x7f6840003520 con 0x7f68580b2000 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 msgr2=0x7f68580d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f68580d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 >> v1:172.21.15.184:6789/0 conn(0x7f68581631f0 legacy=0x7f68581635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580d1a40 con 0x7f68580b2000 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f684c001670 con 0x7f68580b2000 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 >> v1:172.21.15.40:6789/0 conn(0x7f68580b2000 legacy=0x7f68580b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68400039e0 0x7f6840003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f6840005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f684000baf0 con 0x7f6840004800 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f684000bca0 con 0x7f68400039e0 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f684000bea0 con 0x7f685806ea30 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f6856ffd640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68400039e0 0x7f6840003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68577fe640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f6840005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 -- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 msgr2=0x7f6840005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f6840005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 -- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68400039e0 msgr2=0x7f6840003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68400039e0 0x7f6840003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68567fc640 1 -- 172.21.15.40:0/83615687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840003520 con 0x7f6840004800 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.398+0000 7f68577fe640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685806ea30 0x7f6840005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f68567fc640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c210 tx=0x7f6840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f61a5cd2a50557 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c007d30 con 0x7f6840004800 2024-04-26T21:18:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684c005af0 con 0x7f6840004800 2024-04-26T21:18:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f6855ffb640 1 -- 172.21.15.40:0/83615687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c007070 con 0x7f6840004800 2024-04-26T21:18:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 -- 172.21.15.40:0/83615687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6840004800 msgr2=0x7f6840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 --2- 172.21.15.40:0/83615687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c210 tx=0x7f6840005830 comp rx=0 tx=0).stop 2024-04-26T21:18:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f68577fe640 1 -- 172.21.15.40:0/83615687 reap_dead start 2024-04-26T21:18:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 -- 172.21.15.40:0/83615687 shutdown_connections 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 -- 172.21.15.40:0/83615687 >> 172.21.15.40:0/83615687 conn(0x7f685809ba40 msgr2=0x7f685809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 -- 172.21.15.40:0/83615687 shutdown_connections 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.399+0000 7f685d693640 1 -- 172.21.15.40:0/83615687 wait complete. 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 Processor -- start 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 -- start start 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6840004800 0x7f68580c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 0x7f6858161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f68580d0230 con 0x7f6858071600 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68580d2000 con 0x7f6840004800 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.401+0000 7f685d693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f68581629d0 con 0x7f685815d740 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54072/0 (socket says 172.21.15.40:54072) 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 -- 172.21.15.40:0/1251042633 learned_addr learned my addr 172.21.15.40:0/1251042633 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68567fc640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 0x7f6858161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f6856ffd640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6840004800 0x7f68580c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 -- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 msgr2=0x7f6858161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 0x7f6858161590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 -- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6840004800 msgr2=0x7f68580c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6840004800 0x7f68580c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 -- 172.21.15.40:0/1251042633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858162e60 con 0x7f6858071600 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68567fc640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 0x7f6858161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f68577fe640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6840004780 tx=0x7f6850009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685001a030 con 0x7f6858071600 2024-04-26T21:18:35.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6850010e20 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6858161dc0 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850003bd0 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.402+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6858162380 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.403+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685806a290 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.404+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f685001d070 con 0x7f6858071600 2024-04-26T21:18:35.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.404+0000 7f683f7fe640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 0x7f684406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.404+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f685008c200 con 0x7f6858071600 2024-04-26T21:18:35.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.404+0000 7f6856ffd640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 0x7f684406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.405+0000 7f6856ffd640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 0x7f684406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68580ccec0 tx=0x7f6848009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.408+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f685005a9c0 con 0x7f6858071600 2024-04-26T21:18:35.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.592+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f68580b8f10 con 0x7f6858071600 2024-04-26T21:18:35.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.596+0000 7f683f7fe640 1 -- 172.21.15.40:0/1251042633 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v31) ==== 102+0+0 (secure 0 0 0) 0x7f685005a2c0 con 0x7f6858071600 2024-04-26T21:18:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 msgr2=0x7f684406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 0x7f684406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68580ccec0 tx=0x7f6848009040 comp rx=0 tx=0).stop 2024-04-26T21:18:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 msgr2=0x7f6858071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6840004780 tx=0x7f6850009f90 comp rx=0 tx=0).stop 2024-04-26T21:18:35.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 shutdown_connections 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f684406c920 0x7f684406ede0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6858071600 0x7f6858071a80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685815d740 0x7f6858161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 --2- 172.21.15.40:0/1251042633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6840004800 0x7f68580c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 >> 172.21.15.40:0/1251042633 conn(0x7f685809ba40 msgr2=0x7f6858070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 shutdown_connections 2024-04-26T21:18:35.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.598+0000 7f685d693640 1 -- 172.21.15.40:0/1251042633 wait complete. 2024-04-26T21:18:35.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-26T21:18:35.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.713+0000 7fa099cd1640 1 Processor -- start 2024-04-26T21:18:35.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.713+0000 7fa099cd1640 1 -- start start 2024-04-26T21:18:35.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa099cd1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa09415f260 0x7fa09406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa099cd1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa099cd1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa09415f640 con 0x7fa094070560 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa099cd1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa09406c2a0 con 0x7fa09415f260 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa099cd1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa09406c580 con 0x7fa0940b4e00 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa09415f260 0x7fa09406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa09415f260 0x7fa09406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54088/0 (socket says 172.21.15.40:54088) 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 -- 172.21.15.40:0/959991135 learned_addr learned my addr 172.21.15.40:0/959991135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa093fff640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 -- 172.21.15.40:0/959991135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 msgr2=0x7fa0940ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 -- 172.21.15.40:0/959991135 >> v1:172.21.15.40:6789/0 conn(0x7fa094070560 legacy=0x7fa0940b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.714+0000 7fa0937fe640 1 -- 172.21.15.40:0/959991135 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09406c860 con 0x7fa09415f260 2024-04-26T21:18:35.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.715+0000 7fa0937fe640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09415f260 0x7fa09406d620 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa084009870 tx=0x7fa08402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601e8e8d6bc90698 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.715+0000 7fa0927fc640 1 -- 172.21.15.40:0/959991135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084037070 con 0x7fa09415f260 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.715+0000 7fa0927fc640 1 -- 172.21.15.40:0/959991135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa084032440 con 0x7fa09415f260 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.715+0000 7fa0927fc640 1 -- 172.21.15.40:0/959991135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084032810 con 0x7fa09415f260 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 -- 172.21.15.40:0/959991135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09415f260 msgr2=0x7fa09406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09415f260 0x7fa09406d620 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa084009870 tx=0x7fa08402a270 comp rx=0 tx=0).stop 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 -- 172.21.15.40:0/959991135 shutdown_connections 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 --2- 172.21.15.40:0/959991135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa09415f260 0x7fa09406d620 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 -- 172.21.15.40:0/959991135 >> 172.21.15.40:0/959991135 conn(0x7fa0940d1660 msgr2=0x7fa0940d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 -- 172.21.15.40:0/959991135 shutdown_connections 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.716+0000 7fa099cd1640 1 -- 172.21.15.40:0/959991135 wait complete. 2024-04-26T21:18:35.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.717+0000 7fa099cd1640 1 Processor -- start 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.717+0000 7fa099cd1640 1 -- start start 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09415f260 0x7fa0940cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa09406d100 con 0x7fa094070560 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa09406ce20 con 0x7fa09415f260 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa099cd1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa09406db60 con 0x7fa0940b4e00 2024-04-26T21:18:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54100/0 (socket says 172.21.15.40:54100) 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 -- 172.21.15.40:0/189030170 learned_addr learned my addr 172.21.15.40:0/189030170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 -- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 msgr2=0x7fa0940ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 -- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09415f260 msgr2=0x7fa0940cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09415f260 0x7fa0940cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 -- 172.21.15.40:0/189030170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0940ce400 con 0x7fa094070560 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.718+0000 7fa0937fe640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa084009870 tx=0x7fa0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.719+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084041020 con 0x7fa094070560 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.719+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0840306c0 con 0x7fa094070560 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.719+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0940d0f20 con 0x7fa094070560 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.719+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084037040 con 0x7fa094070560 2024-04-26T21:18:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.719+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0940d0c40 con 0x7fa094070560 2024-04-26T21:18:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.720+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0940aec90 con 0x7fa094070560 2024-04-26T21:18:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.720+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa084044020 con 0x7fa094070560 2024-04-26T21:18:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.720+0000 7fa07bfff640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 0x7fa08006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.721+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7fa0840abe40 con 0x7fa094070560 2024-04-26T21:18:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.721+0000 7fa092ffd640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 0x7fa08006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.721+0000 7fa092ffd640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 0x7fa08006edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa088000f60 tx=0x7fa0880079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:35.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.725+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0940aec90 con 0x7fa094070560 2024-04-26T21:18:35.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.912+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7fa0940ce600 con 0x7fa094070560 2024-04-26T21:18:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.915+0000 7fa07bfff640 1 -- 172.21.15.40:0/189030170 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v32) ==== 105+0+0 (secure 0 0 0) 0x7fa08404b070 con 0x7fa094070560 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 msgr2=0x7fa08006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 0x7fa08006edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa088000f60 tx=0x7fa0880079d0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 msgr2=0x7fa0940ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa084009870 tx=0x7fa0840046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 shutdown_connections 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa08006c8f0 0x7fa08006edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa09415f260 0x7fa0940cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0940b4e00 0x7fa0940ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 --2- 172.21.15.40:0/189030170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa094070560 0x7fa0940ce820 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 >> 172.21.15.40:0/189030170 conn(0x7fa0940d1660 msgr2=0x7fa0940aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 shutdown_connections 2024-04-26T21:18:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:35.917+0000 7fa099cd1640 1 -- 172.21.15.40:0/189030170 wait complete. 2024-04-26T21:18:35.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-26T21:18:36.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.031+0000 7f1083d96640 1 Processor -- start 2024-04-26T21:18:36.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.031+0000 7f1083d96640 1 -- start start 2024-04-26T21:18:36.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1083d96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1083d96640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 0x7f107c0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1083d96640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107c09c700 con 0x7f107c0b3210 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1083d96640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f107c0b4480 con 0x7f107c0b3c20 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1083d96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f107c06e050 con 0x7f107c0d0010 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1081b32640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1081331640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 0x7f107c0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1082333640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f107c0b3210 0x7f107c15fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57430/0 (socket says 172.21.15.40:57430) 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.032+0000 7f1082333640 1 -- 172.21.15.40:0/1458476414 learned_addr learned my addr 172.21.15.40:0/1458476414 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081b32640 1 -- 172.21.15.40:0/1458476414 >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 msgr2=0x7f107c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081b32640 1 --2- 172.21.15.40:0/1458476414 >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 0x7f107c0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081b32640 1 -- 172.21.15.40:0/1458476414 >> v1:172.21.15.184:6789/0 conn(0x7f107c0b3210 legacy=0x7f107c15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1080b30640 1 -- 172.21.15.40:0/1458476414 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960345126 0 0) 0x7f107c09c700 con 0x7f107c0b3210 2024-04-26T21:18:36.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081b32640 1 -- 172.21.15.40:0/1458476414 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c0d1c50 con 0x7f107c0d0010 2024-04-26T21:18:36.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081331640 1 --2- 172.21.15.40:0/1458476414 >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 0x7f107c0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.033+0000 7f1081b32640 1 --2- 172.21.15.40:0/1458476414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c06db10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10780097a0 tx=0x7f107802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b2d2f4f3a618be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1080b30640 1 -- 172.21.15.40:0/1458476414 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078037070 con 0x7f107c0d0010 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1080b30640 1 -- 172.21.15.40:0/1458476414 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1078032440 con 0x7f107c0d0010 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1080b30640 1 -- 172.21.15.40:0/1458476414 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078032810 con 0x7f107c0d0010 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 -- 172.21.15.40:0/1458476414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 msgr2=0x7f107c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 --2- 172.21.15.40:0/1458476414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c06db10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10780097a0 tx=0x7f107802a180 comp rx=0 tx=0).stop 2024-04-26T21:18:36.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 -- 172.21.15.40:0/1458476414 shutdown_connections 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 --2- 172.21.15.40:0/1458476414 >> v2:172.21.15.184:3300/0 conn(0x7f107c0b3c20 0x7f107c0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 --2- 172.21.15.40:0/1458476414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c06db10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 -- 172.21.15.40:0/1458476414 >> 172.21.15.40:0/1458476414 conn(0x7f107c09ba40 msgr2=0x7f107c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 -- 172.21.15.40:0/1458476414 shutdown_connections 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.034+0000 7f1083d96640 1 -- 172.21.15.40:0/1458476414 wait complete. 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.035+0000 7f1083d96640 1 Processor -- start 2024-04-26T21:18:36.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.035+0000 7f1083d96640 1 -- start start 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 0x7f107c1eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c09d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f107c0b3820 con 0x7f107c0b3c20 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f107c0b36a0 con 0x7f107c0b3210 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1083d96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f107c0b39a0 con 0x7f107c0d0010 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54102/0 (socket says 172.21.15.40:54102) 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 -- 172.21.15.40:0/3108505579 learned_addr learned my addr 172.21.15.40:0/3108505579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:36.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081b32640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 0x7f107c1eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 -- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 msgr2=0x7f107c09d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1082333640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c09d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c09d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 -- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 msgr2=0x7f107c1eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 0x7f107c1eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.036+0000 7f1081331640 1 -- 172.21.15.40:0/3108505579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c1e9ae0 con 0x7f107c0b3c20 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f1081b32640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 0x7f107c1eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f1082333640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c09d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f1081331640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f106c00e990 tx=0x7f106c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c017070 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c00c9e0 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c00cdb0 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c1e9dc0 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.037+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c1ea380 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.038+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c06a290 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.038+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f106c01a070 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.039+0000 7f10727fc640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 0x7f106006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.039+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f106c08bac0 con 0x7f107c0b3c20 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.039+0000 7f1081b32640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 0x7f106006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.040+0000 7f1081b32640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 0x7f106006edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1078034000 tx=0x7f107802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.043+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f106c05a1f0 con 0x7f107c0b3c20 2024-04-26T21:18:36.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.227+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f107c068e00 con 0x7f107c0b3c20 2024-04-26T21:18:36.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.231+0000 7f10727fc640 1 -- 172.21.15.40:0/3108505579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v33) ==== 107+0+0 (secure 0 0 0) 0x7f106c059af0 con 0x7f107c0b3c20 2024-04-26T21:18:36.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 msgr2=0x7f106006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 0x7f106006edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1078034000 tx=0x7f107802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 msgr2=0x7f107c09ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f106c00e990 tx=0x7f106c00ee50 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 shutdown_connections 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f106006c8f0 0x7f106006edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f107c0d0010 0x7f107c09d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f107c0b3c20 0x7f107c09ce30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 --2- 172.21.15.40:0/3108505579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f107c0b3210 0x7f107c1eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 >> 172.21.15.40:0/3108505579 conn(0x7f107c09ba40 msgr2=0x7f107c0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 shutdown_connections 2024-04-26T21:18:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.233+0000 7f1083d96640 1 -- 172.21.15.40:0/3108505579 wait complete. 2024-04-26T21:18:36.244 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-04-26T21:18:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.345+0000 7f1939e1d640 1 Processor -- start 2024-04-26T21:18:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.346+0000 7f1939e1d640 1 -- start start 2024-04-26T21:18:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f1939e1d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f193415c5f0 0x7f193413f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f1939e1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193413fe00 con 0x7f193415fb70 2024-04-26T21:18:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f1939e1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1934070910 con 0x7f19340731c0 2024-04-26T21:18:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f1939e1d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1934160050 con 0x7f193415c5f0 2024-04-26T21:18:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f193415c5f0 0x7f193413f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f193415c5f0 0x7f193413f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54118/0 (socket says 172.21.15.40:54118) 2024-04-26T21:18:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 -- 172.21.15.40:0/3728468507 learned_addr learned my addr 172.21.15.40:0/3728468507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 -- 172.21.15.40:0/3728468507 >> v1:172.21.15.184:6789/0 conn(0x7f19340731c0 legacy=0x7f1934141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 -- 172.21.15.40:0/3728468507 >> v1:172.21.15.40:6789/0 conn(0x7f193415fb70 legacy=0x7f19340cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.347+0000 7f19337fe640 1 -- 172.21.15.40:0/3728468507 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1934160330 con 0x7f193415c5f0 2024-04-26T21:18:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.348+0000 7f19337fe640 1 --2- 172.21.15.40:0/3728468507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415c5f0 0x7f193413f720 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1924009870 tx=0x7f192402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4311b5f5736e031 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.348+0000 7f19327fc640 1 -- 172.21.15.40:0/3728468507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924037070 con 0x7f193415c5f0 2024-04-26T21:18:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.348+0000 7f19327fc640 1 -- 172.21.15.40:0/3728468507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192402ae50 con 0x7f193415c5f0 2024-04-26T21:18:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.348+0000 7f19327fc640 1 -- 172.21.15.40:0/3728468507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924032860 con 0x7f193415c5f0 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 -- 172.21.15.40:0/3728468507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415c5f0 msgr2=0x7f193413f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 --2- 172.21.15.40:0/3728468507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415c5f0 0x7f193413f720 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1924009870 tx=0x7f192402a300 comp rx=0 tx=0).stop 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 -- 172.21.15.40:0/3728468507 shutdown_connections 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 --2- 172.21.15.40:0/3728468507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415c5f0 0x7f193413f720 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 -- 172.21.15.40:0/3728468507 >> 172.21.15.40:0/3728468507 conn(0x7f19340d1e30 msgr2=0x7f19340d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 -- 172.21.15.40:0/3728468507 shutdown_connections 2024-04-26T21:18:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.349+0000 7f1939e1d640 1 -- 172.21.15.40:0/3728468507 wait complete. 2024-04-26T21:18:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.351+0000 7f1939e1d640 1 Processor -- start 2024-04-26T21:18:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.351+0000 7f1939e1d640 1 -- start start 2024-04-26T21:18:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19340731c0 0x7f1934158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f193415c5f0 0x7f193416a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1934141780 con 0x7f193415fb70 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19341608f0 con 0x7f19340731c0 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1939e1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19340ce550 con 0x7f193415c5f0 2024-04-26T21:18:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1933fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1933fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54132/0 (socket says 172.21.15.40:54132) 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1933fff640 1 -- 172.21.15.40:0/1080932058 learned_addr learned my addr 172.21.15.40:0/1080932058 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f19337fe640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19340731c0 0x7f1934158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.352+0000 7f1932ffd640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f193415c5f0 0x7f193416a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 -- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f193415c5f0 msgr2=0x7f193416a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f193415c5f0 0x7f193416a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 -- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19340731c0 msgr2=0x7f1934158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19340731c0 0x7f1934158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 -- 172.21.15.40:0/1080932058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193413e590 con 0x7f193415fb70 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1933fff640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f192c00c6b0 tx=0x7f192c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c003b70 con 0x7f193415fb70 2024-04-26T21:18:36.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c003d30 con 0x7f193415fb70 2024-04-26T21:18:36.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c005310 con 0x7f193415fb70 2024-04-26T21:18:36.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341407f0 con 0x7f193415fb70 2024-04-26T21:18:36.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.353+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1934157920 con 0x7f193415fb70 2024-04-26T21:18:36.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.355+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19340693d0 con 0x7f193415fb70 2024-04-26T21:18:36.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.355+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f192c0054b0 con 0x7f193415fb70 2024-04-26T21:18:36.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.355+0000 7f191bfff640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.355+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f192c08b610 con 0x7f193415fb70 2024-04-26T21:18:36.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.355+0000 7f19337fe640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.356+0000 7f19337fe640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 0x7f192006edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1924034000 tx=0x7f192402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.360+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f192c0595e0 con 0x7f193415fb70 2024-04-26T21:18:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.620+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f193406cef0 con 0x7f193415fb70 2024-04-26T21:18:36.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.648+0000 7f191bfff640 1 -- 172.21.15.40:0/1080932058 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v333) ==== 160+0+0 (secure 0 0 0) 0x7f192c05d8e0 con 0x7f193415fb70 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.650+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 msgr2=0x7f192006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.650+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 0x7f192006edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1924034000 tx=0x7f192402e000 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.650+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 msgr2=0x7f193413a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.650+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f192c00c6b0 tx=0x7f192c00cb70 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 shutdown_connections 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f193415fb70 0x7f193413a890 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f193415c5f0 0x7f193416a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 --2- 172.21.15.40:0/1080932058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19340731c0 0x7f1934158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 >> 172.21.15.40:0/1080932058 conn(0x7f19340d1e30 msgr2=0x7f193406fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 shutdown_connections 2024-04-26T21:18:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.651+0000 7f1939e1d640 1 -- 172.21.15.40:0/1080932058 wait complete. 2024-04-26T21:18:36.663 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2024-04-26T21:18:36.663 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2024-04-26T21:18:36.663 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:18:36.663 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.762+0000 7f9941fbf640 1 Processor -- start 2024-04-26T21:18:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.762+0000 7f9941fbf640 1 -- start start 2024-04-26T21:18:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f9941fbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f993c15f340 0x7f993c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f9941fbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f993c15f720 0x7f993c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f9941fbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993c0b2510 con 0x7f993c0d4f00 2024-04-26T21:18:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f9941fbf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b27f0 con 0x7f993c15f340 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f9941fbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b2ad0 con 0x7f993c15f720 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f993c15f340 0x7f993c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f993c15f340 0x7f993c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54138/0 (socket says 172.21.15.40:54138) 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 -- 172.21.15.40:0/2050655690 learned_addr learned my addr 172.21.15.40:0/2050655690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993affd640 1 --2- 172.21.15.40:0/2050655690 >> v2:172.21.15.184:3300/0 conn(0x7f993c15f720 0x7f993c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 -- 172.21.15.40:0/2050655690 >> v2:172.21.15.184:3300/0 conn(0x7f993c15f720 msgr2=0x7f993c0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2050655690 >> v2:172.21.15.184:3300/0 conn(0x7f993c15f720 0x7f993c0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 -- 172.21.15.40:0/2050655690 >> v1:172.21.15.184:6789/0 conn(0x7f993c0d4f00 legacy=0x7f993c0d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.763+0000 7f993b7fe640 1 -- 172.21.15.40:0/2050655690 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c0b2db0 con 0x7f993c15f340 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.764+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2050655690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f340 0x7f993c0b1e80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f993c0cf8a0 tx=0x7f992c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636fc5d2f08b9496 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.764+0000 7f993a7fc640 1 -- 172.21.15.40:0/2050655690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c037070 con 0x7f993c15f340 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.764+0000 7f993a7fc640 1 -- 172.21.15.40:0/2050655690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c032440 con 0x7f993c15f340 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.764+0000 7f993a7fc640 1 -- 172.21.15.40:0/2050655690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c032810 con 0x7f993c15f340 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 -- 172.21.15.40:0/2050655690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f340 msgr2=0x7f993c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2050655690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f340 0x7f993c0b1e80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f993c0cf8a0 tx=0x7f992c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 -- 172.21.15.40:0/2050655690 shutdown_connections 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2050655690 >> v2:172.21.15.184:3300/0 conn(0x7f993c15f720 0x7f993c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2050655690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f340 0x7f993c0b1e80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 -- 172.21.15.40:0/2050655690 >> 172.21.15.40:0/2050655690 conn(0x7f993c08ec70 msgr2=0x7f993c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 -- 172.21.15.40:0/2050655690 shutdown_connections 2024-04-26T21:18:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.765+0000 7f9941fbf640 1 -- 172.21.15.40:0/2050655690 wait complete. 2024-04-26T21:18:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.766+0000 7f9941fbf640 1 Processor -- start 2024-04-26T21:18:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.766+0000 7f9941fbf640 1 -- start start 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 0x7f993c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993c15f340 0x7f993c14b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b3940 con 0x7f993c15f720 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b37c0 con 0x7f993c15f340 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f9941fbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f993c0b3ac0 con 0x7f993c0d4f00 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54150/0 (socket says 172.21.15.40:54150) 2024-04-26T21:18:36.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 -- 172.21.15.40:0/2237398009 learned_addr learned my addr 172.21.15.40:0/2237398009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993affd640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993c15f340 0x7f993c14b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 0x7f993c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 -- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 msgr2=0x7f993c14d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 0x7f993c14d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 -- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993c15f340 msgr2=0x7f993c14b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993c15f340 0x7f993c14b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 -- 172.21.15.40:0/2237398009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c14c4c0 con 0x7f993c15f720 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 0x7f993c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.767+0000 7f993bfff640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f993400c700 tx=0x7f993400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.768+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934016020 con 0x7f993c15f720 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.768+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993400aa50 con 0x7f993c15f720 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.768+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993400ce10 con 0x7f993c15f720 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.768+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c14c7a0 con 0x7f993c15f720 2024-04-26T21:18:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.768+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c14cd60 con 0x7f993c15f720 2024-04-26T21:18:36.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.769+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c06a290 con 0x7f993c15f720 2024-04-26T21:18:36.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.769+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9934019020 con 0x7f993c15f720 2024-04-26T21:18:36.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.770+0000 7f9923fff640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 0x7f992806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:36.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.770+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f993408ba80 con 0x7f993c15f720 2024-04-26T21:18:36.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.770+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 0x7f992806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:36.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.771+0000 7f993b7fe640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 0x7f992806edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f992c034000 tx=0x7f992c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:36.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.774+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f993408de20 con 0x7f993c15f720 2024-04-26T21:18:36.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.961+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f993c15edf0 con 0x7f993c15f720 2024-04-26T21:18:36.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.961+0000 7f9923fff640 1 -- 172.21.15.40:0/2237398009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+13023 (secure 0 0 0) 0x7f9934020020 con 0x7f993c15f720 2024-04-26T21:18:36.963 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:36.963 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:34.636527+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6836/1963635253":"2024-04-27T21:18:31.625838+0000","172.21.15.184:6835/1963635253":"2024-04-27T21:18:31.625838+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:36.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.963+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 msgr2=0x7f992806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.963+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 0x7f992806edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f992c034000 tx=0x7f992c02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:36.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.963+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 msgr2=0x7f993c14bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.963+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f993400c700 tx=0x7f993400cbc0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 shutdown_connections 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f992806c8f0 0x7f992806edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f993c15f720 0x7f993c14bae0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f993c15f340 0x7f993c14b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 --2- 172.21.15.40:0/2237398009 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f993c0d4f00 0x7f993c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 >> 172.21.15.40:0/2237398009 conn(0x7f993c08ec70 msgr2=0x7f993c0b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 shutdown_connections 2024-04-26T21:18:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:36.964+0000 7f9941fbf640 1 -- 172.21.15.40:0/2237398009 wait complete. 2024-04-26T21:18:36.976 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:36.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.079+0000 7f3480862640 1 Processor -- start 2024-04-26T21:18:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.079+0000 7f3480862640 1 -- start start 2024-04-26T21:18:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f3480862640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f347c0d36b0 0x7f347c0d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f3480862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d50c0 0x7f347c0d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f3480862640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0d5a60 con 0x7f347c0d1c00 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f3480862640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0d5cd0 con 0x7f347c0d36b0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f3480862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f347c0d3fd0 con 0x7f347c0d50c0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f347a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f347c0d36b0 0x7f347c0d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f347a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f347c0d36b0 0x7f347c0d3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54160/0 (socket says 172.21.15.40:54160) 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.080+0000 7f347a59c640 1 -- 172.21.15.40:0/1636176278 learned_addr learned my addr 172.21.15.40:0/1636176278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f3479d9b640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d50c0 0x7f347c0d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347a59c640 1 -- 172.21.15.40:0/1636176278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d50c0 msgr2=0x7f347c0d5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347a59c640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d50c0 0x7f347c0d5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347a59c640 1 -- 172.21.15.40:0/1636176278 >> v1:172.21.15.184:6789/0 conn(0x7f347c0d1c00 legacy=0x7f347c0d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347a59c640 1 -- 172.21.15.40:0/1636176278 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0b50e0 con 0x7f347c0d36b0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347a59c640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d36b0 0x7f347c0d3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3474009870 tx=0x7f347402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607f8f842b51b362 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.081+0000 7f347959a640 1 -- 172.21.15.40:0/1636176278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474037070 con 0x7f347c0d36b0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f347959a640 1 -- 172.21.15.40:0/1636176278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3474032440 con 0x7f347c0d36b0 2024-04-26T21:18:37.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f347959a640 1 -- 172.21.15.40:0/1636176278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474032810 con 0x7f347c0d36b0 2024-04-26T21:18:37.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 -- 172.21.15.40:0/1636176278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d36b0 msgr2=0x7f347c0d3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d36b0 0x7f347c0d3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3474009870 tx=0x7f347402a210 comp rx=0 tx=0).stop 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 -- 172.21.15.40:0/1636176278 shutdown_connections 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d50c0 0x7f347c0d5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 --2- 172.21.15.40:0/1636176278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d36b0 0x7f347c0d3a90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 -- 172.21.15.40:0/1636176278 >> 172.21.15.40:0/1636176278 conn(0x7f347c08ec30 msgr2=0x7f347c09e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 -- 172.21.15.40:0/1636176278 shutdown_connections 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.082+0000 7f3480862640 1 -- 172.21.15.40:0/1636176278 wait complete. 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.083+0000 7f3480862640 1 Processor -- start 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.083+0000 7f3480862640 1 -- start start 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d1c00 0x7f347c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f347c0d36b0 0x7f347c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b64c0 con 0x7f347c0d50c0 2024-04-26T21:18:37.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b61e0 con 0x7f347c0d36b0 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f3480862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f347c0b67a0 con 0x7f347c0d1c00 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f347ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f347ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54176/0 (socket says 172.21.15.40:54176) 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.084+0000 7f347ad9d640 1 -- 172.21.15.40:0/2517770653 learned_addr learned my addr 172.21.15.40:0/2517770653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347a59c640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d1c00 0x7f347c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f3479d9b640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f347c0d36b0 0x7f347c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 -- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d1c00 msgr2=0x7f347c0cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d1c00 0x7f347c0cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 -- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f347c0d36b0 msgr2=0x7f347c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f347c0d36b0 0x7f347c0ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 -- 172.21.15.40:0/2517770653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0cdf70 con 0x7f347c0d50c0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f347ad9d640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f346c00b970 tx=0x7f346c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c019070 con 0x7f347c0d50c0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c00cb40 con 0x7f347c0d50c0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c0ce250 con 0x7f347c0d50c0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c014440 con 0x7f347c0d50c0 2024-04-26T21:18:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.085+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c0ce810 con 0x7f347c0d50c0 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.086+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c06a290 con 0x7f347c0d50c0 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.087+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f346c01c070 con 0x7f347c0d50c0 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.087+0000 7f346affd640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 0x7f345c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.087+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f346c08b450 con 0x7f347c0d50c0 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.087+0000 7f347a59c640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 0x7f345c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.088+0000 7f347a59c640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 0x7f345c06ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3474034000 tx=0x7f347402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.091+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f346c059b80 con 0x7f347c0d50c0 2024-04-26T21:18:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.273+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f347c0754c0 con 0x7f347c0d50c0 2024-04-26T21:18:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.274+0000 7f346affd640 1 -- 172.21.15.40:0/2517770653 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f346c059480 con 0x7f347c0d50c0 2024-04-26T21:18:37.275 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 msgr2=0x7f345c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 0x7f345c06ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3474034000 tx=0x7f347402e000 comp rx=0 tx=0).stop 2024-04-26T21:18:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 msgr2=0x7f347c156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f346c00b970 tx=0x7f346c00be30 comp rx=0 tx=0).stop 2024-04-26T21:18:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 shutdown_connections 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f345c06c920 0x7f345c06ede0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0d50c0 0x7f347c156570 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f347c0d36b0 0x7f347c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 --2- 172.21.15.40:0/2517770653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f347c0d1c00 0x7f347c0cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 >> 172.21.15.40:0/2517770653 conn(0x7f347c08ec30 msgr2=0x7f347c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 shutdown_connections 2024-04-26T21:18:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.276+0000 7f3480862640 1 -- 172.21.15.40:0/2517770653 wait complete. 2024-04-26T21:18:37.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-04-26T21:18:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.388+0000 7f9da3eb0640 1 Processor -- start 2024-04-26T21:18:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.388+0000 7f9da3eb0640 1 -- start start 2024-04-26T21:18:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da3eb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da3eb0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da3eb0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9c0d0220 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da3eb0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9c0d0500 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da3eb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c0d07e0 con 0x7f9d9c0cfda0 2024-04-26T21:18:37.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da144b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49200/0 (socket says 172.21.15.40:49200) 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da144b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:40344/0 (socket says 172.21.15.40:40344) 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da1c4c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9d9c0d1b80 0x7f9d9c0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57456/0 (socket says 172.21.15.40:57456) 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da144b640 1 -- 172.21.15.40:0/842011393 learned_addr learned my addr 172.21.15.40:0/842011393 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 -- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 msgr2=0x7f9d9c0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 -- 172.21.15.40:0/842011393 >> v1:172.21.15.184:6789/0 conn(0x7f9d9c0d1b80 legacy=0x7f9d9c0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da244d640 1 -- 172.21.15.40:0/842011393 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c0d0ac0 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.389+0000 7f9da144b640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:37.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.390+0000 7f9da244d640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009d90 tx=0x7f9d9402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d12b7bf1b719256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.390+0000 7f9da0c4a640 1 -- 172.21.15.40:0/842011393 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94007740 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.390+0000 7f9da0c4a640 1 -- 172.21.15.40:0/842011393 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d940078e0 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.390+0000 7f9da0c4a640 1 -- 172.21.15.40:0/842011393 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94007cb0 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 -- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 msgr2=0x7f9d9c0b6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009d90 tx=0x7f9d9402a660 comp rx=0 tx=0).stop 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 -- 172.21.15.40:0/842011393 shutdown_connections 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c0b6550 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/842011393 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 -- 172.21.15.40:0/842011393 >> 172.21.15.40:0/842011393 conn(0x7f9d9c09e540 msgr2=0x7f9d9c0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 -- 172.21.15.40:0/842011393 shutdown_connections 2024-04-26T21:18:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.391+0000 7f9da3eb0640 1 -- 172.21.15.40:0/842011393 wait complete. 2024-04-26T21:18:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.392+0000 7f9da3eb0640 1 Processor -- start 2024-04-26T21:18:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.392+0000 7f9da3eb0640 1 -- start start 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0d4170 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0b6a90 con 0x7f9d9c0b60a0 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da3eb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c15ee50 con 0x7f9d9c0cfda0 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54182/0 (socket says 172.21.15.40:54182) 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 -- 172.21.15.40:0/3410193733 learned_addr learned my addr 172.21.15.40:0/3410193733 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da1c4c640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da144b640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 -- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 msgr2=0x7f9d9c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 -- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 msgr2=0x7f9d9c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da244d640 1 -- 172.21.15.40:0/3410193733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c0c9f50 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.393+0000 7f9da1c4c640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9da244d640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009eb0 tx=0x7f9d94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94040070 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94031bc0 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c1f8700 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94009450 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.394+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1f8cc0 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.395+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c06a290 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.395+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d94039040 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.396+0000 7f9d927fc640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 0x7f9d8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.396+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f9d940ad110 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.396+0000 7f9da1c4c640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 0x7f9d8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.396+0000 7f9da1c4c640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 0x7f9d8006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d98007fd0 tx=0x7f9d98007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.400+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d9403ae60 con 0x7f9d9c0d1b80 2024-04-26T21:18:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:37.655+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f9d9c0b90b0 con 0x7f9d9c0d1b80 2024-04-26T21:18:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.654+0000 7f9d927fc640 1 -- 172.21.15.40:0/3410193733 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v335) ==== 153+0+0 (secure 0 0 0) 0x7f9d9407b830 con 0x7f9d9c0d1b80 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 msgr2=0x7f9d8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 0x7f9d8006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d98007fd0 tx=0x7f9d98007d60 comp rx=0 tx=0).stop 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 msgr2=0x7f9d9c151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009eb0 tx=0x7f9d94004a40 comp rx=0 tx=0).stop 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 shutdown_connections 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9d8006c8f0 0x7f9d8006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d9c0d1b80 0x7f9d9c151680 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9c0cfda0 0x7f9d9c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 --2- 172.21.15.40:0/3410193733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9c0b60a0 0x7f9d9c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 >> 172.21.15.40:0/3410193733 conn(0x7f9d9c09e540 msgr2=0x7f9d9c0b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 shutdown_connections 2024-04-26T21:18:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.657+0000 7f9da3eb0640 1 -- 172.21.15.40:0/3410193733 wait complete. 2024-04-26T21:18:38.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:18:38.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.780+0000 7f2e689a4640 1 Processor -- start 2024-04-26T21:18:38.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.780+0000 7f2e689a4640 1 -- start start 2024-04-26T21:18:38.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e689a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e640afe10 0x7f2e640cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e689a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6415d4c0 0x7f2e640b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e689a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e64074430 con 0x7f2e64069ae0 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e689a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e640746f0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e689a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e640ae140 con 0x7f2e640afe10 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e61d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6415d4c0 0x7f2e640b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e61d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e6415d4c0 0x7f2e640b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54196/0 (socket says 172.21.15.40:54196) 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.781+0000 7f2e61d9b640 1 -- 172.21.15.40:0/3419211988 learned_addr learned my addr 172.21.15.40:0/3419211988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e6259c640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e640afe10 0x7f2e640cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e61d9b640 1 -- 172.21.15.40:0/3419211988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e640afe10 msgr2=0x7f2e640cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e61d9b640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e640afe10 0x7f2e640cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e61d9b640 1 -- 172.21.15.40:0/3419211988 >> v1:172.21.15.40:6789/0 conn(0x7f2e64069ae0 legacy=0x7f2e640d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e61d9b640 1 -- 172.21.15.40:0/3419211988 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e640ae420 con 0x7f2e6415d4c0 2024-04-26T21:18:38.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.782+0000 7f2e61d9b640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640b01f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c2a7095dc19d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e6159a640 1 -- 172.21.15.40:0/3419211988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58037070 con 0x7f2e6415d4c0 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e6159a640 1 -- 172.21.15.40:0/3419211988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e58032440 con 0x7f2e6415d4c0 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e6159a640 1 -- 172.21.15.40:0/3419211988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58032810 con 0x7f2e6415d4c0 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 -- 172.21.15.40:0/3419211988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 msgr2=0x7f2e640b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640b01f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802a270 comp rx=0 tx=0).stop 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 -- 172.21.15.40:0/3419211988 shutdown_connections 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640b01f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 --2- 172.21.15.40:0/3419211988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e640afe10 0x7f2e640cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 -- 172.21.15.40:0/3419211988 >> 172.21.15.40:0/3419211988 conn(0x7f2e64073910 msgr2=0x7f2e64073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 -- 172.21.15.40:0/3419211988 shutdown_connections 2024-04-26T21:18:38.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.783+0000 7f2e689a4640 1 -- 172.21.15.40:0/3419211988 wait complete. 2024-04-26T21:18:38.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 Processor -- start 2024-04-26T21:18:38.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 -- start start 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e64069ae0 0x7f2e6415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e640afe10 0x7f2e6414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6406f8e0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640d1140 con 0x7f2e640afe10 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e689a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e640ce100 con 0x7f2e64069ae0 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e62d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e62d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54210/0 (socket says 172.21.15.40:54210) 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.785+0000 7f2e62d9d640 1 -- 172.21.15.40:0/670504850 learned_addr learned my addr 172.21.15.40:0/670504850 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e6259c640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e64069ae0 0x7f2e6415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e61d9b640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e640afe10 0x7f2e6414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 -- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e64069ae0 msgr2=0x7f2e6415ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e64069ae0 0x7f2e6415ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 -- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e640afe10 msgr2=0x7f2e6414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e640afe10 0x7f2e6414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 -- 172.21.15.40:0/670504850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6415f1f0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e62d9d640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c3c0 tx=0x7f2e5c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c017070 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0052d0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6415f4d0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c003830 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.786+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6415fa90 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.787+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6406a290 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.788+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e5c01a070 con 0x7f2e6415d4c0 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.788+0000 7f2e52ffd640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 0x7f2e4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:38.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.788+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f2e5c08aab0 con 0x7f2e6415d4c0 2024-04-26T21:18:38.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.788+0000 7f2e6259c640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 0x7f2e4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:38.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.789+0000 7f2e6259c640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 0x7f2e4406ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e54003b80 tx=0x7f2e540098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:38.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.792+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e5c058b90 con 0x7f2e6415d4c0 2024-04-26T21:18:38.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.977+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e64073080 con 0x7f2e6415d4c0 2024-04-26T21:18:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.980+0000 7f2e52ffd640 1 -- 172.21.15.40:0/670504850 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f2e5c05ce90 con 0x7f2e6415d4c0 2024-04-26T21:18:38.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 msgr2=0x7f2e4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 0x7f2e4406ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e54003b80 tx=0x7f2e540098d0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 msgr2=0x7f2e640cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c3c0 tx=0x7f2e5c00c7c0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 shutdown_connections 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e4406c920 0x7f2e4406ede0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e6415d4c0 0x7f2e640cfef0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e640afe10 0x7f2e6414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 --2- 172.21.15.40:0/670504850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e64069ae0 0x7f2e6415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.982+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 >> 172.21.15.40:0/670504850 conn(0x7f2e64073910 msgr2=0x7f2e64159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.983+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 shutdown_connections 2024-04-26T21:18:38.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:38.983+0000 7f2e689a4640 1 -- 172.21.15.40:0/670504850 wait complete. 2024-04-26T21:18:38.994 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:18:38.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:39.032 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:39.084 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:18:39.084 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:39.084 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:39.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:18:39.100 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:39.100 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:39.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:18:39.117 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:39.117 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:39.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:18:39.155 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:39.155 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:39.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:18:39.173 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:18:39.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.295+0000 7fb95df04640 1 Processor -- start 2024-04-26T21:18:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.295+0000 7fb95df04640 1 -- start start 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb95df04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9580b44e0 0x7fb95806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb95df04640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 0x7fb95806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb95df04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9580b4a10 con 0x7fb95815f1e0 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb95df04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb95806dc60 con 0x7fb9580b44e0 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb95df04640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb95806eeb0 con 0x7fb95806d2b0 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9580b44e0 0x7fb95806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9580b44e0 0x7fb95806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54220/0 (socket says 172.21.15.40:54220) 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 -- 172.21.15.40:0/1353202361 learned_addr learned my addr 172.21.15.40:0/1353202361 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb957fff640 1 --2- 172.21.15.40:0/1353202361 >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 0x7fb95806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 -- 172.21.15.40:0/1353202361 >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 msgr2=0x7fb95806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 --2- 172.21.15.40:0/1353202361 >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 0x7fb95806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 -- 172.21.15.40:0/1353202361 >> v1:172.21.15.184:6789/0 conn(0x7fb95815f1e0 legacy=0x7fb9580ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb956ffd640 1 -- 172.21.15.40:0/1353202361 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9580b4cf0 con 0x7fb9580b44e0 2024-04-26T21:18:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.296+0000 7fb957fff640 1 --2- 172.21.15.40:0/1353202361 >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 0x7fb95806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb956ffd640 1 --2- 172.21.15.40:0/1353202361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb95806e970 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b0ba0 tx=0x7fb94c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1014cdb64b482c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb9567fc640 1 -- 172.21.15.40:0/1353202361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c037070 con 0x7fb9580b44e0 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb9567fc640 1 -- 172.21.15.40:0/1353202361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c032440 con 0x7fb9580b44e0 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb9567fc640 1 -- 172.21.15.40:0/1353202361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c032810 con 0x7fb9580b44e0 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb95df04640 1 -- 172.21.15.40:0/1353202361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 msgr2=0x7fb95806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.297+0000 7fb95df04640 1 --2- 172.21.15.40:0/1353202361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb95806e970 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b0ba0 tx=0x7fb94c02a270 comp rx=0 tx=0).stop 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 -- 172.21.15.40:0/1353202361 shutdown_connections 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 --2- 172.21.15.40:0/1353202361 >> v2:172.21.15.184:3300/0 conn(0x7fb95806d2b0 0x7fb95806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 --2- 172.21.15.40:0/1353202361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb95806e970 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 -- 172.21.15.40:0/1353202361 >> 172.21.15.40:0/1353202361 conn(0x7fb9580728a0 msgr2=0x7fb958072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 -- 172.21.15.40:0/1353202361 shutdown_connections 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.298+0000 7fb95df04640 1 -- 172.21.15.40:0/1353202361 wait complete. 2024-04-26T21:18:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.299+0000 7fb95df04640 1 Processor -- start 2024-04-26T21:18:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.299+0000 7fb95df04640 1 -- start start 2024-04-26T21:18:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 0x7fb9581f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb95815f1e0 0x7fb95815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9580b5590 con 0x7fb9580b44e0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9580b52b0 con 0x7fb95815f1e0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb95df04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb95815f5c0 con 0x7fb95806d2b0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54236/0 (socket says 172.21.15.40:54236) 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 -- 172.21.15.40:0/2626113395 learned_addr learned my addr 172.21.15.40:0/2626113395 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb9577fe640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 0x7fb9581f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 -- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 msgr2=0x7fb9581f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 0x7fb9581f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 -- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb95815f1e0 msgr2=0x7fb95815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb957fff640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb95815f1e0 0x7fb95815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb95815f1e0 0x7fb95815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 -- 172.21.15.40:0/2626113395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95815b7d0 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb9577fe640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 0x7fb9581f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.300+0000 7fb956ffd640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb94c02a2b0 tx=0x7fb94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.301+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c037040 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.301+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c042070 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.301+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95815bab0 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.301+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c030bb0 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.301+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95815c070 con 0x7fb9580b44e0 2024-04-26T21:18:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.302+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95806a290 con 0x7fb9580b44e0 2024-04-26T21:18:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.302+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb94c047460 con 0x7fb9580b44e0 2024-04-26T21:18:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.302+0000 7fb93ffff640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 0x7fb94406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.302+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7fb94c0ac5a0 con 0x7fb9580b44e0 2024-04-26T21:18:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.303+0000 7fb9577fe640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 0x7fb94406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.303+0000 7fb9577fe640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 0x7fb94406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb9480015e0 tx=0x7fb948009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.307+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb94c045360 con 0x7fb9580b44e0 2024-04-26T21:18:39.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.529+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb958068cb0 con 0x7fb9580b44e0 2024-04-26T21:18:39.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.530+0000 7fb93ffff640 1 -- 172.21.15.40:0/2626113395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+3872 (secure 0 0 0) 0x7fb94c07ac40 con 0x7fb9580b44e0 2024-04-26T21:18:39.531 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:39.531 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/105252979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":105252979},{"type":"v1","addr":"172.21.15.184:6837","nonce":105252979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6157,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4115395185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4115395185},{"type":"v1","addr":"172.21.15.40:6834","nonce":4115395185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1282903893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1282903893},{"type":"v1","addr":"172.21.15.184:6836","nonce":1282903893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2383622116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2383622116},{"type":"v1","addr":"172.21.15.40:6835","nonce":2383622116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[]} 2024-04-26T21:18:39.531 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 53 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 msgr2=0x7fb94406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 0x7fb94406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb9480015e0 tx=0x7fb948009040 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 msgr2=0x7fb958161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb94c02a2b0 tx=0x7fb94c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 shutdown_connections 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb94406c8f0 0x7fb94406edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb95815f1e0 0x7fb95815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9580b44e0 0x7fb958161450 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 --2- 172.21.15.40:0/2626113395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb95806d2b0 0x7fb9581f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.531+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 >> 172.21.15.40:0/2626113395 conn(0x7fb9580728a0 msgr2=0x7fb958163270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:39.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.532+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 shutdown_connections 2024-04-26T21:18:39.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.532+0000 7fb95df04640 1 -- 172.21.15.40:0/2626113395 wait complete. 2024-04-26T21:18:39.543 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:18:39.544 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:18:39.544 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:18:39.544 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:18:39.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:18:39.544 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:18:39.544 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:18:39.545 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:18:39.545 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:18:39.545 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:18:39.545 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:18:39.545 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:18:39.546 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:18:39.546 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:18:39.546 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:18:39.546 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:18:39.553 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:18:39.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:18:39.554 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:18:39.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:18:39.558 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:18:39.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:18:39.560 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:18:39.560 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:18:39.596 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:18:39.597 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:39.597 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:18:39.597 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:39.601 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:18:39.601 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:39.601 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:18:39.602 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:39.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:18:39.673 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:18:39.673 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:18:39.682 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:18:39.682 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:18:39.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.706+0000 7f8004197640 1 Processor -- start 2024-04-26T21:18:39.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.706+0000 7f8004197640 1 -- start start 2024-04-26T21:18:39.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8004197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 0x7f7ffc146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8004197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ffc140350 0x7f7ffc1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8004197640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc0d05b0 con 0x7f7ffc0cf810 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8004197640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7ffc0d0390 con 0x7f7ffc140350 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8004197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0d0890 con 0x7f7ffc13ff70 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8001732640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ffc140350 0x7f7ffc1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8001732640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ffc140350 0x7f7ffc1465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54294/0 (socket says 172.21.15.40:54294) 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8001732640 1 -- 172.21.15.40:0/1165718466 learned_addr learned my addr 172.21.15.40:0/1165718466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.707+0000 7f8001f33640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 0x7f7ffc146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001732640 1 -- 172.21.15.40:0/1165718466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 msgr2=0x7f7ffc146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001732640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 0x7f7ffc146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001732640 1 -- 172.21.15.40:0/1165718466 >> v1:172.21.15.184:6789/0 conn(0x7f7ffc0cf810 legacy=0x7f7ffc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001732640 1 -- 172.21.15.40:0/1165718466 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc06e450 con 0x7f7ffc140350 2024-04-26T21:18:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001f33640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 0x7f7ffc146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.708+0000 7f8001732640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc140350 0x7f7ffc1465c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009870 tx=0x7f7ff802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c84e01bfd4ae59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8000f31640 1 -- 172.21.15.40:0/1165718466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8037070 con 0x7f7ffc140350 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8000f31640 1 -- 172.21.15.40:0/1165718466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff8032440 con 0x7f7ffc140350 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8000f31640 1 -- 172.21.15.40:0/1165718466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8032810 con 0x7f7ffc140350 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 -- 172.21.15.40:0/1165718466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc140350 msgr2=0x7f7ffc1465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc140350 0x7f7ffc1465c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009870 tx=0x7f7ff802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 -- 172.21.15.40:0/1165718466 shutdown_connections 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc140350 0x7f7ffc1465c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 --2- 172.21.15.40:0/1165718466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc13ff70 0x7f7ffc146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 -- 172.21.15.40:0/1165718466 >> 172.21.15.40:0/1165718466 conn(0x7f7ffc0d0d30 msgr2=0x7f7ffc0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 -- 172.21.15.40:0/1165718466 shutdown_connections 2024-04-26T21:18:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.709+0000 7f8004197640 1 -- 172.21.15.40:0/1165718466 wait complete. 2024-04-26T21:18:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.710+0000 7f8004197640 1 Processor -- start 2024-04-26T21:18:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.710+0000 7f8004197640 1 -- start start 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ffc13ff70 0x7f7ffc13c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc140350 0x7f7ffc13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc06edd0 con 0x7f7ffc0cf810 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc06ec50 con 0x7f7ffc13ff70 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8004197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc06ef50 con 0x7f7ffc140350 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8001f33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8001f33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54304/0 (socket says 172.21.15.40:54304) 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8001f33640 1 -- 172.21.15.40:0/1624678878 learned_addr learned my addr 172.21.15.40:0/1624678878 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8001732640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ffc13ff70 0x7f7ffc13c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.711+0000 7f8002734640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc140350 0x7f7ffc13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 -- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc140350 msgr2=0x7f7ffc13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc140350 0x7f7ffc13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 -- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ffc13ff70 msgr2=0x7f7ffc13c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ffc13ff70 0x7f7ffc13c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 -- 172.21.15.40:0/1624678878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc13cc20 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8001f33640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e760 tx=0x7f7fec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00f040 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec004230 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00ee70 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc13a650 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.712+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc13c830 con 0x7f7ffc0cf810 2024-04-26T21:18:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.713+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc06a290 con 0x7f7ffc0cf810 2024-04-26T21:18:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.714+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fec00c6d0 con 0x7f7ffc0cf810 2024-04-26T21:18:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.714+0000 7f7ff27fc640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 0x7f7fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.714+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4943+0+0 (secure 0 0 0) 0x7f7fec08b900 con 0x7f7ffc0cf810 2024-04-26T21:18:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.714+0000 7f8001732640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 0x7f7fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.715+0000 7f8001732640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 0x7f7fe006edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8034000 tx=0x7f7ff802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.718+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fec017020 con 0x7f7ffc0cf810 2024-04-26T21:18:39.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:39.914+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7ffc13d1a0 con 0x7f7ffc0cf810 2024-04-26T21:18:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.994+0000 7f7ff27fc640 1 -- 172.21.15.40:0/1624678878 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) ==== 97+0+0 (secure 0 0 0) 0x7f7fec059ac0 con 0x7f7ffc0cf810 2024-04-26T21:18:40.995 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:18:40.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 msgr2=0x7f7fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 0x7f7fe006edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8034000 tx=0x7f7ff802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:40.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 msgr2=0x7f7ffc13b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:40.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e760 tx=0x7f7fec00ec20 comp rx=0 tx=0).stop 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 shutdown_connections 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fe006c8f0 0x7f7fe006edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ffc140350 0x7f7ffc13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ffc13ff70 0x7f7ffc13c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 --2- 172.21.15.40:0/1624678878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ffc0cf810 0x7f7ffc13b130 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.997+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 >> 172.21.15.40:0/1624678878 conn(0x7f7ffc0d0d30 msgr2=0x7f7ffc071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.998+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 shutdown_connections 2024-04-26T21:18:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:40.998+0000 7f8004197640 1 -- 172.21.15.40:0/1624678878 wait complete. 2024-04-26T21:18:41.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:18:41.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.117+0000 7fda833e4640 1 Processor -- start 2024-04-26T21:18:41.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.117+0000 7fda833e4640 1 -- start start 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda833e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda7c15f340 0x7fda7c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda833e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 0x7fda7c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda833e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7c0b2510 con 0x7fda7c15f720 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda833e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fda7c0b27f0 con 0x7fda7c15f340 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda833e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0b2ad0 con 0x7fda7c0d5610 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda8097f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda7c15f720 0x7fda7c0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42974/0 (socket says 172.21.15.40:42974) 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda8097f640 1 -- 172.21.15.40:0/2227210413 learned_addr learned my addr 172.21.15.40:0/2227210413 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda81180640 1 --2- 172.21.15.40:0/2227210413 >> v2:172.21.15.184:3300/0 conn(0x7fda7c15f340 0x7fda7c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda81981640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 0x7fda7c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2262527045 0 0) 0x7fda7c0b2510 con 0x7fda7c15f720 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.118+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda68003520 con 0x7fda7c15f720 2024-04-26T21:18:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2105668209 0 0) 0x7fda68003520 con 0x7fda7c15f720 2024-04-26T21:18:41.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81180640 1 -- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 msgr2=0x7fda7c0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81180640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 0x7fda7c0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81180640 1 -- 172.21.15.40:0/2227210413 >> v1:172.21.15.40:6789/0 conn(0x7fda7c15f720 legacy=0x7fda7c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81180640 1 -- 172.21.15.40:0/2227210413 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0b2db0 con 0x7fda7c15f340 2024-04-26T21:18:41.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81981640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 0x7fda7c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.119+0000 7fda81180640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c15f340 0x7fda7c0b1e80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda7802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1502652a4cac30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78037070 con 0x7fda7c15f340 2024-04-26T21:18:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7802ab10 con 0x7fda7c15f340 2024-04-26T21:18:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda73fff640 1 -- 172.21.15.40:0/2227210413 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c001670 con 0x7fda7c15f340 2024-04-26T21:18:41.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 -- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c15f340 msgr2=0x7fda7c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c15f340 0x7fda7c0b1e80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda780097a0 tx=0x7fda7802a080 comp rx=0 tx=0).stop 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 -- 172.21.15.40:0/2227210413 shutdown_connections 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c0d5610 0x7fda7c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 --2- 172.21.15.40:0/2227210413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c15f340 0x7fda7c0b1e80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 -- 172.21.15.40:0/2227210413 >> 172.21.15.40:0/2227210413 conn(0x7fda7c08ec70 msgr2=0x7fda7c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 -- 172.21.15.40:0/2227210413 shutdown_connections 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.120+0000 7fda833e4640 1 -- 172.21.15.40:0/2227210413 wait complete. 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.121+0000 7fda833e4640 1 Processor -- start 2024-04-26T21:18:41.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 -- start start 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c0d5610 0x7fda7c0b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c15f720 0x7fda7c0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0b3940 con 0x7fda7c15f340 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0b37c0 con 0x7fda7c0d5610 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda833e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0b3ac0 con 0x7fda7c15f720 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda8097f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda8097f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39706/0 (socket says 172.21.15.40:39706) 2024-04-26T21:18:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.122+0000 7fda8097f640 1 -- 172.21.15.40:0/3101357347 learned_addr learned my addr 172.21.15.40:0/3101357347 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda81981640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c15f720 0x7fda7c0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda81180640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c0d5610 0x7fda7c0b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 -- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c15f720 msgr2=0x7fda7c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c15f720 0x7fda7c0ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 -- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c0d5610 msgr2=0x7fda7c0b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c0d5610 0x7fda7c0b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 -- 172.21.15.40:0/3101357347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0ba3f0 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda8097f640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0011e0 tx=0x7fda6c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c017070 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c005ad0 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c0b8570 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c007070 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.123+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c0b8b30 con 0x7fda7c15f340 2024-04-26T21:18:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.124+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c06a290 con 0x7fda7c15f340 2024-04-26T21:18:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.125+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda6c01a070 con 0x7fda7c15f340 2024-04-26T21:18:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.125+0000 7fda71ffb640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.125+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7fda6c08aac0 con 0x7fda7c15f340 2024-04-26T21:18:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.125+0000 7fda81180640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.126+0000 7fda81180640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 0x7fda6006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda78034000 tx=0x7fda7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.130+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda6c058c00 con 0x7fda7c15f340 2024-04-26T21:18:41.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.384+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fda7c0d3020 con 0x7fda7c15f340 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.385+0000 7fda71ffb640 1 -- 172.21.15.40:0/3101357347 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fda6c05cf00 con 0x7fda7c15f340 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:18:41.389 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.390 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:18:41.391 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:18:41.393 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:18:41.395 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:18:41.397 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:18:41.398 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:18:41.399 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:18:41.400 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 msgr2=0x7fda6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 0x7fda6006edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda78034000 tx=0x7fda7802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 msgr2=0x7fda7c0b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0011e0 tx=0x7fda6c00cb90 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 shutdown_connections 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda7c15f720 0x7fda7c0ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c15f340 0x7fda7c0b9eb0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 --2- 172.21.15.40:0/3101357347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda7c0d5610 0x7fda7c0b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 >> 172.21.15.40:0/3101357347 conn(0x7fda7c08ec70 msgr2=0x7fda7c0b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 shutdown_connections 2024-04-26T21:18:41.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.387+0000 7fda833e4640 1 -- 172.21.15.40:0/3101357347 wait complete. 2024-04-26T21:18:41.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.501+0000 7f396de52640 1 Processor -- start 2024-04-26T21:18:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.501+0000 7f396de52640 1 -- start start 2024-04-26T21:18:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f396de52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39680d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f396de52640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f39680d1510 0x7f39680d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f396de52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39680ceef0 con 0x7f396809c450 2024-04-26T21:18:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f396de52640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f396815d740 con 0x7f39680d1510 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f396de52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39680cda60 con 0x7f396815d360 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f3966ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f396809c450 0x7f39680ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42982/0 (socket says 172.21.15.40:42982) 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.502+0000 7f3966ffd640 1 -- 172.21.15.40:0/376655155 learned_addr learned my addr 172.21.15.40:0/376655155 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f3967fff640 1 --2- 172.21.15.40:0/376655155 >> v2:172.21.15.184:3300/0 conn(0x7f39680d1510 0x7f39680d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39680d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713764762 0 0) 0x7f39680ceef0 con 0x7f396809c450 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3950003520 con 0x7f396809c450 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3088757848 0 0) 0x7f3950003520 con 0x7f396809c450 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 msgr2=0x7f39680d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39680d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 >> v2:172.21.15.184:3300/0 conn(0x7f39680d1510 msgr2=0x7f39680d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 --2- 172.21.15.40:0/376655155 >> v2:172.21.15.184:3300/0 conn(0x7f39680d1510 0x7f39680d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680d1100 con 0x7f396809c450 2024-04-26T21:18:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f395c001670 con 0x7f396809c450 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39680d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 >> v1:172.21.15.40:6789/0 conn(0x7f396809c450 legacy=0x7f39680ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f3950003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3950004800 0x7f3950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f396815d740 con 0x7f39500039b0 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39680cda60 con 0x7f3950004800 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f395000b9c0 con 0x7f396815d360 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f3967fff640 1 --2- 172.21.15.40:0/376655155 >> v2:172.21.15.184:3300/0 conn(0x7f39680d1510 0x7f39680d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f3950003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f3966ffd640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3950004800 0x7f3950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f3967fff640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 -- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 msgr2=0x7f39500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 -- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3950004800 msgr2=0x7f3950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3950004800 0x7f3950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f39677fe640 1 -- 172.21.15.40:0/376655155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3950003520 con 0x7f39500039b0 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.503+0000 7f3967fff640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f396815d360 0x7f39500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.504+0000 7f39677fe640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f3950003dd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3958009c90 tx=0x7f3950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d8124355cfe740 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.504+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958036070 con 0x7f39500039b0 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.504+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3958031400 con 0x7f39500039b0 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.504+0000 7f39667fc640 1 -- 172.21.15.40:0/376655155 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958009860 con 0x7f39500039b0 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 -- 172.21.15.40:0/376655155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 msgr2=0x7f3950003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 --2- 172.21.15.40:0/376655155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f3950003dd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3958009c90 tx=0x7f3950005830 comp rx=0 tx=0).stop 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f3967fff640 1 -- 172.21.15.40:0/376655155 reap_dead start 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 -- 172.21.15.40:0/376655155 shutdown_connections 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 -- 172.21.15.40:0/376655155 >> 172.21.15.40:0/376655155 conn(0x7f39680b4a40 msgr2=0x7f39680b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 -- 172.21.15.40:0/376655155 shutdown_connections 2024-04-26T21:18:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.505+0000 7f396de52640 1 -- 172.21.15.40:0/376655155 wait complete. 2024-04-26T21:18:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.506+0000 7f396de52640 1 Processor -- start 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.506+0000 7f396de52640 1 -- start start 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b6490 0x7f39680a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b4e50 0x7f39680cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f396809c8b0 con 0x7f39500039b0 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39680cdfe0 con 0x7f39680b6490 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f396de52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f39680d0b20 con 0x7f39680b4e50 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39724/0 (socket says 172.21.15.40:39724) 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 -- 172.21.15.40:0/4235199768 learned_addr learned my addr 172.21.15.40:0/4235199768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 -- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b4e50 msgr2=0x7f39680cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b4e50 0x7f39680cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 -- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b6490 msgr2=0x7f39680a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b6490 0x7f39680a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.507+0000 7f39677fe640 1 -- 172.21.15.40:0/4235199768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680b53f0 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f39677fe640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3958009db0 tx=0x7f3958004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958040020 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39580093e0 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39680b56d0 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958036040 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.508+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39680b5c90 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.509+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39680693d0 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.509+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3958043020 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.510+0000 7f394ffff640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 0x7f395406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.510+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f39580abe10 con 0x7f39500039b0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.510+0000 7f3966ffd640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 0x7f395406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.511+0000 7f3966ffd640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 0x7f395406edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f395c000f60 tx=0x7f395c000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.514+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f395804a030 con 0x7f39500039b0 2024-04-26T21:18:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.701+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39680d0450 con 0x7f39500039b0 2024-04-26T21:18:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.702+0000 7f394ffff640 1 -- 172.21.15.40:0/4235199768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+12892 (secure 0 0 0) 0x7f3958031e20 con 0x7f39500039b0 2024-04-26T21:18:41.704 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:41.704 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:40.993427+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.703+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 msgr2=0x7f395406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.703+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 0x7f395406edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f395c000f60 tx=0x7f395c000ed0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 msgr2=0x7f396815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3958009db0 tx=0x7f3958004730 comp rx=0 tx=0).stop 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 shutdown_connections 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f395406c8f0 0x7f395406edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f39680b4e50 0x7f39680cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39680b6490 0x7f39680a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 --2- 172.21.15.40:0/4235199768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39500039b0 0x7f396815a1c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 >> 172.21.15.40:0/4235199768 conn(0x7f39680b4a40 msgr2=0x7f39680c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 shutdown_connections 2024-04-26T21:18:41.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.704+0000 7f396de52640 1 -- 172.21.15.40:0/4235199768 wait complete. 2024-04-26T21:18:41.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:41.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:41.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.818+0000 7f984a8d3640 1 Processor -- start 2024-04-26T21:18:41.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.818+0000 7f984a8d3640 1 -- start start 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f984a8d3640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f98440d1be0 0x7f98440d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f984a8d3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98440d4240 con 0x7f984415f490 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f984a8d3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98440d31a0 con 0x7f98440ba3a0 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f984a8d3640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f98440d3480 con 0x7f98440d1be0 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f9843fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f984415f490 0x7f98440d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42986/0 (socket says 172.21.15.40:42986) 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f9843fff640 1 -- 172.21.15.40:0/3655934716 learned_addr learned my addr 172.21.15.40:0/3655934716 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1128413482 0 0) 0x7f98440d4240 con 0x7f984415f490 2024-04-26T21:18:41.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c003520 con 0x7f984415f490 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f98437fe640 1 --2- 172.21.15.40:0/3655934716 >> v2:172.21.15.184:3300/0 conn(0x7f98440d1be0 0x7f98440d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.819+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 597317590 0 0) 0x7f982c003520 con 0x7f984415f490 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 >> v2:172.21.15.184:3300/0 conn(0x7f98440d1be0 msgr2=0x7f98440d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 --2- 172.21.15.40:0/3655934716 >> v2:172.21.15.184:3300/0 conn(0x7f98440d1be0 0x7f98440d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 >> v1:172.21.15.184:6789/0 conn(0x7f98440ba3a0 legacy=0x7f98440ba7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98440d3760 con 0x7f984415f490 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9834001670 con 0x7f984415f490 2024-04-26T21:18:41.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 >> v1:172.21.15.40:6789/0 conn(0x7f984415f490 legacy=0x7f98440d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 0x7f982c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c004800 0x7f982c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c0051c0 0x7f982c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d3480 con 0x7f982c0039b0 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f982c00c060 con 0x7f982c0051c0 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f982c00c260 con 0x7f982c004800 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 0x7f982c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9848e70640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c0051c0 0x7f982c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f98437fe640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c004800 0x7f982c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 -- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c004800 msgr2=0x7f982c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c004800 0x7f982c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 -- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c0051c0 msgr2=0x7f982c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c0051c0 0x7f982c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9843fff640 1 -- 172.21.15.40:0/3655934716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c003520 con 0x7f982c0039b0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9848e70640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c0051c0 0x7f982c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.820+0000 7f9848e70640 1 -- 172.21.15.40:0/3655934716 reap_dead start 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f9843fff640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 0x7f982c003dd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f983400c210 tx=0x7f982c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676611ea9eaddf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834007d30 con 0x7f982c0039b0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834005ad0 con 0x7f982c0039b0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f9842ffd640 1 -- 172.21.15.40:0/3655934716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834007070 con 0x7f982c0039b0 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 -- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 msgr2=0x7f982c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 0x7f982c003dd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f983400c210 tx=0x7f982c005830 comp rx=0 tx=0).stop 2024-04-26T21:18:41.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 -- 172.21.15.40:0/3655934716 shutdown_connections 2024-04-26T21:18:41.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 --2- 172.21.15.40:0/3655934716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0039b0 0x7f982c003dd0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 -- 172.21.15.40:0/3655934716 >> 172.21.15.40:0/3655934716 conn(0x7f984409e530 msgr2=0x7f98440d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:41.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 -- 172.21.15.40:0/3655934716 shutdown_connections 2024-04-26T21:18:41.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.821+0000 7f984a8d3640 1 -- 172.21.15.40:0/3655934716 wait complete. 2024-04-26T21:18:41.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 Processor -- start 2024-04-26T21:18:41.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 -- start start 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0039b0 0x7f98441f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f984406dfe0 0x7f984406e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.823+0000 7f984a8d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f984415f870 con 0x7f98441610b0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f984a8d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d3d20 con 0x7f984406dfe0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f984a8d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98440d2dc0 con 0x7f982c0039b0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39742/0 (socket says 172.21.15.40:39742) 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 -- 172.21.15.40:0/2745101848 learned_addr learned my addr 172.21.15.40:0/2745101848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f9848e70640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f984406dfe0 0x7f984406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f9843fff640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0039b0 0x7f98441f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 -- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0039b0 msgr2=0x7f98441f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0039b0 0x7f98441f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 -- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f984406dfe0 msgr2=0x7f984406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f984406dfe0 0x7f984406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 -- 172.21.15.40:0/2745101848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9844161b90 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f98437fe640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9838011700 tx=0x7f9838011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800f9c0 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983800fb60 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9844161e70 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838015460 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.824+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9844162430 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.825+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984406a290 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.826+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9838015600 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.826+0000 7f9840ff9640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 0x7f983006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.826+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f983808bf40 con 0x7f98441610b0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.826+0000 7f9843fff640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 0x7f983006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.827+0000 7f9843fff640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 0x7f983006edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f983402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:41.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:41.831+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9838019030 con 0x7f98441610b0 2024-04-26T21:18:42.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.015+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9844163ee0 con 0x7f98441610b0 2024-04-26T21:18:42.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.015+0000 7f9840ff9640 1 -- 172.21.15.40:0/2745101848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f983805a100 con 0x7f98441610b0 2024-04-26T21:18:42.017 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 msgr2=0x7f983006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 0x7f983006edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f983402e000 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 msgr2=0x7f9844161530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9838011700 tx=0x7f9838011bc0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 shutdown_connections 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f983006c8f0 0x7f983006edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f984406dfe0 0x7f984406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98441610b0 0x7f9844161530 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 --2- 172.21.15.40:0/2745101848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0039b0 0x7f98441f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 >> 172.21.15.40:0/2745101848 conn(0x7f984409e530 msgr2=0x7f984406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 shutdown_connections 2024-04-26T21:18:42.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.017+0000 7f984a8d3640 1 -- 172.21.15.40:0/2745101848 wait complete. 2024-04-26T21:18:42.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:18:42.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.128+0000 7ff7f2064640 1 Processor -- start 2024-04-26T21:18:42.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.128+0000 7ff7f2064640 1 -- start start 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7f2064640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7ec0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7f2064640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ec0d03a0 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7f2064640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ec0d0680 con 0x7ff7ec15f750 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7f2064640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ec0d0960 con 0x7ff7ec15f370 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7ebfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff7ec0d3ee0 0x7ff7ec0b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42994/0 (socket says 172.21.15.40:42994) 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.129+0000 7ff7ebfff640 1 -- 172.21.15.40:0/3996736443 learned_addr learned my addr 172.21.15.40:0/3996736443 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eb7fe640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7ec0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313068501 0 0) 0x7ff7ec0d03a0 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d4003520 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541892190 0 0) 0x7ff7ec0d0680 con 0x7ff7ec15f750 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7ec0d03a0 con 0x7ff7ec15f750 2024-04-26T21:18:42.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1789585660 0 0) 0x7ff7d4003520 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 msgr2=0x7ff7ec0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7ec0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 >> v1:172.21.15.184:6789/0 conn(0x7ff7ec15f750 legacy=0x7ff7ec0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec0d0c40 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff7e40019f0 con 0x7ff7ec0d3ee0 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 >> v1:172.21.15.40:6789/0 conn(0x7ff7ec0d3ee0 legacy=0x7ff7ec0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d4003b70 0x7ff7d4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7d4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec0d0960 con 0x7ff7d4004800 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d400bb50 con 0x7ff7d4003b70 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d400bd20 con 0x7ff7ec15f370 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7ebfff640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7d4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eb7fe640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d4003b70 0x7ff7d4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eaffd640 1 -- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 msgr2=0x7ff7d4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec15f370 0x7ff7d4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eaffd640 1 -- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d4003b70 msgr2=0x7ff7d4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.130+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d4003b70 0x7ff7d4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.131+0000 7ff7eaffd640 1 -- 172.21.15.40:0/3996736443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d4003520 con 0x7ff7d4004800 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.131+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7d4004c80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0001900 tx=0x7ff7d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4541b5c6a634d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.131+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00014f0 con 0x7ff7d4004800 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.131+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e00052d0 con 0x7ff7d4004800 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.131+0000 7ff7ea7fc640 1 -- 172.21.15.40:0/3996736443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e002ac80 con 0x7ff7d4004800 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 -- 172.21.15.40:0/3996736443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 msgr2=0x7ff7d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3996736443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7d4004c80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0001900 tx=0x7ff7d4005830 comp rx=0 tx=0).stop 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7ebfff640 1 -- 172.21.15.40:0/3996736443 reap_dead start 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 -- 172.21.15.40:0/3996736443 shutdown_connections 2024-04-26T21:18:42.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 -- 172.21.15.40:0/3996736443 >> 172.21.15.40:0/3996736443 conn(0x7ff7ec08ec10 msgr2=0x7ff7ec09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 -- 172.21.15.40:0/3996736443 shutdown_connections 2024-04-26T21:18:42.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.132+0000 7ff7f2064640 1 -- 172.21.15.40:0/3996736443 wait complete. 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.133+0000 7ff7f2064640 1 Processor -- start 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.133+0000 7ff7f2064640 1 -- start start 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0cee80 0x7ff7ec0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0ba290 0x7ff7ec0ba710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec0cfef0 con 0x7ff7d4004800 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec0cfd70 con 0x7ff7ec0ba290 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7f2064640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ec0d0070 con 0x7ff7ec0cee80 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39762/0 (socket says 172.21.15.40:39762) 2024-04-26T21:18:42.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 -- 172.21.15.40:0/3309723550 learned_addr learned my addr 172.21.15.40:0/3309723550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0cee80 0x7ff7ec0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7ebfff640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0ba290 0x7ff7ec0ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 -- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0cee80 msgr2=0x7ff7ec0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0cee80 0x7ff7ec0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 -- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0ba290 msgr2=0x7ff7ec0ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0ba290 0x7ff7ec0ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.134+0000 7ff7eb7fe640 1 -- 172.21.15.40:0/3309723550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec0ca7e0 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7eb7fe640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc011a00 tx=0x7ff7dc011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00fb50 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00fcf0 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec0caac0 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00fea0 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.135+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec0cb080 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.136+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec06a290 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.136+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7dc012040 con 0x7ff7d4004800 2024-04-26T21:18:42.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.137+0000 7ff7d3fff640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 0x7ff7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.137+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7ff7dc08bcd0 con 0x7ff7d4004800 2024-04-26T21:18:42.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.137+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 0x7ff7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.138+0000 7ff7eaffd640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 0x7ff7d806edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0032000 tx=0x7ff7e0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.141+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7dc019030 con 0x7ff7d4004800 2024-04-26T21:18:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.327+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff7ec0d3b50 con 0x7ff7d4004800 2024-04-26T21:18:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.328+0000 7ff7d3fff640 1 -- 172.21.15.40:0/3309723550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) ==== 84+0+65 (secure 0 0 0) 0x7ff7dc059e00 con 0x7ff7d4004800 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:18:42.330 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:18:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 msgr2=0x7ff7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 0x7ff7d806edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0032000 tx=0x7ff7e0009030 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 msgr2=0x7ff7ec1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc011a00 tx=0x7ff7dc011ec0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 shutdown_connections 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff7d806c8f0 0x7ff7d806edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7ec0ba290 0x7ff7ec0ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7ec0cee80 0x7ff7ec0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 --2- 172.21.15.40:0/3309723550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d4004800 0x7ff7ec1625d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.329+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 >> 172.21.15.40:0/3309723550 conn(0x7ff7ec08ec10 msgr2=0x7ff7ec06d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.330+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 shutdown_connections 2024-04-26T21:18:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.330+0000 7ff7f2064640 1 -- 172.21.15.40:0/3309723550 wait complete. 2024-04-26T21:18:42.342 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:18:42.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:18:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.445+0000 7f95396d3640 1 Processor -- start 2024-04-26T21:18:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.445+0000 7f95396d3640 1 -- start start 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f95396d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9534162a10 0x7f95340d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f95396d3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95340b6020 0x7f95340d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f95396d3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95340d33a0 con 0x7f95340d1f70 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f95396d3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b4970 con 0x7f95340b6020 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f95396d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95340d3680 con 0x7f9534162a10 2024-04-26T21:18:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95340b6020 0x7f95340d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95340b6020 0x7f95340d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39778/0 (socket says 172.21.15.40:39778) 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 -- 172.21.15.40:0/918097876 learned_addr learned my addr 172.21.15.40:0/918097876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f9532ffd640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9534162a10 0x7f95340d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 -- 172.21.15.40:0/918097876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9534162a10 msgr2=0x7f95340d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9534162a10 0x7f95340d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 -- 172.21.15.40:0/918097876 >> v1:172.21.15.184:6789/0 conn(0x7f95340d1f70 legacy=0x7f9534163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.446+0000 7f952bfff640 1 -- 172.21.15.40:0/918097876 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340d3960 con 0x7f95340b6020 2024-04-26T21:18:42.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.447+0000 7f952bfff640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95340b6020 0x7f95340d1a30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f951c009870 tx=0x7f951c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d989a6bb521f9454 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.447+0000 7f95327fc640 1 -- 172.21.15.40:0/918097876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c037070 con 0x7f95340b6020 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.447+0000 7f95327fc640 1 -- 172.21.15.40:0/918097876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c032440 con 0x7f95340b6020 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.447+0000 7f95327fc640 1 -- 172.21.15.40:0/918097876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c032810 con 0x7f95340b6020 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.447+0000 7f95396d3640 1 -- 172.21.15.40:0/918097876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95340b6020 msgr2=0x7f95340d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95340b6020 0x7f95340d1a30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f951c009870 tx=0x7f951c02a210 comp rx=0 tx=0).stop 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 -- 172.21.15.40:0/918097876 shutdown_connections 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95340b6020 0x7f95340d1a30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 --2- 172.21.15.40:0/918097876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9534162a10 0x7f95340d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 -- 172.21.15.40:0/918097876 >> 172.21.15.40:0/918097876 conn(0x7f953409e530 msgr2=0x7f95340d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 -- 172.21.15.40:0/918097876 shutdown_connections 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.448+0000 7f95396d3640 1 -- 172.21.15.40:0/918097876 wait complete. 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.449+0000 7f95396d3640 1 Processor -- start 2024-04-26T21:18:42.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.449+0000 7f95396d3640 1 -- start start 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.449+0000 7f95396d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340b6020 0x7f953415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.449+0000 7f95396d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d1f70 0x7f95340ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.449+0000 7f95396d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95396d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9534162df0 con 0x7f9534162a10 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95396d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95340d3f20 con 0x7f95340b6020 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95396d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95340b6600 con 0x7f95340d1f70 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39788/0 (socket says 172.21.15.40:39788) 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 -- 172.21.15.40:0/732570320 learned_addr learned my addr 172.21.15.40:0/732570320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f9532ffd640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340b6020 0x7f953415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f952bfff640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d1f70 0x7f95340ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 -- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d1f70 msgr2=0x7f95340ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d1f70 0x7f95340ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 -- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340b6020 msgr2=0x7f953415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340b6020 0x7f953415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 -- 172.21.15.40:0/732570320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340c96d0 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95337fe640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b970 tx=0x7f952c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c019070 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c00cb40 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340c99b0 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c014440 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.450+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95340c9f70 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.451+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953406a290 con 0x7f9534162a10 2024-04-26T21:18:42.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.452+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f952c01c070 con 0x7f9534162a10 2024-04-26T21:18:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.452+0000 7f952b7fe640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 0x7f951406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.452+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f952c08b360 con 0x7f9534162a10 2024-04-26T21:18:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.453+0000 7f9532ffd640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 0x7f951406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.453+0000 7f9532ffd640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 0x7f951406ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95341f8dc0 tx=0x7f9524009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.457+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f952c059480 con 0x7f9534162a10 2024-04-26T21:18:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.642+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f95340b6ac0 con 0x7f9534162a10 2024-04-26T21:18:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:42.996+0000 7f952b7fe640 1 -- 172.21.15.40:0/732570320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) ==== 138+0+0 (secure 0 0 0) 0x7f952c05d780 con 0x7f9534162a10 2024-04-26T21:18:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:18:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 msgr2=0x7f951406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 0x7f951406ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95341f8dc0 tx=0x7f9524009190 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 msgr2=0x7f9534161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b970 tx=0x7f952c00be30 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 shutdown_connections 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951406c920 0x7f951406ede0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9534162a10 0x7f9534161570 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d1f70 0x7f95340ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 --2- 172.21.15.40:0/732570320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340b6020 0x7f953415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 >> 172.21.15.40:0/732570320 conn(0x7f953409e530 msgr2=0x7f95340d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.000+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 shutdown_connections 2024-04-26T21:18:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.001+0000 7f95396d3640 1 -- 172.21.15.40:0/732570320 wait complete. 2024-04-26T21:18:43.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:18:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.126+0000 7fe48621d640 1 Processor -- start 2024-04-26T21:18:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.126+0000 7fe48621d640 1 -- start start 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe48621d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4800d2170 0x7fe4800d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe48621d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 0x7fe4800cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe48621d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4800d33d0 con 0x7fe480162a40 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe48621d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe4800b49a0 con 0x7fe4800d2170 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe48621d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe4800d36b0 con 0x7fe4800b6050 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4800d2170 0x7fe4800d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4800d2170 0x7fe4800d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39804/0 (socket says 172.21.15.40:39804) 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 -- 172.21.15.40:0/2318312186 learned_addr learned my addr 172.21.15.40:0/2318312186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47ffff640 1 --2- 172.21.15.40:0/2318312186 >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 0x7fe4800cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 -- 172.21.15.40:0/2318312186 >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 msgr2=0x7fe4800cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 --2- 172.21.15.40:0/2318312186 >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 0x7fe4800cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 -- 172.21.15.40:0/2318312186 >> v1:172.21.15.184:6789/0 conn(0x7fe480162a40 legacy=0x7fe4800d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47effd640 1 -- 172.21.15.40:0/2318312186 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4800d3990 con 0x7fe4800d2170 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.127+0000 7fe47ffff640 1 --2- 172.21.15.40:0/2318312186 >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 0x7fe4800cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.128+0000 7fe47effd640 1 --2- 172.21.15.40:0/2318312186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800d2170 0x7fe4800d2df0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe48008e690 tx=0x7fe47002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b334c9edd0e9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.128+0000 7fe47e7fc640 1 -- 172.21.15.40:0/2318312186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470037070 con 0x7fe4800d2170 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.128+0000 7fe47e7fc640 1 -- 172.21.15.40:0/2318312186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe470032440 con 0x7fe4800d2170 2024-04-26T21:18:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.128+0000 7fe47e7fc640 1 -- 172.21.15.40:0/2318312186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470032810 con 0x7fe4800d2170 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 -- 172.21.15.40:0/2318312186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800d2170 msgr2=0x7fe4800d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 --2- 172.21.15.40:0/2318312186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800d2170 0x7fe4800d2df0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe48008e690 tx=0x7fe47002a230 comp rx=0 tx=0).stop 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 -- 172.21.15.40:0/2318312186 shutdown_connections 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 --2- 172.21.15.40:0/2318312186 >> v2:172.21.15.184:3300/0 conn(0x7fe4800b6050 0x7fe4800cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 --2- 172.21.15.40:0/2318312186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800d2170 0x7fe4800d2df0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 -- 172.21.15.40:0/2318312186 >> 172.21.15.40:0/2318312186 conn(0x7fe48009e560 msgr2=0x7fe4800d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 -- 172.21.15.40:0/2318312186 shutdown_connections 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.129+0000 7fe48621d640 1 -- 172.21.15.40:0/2318312186 wait complete. 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.130+0000 7fe48621d640 1 Processor -- start 2024-04-26T21:18:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.130+0000 7fe48621d640 1 -- start start 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4800d2170 0x7fe48015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 0x7fe480161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe480162e20 con 0x7fe4800b6050 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4800d3f50 con 0x7fe480162a40 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe48621d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4800b6630 con 0x7fe4800d2170 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39814/0 (socket says 172.21.15.40:39814) 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 -- 172.21.15.40:0/1723738854 learned_addr learned my addr 172.21.15.40:0/1723738854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47ffff640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 0x7fe480161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47effd640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4800d2170 0x7fe48015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 -- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4800d2170 msgr2=0x7fe48015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4800d2170 0x7fe48015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 -- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 msgr2=0x7fe480161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 0x7fe480161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 -- 172.21.15.40:0/1723738854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4801605f0 con 0x7fe4800b6050 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47ffff640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 0x7fe480161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.131+0000 7fe47f7fe640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b420 tx=0x7fe47400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.132+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474011830 con 0x7fe4800b6050 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.132+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe474016070 con 0x7fe4800b6050 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.132+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4801608b0 con 0x7fe4800b6050 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.132+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474011c10 con 0x7fe4800b6050 2024-04-26T21:18:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.132+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe480160e70 con 0x7fe4800b6050 2024-04-26T21:18:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.133+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4800d45d0 con 0x7fe4800b6050 2024-04-26T21:18:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.133+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe47401b460 con 0x7fe4800b6050 2024-04-26T21:18:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.134+0000 7fe467fff640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 0x7fe46c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.134+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5227+0+0 (secure 0 0 0) 0x7fe47408b240 con 0x7fe4800b6050 2024-04-26T21:18:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.135+0000 7fe47effd640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 0x7fe46c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.137+0000 7fe47effd640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 0x7fe46c06ede0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe470034000 tx=0x7fe47002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.138+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4800d45d0 con 0x7fe4800b6050 2024-04-26T21:18:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:43.323+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe4800b6af0 con 0x7fe4800b6050 2024-04-26T21:18:44.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.002+0000 7fe467fff640 1 -- 172.21.15.40:0/1723738854 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) ==== 170+0+0 (secure 0 0 0) 0x7fe474020020 con 0x7fe4800b6050 2024-04-26T21:18:44.004 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:18:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 msgr2=0x7fe46c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 0x7fe46c06ede0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe470034000 tx=0x7fe47002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 msgr2=0x7fe48015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b420 tx=0x7fe47400b8e0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 shutdown_connections 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe46c06c920 0x7fe46c06ede0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe480162a40 0x7fe480161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4800d2170 0x7fe48015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 --2- 172.21.15.40:0/1723738854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4800b6050 0x7fe48015a5e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 >> 172.21.15.40:0/1723738854 conn(0x7fe48009e560 msgr2=0x7fe48006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 shutdown_connections 2024-04-26T21:18:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.005+0000 7fe48621d640 1 -- 172.21.15.40:0/1723738854 wait complete. 2024-04-26T21:18:44.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:18:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.143+0000 7fd88de07640 1 Processor -- start 2024-04-26T21:18:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.143+0000 7fd88de07640 1 -- start start 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd88de07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8880ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd88de07640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880d07e0 con 0x7fd888160950 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd88de07640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880d2090 con 0x7fd8880b1ef0 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd88de07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd88809c700 con 0x7fd8880d0090 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd8877fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd888160950 0x7fd8880cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43010/0 (socket says 172.21.15.40:43010) 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd8877fe640 1 -- 172.21.15.40:0/1668490697 learned_addr learned my addr 172.21.15.40:0/1668490697 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd886ffd640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8880ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3234316411 0 0) 0x7fd8880d07e0 con 0x7fd888160950 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.144+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd870003520 con 0x7fd888160950 2024-04-26T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1539085812 0 0) 0x7fd870003520 con 0x7fd888160950 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 msgr2=0x7fd8880ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8880ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 >> v1:172.21.15.184:6789/0 conn(0x7fd8880b1ef0 legacy=0x7fd8880ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880d1a40 con 0x7fd888160950 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd878001670 con 0x7fd888160950 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 >> v1:172.21.15.40:6789/0 conn(0x7fd888160950 legacy=0x7fd8880cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd870003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd870004390 0x7fd870004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8700050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd88809c700 con 0x7fd8700039e0 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd87000b9c0 con 0x7fd870004390 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd87000bbc0 con 0x7fd8880d0090 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd870003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd886ffd640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd870004390 0x7fd870004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd887fff640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8700050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 -- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 msgr2=0x7fd8700050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880d0090 0x7fd8700050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 -- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd870004390 msgr2=0x7fd870004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd870004390 0x7fd870004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.145+0000 7fd8877fe640 1 -- 172.21.15.40:0/1668490697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd870003520 con 0x7fd8700039e0 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.146+0000 7fd8877fe640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd870003dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c190 tx=0x7fd870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7d3f9b31c46b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.146+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878007cc0 con 0x7fd8700039e0 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.146+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd878005ad0 con 0x7fd8700039e0 2024-04-26T21:18:44.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.146+0000 7fd8867fc640 1 -- 172.21.15.40:0/1668490697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878007070 con 0x7fd8700039e0 2024-04-26T21:18:44.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 -- 172.21.15.40:0/1668490697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 msgr2=0x7fd870003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 --2- 172.21.15.40:0/1668490697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd870003dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c190 tx=0x7fd870005830 comp rx=0 tx=0).stop 2024-04-26T21:18:44.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd887fff640 1 -- 172.21.15.40:0/1668490697 reap_dead start 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 -- 172.21.15.40:0/1668490697 shutdown_connections 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 -- 172.21.15.40:0/1668490697 >> 172.21.15.40:0/1668490697 conn(0x7fd88809ba40 msgr2=0x7fd88809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 -- 172.21.15.40:0/1668490697 shutdown_connections 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.147+0000 7fd88de07640 1 -- 172.21.15.40:0/1668490697 wait complete. 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.148+0000 7fd88de07640 1 Processor -- start 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.148+0000 7fd88de07640 1 -- start start 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880b6700 0x7fd88815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8880b05c0 0x7fd8880b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880b28f0 con 0x7fd8700039e0 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd888160d30 con 0x7fd8880b05c0 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd88de07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880b2a70 con 0x7fd8880b6700 2024-04-26T21:18:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39844/0 (socket says 172.21.15.40:39844) 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 -- 172.21.15.40:0/2523056893 learned_addr learned my addr 172.21.15.40:0/2523056893 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd887fff640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8880b05c0 0x7fd8880b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd886ffd640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880b6700 0x7fd88815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 -- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880b6700 msgr2=0x7fd88815f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880b6700 0x7fd88815f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 -- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8880b05c0 msgr2=0x7fd8880b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8880b05c0 0x7fd8880b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 -- 172.21.15.40:0/2523056893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880b48d0 con 0x7fd8700039e0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd8877fe640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd87802ad20 tx=0x7fd8780019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.149+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878001430 con 0x7fd8700039e0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.150+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd878042070 con 0x7fd8700039e0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.150+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8880b4bb0 con 0x7fd8700039e0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.150+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878007070 con 0x7fd8700039e0 2024-04-26T21:18:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.150+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88809cfd0 con 0x7fd8700039e0 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.151+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8880b0f80 con 0x7fd8700039e0 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.151+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd878007210 con 0x7fd8700039e0 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.152+0000 7fd86ffff640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 0x7fd87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.152+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5609+0+0 (secure 0 0 0) 0x7fd8780aba40 con 0x7fd8700039e0 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.152+0000 7fd886ffd640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 0x7fd87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.152+0000 7fd886ffd640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 0x7fd87406edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd8880b3cb0 tx=0x7fd87c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.156+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd878079f60 con 0x7fd8700039e0 2024-04-26T21:18:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:44.380+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd8880b50d0 con 0x7fd8700039e0 2024-04-26T21:18:45.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.004+0000 7fd86ffff640 1 -- 172.21.15.40:0/2523056893 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v55) ==== 174+0+0 (secure 0 0 0) 0x7fd8780abe20 con 0x7fd8700039e0 2024-04-26T21:18:45.006 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 14 and data pool 15 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 msgr2=0x7fd87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 0x7fd87406edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd8880b3cb0 tx=0x7fd87c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 msgr2=0x7fd8880b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd87802ad20 tx=0x7fd8780019c0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 shutdown_connections 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd87406c8f0 0x7fd87406edb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8880b05c0 0x7fd8880b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8880b6700 0x7fd88815f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 --2- 172.21.15.40:0/2523056893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8700039e0 0x7fd8880b4080 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 >> 172.21.15.40:0/2523056893 conn(0x7fd88809ba40 msgr2=0x7fd8880b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 shutdown_connections 2024-04-26T21:18:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.007+0000 7fd88de07640 1 -- 172.21.15.40:0/2523056893 wait complete. 2024-04-26T21:18:45.034 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 Processor -- start 2024-04-26T21:18:45.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 -- start start 2024-04-26T21:18:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31580d5610 0x7f31580b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31580b2510 con 0x7f315815f340 2024-04-26T21:18:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31580b27f0 con 0x7f315815f720 2024-04-26T21:18:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.142+0000 7f315d2a9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f31580b2ad0 con 0x7f31580d5610 2024-04-26T21:18:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3156ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f315815f340 0x7f31580d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43014/0 (socket says 172.21.15.40:43014) 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3156ffd640 1 -- 172.21.15.40:0/2580459838 learned_addr learned my addr 172.21.15.40:0/2580459838 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f31577fe640 1 --2- 172.21.15.40:0/2580459838 >> v2:172.21.15.184:3300/0 conn(0x7f31580d5610 0x7f31580b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801889879 0 0) 0x7f31580b2510 con 0x7f315815f340 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3144003520 con 0x7f315815f340 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2465576808 0 0) 0x7f3144003520 con 0x7f315815f340 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 >> v2:172.21.15.184:3300/0 conn(0x7f31580d5610 msgr2=0x7f31580b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 --2- 172.21.15.40:0/2580459838 >> v2:172.21.15.184:3300/0 conn(0x7f31580d5610 0x7f31580b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 >> v1:172.21.15.184:6789/0 conn(0x7f315815f720 legacy=0x7f31580b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31580b2db0 con 0x7f315815f340 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3150001670 con 0x7f315815f340 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 >> v1:172.21.15.40:6789/0 conn(0x7f315815f340 legacy=0x7f31580d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31440039b0 0x7f3144003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f3144004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3144005120 0x7f314400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31580b2ad0 con 0x7f31440043b0 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f314400c060 con 0x7f31440039b0 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f314400c260 con 0x7f3144005120 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f31567fc640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f3144004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f3156ffd640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31440039b0 0x7f3144003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.143+0000 7f31577fe640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3144005120 0x7f314400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 -- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3144005120 msgr2=0x7f314400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3144005120 0x7f314400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 -- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31440039b0 msgr2=0x7f3144003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31440039b0 0x7f3144003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 -- 172.21.15.40:0/2580459838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3144003520 con 0x7f31440043b0 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31577fe640 1 -- 172.21.15.40:0/2580459838 reap_dead start 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f31567fc640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f3144004880 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f31400018e0 tx=0x7f3144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b505032e64167dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140037070 con 0x7f31440043b0 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314002ac10 con 0x7f31440043b0 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.144+0000 7f3155ffb640 1 -- 172.21.15.40:0/2580459838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400014e0 con 0x7f31440043b0 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 -- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 msgr2=0x7f3144004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f3144004880 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f31400018e0 tx=0x7f3144005830 comp rx=0 tx=0).stop 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 -- 172.21.15.40:0/2580459838 shutdown_connections 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 --2- 172.21.15.40:0/2580459838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f3144004880 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 -- 172.21.15.40:0/2580459838 >> 172.21.15.40:0/2580459838 conn(0x7f315808ec70 msgr2=0x7f315809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 -- 172.21.15.40:0/2580459838 shutdown_connections 2024-04-26T21:18:45.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.145+0000 7f315d2a9640 1 -- 172.21.15.40:0/2580459838 wait complete. 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.146+0000 7f315d2a9640 1 Processor -- start 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.146+0000 7f315d2a9640 1 -- start start 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 0x7f31580cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f315815bb50 0x7f315815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31580b2210 con 0x7f31440043b0 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31580b2090 con 0x7f315815bb50 2024-04-26T21:18:45.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f315d2a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31580d3d40 con 0x7f31580ce610 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39852/0 (socket says 172.21.15.40:39852) 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f3156ffd640 1 -- 172.21.15.40:0/1652920993 learned_addr learned my addr 172.21.15.40:0/1652920993 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f31567fc640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 0x7f31580cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.147+0000 7f31577fe640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f315815bb50 0x7f315815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 -- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 msgr2=0x7f31580cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 0x7f31580cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 -- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f315815bb50 msgr2=0x7f315815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f315815bb50 0x7f315815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 -- 172.21.15.40:0/1652920993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3158161620 con 0x7f31440043b0 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f31567fc640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 0x7f31580cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:45.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f3156ffd640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3150004ef0 tx=0x7f3150007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315000f730 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150018070 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3158161900 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315000fb10 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.148+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3158161ec0 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.149+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31580cefb0 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.150+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f315000fcb0 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.150+0000 7f314f7fe640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 0x7f313806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.150+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f315008bab0 con 0x7f31440043b0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.150+0000 7f31567fc640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 0x7f313806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.151+0000 7f31567fc640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 0x7f313806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3140034000 tx=0x7f314002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.154+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3150059e50 con 0x7f31440043b0 2024-04-26T21:18:45.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.339+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f315806a290 con 0x7f31440043b0 2024-04-26T21:18:45.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.340+0000 7f314f7fe640 1 -- 172.21.15.40:0/1652920993 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16621 (secure 0 0 0) 0x7f3150059750 con 0x7f31440043b0 2024-04-26T21:18:45.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:45.342 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":96,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:44.382321+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 msgr2=0x7f313806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 0x7f313806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3140034000 tx=0x7f314002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 msgr2=0x7f315815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3150004ef0 tx=0x7f3150007430 comp rx=0 tx=0).stop 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 shutdown_connections 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f313806c8f0 0x7f313806edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f315815bb50 0x7f315815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31580ce610 0x7f31580cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 --2- 172.21.15.40:0/1652920993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31440043b0 0x7f315815b0d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 >> 172.21.15.40:0/1652920993 conn(0x7f315808ec70 msgr2=0x7f31580d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 shutdown_connections 2024-04-26T21:18:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.342+0000 7f315d2a9640 1 -- 172.21.15.40:0/1652920993 wait complete. 2024-04-26T21:18:45.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.455+0000 7f9604514640 1 Processor -- start 2024-04-26T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.455+0000 7f9604514640 1 -- start start 2024-04-26T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f9604514640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95fc0d5610 0x7f95fc0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f9604514640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc0b2510 con 0x7f95fc15f340 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f9604514640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc0b27f0 con 0x7f95fc15f720 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f9604514640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc0b2ad0 con 0x7f95fc0d5610 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f96022b0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95fc15f340 0x7f95fc0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43018/0 (socket says 172.21.15.40:43018) 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f96022b0640 1 -- 172.21.15.40:0/3532137462 learned_addr learned my addr 172.21.15.40:0/3532137462 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f9602ab1640 1 --2- 172.21.15.40:0/3532137462 >> v2:172.21.15.40:3300/0 conn(0x7f95fc0d5610 0x7f95fc0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f96012ae640 1 -- 172.21.15.40:0/3532137462 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867550757 0 0) 0x7f95fc0b2510 con 0x7f95fc15f340 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.456+0000 7f96012ae640 1 -- 172.21.15.40:0/3532137462 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95e4003520 con 0x7f95fc15f340 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f9602ab1640 1 -- 172.21.15.40:0/3532137462 >> v1:172.21.15.184:6789/0 conn(0x7f95fc15f720 legacy=0x7f95fc0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f9602ab1640 1 -- 172.21.15.40:0/3532137462 >> v1:172.21.15.40:6789/0 conn(0x7f95fc15f340 legacy=0x7f95fc0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f9602ab1640 1 -- 172.21.15.40:0/3532137462 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0b2db0 con 0x7f95fc0d5610 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f9602ab1640 1 --2- 172.21.15.40:0/3532137462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc0d5610 0x7f95fc0b1e80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009d90 tx=0x7f95ec02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55286cec4d6ea4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f96012ae640 1 -- 172.21.15.40:0/3532137462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec031600 con 0x7f95fc0d5610 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f96012ae640 1 -- 172.21.15.40:0/3532137462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec0317a0 con 0x7f95fc0d5610 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.457+0000 7f96012ae640 1 -- 172.21.15.40:0/3532137462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec031b70 con 0x7f95fc0d5610 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 -- 172.21.15.40:0/3532137462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc0d5610 msgr2=0x7f95fc0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 --2- 172.21.15.40:0/3532137462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc0d5610 0x7f95fc0b1e80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009d90 tx=0x7f95ec02a610 comp rx=0 tx=0).stop 2024-04-26T21:18:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 -- 172.21.15.40:0/3532137462 shutdown_connections 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 --2- 172.21.15.40:0/3532137462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc0d5610 0x7f95fc0b1e80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 -- 172.21.15.40:0/3532137462 >> 172.21.15.40:0/3532137462 conn(0x7f95fc08ec70 msgr2=0x7f95fc09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 -- 172.21.15.40:0/3532137462 shutdown_connections 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.458+0000 7f9604514640 1 -- 172.21.15.40:0/3532137462 wait complete. 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.459+0000 7f9604514640 1 Processor -- start 2024-04-26T21:18:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.459+0000 7f9604514640 1 -- start start 2024-04-26T21:18:45.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95fc0d5610 0x7f95fc155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc15f720 0x7f95fc14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0d40d0 con 0x7f95fc15f340 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0d3f50 con 0x7f95fc15f720 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9604514640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0b3cc0 con 0x7f95fc0d5610 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39872/0 (socket says 172.21.15.40:39872) 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 -- 172.21.15.40:0/4271851021 learned_addr learned my addr 172.21.15.40:0/4271851021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9602ab1640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc15f720 0x7f95fc14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f96022b0640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95fc0d5610 0x7f95fc155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 -- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95fc0d5610 msgr2=0x7f95fc155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95fc0d5610 0x7f95fc155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 -- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc15f720 msgr2=0x7f95fc14f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc15f720 0x7f95fc14f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.460+0000 7f9601aaf640 1 -- 172.21.15.40:0/4271851021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc15fbc0 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f9601aaf640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b3c0 tx=0x7f95f000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011830 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f0016070 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc15fea0 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011c50 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.461+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc0b8f80 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.462+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc06a290 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.462+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95f001b460 con 0x7f95fc15f340 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.463+0000 7f95faffd640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 0x7f95e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.463+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f95f008b5e0 con 0x7f95fc15f340 2024-04-26T21:18:45.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.463+0000 7f96022b0640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 0x7f95e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.464+0000 7f96022b0640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 0x7f95e806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f95f4005880 tx=0x7f95f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.467+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95f0020020 con 0x7f95fc15f340 2024-04-26T21:18:45.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.651+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95fc0b2160 con 0x7f95fc15f340 2024-04-26T21:18:45.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.653+0000 7f95faffd640 1 -- 172.21.15.40:0/4271851021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16621 (secure 0 0 0) 0x7f95f0059aa0 con 0x7f95fc15f340 2024-04-26T21:18:45.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:45.655 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":96,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:44.382321+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 msgr2=0x7f95e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 0x7f95e806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f95f4005880 tx=0x7f95f4009040 comp rx=0 tx=0).stop 2024-04-26T21:18:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 msgr2=0x7f95fc0b94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b3c0 tx=0x7f95f000b880 comp rx=0 tx=0).stop 2024-04-26T21:18:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 shutdown_connections 2024-04-26T21:18:45.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f95e806c8f0 0x7f95e806edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc15f720 0x7f95fc14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95fc15f340 0x7f95fc0b94a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 --2- 172.21.15.40:0/4271851021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95fc0d5610 0x7f95fc155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 >> 172.21.15.40:0/4271851021 conn(0x7f95fc08ec70 msgr2=0x7f95fc0d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.654+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 shutdown_connections 2024-04-26T21:18:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.655+0000 7f9604514640 1 -- 172.21.15.40:0/4271851021 wait complete. 2024-04-26T21:18:45.668 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:18:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.770+0000 7f2500a09640 1 Processor -- start 2024-04-26T21:18:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.770+0000 7f2500a09640 1 -- start start 2024-04-26T21:18:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f2500a09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f2500a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f2500a09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24fc0d3c90 con 0x7f24fc15f680 2024-04-26T21:18:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f2500a09640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f24fc15fa60 con 0x7f24fc0d3820 2024-04-26T21:18:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f2500a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc0d3f50 con 0x7f24fc0b60b0 2024-04-26T21:18:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24f9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24fad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24fad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57582/0 (socket says 172.21.15.40:57582) 2024-04-26T21:18:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24f9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52002/0 (socket says 172.21.15.40:52002) 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24fad9d640 1 -- 172.21.15.40:0/591260753 learned_addr learned my addr 172.21.15.40:0/591260753 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.771+0000 7f24fa59c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f24fc15f680 0x7f24fc163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49930/0 (socket says 172.21.15.40:49930) 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24fad9d640 1 -- 172.21.15.40:0/591260753 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 msgr2=0x7f24fc0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24fad9d640 1 --2- 172.21.15.40:0/591260753 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24fad9d640 1 -- 172.21.15.40:0/591260753 >> v1:172.21.15.184:6789/0 conn(0x7f24fc15f680 legacy=0x7f24fc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24fad9d640 1 -- 172.21.15.40:0/591260753 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc0b5090 con 0x7f24fc0b60b0 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24f9d9b640 1 --2- 172.21.15.40:0/591260753 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24fad9d640 1 --2- 172.21.15.40:0/591260753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009e50 tx=0x7f24ec02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fd9b9f0bf41171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.772+0000 7f24f959a640 1 -- 172.21.15.40:0/591260753 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec036070 con 0x7f24fc0b60b0 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f24f959a640 1 -- 172.21.15.40:0/591260753 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec031600 con 0x7f24fc0b60b0 2024-04-26T21:18:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f24f959a640 1 -- 172.21.15.40:0/591260753 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0319d0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 -- 172.21.15.40:0/591260753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 msgr2=0x7f24fc0d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 --2- 172.21.15.40:0/591260753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009e50 tx=0x7f24ec02a710 comp rx=0 tx=0).stop 2024-04-26T21:18:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 -- 172.21.15.40:0/591260753 shutdown_connections 2024-04-26T21:18:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 --2- 172.21.15.40:0/591260753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0b60b0 0x7f24fc0d4f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 --2- 172.21.15.40:0/591260753 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0d3820 0x7f24fc0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 -- 172.21.15.40:0/591260753 >> 172.21.15.40:0/591260753 conn(0x7f24fc08eca0 msgr2=0x7f24fc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 -- 172.21.15.40:0/591260753 shutdown_connections 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.773+0000 7f2500a09640 1 -- 172.21.15.40:0/591260753 wait complete. 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 Processor -- start 2024-04-26T21:18:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 -- start start 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3820 0x7f24fc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc15f680 0x7f24fc0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc0d55e0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc0d5460 con 0x7f24fc15f680 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f2500a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc0d5760 con 0x7f24fc0d3820 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f24fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.775+0000 7f24fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39874/0 (socket says 172.21.15.40:39874) 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 -- 172.21.15.40:0/2037905655 learned_addr learned my addr 172.21.15.40:0/2037905655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24f9d9b640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3820 0x7f24fc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fad9d640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc15f680 0x7f24fc0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 -- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3820 msgr2=0x7f24fc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3820 0x7f24fc0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 -- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc15f680 msgr2=0x7f24fc0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc15f680 0x7f24fc0d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 -- 172.21.15.40:0/2037905655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc1623c0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24fa59c640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c3c0 tx=0x7f24f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40074e0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4007680 con 0x7f24fc0b60b0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc1626a0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4004a00 con 0x7f24fc0b60b0 2024-04-26T21:18:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.776+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc0cc3e0 con 0x7f24fc0b60b0 2024-04-26T21:18:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.777+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc15e630 con 0x7f24fc0b60b0 2024-04-26T21:18:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.778+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24f401a020 con 0x7f24fc0b60b0 2024-04-26T21:18:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.778+0000 7f24f2ffd640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 0x7f24dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.778+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f24f408b190 con 0x7f24fc0b60b0 2024-04-26T21:18:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.778+0000 7f24f9d9b640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 0x7f24dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.779+0000 7f24f9d9b640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 0x7f24dc06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24fc1620e0 tx=0x7f24e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:45.783+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24fc15e630 con 0x7f24fc0b60b0 2024-04-26T21:18:46.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.009+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f24fc06a290 con 0x7f24fc0b60b0 2024-04-26T21:18:46.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.020+0000 7f24f2ffd640 1 -- 172.21.15.40:0/2037905655 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) ==== 118+0+0 (secure 0 0 0) 0x7f24f4058f60 con 0x7f24fc0b60b0 2024-04-26T21:18:46.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 msgr2=0x7f24dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 0x7f24dc06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24fc1620e0 tx=0x7f24e4009190 comp rx=0 tx=0).stop 2024-04-26T21:18:46.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 msgr2=0x7f24fc1560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c3c0 tx=0x7f24f400c880 comp rx=0 tx=0).stop 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 shutdown_connections 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f24dc06c8f0 0x7f24dc06edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc15f680 0x7f24fc0d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc0d3820 0x7f24fc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 --2- 172.21.15.40:0/2037905655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24fc0b60b0 0x7f24fc1560a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.022+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 >> 172.21.15.40:0/2037905655 conn(0x7f24fc08eca0 msgr2=0x7f24fc0d49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.023+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 shutdown_connections 2024-04-26T21:18:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.023+0000 7f2500a09640 1 -- 172.21.15.40:0/2037905655 wait complete. 2024-04-26T21:18:46.036 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:18:46.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:18:46.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.138+0000 7f862f87e640 1 Processor -- start 2024-04-26T21:18:46.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.138+0000 7f862f87e640 1 -- start start 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862f87e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f862806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862f87e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86280b3c20 0x7f86280b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862f87e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862809c700 con 0x7f86280b3210 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862f87e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86280b4480 con 0x7f86280b3c20 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862f87e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f862806e050 con 0x7f86280d0010 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862de1b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86280b3210 0x7f862815fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43022/0 (socket says 172.21.15.40:43022) 2024-04-26T21:18:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862de1b640 1 -- 172.21.15.40:0/577736483 learned_addr learned my addr 172.21.15.40:0/577736483 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862ce19640 1 --2- 172.21.15.40:0/577736483 >> v2:172.21.15.184:3300/0 conn(0x7f86280b3c20 0x7f86280b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f862d61a640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f862806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160947673 0 0) 0x7f862809c700 con 0x7f86280b3210 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.139+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8614003520 con 0x7f86280b3210 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3167171049 0 0) 0x7f8614003520 con 0x7f86280b3210 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 msgr2=0x7f862806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f862806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 >> v2:172.21.15.184:3300/0 conn(0x7f86280b3c20 msgr2=0x7f86280b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 --2- 172.21.15.40:0/577736483 >> v2:172.21.15.184:3300/0 conn(0x7f86280b3c20 0x7f86280b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86280d1c50 con 0x7f86280b3210 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f86200019f0 con 0x7f86280b3210 2024-04-26T21:18:46.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 >> v1:172.21.15.40:6789/0 conn(0x7f86280b3210 legacy=0x7f862815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f86140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8614003f30 0x7f8614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8614004d40 0x7f86140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86280b4480 con 0x7f8614004d40 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f862806e050 con 0x7f8614003f30 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f861400b9c0 con 0x7f86280d0010 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8614004d40 0x7f86140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862d61a640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f86140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862ce19640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8614003f30 0x7f8614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 -- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 msgr2=0x7f86140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f86140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 -- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8614003f30 msgr2=0x7f8614004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8614003f30 0x7f8614004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862de1b640 1 -- 172.21.15.40:0/577736483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8614003520 con 0x7f8614004d40 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862d61a640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86280d0010 0x7f86140039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.140+0000 7f862ce19640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8614003f30 0x7f8614004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.141+0000 7f862de1b640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8614004d40 0x7f86140051c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f86200013d0 tx=0x7f8614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0720b975cfc617 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.141+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000a4e0 con 0x7f8614004d40 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.141+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862002f070 con 0x7f8614004d40 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.141+0000 7f861ffff640 1 -- 172.21.15.40:0/577736483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862002bcd0 con 0x7f8614004d40 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 -- 172.21.15.40:0/577736483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8614004d40 msgr2=0x7f86140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 --2- 172.21.15.40:0/577736483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8614004d40 0x7f86140051c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f86200013d0 tx=0x7f8614005830 comp rx=0 tx=0).stop 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862de1b640 1 -- 172.21.15.40:0/577736483 reap_dead start 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 -- 172.21.15.40:0/577736483 shutdown_connections 2024-04-26T21:18:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 -- 172.21.15.40:0/577736483 >> 172.21.15.40:0/577736483 conn(0x7f862809ba40 msgr2=0x7f862809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:46.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 -- 172.21.15.40:0/577736483 shutdown_connections 2024-04-26T21:18:46.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.142+0000 7f862f87e640 1 -- 172.21.15.40:0/577736483 wait complete. 2024-04-26T21:18:46.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.143+0000 7f862f87e640 1 Processor -- start 2024-04-26T21:18:46.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.143+0000 7f862f87e640 1 -- start start 2024-04-26T21:18:46.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 0x7f86281585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f862814a800 0x7f8628158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f862806dc90 con 0x7f8628157e20 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f862806db10 con 0x7f862814a800 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f862806de10 con 0x7f8628156560 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39896/0 (socket says 172.21.15.40:39896) 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 -- 172.21.15.40:0/4015428702 learned_addr learned my addr 172.21.15.40:0/4015428702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862de1b640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f862814a800 0x7f8628158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862ce19640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 0x7f86281585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 -- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 msgr2=0x7f86281585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 0x7f86281585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 -- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f862814a800 msgr2=0x7f8628158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:46.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f862814a800 0x7f8628158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 -- 172.21.15.40:0/4015428702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862815adb0 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862ce19640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 0x7f86281585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862d61a640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8624011cd0 tx=0x7f862400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400fd20 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8624014440 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.144+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862815b090 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.145+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624014830 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.145+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862815b650 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.146+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86280d07f0 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.146+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8624012040 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.147+0000 7f861dffb640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 0x7f860c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.147+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f862408c2a0 con 0x7f8628157e20 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.147+0000 7f862ce19640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 0x7f860c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:46.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.148+0000 7f862ce19640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 0x7f860c06ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86280cf720 tx=0x7f8618008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:46.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.151+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8624019030 con 0x7f8628157e20 2024-04-26T21:18:46.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:46.378+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f862806a290 con 0x7f8628157e20 2024-04-26T21:18:47.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.025+0000 7f861dffb640 1 -- 172.21.15.40:0/4015428702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) ==== 122+0+0 (secure 0 0 0) 0x7f862405a6d0 con 0x7f8628157e20 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 msgr2=0x7f860c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 0x7f860c06ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86280cf720 tx=0x7f8618008040 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 msgr2=0x7f862815a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8624011cd0 tx=0x7f862400f610 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 shutdown_connections 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f860c06c920 0x7f860c06ede0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f862814a800 0x7f8628158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8628156560 0x7f86281585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.028+0000 7f862f87e640 1 --2- 172.21.15.40:0/4015428702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8628157e20 0x7f862815a540 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.029+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 >> 172.21.15.40:0/4015428702 conn(0x7f862809ba40 msgr2=0x7f862806e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.029+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 shutdown_connections 2024-04-26T21:18:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.029+0000 7f862f87e640 1 -- 172.21.15.40:0/4015428702 wait complete. 2024-04-26T21:18:47.045 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:18:47.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.151+0000 7f8a408b8640 1 Processor -- start 2024-04-26T21:18:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.151+0000 7f8a408b8640 1 -- start start 2024-04-26T21:18:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a408b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0cf2e0 0x7f8a3c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a408b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a3c06e490 0x7f8a3c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a408b8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c0d1740 con 0x7f8a3c0d11b0 2024-04-26T21:18:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a408b8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3c0d1a20 con 0x7f8a3c06e490 2024-04-26T21:18:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a408b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c0d1d00 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a3ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a3c06e490 0x7f8a3c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a3ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a3c06e490 0x7f8a3c15ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39908/0 (socket says 172.21.15.40:39908) 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.152+0000 7f8a3ad9d640 1 -- 172.21.15.40:0/1485717839 learned_addr learned my addr 172.21.15.40:0/1485717839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3ad9d640 1 -- 172.21.15.40:0/1485717839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0cf2e0 msgr2=0x7f8a3c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3ad9d640 1 --2- 172.21.15.40:0/1485717839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0cf2e0 0x7f8a3c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3ad9d640 1 -- 172.21.15.40:0/1485717839 >> v1:172.21.15.184:6789/0 conn(0x7f8a3c0d11b0 legacy=0x7f8a3c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3ad9d640 1 -- 172.21.15.40:0/1485717839 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c0d1fe0 con 0x7f8a3c06e490 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3ad9d640 1 --2- 172.21.15.40:0/1485717839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c06e490 0x7f8a3c15ebe0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009e50 tx=0x7f8a3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13079fbe672ae52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.153+0000 7f8a3959a640 1 -- 172.21.15.40:0/1485717839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34036070 con 0x7f8a3c06e490 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a3959a640 1 -- 172.21.15.40:0/1485717839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a34031600 con 0x7f8a3c06e490 2024-04-26T21:18:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a3959a640 1 -- 172.21.15.40:0/1485717839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340319d0 con 0x7f8a3c06e490 2024-04-26T21:18:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 -- 172.21.15.40:0/1485717839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c06e490 msgr2=0x7f8a3c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 --2- 172.21.15.40:0/1485717839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c06e490 0x7f8a3c15ebe0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009e50 tx=0x7f8a3402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 -- 172.21.15.40:0/1485717839 shutdown_connections 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 --2- 172.21.15.40:0/1485717839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c06e490 0x7f8a3c15ebe0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 --2- 172.21.15.40:0/1485717839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0cf2e0 0x7f8a3c06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 -- 172.21.15.40:0/1485717839 >> 172.21.15.40:0/1485717839 conn(0x7f8a3c0b4a60 msgr2=0x7f8a3c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 -- 172.21.15.40:0/1485717839 shutdown_connections 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.154+0000 7f8a408b8640 1 -- 172.21.15.40:0/1485717839 wait complete. 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.155+0000 7f8a408b8640 1 Processor -- start 2024-04-26T21:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 -- start start 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3c06e490 0x7f8a3c1e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0d11b0 0x7f8a3c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0cf6c0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c06da10 con 0x7f8a3c06e490 2024-04-26T21:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a408b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c0cf840 con 0x7f8a3c0d11b0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a39d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a39d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39914/0 (socket says 172.21.15.40:39914) 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.156+0000 7f8a39d9b640 1 -- 172.21.15.40:0/3972907010 learned_addr learned my addr 172.21.15.40:0/3972907010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a3a59c640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3c06e490 0x7f8a3c1e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a3ad9d640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0d11b0 0x7f8a3c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 -- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0d11b0 msgr2=0x7f8a3c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0d11b0 0x7f8a3c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 -- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3c06e490 msgr2=0x7f8a3c1e92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3c06e490 0x7f8a3c1e92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 -- 172.21.15.40:0/3972907010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1f56a0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a39d9b640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c460 tx=0x7f8a3000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30011790 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a30017070 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30011bb0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c14d270 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.157+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c0ca320 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.158+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c06a290 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.159+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a3001c460 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.159+0000 7f8a2affd640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 0x7f8a1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.159+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f8a3008b910 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.160+0000 7f8a3a59c640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 0x7f8a1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.160+0000 7f8a3a59c640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 0x7f8a1c06ede0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0098e0 tx=0x7f8a2c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.164+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a30059cb0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.387+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a3c0b8ef0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.387+0000 7f8a2affd640 1 -- 172.21.15.40:0/3972907010 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7f8a300595b0 con 0x7f8a3c0cf2e0 2024-04-26T21:18:47.390 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:47.390 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:18:47.390 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.389+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 msgr2=0x7f8a1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.389+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 0x7f8a1c06ede0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0098e0 tx=0x7f8a2c009060 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.389+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 msgr2=0x7f8a3c14cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.389+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c460 tx=0x7f8a3000c920 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 shutdown_connections 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a1c06c920 0x7f8a1c06ede0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a3c0d11b0 0x7f8a3c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a3c0cf2e0 0x7f8a3c14cd30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 --2- 172.21.15.40:0/3972907010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3c06e490 0x7f8a3c1e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 >> 172.21.15.40:0/3972907010 conn(0x7f8a3c0b4a60 msgr2=0x7f8a3c162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 shutdown_connections 2024-04-26T21:18:47.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.390+0000 7f8a408b8640 1 -- 172.21.15.40:0/3972907010 wait complete. 2024-04-26T21:18:47.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.501+0000 7f638a285640 1 Processor -- start 2024-04-26T21:18:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.501+0000 7f638a285640 1 -- start start 2024-04-26T21:18:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f638a285640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840742c0 0x7f638415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f638a285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 0x7f638415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f638a285640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638415e2e0 con 0x7f6384146980 2024-04-26T21:18:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f638a285640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f638415e5c0 con 0x7f63840742c0 2024-04-26T21:18:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f638a285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63841613c0 con 0x7f6384161850 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840742c0 0x7f638415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840742c0 0x7f638415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39922/0 (socket says 172.21.15.40:39922) 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 -- 172.21.15.40:0/2034312427 learned_addr learned my addr 172.21.15.40:0/2034312427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 -- 172.21.15.40:0/2034312427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 msgr2=0x7f638415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f6388822640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 0x7f638415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 0x7f638415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 -- 172.21.15.40:0/2034312427 >> v1:172.21.15.184:6789/0 conn(0x7f6384146980 legacy=0x7f6384073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f63837fe640 1 -- 172.21.15.40:0/2034312427 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6384160d70 con 0x7f63840742c0 2024-04-26T21:18:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.502+0000 7f6388822640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 0x7f638415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.503+0000 7f63837fe640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415d6b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6378009870 tx=0x7f637802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5a0e373350ec6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.503+0000 7f6382ffd640 1 -- 172.21.15.40:0/2034312427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378037070 con 0x7f63840742c0 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.503+0000 7f6382ffd640 1 -- 172.21.15.40:0/2034312427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6378032440 con 0x7f63840742c0 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.503+0000 7f6382ffd640 1 -- 172.21.15.40:0/2034312427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378032810 con 0x7f63840742c0 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 -- 172.21.15.40:0/2034312427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 msgr2=0x7f638415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415d6b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6378009870 tx=0x7f637802a210 comp rx=0 tx=0).stop 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 -- 172.21.15.40:0/2034312427 shutdown_connections 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384161850 0x7f638415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 --2- 172.21.15.40:0/2034312427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415d6b0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 -- 172.21.15.40:0/2034312427 >> 172.21.15.40:0/2034312427 conn(0x7f63840ce8b0 msgr2=0x7f63840cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 -- 172.21.15.40:0/2034312427 shutdown_connections 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.504+0000 7f638a285640 1 -- 172.21.15.40:0/2034312427 wait complete. 2024-04-26T21:18:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.505+0000 7f638a285640 1 Processor -- start 2024-04-26T21:18:47.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.505+0000 7f638a285640 1 -- start start 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 0x7f638413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 0x7f63840b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6384146d60 con 0x7f63840742c0 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63840b5120 con 0x7f6384161850 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f638a285640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63840b6a50 con 0x7f6384146980 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39926/0 (socket says 172.21.15.40:39926) 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 -- 172.21.15.40:0/628443463 learned_addr learned my addr 172.21.15.40:0/628443463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6388822640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 0x7f63840b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 -- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 msgr2=0x7f638413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f63837fe640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 0x7f638413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 0x7f638413cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 -- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 msgr2=0x7f63840b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 0x7f63840b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 -- 172.21.15.40:0/628443463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638415ce40 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f63837fe640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 0x7f638413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6388822640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 0x7f63840b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.506+0000 7f6383fff640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f637400c430 tx=0x7f637400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.507+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374011790 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.507+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6374017070 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.507+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374011bb0 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.507+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63841f30d0 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.507+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63841f3690 con 0x7f63840742c0 2024-04-26T21:18:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.508+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638406a290 con 0x7f63840742c0 2024-04-26T21:18:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.508+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f637401c460 con 0x7f63840742c0 2024-04-26T21:18:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.508+0000 7f6380ff9640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 0x7f637006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.509+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f637408b9f0 con 0x7f63840742c0 2024-04-26T21:18:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.509+0000 7f63837fe640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 0x7f637006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.509+0000 7f63837fe640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 0x7f637006ede0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6378034000 tx=0x7f637802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.513+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6374059e20 con 0x7f63840742c0 2024-04-26T21:18:47.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.697+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6384145170 con 0x7f63840742c0 2024-04-26T21:18:47.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.698+0000 7f6380ff9640 1 -- 172.21.15.40:0/628443463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7f6374059720 con 0x7f63840742c0 2024-04-26T21:18:47.700 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:47.700 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 msgr2=0x7f637006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 0x7f637006ede0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6378034000 tx=0x7f637802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 msgr2=0x7f638415c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f637400c430 tx=0x7f637400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 shutdown_connections 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c920 0x7f637006ede0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6384161850 0x7f63840b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6384146980 0x7f638413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 --2- 172.21.15.40:0/628443463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840742c0 0x7f638415c900 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 >> 172.21.15.40:0/628443463 conn(0x7f63840ce8b0 msgr2=0x7f638415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 shutdown_connections 2024-04-26T21:18:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.700+0000 7f638a285640 1 -- 172.21.15.40:0/628443463 wait complete. 2024-04-26T21:18:47.713 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:18:47.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:47.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.812+0000 7f36b3da4640 1 Processor -- start 2024-04-26T21:18:47.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.812+0000 7f36b3da4640 1 -- start start 2024-04-26T21:18:47.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b3da4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b3da4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b3da4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ac0d1740 con 0x7f36ac06e2c0 2024-04-26T21:18:47.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b3da4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac06e750 con 0x7f36ac0cf2e0 2024-04-26T21:18:47.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b3da4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0d1a20 con 0x7f36ac0d11b0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.813+0000 7f36b1b40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b1b40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:52050/0 (socket says 172.21.15.40:52050) 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:57670/0 (socket says 172.21.15.40:57670) 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b1b40640 1 -- 172.21.15.40:0/2785136256 learned_addr learned my addr 172.21.15.40:0/2785136256 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b2341640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f36ac06e2c0 0x7f36ac15ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:49958/0 (socket says 172.21.15.40:49958) 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b0b3e640 1 -- 172.21.15.40:0/2785136256 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132486125 0 0) 0x7f36ac0d1740 con 0x7f36ac06e2c0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b0b3e640 1 -- 172.21.15.40:0/2785136256 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3694003520 con 0x7f36ac06e2c0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 -- 172.21.15.40:0/2785136256 >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 msgr2=0x7f36ac06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 --2- 172.21.15.40:0/2785136256 >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 -- 172.21.15.40:0/2785136256 >> v1:172.21.15.184:6789/0 conn(0x7f36ac06e2c0 legacy=0x7f36ac15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b133f640 1 -- 172.21.15.40:0/2785136256 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac0d1d00 con 0x7f36ac0d11b0 2024-04-26T21:18:47.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.814+0000 7f36b1b40640 1 --2- 172.21.15.40:0/2785136256 >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:47.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b133f640 1 --2- 172.21.15.40:0/2785136256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097a0 tx=0x7f36a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b59c7d37937790 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b0b3e640 1 -- 172.21.15.40:0/2785136256 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0037070 con 0x7f36ac0d11b0 2024-04-26T21:18:47.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b0b3e640 1 -- 172.21.15.40:0/2785136256 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0032440 con 0x7f36ac0d11b0 2024-04-26T21:18:47.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b0b3e640 1 -- 172.21.15.40:0/2785136256 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a00327f0 con 0x7f36ac0d11b0 2024-04-26T21:18:47.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 -- 172.21.15.40:0/2785136256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 msgr2=0x7f36ac06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 --2- 172.21.15.40:0/2785136256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097a0 tx=0x7f36a002a180 comp rx=0 tx=0).stop 2024-04-26T21:18:47.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 -- 172.21.15.40:0/2785136256 shutdown_connections 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 --2- 172.21.15.40:0/2785136256 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0d11b0 0x7f36ac06dd80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 --2- 172.21.15.40:0/2785136256 >> v2:172.21.15.184:3300/0 conn(0x7f36ac0cf2e0 0x7f36ac06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.815+0000 7f36b3da4640 1 -- 172.21.15.40:0/2785136256 >> 172.21.15.40:0/2785136256 conn(0x7f36ac0b4a60 msgr2=0x7f36ac0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.816+0000 7f36b3da4640 1 -- 172.21.15.40:0/2785136256 shutdown_connections 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.816+0000 7f36b3da4640 1 -- 172.21.15.40:0/2785136256 wait complete. 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 Processor -- start 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 -- start start 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac06e2c0 0x7f36ac14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0cf2e0 0x7f36ac152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac06da10 con 0x7f36ac0d11b0 2024-04-26T21:18:47.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0d22c0 con 0x7f36ac06e2c0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.817+0000 7f36b3da4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0cf6c0 con 0x7f36ac0cf2e0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39942/0 (socket says 172.21.15.40:39942) 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 -- 172.21.15.40:0/58209633 learned_addr learned my addr 172.21.15.40:0/58209633 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b1b40640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac06e2c0 0x7f36ac14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b133f640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0cf2e0 0x7f36ac152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 -- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0cf2e0 msgr2=0x7f36ac152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0cf2e0 0x7f36ac152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 -- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac06e2c0 msgr2=0x7f36ac14ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac06e2c0 0x7f36ac14ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 -- 172.21.15.40:0/58209633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac0a07b0 con 0x7f36ac0d11b0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b2341640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c460 tx=0x7f36a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4018070 con 0x7f36ac0d11b0 2024-04-26T21:18:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a4007480 con 0x7f36ac0d11b0 2024-04-26T21:18:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40047e0 con 0x7f36ac0d11b0 2024-04-26T21:18:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac0a0a70 con 0x7f36ac0d11b0 2024-04-26T21:18:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.818+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac1506e0 con 0x7f36ac0d11b0 2024-04-26T21:18:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.819+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac06a290 con 0x7f36ac0d11b0 2024-04-26T21:18:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.820+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36a401b070 con 0x7f36ac0d11b0 2024-04-26T21:18:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.820+0000 7f36aa7fc640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.820+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f36a408ad40 con 0x7f36ac0d11b0 2024-04-26T21:18:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.820+0000 7f36b1b40640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.821+0000 7f36b1b40640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 0x7f369806edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f369c0072c0 tx=0x7f369c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:47.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:47.824+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36a4058b20 con 0x7f36ac0d11b0 2024-04-26T21:18:48.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.005+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36ac0b8ef0 con 0x7f369806c8f0 2024-04-26T21:18:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.011+0000 7f36aa7fc640 1 -- 172.21.15.40:0/58209633 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238413 (secure 0 0 0) 0x7f36ac0b8ef0 con 0x7f369806c8f0 2024-04-26T21:18:48.016 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:48.017 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 msgr2=0x7f369806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 0x7f369806edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f369c0072c0 tx=0x7f369c00b040 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 msgr2=0x7f36ac0b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c460 tx=0x7f36a400c920 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 shutdown_connections 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f369806c8f0 0x7f369806edb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36ac0d11b0 0x7f36ac0b6d80 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36ac0cf2e0 0x7f36ac152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 --2- 172.21.15.40:0/58209633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ac06e2c0 0x7f36ac14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 >> 172.21.15.40:0/58209633 conn(0x7f36ac0b4a60 msgr2=0x7f36ac0c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 shutdown_connections 2024-04-26T21:18:48.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.014+0000 7f36b3da4640 1 -- 172.21.15.40:0/58209633 wait complete. 2024-04-26T21:18:48.027 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-26T21:18:47.016512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":263720,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711000,"statfs":{"total":773094113280,"available":772824064000,"internally_reserved":0,"allocated":10342400,"data_stored":5049077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016202"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018917+0000","last_change":"2024-04-26T21:18:44.018917+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018917+0000","last_undegraded":"2024-04-26T21:18:44.018917+0000","last_fullsized":"2024-04-26T21:18:44.018917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015819+0000","last_change":"2024-04-26T21:18:44.015819+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015819+0000","last_undegraded":"2024-04-26T21:18:44.015819+0000","last_fullsized":"2024-04-26T21:18:44.015819+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016468+0000","last_change":"2024-04-26T21:18:44.016468+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016468+0000","last_undegraded":"2024-04-26T21:18:44.016468+0000","last_fullsized":"2024-04-26T21:18:44.016468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018687+0000","last_change":"2024-04-26T21:18:44.018687+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018687+0000","last_undegraded":"2024-04-26T21:18:44.018687+0000","last_fullsized":"2024-04-26T21:18:44.018687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029601+0000","last_change":"2024-04-26T21:18:44.024416+0000","last_active":"2024-04-26T21:18:44.029601+0000","last_peered":"2024-04-26T21:18:44.029601+0000","last_clean":"2024-04-26T21:18:44.029601+0000","last_became_active":"2024-04-26T21:18:44.024129+0000","last_became_peered":"2024-04-26T21:18:44.024129+0000","last_unstale":"2024-04-26T21:18:44.029601+0000","last_undegraded":"2024-04-26T21:18:44.029601+0000","last_fullsized":"2024-04-26T21:18:44.029601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:11.530320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029700+0000","last_change":"2024-04-26T21:18:44.025572+0000","last_active":"2024-04-26T21:18:44.029700+0000","last_peered":"2024-04-26T21:18:44.029700+0000","last_clean":"2024-04-26T21:18:44.029700+0000","last_became_active":"2024-04-26T21:18:44.025260+0000","last_became_peered":"2024-04-26T21:18:44.025260+0000","last_unstale":"2024-04-26T21:18:44.029700+0000","last_undegraded":"2024-04-26T21:18:44.029700+0000","last_fullsized":"2024-04-26T21:18:44.029700+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:57.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029675+0000","last_change":"2024-04-26T21:18:44.025032+0000","last_active":"2024-04-26T21:18:44.029675+0000","last_peered":"2024-04-26T21:18:44.029675+0000","last_clean":"2024-04-26T21:18:44.029675+0000","last_became_active":"2024-04-26T21:18:44.024732+0000","last_became_peered":"2024-04-26T21:18:44.024732+0000","last_unstale":"2024-04-26T21:18:44.029675+0000","last_undegraded":"2024-04-26T21:18:44.029675+0000","last_fullsized":"2024-04-26T21:18:44.029675+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:41.641506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015981+0000","last_change":"2024-04-26T21:18:44.015981+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015981+0000","last_undegraded":"2024-04-26T21:18:44.015981+0000","last_fullsized":"2024-04-26T21:18:44.015981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.031512+0000","last_change":"2024-04-26T21:18:44.028840+0000","last_active":"2024-04-26T21:18:44.031512+0000","last_peered":"2024-04-26T21:18:44.031512+0000","last_clean":"2024-04-26T21:18:44.031512+0000","last_became_active":"2024-04-26T21:18:44.028560+0000","last_became_peered":"2024-04-26T21:18:44.028560+0000","last_unstale":"2024-04-26T21:18:44.031512+0000","last_undegraded":"2024-04-26T21:18:44.031512+0000","last_fullsized":"2024-04-26T21:18:44.031512+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:00.234414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016765+0000","last_change":"2024-04-26T21:18:44.016765+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016765+0000","last_undegraded":"2024-04-26T21:18:44.016765+0000","last_fullsized":"2024-04-26T21:18:44.016765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016771+0000","last_change":"2024-04-26T21:18:44.016771+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016771+0000","last_undegraded":"2024-04-26T21:18:44.016771+0000","last_fullsized":"2024-04-26T21:18:44.016771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":297,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T21:18:42.244752+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:18:42.244752+0000","last_peered":"2024-04-26T21:18:42.244752+0000","last_clean":"2024-04-26T21:18:42.244752+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:18:42.244752+0000","last_undegraded":"2024-04-26T21:18:42.244752+0000","last_fullsized":"2024-04-26T21:18:42.244752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509527,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1067913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1073040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":494764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:48.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:48.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.165+0000 7f2c4a22a640 1 Processor -- start 2024-04-26T21:18:48.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.165+0000 7f2c4a22a640 1 -- start start 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c4a22a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c440d3ee0 0x7f2c440cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c4a22a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c4415f750 0x7f2c440b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c4a22a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c440d03a0 con 0x7f2c4415f370 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c4a22a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2c440b4d10 con 0x7f2c4415f750 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c4a22a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c440d0680 con 0x7f2c440d3ee0 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c43fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c4415f750 0x7f2c440b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c437fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2c4415f370 0x7f2c440d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43024/0 (socket says 172.21.15.40:43024) 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c43fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c4415f750 0x7f2c440b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39948/0 (socket says 172.21.15.40:39948) 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.166+0000 7f2c437fe640 1 -- 172.21.15.40:0/1073976578 learned_addr learned my addr 172.21.15.40:0/1073976578 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c43fff640 1 -- 172.21.15.40:0/1073976578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c440d3ee0 msgr2=0x7f2c440cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c43fff640 1 --2- 172.21.15.40:0/1073976578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c440d3ee0 0x7f2c440cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c43fff640 1 -- 172.21.15.40:0/1073976578 >> v1:172.21.15.40:6789/0 conn(0x7f2c4415f370 legacy=0x7f2c440d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c43fff640 1 -- 172.21.15.40:0/1073976578 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c440d0960 con 0x7f2c4415f750 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c427fc640 1 -- 172.21.15.40:0/1073976578 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191084194 0 0) 0x7f2c440d03a0 con 0x7f2c4415f370 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.167+0000 7f2c43fff640 1 --2- 172.21.15.40:0/1073976578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c4415f750 0x7f2c440b47d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009d90 tx=0x7f2c3c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fbe6bb4de30b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c427fc640 1 -- 172.21.15.40:0/1073976578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007740 con 0x7f2c4415f750 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c427fc640 1 -- 172.21.15.40:0/1073976578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0078e0 con 0x7f2c4415f750 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c427fc640 1 -- 172.21.15.40:0/1073976578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007cb0 con 0x7f2c4415f750 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1073976578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c4415f750 msgr2=0x7f2c440b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1073976578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c4415f750 0x7f2c440b47d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009d90 tx=0x7f2c3c02a660 comp rx=0 tx=0).stop 2024-04-26T21:18:48.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1073976578 shutdown_connections 2024-04-26T21:18:48.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1073976578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c4415f750 0x7f2c440b47d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1073976578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c440d3ee0 0x7f2c440cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1073976578 >> 172.21.15.40:0/1073976578 conn(0x7f2c4408ec10 msgr2=0x7f2c4409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1073976578 shutdown_connections 2024-04-26T21:18:48.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.168+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1073976578 wait complete. 2024-04-26T21:18:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.169+0000 7f2c4a22a640 1 Processor -- start 2024-04-26T21:18:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 -- start start 2024-04-26T21:18:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4415f370 0x7f2c440c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c4415f750 0x7f2c4415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440b51c0 con 0x7f2c440d3ee0 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440d0f20 con 0x7f2c4415f750 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c4a22a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2c440b5340 con 0x7f2c4415f370 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39964/0 (socket says 172.21.15.40:39964) 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.170+0000 7f2c437fe640 1 -- 172.21.15.40:0/1537549546 learned_addr learned my addr 172.21.15.40:0/1537549546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 -- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4415f370 msgr2=0x7f2c440c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4415f370 0x7f2c440c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 -- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c4415f750 msgr2=0x7f2c4415c870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c4415f750 0x7f2c4415c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 -- 172.21.15.40:0/1537549546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c440b8040 con 0x7f2c440d3ee0 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c437fe640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c500 tx=0x7f2c3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34018070 con 0x7f2c440d3ee0 2024-04-26T21:18:48.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34007580 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c440b8320 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34004a20 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.171+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c440b88e0 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.172+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4406a290 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.172+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c3401b070 con 0x7f2c440d3ee0 2024-04-26T21:18:48.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.173+0000 7f2c2bfff640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 0x7f2c3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.173+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f2c3408ae00 con 0x7f2c440d3ee0 2024-04-26T21:18:48.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.174+0000 7f2c42ffd640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 0x7f2c3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.175+0000 7f2c42ffd640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 0x7f2c3006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c440cbca0 tx=0x7f2c38006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.177+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c340592c0 con 0x7f2c440d3ee0 2024-04-26T21:18:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.360+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c440d3900 con 0x7f2c3006c8f0 2024-04-26T21:18:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.366+0000 7f2c2bfff640 1 -- 172.21.15.40:0/1537549546 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238413 (secure 0 0 0) 0x7f2c440d3900 con 0x7f2c3006c8f0 2024-04-26T21:18:48.370 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:18:48.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 msgr2=0x7f2c3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 0x7f2c3006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c440cbca0 tx=0x7f2c38006c90 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 msgr2=0x7f2c440cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c500 tx=0x7f2c3400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 shutdown_connections 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2c3006c8f0 0x7f2c3006edb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c4415f750 0x7f2c4415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c4415f370 0x7f2c440c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 --2- 172.21.15.40:0/1537549546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c440d3ee0 0x7f2c440cc960 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 >> 172.21.15.40:0/1537549546 conn(0x7f2c4408ec10 msgr2=0x7f2c4415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 shutdown_connections 2024-04-26T21:18:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.368+0000 7f2c4a22a640 1 -- 172.21.15.40:0/1537549546 wait complete. 2024-04-26T21:18:48.381 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-26T21:18:47.016512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":263720,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711000,"statfs":{"total":773094113280,"available":772824064000,"internally_reserved":0,"allocated":10342400,"data_stored":5049077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016202"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018917+0000","last_change":"2024-04-26T21:18:44.018917+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018917+0000","last_undegraded":"2024-04-26T21:18:44.018917+0000","last_fullsized":"2024-04-26T21:18:44.018917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015819+0000","last_change":"2024-04-26T21:18:44.015819+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015819+0000","last_undegraded":"2024-04-26T21:18:44.015819+0000","last_fullsized":"2024-04-26T21:18:44.015819+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016468+0000","last_change":"2024-04-26T21:18:44.016468+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016468+0000","last_undegraded":"2024-04-26T21:18:44.016468+0000","last_fullsized":"2024-04-26T21:18:44.016468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018687+0000","last_change":"2024-04-26T21:18:44.018687+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018687+0000","last_undegraded":"2024-04-26T21:18:44.018687+0000","last_fullsized":"2024-04-26T21:18:44.018687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029601+0000","last_change":"2024-04-26T21:18:44.024416+0000","last_active":"2024-04-26T21:18:44.029601+0000","last_peered":"2024-04-26T21:18:44.029601+0000","last_clean":"2024-04-26T21:18:44.029601+0000","last_became_active":"2024-04-26T21:18:44.024129+0000","last_became_peered":"2024-04-26T21:18:44.024129+0000","last_unstale":"2024-04-26T21:18:44.029601+0000","last_undegraded":"2024-04-26T21:18:44.029601+0000","last_fullsized":"2024-04-26T21:18:44.029601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:11.530320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029700+0000","last_change":"2024-04-26T21:18:44.025572+0000","last_active":"2024-04-26T21:18:44.029700+0000","last_peered":"2024-04-26T21:18:44.029700+0000","last_clean":"2024-04-26T21:18:44.029700+0000","last_became_active":"2024-04-26T21:18:44.025260+0000","last_became_peered":"2024-04-26T21:18:44.025260+0000","last_unstale":"2024-04-26T21:18:44.029700+0000","last_undegraded":"2024-04-26T21:18:44.029700+0000","last_fullsized":"2024-04-26T21:18:44.029700+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:57.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029675+0000","last_change":"2024-04-26T21:18:44.025032+0000","last_active":"2024-04-26T21:18:44.029675+0000","last_peered":"2024-04-26T21:18:44.029675+0000","last_clean":"2024-04-26T21:18:44.029675+0000","last_became_active":"2024-04-26T21:18:44.024732+0000","last_became_peered":"2024-04-26T21:18:44.024732+0000","last_unstale":"2024-04-26T21:18:44.029675+0000","last_undegraded":"2024-04-26T21:18:44.029675+0000","last_fullsized":"2024-04-26T21:18:44.029675+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:41.641506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015981+0000","last_change":"2024-04-26T21:18:44.015981+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015981+0000","last_undegraded":"2024-04-26T21:18:44.015981+0000","last_fullsized":"2024-04-26T21:18:44.015981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.031512+0000","last_change":"2024-04-26T21:18:44.028840+0000","last_active":"2024-04-26T21:18:44.031512+0000","last_peered":"2024-04-26T21:18:44.031512+0000","last_clean":"2024-04-26T21:18:44.031512+0000","last_became_active":"2024-04-26T21:18:44.028560+0000","last_became_peered":"2024-04-26T21:18:44.028560+0000","last_unstale":"2024-04-26T21:18:44.031512+0000","last_undegraded":"2024-04-26T21:18:44.031512+0000","last_fullsized":"2024-04-26T21:18:44.031512+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:00.234414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016765+0000","last_change":"2024-04-26T21:18:44.016765+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016765+0000","last_undegraded":"2024-04-26T21:18:44.016765+0000","last_fullsized":"2024-04-26T21:18:44.016765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016771+0000","last_change":"2024-04-26T21:18:44.016771+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016771+0000","last_undegraded":"2024-04-26T21:18:44.016771+0000","last_fullsized":"2024-04-26T21:18:44.016771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":297,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T21:18:42.244752+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:18:42.244752+0000","last_peered":"2024-04-26T21:18:42.244752+0000","last_clean":"2024-04-26T21:18:42.244752+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:18:42.244752+0000","last_undegraded":"2024-04-26T21:18:42.244752+0000","last_fullsized":"2024-04-26T21:18:42.244752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509527,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1067913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1073040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":494764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:48.385 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.520+0000 7f27a735f640 1 Processor -- start 2024-04-26T21:18:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.520+0000 7f27a735f640 1 -- start start 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a735f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f27a0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a735f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f27a015b8c0 0x7f27a00cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a735f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a00733e0 con 0x7f27a015b4e0 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a735f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f27a00ce230 con 0x7f27a015b8c0 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a735f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f27a0072f00 con 0x7f27a006f2c0 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a50fb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27a015b4e0 0x7f27a006ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43026/0 (socket says 172.21.15.40:43026) 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a50fb640 1 -- 172.21.15.40:0/41672522 learned_addr learned my addr 172.21.15.40:0/41672522 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a48fa640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f27a0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> v2:172.21.15.184:3300/0 conn(0x7f27a015b8c0 0x7f27a00cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320090814 0 0) 0x7f27a00733e0 con 0x7f27a015b4e0 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.521+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2788003520 con 0x7f27a015b4e0 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2829463694 0 0) 0x7f2788003520 con 0x7f27a015b4e0 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 msgr2=0x7f27a0071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f27a0071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 >> v2:172.21.15.184:3300/0 conn(0x7f27a015b8c0 msgr2=0x7f27a00cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 --2- 172.21.15.40:0/41672522 >> v2:172.21.15.184:3300/0 conn(0x7f27a015b8c0 0x7f27a00cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0072660 con 0x7f27a015b4e0 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f2798001670 con 0x7f27a015b4e0 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> v2:172.21.15.184:3300/0 conn(0x7f27a015b8c0 0x7f27a00cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 >> v1:172.21.15.40:6789/0 conn(0x7f27a015b4e0 legacy=0x7f27a006ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27880039b0 0x7f2788003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f2788004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2788004d40 0x7f27880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00ce230 con 0x7f2788004d40 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0072f00 con 0x7f27880039b0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f278800b9c0 con 0x7f27a006f2c0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2788004d40 0x7f27880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a50fb640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27880039b0 0x7f2788003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a48fa640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f2788004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 -- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 msgr2=0x7f2788004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a006f2c0 0x7f2788004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 -- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27880039b0 msgr2=0x7f2788003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27880039b0 0x7f2788003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a58fc640 1 -- 172.21.15.40:0/41672522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2788003520 con 0x7f2788004d40 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.522+0000 7f27a50fb640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27880039b0 0x7f2788003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.523+0000 7f27a58fc640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2788004d40 0x7f27880051c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b6e70 tx=0x7f2788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48002ee1680c4223 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.523+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790010160 con 0x7f2788004d40 2024-04-26T21:18:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.523+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2790003b90 con 0x7f2788004d40 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.523+0000 7f279ffff640 1 -- 172.21.15.40:0/41672522 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790004020 con 0x7f2788004d40 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 -- 172.21.15.40:0/41672522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2788004d40 msgr2=0x7f27880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 --2- 172.21.15.40:0/41672522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2788004d40 0x7f27880051c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b6e70 tx=0x7f2788005830 comp rx=0 tx=0).stop 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a58fc640 1 -- 172.21.15.40:0/41672522 reap_dead start 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 -- 172.21.15.40:0/41672522 shutdown_connections 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 -- 172.21.15.40:0/41672522 >> 172.21.15.40:0/41672522 conn(0x7f27a00d0d30 msgr2=0x7f27a00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 -- 172.21.15.40:0/41672522 shutdown_connections 2024-04-26T21:18:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.524+0000 7f27a735f640 1 -- 172.21.15.40:0/41672522 wait complete. 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.525+0000 7f27a735f640 1 Processor -- start 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.525+0000 7f27a735f640 1 -- start start 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a01416b0 0x7f27a0141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 0x7f27a0151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00b3e80 con 0x7f27a0142480 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0072c20 con 0x7f27a015ea00 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f27a0073820 con 0x7f27a01416b0 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39982/0 (socket says 172.21.15.40:39982) 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 -- 172.21.15.40:0/1839183141 learned_addr learned my addr 172.21.15.40:0/1839183141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a50fb640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a01416b0 0x7f27a0141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a58fc640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 0x7f27a0151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 -- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a01416b0 msgr2=0x7f27a0141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a01416b0 0x7f27a0141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 -- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 msgr2=0x7f27a0151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 0x7f27a0151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 -- 172.21.15.40:0/1839183141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01482f0 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a58fc640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 0x7f27a0151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a48fa640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2794011990 tx=0x7f2794011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400fa70 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279400fc10 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a0142ab0 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.526+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794014440 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.527+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a0143070 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.528+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a006a290 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.528+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2794012040 con 0x7f27a0142480 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.529+0000 7f279dffb640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 0x7f278c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.529+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f279408c140 con 0x7f27a0142480 2024-04-26T21:18:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.529+0000 7f27a50fb640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 0x7f278c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.530+0000 7f27a50fb640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 0x7f278c06ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2798004840 tx=0x7f2798009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.533+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2794019030 con 0x7f27a0142480 2024-04-26T21:18:48.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.716+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27a0072120 con 0x7f278c06c920 2024-04-26T21:18:48.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.722+0000 7f279dffb640 1 -- 172.21.15.40:0/1839183141 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238413 (secure 0 0 0) 0x7f27a0072120 con 0x7f278c06c920 2024-04-26T21:18:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:48.727 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:18:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 msgr2=0x7f278c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 0x7f278c06ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2798004840 tx=0x7f2798009040 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 msgr2=0x7f27a015e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2794011990 tx=0x7f2794011e50 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 shutdown_connections 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f278c06c920 0x7f278c06ede0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27a015ea00 0x7f27a0151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27a0142480 0x7f27a015e4c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 --2- 172.21.15.40:0/1839183141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27a01416b0 0x7f27a0141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 >> 172.21.15.40:0/1839183141 conn(0x7f27a00d0d30 msgr2=0x7f27a01517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 shutdown_connections 2024-04-26T21:18:48.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.725+0000 7f27a735f640 1 -- 172.21.15.40:0/1839183141 wait complete. 2024-04-26T21:18:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-26T21:18:47.016512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":263720,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711000,"statfs":{"total":773094113280,"available":772824064000,"internally_reserved":0,"allocated":10342400,"data_stored":5049077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016202"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018917+0000","last_change":"2024-04-26T21:18:44.018917+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018917+0000","last_undegraded":"2024-04-26T21:18:44.018917+0000","last_fullsized":"2024-04-26T21:18:44.018917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015819+0000","last_change":"2024-04-26T21:18:44.015819+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015819+0000","last_undegraded":"2024-04-26T21:18:44.015819+0000","last_fullsized":"2024-04-26T21:18:44.015819+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016468+0000","last_change":"2024-04-26T21:18:44.016468+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016468+0000","last_undegraded":"2024-04-26T21:18:44.016468+0000","last_fullsized":"2024-04-26T21:18:44.016468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018687+0000","last_change":"2024-04-26T21:18:44.018687+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018687+0000","last_undegraded":"2024-04-26T21:18:44.018687+0000","last_fullsized":"2024-04-26T21:18:44.018687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029601+0000","last_change":"2024-04-26T21:18:44.024416+0000","last_active":"2024-04-26T21:18:44.029601+0000","last_peered":"2024-04-26T21:18:44.029601+0000","last_clean":"2024-04-26T21:18:44.029601+0000","last_became_active":"2024-04-26T21:18:44.024129+0000","last_became_peered":"2024-04-26T21:18:44.024129+0000","last_unstale":"2024-04-26T21:18:44.029601+0000","last_undegraded":"2024-04-26T21:18:44.029601+0000","last_fullsized":"2024-04-26T21:18:44.029601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:11.530320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029700+0000","last_change":"2024-04-26T21:18:44.025572+0000","last_active":"2024-04-26T21:18:44.029700+0000","last_peered":"2024-04-26T21:18:44.029700+0000","last_clean":"2024-04-26T21:18:44.029700+0000","last_became_active":"2024-04-26T21:18:44.025260+0000","last_became_peered":"2024-04-26T21:18:44.025260+0000","last_unstale":"2024-04-26T21:18:44.029700+0000","last_undegraded":"2024-04-26T21:18:44.029700+0000","last_fullsized":"2024-04-26T21:18:44.029700+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:57.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029675+0000","last_change":"2024-04-26T21:18:44.025032+0000","last_active":"2024-04-26T21:18:44.029675+0000","last_peered":"2024-04-26T21:18:44.029675+0000","last_clean":"2024-04-26T21:18:44.029675+0000","last_became_active":"2024-04-26T21:18:44.024732+0000","last_became_peered":"2024-04-26T21:18:44.024732+0000","last_unstale":"2024-04-26T21:18:44.029675+0000","last_undegraded":"2024-04-26T21:18:44.029675+0000","last_fullsized":"2024-04-26T21:18:44.029675+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:41.641506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015981+0000","last_change":"2024-04-26T21:18:44.015981+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015981+0000","last_undegraded":"2024-04-26T21:18:44.015981+0000","last_fullsized":"2024-04-26T21:18:44.015981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.031512+0000","last_change":"2024-04-26T21:18:44.028840+0000","last_active":"2024-04-26T21:18:44.031512+0000","last_peered":"2024-04-26T21:18:44.031512+0000","last_clean":"2024-04-26T21:18:44.031512+0000","last_became_active":"2024-04-26T21:18:44.028560+0000","last_became_peered":"2024-04-26T21:18:44.028560+0000","last_unstale":"2024-04-26T21:18:44.031512+0000","last_undegraded":"2024-04-26T21:18:44.031512+0000","last_fullsized":"2024-04-26T21:18:44.031512+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:00.234414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016765+0000","last_change":"2024-04-26T21:18:44.016765+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016765+0000","last_undegraded":"2024-04-26T21:18:44.016765+0000","last_fullsized":"2024-04-26T21:18:44.016765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016771+0000","last_change":"2024-04-26T21:18:44.016771+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016771+0000","last_undegraded":"2024-04-26T21:18:44.016771+0000","last_fullsized":"2024-04-26T21:18:44.016771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:43.999199+0000","last_change":"2024-04-26T21:18:43.999199+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:43.999199+0000","last_undegraded":"2024-04-26T21:18:43.999199+0000","last_fullsized":"2024-04-26T21:18:43.999199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:18:42.996696+0000","last_change":"2024-04-26T21:18:42.996696+0000","last_active":"2024-04-26T21:18:42.996696+0000","last_peered":"2024-04-26T21:18:42.996696+0000","last_clean":"2024-04-26T21:18:42.996696+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:42.996696+0000","last_undegraded":"2024-04-26T21:18:42.996696+0000","last_fullsized":"2024-04-26T21:18:42.996696+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":297,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T21:18:42.244752+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:18:42.244752+0000","last_peered":"2024-04-26T21:18:42.244752+0000","last_clean":"2024-04-26T21:18:42.244752+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:18:42.244752+0000","last_undegraded":"2024-04-26T21:18:42.244752+0000","last_fullsized":"2024-04-26T21:18:42.244752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509527,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1067913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":482672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1073040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":494764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:48.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-04-26T21:18:48.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.018917+0000', 'last_change': '2024-04-26T21:18:44.018917+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.018917+0000', 'last_undegraded': '2024-04-26T21:18:44.018917+0000', 'last_fullsized': '2024-04-26T21:18:44.018917+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.015819+0000', 'last_change': '2024-04-26T21:18:44.015819+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.015819+0000', 'last_undegraded': '2024-04-26T21:18:44.015819+0000', 'last_fullsized': '2024-04-26T21:18:44.015819+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-04-26T21:18:48.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-04-26T21:18:48.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-04-26T21:18:48.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.016468+0000', 'last_change': '2024-04-26T21:18:44.016468+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.016468+0000', 'last_undegraded': '2024-04-26T21:18:44.016468+0000', 'last_fullsized': '2024-04-26T21:18:44.016468+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.018687+0000', 'last_change': '2024-04-26T21:18:44.018687+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.018687+0000', 'last_undegraded': '2024-04-26T21:18:44.018687+0000', 'last_fullsized': '2024-04-26T21:18:44.018687+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-26T21:18:48.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-26T21:18:48.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-26T21:18:48.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.015981+0000', 'last_change': '2024-04-26T21:18:44.015981+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.015981+0000', 'last_undegraded': '2024-04-26T21:18:44.015981+0000', 'last_fullsized': '2024-04-26T21:18:44.015981+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-26T21:18:48.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-26T21:18:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.016765+0000', 'last_change': '2024-04-26T21:18:44.016765+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.016765+0000', 'last_undegraded': '2024-04-26T21:18:44.016765+0000', 'last_fullsized': '2024-04-26T21:18:44.016765+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:18:44.016771+0000', 'last_change': '2024-04-26T21:18:44.016771+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:44.016771+0000', 'last_undegraded': '2024-04-26T21:18:44.016771+0000', 'last_fullsized': '2024-04-26T21:18:44.016771+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-26T21:18:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-26T21:18:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:43.999199+0000', 'last_change': '2024-04-26T21:18:43.999199+0000', 'last_active': '2024-04-26T21:18:43.999199+0000', 'last_peered': '2024-04-26T21:18:43.999199+0000', 'last_clean': '2024-04-26T21:18:43.999199+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:43.999199+0000', 'last_undegraded': '2024-04-26T21:18:43.999199+0000', 'last_fullsized': '2024-04-26T21:18:43.999199+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:43.999199+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-26T21:18:48.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T21:18:42.996696+0000', 'last_change': '2024-04-26T21:18:42.996696+0000', 'last_active': '2024-04-26T21:18:42.996696+0000', 'last_peered': '2024-04-26T21:18:42.996696+0000', 'last_clean': '2024-04-26T21:18:42.996696+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:18:42.996696+0000', 'last_undegraded': '2024-04-26T21:18:42.996696+0000', 'last_fullsized': '2024-04-26T21:18:42.996696+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'last_clean_scrub_stamp': '2024-04-26T21:18:42.996696+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T21:18:48.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:18:48.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.876+0000 7f8cfaf4e640 1 Processor -- start 2024-04-26T21:18:48.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.876+0000 7f8cfaf4e640 1 -- start start 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cfaf4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8cf4070570 0x7f8cf40b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cfaf4e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf415fce0 con 0x7f8cf415f6a0 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cfaf4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf415ffc0 con 0x7f8cf409ba40 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cfaf4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf4160610 con 0x7f8cf4070570 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cf3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8cf415f6a0 0x7f8cf40d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43040/0 (socket says 172.21.15.40:43040) 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cf3fff640 1 -- 172.21.15.40:0/3485763189 learned_addr learned my addr 172.21.15.40:0/3485763189 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/3485763189 >> v2:172.21.15.184:3300/0 conn(0x7f8cf4070570 0x7f8cf40b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691404688 0 0) 0x7f8cf415fce0 con 0x7f8cf415f6a0 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.877+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ce4003520 con 0x7f8cf415f6a0 2024-04-26T21:18:48.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 357742441 0 0) 0x7f8ce4003520 con 0x7f8cf415f6a0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 >> v2:172.21.15.184:3300/0 conn(0x7f8cf4070570 msgr2=0x7f8cf40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 --2- 172.21.15.40:0/3485763189 >> v2:172.21.15.184:3300/0 conn(0x7f8cf4070570 0x7f8cf40b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 >> v1:172.21.15.184:6789/0 conn(0x7f8cf409ba40 legacy=0x7f8cf409be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41602a0 con 0x7f8cf415f6a0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f8ce0001670 con 0x7f8cf415f6a0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 >> v1:172.21.15.40:6789/0 conn(0x7f8cf415f6a0 legacy=0x7f8cf40d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce4004800 0x7f8ce4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 0x7f8ce400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4160610 con 0x7f8ce40051c0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce400c060 con 0x7f8ce4004800 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce400c260 con 0x7f8ce40039b0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 0x7f8ce400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf3fff640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce4004800 0x7f8ce4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf8cea640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 -- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ce40039b0 msgr2=0x7f8ce4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 -- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce4004800 msgr2=0x7f8ce4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce4004800 0x7f8ce4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.878+0000 7f8cf94eb640 1 -- 172.21.15.40:0/3485763189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce4003520 con 0x7f8ce40051c0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.879+0000 7f8cf94eb640 1 -- 172.21.15.40:0/3485763189 reap_dead start 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.879+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 0x7f8ce400b9c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009fc0 tx=0x7f8ce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce35c84f214a2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.879+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec035020 con 0x7f8ce40051c0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.879+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec003bd0 con 0x7f8ce40051c0 2024-04-26T21:18:48.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.879+0000 7f8cf37fe640 1 -- 172.21.15.40:0/3485763189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec009c10 con 0x7f8ce40051c0 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 msgr2=0x7f8ce400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 0x7f8ce400b9c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009fc0 tx=0x7f8ce4005830 comp rx=0 tx=0).stop 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/3485763189 shutdown_connections 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/3485763189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ce40051c0 0x7f8ce400b9c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/3485763189 >> 172.21.15.40:0/3485763189 conn(0x7f8cf40b4a10 msgr2=0x7f8cf40b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/3485763189 shutdown_connections 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.880+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/3485763189 wait complete. 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.881+0000 7f8cfaf4e640 1 Processor -- start 2024-04-26T21:18:48.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.881+0000 7f8cfaf4e640 1 -- start start 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce40051c0 0x7f8cf41f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf40b7ca0 0x7f8cf40b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf40b58c0 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf40b5740 con 0x7f8ce40051c0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cfaf4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf409be20 con 0x7f8cf40b7ca0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40002/0 (socket says 172.21.15.40:40002) 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 -- 172.21.15.40:0/495412123 learned_addr learned my addr 172.21.15.40:0/495412123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf8cea640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce40051c0 0x7f8cf41f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf94eb640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf40b7ca0 0x7f8cf40b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 -- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf40b7ca0 msgr2=0x7f8cf40b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf40b7ca0 0x7f8cf40b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 -- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce40051c0 msgr2=0x7f8cf41f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce40051c0 0x7f8cf41f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 -- 172.21.15.40:0/495412123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf40b79e0 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf3fff640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00072c0 tx=0x7f8ce0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.882+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0001660 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.883+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce0018070 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.883+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf40cee00 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.883+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000fb80 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.883+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf40cf3c0 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.884+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf406a290 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.884+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ce000fd20 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.884+0000 7f8cf17fa640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 0x7f8cd806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.884+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f8ce008bb20 con 0x7f8cf40cfbc0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.885+0000 7f8cf8cea640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 0x7f8cd806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:48.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.885+0000 7f8cf8cea640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 0x7f8cd806ef50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:48.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:48.889+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ce0059fe0 con 0x7f8cf40cfbc0 2024-04-26T21:18:49.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.146+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8cf415afd0 con 0x7f8cf40cfbc0 2024-04-26T21:18:49.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.147+0000 7f8cf17fa640 1 -- 172.21.15.40:0/495412123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1100 (secure 0 0 0) 0x7f8ce005ee20 con 0x7f8cf40cfbc0 2024-04-26T21:18:49.149 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592778,"bytes_used":270479360,"bytes_avail":772823633920,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":57,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":7545}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-26T21:17:04.990352+0000","services":{}},"progress_events":{}} 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 msgr2=0x7f8cd806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 0x7f8cd806ef50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce8009040 comp rx=0 tx=0).stop 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 msgr2=0x7f8cf40b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00072c0 tx=0x7f8ce0007780 comp rx=0 tx=0).stop 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 shutdown_connections 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cd806ca90 0x7f8cd806ef50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8cf40b7ca0 0x7f8cf40b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8cf40cfbc0 0x7f8cf40b6ef0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 --2- 172.21.15.40:0/495412123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ce40051c0 0x7f8cf41f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 >> 172.21.15.40:0/495412123 conn(0x7f8cf40b4a10 msgr2=0x7f8cf406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 shutdown_connections 2024-04-26T21:18:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.149+0000 7f8cfaf4e640 1 -- 172.21.15.40:0/495412123 wait complete. 2024-04-26T21:18:49.162 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:18:49.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.260+0000 7fe53fb6a640 1 Processor -- start 2024-04-26T21:18:49.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.260+0000 7fe53fb6a640 1 -- start start 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53fb6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5380b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53fb6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe53815f720 0x7fe5380d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53fb6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5380b2510 con 0x7fe5380d4f00 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53fb6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe5380b27f0 con 0x7fe53815f720 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53fb6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5380b2ad0 con 0x7fe53815f340 2024-04-26T21:18:49.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53e107640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5380d4f00 0x7fe5380d5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43044/0 (socket says 172.21.15.40:43044) 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53e107640 1 -- 172.21.15.40:0/563419201 learned_addr learned my addr 172.21.15.40:0/563419201 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53d105640 1 --2- 172.21.15.40:0/563419201 >> v2:172.21.15.184:3300/0 conn(0x7fe53815f720 0x7fe5380d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53d906640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5380b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.261+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916407626 0 0) 0x7fe5380b2510 con 0x7fe5380d4f00 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe520003520 con 0x7fe5380d4f00 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3808318809 0 0) 0x7fe520003520 con 0x7fe5380d4f00 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 msgr2=0x7fe5380b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5380b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 >> v2:172.21.15.184:3300/0 conn(0x7fe53815f720 msgr2=0x7fe5380d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 --2- 172.21.15.40:0/563419201 >> v2:172.21.15.184:3300/0 conn(0x7fe53815f720 0x7fe5380d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5380b2db0 con 0x7fe5380d4f00 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe530001670 con 0x7fe5380d4f00 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 >> v1:172.21.15.40:6789/0 conn(0x7fe5380d4f00 legacy=0x7fe5380d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe520003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe520004800 0x7fe520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380b27f0 con 0x7fe5200039b0 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380b2ad0 con 0x7fe520004800 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe52000b9c0 con 0x7fe53815f340 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe520003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d105640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe520004800 0x7fe520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53e107640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 -- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 msgr2=0x7fe5200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe53815f340 0x7fe5200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 -- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe520004800 msgr2=0x7fe520004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe520004800 0x7fe520004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d906640 1 -- 172.21.15.40:0/563419201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe520003520 con 0x7fe5200039b0 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.262+0000 7fe53d105640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe520004800 0x7fe520004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:49.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.263+0000 7fe53d906640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe520003dd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e170 tx=0x7fe520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4335de40fe4ecd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.263+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528034020 con 0x7fe5200039b0 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.263+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5280305f0 con 0x7fe5200039b0 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.263+0000 7fe53c904640 1 -- 172.21.15.40:0/563419201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528009c10 con 0x7fe5200039b0 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 -- 172.21.15.40:0/563419201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 msgr2=0x7fe520003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/563419201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe520003dd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e170 tx=0x7fe520005830 comp rx=0 tx=0).stop 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53e107640 1 -- 172.21.15.40:0/563419201 reap_dead start 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 -- 172.21.15.40:0/563419201 shutdown_connections 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 -- 172.21.15.40:0/563419201 >> 172.21.15.40:0/563419201 conn(0x7fe53808ec70 msgr2=0x7fe53809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 -- 172.21.15.40:0/563419201 shutdown_connections 2024-04-26T21:18:49.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.264+0000 7fe53fb6a640 1 -- 172.21.15.40:0/563419201 wait complete. 2024-04-26T21:18:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.265+0000 7fe53fb6a640 1 Processor -- start 2024-04-26T21:18:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.265+0000 7fe53fb6a640 1 -- start start 2024-04-26T21:18:49.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 0x7fe53809f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 0x7fe5380a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380b2290 con 0x7fe5200039b0 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380b2110 con 0x7fe53809f000 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe5380b35b0 con 0x7fe5380a0f10 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40020/0 (socket says 172.21.15.40:40020) 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 -- 172.21.15.40:0/743931224 learned_addr learned my addr 172.21.15.40:0/743931224 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53e107640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 0x7fe5380a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 -- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 msgr2=0x7fe5380a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d105640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 0x7fe53809f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 0x7fe5380a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 -- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 msgr2=0x7fe53809f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 0x7fe53809f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 -- 172.21.15.40:0/743931224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe538154ff0 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53e107640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 0x7fe5380a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d105640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 0x7fe53809f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53d906640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e170 tx=0x7fe528009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528040070 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe528039a00 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5380a2c90 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528034040 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.266+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5380a1ed0 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.267+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53806a290 con 0x7fe5200039b0 2024-04-26T21:18:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.268+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe528037020 con 0x7fe5200039b0 2024-04-26T21:18:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.268+0000 7fe5367fc640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 0x7fe52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.268+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe5280ac6e0 con 0x7fe5200039b0 2024-04-26T21:18:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.268+0000 7fe53d105640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 0x7fe52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.269+0000 7fe53d105640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 0x7fe52406edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe53815dc80 tx=0x7fe52c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:49.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.273+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe52807ab10 con 0x7fe5200039b0 2024-04-26T21:18:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.454+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5380d3020 con 0x7fe52406c8f0 2024-04-26T21:18:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.461+0000 7fe5367fc640 1 -- 172.21.15.40:0/743931224 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251530 (secure 0 0 0) 0x7fe5380d3020 con 0x7fe52406c8f0 2024-04-26T21:18:49.465 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:49.466 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:18:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 msgr2=0x7fe52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 0x7fe52406edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe53815dc80 tx=0x7fe52c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 msgr2=0x7fe5381545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e170 tx=0x7fe528009750 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 shutdown_connections 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe52406c8f0 0x7fe52406edb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5380a0f10 0x7fe5380a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe53809f000 0x7fe53809f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 --2- 172.21.15.40:0/743931224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5200039b0 0x7fe5381545c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 >> 172.21.15.40:0/743931224 conn(0x7fe53808ec70 msgr2=0x7fe5380cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 shutdown_connections 2024-04-26T21:18:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:49.464+0000 7fe53fb6a640 1 -- 172.21.15.40:0/743931224 wait complete. 2024-04-26T21:18:49.476 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-04-26T21:18:49.016886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264140,"kb_used_data":10512,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710580,"statfs":{"total":773094113280,"available":772823633920,"internally_reserved":0,"allocated":10764288,"data_stored":5235193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018917+0000","last_change":"2024-04-26T21:18:44.018917+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018917+0000","last_undegraded":"2024-04-26T21:18:44.018917+0000","last_fullsized":"2024-04-26T21:18:44.018917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015284+0000","last_change":"2024-04-26T21:18:45.028537+0000","last_active":"2024-04-26T21:18:47.015284+0000","last_peered":"2024-04-26T21:18:47.015284+0000","last_clean":"2024-04-26T21:18:47.015284+0000","last_became_active":"2024-04-26T21:18:45.028229+0000","last_became_peered":"2024-04-26T21:18:45.028229+0000","last_unstale":"2024-04-26T21:18:47.015284+0000","last_undegraded":"2024-04-26T21:18:47.015284+0000","last_fullsized":"2024-04-26T21:18:47.015284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:06.291628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015819+0000","last_change":"2024-04-26T21:18:44.015819+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015819+0000","last_undegraded":"2024-04-26T21:18:44.015819+0000","last_fullsized":"2024-04-26T21:18:44.015819+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025293+0000","last_change":"2024-04-26T21:18:45.027534+0000","last_active":"2024-04-26T21:18:46.025293+0000","last_peered":"2024-04-26T21:18:46.025293+0000","last_clean":"2024-04-26T21:18:46.025293+0000","last_became_active":"2024-04-26T21:18:45.027256+0000","last_became_peered":"2024-04-26T21:18:45.027256+0000","last_unstale":"2024-04-26T21:18:46.025293+0000","last_undegraded":"2024-04-26T21:18:46.025293+0000","last_fullsized":"2024-04-26T21:18:46.025293+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633905+0000","last_change":"2024-04-26T21:18:45.032479+0000","last_active":"2024-04-26T21:18:46.633905+0000","last_peered":"2024-04-26T21:18:46.633905+0000","last_clean":"2024-04-26T21:18:46.633905+0000","last_became_active":"2024-04-26T21:18:45.032219+0000","last_became_peered":"2024-04-26T21:18:45.032219+0000","last_unstale":"2024-04-26T21:18:46.633905+0000","last_undegraded":"2024-04-26T21:18:46.633905+0000","last_fullsized":"2024-04-26T21:18:46.633905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:30.178243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124480+0000","last_change":"2024-04-26T21:18:45.023238+0000","last_active":"2024-04-26T21:18:47.124480+0000","last_peered":"2024-04-26T21:18:47.124480+0000","last_clean":"2024-04-26T21:18:47.124480+0000","last_became_active":"2024-04-26T21:18:45.022988+0000","last_became_peered":"2024-04-26T21:18:45.022988+0000","last_unstale":"2024-04-26T21:18:47.124480+0000","last_undegraded":"2024-04-26T21:18:47.124480+0000","last_fullsized":"2024-04-26T21:18:47.124480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:20.109826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124470+0000","last_change":"2024-04-26T21:18:45.027725+0000","last_active":"2024-04-26T21:18:47.124470+0000","last_peered":"2024-04-26T21:18:47.124470+0000","last_clean":"2024-04-26T21:18:47.124470+0000","last_became_active":"2024-04-26T21:18:45.027450+0000","last_became_peered":"2024-04-26T21:18:45.027450+0000","last_unstale":"2024-04-26T21:18:47.124470+0000","last_undegraded":"2024-04-26T21:18:47.124470+0000","last_fullsized":"2024-04-26T21:18:47.124470+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015142+0000","last_change":"2024-04-26T21:18:45.028833+0000","last_active":"2024-04-26T21:18:47.015142+0000","last_peered":"2024-04-26T21:18:47.015142+0000","last_clean":"2024-04-26T21:18:47.015142+0000","last_became_active":"2024-04-26T21:18:45.028509+0000","last_became_peered":"2024-04-26T21:18:45.028509+0000","last_unstale":"2024-04-26T21:18:47.015142+0000","last_undegraded":"2024-04-26T21:18:47.015142+0000","last_fullsized":"2024-04-26T21:18:47.015142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:12.695915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025269+0000","last_change":"2024-04-26T21:18:45.031345+0000","last_active":"2024-04-26T21:18:46.025269+0000","last_peered":"2024-04-26T21:18:46.025269+0000","last_clean":"2024-04-26T21:18:46.025269+0000","last_became_active":"2024-04-26T21:18:45.031042+0000","last_became_peered":"2024-04-26T21:18:45.031042+0000","last_unstale":"2024-04-26T21:18:46.025269+0000","last_undegraded":"2024-04-26T21:18:46.025269+0000","last_fullsized":"2024-04-26T21:18:46.025269+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:24.464304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634010+0000","last_change":"2024-04-26T21:18:45.024931+0000","last_active":"2024-04-26T21:18:46.634010+0000","last_peered":"2024-04-26T21:18:46.634010+0000","last_clean":"2024-04-26T21:18:46.634010+0000","last_became_active":"2024-04-26T21:18:45.024628+0000","last_became_peered":"2024-04-26T21:18:45.024628+0000","last_unstale":"2024-04-26T21:18:46.634010+0000","last_undegraded":"2024-04-26T21:18:46.634010+0000","last_fullsized":"2024-04-26T21:18:46.634010+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:56.384292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124474+0000","last_change":"2024-04-26T21:18:45.028433+0000","last_active":"2024-04-26T21:18:47.124474+0000","last_peered":"2024-04-26T21:18:47.124474+0000","last_clean":"2024-04-26T21:18:47.124474+0000","last_became_active":"2024-04-26T21:18:45.028113+0000","last_became_peered":"2024-04-26T21:18:45.028113+0000","last_unstale":"2024-04-26T21:18:47.124474+0000","last_undegraded":"2024-04-26T21:18:47.124474+0000","last_fullsized":"2024-04-26T21:18:47.124474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:26.756819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025096+0000","last_change":"2024-04-26T21:18:45.024858+0000","last_active":"2024-04-26T21:18:46.025096+0000","last_peered":"2024-04-26T21:18:46.025096+0000","last_clean":"2024-04-26T21:18:46.025096+0000","last_became_active":"2024-04-26T21:18:45.024560+0000","last_became_peered":"2024-04-26T21:18:45.024560+0000","last_unstale":"2024-04-26T21:18:46.025096+0000","last_undegraded":"2024-04-26T21:18:46.025096+0000","last_fullsized":"2024-04-26T21:18:46.025096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.434562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025139+0000","last_change":"2024-04-26T21:18:45.035850+0000","last_active":"2024-04-26T21:18:46.025139+0000","last_peered":"2024-04-26T21:18:46.025139+0000","last_clean":"2024-04-26T21:18:46.025139+0000","last_became_active":"2024-04-26T21:18:45.035565+0000","last_became_peered":"2024-04-26T21:18:45.035565+0000","last_unstale":"2024-04-26T21:18:46.025139+0000","last_undegraded":"2024-04-26T21:18:46.025139+0000","last_fullsized":"2024-04-26T21:18:46.025139+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:52.859671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017637+0000","last_change":"2024-04-26T21:18:45.029826+0000","last_active":"2024-04-26T21:18:47.017637+0000","last_peered":"2024-04-26T21:18:47.017637+0000","last_clean":"2024-04-26T21:18:47.017637+0000","last_became_active":"2024-04-26T21:18:45.029236+0000","last_became_peered":"2024-04-26T21:18:45.029236+0000","last_unstale":"2024-04-26T21:18:47.017637+0000","last_undegraded":"2024-04-26T21:18:47.017637+0000","last_fullsized":"2024-04-26T21:18:47.017637+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:02.758562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024802+0000","last_change":"2024-04-26T21:18:45.036314+0000","last_active":"2024-04-26T21:18:46.024802+0000","last_peered":"2024-04-26T21:18:46.024802+0000","last_clean":"2024-04-26T21:18:46.024802+0000","last_became_active":"2024-04-26T21:18:45.035995+0000","last_became_peered":"2024-04-26T21:18:45.035995+0000","last_unstale":"2024-04-26T21:18:46.024802+0000","last_undegraded":"2024-04-26T21:18:46.024802+0000","last_fullsized":"2024-04-26T21:18:46.024802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:30.178243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124549+0000","last_change":"2024-04-26T21:18:45.029801+0000","last_active":"2024-04-26T21:18:47.124549+0000","last_peered":"2024-04-26T21:18:47.124549+0000","last_clean":"2024-04-26T21:18:47.124549+0000","last_became_active":"2024-04-26T21:18:45.029535+0000","last_became_peered":"2024-04-26T21:18:45.029535+0000","last_unstale":"2024-04-26T21:18:47.124549+0000","last_undegraded":"2024-04-26T21:18:47.124549+0000","last_fullsized":"2024-04-26T21:18:47.124549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:23.849073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015536+0000","last_change":"2024-04-26T21:18:45.037043+0000","last_active":"2024-04-26T21:18:47.015536+0000","last_peered":"2024-04-26T21:18:47.015536+0000","last_clean":"2024-04-26T21:18:47.015536+0000","last_became_active":"2024-04-26T21:18:45.036743+0000","last_became_peered":"2024-04-26T21:18:45.036743+0000","last_unstale":"2024-04-26T21:18:47.015536+0000","last_undegraded":"2024-04-26T21:18:47.015536+0000","last_fullsized":"2024-04-26T21:18:47.015536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:43.951451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024809+0000","last_change":"2024-04-26T21:18:45.025803+0000","last_active":"2024-04-26T21:18:46.024809+0000","last_peered":"2024-04-26T21:18:46.024809+0000","last_clean":"2024-04-26T21:18:46.024809+0000","last_became_active":"2024-04-26T21:18:45.025416+0000","last_became_peered":"2024-04-26T21:18:45.025416+0000","last_unstale":"2024-04-26T21:18:46.024809+0000","last_undegraded":"2024-04-26T21:18:46.024809+0000","last_fullsized":"2024-04-26T21:18:46.024809+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:11.034280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018083+0000","last_change":"2024-04-26T21:18:45.024838+0000","last_active":"2024-04-26T21:18:47.018083+0000","last_peered":"2024-04-26T21:18:47.018083+0000","last_clean":"2024-04-26T21:18:47.018083+0000","last_became_active":"2024-04-26T21:18:45.024509+0000","last_became_peered":"2024-04-26T21:18:45.024509+0000","last_unstale":"2024-04-26T21:18:47.018083+0000","last_undegraded":"2024-04-26T21:18:47.018083+0000","last_fullsized":"2024-04-26T21:18:47.018083+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:10.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025241+0000","last_change":"2024-04-26T21:18:45.031847+0000","last_active":"2024-04-26T21:18:46.025241+0000","last_peered":"2024-04-26T21:18:46.025241+0000","last_clean":"2024-04-26T21:18:46.025241+0000","last_became_active":"2024-04-26T21:18:45.030655+0000","last_became_peered":"2024-04-26T21:18:45.030655+0000","last_unstale":"2024-04-26T21:18:46.025241+0000","last_undegraded":"2024-04-26T21:18:46.025241+0000","last_fullsized":"2024-04-26T21:18:46.025241+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:17.288331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017962+0000","last_change":"2024-04-26T21:18:45.030714+0000","last_active":"2024-04-26T21:18:47.017962+0000","last_peered":"2024-04-26T21:18:47.017962+0000","last_clean":"2024-04-26T21:18:47.017962+0000","last_became_active":"2024-04-26T21:18:45.028965+0000","last_became_peered":"2024-04-26T21:18:45.028965+0000","last_unstale":"2024-04-26T21:18:47.017962+0000","last_undegraded":"2024-04-26T21:18:47.017962+0000","last_fullsized":"2024-04-26T21:18:47.017962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:45:01.726761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014814+0000","last_change":"2024-04-26T21:18:45.030869+0000","last_active":"2024-04-26T21:18:47.014814+0000","last_peered":"2024-04-26T21:18:47.014814+0000","last_clean":"2024-04-26T21:18:47.014814+0000","last_became_active":"2024-04-26T21:18:45.030616+0000","last_became_peered":"2024-04-26T21:18:45.030616+0000","last_unstale":"2024-04-26T21:18:47.014814+0000","last_undegraded":"2024-04-26T21:18:47.014814+0000","last_fullsized":"2024-04-26T21:18:47.014814+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:11.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015024+0000","last_change":"2024-04-26T21:18:45.030225+0000","last_active":"2024-04-26T21:18:47.015024+0000","last_peered":"2024-04-26T21:18:47.015024+0000","last_clean":"2024-04-26T21:18:47.015024+0000","last_became_active":"2024-04-26T21:18:45.029962+0000","last_became_peered":"2024-04-26T21:18:45.029962+0000","last_unstale":"2024-04-26T21:18:47.015024+0000","last_undegraded":"2024-04-26T21:18:47.015024+0000","last_fullsized":"2024-04-26T21:18:47.015024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:35.951269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124568+0000","last_change":"2024-04-26T21:18:45.039040+0000","last_active":"2024-04-26T21:18:47.124568+0000","last_peered":"2024-04-26T21:18:47.124568+0000","last_clean":"2024-04-26T21:18:47.124568+0000","last_became_active":"2024-04-26T21:18:45.038765+0000","last_became_peered":"2024-04-26T21:18:45.038765+0000","last_unstale":"2024-04-26T21:18:47.124568+0000","last_undegraded":"2024-04-26T21:18:47.124568+0000","last_fullsized":"2024-04-26T21:18:47.124568+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:59.961187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634109+0000","last_change":"2024-04-26T21:18:45.029767+0000","last_active":"2024-04-26T21:18:46.634109+0000","last_peered":"2024-04-26T21:18:46.634109+0000","last_clean":"2024-04-26T21:18:46.634109+0000","last_became_active":"2024-04-26T21:18:45.029316+0000","last_became_peered":"2024-04-26T21:18:45.029316+0000","last_unstale":"2024-04-26T21:18:46.634109+0000","last_undegraded":"2024-04-26T21:18:46.634109+0000","last_fullsized":"2024-04-26T21:18:46.634109+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015228+0000","last_change":"2024-04-26T21:18:45.028396+0000","last_active":"2024-04-26T21:18:47.015228+0000","last_peered":"2024-04-26T21:18:47.015228+0000","last_clean":"2024-04-26T21:18:47.015228+0000","last_became_active":"2024-04-26T21:18:45.028127+0000","last_became_peered":"2024-04-26T21:18:45.028127+0000","last_unstale":"2024-04-26T21:18:47.015228+0000","last_undegraded":"2024-04-26T21:18:47.015228+0000","last_fullsized":"2024-04-26T21:18:47.015228+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:47.380274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124674+0000","last_change":"2024-04-26T21:18:45.024115+0000","last_active":"2024-04-26T21:18:47.124674+0000","last_peered":"2024-04-26T21:18:47.124674+0000","last_clean":"2024-04-26T21:18:47.124674+0000","last_became_active":"2024-04-26T21:18:45.023816+0000","last_became_peered":"2024-04-26T21:18:45.023816+0000","last_unstale":"2024-04-26T21:18:47.124674+0000","last_undegraded":"2024-04-26T21:18:47.124674+0000","last_fullsized":"2024-04-26T21:18:47.124674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:11.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633928+0000","last_change":"2024-04-26T21:18:45.031007+0000","last_active":"2024-04-26T21:18:46.633928+0000","last_peered":"2024-04-26T21:18:46.633928+0000","last_clean":"2024-04-26T21:18:46.633928+0000","last_became_active":"2024-04-26T21:18:45.030689+0000","last_became_peered":"2024-04-26T21:18:45.030689+0000","last_unstale":"2024-04-26T21:18:46.633928+0000","last_undegraded":"2024-04-26T21:18:46.633928+0000","last_fullsized":"2024-04-26T21:18:46.633928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:11.034280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014986+0000","last_change":"2024-04-26T21:18:45.039935+0000","last_active":"2024-04-26T21:18:47.014986+0000","last_peered":"2024-04-26T21:18:47.014986+0000","last_clean":"2024-04-26T21:18:47.014986+0000","last_became_active":"2024-04-26T21:18:45.039657+0000","last_became_peered":"2024-04-26T21:18:45.039657+0000","last_unstale":"2024-04-26T21:18:47.014986+0000","last_undegraded":"2024-04-26T21:18:47.014986+0000","last_fullsized":"2024-04-26T21:18:47.014986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:16.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017621+0000","last_change":"2024-04-26T21:18:45.025483+0000","last_active":"2024-04-26T21:18:47.017621+0000","last_peered":"2024-04-26T21:18:47.017621+0000","last_clean":"2024-04-26T21:18:47.017621+0000","last_became_active":"2024-04-26T21:18:45.025217+0000","last_became_peered":"2024-04-26T21:18:45.025217+0000","last_unstale":"2024-04-26T21:18:47.017621+0000","last_undegraded":"2024-04-26T21:18:47.017621+0000","last_fullsized":"2024-04-26T21:18:47.017621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:30.599056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024925+0000","last_change":"2024-04-26T21:18:45.030588+0000","last_active":"2024-04-26T21:18:46.024925+0000","last_peered":"2024-04-26T21:18:46.024925+0000","last_clean":"2024-04-26T21:18:46.024925+0000","last_became_active":"2024-04-26T21:18:45.030329+0000","last_became_peered":"2024-04-26T21:18:45.030329+0000","last_unstale":"2024-04-26T21:18:46.024925+0000","last_undegraded":"2024-04-26T21:18:46.024925+0000","last_fullsized":"2024-04-26T21:18:46.024925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:56.384292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016468+0000","last_change":"2024-04-26T21:18:44.016468+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016468+0000","last_undegraded":"2024-04-26T21:18:44.016468+0000","last_fullsized":"2024-04-26T21:18:44.016468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018187+0000","last_change":"2024-04-26T21:18:45.024508+0000","last_active":"2024-04-26T21:18:47.018187+0000","last_peered":"2024-04-26T21:18:47.018187+0000","last_clean":"2024-04-26T21:18:47.018187+0000","last_became_active":"2024-04-26T21:18:45.024246+0000","last_became_peered":"2024-04-26T21:18:45.024246+0000","last_unstale":"2024-04-26T21:18:47.018187+0000","last_undegraded":"2024-04-26T21:18:47.018187+0000","last_fullsized":"2024-04-26T21:18:47.018187+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:17.239874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015588+0000","last_change":"2024-04-26T21:18:44.028474+0000","last_active":"2024-04-26T21:18:47.015588+0000","last_peered":"2024-04-26T21:18:47.015588+0000","last_clean":"2024-04-26T21:18:47.015588+0000","last_became_active":"2024-04-26T21:18:44.028207+0000","last_became_peered":"2024-04-26T21:18:44.028207+0000","last_unstale":"2024-04-26T21:18:47.015588+0000","last_undegraded":"2024-04-26T21:18:47.015588+0000","last_fullsized":"2024-04-26T21:18:47.015588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:58.541139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025468+0000","last_change":"2024-04-26T21:18:45.034871+0000","last_active":"2024-04-26T21:18:46.025468+0000","last_peered":"2024-04-26T21:18:46.025468+0000","last_clean":"2024-04-26T21:18:46.025468+0000","last_became_active":"2024-04-26T21:18:45.034602+0000","last_became_peered":"2024-04-26T21:18:45.034602+0000","last_unstale":"2024-04-26T21:18:46.025468+0000","last_undegraded":"2024-04-26T21:18:46.025468+0000","last_fullsized":"2024-04-26T21:18:46.025468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124650+0000","last_change":"2024-04-26T21:18:44.030474+0000","last_active":"2024-04-26T21:18:47.124650+0000","last_peered":"2024-04-26T21:18:47.124650+0000","last_clean":"2024-04-26T21:18:47.124650+0000","last_became_active":"2024-04-26T21:18:44.029983+0000","last_became_peered":"2024-04-26T21:18:44.029983+0000","last_unstale":"2024-04-26T21:18:47.124650+0000","last_undegraded":"2024-04-26T21:18:47.124650+0000","last_fullsized":"2024-04-26T21:18:47.124650+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.432059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.018687+0000","last_change":"2024-04-26T21:18:44.018687+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.018687+0000","last_undegraded":"2024-04-26T21:18:44.018687+0000","last_fullsized":"2024-04-26T21:18:44.018687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015134+0000","last_change":"2024-04-26T21:18:44.021771+0000","last_active":"2024-04-26T21:18:47.015134+0000","last_peered":"2024-04-26T21:18:47.015134+0000","last_clean":"2024-04-26T21:18:47.015134+0000","last_became_active":"2024-04-26T21:18:44.021492+0000","last_became_peered":"2024-04-26T21:18:44.021492+0000","last_unstale":"2024-04-26T21:18:47.015134+0000","last_undegraded":"2024-04-26T21:18:47.015134+0000","last_fullsized":"2024-04-26T21:18:47.015134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:27.301453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025282+0000","last_change":"2024-04-26T21:18:45.023563+0000","last_active":"2024-04-26T21:18:46.025282+0000","last_peered":"2024-04-26T21:18:46.025282+0000","last_clean":"2024-04-26T21:18:46.025282+0000","last_became_active":"2024-04-26T21:18:45.023300+0000","last_became_peered":"2024-04-26T21:18:45.023300+0000","last_unstale":"2024-04-26T21:18:46.025282+0000","last_undegraded":"2024-04-26T21:18:46.025282+0000","last_fullsized":"2024-04-26T21:18:46.025282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:34.810442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014986+0000","last_change":"2024-04-26T21:18:44.023307+0000","last_active":"2024-04-26T21:18:47.014986+0000","last_peered":"2024-04-26T21:18:47.014986+0000","last_clean":"2024-04-26T21:18:47.014986+0000","last_became_active":"2024-04-26T21:18:44.023006+0000","last_became_peered":"2024-04-26T21:18:44.023006+0000","last_unstale":"2024-04-26T21:18:47.014986+0000","last_undegraded":"2024-04-26T21:18:47.014986+0000","last_fullsized":"2024-04-26T21:18:47.014986+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:16.285828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669363+0000","last_change":"2024-04-26T21:18:45.014108+0000","last_active":"2024-04-26T21:18:46.669363+0000","last_peered":"2024-04-26T21:18:46.669363+0000","last_clean":"2024-04-26T21:18:46.669363+0000","last_became_active":"2024-04-26T21:18:45.013852+0000","last_became_peered":"2024-04-26T21:18:45.013852+0000","last_unstale":"2024-04-26T21:18:46.669363+0000","last_undegraded":"2024-04-26T21:18:46.669363+0000","last_fullsized":"2024-04-26T21:18:46.669363+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:15.135229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029601+0000","last_change":"2024-04-26T21:18:44.024416+0000","last_active":"2024-04-26T21:18:44.029601+0000","last_peered":"2024-04-26T21:18:44.029601+0000","last_clean":"2024-04-26T21:18:44.029601+0000","last_became_active":"2024-04-26T21:18:44.024129+0000","last_became_peered":"2024-04-26T21:18:44.024129+0000","last_unstale":"2024-04-26T21:18:44.029601+0000","last_undegraded":"2024-04-26T21:18:44.029601+0000","last_fullsized":"2024-04-26T21:18:44.029601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:11.530320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124465+0000","last_change":"2024-04-26T21:18:45.026037+0000","last_active":"2024-04-26T21:18:47.124465+0000","last_peered":"2024-04-26T21:18:47.124465+0000","last_clean":"2024-04-26T21:18:47.124465+0000","last_became_active":"2024-04-26T21:18:45.025727+0000","last_became_peered":"2024-04-26T21:18:45.025727+0000","last_unstale":"2024-04-26T21:18:47.124465+0000","last_undegraded":"2024-04-26T21:18:47.124465+0000","last_fullsized":"2024-04-26T21:18:47.124465+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014954+0000","last_change":"2024-04-26T21:18:44.024846+0000","last_active":"2024-04-26T21:18:47.014954+0000","last_peered":"2024-04-26T21:18:47.014954+0000","last_clean":"2024-04-26T21:18:47.014954+0000","last_became_active":"2024-04-26T21:18:44.024565+0000","last_became_peered":"2024-04-26T21:18:44.024565+0000","last_unstale":"2024-04-26T21:18:47.014954+0000","last_undegraded":"2024-04-26T21:18:47.014954+0000","last_fullsized":"2024-04-26T21:18:47.014954+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:34.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024686+0000","last_change":"2024-04-26T21:18:45.038538+0000","last_active":"2024-04-26T21:18:46.024686+0000","last_peered":"2024-04-26T21:18:46.024686+0000","last_clean":"2024-04-26T21:18:46.024686+0000","last_became_active":"2024-04-26T21:18:45.038279+0000","last_became_peered":"2024-04-26T21:18:45.038279+0000","last_unstale":"2024-04-26T21:18:46.024686+0000","last_undegraded":"2024-04-26T21:18:46.024686+0000","last_fullsized":"2024-04-26T21:18:46.024686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:54.286661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633861+0000","last_change":"2024-04-26T21:18:44.023069+0000","last_active":"2024-04-26T21:18:46.633861+0000","last_peered":"2024-04-26T21:18:46.633861+0000","last_clean":"2024-04-26T21:18:46.633861+0000","last_became_active":"2024-04-26T21:18:44.022719+0000","last_became_peered":"2024-04-26T21:18:44.022719+0000","last_unstale":"2024-04-26T21:18:46.633861+0000","last_undegraded":"2024-04-26T21:18:46.633861+0000","last_fullsized":"2024-04-26T21:18:46.633861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:13.337074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024639+0000","last_change":"2024-04-26T21:18:45.027344+0000","last_active":"2024-04-26T21:18:46.024639+0000","last_peered":"2024-04-26T21:18:46.024639+0000","last_clean":"2024-04-26T21:18:46.024639+0000","last_became_active":"2024-04-26T21:18:45.026788+0000","last_became_peered":"2024-04-26T21:18:45.026788+0000","last_unstale":"2024-04-26T21:18:46.024639+0000","last_undegraded":"2024-04-26T21:18:46.024639+0000","last_fullsized":"2024-04-26T21:18:46.024639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:59.543642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029700+0000","last_change":"2024-04-26T21:18:44.025572+0000","last_active":"2024-04-26T21:18:44.029700+0000","last_peered":"2024-04-26T21:18:44.029700+0000","last_clean":"2024-04-26T21:18:44.029700+0000","last_became_active":"2024-04-26T21:18:44.025260+0000","last_became_peered":"2024-04-26T21:18:44.025260+0000","last_unstale":"2024-04-26T21:18:44.029700+0000","last_undegraded":"2024-04-26T21:18:44.029700+0000","last_fullsized":"2024-04-26T21:18:44.029700+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:57.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015496+0000","last_change":"2024-04-26T21:18:45.029555+0000","last_active":"2024-04-26T21:18:47.015496+0000","last_peered":"2024-04-26T21:18:47.015496+0000","last_clean":"2024-04-26T21:18:47.015496+0000","last_became_active":"2024-04-26T21:18:45.029277+0000","last_became_peered":"2024-04-26T21:18:45.029277+0000","last_unstale":"2024-04-26T21:18:47.015496+0000","last_undegraded":"2024-04-26T21:18:47.015496+0000","last_fullsized":"2024-04-26T21:18:47.015496+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:37.275648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.029675+0000","last_change":"2024-04-26T21:18:44.025032+0000","last_active":"2024-04-26T21:18:44.029675+0000","last_peered":"2024-04-26T21:18:44.029675+0000","last_clean":"2024-04-26T21:18:44.029675+0000","last_became_active":"2024-04-26T21:18:44.024732+0000","last_became_peered":"2024-04-26T21:18:44.024732+0000","last_unstale":"2024-04-26T21:18:44.029675+0000","last_undegraded":"2024-04-26T21:18:44.029675+0000","last_fullsized":"2024-04-26T21:18:44.029675+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:41.641506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025356+0000","last_change":"2024-04-26T21:18:45.027556+0000","last_active":"2024-04-26T21:18:46.025356+0000","last_peered":"2024-04-26T21:18:46.025356+0000","last_clean":"2024-04-26T21:18:46.025356+0000","last_became_active":"2024-04-26T21:18:45.027255+0000","last_became_peered":"2024-04-26T21:18:45.027255+0000","last_unstale":"2024-04-26T21:18:46.025356+0000","last_undegraded":"2024-04-26T21:18:46.025356+0000","last_fullsized":"2024-04-26T21:18:46.025356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:28.303956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024671+0000","last_change":"2024-04-26T21:18:44.023148+0000","last_active":"2024-04-26T21:18:46.024671+0000","last_peered":"2024-04-26T21:18:46.024671+0000","last_clean":"2024-04-26T21:18:46.024671+0000","last_became_active":"2024-04-26T21:18:44.022721+0000","last_became_peered":"2024-04-26T21:18:44.022721+0000","last_unstale":"2024-04-26T21:18:46.024671+0000","last_undegraded":"2024-04-26T21:18:46.024671+0000","last_fullsized":"2024-04-26T21:18:46.024671+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:34.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634061+0000","last_change":"2024-04-26T21:18:45.029957+0000","last_active":"2024-04-26T21:18:46.634061+0000","last_peered":"2024-04-26T21:18:46.634061+0000","last_clean":"2024-04-26T21:18:46.634061+0000","last_became_active":"2024-04-26T21:18:45.029612+0000","last_became_peered":"2024-04-26T21:18:45.029612+0000","last_unstale":"2024-04-26T21:18:46.634061+0000","last_undegraded":"2024-04-26T21:18:46.634061+0000","last_fullsized":"2024-04-26T21:18:46.634061+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:30.119656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124776+0000","last_change":"2024-04-26T21:18:44.025750+0000","last_active":"2024-04-26T21:18:47.124776+0000","last_peered":"2024-04-26T21:18:47.124776+0000","last_clean":"2024-04-26T21:18:47.124776+0000","last_became_active":"2024-04-26T21:18:44.025402+0000","last_became_peered":"2024-04-26T21:18:44.025402+0000","last_unstale":"2024-04-26T21:18:47.124776+0000","last_undegraded":"2024-04-26T21:18:47.124776+0000","last_fullsized":"2024-04-26T21:18:47.124776+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:16.285828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124706+0000","last_change":"2024-04-26T21:18:45.029679+0000","last_active":"2024-04-26T21:18:47.124706+0000","last_peered":"2024-04-26T21:18:47.124706+0000","last_clean":"2024-04-26T21:18:47.124706+0000","last_became_active":"2024-04-26T21:18:45.029410+0000","last_became_peered":"2024-04-26T21:18:45.029410+0000","last_unstale":"2024-04-26T21:18:47.124706+0000","last_undegraded":"2024-04-26T21:18:47.124706+0000","last_fullsized":"2024-04-26T21:18:47.124706+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:16.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025208+0000","last_change":"2024-04-26T21:18:44.018742+0000","last_active":"2024-04-26T21:18:46.025208+0000","last_peered":"2024-04-26T21:18:46.025208+0000","last_clean":"2024-04-26T21:18:46.025208+0000","last_became_active":"2024-04-26T21:18:44.018439+0000","last_became_peered":"2024-04-26T21:18:44.018439+0000","last_unstale":"2024-04-26T21:18:46.025208+0000","last_undegraded":"2024-04-26T21:18:46.025208+0000","last_fullsized":"2024-04-26T21:18:46.025208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:16.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025141+0000","last_change":"2024-04-26T21:18:45.029655+0000","last_active":"2024-04-26T21:18:46.025141+0000","last_peered":"2024-04-26T21:18:46.025141+0000","last_clean":"2024-04-26T21:18:46.025141+0000","last_became_active":"2024-04-26T21:18:45.029376+0000","last_became_peered":"2024-04-26T21:18:45.029376+0000","last_unstale":"2024-04-26T21:18:46.025141+0000","last_undegraded":"2024-04-26T21:18:46.025141+0000","last_fullsized":"2024-04-26T21:18:46.025141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:30.119656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018135+0000","last_change":"2024-04-26T21:18:44.020936+0000","last_active":"2024-04-26T21:18:47.018135+0000","last_peered":"2024-04-26T21:18:47.018135+0000","last_clean":"2024-04-26T21:18:47.018135+0000","last_became_active":"2024-04-26T21:18:44.020631+0000","last_became_peered":"2024-04-26T21:18:44.020631+0000","last_unstale":"2024-04-26T21:18:47.018135+0000","last_undegraded":"2024-04-26T21:18:47.018135+0000","last_fullsized":"2024-04-26T21:18:47.018135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.271969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.015981+0000","last_change":"2024-04-26T21:18:44.015981+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.015981+0000","last_undegraded":"2024-04-26T21:18:44.015981+0000","last_fullsized":"2024-04-26T21:18:44.015981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018060+0000","last_change":"2024-04-26T21:18:44.024830+0000","last_active":"2024-04-26T21:18:47.018060+0000","last_peered":"2024-04-26T21:18:47.018060+0000","last_clean":"2024-04-26T21:18:47.018060+0000","last_became_active":"2024-04-26T21:18:44.024559+0000","last_became_peered":"2024-04-26T21:18:44.024559+0000","last_unstale":"2024-04-26T21:18:47.018060+0000","last_undegraded":"2024-04-26T21:18:47.018060+0000","last_fullsized":"2024-04-26T21:18:47.018060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:14.711395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024639+0000","last_change":"2024-04-26T21:18:45.023099+0000","last_active":"2024-04-26T21:18:46.024639+0000","last_peered":"2024-04-26T21:18:46.024639+0000","last_clean":"2024-04-26T21:18:46.024639+0000","last_became_active":"2024-04-26T21:18:45.022700+0000","last_became_peered":"2024-04-26T21:18:45.022700+0000","last_unstale":"2024-04-26T21:18:46.024639+0000","last_undegraded":"2024-04-26T21:18:46.024639+0000","last_fullsized":"2024-04-26T21:18:46.024639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:35.193907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024751+0000","last_change":"2024-04-26T21:18:44.023352+0000","last_active":"2024-04-26T21:18:46.024751+0000","last_peered":"2024-04-26T21:18:46.024751+0000","last_clean":"2024-04-26T21:18:46.024751+0000","last_became_active":"2024-04-26T21:18:44.023050+0000","last_became_peered":"2024-04-26T21:18:44.023050+0000","last_unstale":"2024-04-26T21:18:46.024751+0000","last_undegraded":"2024-04-26T21:18:46.024751+0000","last_fullsized":"2024-04-26T21:18:46.024751+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:35.391866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017620+0000","last_change":"2024-04-26T21:18:45.030476+0000","last_active":"2024-04-26T21:18:47.017620+0000","last_peered":"2024-04-26T21:18:47.017620+0000","last_clean":"2024-04-26T21:18:47.017620+0000","last_became_active":"2024-04-26T21:18:45.029032+0000","last_became_peered":"2024-04-26T21:18:45.029032+0000","last_unstale":"2024-04-26T21:18:47.017620+0000","last_undegraded":"2024-04-26T21:18:47.017620+0000","last_fullsized":"2024-04-26T21:18:47.017620+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:38.349495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-26T21:18:44.031512+0000","last_change":"2024-04-26T21:18:44.028840+0000","last_active":"2024-04-26T21:18:44.031512+0000","last_peered":"2024-04-26T21:18:44.031512+0000","last_clean":"2024-04-26T21:18:44.031512+0000","last_became_active":"2024-04-26T21:18:44.028560+0000","last_became_peered":"2024-04-26T21:18:44.028560+0000","last_unstale":"2024-04-26T21:18:44.031512+0000","last_undegraded":"2024-04-26T21:18:44.031512+0000","last_fullsized":"2024-04-26T21:18:44.031512+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:00.234414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634133+0000","last_change":"2024-04-26T21:18:45.027746+0000","last_active":"2024-04-26T21:18:46.634133+0000","last_peered":"2024-04-26T21:18:46.634133+0000","last_clean":"2024-04-26T21:18:46.634133+0000","last_became_active":"2024-04-26T21:18:45.027456+0000","last_became_peered":"2024-04-26T21:18:45.027456+0000","last_unstale":"2024-04-26T21:18:46.634133+0000","last_undegraded":"2024-04-26T21:18:46.634133+0000","last_fullsized":"2024-04-26T21:18:46.634133+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025108+0000","last_change":"2024-04-26T21:18:44.025797+0000","last_active":"2024-04-26T21:18:46.025108+0000","last_peered":"2024-04-26T21:18:46.025108+0000","last_clean":"2024-04-26T21:18:46.025108+0000","last_became_active":"2024-04-26T21:18:44.025461+0000","last_became_peered":"2024-04-26T21:18:44.025461+0000","last_unstale":"2024-04-26T21:18:46.025108+0000","last_undegraded":"2024-04-26T21:18:46.025108+0000","last_fullsized":"2024-04-26T21:18:46.025108+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:34.579949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669361+0000","last_change":"2024-04-26T21:18:45.022240+0000","last_active":"2024-04-26T21:18:46.669361+0000","last_peered":"2024-04-26T21:18:46.669361+0000","last_clean":"2024-04-26T21:18:46.669361+0000","last_became_active":"2024-04-26T21:18:45.019481+0000","last_became_peered":"2024-04-26T21:18:45.019481+0000","last_unstale":"2024-04-26T21:18:46.669361+0000","last_undegraded":"2024-04-26T21:18:46.669361+0000","last_fullsized":"2024-04-26T21:18:46.669361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:45.002035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633927+0000","last_change":"2024-04-26T21:18:44.023439+0000","last_active":"2024-04-26T21:18:46.633927+0000","last_peered":"2024-04-26T21:18:46.633927+0000","last_clean":"2024-04-26T21:18:46.633927+0000","last_became_active":"2024-04-26T21:18:44.023138+0000","last_became_peered":"2024-04-26T21:18:44.023138+0000","last_unstale":"2024-04-26T21:18:46.633927+0000","last_undegraded":"2024-04-26T21:18:46.633927+0000","last_fullsized":"2024-04-26T21:18:46.633927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:32.892090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124777+0000","last_change":"2024-04-26T21:18:45.034899+0000","last_active":"2024-04-26T21:18:47.124777+0000","last_peered":"2024-04-26T21:18:47.124777+0000","last_clean":"2024-04-26T21:18:47.124777+0000","last_became_active":"2024-04-26T21:18:45.034622+0000","last_became_peered":"2024-04-26T21:18:45.034622+0000","last_unstale":"2024-04-26T21:18:47.124777+0000","last_undegraded":"2024-04-26T21:18:47.124777+0000","last_fullsized":"2024-04-26T21:18:47.124777+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:35.951269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015076+0000","last_change":"2024-04-26T21:18:44.020211+0000","last_active":"2024-04-26T21:18:47.015076+0000","last_peered":"2024-04-26T21:18:47.015076+0000","last_clean":"2024-04-26T21:18:47.015076+0000","last_became_active":"2024-04-26T21:18:44.019816+0000","last_became_peered":"2024-04-26T21:18:44.019816+0000","last_unstale":"2024-04-26T21:18:47.015076+0000","last_undegraded":"2024-04-26T21:18:47.015076+0000","last_fullsized":"2024-04-26T21:18:47.015076+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:23.461801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017906+0000","last_change":"2024-04-26T21:18:45.034560+0000","last_active":"2024-04-26T21:18:47.017906+0000","last_peered":"2024-04-26T21:18:47.017906+0000","last_clean":"2024-04-26T21:18:47.017906+0000","last_became_active":"2024-04-26T21:18:45.034288+0000","last_became_peered":"2024-04-26T21:18:45.034288+0000","last_unstale":"2024-04-26T21:18:47.017906+0000","last_undegraded":"2024-04-26T21:18:47.017906+0000","last_fullsized":"2024-04-26T21:18:47.017906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:13.142781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014911+0000","last_change":"2024-04-26T21:18:44.026367+0000","last_active":"2024-04-26T21:18:47.014911+0000","last_peered":"2024-04-26T21:18:47.014911+0000","last_clean":"2024-04-26T21:18:47.014911+0000","last_became_active":"2024-04-26T21:18:44.026065+0000","last_became_peered":"2024-04-26T21:18:44.026065+0000","last_unstale":"2024-04-26T21:18:47.014911+0000","last_undegraded":"2024-04-26T21:18:47.014911+0000","last_fullsized":"2024-04-26T21:18:47.014911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:33.807940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018107+0000","last_change":"2024-04-26T21:18:45.034062+0000","last_active":"2024-04-26T21:18:47.018107+0000","last_peered":"2024-04-26T21:18:47.018107+0000","last_clean":"2024-04-26T21:18:47.018107+0000","last_became_active":"2024-04-26T21:18:45.032456+0000","last_became_peered":"2024-04-26T21:18:45.032456+0000","last_unstale":"2024-04-26T21:18:47.018107+0000","last_undegraded":"2024-04-26T21:18:47.018107+0000","last_fullsized":"2024-04-26T21:18:47.018107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:57.534083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669254+0000","last_change":"2024-04-26T21:18:44.024805+0000","last_active":"2024-04-26T21:18:46.669254+0000","last_peered":"2024-04-26T21:18:46.669254+0000","last_clean":"2024-04-26T21:18:46.669254+0000","last_became_active":"2024-04-26T21:18:44.024452+0000","last_became_peered":"2024-04-26T21:18:44.024452+0000","last_unstale":"2024-04-26T21:18:46.669254+0000","last_undegraded":"2024-04-26T21:18:46.669254+0000","last_fullsized":"2024-04-26T21:18:46.669254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:13.814207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017668+0000","last_change":"2024-04-26T21:18:45.036794+0000","last_active":"2024-04-26T21:18:47.017668+0000","last_peered":"2024-04-26T21:18:47.017668+0000","last_clean":"2024-04-26T21:18:47.017668+0000","last_became_active":"2024-04-26T21:18:45.036523+0000","last_became_peered":"2024-04-26T21:18:45.036523+0000","last_unstale":"2024-04-26T21:18:47.017668+0000","last_undegraded":"2024-04-26T21:18:47.017668+0000","last_fullsized":"2024-04-26T21:18:47.017668+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:51.318664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669204+0000","last_change":"2024-04-26T21:18:44.029908+0000","last_active":"2024-04-26T21:18:46.669204+0000","last_peered":"2024-04-26T21:18:46.669204+0000","last_clean":"2024-04-26T21:18:46.669204+0000","last_became_active":"2024-04-26T21:18:44.029533+0000","last_became_peered":"2024-04-26T21:18:44.029533+0000","last_unstale":"2024-04-26T21:18:46.669204+0000","last_undegraded":"2024-04-26T21:18:46.669204+0000","last_fullsized":"2024-04-26T21:18:46.669204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:02.622477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633630+0000","last_change":"2024-04-26T21:18:45.024869+0000","last_active":"2024-04-26T21:18:46.633630+0000","last_peered":"2024-04-26T21:18:46.633630+0000","last_clean":"2024-04-26T21:18:46.633630+0000","last_became_active":"2024-04-26T21:18:45.024583+0000","last_became_peered":"2024-04-26T21:18:45.024583+0000","last_unstale":"2024-04-26T21:18:46.633630+0000","last_undegraded":"2024-04-26T21:18:46.633630+0000","last_fullsized":"2024-04-26T21:18:46.633630+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:28.303956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014897+0000","last_change":"2024-04-26T21:18:44.030815+0000","last_active":"2024-04-26T21:18:47.014897+0000","last_peered":"2024-04-26T21:18:47.014897+0000","last_clean":"2024-04-26T21:18:47.014897+0000","last_became_active":"2024-04-26T21:18:44.030316+0000","last_became_peered":"2024-04-26T21:18:44.030316+0000","last_unstale":"2024-04-26T21:18:47.014897+0000","last_undegraded":"2024-04-26T21:18:47.014897+0000","last_fullsized":"2024-04-26T21:18:47.014897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:16.699824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015481+0000","last_change":"2024-04-26T21:18:45.027423+0000","last_active":"2024-04-26T21:18:47.015481+0000","last_peered":"2024-04-26T21:18:47.015481+0000","last_clean":"2024-04-26T21:18:47.015481+0000","last_became_active":"2024-04-26T21:18:45.027130+0000","last_became_peered":"2024-04-26T21:18:45.027130+0000","last_unstale":"2024-04-26T21:18:47.015481+0000","last_undegraded":"2024-04-26T21:18:47.015481+0000","last_fullsized":"2024-04-26T21:18:47.015481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:00.298609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669437+0000","last_change":"2024-04-26T21:18:44.023137+0000","last_active":"2024-04-26T21:18:46.669437+0000","last_peered":"2024-04-26T21:18:46.669437+0000","last_clean":"2024-04-26T21:18:46.669437+0000","last_became_active":"2024-04-26T21:18:44.022828+0000","last_became_peered":"2024-04-26T21:18:44.022828+0000","last_unstale":"2024-04-26T21:18:46.669437+0000","last_undegraded":"2024-04-26T21:18:46.669437+0000","last_fullsized":"2024-04-26T21:18:46.669437+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:16.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016765+0000","last_change":"2024-04-26T21:18:44.016765+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016765+0000","last_undegraded":"2024-04-26T21:18:44.016765+0000","last_fullsized":"2024-04-26T21:18:44.016765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017049+0000","last_change":"2024-04-26T21:18:44.026211+0000","last_active":"2024-04-26T21:18:47.017049+0000","last_peered":"2024-04-26T21:18:47.017049+0000","last_clean":"2024-04-26T21:18:47.017049+0000","last_became_active":"2024-04-26T21:18:44.025593+0000","last_became_peered":"2024-04-26T21:18:44.025593+0000","last_unstale":"2024-04-26T21:18:47.017049+0000","last_undegraded":"2024-04-26T21:18:47.017049+0000","last_fullsized":"2024-04-26T21:18:47.017049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.432059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-26T21:18:44.016771+0000","last_change":"2024-04-26T21:18:44.016771+0000","last_active":"2024-04-26T21:18:43.999199+0000","last_peered":"2024-04-26T21:18:43.999199+0000","last_clean":"2024-04-26T21:18:43.999199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:18:44.016771+0000","last_undegraded":"2024-04-26T21:18:44.016771+0000","last_fullsized":"2024-04-26T21:18:44.016771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015217+0000","last_change":"2024-04-26T21:18:44.023842+0000","last_active":"2024-04-26T21:18:47.015217+0000","last_peered":"2024-04-26T21:18:47.015217+0000","last_clean":"2024-04-26T21:18:47.015217+0000","last_became_active":"2024-04-26T21:18:44.023457+0000","last_became_peered":"2024-04-26T21:18:44.023457+0000","last_unstale":"2024-04-26T21:18:47.015217+0000","last_undegraded":"2024-04-26T21:18:47.015217+0000","last_fullsized":"2024-04-26T21:18:47.015217+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:53.284159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634004+0000","last_change":"2024-04-26T21:18:45.031489+0000","last_active":"2024-04-26T21:18:46.634004+0000","last_peered":"2024-04-26T21:18:46.634004+0000","last_clean":"2024-04-26T21:18:46.634004+0000","last_became_active":"2024-04-26T21:18:45.030496+0000","last_became_peered":"2024-04-26T21:18:45.030496+0000","last_unstale":"2024-04-26T21:18:46.634004+0000","last_undegraded":"2024-04-26T21:18:46.634004+0000","last_fullsized":"2024-04-26T21:18:46.634004+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:52.859671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634063+0000","last_change":"2024-04-26T21:18:44.027874+0000","last_active":"2024-04-26T21:18:46.634063+0000","last_peered":"2024-04-26T21:18:46.634063+0000","last_clean":"2024-04-26T21:18:46.634063+0000","last_became_active":"2024-04-26T21:18:44.027571+0000","last_became_peered":"2024-04-26T21:18:44.027571+0000","last_unstale":"2024-04-26T21:18:46.634063+0000","last_undegraded":"2024-04-26T21:18:46.634063+0000","last_fullsized":"2024-04-26T21:18:46.634063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:30.599389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018019+0000","last_change":"2024-04-26T21:18:45.030063+0000","last_active":"2024-04-26T21:18:47.018019+0000","last_peered":"2024-04-26T21:18:47.018019+0000","last_clean":"2024-04-26T21:18:47.018019+0000","last_became_active":"2024-04-26T21:18:45.029558+0000","last_became_peered":"2024-04-26T21:18:45.029558+0000","last_unstale":"2024-04-26T21:18:47.018019+0000","last_undegraded":"2024-04-26T21:18:47.018019+0000","last_fullsized":"2024-04-26T21:18:47.018019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:42.797366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015172+0000","last_change":"2024-04-26T21:18:44.022574+0000","last_active":"2024-04-26T21:18:47.015172+0000","last_peered":"2024-04-26T21:18:47.015172+0000","last_clean":"2024-04-26T21:18:47.015172+0000","last_became_active":"2024-04-26T21:18:44.021973+0000","last_became_peered":"2024-04-26T21:18:44.021973+0000","last_unstale":"2024-04-26T21:18:47.015172+0000","last_undegraded":"2024-04-26T21:18:47.015172+0000","last_fullsized":"2024-04-26T21:18:47.015172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:29.175740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024881+0000","last_change":"2024-04-26T21:18:45.021243+0000","last_active":"2024-04-26T21:18:46.024881+0000","last_peered":"2024-04-26T21:18:46.024881+0000","last_clean":"2024-04-26T21:18:46.024881+0000","last_became_active":"2024-04-26T21:18:45.020950+0000","last_became_peered":"2024-04-26T21:18:45.020950+0000","last_unstale":"2024-04-26T21:18:46.024881+0000","last_undegraded":"2024-04-26T21:18:46.024881+0000","last_fullsized":"2024-04-26T21:18:46.024881+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:17.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633658+0000","last_change":"2024-04-26T21:18:44.029874+0000","last_active":"2024-04-26T21:18:46.633658+0000","last_peered":"2024-04-26T21:18:46.633658+0000","last_clean":"2024-04-26T21:18:46.633658+0000","last_became_active":"2024-04-26T21:18:44.029509+0000","last_became_peered":"2024-04-26T21:18:44.029509+0000","last_unstale":"2024-04-26T21:18:46.633658+0000","last_undegraded":"2024-04-26T21:18:46.633658+0000","last_fullsized":"2024-04-26T21:18:46.633658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:14.132727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017578+0000","last_change":"2024-04-26T21:18:45.033715+0000","last_active":"2024-04-26T21:18:47.017578+0000","last_peered":"2024-04-26T21:18:47.017578+0000","last_clean":"2024-04-26T21:18:47.017578+0000","last_became_active":"2024-04-26T21:18:45.032114+0000","last_became_peered":"2024-04-26T21:18:45.032114+0000","last_unstale":"2024-04-26T21:18:47.017578+0000","last_undegraded":"2024-04-26T21:18:47.017578+0000","last_fullsized":"2024-04-26T21:18:47.017578+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:22.740080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024906+0000","last_change":"2024-04-26T21:18:44.025333+0000","last_active":"2024-04-26T21:18:46.024906+0000","last_peered":"2024-04-26T21:18:46.024906+0000","last_clean":"2024-04-26T21:18:46.024906+0000","last_became_active":"2024-04-26T21:18:44.025063+0000","last_became_peered":"2024-04-26T21:18:44.025063+0000","last_unstale":"2024-04-26T21:18:46.024906+0000","last_undegraded":"2024-04-26T21:18:46.024906+0000","last_fullsized":"2024-04-26T21:18:46.024906+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:50.619652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633740+0000","last_change":"2024-04-26T21:18:45.030906+0000","last_active":"2024-04-26T21:18:46.633740+0000","last_peered":"2024-04-26T21:18:46.633740+0000","last_clean":"2024-04-26T21:18:46.633740+0000","last_became_active":"2024-04-26T21:18:45.030635+0000","last_became_peered":"2024-04-26T21:18:45.030635+0000","last_unstale":"2024-04-26T21:18:46.633740+0000","last_undegraded":"2024-04-26T21:18:46.633740+0000","last_fullsized":"2024-04-26T21:18:46.633740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:54.286661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017704+0000","last_change":"2024-04-26T21:18:44.025669+0000","last_active":"2024-04-26T21:18:47.017704+0000","last_peered":"2024-04-26T21:18:47.017704+0000","last_clean":"2024-04-26T21:18:47.017704+0000","last_became_active":"2024-04-26T21:18:44.025366+0000","last_became_peered":"2024-04-26T21:18:44.025366+0000","last_unstale":"2024-04-26T21:18:47.017704+0000","last_undegraded":"2024-04-26T21:18:47.017704+0000","last_fullsized":"2024-04-26T21:18:47.017704+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:11.988615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018188+0000","last_change":"2024-04-26T21:18:45.032849+0000","last_active":"2024-04-26T21:18:47.018188+0000","last_peered":"2024-04-26T21:18:47.018188+0000","last_clean":"2024-04-26T21:18:47.018188+0000","last_became_active":"2024-04-26T21:18:45.028723+0000","last_became_peered":"2024-04-26T21:18:45.028723+0000","last_unstale":"2024-04-26T21:18:47.018188+0000","last_undegraded":"2024-04-26T21:18:47.018188+0000","last_fullsized":"2024-04-26T21:18:47.018188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:01.743888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124635+0000","last_change":"2024-04-26T21:18:44.027770+0000","last_active":"2024-04-26T21:18:47.124635+0000","last_peered":"2024-04-26T21:18:47.124635+0000","last_clean":"2024-04-26T21:18:47.124635+0000","last_became_active":"2024-04-26T21:18:44.027461+0000","last_became_peered":"2024-04-26T21:18:44.027461+0000","last_unstale":"2024-04-26T21:18:47.124635+0000","last_undegraded":"2024-04-26T21:18:47.124635+0000","last_fullsized":"2024-04-26T21:18:47.124635+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:16.699824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124179+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:18:47.124179+0000","last_peered":"2024-04-26T21:18:47.124179+0000","last_clean":"2024-04-26T21:18:47.124179+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:18:47.124179+0000","last_undegraded":"2024-04-26T21:18:47.124179+0000","last_fullsized":"2024-04-26T21:18:47.124179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1098319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":508491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":509575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":509473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1789952,"data_stored":1098789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1167360,"data_stored":494764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:50.669 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:18:50.668+0000 7fb2ef917640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:18:52.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:18:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 Processor -- start 2024-04-26T21:18:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 -- start start 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa37c0d3ee0 0x7fa37c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa37c15f750 0x7fa37c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37c0d03a0 con 0x7fa37c15f370 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c0b4d10 con 0x7fa37c15f750 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.592+0000 7fa381f11640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c0d0680 con 0x7fa37c0d3ee0 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa37c15f750 0x7fa37c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa37c15f750 0x7fa37c0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57340/0 (socket says 172.21.15.40:57340) 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 -- 172.21.15.40:0/3398056752 learned_addr learned my addr 172.21.15.40:0/3398056752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37affd640 1 --2- 172.21.15.40:0/3398056752 >> v2:172.21.15.184:3300/0 conn(0x7fa37c0d3ee0 0x7fa37c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 -- 172.21.15.40:0/3398056752 >> v2:172.21.15.184:3300/0 conn(0x7fa37c0d3ee0 msgr2=0x7fa37c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 --2- 172.21.15.40:0/3398056752 >> v2:172.21.15.184:3300/0 conn(0x7fa37c0d3ee0 0x7fa37c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 -- 172.21.15.40:0/3398056752 >> v1:172.21.15.184:6789/0 conn(0x7fa37c15f370 legacy=0x7fa37c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.593+0000 7fa37bfff640 1 -- 172.21.15.40:0/3398056752 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c0d0960 con 0x7fa37c15f750 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa37bfff640 1 --2- 172.21.15.40:0/3398056752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f750 0x7fa37c0b47d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa374009e50 tx=0x7fa37402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf71179955c7c425 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa37a7fc640 1 -- 172.21.15.40:0/3398056752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374036070 con 0x7fa37c15f750 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa37a7fc640 1 -- 172.21.15.40:0/3398056752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa374031600 con 0x7fa37c15f750 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa37a7fc640 1 -- 172.21.15.40:0/3398056752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3740319d0 con 0x7fa37c15f750 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 -- 172.21.15.40:0/3398056752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f750 msgr2=0x7fa37c0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 --2- 172.21.15.40:0/3398056752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f750 0x7fa37c0b47d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa374009e50 tx=0x7fa37402a760 comp rx=0 tx=0).stop 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 -- 172.21.15.40:0/3398056752 shutdown_connections 2024-04-26T21:18:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 --2- 172.21.15.40:0/3398056752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f750 0x7fa37c0b47d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 --2- 172.21.15.40:0/3398056752 >> v2:172.21.15.184:3300/0 conn(0x7fa37c0d3ee0 0x7fa37c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.594+0000 7fa381f11640 1 -- 172.21.15.40:0/3398056752 >> 172.21.15.40:0/3398056752 conn(0x7fa37c08ec10 msgr2=0x7fa37c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.595+0000 7fa381f11640 1 -- 172.21.15.40:0/3398056752 shutdown_connections 2024-04-26T21:18:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.595+0000 7fa381f11640 1 -- 172.21.15.40:0/3398056752 wait complete. 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.596+0000 7fa381f11640 1 Processor -- start 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.596+0000 7fa381f11640 1 -- start start 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 0x7fa37c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37c15f750 0x7fa37c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0b51c0 con 0x7fa37c15f370 2024-04-26T21:18:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0d0f20 con 0x7fa37c0d3ee0 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa381f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c0b5340 con 0x7fa37c15f750 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57344/0 (socket says 172.21.15.40:57344) 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 -- 172.21.15.40:0/3479588318 learned_addr learned my addr 172.21.15.40:0/3479588318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37bfff640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37c15f750 0x7fa37c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37b7fe640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 0x7fa37c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 -- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37c15f750 msgr2=0x7fa37c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37c15f750 0x7fa37c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 -- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 msgr2=0x7fa37c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 0x7fa37c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 -- 172.21.15.40:0/3479588318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c0c9de0 con 0x7fa37c15f370 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37b7fe640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 0x7fa37c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa37affd640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa37000e8b0 tx=0x7fa37000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.597+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370017070 con 0x7fa37c15f370 2024-04-26T21:18:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.598+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37000c8e0 con 0x7fa37c15f370 2024-04-26T21:18:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.598+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c0ca0c0 con 0x7fa37c15f370 2024-04-26T21:18:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.598+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000ccd0 con 0x7fa37c15f370 2024-04-26T21:18:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.598+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c0ca680 con 0x7fa37c15f370 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.599+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c069ae0 con 0x7fa37c15f370 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.599+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa37001a070 con 0x7fa37c15f370 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.600+0000 7fa363fff640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 0x7fa36806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.600+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fa37008bd70 con 0x7fa37c15f370 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.600+0000 7fa37b7fe640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 0x7fa36806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.600+0000 7fa37b7fe640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 0x7fa36806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa36c001700 tx=0x7fa36c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.604+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa37c069ae0 con 0x7fa37c15f370 2024-04-26T21:18:52.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.786+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa37c06a290 con 0x7fa36806c8f0 2024-04-26T21:18:52.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.794+0000 7fa363fff640 1 -- 172.21.15.40:0/3479588318 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252105 (secure 0 0 0) 0x7fa37c06a290 con 0x7fa36806c8f0 2024-04-26T21:18:52.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:52.799 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 msgr2=0x7fa36806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 0x7fa36806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa36c001700 tx=0x7fa36c009040 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 msgr2=0x7fa37c15d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa37000e8b0 tx=0x7fa37000ed70 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 shutdown_connections 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa36806c8f0 0x7fa36806edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa37c15f750 0x7fa37c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa37c15f370 0x7fa37c15d7e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 --2- 172.21.15.40:0/3479588318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0d3ee0 0x7fa37c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 >> 172.21.15.40:0/3479588318 conn(0x7fa37c08ec10 msgr2=0x7fa37c06e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 shutdown_connections 2024-04-26T21:18:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.796+0000 7fa381f11640 1 -- 172.21.15.40:0/3479588318 wait complete. 2024-04-26T21:18:52.809 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-04-26T21:18:51.017468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264172,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710548,"statfs":{"total":773094113280,"available":772823601152,"internally_reserved":0,"allocated":10788864,"data_stored":5248268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.007471"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025612+0000","last_change":"2024-04-26T21:18:45.040167+0000","last_active":"2024-04-26T21:18:46.025612+0000","last_peered":"2024-04-26T21:18:46.025612+0000","last_clean":"2024-04-26T21:18:46.025612+0000","last_became_active":"2024-04-26T21:18:45.039871+0000","last_became_peered":"2024-04-26T21:18:45.039871+0000","last_unstale":"2024-04-26T21:18:46.025612+0000","last_undegraded":"2024-04-26T21:18:46.025612+0000","last_fullsized":"2024-04-26T21:18:46.025612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:58.215420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015284+0000","last_change":"2024-04-26T21:18:45.028537+0000","last_active":"2024-04-26T21:18:47.015284+0000","last_peered":"2024-04-26T21:18:47.015284+0000","last_clean":"2024-04-26T21:18:47.015284+0000","last_became_active":"2024-04-26T21:18:45.028229+0000","last_became_peered":"2024-04-26T21:18:45.028229+0000","last_unstale":"2024-04-26T21:18:47.015284+0000","last_undegraded":"2024-04-26T21:18:47.015284+0000","last_fullsized":"2024-04-26T21:18:47.015284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:06.291628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025578+0000","last_change":"2024-04-26T21:18:45.038833+0000","last_active":"2024-04-26T21:18:46.025578+0000","last_peered":"2024-04-26T21:18:46.025578+0000","last_clean":"2024-04-26T21:18:46.025578+0000","last_became_active":"2024-04-26T21:18:45.038546+0000","last_became_peered":"2024-04-26T21:18:45.038546+0000","last_unstale":"2024-04-26T21:18:46.025578+0000","last_undegraded":"2024-04-26T21:18:46.025578+0000","last_fullsized":"2024-04-26T21:18:46.025578+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:16.477871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025293+0000","last_change":"2024-04-26T21:18:45.027534+0000","last_active":"2024-04-26T21:18:46.025293+0000","last_peered":"2024-04-26T21:18:46.025293+0000","last_clean":"2024-04-26T21:18:46.025293+0000","last_became_active":"2024-04-26T21:18:45.027256+0000","last_became_peered":"2024-04-26T21:18:45.027256+0000","last_unstale":"2024-04-26T21:18:46.025293+0000","last_undegraded":"2024-04-26T21:18:46.025293+0000","last_fullsized":"2024-04-26T21:18:46.025293+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633905+0000","last_change":"2024-04-26T21:18:45.032479+0000","last_active":"2024-04-26T21:18:46.633905+0000","last_peered":"2024-04-26T21:18:46.633905+0000","last_clean":"2024-04-26T21:18:46.633905+0000","last_became_active":"2024-04-26T21:18:45.032219+0000","last_became_peered":"2024-04-26T21:18:45.032219+0000","last_unstale":"2024-04-26T21:18:46.633905+0000","last_undegraded":"2024-04-26T21:18:46.633905+0000","last_fullsized":"2024-04-26T21:18:46.633905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:30.178243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124480+0000","last_change":"2024-04-26T21:18:45.023238+0000","last_active":"2024-04-26T21:18:47.124480+0000","last_peered":"2024-04-26T21:18:47.124480+0000","last_clean":"2024-04-26T21:18:47.124480+0000","last_became_active":"2024-04-26T21:18:45.022988+0000","last_became_peered":"2024-04-26T21:18:45.022988+0000","last_unstale":"2024-04-26T21:18:47.124480+0000","last_undegraded":"2024-04-26T21:18:47.124480+0000","last_fullsized":"2024-04-26T21:18:47.124480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:20.109826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124470+0000","last_change":"2024-04-26T21:18:45.027725+0000","last_active":"2024-04-26T21:18:47.124470+0000","last_peered":"2024-04-26T21:18:47.124470+0000","last_clean":"2024-04-26T21:18:47.124470+0000","last_became_active":"2024-04-26T21:18:45.027450+0000","last_became_peered":"2024-04-26T21:18:45.027450+0000","last_unstale":"2024-04-26T21:18:47.124470+0000","last_undegraded":"2024-04-26T21:18:47.124470+0000","last_fullsized":"2024-04-26T21:18:47.124470+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015142+0000","last_change":"2024-04-26T21:18:45.028833+0000","last_active":"2024-04-26T21:18:47.015142+0000","last_peered":"2024-04-26T21:18:47.015142+0000","last_clean":"2024-04-26T21:18:47.015142+0000","last_became_active":"2024-04-26T21:18:45.028509+0000","last_became_peered":"2024-04-26T21:18:45.028509+0000","last_unstale":"2024-04-26T21:18:47.015142+0000","last_undegraded":"2024-04-26T21:18:47.015142+0000","last_fullsized":"2024-04-26T21:18:47.015142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:12.695915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025269+0000","last_change":"2024-04-26T21:18:45.031345+0000","last_active":"2024-04-26T21:18:46.025269+0000","last_peered":"2024-04-26T21:18:46.025269+0000","last_clean":"2024-04-26T21:18:46.025269+0000","last_became_active":"2024-04-26T21:18:45.031042+0000","last_became_peered":"2024-04-26T21:18:45.031042+0000","last_unstale":"2024-04-26T21:18:46.025269+0000","last_undegraded":"2024-04-26T21:18:46.025269+0000","last_fullsized":"2024-04-26T21:18:46.025269+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:24.464304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634010+0000","last_change":"2024-04-26T21:18:45.024931+0000","last_active":"2024-04-26T21:18:46.634010+0000","last_peered":"2024-04-26T21:18:46.634010+0000","last_clean":"2024-04-26T21:18:46.634010+0000","last_became_active":"2024-04-26T21:18:45.024628+0000","last_became_peered":"2024-04-26T21:18:45.024628+0000","last_unstale":"2024-04-26T21:18:46.634010+0000","last_undegraded":"2024-04-26T21:18:46.634010+0000","last_fullsized":"2024-04-26T21:18:46.634010+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:56.384292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124474+0000","last_change":"2024-04-26T21:18:45.028433+0000","last_active":"2024-04-26T21:18:47.124474+0000","last_peered":"2024-04-26T21:18:47.124474+0000","last_clean":"2024-04-26T21:18:47.124474+0000","last_became_active":"2024-04-26T21:18:45.028113+0000","last_became_peered":"2024-04-26T21:18:45.028113+0000","last_unstale":"2024-04-26T21:18:47.124474+0000","last_undegraded":"2024-04-26T21:18:47.124474+0000","last_fullsized":"2024-04-26T21:18:47.124474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:26.756819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025096+0000","last_change":"2024-04-26T21:18:45.024858+0000","last_active":"2024-04-26T21:18:46.025096+0000","last_peered":"2024-04-26T21:18:46.025096+0000","last_clean":"2024-04-26T21:18:46.025096+0000","last_became_active":"2024-04-26T21:18:45.024560+0000","last_became_peered":"2024-04-26T21:18:45.024560+0000","last_unstale":"2024-04-26T21:18:46.025096+0000","last_undegraded":"2024-04-26T21:18:46.025096+0000","last_fullsized":"2024-04-26T21:18:46.025096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.434562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025139+0000","last_change":"2024-04-26T21:18:45.035850+0000","last_active":"2024-04-26T21:18:46.025139+0000","last_peered":"2024-04-26T21:18:46.025139+0000","last_clean":"2024-04-26T21:18:46.025139+0000","last_became_active":"2024-04-26T21:18:45.035565+0000","last_became_peered":"2024-04-26T21:18:45.035565+0000","last_unstale":"2024-04-26T21:18:46.025139+0000","last_undegraded":"2024-04-26T21:18:46.025139+0000","last_fullsized":"2024-04-26T21:18:46.025139+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:52.859671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017637+0000","last_change":"2024-04-26T21:18:45.029826+0000","last_active":"2024-04-26T21:18:47.017637+0000","last_peered":"2024-04-26T21:18:47.017637+0000","last_clean":"2024-04-26T21:18:47.017637+0000","last_became_active":"2024-04-26T21:18:45.029236+0000","last_became_peered":"2024-04-26T21:18:45.029236+0000","last_unstale":"2024-04-26T21:18:47.017637+0000","last_undegraded":"2024-04-26T21:18:47.017637+0000","last_fullsized":"2024-04-26T21:18:47.017637+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:02.758562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024802+0000","last_change":"2024-04-26T21:18:45.036314+0000","last_active":"2024-04-26T21:18:46.024802+0000","last_peered":"2024-04-26T21:18:46.024802+0000","last_clean":"2024-04-26T21:18:46.024802+0000","last_became_active":"2024-04-26T21:18:45.035995+0000","last_became_peered":"2024-04-26T21:18:45.035995+0000","last_unstale":"2024-04-26T21:18:46.024802+0000","last_undegraded":"2024-04-26T21:18:46.024802+0000","last_fullsized":"2024-04-26T21:18:46.024802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:30.178243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124549+0000","last_change":"2024-04-26T21:18:45.029801+0000","last_active":"2024-04-26T21:18:47.124549+0000","last_peered":"2024-04-26T21:18:47.124549+0000","last_clean":"2024-04-26T21:18:47.124549+0000","last_became_active":"2024-04-26T21:18:45.029535+0000","last_became_peered":"2024-04-26T21:18:45.029535+0000","last_unstale":"2024-04-26T21:18:47.124549+0000","last_undegraded":"2024-04-26T21:18:47.124549+0000","last_fullsized":"2024-04-26T21:18:47.124549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:23.849073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015536+0000","last_change":"2024-04-26T21:18:45.037043+0000","last_active":"2024-04-26T21:18:47.015536+0000","last_peered":"2024-04-26T21:18:47.015536+0000","last_clean":"2024-04-26T21:18:47.015536+0000","last_became_active":"2024-04-26T21:18:45.036743+0000","last_became_peered":"2024-04-26T21:18:45.036743+0000","last_unstale":"2024-04-26T21:18:47.015536+0000","last_undegraded":"2024-04-26T21:18:47.015536+0000","last_fullsized":"2024-04-26T21:18:47.015536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:43.951451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024809+0000","last_change":"2024-04-26T21:18:45.025803+0000","last_active":"2024-04-26T21:18:46.024809+0000","last_peered":"2024-04-26T21:18:46.024809+0000","last_clean":"2024-04-26T21:18:46.024809+0000","last_became_active":"2024-04-26T21:18:45.025416+0000","last_became_peered":"2024-04-26T21:18:45.025416+0000","last_unstale":"2024-04-26T21:18:46.024809+0000","last_undegraded":"2024-04-26T21:18:46.024809+0000","last_fullsized":"2024-04-26T21:18:46.024809+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:11.034280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018083+0000","last_change":"2024-04-26T21:18:45.024838+0000","last_active":"2024-04-26T21:18:47.018083+0000","last_peered":"2024-04-26T21:18:47.018083+0000","last_clean":"2024-04-26T21:18:47.018083+0000","last_became_active":"2024-04-26T21:18:45.024509+0000","last_became_peered":"2024-04-26T21:18:45.024509+0000","last_unstale":"2024-04-26T21:18:47.018083+0000","last_undegraded":"2024-04-26T21:18:47.018083+0000","last_fullsized":"2024-04-26T21:18:47.018083+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:10.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025241+0000","last_change":"2024-04-26T21:18:45.031847+0000","last_active":"2024-04-26T21:18:46.025241+0000","last_peered":"2024-04-26T21:18:46.025241+0000","last_clean":"2024-04-26T21:18:46.025241+0000","last_became_active":"2024-04-26T21:18:45.030655+0000","last_became_peered":"2024-04-26T21:18:45.030655+0000","last_unstale":"2024-04-26T21:18:46.025241+0000","last_undegraded":"2024-04-26T21:18:46.025241+0000","last_fullsized":"2024-04-26T21:18:46.025241+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:17.288331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017962+0000","last_change":"2024-04-26T21:18:45.030714+0000","last_active":"2024-04-26T21:18:47.017962+0000","last_peered":"2024-04-26T21:18:47.017962+0000","last_clean":"2024-04-26T21:18:47.017962+0000","last_became_active":"2024-04-26T21:18:45.028965+0000","last_became_peered":"2024-04-26T21:18:45.028965+0000","last_unstale":"2024-04-26T21:18:47.017962+0000","last_undegraded":"2024-04-26T21:18:47.017962+0000","last_fullsized":"2024-04-26T21:18:47.017962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:45:01.726761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014814+0000","last_change":"2024-04-26T21:18:45.030869+0000","last_active":"2024-04-26T21:18:47.014814+0000","last_peered":"2024-04-26T21:18:47.014814+0000","last_clean":"2024-04-26T21:18:47.014814+0000","last_became_active":"2024-04-26T21:18:45.030616+0000","last_became_peered":"2024-04-26T21:18:45.030616+0000","last_unstale":"2024-04-26T21:18:47.014814+0000","last_undegraded":"2024-04-26T21:18:47.014814+0000","last_fullsized":"2024-04-26T21:18:47.014814+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:11.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015024+0000","last_change":"2024-04-26T21:18:45.030225+0000","last_active":"2024-04-26T21:18:47.015024+0000","last_peered":"2024-04-26T21:18:47.015024+0000","last_clean":"2024-04-26T21:18:47.015024+0000","last_became_active":"2024-04-26T21:18:45.029962+0000","last_became_peered":"2024-04-26T21:18:45.029962+0000","last_unstale":"2024-04-26T21:18:47.015024+0000","last_undegraded":"2024-04-26T21:18:47.015024+0000","last_fullsized":"2024-04-26T21:18:47.015024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:35.951269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124568+0000","last_change":"2024-04-26T21:18:45.039040+0000","last_active":"2024-04-26T21:18:47.124568+0000","last_peered":"2024-04-26T21:18:47.124568+0000","last_clean":"2024-04-26T21:18:47.124568+0000","last_became_active":"2024-04-26T21:18:45.038765+0000","last_became_peered":"2024-04-26T21:18:45.038765+0000","last_unstale":"2024-04-26T21:18:47.124568+0000","last_undegraded":"2024-04-26T21:18:47.124568+0000","last_fullsized":"2024-04-26T21:18:47.124568+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:59.961187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634109+0000","last_change":"2024-04-26T21:18:45.029767+0000","last_active":"2024-04-26T21:18:46.634109+0000","last_peered":"2024-04-26T21:18:46.634109+0000","last_clean":"2024-04-26T21:18:46.634109+0000","last_became_active":"2024-04-26T21:18:45.029316+0000","last_became_peered":"2024-04-26T21:18:45.029316+0000","last_unstale":"2024-04-26T21:18:46.634109+0000","last_undegraded":"2024-04-26T21:18:46.634109+0000","last_fullsized":"2024-04-26T21:18:46.634109+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015228+0000","last_change":"2024-04-26T21:18:45.028396+0000","last_active":"2024-04-26T21:18:47.015228+0000","last_peered":"2024-04-26T21:18:47.015228+0000","last_clean":"2024-04-26T21:18:47.015228+0000","last_became_active":"2024-04-26T21:18:45.028127+0000","last_became_peered":"2024-04-26T21:18:45.028127+0000","last_unstale":"2024-04-26T21:18:47.015228+0000","last_undegraded":"2024-04-26T21:18:47.015228+0000","last_fullsized":"2024-04-26T21:18:47.015228+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:47.380274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124674+0000","last_change":"2024-04-26T21:18:45.024115+0000","last_active":"2024-04-26T21:18:47.124674+0000","last_peered":"2024-04-26T21:18:47.124674+0000","last_clean":"2024-04-26T21:18:47.124674+0000","last_became_active":"2024-04-26T21:18:45.023816+0000","last_became_peered":"2024-04-26T21:18:45.023816+0000","last_unstale":"2024-04-26T21:18:47.124674+0000","last_undegraded":"2024-04-26T21:18:47.124674+0000","last_fullsized":"2024-04-26T21:18:47.124674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:11.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633928+0000","last_change":"2024-04-26T21:18:45.031007+0000","last_active":"2024-04-26T21:18:46.633928+0000","last_peered":"2024-04-26T21:18:46.633928+0000","last_clean":"2024-04-26T21:18:46.633928+0000","last_became_active":"2024-04-26T21:18:45.030689+0000","last_became_peered":"2024-04-26T21:18:45.030689+0000","last_unstale":"2024-04-26T21:18:46.633928+0000","last_undegraded":"2024-04-26T21:18:46.633928+0000","last_fullsized":"2024-04-26T21:18:46.633928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:11.034280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014986+0000","last_change":"2024-04-26T21:18:45.039935+0000","last_active":"2024-04-26T21:18:47.014986+0000","last_peered":"2024-04-26T21:18:47.014986+0000","last_clean":"2024-04-26T21:18:47.014986+0000","last_became_active":"2024-04-26T21:18:45.039657+0000","last_became_peered":"2024-04-26T21:18:45.039657+0000","last_unstale":"2024-04-26T21:18:47.014986+0000","last_undegraded":"2024-04-26T21:18:47.014986+0000","last_fullsized":"2024-04-26T21:18:47.014986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:16.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017621+0000","last_change":"2024-04-26T21:18:45.025483+0000","last_active":"2024-04-26T21:18:47.017621+0000","last_peered":"2024-04-26T21:18:47.017621+0000","last_clean":"2024-04-26T21:18:47.017621+0000","last_became_active":"2024-04-26T21:18:45.025217+0000","last_became_peered":"2024-04-26T21:18:45.025217+0000","last_unstale":"2024-04-26T21:18:47.017621+0000","last_undegraded":"2024-04-26T21:18:47.017621+0000","last_fullsized":"2024-04-26T21:18:47.017621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:30.599056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024925+0000","last_change":"2024-04-26T21:18:45.030588+0000","last_active":"2024-04-26T21:18:46.024925+0000","last_peered":"2024-04-26T21:18:46.024925+0000","last_clean":"2024-04-26T21:18:46.024925+0000","last_became_active":"2024-04-26T21:18:45.030329+0000","last_became_peered":"2024-04-26T21:18:45.030329+0000","last_unstale":"2024-04-26T21:18:46.024925+0000","last_undegraded":"2024-04-26T21:18:46.024925+0000","last_fullsized":"2024-04-26T21:18:46.024925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:56.384292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025068+0000","last_change":"2024-04-26T21:18:45.042786+0000","last_active":"2024-04-26T21:18:46.025068+0000","last_peered":"2024-04-26T21:18:46.025068+0000","last_clean":"2024-04-26T21:18:46.025068+0000","last_became_active":"2024-04-26T21:18:45.042069+0000","last_became_peered":"2024-04-26T21:18:45.042069+0000","last_unstale":"2024-04-26T21:18:46.025068+0000","last_undegraded":"2024-04-26T21:18:46.025068+0000","last_fullsized":"2024-04-26T21:18:46.025068+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:59.042515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018187+0000","last_change":"2024-04-26T21:18:45.024508+0000","last_active":"2024-04-26T21:18:47.018187+0000","last_peered":"2024-04-26T21:18:47.018187+0000","last_clean":"2024-04-26T21:18:47.018187+0000","last_became_active":"2024-04-26T21:18:45.024246+0000","last_became_peered":"2024-04-26T21:18:45.024246+0000","last_unstale":"2024-04-26T21:18:47.018187+0000","last_undegraded":"2024-04-26T21:18:47.018187+0000","last_fullsized":"2024-04-26T21:18:47.018187+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:17.239874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015588+0000","last_change":"2024-04-26T21:18:44.028474+0000","last_active":"2024-04-26T21:18:47.015588+0000","last_peered":"2024-04-26T21:18:47.015588+0000","last_clean":"2024-04-26T21:18:47.015588+0000","last_became_active":"2024-04-26T21:18:44.028207+0000","last_became_peered":"2024-04-26T21:18:44.028207+0000","last_unstale":"2024-04-26T21:18:47.015588+0000","last_undegraded":"2024-04-26T21:18:47.015588+0000","last_fullsized":"2024-04-26T21:18:47.015588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:58.541139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025468+0000","last_change":"2024-04-26T21:18:45.034871+0000","last_active":"2024-04-26T21:18:46.025468+0000","last_peered":"2024-04-26T21:18:46.025468+0000","last_clean":"2024-04-26T21:18:46.025468+0000","last_became_active":"2024-04-26T21:18:45.034602+0000","last_became_peered":"2024-04-26T21:18:45.034602+0000","last_unstale":"2024-04-26T21:18:46.025468+0000","last_undegraded":"2024-04-26T21:18:46.025468+0000","last_fullsized":"2024-04-26T21:18:46.025468+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124650+0000","last_change":"2024-04-26T21:18:44.030474+0000","last_active":"2024-04-26T21:18:47.124650+0000","last_peered":"2024-04-26T21:18:47.124650+0000","last_clean":"2024-04-26T21:18:47.124650+0000","last_became_active":"2024-04-26T21:18:44.029983+0000","last_became_peered":"2024-04-26T21:18:44.029983+0000","last_unstale":"2024-04-26T21:18:47.124650+0000","last_undegraded":"2024-04-26T21:18:47.124650+0000","last_fullsized":"2024-04-26T21:18:47.124650+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.432059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025368+0000","last_change":"2024-04-26T21:18:45.040810+0000","last_active":"2024-04-26T21:18:46.025368+0000","last_peered":"2024-04-26T21:18:46.025368+0000","last_clean":"2024-04-26T21:18:46.025368+0000","last_became_active":"2024-04-26T21:18:45.040522+0000","last_became_peered":"2024-04-26T21:18:45.040522+0000","last_unstale":"2024-04-26T21:18:46.025368+0000","last_undegraded":"2024-04-26T21:18:46.025368+0000","last_fullsized":"2024-04-26T21:18:46.025368+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:57.090982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015134+0000","last_change":"2024-04-26T21:18:44.021771+0000","last_active":"2024-04-26T21:18:47.015134+0000","last_peered":"2024-04-26T21:18:47.015134+0000","last_clean":"2024-04-26T21:18:47.015134+0000","last_became_active":"2024-04-26T21:18:44.021492+0000","last_became_peered":"2024-04-26T21:18:44.021492+0000","last_unstale":"2024-04-26T21:18:47.015134+0000","last_undegraded":"2024-04-26T21:18:47.015134+0000","last_fullsized":"2024-04-26T21:18:47.015134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:27.301453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025282+0000","last_change":"2024-04-26T21:18:45.023563+0000","last_active":"2024-04-26T21:18:46.025282+0000","last_peered":"2024-04-26T21:18:46.025282+0000","last_clean":"2024-04-26T21:18:46.025282+0000","last_became_active":"2024-04-26T21:18:45.023300+0000","last_became_peered":"2024-04-26T21:18:45.023300+0000","last_unstale":"2024-04-26T21:18:46.025282+0000","last_undegraded":"2024-04-26T21:18:46.025282+0000","last_fullsized":"2024-04-26T21:18:46.025282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:34.810442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014986+0000","last_change":"2024-04-26T21:18:44.023307+0000","last_active":"2024-04-26T21:18:47.014986+0000","last_peered":"2024-04-26T21:18:47.014986+0000","last_clean":"2024-04-26T21:18:47.014986+0000","last_became_active":"2024-04-26T21:18:44.023006+0000","last_became_peered":"2024-04-26T21:18:44.023006+0000","last_unstale":"2024-04-26T21:18:47.014986+0000","last_undegraded":"2024-04-26T21:18:47.014986+0000","last_fullsized":"2024-04-26T21:18:47.014986+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:16.285828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669363+0000","last_change":"2024-04-26T21:18:45.014108+0000","last_active":"2024-04-26T21:18:46.669363+0000","last_peered":"2024-04-26T21:18:46.669363+0000","last_clean":"2024-04-26T21:18:46.669363+0000","last_became_active":"2024-04-26T21:18:45.013852+0000","last_became_peered":"2024-04-26T21:18:45.013852+0000","last_unstale":"2024-04-26T21:18:46.669363+0000","last_undegraded":"2024-04-26T21:18:46.669363+0000","last_fullsized":"2024-04-26T21:18:46.669363+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:15.135229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025423+0000","last_change":"2024-04-26T21:18:44.024416+0000","last_active":"2024-04-26T21:18:46.025423+0000","last_peered":"2024-04-26T21:18:46.025423+0000","last_clean":"2024-04-26T21:18:46.025423+0000","last_became_active":"2024-04-26T21:18:44.024129+0000","last_became_peered":"2024-04-26T21:18:44.024129+0000","last_unstale":"2024-04-26T21:18:46.025423+0000","last_undegraded":"2024-04-26T21:18:46.025423+0000","last_fullsized":"2024-04-26T21:18:46.025423+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:49.290460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124465+0000","last_change":"2024-04-26T21:18:45.026037+0000","last_active":"2024-04-26T21:18:47.124465+0000","last_peered":"2024-04-26T21:18:47.124465+0000","last_clean":"2024-04-26T21:18:47.124465+0000","last_became_active":"2024-04-26T21:18:45.025727+0000","last_became_peered":"2024-04-26T21:18:45.025727+0000","last_unstale":"2024-04-26T21:18:47.124465+0000","last_undegraded":"2024-04-26T21:18:47.124465+0000","last_fullsized":"2024-04-26T21:18:47.124465+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:00.484065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014954+0000","last_change":"2024-04-26T21:18:44.024846+0000","last_active":"2024-04-26T21:18:47.014954+0000","last_peered":"2024-04-26T21:18:47.014954+0000","last_clean":"2024-04-26T21:18:47.014954+0000","last_became_active":"2024-04-26T21:18:44.024565+0000","last_became_peered":"2024-04-26T21:18:44.024565+0000","last_unstale":"2024-04-26T21:18:47.014954+0000","last_undegraded":"2024-04-26T21:18:47.014954+0000","last_fullsized":"2024-04-26T21:18:47.014954+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:34.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024686+0000","last_change":"2024-04-26T21:18:45.038538+0000","last_active":"2024-04-26T21:18:46.024686+0000","last_peered":"2024-04-26T21:18:46.024686+0000","last_clean":"2024-04-26T21:18:46.024686+0000","last_became_active":"2024-04-26T21:18:45.038279+0000","last_became_peered":"2024-04-26T21:18:45.038279+0000","last_unstale":"2024-04-26T21:18:46.024686+0000","last_undegraded":"2024-04-26T21:18:46.024686+0000","last_fullsized":"2024-04-26T21:18:46.024686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:54.286661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633861+0000","last_change":"2024-04-26T21:18:44.023069+0000","last_active":"2024-04-26T21:18:46.633861+0000","last_peered":"2024-04-26T21:18:46.633861+0000","last_clean":"2024-04-26T21:18:46.633861+0000","last_became_active":"2024-04-26T21:18:44.022719+0000","last_became_peered":"2024-04-26T21:18:44.022719+0000","last_unstale":"2024-04-26T21:18:46.633861+0000","last_undegraded":"2024-04-26T21:18:46.633861+0000","last_fullsized":"2024-04-26T21:18:46.633861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:13.337074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024639+0000","last_change":"2024-04-26T21:18:45.027344+0000","last_active":"2024-04-26T21:18:46.024639+0000","last_peered":"2024-04-26T21:18:46.024639+0000","last_clean":"2024-04-26T21:18:46.024639+0000","last_became_active":"2024-04-26T21:18:45.026788+0000","last_became_peered":"2024-04-26T21:18:45.026788+0000","last_unstale":"2024-04-26T21:18:46.024639+0000","last_undegraded":"2024-04-26T21:18:46.024639+0000","last_fullsized":"2024-04-26T21:18:46.024639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:59.543642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025050+0000","last_change":"2024-04-26T21:18:44.025572+0000","last_active":"2024-04-26T21:18:46.025050+0000","last_peered":"2024-04-26T21:18:46.025050+0000","last_clean":"2024-04-26T21:18:46.025050+0000","last_became_active":"2024-04-26T21:18:44.025260+0000","last_became_peered":"2024-04-26T21:18:44.025260+0000","last_unstale":"2024-04-26T21:18:46.025050+0000","last_undegraded":"2024-04-26T21:18:46.025050+0000","last_fullsized":"2024-04-26T21:18:46.025050+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.971224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015496+0000","last_change":"2024-04-26T21:18:45.029555+0000","last_active":"2024-04-26T21:18:47.015496+0000","last_peered":"2024-04-26T21:18:47.015496+0000","last_clean":"2024-04-26T21:18:47.015496+0000","last_became_active":"2024-04-26T21:18:45.029277+0000","last_became_peered":"2024-04-26T21:18:45.029277+0000","last_unstale":"2024-04-26T21:18:47.015496+0000","last_undegraded":"2024-04-26T21:18:47.015496+0000","last_fullsized":"2024-04-26T21:18:47.015496+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:37.275648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025673+0000","last_change":"2024-04-26T21:18:44.025032+0000","last_active":"2024-04-26T21:18:46.025673+0000","last_peered":"2024-04-26T21:18:46.025673+0000","last_clean":"2024-04-26T21:18:46.025673+0000","last_became_active":"2024-04-26T21:18:44.024732+0000","last_became_peered":"2024-04-26T21:18:44.024732+0000","last_unstale":"2024-04-26T21:18:46.025673+0000","last_undegraded":"2024-04-26T21:18:46.025673+0000","last_fullsized":"2024-04-26T21:18:46.025673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:27.955512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025356+0000","last_change":"2024-04-26T21:18:45.027556+0000","last_active":"2024-04-26T21:18:46.025356+0000","last_peered":"2024-04-26T21:18:46.025356+0000","last_clean":"2024-04-26T21:18:46.025356+0000","last_became_active":"2024-04-26T21:18:45.027255+0000","last_became_peered":"2024-04-26T21:18:45.027255+0000","last_unstale":"2024-04-26T21:18:46.025356+0000","last_undegraded":"2024-04-26T21:18:46.025356+0000","last_fullsized":"2024-04-26T21:18:46.025356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:28.303956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024671+0000","last_change":"2024-04-26T21:18:44.023148+0000","last_active":"2024-04-26T21:18:46.024671+0000","last_peered":"2024-04-26T21:18:46.024671+0000","last_clean":"2024-04-26T21:18:46.024671+0000","last_became_active":"2024-04-26T21:18:44.022721+0000","last_became_peered":"2024-04-26T21:18:44.022721+0000","last_unstale":"2024-04-26T21:18:46.024671+0000","last_undegraded":"2024-04-26T21:18:46.024671+0000","last_fullsized":"2024-04-26T21:18:46.024671+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:34.191404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634061+0000","last_change":"2024-04-26T21:18:45.029957+0000","last_active":"2024-04-26T21:18:46.634061+0000","last_peered":"2024-04-26T21:18:46.634061+0000","last_clean":"2024-04-26T21:18:46.634061+0000","last_became_active":"2024-04-26T21:18:45.029612+0000","last_became_peered":"2024-04-26T21:18:45.029612+0000","last_unstale":"2024-04-26T21:18:46.634061+0000","last_undegraded":"2024-04-26T21:18:46.634061+0000","last_fullsized":"2024-04-26T21:18:46.634061+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:30.119656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124776+0000","last_change":"2024-04-26T21:18:44.025750+0000","last_active":"2024-04-26T21:18:47.124776+0000","last_peered":"2024-04-26T21:18:47.124776+0000","last_clean":"2024-04-26T21:18:47.124776+0000","last_became_active":"2024-04-26T21:18:44.025402+0000","last_became_peered":"2024-04-26T21:18:44.025402+0000","last_unstale":"2024-04-26T21:18:47.124776+0000","last_undegraded":"2024-04-26T21:18:47.124776+0000","last_fullsized":"2024-04-26T21:18:47.124776+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:16.285828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124706+0000","last_change":"2024-04-26T21:18:45.029679+0000","last_active":"2024-04-26T21:18:47.124706+0000","last_peered":"2024-04-26T21:18:47.124706+0000","last_clean":"2024-04-26T21:18:47.124706+0000","last_became_active":"2024-04-26T21:18:45.029410+0000","last_became_peered":"2024-04-26T21:18:45.029410+0000","last_unstale":"2024-04-26T21:18:47.124706+0000","last_undegraded":"2024-04-26T21:18:47.124706+0000","last_fullsized":"2024-04-26T21:18:47.124706+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:16.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025208+0000","last_change":"2024-04-26T21:18:44.018742+0000","last_active":"2024-04-26T21:18:46.025208+0000","last_peered":"2024-04-26T21:18:46.025208+0000","last_clean":"2024-04-26T21:18:46.025208+0000","last_became_active":"2024-04-26T21:18:44.018439+0000","last_became_peered":"2024-04-26T21:18:44.018439+0000","last_unstale":"2024-04-26T21:18:46.025208+0000","last_undegraded":"2024-04-26T21:18:46.025208+0000","last_fullsized":"2024-04-26T21:18:46.025208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:16.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025141+0000","last_change":"2024-04-26T21:18:45.029655+0000","last_active":"2024-04-26T21:18:46.025141+0000","last_peered":"2024-04-26T21:18:46.025141+0000","last_clean":"2024-04-26T21:18:46.025141+0000","last_became_active":"2024-04-26T21:18:45.029376+0000","last_became_peered":"2024-04-26T21:18:45.029376+0000","last_unstale":"2024-04-26T21:18:46.025141+0000","last_undegraded":"2024-04-26T21:18:46.025141+0000","last_fullsized":"2024-04-26T21:18:46.025141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:30.119656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018135+0000","last_change":"2024-04-26T21:18:44.020936+0000","last_active":"2024-04-26T21:18:47.018135+0000","last_peered":"2024-04-26T21:18:47.018135+0000","last_clean":"2024-04-26T21:18:47.018135+0000","last_became_active":"2024-04-26T21:18:44.020631+0000","last_became_peered":"2024-04-26T21:18:44.020631+0000","last_unstale":"2024-04-26T21:18:47.018135+0000","last_undegraded":"2024-04-26T21:18:47.018135+0000","last_fullsized":"2024-04-26T21:18:47.018135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.271969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025339+0000","last_change":"2024-04-26T21:18:45.028970+0000","last_active":"2024-04-26T21:18:46.025339+0000","last_peered":"2024-04-26T21:18:46.025339+0000","last_clean":"2024-04-26T21:18:46.025339+0000","last_became_active":"2024-04-26T21:18:45.028684+0000","last_became_peered":"2024-04-26T21:18:45.028684+0000","last_unstale":"2024-04-26T21:18:46.025339+0000","last_undegraded":"2024-04-26T21:18:46.025339+0000","last_fullsized":"2024-04-26T21:18:46.025339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:13.102115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018060+0000","last_change":"2024-04-26T21:18:44.024830+0000","last_active":"2024-04-26T21:18:47.018060+0000","last_peered":"2024-04-26T21:18:47.018060+0000","last_clean":"2024-04-26T21:18:47.018060+0000","last_became_active":"2024-04-26T21:18:44.024559+0000","last_became_peered":"2024-04-26T21:18:44.024559+0000","last_unstale":"2024-04-26T21:18:47.018060+0000","last_undegraded":"2024-04-26T21:18:47.018060+0000","last_fullsized":"2024-04-26T21:18:47.018060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:14.711395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024639+0000","last_change":"2024-04-26T21:18:45.023099+0000","last_active":"2024-04-26T21:18:46.024639+0000","last_peered":"2024-04-26T21:18:46.024639+0000","last_clean":"2024-04-26T21:18:46.024639+0000","last_became_active":"2024-04-26T21:18:45.022700+0000","last_became_peered":"2024-04-26T21:18:45.022700+0000","last_unstale":"2024-04-26T21:18:46.024639+0000","last_undegraded":"2024-04-26T21:18:46.024639+0000","last_fullsized":"2024-04-26T21:18:46.024639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:35.193907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024751+0000","last_change":"2024-04-26T21:18:44.023352+0000","last_active":"2024-04-26T21:18:46.024751+0000","last_peered":"2024-04-26T21:18:46.024751+0000","last_clean":"2024-04-26T21:18:46.024751+0000","last_became_active":"2024-04-26T21:18:44.023050+0000","last_became_peered":"2024-04-26T21:18:44.023050+0000","last_unstale":"2024-04-26T21:18:46.024751+0000","last_undegraded":"2024-04-26T21:18:46.024751+0000","last_fullsized":"2024-04-26T21:18:46.024751+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:00:35.391866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017620+0000","last_change":"2024-04-26T21:18:45.030476+0000","last_active":"2024-04-26T21:18:47.017620+0000","last_peered":"2024-04-26T21:18:47.017620+0000","last_clean":"2024-04-26T21:18:47.017620+0000","last_became_active":"2024-04-26T21:18:45.029032+0000","last_became_peered":"2024-04-26T21:18:45.029032+0000","last_unstale":"2024-04-26T21:18:47.017620+0000","last_undegraded":"2024-04-26T21:18:47.017620+0000","last_fullsized":"2024-04-26T21:18:47.017620+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:38.349495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025166+0000","last_change":"2024-04-26T21:18:44.028840+0000","last_active":"2024-04-26T21:18:46.025166+0000","last_peered":"2024-04-26T21:18:46.025166+0000","last_clean":"2024-04-26T21:18:46.025166+0000","last_became_active":"2024-04-26T21:18:44.028560+0000","last_became_peered":"2024-04-26T21:18:44.028560+0000","last_unstale":"2024-04-26T21:18:46.025166+0000","last_undegraded":"2024-04-26T21:18:46.025166+0000","last_fullsized":"2024-04-26T21:18:46.025166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:33.361412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634133+0000","last_change":"2024-04-26T21:18:45.027746+0000","last_active":"2024-04-26T21:18:46.634133+0000","last_peered":"2024-04-26T21:18:46.634133+0000","last_clean":"2024-04-26T21:18:46.634133+0000","last_became_active":"2024-04-26T21:18:45.027456+0000","last_became_peered":"2024-04-26T21:18:45.027456+0000","last_unstale":"2024-04-26T21:18:46.634133+0000","last_undegraded":"2024-04-26T21:18:46.634133+0000","last_fullsized":"2024-04-26T21:18:46.634133+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:53.031582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025108+0000","last_change":"2024-04-26T21:18:44.025797+0000","last_active":"2024-04-26T21:18:46.025108+0000","last_peered":"2024-04-26T21:18:46.025108+0000","last_clean":"2024-04-26T21:18:46.025108+0000","last_became_active":"2024-04-26T21:18:44.025461+0000","last_became_peered":"2024-04-26T21:18:44.025461+0000","last_unstale":"2024-04-26T21:18:46.025108+0000","last_undegraded":"2024-04-26T21:18:46.025108+0000","last_fullsized":"2024-04-26T21:18:46.025108+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:34.579949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669361+0000","last_change":"2024-04-26T21:18:45.022240+0000","last_active":"2024-04-26T21:18:46.669361+0000","last_peered":"2024-04-26T21:18:46.669361+0000","last_clean":"2024-04-26T21:18:46.669361+0000","last_became_active":"2024-04-26T21:18:45.019481+0000","last_became_peered":"2024-04-26T21:18:45.019481+0000","last_unstale":"2024-04-26T21:18:46.669361+0000","last_undegraded":"2024-04-26T21:18:46.669361+0000","last_fullsized":"2024-04-26T21:18:46.669361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:45.002035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633927+0000","last_change":"2024-04-26T21:18:44.023439+0000","last_active":"2024-04-26T21:18:46.633927+0000","last_peered":"2024-04-26T21:18:46.633927+0000","last_clean":"2024-04-26T21:18:46.633927+0000","last_became_active":"2024-04-26T21:18:44.023138+0000","last_became_peered":"2024-04-26T21:18:44.023138+0000","last_unstale":"2024-04-26T21:18:46.633927+0000","last_undegraded":"2024-04-26T21:18:46.633927+0000","last_fullsized":"2024-04-26T21:18:46.633927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:32.892090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124777+0000","last_change":"2024-04-26T21:18:45.034899+0000","last_active":"2024-04-26T21:18:47.124777+0000","last_peered":"2024-04-26T21:18:47.124777+0000","last_clean":"2024-04-26T21:18:47.124777+0000","last_became_active":"2024-04-26T21:18:45.034622+0000","last_became_peered":"2024-04-26T21:18:45.034622+0000","last_unstale":"2024-04-26T21:18:47.124777+0000","last_undegraded":"2024-04-26T21:18:47.124777+0000","last_fullsized":"2024-04-26T21:18:47.124777+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:35.951269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015076+0000","last_change":"2024-04-26T21:18:44.020211+0000","last_active":"2024-04-26T21:18:47.015076+0000","last_peered":"2024-04-26T21:18:47.015076+0000","last_clean":"2024-04-26T21:18:47.015076+0000","last_became_active":"2024-04-26T21:18:44.019816+0000","last_became_peered":"2024-04-26T21:18:44.019816+0000","last_unstale":"2024-04-26T21:18:47.015076+0000","last_undegraded":"2024-04-26T21:18:47.015076+0000","last_fullsized":"2024-04-26T21:18:47.015076+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:23.461801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017906+0000","last_change":"2024-04-26T21:18:45.034560+0000","last_active":"2024-04-26T21:18:47.017906+0000","last_peered":"2024-04-26T21:18:47.017906+0000","last_clean":"2024-04-26T21:18:47.017906+0000","last_became_active":"2024-04-26T21:18:45.034288+0000","last_became_peered":"2024-04-26T21:18:45.034288+0000","last_unstale":"2024-04-26T21:18:47.017906+0000","last_undegraded":"2024-04-26T21:18:47.017906+0000","last_fullsized":"2024-04-26T21:18:47.017906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:13.142781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014911+0000","last_change":"2024-04-26T21:18:44.026367+0000","last_active":"2024-04-26T21:18:47.014911+0000","last_peered":"2024-04-26T21:18:47.014911+0000","last_clean":"2024-04-26T21:18:47.014911+0000","last_became_active":"2024-04-26T21:18:44.026065+0000","last_became_peered":"2024-04-26T21:18:44.026065+0000","last_unstale":"2024-04-26T21:18:47.014911+0000","last_undegraded":"2024-04-26T21:18:47.014911+0000","last_fullsized":"2024-04-26T21:18:47.014911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:33.807940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018107+0000","last_change":"2024-04-26T21:18:45.034062+0000","last_active":"2024-04-26T21:18:47.018107+0000","last_peered":"2024-04-26T21:18:47.018107+0000","last_clean":"2024-04-26T21:18:47.018107+0000","last_became_active":"2024-04-26T21:18:45.032456+0000","last_became_peered":"2024-04-26T21:18:45.032456+0000","last_unstale":"2024-04-26T21:18:47.018107+0000","last_undegraded":"2024-04-26T21:18:47.018107+0000","last_fullsized":"2024-04-26T21:18:47.018107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:57.534083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669254+0000","last_change":"2024-04-26T21:18:44.024805+0000","last_active":"2024-04-26T21:18:46.669254+0000","last_peered":"2024-04-26T21:18:46.669254+0000","last_clean":"2024-04-26T21:18:46.669254+0000","last_became_active":"2024-04-26T21:18:44.024452+0000","last_became_peered":"2024-04-26T21:18:44.024452+0000","last_unstale":"2024-04-26T21:18:46.669254+0000","last_undegraded":"2024-04-26T21:18:46.669254+0000","last_fullsized":"2024-04-26T21:18:46.669254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:13.814207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017668+0000","last_change":"2024-04-26T21:18:45.036794+0000","last_active":"2024-04-26T21:18:47.017668+0000","last_peered":"2024-04-26T21:18:47.017668+0000","last_clean":"2024-04-26T21:18:47.017668+0000","last_became_active":"2024-04-26T21:18:45.036523+0000","last_became_peered":"2024-04-26T21:18:45.036523+0000","last_unstale":"2024-04-26T21:18:47.017668+0000","last_undegraded":"2024-04-26T21:18:47.017668+0000","last_fullsized":"2024-04-26T21:18:47.017668+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:51.318664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669204+0000","last_change":"2024-04-26T21:18:44.029908+0000","last_active":"2024-04-26T21:18:46.669204+0000","last_peered":"2024-04-26T21:18:46.669204+0000","last_clean":"2024-04-26T21:18:46.669204+0000","last_became_active":"2024-04-26T21:18:44.029533+0000","last_became_peered":"2024-04-26T21:18:44.029533+0000","last_unstale":"2024-04-26T21:18:46.669204+0000","last_undegraded":"2024-04-26T21:18:46.669204+0000","last_fullsized":"2024-04-26T21:18:46.669204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:02.622477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633630+0000","last_change":"2024-04-26T21:18:45.024869+0000","last_active":"2024-04-26T21:18:46.633630+0000","last_peered":"2024-04-26T21:18:46.633630+0000","last_clean":"2024-04-26T21:18:46.633630+0000","last_became_active":"2024-04-26T21:18:45.024583+0000","last_became_peered":"2024-04-26T21:18:45.024583+0000","last_unstale":"2024-04-26T21:18:46.633630+0000","last_undegraded":"2024-04-26T21:18:46.633630+0000","last_fullsized":"2024-04-26T21:18:46.633630+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:28.303956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.014897+0000","last_change":"2024-04-26T21:18:44.030815+0000","last_active":"2024-04-26T21:18:47.014897+0000","last_peered":"2024-04-26T21:18:47.014897+0000","last_clean":"2024-04-26T21:18:47.014897+0000","last_became_active":"2024-04-26T21:18:44.030316+0000","last_became_peered":"2024-04-26T21:18:44.030316+0000","last_unstale":"2024-04-26T21:18:47.014897+0000","last_undegraded":"2024-04-26T21:18:47.014897+0000","last_fullsized":"2024-04-26T21:18:47.014897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:16.699824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015481+0000","last_change":"2024-04-26T21:18:45.027423+0000","last_active":"2024-04-26T21:18:47.015481+0000","last_peered":"2024-04-26T21:18:47.015481+0000","last_clean":"2024-04-26T21:18:47.015481+0000","last_became_active":"2024-04-26T21:18:45.027130+0000","last_became_peered":"2024-04-26T21:18:45.027130+0000","last_unstale":"2024-04-26T21:18:47.015481+0000","last_undegraded":"2024-04-26T21:18:47.015481+0000","last_fullsized":"2024-04-26T21:18:47.015481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:00.298609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.669437+0000","last_change":"2024-04-26T21:18:44.023137+0000","last_active":"2024-04-26T21:18:46.669437+0000","last_peered":"2024-04-26T21:18:46.669437+0000","last_clean":"2024-04-26T21:18:46.669437+0000","last_became_active":"2024-04-26T21:18:44.022828+0000","last_became_peered":"2024-04-26T21:18:44.022828+0000","last_unstale":"2024-04-26T21:18:46.669437+0000","last_undegraded":"2024-04-26T21:18:46.669437+0000","last_fullsized":"2024-04-26T21:18:46.669437+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:16.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025600+0000","last_change":"2024-04-26T21:18:45.028187+0000","last_active":"2024-04-26T21:18:46.025600+0000","last_peered":"2024-04-26T21:18:46.025600+0000","last_clean":"2024-04-26T21:18:46.025600+0000","last_became_active":"2024-04-26T21:18:45.026300+0000","last_became_peered":"2024-04-26T21:18:45.026300+0000","last_unstale":"2024-04-26T21:18:46.025600+0000","last_undegraded":"2024-04-26T21:18:46.025600+0000","last_fullsized":"2024-04-26T21:18:46.025600+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:59.339175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017049+0000","last_change":"2024-04-26T21:18:44.026211+0000","last_active":"2024-04-26T21:18:47.017049+0000","last_peered":"2024-04-26T21:18:47.017049+0000","last_clean":"2024-04-26T21:18:47.017049+0000","last_became_active":"2024-04-26T21:18:44.025593+0000","last_became_peered":"2024-04-26T21:18:44.025593+0000","last_unstale":"2024-04-26T21:18:47.017049+0000","last_undegraded":"2024-04-26T21:18:47.017049+0000","last_fullsized":"2024-04-26T21:18:47.017049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.432059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.025525+0000","last_change":"2024-04-26T21:18:45.040341+0000","last_active":"2024-04-26T21:18:46.025525+0000","last_peered":"2024-04-26T21:18:46.025525+0000","last_clean":"2024-04-26T21:18:46.025525+0000","last_became_active":"2024-04-26T21:18:45.037900+0000","last_became_peered":"2024-04-26T21:18:45.037900+0000","last_unstale":"2024-04-26T21:18:46.025525+0000","last_undegraded":"2024-04-26T21:18:46.025525+0000","last_fullsized":"2024-04-26T21:18:46.025525+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:36.462509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015217+0000","last_change":"2024-04-26T21:18:44.023842+0000","last_active":"2024-04-26T21:18:47.015217+0000","last_peered":"2024-04-26T21:18:47.015217+0000","last_clean":"2024-04-26T21:18:47.015217+0000","last_became_active":"2024-04-26T21:18:44.023457+0000","last_became_peered":"2024-04-26T21:18:44.023457+0000","last_unstale":"2024-04-26T21:18:47.015217+0000","last_undegraded":"2024-04-26T21:18:47.015217+0000","last_fullsized":"2024-04-26T21:18:47.015217+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:53.284159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634004+0000","last_change":"2024-04-26T21:18:45.031489+0000","last_active":"2024-04-26T21:18:46.634004+0000","last_peered":"2024-04-26T21:18:46.634004+0000","last_clean":"2024-04-26T21:18:46.634004+0000","last_became_active":"2024-04-26T21:18:45.030496+0000","last_became_peered":"2024-04-26T21:18:45.030496+0000","last_unstale":"2024-04-26T21:18:46.634004+0000","last_undegraded":"2024-04-26T21:18:46.634004+0000","last_fullsized":"2024-04-26T21:18:46.634004+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:52.859671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.634063+0000","last_change":"2024-04-26T21:18:44.027874+0000","last_active":"2024-04-26T21:18:46.634063+0000","last_peered":"2024-04-26T21:18:46.634063+0000","last_clean":"2024-04-26T21:18:46.634063+0000","last_became_active":"2024-04-26T21:18:44.027571+0000","last_became_peered":"2024-04-26T21:18:44.027571+0000","last_unstale":"2024-04-26T21:18:46.634063+0000","last_undegraded":"2024-04-26T21:18:46.634063+0000","last_fullsized":"2024-04-26T21:18:46.634063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:30.599389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018019+0000","last_change":"2024-04-26T21:18:45.030063+0000","last_active":"2024-04-26T21:18:47.018019+0000","last_peered":"2024-04-26T21:18:47.018019+0000","last_clean":"2024-04-26T21:18:47.018019+0000","last_became_active":"2024-04-26T21:18:45.029558+0000","last_became_peered":"2024-04-26T21:18:45.029558+0000","last_unstale":"2024-04-26T21:18:47.018019+0000","last_undegraded":"2024-04-26T21:18:47.018019+0000","last_fullsized":"2024-04-26T21:18:47.018019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:42.797366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.015172+0000","last_change":"2024-04-26T21:18:44.022574+0000","last_active":"2024-04-26T21:18:47.015172+0000","last_peered":"2024-04-26T21:18:47.015172+0000","last_clean":"2024-04-26T21:18:47.015172+0000","last_became_active":"2024-04-26T21:18:44.021973+0000","last_became_peered":"2024-04-26T21:18:44.021973+0000","last_unstale":"2024-04-26T21:18:47.015172+0000","last_undegraded":"2024-04-26T21:18:47.015172+0000","last_fullsized":"2024-04-26T21:18:47.015172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:29.175740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024881+0000","last_change":"2024-04-26T21:18:45.021243+0000","last_active":"2024-04-26T21:18:46.024881+0000","last_peered":"2024-04-26T21:18:46.024881+0000","last_clean":"2024-04-26T21:18:46.024881+0000","last_became_active":"2024-04-26T21:18:45.020950+0000","last_became_peered":"2024-04-26T21:18:45.020950+0000","last_unstale":"2024-04-26T21:18:46.024881+0000","last_undegraded":"2024-04-26T21:18:46.024881+0000","last_fullsized":"2024-04-26T21:18:46.024881+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:17.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633658+0000","last_change":"2024-04-26T21:18:44.029874+0000","last_active":"2024-04-26T21:18:46.633658+0000","last_peered":"2024-04-26T21:18:46.633658+0000","last_clean":"2024-04-26T21:18:46.633658+0000","last_became_active":"2024-04-26T21:18:44.029509+0000","last_became_peered":"2024-04-26T21:18:44.029509+0000","last_unstale":"2024-04-26T21:18:46.633658+0000","last_undegraded":"2024-04-26T21:18:46.633658+0000","last_fullsized":"2024-04-26T21:18:46.633658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:14.132727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017578+0000","last_change":"2024-04-26T21:18:45.033715+0000","last_active":"2024-04-26T21:18:47.017578+0000","last_peered":"2024-04-26T21:18:47.017578+0000","last_clean":"2024-04-26T21:18:47.017578+0000","last_became_active":"2024-04-26T21:18:45.032114+0000","last_became_peered":"2024-04-26T21:18:45.032114+0000","last_unstale":"2024-04-26T21:18:47.017578+0000","last_undegraded":"2024-04-26T21:18:47.017578+0000","last_fullsized":"2024-04-26T21:18:47.017578+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:22.740080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.024906+0000","last_change":"2024-04-26T21:18:44.025333+0000","last_active":"2024-04-26T21:18:46.024906+0000","last_peered":"2024-04-26T21:18:46.024906+0000","last_clean":"2024-04-26T21:18:46.024906+0000","last_became_active":"2024-04-26T21:18:44.025063+0000","last_became_peered":"2024-04-26T21:18:44.025063+0000","last_unstale":"2024-04-26T21:18:46.024906+0000","last_undegraded":"2024-04-26T21:18:46.024906+0000","last_fullsized":"2024-04-26T21:18:46.024906+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:50.619652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:46.633740+0000","last_change":"2024-04-26T21:18:45.030906+0000","last_active":"2024-04-26T21:18:46.633740+0000","last_peered":"2024-04-26T21:18:46.633740+0000","last_clean":"2024-04-26T21:18:46.633740+0000","last_became_active":"2024-04-26T21:18:45.030635+0000","last_became_peered":"2024-04-26T21:18:45.030635+0000","last_unstale":"2024-04-26T21:18:46.633740+0000","last_undegraded":"2024-04-26T21:18:46.633740+0000","last_fullsized":"2024-04-26T21:18:46.633740+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:54.286661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.017704+0000","last_change":"2024-04-26T21:18:44.025669+0000","last_active":"2024-04-26T21:18:47.017704+0000","last_peered":"2024-04-26T21:18:47.017704+0000","last_clean":"2024-04-26T21:18:47.017704+0000","last_became_active":"2024-04-26T21:18:44.025366+0000","last_became_peered":"2024-04-26T21:18:44.025366+0000","last_unstale":"2024-04-26T21:18:47.017704+0000","last_undegraded":"2024-04-26T21:18:47.017704+0000","last_fullsized":"2024-04-26T21:18:47.017704+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:11.988615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.018188+0000","last_change":"2024-04-26T21:18:45.032849+0000","last_active":"2024-04-26T21:18:47.018188+0000","last_peered":"2024-04-26T21:18:47.018188+0000","last_clean":"2024-04-26T21:18:47.018188+0000","last_became_active":"2024-04-26T21:18:45.028723+0000","last_became_peered":"2024-04-26T21:18:45.028723+0000","last_unstale":"2024-04-26T21:18:47.018188+0000","last_undegraded":"2024-04-26T21:18:47.018188+0000","last_fullsized":"2024-04-26T21:18:47.018188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:43.999199+0000","last_clean_scrub_stamp":"2024-04-26T21:18:43.999199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:01.743888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124635+0000","last_change":"2024-04-26T21:18:44.027770+0000","last_active":"2024-04-26T21:18:47.124635+0000","last_peered":"2024-04-26T21:18:47.124635+0000","last_clean":"2024-04-26T21:18:47.124635+0000","last_became_active":"2024-04-26T21:18:44.027461+0000","last_became_peered":"2024-04-26T21:18:44.027461+0000","last_unstale":"2024-04-26T21:18:47.124635+0000","last_undegraded":"2024-04-26T21:18:47.124635+0000","last_fullsized":"2024-04-26T21:18:47.124635+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:18:42.996696+0000","last_clean_scrub_stamp":"2024-04-26T21:18:42.996696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:16.699824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":305,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-26T21:18:47.124179+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:18:47.124179+0000","last_peered":"2024-04-26T21:18:47.124179+0000","last_clean":"2024-04-26T21:18:47.124179+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:18:47.124179+0000","last_undegraded":"2024-04-26T21:18:47.124179+0000","last_fullsized":"2024-04-26T21:18:47.124179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1098319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":508491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":509575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":509473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1789952,"data_stored":1098789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1191936,"data_stored":507839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:18:52.812 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:18:52.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:18:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.956+0000 7fbc3e1a4640 1 Processor -- start 2024-04-26T21:18:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.956+0000 7fbc3e1a4640 1 -- start start 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc3e1a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3620 0x7fbc38160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc3e1a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbc380b3ac0 0x7fbc3809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc3e1a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc380b46f0 con 0x7fbc380b3ff0 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc3e1a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3806d3b0 con 0x7fbc380b3ac0 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc3e1a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbc380d1f60 con 0x7fbc380b3620 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbc380b3ac0 0x7fbc3809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbc380b3ff0 0x7fbc3806db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53944/0 (socket says 172.21.15.40:53944) 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbc380b3ac0 0x7fbc3809ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57358/0 (socket says 172.21.15.40:57358) 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc37fff640 1 -- 172.21.15.40:0/3517508346 learned_addr learned my addr 172.21.15.40:0/3517508346 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc377fe640 1 --2- 172.21.15.40:0/3517508346 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3620 0x7fbc38160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 -- 172.21.15.40:0/3517508346 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3620 msgr2=0x7fbc38160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 --2- 172.21.15.40:0/3517508346 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3620 0x7fbc38160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 -- 172.21.15.40:0/3517508346 >> v1:172.21.15.40:6789/0 conn(0x7fbc380b3ff0 legacy=0x7fbc3806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 -- 172.21.15.40:0/3517508346 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc380d2240 con 0x7fbc380b3ac0 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc36ffd640 1 --2- 172.21.15.40:0/3517508346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3ac0 0x7fbc3809ba40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbc380736a0 tx=0x7fbc2c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d737298d7038d8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc367fc640 1 -- 172.21.15.40:0/3517508346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c037070 con 0x7fbc380b3ac0 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc367fc640 1 -- 172.21.15.40:0/3517508346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2c032440 con 0x7fbc380b3ac0 2024-04-26T21:18:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.957+0000 7fbc367fc640 1 -- 172.21.15.40:0/3517508346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c032810 con 0x7fbc380b3ac0 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3517508346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3ac0 msgr2=0x7fbc3809ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3517508346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3ac0 0x7fbc3809ba40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbc380736a0 tx=0x7fbc2c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3517508346 shutdown_connections 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3517508346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3ac0 0x7fbc3809ba40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3517508346 >> v2:172.21.15.184:3300/0 conn(0x7fbc380b3620 0x7fbc38160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3517508346 >> 172.21.15.40:0/3517508346 conn(0x7fbc380b4ab0 msgr2=0x7fbc380b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:52.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3517508346 shutdown_connections 2024-04-26T21:18:52.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.958+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3517508346 wait complete. 2024-04-26T21:18:52.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.959+0000 7fbc3e1a4640 1 Processor -- start 2024-04-26T21:18:52.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.959+0000 7fbc3e1a4640 1 -- start start 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ac0 0x7fbc380a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 0x7fbc380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3809bf80 con 0x7fbc380b3620 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3806d600 con 0x7fbc380b3ff0 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc3e1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3809c100 con 0x7fbc380b3ac0 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57374/0 (socket says 172.21.15.40:57374) 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 -- 172.21.15.40:0/3798831116 learned_addr learned my addr 172.21.15.40:0/3798831116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc37fff640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 0x7fbc380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc36ffd640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ac0 0x7fbc380a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 -- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ac0 msgr2=0x7fbc380a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ac0 0x7fbc380a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 -- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 msgr2=0x7fbc380b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 0x7fbc380b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 -- 172.21.15.40:0/3798831116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3815c6c0 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc37fff640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 0x7fbc380b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc377fe640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800d800 tx=0x7fbc2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.960+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800b6c0 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.961+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc28019070 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.961+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc381f5940 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.961+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800ba80 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.961+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381f5f00 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.962+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3806a290 con 0x7fbc380b3620 2024-04-26T21:18:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.962+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc2800bc20 con 0x7fbc380b3620 2024-04-26T21:18:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.962+0000 7fbc1ffff640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 0x7fbc2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.963+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fbc2808bf90 con 0x7fbc380b3620 2024-04-26T21:18:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.963+0000 7fbc36ffd640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 0x7fbc2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.963+0000 7fbc36ffd640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 0x7fbc2406ede0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c034000 tx=0x7fbc2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:52.968+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc2801c360 con 0x7fbc380b3620 2024-04-26T21:18:53.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.228+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbc380745c0 con 0x7fbc380b3620 2024-04-26T21:18:53.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.230+0000 7fbc1ffff640 1 -- 172.21.15.40:0/3798831116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fbc2805a3c0 con 0x7fbc380b3620 2024-04-26T21:18:53.290 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:18:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:18:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:18:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:18:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 msgr2=0x7fbc2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 0x7fbc2406ede0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c034000 tx=0x7fbc2c02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 msgr2=0x7fbc3815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800d800 tx=0x7fbc2800dcc0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 shutdown_connections 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbc2406c920 0x7fbc2406ede0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc380b3ff0 0x7fbc380b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc380b3ac0 0x7fbc380a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 --2- 172.21.15.40:0/3798831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc380b3620 0x7fbc3815c180 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.232+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 >> 172.21.15.40:0/3798831116 conn(0x7fbc380b4ab0 msgr2=0x7fbc380b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.233+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 shutdown_connections 2024-04-26T21:18:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.233+0000 7fbc3e1a4640 1 -- 172.21.15.40:0/3798831116 wait complete. 2024-04-26T21:18:53.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:18:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.392+0000 7f44dd140640 1 Processor -- start 2024-04-26T21:18:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.392+0000 7f44dd140640 1 -- start start 2024-04-26T21:18:53.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44dd140640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44d815f720 0x7f44d80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44dd140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 0x7f44d80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44dd140640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d80b2510 con 0x7f44d815f340 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44dd140640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f44d80b27f0 con 0x7f44d815f720 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44dd140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f44d80b2ad0 con 0x7f44d80d5610 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44d815f720 0x7f44d80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f44d815f340 0x7f44d80d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53952/0 (socket says 172.21.15.40:53952) 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d6d9d640 1 -- 172.21.15.40:0/3410061361 learned_addr learned my addr 172.21.15.40:0/3410061361 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44d815f720 0x7f44d80b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57382/0 (socket says 172.21.15.40:57382) 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d759e640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 0x7f44d80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 -- 172.21.15.40:0/3410061361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 msgr2=0x7f44d80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 0x7f44d80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 -- 172.21.15.40:0/3410061361 >> v1:172.21.15.40:6789/0 conn(0x7f44d815f340 legacy=0x7f44d80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d659c640 1 -- 172.21.15.40:0/3410061361 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d80b2db0 con 0x7f44d815f720 2024-04-26T21:18:53.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.393+0000 7f44d5d9b640 1 -- 172.21.15.40:0/3410061361 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683860423 0 0) 0x7f44d80b2510 con 0x7f44d815f340 2024-04-26T21:18:53.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.394+0000 7f44d759e640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 0x7f44d80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.394+0000 7f44d659c640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d815f720 0x7f44d80b1e80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0097a0 tx=0x7f44cc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81608bd3cd77f5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.394+0000 7f44d5d9b640 1 -- 172.21.15.40:0/3410061361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc037070 con 0x7f44d815f720 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.394+0000 7f44d5d9b640 1 -- 172.21.15.40:0/3410061361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc02ae50 con 0x7f44d815f720 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.394+0000 7f44d5d9b640 1 -- 172.21.15.40:0/3410061361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc032660 con 0x7f44d815f720 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 -- 172.21.15.40:0/3410061361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d815f720 msgr2=0x7f44d80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d815f720 0x7f44d80b1e80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0097a0 tx=0x7f44cc02a100 comp rx=0 tx=0).stop 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 -- 172.21.15.40:0/3410061361 shutdown_connections 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80d5610 0x7f44d80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 --2- 172.21.15.40:0/3410061361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d815f720 0x7f44d80b1e80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 -- 172.21.15.40:0/3410061361 >> 172.21.15.40:0/3410061361 conn(0x7f44d808ec70 msgr2=0x7f44d809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 -- 172.21.15.40:0/3410061361 shutdown_connections 2024-04-26T21:18:53.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.395+0000 7f44dd140640 1 -- 172.21.15.40:0/3410061361 wait complete. 2024-04-26T21:18:53.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.396+0000 7f44dd140640 1 Processor -- start 2024-04-26T21:18:53.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.396+0000 7f44dd140640 1 -- start start 2024-04-26T21:18:53.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d815f340 0x7f44d815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d815f720 0x7f44d8160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44d80b3940 con 0x7f44d80d5610 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44d80b37c0 con 0x7f44d815f340 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44dd140640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f44d80b3ac0 con 0x7f44d815f720 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57394/0 (socket says 172.21.15.40:57394) 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44d6d9d640 1 -- 172.21.15.40:0/4244675877 learned_addr learned my addr 172.21.15.40:0/4244675877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44d759e640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d815f720 0x7f44d8160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.397+0000 7f44d659c640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d815f340 0x7f44d815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 -- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d815f720 msgr2=0x7f44d8160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d815f720 0x7f44d8160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 -- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d815f340 msgr2=0x7f44d815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d815f340 0x7f44d815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 -- 172.21.15.40:0/4244675877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d81621e0 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44d6d9d640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c4d0 tx=0x7f44d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018070 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d0007500 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d81624c0 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0004a20 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.398+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d8162a80 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.399+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d806a290 con 0x7f44d80d5610 2024-04-26T21:18:53.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.399+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d001b070 con 0x7f44d80d5610 2024-04-26T21:18:53.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.400+0000 7f44bf7fe640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 0x7f44c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:53.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.400+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f44d008add0 con 0x7f44d80d5610 2024-04-26T21:18:53.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.400+0000 7f44d659c640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 0x7f44c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:53.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.401+0000 7f44d659c640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 0x7f44c406edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f44cc034000 tx=0x7f44cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:53.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.405+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f44d0059290 con 0x7f44d80d5610 2024-04-26T21:18:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.626+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44d80d3dc0 con 0x7f44d80d5610 2024-04-26T21:18:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.627+0000 7f44bf7fe640 1 -- 172.21.15.40:0/4244675877 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7f44d0058b90 con 0x7f44d80d5610 2024-04-26T21:18:53.628 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:53.628 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:18:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 msgr2=0x7f44c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 0x7f44c406edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f44cc034000 tx=0x7f44cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 msgr2=0x7f44d8161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c4d0 tx=0x7f44d000c990 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 shutdown_connections 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f44c406c8f0 0x7f44c406edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d815f720 0x7f44d8160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d815f340 0x7f44d815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 --2- 172.21.15.40:0/4244675877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d80d5610 0x7f44d8161b20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 >> 172.21.15.40:0/4244675877 conn(0x7f44d808ec70 msgr2=0x7f44d815e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 shutdown_connections 2024-04-26T21:18:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:53.629+0000 7f44dd140640 1 -- 172.21.15.40:0/4244675877 wait complete. 2024-04-26T21:18:54.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:18:44.382283+0000', 'modified': '2024-04-26T21:18:46.380682+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6837/2864100946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 2864100946}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2864100946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7545, 'qdb_cluster': [7545]} 2024-04-26T21:18:54.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:18:54.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:18:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.151+0000 7f89d3b89640 1 Processor -- start 2024-04-26T21:18:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.151+0000 7f89d3b89640 1 -- start start 2024-04-26T21:18:54.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d3b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d3b89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0d1740 con 0x7f89cc0cf2e0 2024-04-26T21:18:54.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d3b89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0d1a20 con 0x7f89cc06e2c0 2024-04-26T21:18:54.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d3b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc06e750 con 0x7f89cc0d11b0 2024-04-26T21:18:54.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1925640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f89cc0cf2e0 0x7f89cc06d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53958/0 (socket says 172.21.15.40:53958) 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1925640 1 -- 172.21.15.40:0/3099447022 learned_addr learned my addr 172.21.15.40:0/3099447022 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1124640 1 --2- 172.21.15.40:0/3099447022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3663341605 0 0) 0x7f89cc0d1740 con 0x7f89cc0cf2e0 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b8003520 con 0x7f89cc0cf2e0 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030962310 0 0) 0x7f89cc0d1a20 con 0x7f89cc06e2c0 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89cc0d1740 con 0x7f89cc06e2c0 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1124640 1 -- 172.21.15.40:0/3099447022 >> v1:172.21.15.184:6789/0 conn(0x7f89cc06e2c0 legacy=0x7f89cc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1124640 1 -- 172.21.15.40:0/3099447022 >> v1:172.21.15.40:6789/0 conn(0x7f89cc0cf2e0 legacy=0x7f89cc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.152+0000 7f89d1124640 1 -- 172.21.15.40:0/3099447022 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0d1d00 con 0x7f89cc0d11b0 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.153+0000 7f89d1124640 1 --2- 172.21.15.40:0/3099447022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc06dd80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0097d0 tx=0x7f89bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb94ba09d433b27e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.153+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc037070 con 0x7f89cc0d11b0 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.153+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc032440 con 0x7f89cc0d11b0 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.153+0000 7f89d0923640 1 -- 172.21.15.40:0/3099447022 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0327f0 con 0x7f89cc0d11b0 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 -- 172.21.15.40:0/3099447022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 msgr2=0x7f89cc06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3099447022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc06dd80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0097d0 tx=0x7f89bc02a210 comp rx=0 tx=0).stop 2024-04-26T21:18:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 -- 172.21.15.40:0/3099447022 shutdown_connections 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3099447022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc06dd80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 -- 172.21.15.40:0/3099447022 >> 172.21.15.40:0/3099447022 conn(0x7f89cc0b4a60 msgr2=0x7f89cc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 -- 172.21.15.40:0/3099447022 shutdown_connections 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.154+0000 7f89d3b89640 1 -- 172.21.15.40:0/3099447022 wait complete. 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.155+0000 7f89d3b89640 1 Processor -- start 2024-04-26T21:18:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.155+0000 7f89d3b89640 1 -- start start 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89cc0cf2e0 0x7f89cc0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc14ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0cf6c0 con 0x7f89cc06e2c0 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0d22c0 con 0x7f89cc0cf2e0 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d3b89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0cf840 con 0x7f89cc0d11b0 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57402/0 (socket says 172.21.15.40:57402) 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 -- 172.21.15.40:0/3341216701 learned_addr learned my addr 172.21.15.40:0/3341216701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d2126640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc14ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1124640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89cc0cf2e0 0x7f89cc0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 -- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 msgr2=0x7f89cc14ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc14ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 -- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89cc0cf2e0 msgr2=0x7f89cc0cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89cc0cf2e0 0x7f89cc0cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d1925640 1 -- 172.21.15.40:0/3341216701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc1f4ef0 con 0x7f89cc06e2c0 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.156+0000 7f89d2126640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc14ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89d1925640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c430 tx=0x7f89c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89c27fc640 1 -- 172.21.15.40:0/3341216701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8017070 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89c27fc640 1 -- 172.21.15.40:0/3341216701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c8005ce0 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc0cc100 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89c27fc640 1 -- 172.21.15.40:0/3341216701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80072f0 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.157+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc0cc6c0 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.159+0000 7f89c27fc640 1 -- 172.21.15.40:0/3341216701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89c801a070 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.159+0000 7f89c27fc640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.159+0000 7f89c27fc640 1 -- 172.21.15.40:0/3341216701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f89c808af70 con 0x7f89cc06e2c0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.160+0000 7f89d1124640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.160+0000 7f89d1124640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 0x7f89b006ede0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89bc034000 tx=0x7f89bc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.160+0000 7f89d697e740 1 Processor -- start 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.160+0000 7f89d697e740 1 -- start start 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dceeee120 0x562dcef343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562dcee52740 con 0x562dcef35ce0 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562dcedee3a0 con 0x562dcef18440 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562dcee4e700 con 0x562dceeee120 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57404/0 (socket says 172.21.15.40:57404) 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 -- 172.21.15.40:0/1448877938 learned_addr learned my addr 172.21.15.40:0/1448877938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1925640 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d2126640 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dceeee120 0x562dcef343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 -- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dceeee120 msgr2=0x562dcef343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dceeee120 0x562dcef343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 -- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 msgr2=0x562dcef18820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef18820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 -- 172.21.15.40:0/1448877938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcee0a750 con 0x562dcef35ce0 2024-04-26T21:18:54.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.161+0000 7f89d1124640 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0098f0 tx=0x7f89bc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db3a07744e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89af7fe640 1 -- 172.21.15.40:0/1448877938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc04d070 con 0x562dcef35ce0 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89af7fe640 1 -- 172.21.15.40:0/1448877938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc032d40 con 0x562dcef35ce0 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89af7fe640 1 -- 172.21.15.40:0/1448877938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc037040 con 0x562dcef35ce0 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 -- 172.21.15.40:0/1448877938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 msgr2=0x562dcef36230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0098f0 tx=0x7f89bc02c000 comp rx=0 tx=0).stop 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 -- 172.21.15.40:0/1448877938 shutdown_connections 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dceeee120 0x562dcef343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef36230 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 --2- 172.21.15.40:0/1448877938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef18820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 -- 172.21.15.40:0/1448877938 >> 172.21.15.40:0/1448877938 conn(0x562dceeeaa50 msgr2=0x562dcef33250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 -- 172.21.15.40:0/1448877938 shutdown_connections 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.162+0000 7f89d697e740 1 -- 172.21.15.40:0/1448877938 wait complete. 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.163+0000 7f89d697e740 1 -- start start 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.163+0000 7f89d697e740 1 Processor -- start 2024-04-26T21:18:54.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef18440 0x562dcef07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef35ce0 0x562dcef47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562dceeeeec0 con 0x562dcef07f70 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562dcedd9fe0 con 0x562dcef35ce0 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562dceee8980 con 0x562dcef18440 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d697e740 10 client.0 _ll_register_callbacks cb 0x562dcedf0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57418/0 (socket says 172.21.15.40:57418) 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 -- 172.21.15.40:0/1829885501 learned_addr learned my addr 172.21.15.40:0/1829885501 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d1124640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef35ce0 0x562dcef47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d1925640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef18440 0x562dcef07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 -- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef18440 msgr2=0x562dcef07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef18440 0x562dcef07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 -- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef35ce0 msgr2=0x562dcef47250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef35ce0 0x562dcef47250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.164+0000 7f89d2126640 1 -- 172.21.15.40:0/1829885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dceb642d0 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d2126640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f89c4010550 tx=0x7f89c4010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6617d8b9b707baf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003da0 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f89c40050d0 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc032d40 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f89c4021050 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 10 client.7695 fetch_fsmap learned FSMap version 57 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x562dcec2e6b0 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc032d40 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f89c4024020 con 0x562dcef07f70 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 10 client.7695 fetch_fsmap finished waiting for FSMap version 57 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 10 client.7695 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:18:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 20 client.7695 populate_metadata read hostname 'smithi040' 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef343f0 0x562dcef1b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 4 client.7695 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.165+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 --> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dcee60350 con 0x562dcef343f0 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.166+0000 7f89d1925640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef343f0 0x562dcef1b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.166+0000 7f89acff9640 10 client.7695 ms_handle_connect on v2:172.21.15.184:6835/2864100946 2024-04-26T21:18:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.166+0000 7f89d1925640 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef343f0 0x562dcef1b260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d887d3e01924ad server_cookie=1ba9cb6ce22df289 in_seq=0 out_seq=0 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89acff9640 1 -- 172.21.15.40:0/1829885501 <== mds.0 v2:172.21.15.184:6835/2864100946 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dcee60350 con 0x562dcef343f0 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89acff9640 10 client.7695 handle_command_reply: tid=0 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 client.7695 shutdown 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 msgr2=0x562dceee7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f89c4010550 tx=0x7f89c4010a10 comp rx=0 tx=0).stop 2024-04-26T21:18:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 shutdown_connections 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef343f0 0x562dcef1b260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef07f70 0x562dceee7730 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef35ce0 0x562dcef47250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 --2- 172.21.15.40:0/1829885501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef18440 0x562dcef07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89acff9640 0 client.7695 ms_handle_reset on v2:172.21.15.184:6835/2864100946 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 >> 172.21.15.40:0/1829885501 conn(0x562dceeee120 msgr2=0x562dcef367d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 shutdown_connections 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 1 -- 172.21.15.40:0/1829885501 wait complete. 2024-04-26T21:18:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.167+0000 7f89d697e740 20 client.7695 trim_cache size 0 max 16384 2024-04-26T21:18:54.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.185+0000 7f89d697e740 1 Processor -- start 2024-04-26T21:18:54.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 -- start start 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcef28d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dceeee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc200830 con 0x562dcef35ce0 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b7440 con 0x562dcef18440 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0cda40 con 0x562dcef07f70 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57424/0 (socket says 172.21.15.40:57424) 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.186+0000 7f89d2126640 1 -- 172.21.15.40:0/2114520402 learned_addr learned my addr 172.21.15.40:0/2114520402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 -- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 msgr2=0x562dcef28d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcef28d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 -- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 msgr2=0x562dceeee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dceeee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 -- 172.21.15.40:0/2114520402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dceee4700 con 0x562dcef35ce0 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89d2126640 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001580 tx=0x7f89c400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8470186ca4e8e841 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89aeffd640 1 -- 172.21.15.40:0/2114520402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4005390 con 0x562dcef35ce0 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89aeffd640 1 -- 172.21.15.40:0/2114520402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4019070 con 0x562dcef35ce0 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.187+0000 7f89aeffd640 1 -- 172.21.15.40:0/2114520402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4005790 con 0x562dcef35ce0 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- 172.21.15.40:0/2114520402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 msgr2=0x562dcef41800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001580 tx=0x7f89c400b000 comp rx=0 tx=0).stop 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- 172.21.15.40:0/2114520402 shutdown_connections 2024-04-26T21:18:54.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef41800 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dceeee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 --2- 172.21.15.40:0/2114520402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcef28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- 172.21.15.40:0/2114520402 >> 172.21.15.40:0/2114520402 conn(0x562dceeee120 msgr2=0x562dcef19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- 172.21.15.40:0/2114520402 shutdown_connections 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- 172.21.15.40:0/2114520402 wait complete. 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 -- start start 2024-04-26T21:18:54.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.188+0000 7f89d697e740 1 Processor -- start 2024-04-26T21:18:54.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcefa9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562dcef34270 con 0x562dcef35ce0 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562dcef2ae90 con 0x562dcef18440 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x562dcef348c0 con 0x562dcef07f70 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d697e740 10 client.0 _ll_register_callbacks cb 0x7f89c4001900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d2126640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57436/0 (socket says 172.21.15.40:57436) 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.189+0000 7f89d2126640 1 -- 172.21.15.40:0/659495135 learned_addr learned my addr 172.21.15.40:0/659495135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 -- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 msgr2=0x562dcefa9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcefa9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 -- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 msgr2=0x562dcef16fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 -- 172.21.15.40:0/659495135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcef2a350 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d2126640 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f89c40021b0 tx=0x7f89c40014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c021474a9132dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4005830 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f89c4039ae0 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c4019070 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f89c4039ce0 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 10 client.7701 fetch_fsmap learned FSMap version 57 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x562dcef14aa0 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4019070 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) ==== 3494+0+0 (secure 0 0 0) 0x7f89c4040050 con 0x562dcef35ce0 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 10 client.7701 fetch_fsmap finished waiting for FSMap version 57 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 10 client.7701 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 20 client.7701 populate_metadata read hostname 'smithi040' 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef45840 0x562dcef55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 4 client.7701 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.190+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 --> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] -- command(tid 0: {"prefix": "status"}) -- 0x562dcee5a1c0 con 0x562dcef45840 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d1925640 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef45840 0x562dcef55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d1925640 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef45840 0x562dcef55cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4af26525b6d340 server_cookie=d7c88c78ffddf3ce in_seq=0 out_seq=0 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89acff9640 10 client.7701 ms_handle_connect on v2:172.21.15.184:6835/2864100946 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89acff9640 1 -- 172.21.15.40:0/659495135 <== mds.0 v2:172.21.15.184:6835/2864100946 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562dcee5a1c0 con 0x562dcef45840 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89acff9640 10 client.7701 handle_command_reply: tid=0 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 client.7701 shutdown 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 msgr2=0x562dcef451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f89c40021b0 tx=0x7f89c40014d0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 shutdown_connections 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:6835/2864100946,v1:172.21.15.184:6837/2864100946] conn(0x562dcef45840 0x562dcef55cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89acff9640 0 client.7701 ms_handle_reset on v2:172.21.15.184:6835/2864100946 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562dcef35ce0 0x562dcef451c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dcef18440 0x562dcef16fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 --2- 172.21.15.40:0/659495135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562dcef07f70 0x562dcefa9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.191+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 >> 172.21.15.40:0/659495135 conn(0x562dceeee120 msgr2=0x562dcef07820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 shutdown_connections 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d697e740 1 -- 172.21.15.40:0/659495135 wait complete. 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d697e740 20 client.7701 trim_cache size 0 max 16384 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7545, 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:18:54.198 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 9.1855748629999994, 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 57, 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 97, 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 97, 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 14.508375494999999 2024-04-26T21:18:54.199 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 msgr2=0x7f89b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 0x7f89b006ede0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89bc034000 tx=0x7f89bc02e000 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 msgr2=0x7f89cc1f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c430 tx=0x7f89c800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 shutdown_connections 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89b006c920 0x7f89b006ede0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89cc0d11b0 0x7f89cc14ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89cc0cf2e0 0x7f89cc0cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 --2- 172.21.15.40:0/3341216701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc06e2c0 0x7f89cc1f8710 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 >> 172.21.15.40:0/3341216701 conn(0x7f89cc0b4a60 msgr2=0x7f89cc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 shutdown_connections 2024-04-26T21:18:54.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.192+0000 7f89d3b89640 1 -- 172.21.15.40:0/3341216701 wait complete. 2024-04-26T21:18:54.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:18:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.303+0000 7f1100dc6640 1 Processor -- start 2024-04-26T21:18:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.303+0000 7f1100dc6640 1 -- start start 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f1100dc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10fc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f1100dc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f10fc06e2c0 0x7f10fc15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f1100dc6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc0d1740 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f1100dc6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc06e750 con 0x7f10fc06e2c0 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f1100dc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc0d1a20 con 0x7f10fc0d11b0 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10fa59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10fc0cf2e0 0x7f10fc06d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53970/0 (socket says 172.21.15.40:53970) 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10fa59c640 1 -- 172.21.15.40:0/2705425487 learned_addr learned my addr 172.21.15.40:0/2705425487 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10fad9d640 1 --2- 172.21.15.40:0/2705425487 >> v2:172.21.15.184:3300/0 conn(0x7f10fc06e2c0 0x7f10fc15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10fc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850383633 0 0) 0x7f10fc0d1740 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.304+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e4003520 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2150113026 0 0) 0x7f10e4003520 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 msgr2=0x7f10fc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10fc06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 >> v2:172.21.15.184:3300/0 conn(0x7f10fc06e2c0 msgr2=0x7f10fc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 --2- 172.21.15.40:0/2705425487 >> v2:172.21.15.184:3300/0 conn(0x7f10fc06e2c0 0x7f10fc15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0d1d00 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f10f4001670 con 0x7f10fc0cf2e0 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10fc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10fad9d640 1 --2- 172.21.15.40:0/2705425487 >> v2:172.21.15.184:3300/0 conn(0x7f10fc06e2c0 0x7f10fc15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 >> v1:172.21.15.40:6789/0 conn(0x7f10fc0cf2e0 legacy=0x7f10fc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40039b0 0x7f10e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e4004800 0x7f10e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc06e750 con 0x7f10e4004800 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0d1a20 con 0x7f10e40039b0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10e400b9c0 con 0x7f10fc0d11b0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e4004800 0x7f10e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10fad9d640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10fa59c640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40039b0 0x7f10e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 -- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 msgr2=0x7f10e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10fc0d11b0 0x7f10e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 -- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40039b0 msgr2=0x7f10e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40039b0 0x7f10e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10f9d9b640 1 -- 172.21.15.40:0/2705425487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e4003520 con 0x7f10e4004800 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.305+0000 7f10fa59c640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10e40039b0 0x7f10e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.306+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e4004800 0x7f10e4004c80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009c90 tx=0x7f10e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bce97aec16d968 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.306+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0036070 con 0x7f10e4004800 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.306+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f0031400 con 0x7f10e4004800 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.306+0000 7f10f959a640 1 -- 172.21.15.40:0/2705425487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0009860 con 0x7f10e4004800 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 -- 172.21.15.40:0/2705425487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e4004800 msgr2=0x7f10e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 --2- 172.21.15.40:0/2705425487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10e4004800 0x7f10e4004c80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009c90 tx=0x7f10e4005830 comp rx=0 tx=0).stop 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f10fad9d640 1 -- 172.21.15.40:0/2705425487 reap_dead start 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 -- 172.21.15.40:0/2705425487 shutdown_connections 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 -- 172.21.15.40:0/2705425487 >> 172.21.15.40:0/2705425487 conn(0x7f10fc0b4a60 msgr2=0x7f10fc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 -- 172.21.15.40:0/2705425487 shutdown_connections 2024-04-26T21:18:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.307+0000 7f1100dc6640 1 -- 172.21.15.40:0/2705425487 wait complete. 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.308+0000 7f1100dc6640 1 Processor -- start 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.308+0000 7f1100dc6640 1 -- start start 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e4004800 0x7f10fc15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 0x7f10fc0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0cf6c0 con 0x7f10fc0b65e0 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0d22c0 con 0x7f10fc1f9150 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f1100dc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc0cf840 con 0x7f10e4004800 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57446/0 (socket says 172.21.15.40:57446) 2024-04-26T21:18:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 -- 172.21.15.40:0/1052379566 learned_addr learned my addr 172.21.15.40:0/1052379566 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10fad9d640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 0x7f10fc0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 -- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e4004800 msgr2=0x7f10fc15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10fa59c640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e4004800 0x7f10fc15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e4004800 0x7f10fc15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 -- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 msgr2=0x7f10fc0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 0x7f10fc0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 -- 172.21.15.40:0/1052379566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0b6a40 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10fad9d640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 0x7f10fc0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.309+0000 7f10f9d9b640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f10f002ad30 tx=0x7f10f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.310+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0040020 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.310+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f00093e0 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.310+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc0b6ce0 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.310+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0036040 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.310+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1f8950 con 0x7f10fc0b65e0 2024-04-26T21:18:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.311+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc0b7200 con 0x7f10fc0b65e0 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.311+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f0043020 con 0x7f10fc0b65e0 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.311+0000 7f10eaffd640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.312+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f10f00ac050 con 0x7f10fc0b65e0 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.312+0000 7f10fa59c640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.312+0000 7f10fa59c640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 0x7f10dc06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f10f4001eb0 tx=0x7f10f4000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:54.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.316+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10f004a030 con 0x7f10fc0b65e0 2024-04-26T21:18:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.536+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f10fc06a290 con 0x7f10fc0b65e0 2024-04-26T21:18:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.537+0000 7f10eaffd640 1 -- 172.21.15.40:0/1052379566 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7f10f007a480 con 0x7f10fc0b65e0 2024-04-26T21:18:54.538 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:18:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 msgr2=0x7f10dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 0x7f10dc06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f10f4001eb0 tx=0x7f10f4000ed0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 msgr2=0x7f10fc0b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f10f002ad30 tx=0x7f10f0004730 comp rx=0 tx=0).stop 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 shutdown_connections 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10fc1f9150 0x7f10fc0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10fc0b65e0 0x7f10fc0b8800 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 --2- 172.21.15.40:0/1052379566 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10e4004800 0x7f10fc15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 >> 172.21.15.40:0/1052379566 conn(0x7f10fc0b4a60 msgr2=0x7f10fc0c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 shutdown_connections 2024-04-26T21:18:54.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54.539+0000 7f1100dc6640 1 -- 172.21.15.40:0/1052379566 wait complete. 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:18:54.551 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:18:54.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:54.552 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:54.567 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:18:54.568 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:18:54.569 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:54.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:54.570 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:18:54.630 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:18:54.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:54.632 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:18:54.687 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:18:54.687 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:18:54.687 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:18:54.688 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:18:54.688 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:18:54.688 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:18:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:54.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:54.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:54.813 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:18:54.829 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:18:54.829 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:18:54.829 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:18:54.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:54.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:18:54.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:54.893 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:54.949 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:18:54.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:54.949 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:55.005 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:18:55.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:18:55.087 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:18:55.087 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:18:55.087 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:18:55.095 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:18:55.095 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:18:55.096 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:18:55.096 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:18:55.096 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:18:55.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:18:55.102 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:18:55.102 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:18:55.102 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:18:55.102 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:18:55.103 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:18:55.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:55.168 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:18:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:18:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:55.271 INFO:teuthology.orchestra.run.smithi040.stdout:client7710 2024-04-26T21:18:55.271 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:55.272 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:55.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:18:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.427+0000 7f21f46fc640 1 Processor -- start 2024-04-26T21:18:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.427+0000 7f21f46fc640 1 -- start start 2024-04-26T21:18:55.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f46fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21ec15f720 0x7f21ec0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f46fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d5610 0x7f21ec0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f46fc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0b2510 con 0x7f21ec15f340 2024-04-26T21:18:55.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f46fc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0b27f0 con 0x7f21ec15f720 2024-04-26T21:18:55.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f46fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0b2ad0 con 0x7f21ec0d5610 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21ec15f720 0x7f21ec0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21ec15f720 0x7f21ec0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57452/0 (socket says 172.21.15.40:57452) 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 -- 172.21.15.40:0/753506508 learned_addr learned my addr 172.21.15.40:0/753506508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f2c99640 1 --2- 172.21.15.40:0/753506508 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d5610 0x7f21ec0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 -- 172.21.15.40:0/753506508 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d5610 msgr2=0x7f21ec0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 --2- 172.21.15.40:0/753506508 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d5610 0x7f21ec0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 -- 172.21.15.40:0/753506508 >> v1:172.21.15.184:6789/0 conn(0x7f21ec15f340 legacy=0x7f21ec0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.428+0000 7f21f1c97640 1 -- 172.21.15.40:0/753506508 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0b2db0 con 0x7f21ec15f720 2024-04-26T21:18:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.429+0000 7f21f1c97640 1 --2- 172.21.15.40:0/753506508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f720 0x7f21ec0b1e80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0d4c20 tx=0x7f21e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcaf3c5eb6c2ede server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.429+0000 7f21f1496640 1 -- 172.21.15.40:0/753506508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0037070 con 0x7f21ec15f720 2024-04-26T21:18:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.429+0000 7f21f1496640 1 -- 172.21.15.40:0/753506508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e0032440 con 0x7f21ec15f720 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.429+0000 7f21f1496640 1 -- 172.21.15.40:0/753506508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0032810 con 0x7f21ec15f720 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 -- 172.21.15.40:0/753506508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f720 msgr2=0x7f21ec0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 --2- 172.21.15.40:0/753506508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f720 0x7f21ec0b1e80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0d4c20 tx=0x7f21e002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 -- 172.21.15.40:0/753506508 shutdown_connections 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 --2- 172.21.15.40:0/753506508 >> v2:172.21.15.184:3300/0 conn(0x7f21ec0d5610 0x7f21ec0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 --2- 172.21.15.40:0/753506508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f720 0x7f21ec0b1e80 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 -- 172.21.15.40:0/753506508 >> 172.21.15.40:0/753506508 conn(0x7f21ec08ec70 msgr2=0x7f21ec09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 -- 172.21.15.40:0/753506508 shutdown_connections 2024-04-26T21:18:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.430+0000 7f21f46fc640 1 -- 172.21.15.40:0/753506508 wait complete. 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.431+0000 7f21f46fc640 1 Processor -- start 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.431+0000 7f21f46fc640 1 -- start start 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0d5610 0x7f21ec0bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec15f720 0x7f21ec1fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3940 con 0x7f21ec15f340 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0b37c0 con 0x7f21ec0d5610 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f46fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3ac0 con 0x7f21ec15f720 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57456/0 (socket says 172.21.15.40:57456) 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 -- 172.21.15.40:0/2142735691 learned_addr learned my addr 172.21.15.40:0/2142735691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 -- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec15f720 msgr2=0x7f21ec1fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec15f720 0x7f21ec1fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f2498640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0d5610 0x7f21ec0bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 -- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0d5610 msgr2=0x7f21ec0bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0d5610 0x7f21ec0bacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 -- 172.21.15.40:0/2142735691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec160d40 con 0x7f21ec15f340 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.432+0000 7f21f1c97640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f21e002a220 tx=0x7f21e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0041020 con 0x7f21ec15f340 2024-04-26T21:18:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e00306c0 con 0x7f21ec15f340 2024-04-26T21:18:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec161020 con 0x7f21ec15f340 2024-04-26T21:18:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0037040 con 0x7f21ec15f340 2024-04-26T21:18:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1615e0 con 0x7f21ec15f340 2024-04-26T21:18:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.433+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec1fa5f0 con 0x7f21ec15f340 2024-04-26T21:18:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.434+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e0044020 con 0x7f21ec15f340 2024-04-26T21:18:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.435+0000 7f21eaffd640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.435+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f21e00ac480 con 0x7f21ec15f340 2024-04-26T21:18:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.435+0000 7f21f2498640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.435+0000 7f21f2498640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21e40015a0 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.439+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21e004b030 con 0x7f21ec15f340 2024-04-26T21:18:55.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.662+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f21ec0d3020 con 0x7f21ec15f340 2024-04-26T21:18:55.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.662+0000 7f21eaffd640 1 -- 172.21.15.40:0/2142735691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7f21e007a820 con 0x7f21ec15f340 2024-04-26T21:18:55.663 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:18:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 msgr2=0x7f21d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21e40015a0 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2024-04-26T21:18:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 msgr2=0x7f21ec160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f21e002a220 tx=0x7f21e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 shutdown_connections 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21ec15f720 0x7f21ec1fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec15f340 0x7f21ec160510 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 --2- 172.21.15.40:0/2142735691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21ec0d5610 0x7f21ec0bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 >> 172.21.15.40:0/2142735691 conn(0x7f21ec08ec70 msgr2=0x7f21ec0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 shutdown_connections 2024-04-26T21:18:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:55.664+0000 7f21f46fc640 1 -- 172.21.15.40:0/2142735691 wait complete. 2024-04-26T21:18:55.676 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:18:55.676 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:18:55.677 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:18:55.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.678 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:18:55.697 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:18:55.697 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:18:55.697 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:55.698 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42338sec preferred_lft 42338sec 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.699 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:18:55.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.700 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:18:55.765 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:18:55.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.767 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:18:55.825 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:18:55.825 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:18:55.825 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:18:55.826 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:18:55.826 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:18:55.826 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:18:55.921 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:18:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:18:55.952 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:18:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:55.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.959 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:18:55.975 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:18:55.975 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:18:55.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:18:55.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:55.975 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:18:56.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:56.039 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:56.094 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:18:56.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:18:56.095 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:56.149 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:18:56.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:56.149 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:18:56.230 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:18:56.231 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:18:56.231 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:18:56.239 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:18:56.239 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:18:56.239 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:18:56.239 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:18:56.240 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:18:56.247 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:18:56.248 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:18:56.249 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:18:56.249 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:18:56.249 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:18:56.250 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:18:56.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:56.250 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:56.314 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:18:56.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:18:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:18:56.403 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:18:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:18:56.409 INFO:teuthology.orchestra.run.smithi184.stdout:client6736 2024-04-26T21:18:56.409 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:56.409 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:56.424 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:18:56.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:56.469 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:18:56.504 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:18:56.504 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:56.504 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:56.504 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:56.522 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:18:56.522 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:56.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.726+0000 7fd0854bd640 1 Processor -- start 2024-04-26T21:18:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.726+0000 7fd0854bd640 1 -- start start 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd0854bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd08015f340 0x7fd0800b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd0854bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0800b2510 con 0x7fd08015f720 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd0854bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0800b27f0 con 0x7fd0800d5610 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd0854bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd0800b2ad0 con 0x7fd08015f340 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd07e7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd08015f720 0x7fd0800d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53986/0 (socket says 172.21.15.40:53986) 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd07e7fc640 1 -- 172.21.15.40:0/3996170126 learned_addr learned my addr 172.21.15.40:0/3996170126 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.727+0000 7fd07effd640 1 --2- 172.21.15.40:0/3996170126 >> v2:172.21.15.184:3300/0 conn(0x7fd08015f340 0x7fd0800b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3097177947 0 0) 0x7fd0800b2510 con 0x7fd08015f720 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd068003520 con 0x7fd08015f720 2024-04-26T21:18:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 101780451 0 0) 0x7fd068003520 con 0x7fd08015f720 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 >> v2:172.21.15.184:3300/0 conn(0x7fd08015f340 msgr2=0x7fd0800b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 --2- 172.21.15.40:0/3996170126 >> v2:172.21.15.184:3300/0 conn(0x7fd08015f340 0x7fd0800b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 >> v1:172.21.15.184:6789/0 conn(0x7fd0800d5610 legacy=0x7fd0800b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0800b2db0 con 0x7fd08015f720 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd074001670 con 0x7fd08015f720 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 >> v1:172.21.15.40:6789/0 conn(0x7fd08015f720 legacy=0x7fd0800d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0680039b0 0x7fd068003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 0x7fd068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0680051c0 0x7fd06800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800b2ad0 con 0x7fd0680039b0 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd06800c160 con 0x7fd068004800 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd06800c360 con 0x7fd0680051c0 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07effd640 1 --2- 172.21.15.40:0/3996170126 >> v2:172.21.15.184:3300/0 conn(0x7fd08015f340 0x7fd0800b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 0x7fd068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07effd640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0680039b0 0x7fd068003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07f7fe640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0680051c0 0x7fd06800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 -- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0680051c0 msgr2=0x7fd06800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0680051c0 0x7fd06800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 -- 172.21.15.40:0/3996170126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0680039b0 msgr2=0x7fd068003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0680039b0 0x7fd068003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07e7fc640 1 -- 172.21.15.40:0/3996170126 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd068003520 con 0x7fd068004800 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07f7fe640 1 -- 172.21.15.40:0/3996170126 reap_dead start 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.728+0000 7fd07effd640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0680039b0 0x7fd068003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.729+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 0x7fd068004c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c210 tx=0x7fd068005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c898dfe236dac96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.729+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074007d10 con 0x7fd068004800 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.729+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd074005ad0 con 0x7fd068004800 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.729+0000 7fd07dffb640 1 -- 172.21.15.40:0/3996170126 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074007070 con 0x7fd068004800 2024-04-26T21:18:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 -- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 msgr2=0x7fd068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 0x7fd068004c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c210 tx=0x7fd068005830 comp rx=0 tx=0).stop 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 -- 172.21.15.40:0/3996170126 shutdown_connections 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 --2- 172.21.15.40:0/3996170126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd068004800 0x7fd068004c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 -- 172.21.15.40:0/3996170126 >> 172.21.15.40:0/3996170126 conn(0x7fd08008ec70 msgr2=0x7fd08009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 -- 172.21.15.40:0/3996170126 shutdown_connections 2024-04-26T21:18:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.730+0000 7fd0854bd640 1 -- 172.21.15.40:0/3996170126 wait complete. 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.731+0000 7fd0854bd640 1 Processor -- start 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.731+0000 7fd0854bd640 1 -- start start 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 0x7fd0800cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0800babc0 0x7fd0800b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800b2210 con 0x7fd068004800 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800b2090 con 0x7fd0800ba760 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd0854bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0800d3d40 con 0x7fd0800babc0 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd07effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd07effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57484/0 (socket says 172.21.15.40:57484) 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd07effd640 1 -- 172.21.15.40:0/1101039299 learned_addr learned my addr 172.21.15.40:0/1101039299 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 0x7fd0800cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.732+0000 7fd07f7fe640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0800babc0 0x7fd0800b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 -- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0800babc0 msgr2=0x7fd0800b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0800babc0 0x7fd0800b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 -- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 msgr2=0x7fd0800cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 0x7fd0800cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 -- 172.21.15.40:0/1101039299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0800cc4c0 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 0x7fd0800cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd07effd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd07000f3e0 tx=0x7fd07000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070019070 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070003da0 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070009dc0 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0800cc7a0 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.733+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0800ccd60 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.734+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08006a290 con 0x7fd068004800 2024-04-26T21:18:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.735+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd07001c070 con 0x7fd068004800 2024-04-26T21:18:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.735+0000 7fd0677fe640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 0x7fd06c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.735+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd07008c870 con 0x7fd068004800 2024-04-26T21:18:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.735+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 0x7fd06c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.736+0000 7fd07e7fc640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 0x7fd06c06ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd074032000 tx=0x7fd07402d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:56.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.740+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd070010040 con 0x7fd068004800 2024-04-26T21:18:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.924+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0800b6090 con 0x7fd068004800 2024-04-26T21:18:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.925+0000 7fd0677fe640 1 -- 172.21.15.40:0/1101039299 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7fd07005a6e0 con 0x7fd068004800 2024-04-26T21:18:56.926 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:56.927 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:56.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 msgr2=0x7fd06c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 0x7fd06c06ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd074032000 tx=0x7fd07402d000 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 msgr2=0x7fd0800ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd07000f3e0 tx=0x7fd07000f8a0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 shutdown_connections 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd06c06c920 0x7fd06c06ede0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0800babc0 0x7fd0800b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0800ba760 0x7fd0800cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 --2- 172.21.15.40:0/1101039299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd068004800 0x7fd0800ce520 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 >> 172.21.15.40:0/1101039299 conn(0x7fd08008ec70 msgr2=0x7fd0800b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 shutdown_connections 2024-04-26T21:18:56.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:56.927+0000 7fd0854bd640 1 -- 172.21.15.40:0/1101039299 wait complete. 2024-04-26T21:18:56.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:18:42.648529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:18:43.328674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:56.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:57.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.042+0000 7f730ba7e640 1 Processor -- start 2024-04-26T21:18:57.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 -- start start 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 0x7f73040d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73040d3f50 con 0x7f730415f680 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f730415fa60 con 0x7f73040b60b0 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.043+0000 7f730ba7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73040b5090 con 0x7f73040d3820 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f730a01b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 0x7f73040d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f730981a640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f730415f680 0x7f7304163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41850/0 (socket says 172.21.15.40:41850) 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f730981a640 1 -- 172.21.15.40:0/1660805997 learned_addr learned my addr 172.21.15.40:0/1660805997 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:43486/0 (socket says 172.21.15.40:43486) 2024-04-26T21:18:57.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f730a01b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 0x7f73040d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:40432/0 (socket says 172.21.15.40:40432) 2024-04-26T21:18:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 -- 172.21.15.40:0/1660805997 >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 msgr2=0x7f73040d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 --2- 172.21.15.40:0/1660805997 >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 0x7f73040d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 -- 172.21.15.40:0/1660805997 >> v1:172.21.15.184:6789/0 conn(0x7f730415f680 legacy=0x7f7304163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.044+0000 7f7309019640 1 -- 172.21.15.40:0/1660805997 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73040b5370 con 0x7f73040d3820 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f7309019640 1 --2- 172.21.15.40:0/1660805997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097a0 tx=0x7f72f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e4eb53809a19a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f7308818640 1 -- 172.21.15.40:0/1660805997 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8037070 con 0x7f73040d3820 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f7308818640 1 -- 172.21.15.40:0/1660805997 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f8032440 con 0x7f73040d3820 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f7308818640 1 -- 172.21.15.40:0/1660805997 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8032810 con 0x7f73040d3820 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 -- 172.21.15.40:0/1660805997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 msgr2=0x7f73040b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 --2- 172.21.15.40:0/1660805997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097a0 tx=0x7f72f802a180 comp rx=0 tx=0).stop 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 -- 172.21.15.40:0/1660805997 shutdown_connections 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 --2- 172.21.15.40:0/1660805997 >> v2:172.21.15.184:3300/0 conn(0x7f73040b60b0 0x7f73040d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 --2- 172.21.15.40:0/1660805997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040b5b70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 -- 172.21.15.40:0/1660805997 >> 172.21.15.40:0/1660805997 conn(0x7f730408eca0 msgr2=0x7f730409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 -- 172.21.15.40:0/1660805997 shutdown_connections 2024-04-26T21:18:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.045+0000 7f730ba7e640 1 -- 172.21.15.40:0/1660805997 wait complete. 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 Processor -- start 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 -- start start 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f730415f680 0x7f73040cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73040d55e0 con 0x7f73040b60b0 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73040d5460 con 0x7f730415f680 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.047+0000 7f730ba7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73040d5760 con 0x7f73040d3820 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57494/0 (socket says 172.21.15.40:57494) 2024-04-26T21:18:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 -- 172.21.15.40:0/429093907 learned_addr learned my addr 172.21.15.40:0/429093907 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f7309019640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730a01b640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f730415f680 0x7f73040cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 -- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 msgr2=0x7f73040c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 -- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f730415f680 msgr2=0x7f73040cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f730415f680 0x7f73040cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 -- 172.21.15.40:0/429093907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73040ccd60 con 0x7f73040b60b0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730981a640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c3c0 tx=0x7f72fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc017070 con 0x7f73040b60b0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc005ce0 con 0x7f73040b60b0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73040cd030 con 0x7f73040b60b0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0072f0 con 0x7f73040b60b0 2024-04-26T21:18:57.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.048+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73040cd870 con 0x7f73040b60b0 2024-04-26T21:18:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.049+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73040d5220 con 0x7f73040b60b0 2024-04-26T21:18:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.050+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72fc01a070 con 0x7f73040b60b0 2024-04-26T21:18:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.050+0000 7f73027fc640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 0x7f72f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.050+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f72fc08aec0 con 0x7f73040b60b0 2024-04-26T21:18:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.050+0000 7f7309019640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 0x7f72f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.051+0000 7f7309019640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 0x7f72f006edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72f8034000 tx=0x7f72f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.056+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73040d5220 con 0x7f73040b60b0 2024-04-26T21:18:57.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.243+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f730406a290 con 0x7f73040b60b0 2024-04-26T21:18:57.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.244+0000 7f73027fc640 1 -- 172.21.15.40:0/429093907 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f73040cd030 con 0x7f73040b60b0 2024-04-26T21:18:57.245 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:57.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 msgr2=0x7f72f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 0x7f72f006edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72f8034000 tx=0x7f72f802e000 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 msgr2=0x7f73040b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c3c0 tx=0x7f72fc00c880 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 shutdown_connections 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f72f006c8f0 0x7f72f006edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f730415f680 0x7f73040cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73040d3820 0x7f73040c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 --2- 172.21.15.40:0/429093907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73040b60b0 0x7f73040b2dd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 >> 172.21.15.40:0/429093907 conn(0x7f730408eca0 msgr2=0x7f73040d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 shutdown_connections 2024-04-26T21:18:57.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.245+0000 7f730ba7e640 1 -- 172.21.15.40:0/429093907 wait complete. 2024-04-26T21:18:57.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.360+0000 7fae22220640 1 Processor -- start 2024-04-26T21:18:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.360+0000 7fae22220640 1 -- start start 2024-04-26T21:18:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae22220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0d0010 0x7fae1c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae22220640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae1c0b33e0 0x7fae1c163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae22220640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1c09c700 con 0x7fae1c0b3c20 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae22220640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c0b4480 con 0x7fae1c0b33e0 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae22220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c06e050 con 0x7fae1c0d0010 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae1c0b33e0 0x7fae1c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae1c0b3c20 0x7fae1c0b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53994/0 (socket says 172.21.15.40:53994) 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae1c0b33e0 0x7fae1c163280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57496/0 (socket says 172.21.15.40:57496) 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 -- 172.21.15.40:0/2554232647 learned_addr learned my addr 172.21.15.40:0/2554232647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 -- 172.21.15.40:0/2554232647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0d0010 msgr2=0x7fae1c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 --2- 172.21.15.40:0/2554232647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0d0010 0x7fae1c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 -- 172.21.15.40:0/2554232647 >> v1:172.21.15.40:6789/0 conn(0x7fae1c0b3c20 legacy=0x7fae1c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1bfff640 1 -- 172.21.15.40:0/2554232647 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c0d1c50 con 0x7fae1c0b33e0 2024-04-26T21:18:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.361+0000 7fae1a7fc640 1 -- 172.21.15.40:0/2554232647 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 413423852 0 0) 0x7fae1c09c700 con 0x7fae1c0b3c20 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.362+0000 7fae1bfff640 1 --2- 172.21.15.40:0/2554232647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0b33e0 0x7fae1c163280 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd44bb4601b65747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.362+0000 7fae1a7fc640 1 -- 172.21.15.40:0/2554232647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14036070 con 0x7fae1c0b33e0 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.362+0000 7fae1a7fc640 1 -- 172.21.15.40:0/2554232647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae14031600 con 0x7fae1c0b33e0 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.362+0000 7fae1a7fc640 1 -- 172.21.15.40:0/2554232647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae140319d0 con 0x7fae1c0b33e0 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 -- 172.21.15.40:0/2554232647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0b33e0 msgr2=0x7fae1c163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 --2- 172.21.15.40:0/2554232647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0b33e0 0x7fae1c163280 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1402a710 comp rx=0 tx=0).stop 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 -- 172.21.15.40:0/2554232647 shutdown_connections 2024-04-26T21:18:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 --2- 172.21.15.40:0/2554232647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0b33e0 0x7fae1c163280 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 --2- 172.21.15.40:0/2554232647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0d0010 0x7fae1c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 -- 172.21.15.40:0/2554232647 >> 172.21.15.40:0/2554232647 conn(0x7fae1c09ba40 msgr2=0x7fae1c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 -- 172.21.15.40:0/2554232647 shutdown_connections 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.363+0000 7fae22220640 1 -- 172.21.15.40:0/2554232647 wait complete. 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.364+0000 7fae22220640 1 Processor -- start 2024-04-26T21:18:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.364+0000 7fae22220640 1 -- start start 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c0b33e0 0x7fae1c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0b3c20 0x7fae1c0a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c0b39f0 con 0x7fae1c0d0010 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c0b3870 con 0x7fae1c0b33e0 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae22220640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c1637c0 con 0x7fae1c0b3c20 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57510/0 (socket says 172.21.15.40:57510) 2024-04-26T21:18:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 -- 172.21.15.40:0/1950495654 learned_addr learned my addr 172.21.15.40:0/1950495654 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1b7fe640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c0b33e0 0x7fae1c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1affd640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0b3c20 0x7fae1c0a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 -- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0b3c20 msgr2=0x7fae1c0a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0b3c20 0x7fae1c0a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 -- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c0b33e0 msgr2=0x7fae1c14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c0b33e0 0x7fae1c14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.365+0000 7fae1bfff640 1 -- 172.21.15.40:0/1950495654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c0a0c40 con 0x7fae1c0d0010 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae1bfff640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14041020 con 0x7fae1c0d0010 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae14037040 con 0x7fae1c0d0010 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c150d50 con 0x7fae1c0d0010 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14036050 con 0x7fae1c0d0010 2024-04-26T21:18:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.366+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c151310 con 0x7fae1c0d0010 2024-04-26T21:18:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.367+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c06a290 con 0x7fae1c0d0010 2024-04-26T21:18:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.367+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae14007c30 con 0x7fae1c0d0010 2024-04-26T21:18:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.368+0000 7fae03fff640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.368+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fae140ac410 con 0x7fae1c0d0010 2024-04-26T21:18:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.368+0000 7fae1b7fe640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.368+0000 7fae1b7fe640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 0x7fae0806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b810 tx=0x7fae0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.372+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae1404b060 con 0x7fae1c0d0010 2024-04-26T21:18:57.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.556+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae1c068e00 con 0x7fae1c0d0010 2024-04-26T21:18:57.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.556+0000 7fae03fff640 1 -- 172.21.15.40:0/1950495654 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fae1407a8d0 con 0x7fae1c0d0010 2024-04-26T21:18:57.557 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 msgr2=0x7fae0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 0x7fae0806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b810 tx=0x7fae0c008040 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 msgr2=0x7fae1c06ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae14004a40 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 shutdown_connections 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fae0806c8f0 0x7fae0806edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae1c0d0010 0x7fae1c06ea90 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae1c0b3c20 0x7fae1c0a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 --2- 172.21.15.40:0/1950495654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c0b33e0 0x7fae1c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 >> 172.21.15.40:0/1950495654 conn(0x7fae1c09ba40 msgr2=0x7fae1c15f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 shutdown_connections 2024-04-26T21:18:57.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.558+0000 7fae22220640 1 -- 172.21.15.40:0/1950495654 wait complete. 2024-04-26T21:18:57.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.672+0000 7f3f3e2d1640 1 Processor -- start 2024-04-26T21:18:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.672+0000 7f3f3e2d1640 1 -- start start 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f3e2d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f38070640 0x7f3f38070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f3e2d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f380d2470 con 0x7f3f38162ca0 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f3e2d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f380d2750 con 0x7f3f38071160 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f3e2d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f380cdc40 con 0x7f3f38070640 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f377fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f38070640 0x7f3f38070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3f38162ca0 0x7f3f380d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54010/0 (socket says 172.21.15.40:54010) 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.673+0000 7f3f37fff640 1 -- 172.21.15.40:0/3567979589 learned_addr learned my addr 172.21.15.40:0/3567979589 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.674+0000 7f3f377fe640 1 -- 172.21.15.40:0/3567979589 >> v1:172.21.15.184:6789/0 conn(0x7f3f38071160 legacy=0x7f3f380ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.674+0000 7f3f377fe640 1 -- 172.21.15.40:0/3567979589 >> v1:172.21.15.40:6789/0 conn(0x7f3f38162ca0 legacy=0x7f3f380d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.674+0000 7f3f36ffd640 1 -- 172.21.15.40:0/3567979589 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166552586 0 0) 0x7f3f380d2470 con 0x7f3f38162ca0 2024-04-26T21:18:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.674+0000 7f3f377fe640 1 -- 172.21.15.40:0/3567979589 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380cd5f0 con 0x7f3f38070640 2024-04-26T21:18:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.674+0000 7f3f377fe640 1 --2- 172.21.15.40:0/3567979589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070640 0x7f3f38070c20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9eb7ce91f50da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f36ffd640 1 -- 172.21.15.40:0/3567979589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20037070 con 0x7f3f38070640 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f36ffd640 1 -- 172.21.15.40:0/3567979589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f20032440 con 0x7f3f38070640 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f36ffd640 1 -- 172.21.15.40:0/3567979589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20032810 con 0x7f3f38070640 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/3567979589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070640 msgr2=0x7f3f38070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/3567979589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070640 0x7f3f38070c20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002a230 comp rx=0 tx=0).stop 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/3567979589 shutdown_connections 2024-04-26T21:18:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/3567979589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38070640 0x7f3f38070c20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/3567979589 >> 172.21.15.40:0/3567979589 conn(0x7f3f3809ba40 msgr2=0x7f3f3809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/3567979589 shutdown_connections 2024-04-26T21:18:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.675+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/3567979589 wait complete. 2024-04-26T21:18:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.676+0000 7f3f3e2d1640 1 Processor -- start 2024-04-26T21:18:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 -- start start 2024-04-26T21:18:57.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070640 0x7f3f3814a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f38162ca0 0x7f3f380ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380ce7e0 con 0x7f3f38071160 2024-04-26T21:18:57.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38071590 con 0x7f3f38162ca0 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f3e2d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3f380ce960 con 0x7f3f38070640 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57524/0 (socket says 172.21.15.40:57524) 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.677+0000 7f3f377fe640 1 -- 172.21.15.40:0/1283266231 learned_addr learned my addr 172.21.15.40:0/1283266231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 -- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070640 msgr2=0x7f3f3814a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070640 0x7f3f3814a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 -- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f38162ca0 msgr2=0x7f3f380ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f3c86e640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f38162ca0 0x7f3f380ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f38162ca0 0x7f3f380ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 -- 172.21.15.40:0/1283266231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f380b0ba0 con 0x7f3f38071160 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f377fe640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20037040 con 0x7f3f38071160 2024-04-26T21:18:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f20042070 con 0x7f3f38071160 2024-04-26T21:18:57.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f380b0e80 con 0x7f3f38071160 2024-04-26T21:18:57.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20030a70 con 0x7f3f38071160 2024-04-26T21:18:57.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.678+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f380b1440 con 0x7f3f38071160 2024-04-26T21:18:57.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.679+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f380693d0 con 0x7f3f38071160 2024-04-26T21:18:57.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.680+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f20030c10 con 0x7f3f38071160 2024-04-26T21:18:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.680+0000 7f3f34ff9640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.680+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f3f200ac580 con 0x7f3f38071160 2024-04-26T21:18:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.680+0000 7f3f37fff640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.681+0000 7f3f37fff640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 0x7f3f1806edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000c00 tx=0x7f3f280079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.685+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f20045360 con 0x7f3f38071160 2024-04-26T21:18:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.867+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f380b26e0 con 0x7f3f38071160 2024-04-26T21:18:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.867+0000 7f3f34ff9640 1 -- 172.21.15.40:0/1283266231 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f3f2007a9b0 con 0x7f3f38071160 2024-04-26T21:18:57.868 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:18:57.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 msgr2=0x7f3f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 0x7f3f1806edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000c00 tx=0x7f3f280079c0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 msgr2=0x7f3f380cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f200046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 shutdown_connections 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3f1806c8f0 0x7f3f1806edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f38162ca0 0x7f3f380ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f38071160 0x7f3f380cc6c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 --2- 172.21.15.40:0/1283266231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f38070640 0x7f3f3814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 >> 172.21.15.40:0/1283266231 conn(0x7f3f3809ba40 msgr2=0x7f3f380cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 shutdown_connections 2024-04-26T21:18:57.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.869+0000 7f3f3e2d1640 1 -- 172.21.15.40:0/1283266231 wait complete. 2024-04-26T21:18:57.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.982+0000 7f28aea64640 1 Processor -- start 2024-04-26T21:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 -- start start 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f28a80d3ee0 0x7f28a80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a80d03a0 con 0x7f28a815f750 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28a80b4d10 con 0x7f28a80d3ee0 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28aea64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80d0680 con 0x7f28a815f370 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.983+0000 7f28a7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28a815f750 0x7f28a80d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54012/0 (socket says 172.21.15.40:54012) 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28a7fff640 1 -- 172.21.15.40:0/1071485078 learned_addr learned my addr 172.21.15.40:0/1071485078 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ac800640 1 --2- 172.21.15.40:0/1071485078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ad001640 1 --2- 172.21.15.40:0/1071485078 >> v2:172.21.15.184:3300/0 conn(0x7f28a80d3ee0 0x7f28a80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28a77fe640 1 -- 172.21.15.40:0/1071485078 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999039136 0 0) 0x7f28a80d03a0 con 0x7f28a815f750 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28a77fe640 1 -- 172.21.15.40:0/1071485078 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2890003520 con 0x7f28a815f750 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ac800640 1 -- 172.21.15.40:0/1071485078 >> v2:172.21.15.184:3300/0 conn(0x7f28a80d3ee0 msgr2=0x7f28a80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ac800640 1 --2- 172.21.15.40:0/1071485078 >> v2:172.21.15.184:3300/0 conn(0x7f28a80d3ee0 0x7f28a80b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ac800640 1 -- 172.21.15.40:0/1071485078 >> v1:172.21.15.40:6789/0 conn(0x7f28a815f750 legacy=0x7f28a80d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.984+0000 7f28ac800640 1 -- 172.21.15.40:0/1071485078 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80d0960 con 0x7f28a815f370 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.985+0000 7f28ac800640 1 --2- 172.21.15.40:0/1071485078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80cfd70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f28980097a0 tx=0x7f289802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bbdb81aba0e7deb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.985+0000 7f28a77fe640 1 -- 172.21.15.40:0/1071485078 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898037070 con 0x7f28a815f370 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.985+0000 7f28a77fe640 1 -- 172.21.15.40:0/1071485078 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289802ae50 con 0x7f28a815f370 2024-04-26T21:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.985+0000 7f28a77fe640 1 -- 172.21.15.40:0/1071485078 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898032640 con 0x7f28a815f370 2024-04-26T21:18:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 -- 172.21.15.40:0/1071485078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 msgr2=0x7f28a80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 --2- 172.21.15.40:0/1071485078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80cfd70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f28980097a0 tx=0x7f289802a100 comp rx=0 tx=0).stop 2024-04-26T21:18:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 -- 172.21.15.40:0/1071485078 shutdown_connections 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 --2- 172.21.15.40:0/1071485078 >> v2:172.21.15.184:3300/0 conn(0x7f28a80d3ee0 0x7f28a80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 --2- 172.21.15.40:0/1071485078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80cfd70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 -- 172.21.15.40:0/1071485078 >> 172.21.15.40:0/1071485078 conn(0x7f28a808ec10 msgr2=0x7f28a809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 -- 172.21.15.40:0/1071485078 shutdown_connections 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.986+0000 7f28aea64640 1 -- 172.21.15.40:0/1071485078 wait complete. 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.987+0000 7f28aea64640 1 Processor -- start 2024-04-26T21:18:57.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.987+0000 7f28aea64640 1 -- start start 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a815f750 0x7f28a809fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80b51c0 con 0x7f28a80d3ee0 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80d0f20 con 0x7f28a815f750 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28aea64640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80b5340 con 0x7f28a815f370 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57534/0 (socket says 172.21.15.40:57534) 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 -- 172.21.15.40:0/3889809511 learned_addr learned my addr 172.21.15.40:0/3889809511 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ad001640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a815f750 0x7f28a809fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28a7fff640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 -- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 msgr2=0x7f28a80a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 -- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a815f750 msgr2=0x7f28a809fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a815f750 0x7f28a809fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 -- 172.21.15.40:0/3889809511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80a1e70 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28a7fff640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.988+0000 7f28ac800640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f289802a140 tx=0x7f28980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.989+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898041070 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.989+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289803c460 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.989+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a80a0450 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.989+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898037040 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.989+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a80a23f0 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.990+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a806a290 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.990+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2898044070 con 0x7f28a80d3ee0 2024-04-26T21:18:57.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.991+0000 7f28a57fa640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 0x7f289406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:57.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.991+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f28980ac070 con 0x7f28a80d3ee0 2024-04-26T21:18:57.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.991+0000 7f28a7fff640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 0x7f289406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:57.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.991+0000 7f28a7fff640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 0x7f289406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28a80ba9a0 tx=0x7f289c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:57.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:57.995+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f289807a530 con 0x7f28a80d3ee0 2024-04-26T21:18:58.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.196+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28a806d620 con 0x7f28a80d3ee0 2024-04-26T21:18:58.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.196+0000 7f28a57fa640 1 -- 172.21.15.40:0/3889809511 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f2898079e30 con 0x7f28a80d3ee0 2024-04-26T21:18:58.197 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:18:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 msgr2=0x7f289406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 0x7f289406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28a80ba9a0 tx=0x7f289c009040 comp rx=0 tx=0).stop 2024-04-26T21:18:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 msgr2=0x7f28a815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f289802a140 tx=0x7f28980046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 shutdown_connections 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f289406c8f0 0x7f289406edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a815f750 0x7f28a809fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28a815f370 0x7f28a80a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 --2- 172.21.15.40:0/3889809511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a80d3ee0 0x7f28a815c120 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 >> 172.21.15.40:0/3889809511 conn(0x7f28a808ec10 msgr2=0x7f28a80b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 shutdown_connections 2024-04-26T21:18:58.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.198+0000 7f28aea64640 1 -- 172.21.15.40:0/3889809511 wait complete. 2024-04-26T21:18:58.210 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:18:58.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:58.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:18:58.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:18:58.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:58.239 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:18:58.294 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:18:58.294 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:58.294 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:58.313 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:18:58.313 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:18:58.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:18:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 Processor -- start 2024-04-26T21:18:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 -- start start 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a0140350 0x7fb8a00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a00d05b0 con 0x7fb8a013ff70 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a00d0890 con 0x7fb8a0140350 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.472+0000 7fb8a6978640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a006e450 con 0x7fb8a0146790 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a0140350 0x7fb8a00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8a013ff70 0x7fb8a0146080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54024/0 (socket says 172.21.15.40:54024) 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89ffff640 1 -- 172.21.15.40:0/167697607 learned_addr learned my addr 172.21.15.40:0/167697607 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8a0140350 0x7fb8a00cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57538/0 (socket says 172.21.15.40:57538) 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb8a4f15640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 -- 172.21.15.40:0/167697607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 msgr2=0x7fb8a00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 -- 172.21.15.40:0/167697607 >> v1:172.21.15.40:6789/0 conn(0x7fb8a013ff70 legacy=0x7fb8a0146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89f7fe640 1 -- 172.21.15.40:0/167697607 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a006e730 con 0x7fb8a0140350 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.473+0000 7fb89effd640 1 -- 172.21.15.40:0/167697607 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140370350 0 0) 0x7fb8a00d05b0 con 0x7fb8a013ff70 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a0140350 0x7fb8a00cf810 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097a0 tx=0x7fb89002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb4048fc05f701b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb89effd640 1 -- 172.21.15.40:0/167697607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890037070 con 0x7fb8a0140350 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb89effd640 1 -- 172.21.15.40:0/167697607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89002ae50 con 0x7fb8a0140350 2024-04-26T21:18:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb89effd640 1 -- 172.21.15.40:0/167697607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890032660 con 0x7fb8a0140350 2024-04-26T21:18:58.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 -- 172.21.15.40:0/167697607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a0140350 msgr2=0x7fb8a00cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a0140350 0x7fb8a00cf810 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb8900097a0 tx=0x7fb89002a100 comp rx=0 tx=0).stop 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 -- 172.21.15.40:0/167697607 shutdown_connections 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a00cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 --2- 172.21.15.40:0/167697607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a0140350 0x7fb8a00cf810 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 -- 172.21.15.40:0/167697607 >> 172.21.15.40:0/167697607 conn(0x7fb8a00d0d30 msgr2=0x7fb8a00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 -- 172.21.15.40:0/167697607 shutdown_connections 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.474+0000 7fb8a6978640 1 -- 172.21.15.40:0/167697607 wait complete. 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.476+0000 7fb8a6978640 1 Processor -- start 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.476+0000 7fb8a6978640 1 -- start start 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.476+0000 7fb8a6978640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.476+0000 7fb8a6978640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 0x7fb8a015f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a006edd0 con 0x7fb8a013ff70 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a006ec50 con 0x7fb8a0140350 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a006ef50 con 0x7fb8a0146790 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57544/0 (socket says 172.21.15.40:57544) 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 -- 172.21.15.40:0/3415668199 learned_addr learned my addr 172.21.15.40:0/3415668199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a4f15640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 -- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 msgr2=0x7fb8a015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a015ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 0x7fb8a015f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 -- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 msgr2=0x7fb8a015f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 0x7fb8a015f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 -- 172.21.15.40:0/3415668199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00b5ae0 con 0x7fb8a013ff70 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 0x7fb8a015f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89ffff640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c500 tx=0x7fb89400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894018070 con 0x7fb8a013ff70 2024-04-26T21:18:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894007580 con 0x7fb8a013ff70 2024-04-26T21:18:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a00b5dc0 con 0x7fb8a013ff70 2024-04-26T21:18:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894004a20 con 0x7fb8a013ff70 2024-04-26T21:18:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.477+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a00b6380 con 0x7fb8a013ff70 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.478+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a006a290 con 0x7fb8a013ff70 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.479+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb89401b070 con 0x7fb8a013ff70 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.479+0000 7fb89cff9640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 0x7fb88c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.479+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb89408ae20 con 0x7fb8a013ff70 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.479+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 0x7fb88c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.480+0000 7fb89f7fe640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 0x7fb88c06edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb890034000 tx=0x7fb89002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.483+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8940592e0 con 0x7fb8a013ff70 2024-04-26T21:18:58.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.668+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8a00b45e0 con 0x7fb8a013ff70 2024-04-26T21:18:58.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.669+0000 7fb89cff9640 1 -- 172.21.15.40:0/3415668199 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7fb894058be0 con 0x7fb8a013ff70 2024-04-26T21:18:58.670 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:18:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:18:58.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 msgr2=0x7fb88c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 0x7fb88c06edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb890034000 tx=0x7fb89002e000 comp rx=0 tx=0).stop 2024-04-26T21:18:58.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 msgr2=0x7fb8a01f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c500 tx=0x7fb89400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 shutdown_connections 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb88c06c8f0 0x7fb88c06edb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8a0146790 0x7fb8a015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a0140350 0x7fb8a015f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 --2- 172.21.15.40:0/3415668199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8a013ff70 0x7fb8a01f5c90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 >> 172.21.15.40:0/3415668199 conn(0x7fb8a00d0d30 msgr2=0x7fb8a00b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 shutdown_connections 2024-04-26T21:18:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.671+0000 7fb8a6978640 1 -- 172.21.15.40:0/3415668199 wait complete. 2024-04-26T21:18:58.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:18:42.648529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:18:43.328674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:18:58.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.786+0000 7fe6775e4640 1 Processor -- start 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.786+0000 7fe6775e4640 1 -- start start 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe6775e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6700ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe6775e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe670161cb0 0x7fe6700d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe6775e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6700cdd40 con 0x7fe6700d1d90 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe6775e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe6700d2280 con 0x7fe670161cb0 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe6775e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700cdfe0 con 0x7fe67015e7e0 2024-04-26T21:18:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe675b81640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe6700d1d90 0x7fe670161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe675b81640 1 -- 172.21.15.40:0/105728350 learned_addr learned my addr 172.21.15.40:0/105728350 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe674b7f640 1 --2- 172.21.15.40:0/105728350 >> v2:172.21.15.184:3300/0 conn(0x7fe670161cb0 0x7fe6700d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6700ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024063463 0 0) 0x7fe6700cdd40 con 0x7fe6700d1d90 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.787+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe658003520 con 0x7fe6700d1d90 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1742404747 0 0) 0x7fe658003520 con 0x7fe6700d1d90 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 msgr2=0x7fe6700ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6700ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 >> v2:172.21.15.184:3300/0 conn(0x7fe670161cb0 msgr2=0x7fe6700d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 --2- 172.21.15.40:0/105728350 >> v2:172.21.15.184:3300/0 conn(0x7fe670161cb0 0x7fe6700d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6700ce2c0 con 0x7fe6700d1d90 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6700ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe6600019f0 con 0x7fe6700d1d90 2024-04-26T21:18:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe674b7f640 1 --2- 172.21.15.40:0/105728350 >> v2:172.21.15.184:3300/0 conn(0x7fe670161cb0 0x7fe6700d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 >> v1:172.21.15.40:6789/0 conn(0x7fe6700d1d90 legacy=0x7fe670161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe658004800 0x7fe658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700d2280 con 0x7fe6580039b0 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700cdfe0 con 0x7fe658004800 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe65800b9c0 con 0x7fe67015e7e0 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe674b7f640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe658004800 0x7fe658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675b81640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 -- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 msgr2=0x7fe6580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 -- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe658004800 msgr2=0x7fe658004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe658004800 0x7fe658004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675380640 1 -- 172.21.15.40:0/105728350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe658003520 con 0x7fe6580039b0 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe674b7f640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe658004800 0x7fe658004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.788+0000 7fe675b81640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67015e7e0 0x7fe6580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.789+0000 7fe675380640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe658003dd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fbd6c125b33a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.789+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668036070 con 0x7fe6580039b0 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.789+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe668031400 con 0x7fe6580039b0 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.789+0000 7fe66ffff640 1 -- 172.21.15.40:0/105728350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668009860 con 0x7fe6580039b0 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 -- 172.21.15.40:0/105728350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 msgr2=0x7fe658003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 --2- 172.21.15.40:0/105728350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe658003dd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe658005830 comp rx=0 tx=0).stop 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe675b81640 1 -- 172.21.15.40:0/105728350 reap_dead start 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 -- 172.21.15.40:0/105728350 shutdown_connections 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 -- 172.21.15.40:0/105728350 >> 172.21.15.40:0/105728350 conn(0x7fe67009ba40 msgr2=0x7fe67009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 -- 172.21.15.40:0/105728350 shutdown_connections 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.790+0000 7fe6775e4640 1 -- 172.21.15.40:0/105728350 wait complete. 2024-04-26T21:18:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.791+0000 7fe6775e4640 1 Processor -- start 2024-04-26T21:18:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.791+0000 7fe6775e4640 1 -- start start 2024-04-26T21:18:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 0x7fe6700cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67014e790 0x7fe6700cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe670161790 con 0x7fe6580039b0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700ce880 con 0x7fe670158790 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe670161910 con 0x7fe67014e790 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57560/0 (socket says 172.21.15.40:57560) 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 -- 172.21.15.40:0/2553084538 learned_addr learned my addr 172.21.15.40:0/2553084538 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675b81640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67014e790 0x7fe6700cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe674b7f640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 0x7fe6700cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 -- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67014e790 msgr2=0x7fe6700cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67014e790 0x7fe6700cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 -- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 msgr2=0x7fe6700cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 0x7fe6700cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 -- 172.21.15.40:0/2553084538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6700c71f0 con 0x7fe6580039b0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe674b7f640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 0x7fe6700cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe675380640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe668004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668041020 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680093e0 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668036040 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.792+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6700c7460 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.793+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6700c7a20 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.794+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67006a290 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.794+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe668044020 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.794+0000 7fe66dffb640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 0x7fe65c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.795+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe6680abe70 con 0x7fe6580039b0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.795+0000 7fe674b7f640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 0x7fe65c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.795+0000 7fe674b7f640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 0x7fe65c06edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe6701613c0 tx=0x7fe664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.799+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe66804b070 con 0x7fe6580039b0 2024-04-26T21:18:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.983+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe67015f060 con 0x7fe6580039b0 2024-04-26T21:18:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.984+0000 7fe66dffb640 1 -- 172.21.15.40:0/2553084538 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fe668079ce0 con 0x7fe6580039b0 2024-04-26T21:18:58.985 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 msgr2=0x7fe65c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 0x7fe65c06edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe6701613c0 tx=0x7fe664008040 comp rx=0 tx=0).stop 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 msgr2=0x7fe6700c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe668004730 comp rx=0 tx=0).stop 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 shutdown_connections 2024-04-26T21:18:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe65c06c8f0 0x7fe65c06edb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe67014e790 0x7fe6700cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670158790 0x7fe6700cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 --2- 172.21.15.40:0/2553084538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6580039b0 0x7fe6700c9160 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 >> 172.21.15.40:0/2553084538 conn(0x7fe67009ba40 msgr2=0x7fe6700b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 shutdown_connections 2024-04-26T21:18:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:58.986+0000 7fe6775e4640 1 -- 172.21.15.40:0/2553084538 wait complete. 2024-04-26T21:18:58.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:18:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.104+0000 7f638a8d1640 1 Processor -- start 2024-04-26T21:18:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.104+0000 7f638a8d1640 1 -- start start 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.104+0000 7f638a8d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f638415f490 0x7f63840d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.104+0000 7f638a8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63840d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.104+0000 7f638a8d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840d2120 0x7f63840ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f638a8d1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f63840d2570 con 0x7f63840d2120 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f638a8d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f63840d4240 con 0x7f638415f490 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f638a8d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63840d31a0 con 0x7f63840d0ff0 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840d2120 0x7f63840ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63840d2120 0x7f63840ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57568/0 (socket says 172.21.15.40:57568) 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 -- 172.21.15.40:0/1049487291 learned_addr learned my addr 172.21.15.40:0/1049487291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 -- 172.21.15.40:0/1049487291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 msgr2=0x7f63840d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 --2- 172.21.15.40:0/1049487291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63840d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 -- 172.21.15.40:0/1049487291 >> v2:172.21.15.184:3300/0 conn(0x7f638415f490 msgr2=0x7f63840d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 --2- 172.21.15.40:0/1049487291 >> v2:172.21.15.184:3300/0 conn(0x7f638415f490 0x7f63840d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.105+0000 7f6388e6e640 1 -- 172.21.15.40:0/1049487291 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63840d3480 con 0x7f63840d2120 2024-04-26T21:18:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f6388e6e640 1 --2- 172.21.15.40:0/1049487291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63840ba3a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f63840b7cf0 tx=0x7f637c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8e509fe867fee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f6382ffd640 1 -- 172.21.15.40:0/1049487291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c036070 con 0x7f63840d2120 2024-04-26T21:18:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f6382ffd640 1 -- 172.21.15.40:0/1049487291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c031600 con 0x7f63840d2120 2024-04-26T21:18:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f6382ffd640 1 -- 172.21.15.40:0/1049487291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0319d0 con 0x7f63840d2120 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 -- 172.21.15.40:0/1049487291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 msgr2=0x7f63840ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 --2- 172.21.15.40:0/1049487291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63840ba3a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f63840b7cf0 tx=0x7f637c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 -- 172.21.15.40:0/1049487291 shutdown_connections 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 --2- 172.21.15.40:0/1049487291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63840ba3a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 --2- 172.21.15.40:0/1049487291 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63840d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 --2- 172.21.15.40:0/1049487291 >> v2:172.21.15.184:3300/0 conn(0x7f638415f490 0x7f63840d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 -- 172.21.15.40:0/1049487291 >> 172.21.15.40:0/1049487291 conn(0x7f638409e530 msgr2=0x7f63840d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 -- 172.21.15.40:0/1049487291 shutdown_connections 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.106+0000 7f638a8d1640 1 -- 172.21.15.40:0/1049487291 wait complete. 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.108+0000 7f638a8d1640 1 Processor -- start 2024-04-26T21:18:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.108+0000 7f638a8d1640 1 -- start start 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.108+0000 7f638a8d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63841f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.108+0000 7f638a8d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638415f490 0x7f63840ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63840d3d20 con 0x7f63840d2120 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63840d3a40 con 0x7f638415f490 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f638415f870 con 0x7f63840d0ff0 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57584/0 (socket says 172.21.15.40:57584) 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 -- 172.21.15.40:0/2982493225 learned_addr learned my addr 172.21.15.40:0/2982493225 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f6383fff640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63841f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f6388e6e640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638415f490 0x7f63840ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 -- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 msgr2=0x7f63841f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63841f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 -- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638415f490 msgr2=0x7f63840ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638415f490 0x7f63840ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 -- 172.21.15.40:0/2982493225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63840cd490 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f63837fe640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f637800b410 tx=0x7f637800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378015020 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6378011830 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63840cd770 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378011c20 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.109+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63840cdd30 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.111+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638406a290 con 0x7f63840d2120 2024-04-26T21:18:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.111+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6378018020 con 0x7f63840d2120 2024-04-26T21:18:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.111+0000 7f6380ff9640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 0x7f637006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.111+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f637808b890 con 0x7f63840d2120 2024-04-26T21:18:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.112+0000 7f6383fff640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 0x7f637006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.112+0000 7f6383fff640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 0x7f637006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6374009990 tx=0x7f6374009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.116+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f637801f020 con 0x7f63840d2120 2024-04-26T21:18:59.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.301+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f638406eba0 con 0x7f63840d2120 2024-04-26T21:18:59.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.301+0000 7f6380ff9640 1 -- 172.21.15.40:0/2982493225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f6378059d50 con 0x7f63840d2120 2024-04-26T21:18:59.302 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:18:59.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 msgr2=0x7f637006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 0x7f637006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6374009990 tx=0x7f6374009110 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 msgr2=0x7f63841f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f637800b410 tx=0x7f637800b810 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 shutdown_connections 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f637006c8f0 0x7f637006edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638415f490 0x7f63840ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63840d2120 0x7f63841f6260 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 --2- 172.21.15.40:0/2982493225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63840d0ff0 0x7f63841f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 >> 172.21.15.40:0/2982493225 conn(0x7f638409e530 msgr2=0x7f63840d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 shutdown_connections 2024-04-26T21:18:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.303+0000 7f638a8d1640 1 -- 172.21.15.40:0/2982493225 wait complete. 2024-04-26T21:18:59.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:18:59.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.417+0000 7fb156662640 1 Processor -- start 2024-04-26T21:18:59.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 -- start start 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb150073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb150074030 0x7fb1500cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15015c530 con 0x7fb15015b930 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb15015c810 con 0x7fb150074030 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.418+0000 7fb156662640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb15015caf0 con 0x7fb15015f4f0 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb150074030 0x7fb1500cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb150074030 0x7fb1500cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57594/0 (socket says 172.21.15.40:57594) 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 -- 172.21.15.40:0/1528060556 learned_addr learned my addr 172.21.15.40:0/1528060556 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb14ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb15015b930 0x7fb15015bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54038/0 (socket says 172.21.15.40:54038) 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 -- 172.21.15.40:0/1528060556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 msgr2=0x7fb150073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:59.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb14f7fe640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb150073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb14effd640 1 -- 172.21.15.40:0/1528060556 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069497151 0 0) 0x7fb15015c530 con 0x7fb15015b930 2024-04-26T21:18:59.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb150073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 -- 172.21.15.40:0/1528060556 >> v1:172.21.15.40:6789/0 conn(0x7fb15015b930 legacy=0x7fb15015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb154bff640 1 -- 172.21.15.40:0/1528060556 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15015cdd0 con 0x7fb150074030 2024-04-26T21:18:59.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.419+0000 7fb14f7fe640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb150073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb154bff640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb150074030 0x7fb1500cf260 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d150 tx=0x7fb14802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35abbced6552df81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb14effd640 1 -- 172.21.15.40:0/1528060556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148009600 con 0x7fb150074030 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb14effd640 1 -- 172.21.15.40:0/1528060556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb148037070 con 0x7fb150074030 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb14effd640 1 -- 172.21.15.40:0/1528060556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1480099c0 con 0x7fb150074030 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 -- 172.21.15.40:0/1528060556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb150074030 msgr2=0x7fb1500cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb150074030 0x7fb1500cf260 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d150 tx=0x7fb14802c930 comp rx=0 tx=0).stop 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 -- 172.21.15.40:0/1528060556 shutdown_connections 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb150074030 0x7fb1500cf260 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 --2- 172.21.15.40:0/1528060556 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb150073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 -- 172.21.15.40:0/1528060556 >> 172.21.15.40:0/1528060556 conn(0x7fb1500cd8a0 msgr2=0x7fb1500cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.420+0000 7fb156662640 1 -- 172.21.15.40:0/1528060556 shutdown_connections 2024-04-26T21:18:59.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.421+0000 7fb156662640 1 -- 172.21.15.40:0/1528060556 wait complete. 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.422+0000 7fb156662640 1 Processor -- start 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.422+0000 7fb156662640 1 -- start start 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.422+0000 7fb156662640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 0x7fb150152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.422+0000 7fb156662640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb1501558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb15006f340 con 0x7fb15015b930 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb15006fdd0 con 0x7fb150074030 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb15006f4c0 con 0x7fb15015f4f0 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57608/0 (socket says 172.21.15.40:57608) 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 -- 172.21.15.40:0/3631289469 learned_addr learned my addr 172.21.15.40:0/3631289469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb154bff640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb1501558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14ffff640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 0x7fb150152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 -- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 msgr2=0x7fb1501558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb1501558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 -- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 msgr2=0x7fb150152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 0x7fb150152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 -- 172.21.15.40:0/3631289469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb150153fc0 con 0x7fb15015b930 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14ffff640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 0x7fb150152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14f7fe640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c7f0 tx=0x7fb13800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138015070 con 0x7fb15015b930 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb138007820 con 0x7fb15015b930 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15014bf30 con 0x7fb15015b930 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138007bd0 con 0x7fb15015b930 2024-04-26T21:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.423+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb150153ce0 con 0x7fb15015b930 2024-04-26T21:18:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.424+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15006a290 con 0x7fb15015b930 2024-04-26T21:18:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.425+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb138018070 con 0x7fb15015b930 2024-04-26T21:18:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.425+0000 7fb14cff9640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 0x7fb13006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.425+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb13808ba40 con 0x7fb15015b930 2024-04-26T21:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.426+0000 7fb14ffff640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 0x7fb13006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.426+0000 7fb14ffff640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 0x7fb13006edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb140000f60 tx=0x7fb140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.430+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb138091020 con 0x7fb15015b930 2024-04-26T21:18:59.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.616+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb1500cdd90 con 0x7fb15015b930 2024-04-26T21:18:59.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.617+0000 7fb14cff9640 1 -- 172.21.15.40:0/3631289469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fb138059e70 con 0x7fb15015b930 2024-04-26T21:18:59.618 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 msgr2=0x7fb13006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 0x7fb13006edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb140000f60 tx=0x7fb140009040 comp rx=0 tx=0).stop 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 msgr2=0x7fb15015a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c7f0 tx=0x7fb13800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 shutdown_connections 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb13006c8f0 0x7fb13006edb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb15015f4f0 0x7fb1501558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15015b930 0x7fb15015a2a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 --2- 172.21.15.40:0/3631289469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb150074030 0x7fb150152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 >> 172.21.15.40:0/3631289469 conn(0x7fb1500cd8a0 msgr2=0x7fb1500b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 shutdown_connections 2024-04-26T21:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.619+0000 7fb156662640 1 -- 172.21.15.40:0/3631289469 wait complete. 2024-04-26T21:18:59.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:18:59.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.732+0000 7f0d1c797640 1 Processor -- start 2024-04-26T21:18:59.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.732+0000 7f0d1c797640 1 -- start start 2024-04-26T21:18:59.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1c797640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d1415f370 0x7f0d140cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1c797640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d140d03a0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1c797640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d140d0680 con 0x7f0d1415f750 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1c797640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0d140d0960 con 0x7f0d1415f370 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1a533640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d1415f370 0x7f0d140cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1ad34640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d140d3ee0 0x7f0d140b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54040/0 (socket says 172.21.15.40:54040) 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.733+0000 7f0d1ad34640 1 -- 172.21.15.40:0/3426965822 learned_addr learned my addr 172.21.15.40:0/3426965822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.734+0000 7f0d1a533640 1 -- 172.21.15.40:0/3426965822 >> v1:172.21.15.184:6789/0 conn(0x7f0d1415f750 legacy=0x7f0d140d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.734+0000 7f0d1a533640 1 -- 172.21.15.40:0/3426965822 >> v1:172.21.15.40:6789/0 conn(0x7f0d140d3ee0 legacy=0x7f0d140b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.734+0000 7f0d19531640 1 -- 172.21.15.40:0/3426965822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241310089 0 0) 0x7f0d140d03a0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.734+0000 7f0d1a533640 1 -- 172.21.15.40:0/3426965822 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d140d0c40 con 0x7f0d1415f370 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.734+0000 7f0d1a533640 1 --2- 172.21.15.40:0/3426965822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d1415f370 0x7f0d140cfd70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0097a0 tx=0x7f0d0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518fb14f9b4649bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d19531640 1 -- 172.21.15.40:0/3426965822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c037070 con 0x7f0d1415f370 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d19531640 1 -- 172.21.15.40:0/3426965822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c032440 con 0x7f0d1415f370 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d19531640 1 -- 172.21.15.40:0/3426965822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c032810 con 0x7f0d1415f370 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 -- 172.21.15.40:0/3426965822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d1415f370 msgr2=0x7f0d140cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 --2- 172.21.15.40:0/3426965822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d1415f370 0x7f0d140cfd70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0097a0 tx=0x7f0d0c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 -- 172.21.15.40:0/3426965822 shutdown_connections 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 --2- 172.21.15.40:0/3426965822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d1415f370 0x7f0d140cfd70 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 -- 172.21.15.40:0/3426965822 >> 172.21.15.40:0/3426965822 conn(0x7f0d1408ec10 msgr2=0x7f0d1409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 -- 172.21.15.40:0/3426965822 shutdown_connections 2024-04-26T21:18:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.735+0000 7f0d1c797640 1 -- 172.21.15.40:0/3426965822 wait complete. 2024-04-26T21:18:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.736+0000 7f0d1c797640 1 Processor -- start 2024-04-26T21:18:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.736+0000 7f0d1c797640 1 -- start start 2024-04-26T21:18:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d1415f370 0x7f0d14162710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d1415f750 0x7f0d1414c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140b4950 con 0x7f0d140d3ee0 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140b47d0 con 0x7f0d1415f750 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1c797640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d140b4ad0 con 0x7f0d1415f370 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57614/0 (socket says 172.21.15.40:57614) 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 -- 172.21.15.40:0/1706046124 learned_addr learned my addr 172.21.15.40:0/1706046124 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 -- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d1415f370 msgr2=0x7f0d14162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d1415f370 0x7f0d14162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 -- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d1415f750 msgr2=0x7f0d1414c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d1415f750 0x7f0d1414c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.737+0000 7f0d1a533640 1 -- 172.21.15.40:0/1706046124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d14160460 con 0x7f0d140d3ee0 2024-04-26T21:18:59.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d1a533640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0098c0 tx=0x7f0d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c041070 con 0x7f0d140d3ee0 2024-04-26T21:18:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c03c870 con 0x7f0d140d3ee0 2024-04-26T21:18:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d140b27c0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c037040 con 0x7f0d140d3ee0 2024-04-26T21:18:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.738+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d140b2d80 con 0x7f0d140d3ee0 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.739+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1406a290 con 0x7f0d140d3ee0 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.739+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d0c044070 con 0x7f0d140d3ee0 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.740+0000 7f0d12ffd640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 0x7f0d0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.740+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f0d0c0ac600 con 0x7f0d140d3ee0 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.740+0000 7f0d19d32640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 0x7f0d0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:18:59.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.741+0000 7f0d19d32640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 0x7f0d0006ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0d1406d150 tx=0x7f0d08007990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:18:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.744+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d0c07abe0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.945+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0d140c8ab0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.946+0000 7f0d12ffd640 1 -- 172.21.15.40:0/1706046124 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f0d0c07a4e0 con 0x7f0d140d3ee0 2024-04-26T21:18:59.947 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 msgr2=0x7f0d0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 0x7f0d0006ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0d1406d150 tx=0x7f0d08007990 comp rx=0 tx=0).stop 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 msgr2=0x7f0d1415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0098c0 tx=0x7f0d0c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 shutdown_connections 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d0006c920 0x7f0d0006ede0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d1415f750 0x7f0d1414c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d1415f370 0x7f0d14162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 --2- 172.21.15.40:0/1706046124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140d3ee0 0x7f0d1415fcf0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:18:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 >> 172.21.15.40:0/1706046124 conn(0x7f0d1408ec10 msgr2=0x7f0d1406dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:18:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 shutdown_connections 2024-04-26T21:18:59.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:18:59.948+0000 7f0d1c797640 1 -- 172.21.15.40:0/1706046124 wait complete. 2024-04-26T21:18:59.959 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:18:59.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:59.960 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:18:59.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:18:59.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:18:59.992 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:19:00.050 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:00.050 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:00.051 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:00.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:00.067 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:00.067 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:00.067 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:00.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:00.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.208+0000 7f23a91af640 1 Processor -- start 2024-04-26T21:19:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.208+0000 7f23a91af640 1 -- start start 2024-04-26T21:19:00.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a91af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f23a406f2c0 0x7f23a4071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a91af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a40733e0 con 0x7f23a415b8c0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a91af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a4072f00 con 0x7f23a415b4e0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a91af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f23a4072660 con 0x7f23a406f2c0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a359e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23a415b8c0 0x7f23a40b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54048/0 (socket says 172.21.15.40:54048) 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a359e640 1 -- 172.21.15.40:0/3259524771 learned_addr learned my addr 172.21.15.40:0/3259524771 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a259c640 1 --2- 172.21.15.40:0/3259524771 >> v2:172.21.15.184:3300/0 conn(0x7f23a406f2c0 0x7f23a4071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783013972 0 0) 0x7f23a40733e0 con 0x7f23a415b8c0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.209+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2390003520 con 0x7f23a415b8c0 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.210+0000 7f23a259c640 1 -- 172.21.15.40:0/3259524771 >> v1:172.21.15.184:6789/0 conn(0x7f23a415b4e0 legacy=0x7f23a406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.210+0000 7f23a259c640 1 -- 172.21.15.40:0/3259524771 >> v1:172.21.15.40:6789/0 conn(0x7f23a415b8c0 legacy=0x7f23a40b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.210+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1772654881 0 0) 0x7f2390003520 con 0x7f23a415b8c0 2024-04-26T21:19:00.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.210+0000 7f23a259c640 1 -- 172.21.15.40:0/3259524771 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a4072940 con 0x7f23a406f2c0 2024-04-26T21:19:00.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.210+0000 7f23a259c640 1 --2- 172.21.15.40:0/3259524771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a406f2c0 0x7f23a4071fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097a0 tx=0x7f238c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f69c42333f2453b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c037070 con 0x7f23a406f2c0 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f238c02abc0 con 0x7f23a406f2c0 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a1d9b640 1 -- 172.21.15.40:0/3259524771 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23940019f0 con 0x7f23a406f2c0 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 -- 172.21.15.40:0/3259524771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a406f2c0 msgr2=0x7f23a4071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 --2- 172.21.15.40:0/3259524771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a406f2c0 0x7f23a4071fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097a0 tx=0x7f238c02a130 comp rx=0 tx=0).stop 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 -- 172.21.15.40:0/3259524771 shutdown_connections 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 --2- 172.21.15.40:0/3259524771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a406f2c0 0x7f23a4071fd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 -- 172.21.15.40:0/3259524771 >> 172.21.15.40:0/3259524771 conn(0x7f23a40d0d30 msgr2=0x7f23a40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 -- 172.21.15.40:0/3259524771 shutdown_connections 2024-04-26T21:19:00.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.211+0000 7f23a91af640 1 -- 172.21.15.40:0/3259524771 wait complete. 2024-04-26T21:19:00.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 Processor -- start 2024-04-26T21:19:00.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 -- start start 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 0x7f23a40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a415b8c0 0x7f23a41541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4073820 con 0x7f23a406f2c0 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40b3e80 con 0x7f23a415b8c0 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a91af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40739a0 con 0x7f23a415b4e0 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57626/0 (socket says 172.21.15.40:57626) 2024-04-26T21:19:00.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.213+0000 7f23a2d9d640 1 -- 172.21.15.40:0/1036135862 learned_addr learned my addr 172.21.15.40:0/1036135862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a359e640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a415b8c0 0x7f23a41541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a259c640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 0x7f23a40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 -- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 msgr2=0x7f23a40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 0x7f23a40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 -- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a415b8c0 msgr2=0x7f23a41541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a415b8c0 0x7f23a41541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 -- 172.21.15.40:0/1036135862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a41f3fb0 con 0x7f23a406f2c0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a259c640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 0x7f23a40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a2d9d640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c480 tx=0x7f239c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c0052d0 con 0x7f23a406f2c0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c005470 con 0x7f23a406f2c0 2024-04-26T21:19:00.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a41f4290 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c012870 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.214+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a40ce260 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.215+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a406a290 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.216+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f239c012a10 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.216+0000 7f239b7fe640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 0x7f238406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.216+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f239c08b760 con 0x7f23a406f2c0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.216+0000 7f23a259c640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 0x7f238406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.217+0000 7f23a259c640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 0x7f238406ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f238c034000 tx=0x7f238c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.220+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f239c059c20 con 0x7f23a406f2c0 2024-04-26T21:19:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.442+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23a40d1560 con 0x7f23a406f2c0 2024-04-26T21:19:00.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.443+0000 7f239b7fe640 1 -- 172.21.15.40:0/1036135862 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7f239c059520 con 0x7f23a406f2c0 2024-04-26T21:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:19:00.445 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:19:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 msgr2=0x7f238406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 0x7f238406ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f238c034000 tx=0x7f238c02e000 comp rx=0 tx=0).stop 2024-04-26T21:19:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 msgr2=0x7f23a41f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c480 tx=0x7f239c00c940 comp rx=0 tx=0).stop 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 shutdown_connections 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f238406c920 0x7f238406ede0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23a415b8c0 0x7f23a41541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23a415b4e0 0x7f23a40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 --2- 172.21.15.40:0/1036135862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a406f2c0 0x7f23a41f35f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 >> 172.21.15.40:0/1036135862 conn(0x7f23a40d0d30 msgr2=0x7f23a41509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 shutdown_connections 2024-04-26T21:19:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.445+0000 7f23a91af640 1 -- 172.21.15.40:0/1036135862 wait complete. 2024-04-26T21:19:00.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.558+0000 7fcde1c9c640 1 Processor -- start 2024-04-26T21:19:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.558+0000 7fcde1c9c640 1 -- start start 2024-04-26T21:19:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcde1c9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc0d11b0 0x7fcddc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcde1c9c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcddc06e2c0 0x7fcddc15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcde1c9c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcddc0d1740 con 0x7fcddc0cf2e0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcde1c9c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcddc06e710 con 0x7fcddc0d11b0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcde1c9c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcddc0d1a20 con 0x7fcddc06e2c0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc0d11b0 0x7fcddc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc0d11b0 0x7fcddc06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57628/0 (socket says 172.21.15.40:57628) 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 -- 172.21.15.40:0/3074461397 learned_addr learned my addr 172.21.15.40:0/3074461397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddbfff640 1 --2- 172.21.15.40:0/3074461397 >> v2:172.21.15.184:3300/0 conn(0x7fcddc06e2c0 0x7fcddc15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 -- 172.21.15.40:0/3074461397 >> v2:172.21.15.184:3300/0 conn(0x7fcddc06e2c0 msgr2=0x7fcddc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 --2- 172.21.15.40:0/3074461397 >> v2:172.21.15.184:3300/0 conn(0x7fcddc06e2c0 0x7fcddc15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 -- 172.21.15.40:0/3074461397 >> v1:172.21.15.184:6789/0 conn(0x7fcddc0cf2e0 legacy=0x7fcddc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.559+0000 7fcddaffd640 1 -- 172.21.15.40:0/3074461397 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc0d1d00 con 0x7fcddc0d11b0 2024-04-26T21:19:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.560+0000 7fcddaffd640 1 --2- 172.21.15.40:0/3074461397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc06dd80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009870 tx=0x7fcdd002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70886474d2d6a47f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.560+0000 7fcdda7fc640 1 -- 172.21.15.40:0/3074461397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0037070 con 0x7fcddc0d11b0 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.560+0000 7fcdda7fc640 1 -- 172.21.15.40:0/3074461397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0032440 con 0x7fcddc0d11b0 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.560+0000 7fcdda7fc640 1 -- 172.21.15.40:0/3074461397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0032810 con 0x7fcddc0d11b0 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 -- 172.21.15.40:0/3074461397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 msgr2=0x7fcddc06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/3074461397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc06dd80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009870 tx=0x7fcdd002a2a0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 -- 172.21.15.40:0/3074461397 shutdown_connections 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/3074461397 >> v2:172.21.15.184:3300/0 conn(0x7fcddc06e2c0 0x7fcddc15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/3074461397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc06dd80 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 -- 172.21.15.40:0/3074461397 >> 172.21.15.40:0/3074461397 conn(0x7fcddc0b4a60 msgr2=0x7fcddc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:00.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 -- 172.21.15.40:0/3074461397 shutdown_connections 2024-04-26T21:19:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.561+0000 7fcde1c9c640 1 -- 172.21.15.40:0/3074461397 wait complete. 2024-04-26T21:19:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.562+0000 7fcde1c9c640 1 Processor -- start 2024-04-26T21:19:00.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.562+0000 7fcde1c9c640 1 -- start start 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcddc06e2c0 0x7fcddc15e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcddc0cf2e0 0x7fcddc0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc0cf6c0 con 0x7fcddc0d11b0 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc0d22c0 con 0x7fcddc0cf2e0 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcde1c9c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcddc0cf840 con 0x7fcddc06e2c0 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57642/0 (socket says 172.21.15.40:57642) 2024-04-26T21:19:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 -- 172.21.15.40:0/2750234608 learned_addr learned my addr 172.21.15.40:0/2750234608 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddaffd640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcddc0cf2e0 0x7fcddc0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddb7fe640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcddc06e2c0 0x7fcddc15e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 -- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcddc06e2c0 msgr2=0x7fcddc15e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcddc06e2c0 0x7fcddc15e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 -- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcddc0cf2e0 msgr2=0x7fcddc0b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcddc0cf2e0 0x7fcddc0b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.563+0000 7fcddbfff640 1 -- 172.21.15.40:0/2750234608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc1ea5b0 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcddbfff640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400efc0 tx=0x7fcdd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4018020 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd4014480 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc1f5510 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4014870 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.564+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1f5ad0 con 0x7fcddc0d11b0 2024-04-26T21:19:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.565+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc1f5330 con 0x7fcddc0d11b0 2024-04-26T21:19:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.565+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdd4005d10 con 0x7fcddc0d11b0 2024-04-26T21:19:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.566+0000 7fcdc3fff640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 0x7fcdc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.566+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd408c3d0 con 0x7fcddc0d11b0 2024-04-26T21:19:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.566+0000 7fcddb7fe640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 0x7fcdc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.566+0000 7fcddb7fe640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 0x7fcdc806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0016e0 tx=0x7fcdcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.570+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcddc1f5330 con 0x7fcddc0d11b0 2024-04-26T21:19:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.756+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcddc06a290 con 0x7fcddc0d11b0 2024-04-26T21:19:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.757+0000 7fcdc3fff640 1 -- 172.21.15.40:0/2750234608 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7fcdd4022070 con 0x7fcddc0d11b0 2024-04-26T21:19:00.758 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:00.758 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 msgr2=0x7fcdc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 0x7fcdc806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0016e0 tx=0x7fcdcc009040 comp rx=0 tx=0).stop 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 msgr2=0x7fcddc0b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400efc0 tx=0x7fcdd400c490 comp rx=0 tx=0).stop 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 shutdown_connections 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcdc806c8f0 0x7fcdc806edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc0d11b0 0x7fcddc0b75f0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcddc0cf2e0 0x7fcddc0b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 --2- 172.21.15.40:0/2750234608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcddc06e2c0 0x7fcddc15e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 >> 172.21.15.40:0/2750234608 conn(0x7fcddc0b4a60 msgr2=0x7fcddc0c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 shutdown_connections 2024-04-26T21:19:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.759+0000 7fcde1c9c640 1 -- 172.21.15.40:0/2750234608 wait complete. 2024-04-26T21:19:00.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:18:42.648529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:18:43.328674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:00.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:00.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.874+0000 7f5b1e42a640 1 Processor -- start 2024-04-26T21:19:00.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.874+0000 7f5b1e42a640 1 -- start start 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1e42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1e42a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 0x7f5b181635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1e42a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1809c790 con 0x7f5b180b3300 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1e42a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5b180d15f0 con 0x7f5b180d1170 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1e42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b18163ec0 con 0x7f5b181631f0 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b17fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b177fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 0x7f5b181635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1c9c7640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f5b180b3300 0x7f5b180b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58572/0 (socket says 172.21.15.40:58572) 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b17fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38990/0 (socket says 172.21.15.40:38990) 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b1c9c7640 1 -- 172.21.15.40:0/3760794282 learned_addr learned my addr 172.21.15.40:0/3760794282 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.875+0000 7f5b177fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 0x7f5b181635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:58538/0 (socket says 172.21.15.40:58538) 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b16ffd640 1 -- 172.21.15.40:0/3760794282 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562347222 0 0) 0x7f5b1809c790 con 0x7f5b180b3300 2024-04-26T21:19:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b17fff640 1 -- 172.21.15.40:0/3760794282 >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 msgr2=0x7f5b181635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b17fff640 1 --2- 172.21.15.40:0/3760794282 >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 0x7f5b181635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b17fff640 1 -- 172.21.15.40:0/3760794282 >> v1:172.21.15.184:6789/0 conn(0x7f5b180b3300 legacy=0x7f5b180b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b17fff640 1 -- 172.21.15.40:0/3760794282 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180d1a40 con 0x7f5b181631f0 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.876+0000 7f5b17fff640 1 --2- 172.21.15.40:0/3760794282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b0002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0ad06183c47be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b16ffd640 1 -- 172.21.15.40:0/3760794282 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00037070 con 0x7f5b181631f0 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b16ffd640 1 -- 172.21.15.40:0/3760794282 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0002ae50 con 0x7f5b181631f0 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b16ffd640 1 -- 172.21.15.40:0/3760794282 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00032660 con 0x7f5b181631f0 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 -- 172.21.15.40:0/3760794282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 msgr2=0x7f5b180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/3760794282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b0002a100 comp rx=0 tx=0).stop 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 -- 172.21.15.40:0/3760794282 shutdown_connections 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/3760794282 >> v2:172.21.15.184:3300/0 conn(0x7f5b180d1170 0x7f5b181635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/3760794282 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b181631f0 0x7f5b180b2000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 -- 172.21.15.40:0/3760794282 >> 172.21.15.40:0/3760794282 conn(0x7f5b1809ba40 msgr2=0x7f5b1809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 -- 172.21.15.40:0/3760794282 shutdown_connections 2024-04-26T21:19:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.877+0000 7f5b1e42a640 1 -- 172.21.15.40:0/3760794282 wait complete. 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.878+0000 7f5b1e42a640 1 Processor -- start 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 -- start start 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 0x7f5b180b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b181631f0 0x7f5b180b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180d0230 con 0x7f5b180b3300 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180d2000 con 0x7f5b181631f0 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.879+0000 7f5b1e42a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5b181f9d00 con 0x7f5b180d1170 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56988/0 (socket says 172.21.15.40:56988) 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 -- 172.21.15.40:0/246973668 learned_addr learned my addr 172.21.15.40:0/246973668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b177fe640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 0x7f5b180b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b1c9c7640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b181631f0 0x7f5b180b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 -- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 msgr2=0x7f5b180b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 0x7f5b180b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 -- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b181631f0 msgr2=0x7f5b180b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b181631f0 0x7f5b180b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 -- 172.21.15.40:0/246973668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b18153990 con 0x7f5b180b3300 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b177fe640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 0x7f5b180b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b17fff640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00037040 con 0x7f5b180b3300 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00004030 con 0x7f5b180b3300 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b180b4b90 con 0x7f5b180b3300 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00004420 con 0x7f5b180b3300 2024-04-26T21:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.880+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b180b5150 con 0x7f5b180b3300 2024-04-26T21:19:00.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.881+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1806a290 con 0x7f5b180b3300 2024-04-26T21:19:00.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.882+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b00003610 con 0x7f5b180b3300 2024-04-26T21:19:00.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.882+0000 7f5b14ff9640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 0x7f5af806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.882+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5b000ac860 con 0x7f5b180b3300 2024-04-26T21:19:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.882+0000 7f5b177fe640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 0x7f5af806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.883+0000 7f5b177fe640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 0x7f5af806ede0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b180afdd0 tx=0x7f5b08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:00.887+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b18153990 con 0x7f5b180b3300 2024-04-26T21:19:01.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.073+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b18071600 con 0x7f5b180b3300 2024-04-26T21:19:01.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.073+0000 7f5b14ff9640 1 -- 172.21.15.40:0/246973668 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f5b000ac070 con 0x7f5b180b3300 2024-04-26T21:19:01.075 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 msgr2=0x7f5af806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 0x7f5af806ede0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b180afdd0 tx=0x7f5b08008040 comp rx=0 tx=0).stop 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 msgr2=0x7f5b18152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5b000097a0 tx=0x7f5b000046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 shutdown_connections 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5af806c920 0x7f5af806ede0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b181631f0 0x7f5b180b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b180d1170 0x7f5b180b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 --2- 172.21.15.40:0/246973668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b180b3300 0x7f5b18152fe0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 >> 172.21.15.40:0/246973668 conn(0x7f5b1809ba40 msgr2=0x7f5b180b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 shutdown_connections 2024-04-26T21:19:01.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.076+0000 7f5b1e42a640 1 -- 172.21.15.40:0/246973668 wait complete. 2024-04-26T21:19:01.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:19:01.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.190+0000 7f67eb493640 1 Processor -- start 2024-04-26T21:19:01.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.190+0000 7f67eb493640 1 -- start start 2024-04-26T21:19:01.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67eb493640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67e40d0010 0x7f67e406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67eb493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40b3c20 0x7f67e40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67eb493640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e409c700 con 0x7f67e40b3210 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67eb493640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f67e40b4480 con 0x7f67e40d0010 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67eb493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67e406e050 con 0x7f67e40b3c20 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67e40d0010 0x7f67e406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67e40d0010 0x7f67e406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56996/0 (socket says 172.21.15.40:56996) 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 -- 172.21.15.40:0/737766611 learned_addr learned my addr 172.21.15.40:0/737766611 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e9a30640 1 --1- 172.21.15.40:0/737766611 >> v1:172.21.15.40:6789/0 conn(0x7f67e40b3210 0x7f67e415fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47488/0 (socket says 172.21.15.40:47488) 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e8a2e640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40b3c20 0x7f67e40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 -- 172.21.15.40:0/737766611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40b3c20 msgr2=0x7f67e40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40b3c20 0x7f67e40b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 -- 172.21.15.40:0/737766611 >> v1:172.21.15.40:6789/0 conn(0x7f67e40b3210 legacy=0x7f67e415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67e922f640 1 -- 172.21.15.40:0/737766611 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e40d1c50 con 0x7f67e40d0010 2024-04-26T21:19:01.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.191+0000 7f67dbfff640 1 -- 172.21.15.40:0/737766611 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150703622 0 0) 0x7f67e409c700 con 0x7f67e40b3210 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.192+0000 7f67e922f640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40d0010 0x7f67e406db10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009870 tx=0x7f67e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea1d70b8bd46314 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.192+0000 7f67dbfff640 1 -- 172.21.15.40:0/737766611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0037070 con 0x7f67e40d0010 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.192+0000 7f67dbfff640 1 -- 172.21.15.40:0/737766611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e0032440 con 0x7f67e40d0010 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.192+0000 7f67dbfff640 1 -- 172.21.15.40:0/737766611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0032810 con 0x7f67e40d0010 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 -- 172.21.15.40:0/737766611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40d0010 msgr2=0x7f67e406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40d0010 0x7f67e406db10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009870 tx=0x7f67e002a270 comp rx=0 tx=0).stop 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 -- 172.21.15.40:0/737766611 shutdown_connections 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40b3c20 0x7f67e40b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 --2- 172.21.15.40:0/737766611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40d0010 0x7f67e406db10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 -- 172.21.15.40:0/737766611 >> 172.21.15.40:0/737766611 conn(0x7f67e409ba40 msgr2=0x7f67e409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 -- 172.21.15.40:0/737766611 shutdown_connections 2024-04-26T21:19:01.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.193+0000 7f67eb493640 1 -- 172.21.15.40:0/737766611 wait complete. 2024-04-26T21:19:01.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.194+0000 7f67eb493640 1 Processor -- start 2024-04-26T21:19:01.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.194+0000 7f67eb493640 1 -- start start 2024-04-26T21:19:01.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67e40b3c20 0x7f67e4161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40d0010 0x7f67e4162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40b3820 con 0x7f67e40b3210 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40b36a0 con 0x7f67e40b3c20 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67eb493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67e40b39a0 con 0x7f67e40d0010 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57006/0 (socket says 172.21.15.40:57006) 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 -- 172.21.15.40:0/2643376789 learned_addr learned my addr 172.21.15.40:0/2643376789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 -- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40d0010 msgr2=0x7f67e4162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40d0010 0x7f67e4162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 -- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67e40b3c20 msgr2=0x7f67e4161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67e40b3c20 0x7f67e4161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 -- 172.21.15.40:0/2643376789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e415a8a0 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.195+0000 7f67e922f640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000c00 tx=0x7f67e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.196+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0041020 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.196+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e00306c0 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.196+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e415ab80 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.196+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0037040 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.196+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41552c0 con 0x7f67e40b3210 2024-04-26T21:19:01.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.197+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e406a290 con 0x7f67e40b3210 2024-04-26T21:19:01.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.197+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67e0044020 con 0x7f67e40b3210 2024-04-26T21:19:01.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.198+0000 7f67d9ffb640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 0x7f67c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.198+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f67e00ac300 con 0x7f67e40b3210 2024-04-26T21:19:01.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.198+0000 7f67e8a2e640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 0x7f67c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.199+0000 7f67e8a2e640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 0x7f67c806ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f67d4006fd0 tx=0x7f67d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.202+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67e004b030 con 0x7f67e40b3210 2024-04-26T21:19:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.388+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67e4068e00 con 0x7f67e40b3210 2024-04-26T21:19:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.388+0000 7f67d9ffb640 1 -- 172.21.15.40:0/2643376789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f67e007a730 con 0x7f67e40b3210 2024-04-26T21:19:01.390 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:19:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 msgr2=0x7f67c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 0x7f67c806ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f67d4006fd0 tx=0x7f67d4008040 comp rx=0 tx=0).stop 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 msgr2=0x7f67e40b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000c00 tx=0x7f67e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 shutdown_connections 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f67c806c920 0x7f67c806ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67e40d0010 0x7f67e4162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67e40b3c20 0x7f67e4161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 --2- 172.21.15.40:0/2643376789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e40b3210 0x7f67e40b0fa0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 >> 172.21.15.40:0/2643376789 conn(0x7f67e409ba40 msgr2=0x7f67e406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 shutdown_connections 2024-04-26T21:19:01.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.390+0000 7f67eb493640 1 -- 172.21.15.40:0/2643376789 wait complete. 2024-04-26T21:19:01.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:19:01.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.505+0000 7fe578664640 1 Processor -- start 2024-04-26T21:19:01.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.505+0000 7fe578664640 1 -- start start 2024-04-26T21:19:01.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe578664640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe57006db10 0x7fe57016a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe578664640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57009c700 con 0x7fe5700b3c20 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe578664640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5700d1c50 con 0x7fe5700d0010 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe578664640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe5700b4480 con 0x7fe57006db10 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe575bff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5700b3c20 0x7fe5700b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47504/0 (socket says 172.21.15.40:47504) 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe575bff640 1 -- 172.21.15.40:0/1944198943 learned_addr learned my addr 172.21.15.40:0/1944198943 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.506+0000 7fe576c01640 1 --2- 172.21.15.40:0/1944198943 >> v2:172.21.15.40:3300/0 conn(0x7fe57006db10 0x7fe57016a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe5753fe640 1 -- 172.21.15.40:0/1944198943 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643904461 0 0) 0x7fe57009c700 con 0x7fe5700b3c20 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe5753fe640 1 -- 172.21.15.40:0/1944198943 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe55c003520 con 0x7fe5700b3c20 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe576c01640 1 -- 172.21.15.40:0/1944198943 >> v1:172.21.15.184:6789/0 conn(0x7fe5700d0010 legacy=0x7fe5700b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe576c01640 1 -- 172.21.15.40:0/1944198943 >> v1:172.21.15.40:6789/0 conn(0x7fe5700b3c20 legacy=0x7fe5700b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe576c01640 1 -- 172.21.15.40:0/1944198943 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5700d1f30 con 0x7fe57006db10 2024-04-26T21:19:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.507+0000 7fe576c01640 1 --2- 172.21.15.40:0/1944198943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe57006db10 0x7fe57016a5b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c11ca0df78b598e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe5753fe640 1 -- 172.21.15.40:0/1944198943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568037070 con 0x7fe57006db10 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe5753fe640 1 -- 172.21.15.40:0/1944198943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe568032440 con 0x7fe57006db10 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe5753fe640 1 -- 172.21.15.40:0/1944198943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568032810 con 0x7fe57006db10 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 -- 172.21.15.40:0/1944198943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe57006db10 msgr2=0x7fe57016a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 --2- 172.21.15.40:0/1944198943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe57006db10 0x7fe57016a5b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a230 comp rx=0 tx=0).stop 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 -- 172.21.15.40:0/1944198943 shutdown_connections 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 --2- 172.21.15.40:0/1944198943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe57006db10 0x7fe57016a5b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 -- 172.21.15.40:0/1944198943 >> 172.21.15.40:0/1944198943 conn(0x7fe57009ba40 msgr2=0x7fe57009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 -- 172.21.15.40:0/1944198943 shutdown_connections 2024-04-26T21:19:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.508+0000 7fe578664640 1 -- 172.21.15.40:0/1944198943 wait complete. 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.509+0000 7fe578664640 1 Processor -- start 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 -- start start 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57006db10 0x7fe570162330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5700d0010 0x7fe570162d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe57016ac70 con 0x7fe5700b3c20 2024-04-26T21:19:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe57016aaf0 con 0x7fe57006db10 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe578664640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe57016adf0 con 0x7fe5700d0010 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe575bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe575bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57020/0 (socket says 172.21.15.40:57020) 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.510+0000 7fe575bff640 1 -- 172.21.15.40:0/484359354 learned_addr learned my addr 172.21.15.40:0/484359354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe576400640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57006db10 0x7fe570162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe576c01640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5700d0010 0x7fe570162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 -- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5700d0010 msgr2=0x7fe570162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5700d0010 0x7fe570162d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 -- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57006db10 msgr2=0x7fe570162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57006db10 0x7fe570162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 -- 172.21.15.40:0/484359354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570161290 con 0x7fe5700b3c20 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe575bff640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c500 tx=0x7fe56000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560017070 con 0x7fe5700b3c20 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe560005ce0 con 0x7fe5700b3c20 2024-04-26T21:19:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe570161570 con 0x7fe5700b3c20 2024-04-26T21:19:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560007330 con 0x7fe5700b3c20 2024-04-26T21:19:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.511+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe570161b30 con 0x7fe5700b3c20 2024-04-26T21:19:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.512+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe570162010 con 0x7fe5700b3c20 2024-04-26T21:19:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.513+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe56001a070 con 0x7fe5700b3c20 2024-04-26T21:19:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.513+0000 7fe566ffd640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 0x7fe55406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.513+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe56008af60 con 0x7fe5700b3c20 2024-04-26T21:19:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.513+0000 7fe576400640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 0x7fe55406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.514+0000 7fe576400640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 0x7fe55406edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.517+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe570162010 con 0x7fe5700b3c20 2024-04-26T21:19:01.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.703+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe57006a290 con 0x7fe5700b3c20 2024-04-26T21:19:01.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.704+0000 7fe566ffd640 1 -- 172.21.15.40:0/484359354 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fe5600594e0 con 0x7fe5700b3c20 2024-04-26T21:19:01.705 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.705+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 msgr2=0x7fe55406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.705+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 0x7fe55406edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).stop 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 msgr2=0x7fe5700b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c500 tx=0x7fe56000c9c0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 shutdown_connections 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe55406c8f0 0x7fe55406edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5700d0010 0x7fe570162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5700b3c20 0x7fe5700b2f40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 --2- 172.21.15.40:0/484359354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57006db10 0x7fe570162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 >> 172.21.15.40:0/484359354 conn(0x7fe57009ba40 msgr2=0x7fe5700b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 shutdown_connections 2024-04-26T21:19:01.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.706+0000 7fe578664640 1 -- 172.21.15.40:0/484359354 wait complete. 2024-04-26T21:19:01.718 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:01.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.817+0000 7f772f01d640 1 Processor -- start 2024-04-26T21:19:01.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.817+0000 7f772f01d640 1 -- start start 2024-04-26T21:19:01.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772f01d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f772815f720 0x7f77280b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772f01d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77280d5610 0x7f77280d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772f01d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77280b2510 con 0x7f772815f340 2024-04-26T21:19:01.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772f01d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f77280b27f0 con 0x7f772815f720 2024-04-26T21:19:01.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772f01d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77280b2ad0 con 0x7f77280d5610 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f772815f720 0x7f77280b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f772815f720 0x7f77280b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57026/0 (socket says 172.21.15.40:57026) 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 -- 172.21.15.40:0/3459199214 learned_addr learned my addr 172.21.15.40:0/3459199214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f772d5ba640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77280d5610 0x7f77280d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 -- 172.21.15.40:0/3459199214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77280d5610 msgr2=0x7f77280d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77280d5610 0x7f77280d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 -- 172.21.15.40:0/3459199214 >> v1:172.21.15.184:6789/0 conn(0x7f772815f340 legacy=0x7f77280d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.818+0000 7f7727fff640 1 -- 172.21.15.40:0/3459199214 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77280b2db0 con 0x7f772815f720 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.819+0000 7f7727fff640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f720 0x7f77280b1e80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f771c009870 tx=0x7f771c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d3f2e0868dbd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.819+0000 7f77277fe640 1 -- 172.21.15.40:0/3459199214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c037070 con 0x7f772815f720 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.819+0000 7f77277fe640 1 -- 172.21.15.40:0/3459199214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771c032440 con 0x7f772815f720 2024-04-26T21:19:01.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.819+0000 7f77277fe640 1 -- 172.21.15.40:0/3459199214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c032810 con 0x7f772815f720 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 -- 172.21.15.40:0/3459199214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f720 msgr2=0x7f77280b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f720 0x7f77280b1e80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f771c009870 tx=0x7f771c02a180 comp rx=0 tx=0).stop 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 -- 172.21.15.40:0/3459199214 shutdown_connections 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77280d5610 0x7f77280d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 --2- 172.21.15.40:0/3459199214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f720 0x7f77280b1e80 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 -- 172.21.15.40:0/3459199214 >> 172.21.15.40:0/3459199214 conn(0x7f772808ec70 msgr2=0x7f772809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 -- 172.21.15.40:0/3459199214 shutdown_connections 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.820+0000 7f772f01d640 1 -- 172.21.15.40:0/3459199214 wait complete. 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.821+0000 7f772f01d640 1 Processor -- start 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.821+0000 7f772f01d640 1 -- start start 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77280d5610 0x7f772814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f772815f720 0x7f77280cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77280b3940 con 0x7f772815f340 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77280b37c0 con 0x7f77280d5610 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f772f01d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77280b3ac0 con 0x7f772815f720 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57038/0 (socket says 172.21.15.40:57038) 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 -- 172.21.15.40:0/2785706910 learned_addr learned my addr 172.21.15.40:0/2785706910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 -- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f772815f720 msgr2=0x7f77280cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f772815f720 0x7f77280cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 -- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77280d5610 msgr2=0x7f772814eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77280d5610 0x7f772814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.822+0000 7f7727fff640 1 -- 172.21.15.40:0/2785706910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7728160ea0 con 0x7f772815f340 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f7727fff640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f771c009870 tx=0x7f771c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c041020 con 0x7f772815f340 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771c030970 con 0x7f772815f340 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7728161180 con 0x7f772815f340 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c037040 con 0x7f772815f340 2024-04-26T21:19:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.823+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7728161740 con 0x7f772815f340 2024-04-26T21:19:01.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.824+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772806a290 con 0x7f772815f340 2024-04-26T21:19:01.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.824+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f771c044020 con 0x7f772815f340 2024-04-26T21:19:01.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.825+0000 7f77257fa640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 0x7f771406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:01.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.825+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f771c0ac540 con 0x7f772815f340 2024-04-26T21:19:01.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.825+0000 7f772cdb9640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 0x7f771406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:01.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.826+0000 7f772cdb9640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 0x7f771406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7718003cc0 tx=0x7f77180079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:01.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:01.829+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f771c04b030 con 0x7f772815f340 2024-04-26T21:19:02.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.054+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77280d3020 con 0x7f772815f340 2024-04-26T21:19:02.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.055+0000 7f77257fa640 1 -- 172.21.15.40:0/2785706910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7f771c07a970 con 0x7f772815f340 2024-04-26T21:19:02.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:02.056 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:19:02.057 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:19:02.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 msgr2=0x7f771406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 0x7f771406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7718003cc0 tx=0x7f77180079b0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 msgr2=0x7f77280cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f771c009870 tx=0x7f771c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 shutdown_connections 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f771406c8f0 0x7f771406edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f772815f720 0x7f77280cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f772815f340 0x7f77280cc320 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 --2- 172.21.15.40:0/2785706910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77280d5610 0x7f772814eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 >> 172.21.15.40:0/2785706910 conn(0x7f772808ec70 msgr2=0x7f77280b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 shutdown_connections 2024-04-26T21:19:02.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.057+0000 7f772f01d640 1 -- 172.21.15.40:0/2785706910 wait complete. 2024-04-26T21:19:02.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.169+0000 7fb56df14640 1 Processor -- start 2024-04-26T21:19:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.169+0000 7fb56df14640 1 -- start start 2024-04-26T21:19:02.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb56df14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5680d3d00 0x7fb5680d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb56df14640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5680d05f0 con 0x7fb56815f3e0 2024-04-26T21:19:02.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb56df14640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5680d08d0 con 0x7fb5680d5330 2024-04-26T21:19:02.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb56df14640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5680d4190 con 0x7fb5680d3d00 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5680d3d00 0x7fb5680d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5680d3d00 0x7fb5680d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57054/0 (socket says 172.21.15.40:57054) 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 -- 172.21.15.40:0/2900937905 learned_addr learned my addr 172.21.15.40:0/2900937905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb5677fe640 1 --1- 172.21.15.40:0/2900937905 >> v1:172.21.15.40:6789/0 conn(0x7fb56815f3e0 0x7fb5680cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47508/0 (socket says 172.21.15.40:47508) 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 -- 172.21.15.40:0/2900937905 >> v1:172.21.15.184:6789/0 conn(0x7fb5680d5330 legacy=0x7fb5680cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 -- 172.21.15.40:0/2900937905 >> v1:172.21.15.40:6789/0 conn(0x7fb56815f3e0 legacy=0x7fb5680cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb566ffd640 1 -- 172.21.15.40:0/2900937905 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5680d0bb0 con 0x7fb5680d3d00 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.170+0000 7fb5667fc640 1 -- 172.21.15.40:0/2900937905 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175443520 0 0) 0x7fb5680d05f0 con 0x7fb56815f3e0 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.171+0000 7fb566ffd640 1 --2- 172.21.15.40:0/2900937905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d3d00 0x7fb5680d4df0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f3ac9535c11c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.171+0000 7fb5667fc640 1 -- 172.21.15.40:0/2900937905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037070 con 0x7fb5680d3d00 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.171+0000 7fb5667fc640 1 -- 172.21.15.40:0/2900937905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c032440 con 0x7fb5680d3d00 2024-04-26T21:19:02.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.171+0000 7fb5667fc640 1 -- 172.21.15.40:0/2900937905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c032810 con 0x7fb5680d3d00 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 -- 172.21.15.40:0/2900937905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d3d00 msgr2=0x7fb5680d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 --2- 172.21.15.40:0/2900937905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d3d00 0x7fb5680d4df0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 -- 172.21.15.40:0/2900937905 shutdown_connections 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 --2- 172.21.15.40:0/2900937905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d3d00 0x7fb5680d4df0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 -- 172.21.15.40:0/2900937905 >> 172.21.15.40:0/2900937905 conn(0x7fb56808ebf0 msgr2=0x7fb56809e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 -- 172.21.15.40:0/2900937905 shutdown_connections 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.172+0000 7fb56df14640 1 -- 172.21.15.40:0/2900937905 wait complete. 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.173+0000 7fb56df14640 1 Processor -- start 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.173+0000 7fb56df14640 1 -- start start 2024-04-26T21:19:02.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5680d3d00 0x7fb5680b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56815f3e0 0x7fb5680c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb56815f7c0 con 0x7fb5680d5330 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5680d5780 con 0x7fb5680d3d00 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb56df14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb56815f940 con 0x7fb56815f3e0 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57064/0 (socket says 172.21.15.40:57064) 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 -- 172.21.15.40:0/232988514 learned_addr learned my addr 172.21.15.40:0/232988514 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb5677fe640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5680d3d00 0x7fb5680b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 -- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56815f3e0 msgr2=0x7fb5680c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb567fff640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56815f3e0 0x7fb5680c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56815f3e0 0x7fb5680c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 -- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5680d3d00 msgr2=0x7fb5680b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5680d3d00 0x7fb5680b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.174+0000 7fb566ffd640 1 -- 172.21.15.40:0/232988514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56809f610 con 0x7fb5680d5330 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb566ffd640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037040 con 0x7fb5680d5330 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c004250 con 0x7fb5680d5330 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c03c670 con 0x7fb5680d5330 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5680b8aa0 con 0x7fb5680d5330 2024-04-26T21:19:02.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.175+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5680b9060 con 0x7fb5680d5330 2024-04-26T21:19:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.176+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56806a290 con 0x7fb5680d5330 2024-04-26T21:19:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.176+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb55c03a070 con 0x7fb5680d5330 2024-04-26T21:19:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.177+0000 7fb54ffff640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 0x7fb55406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.177+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb55c0ac440 con 0x7fb5680d5330 2024-04-26T21:19:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.177+0000 7fb5677fe640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 0x7fb55406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.178+0000 7fb5677fe640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 0x7fb55406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb5580012e0 tx=0x7fb558005eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.181+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb56809f610 con 0x7fb5680d5330 2024-04-26T21:19:02.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:19:02.300 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:19:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.367+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb568075fb0 con 0x7fb5680d5330 2024-04-26T21:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.368+0000 7fb54ffff640 1 -- 172.21.15.40:0/232988514 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7fb55c041020 con 0x7fb5680d5330 2024-04-26T21:19:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:02.369 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 msgr2=0x7fb55406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 0x7fb55406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb5580012e0 tx=0x7fb558005eb0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 msgr2=0x7fb56809f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009870 tx=0x7fb55c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 shutdown_connections 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb55406c8f0 0x7fb55406edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56815f3e0 0x7fb5680c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5680d5330 0x7fb56809f0d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 --2- 172.21.15.40:0/232988514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5680d3d00 0x7fb5680b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 >> 172.21.15.40:0/232988514 conn(0x7fb56808ebf0 msgr2=0x7fb5680ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 shutdown_connections 2024-04-26T21:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.370+0000 7fb56df14640 1 -- 172.21.15.40:0/232988514 wait complete. 2024-04-26T21:19:02.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:19:02.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.522+0000 7f9571589640 1 Processor -- start 2024-04-26T21:19:02.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.522+0000 7f9571589640 1 -- start start 2024-04-26T21:19:02.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f9571589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f956c15f340 0x7f956c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f9571589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f956c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f9571589640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c0b2510 con 0x7f956c15f720 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f9571589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f956c0b27f0 con 0x7f956c15f340 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f9571589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f956c0b2ad0 con 0x7f956c0d5610 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f956a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f956c15f720 0x7f956c0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47524/0 (socket says 172.21.15.40:47524) 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.523+0000 7f956a7fc640 1 -- 172.21.15.40:0/2358485354 learned_addr learned my addr 172.21.15.40:0/2358485354 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f956b7fe640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f956c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126772288 0 0) 0x7f956c0b2510 con 0x7f956c15f720 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9558003520 con 0x7f956c15f720 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f956affd640 1 --2- 172.21.15.40:0/2358485354 >> v2:172.21.15.184:3300/0 conn(0x7f956c15f340 0x7f956c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1664489407 0 0) 0x7f9558003520 con 0x7f956c15f720 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 msgr2=0x7f956c0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f956c0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 >> v2:172.21.15.184:3300/0 conn(0x7f956c15f340 msgr2=0x7f956c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 --2- 172.21.15.40:0/2358485354 >> v2:172.21.15.184:3300/0 conn(0x7f956c15f340 0x7f956c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0b2db0 con 0x7f956c15f720 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9554001670 con 0x7f956c15f720 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 >> v1:172.21.15.40:6789/0 conn(0x7f956c15f720 legacy=0x7f956c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f95580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9558003f30 0x7f9558004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558004d40 0x7f95580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0b27f0 con 0x7f9558003f30 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0b2ad0 con 0x7f9558004d40 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f955800ba10 con 0x7f956c0d5610 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f956a7fc640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9558003f30 0x7f9558004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f956affd640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f95580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.524+0000 7f956b7fe640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558004d40 0x7f95580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 -- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 msgr2=0x7f95580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0d5610 0x7f95580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 -- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558004d40 msgr2=0x7f95580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9558004d40 0x7f95580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 -- 172.21.15.40:0/2358485354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558003520 con 0x7f9558003f30 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f956a7fc640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9558003f30 0x7f9558004800 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9554007fd0 tx=0x7f9558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf73ceda3ddce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554007d50 con 0x7f9558003f30 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9554005ad0 con 0x7f9558003f30 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.525+0000 7f9569ffb640 1 -- 172.21.15.40:0/2358485354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554007070 con 0x7f9558003f30 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 -- 172.21.15.40:0/2358485354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9558003f30 msgr2=0x7f9558004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 --2- 172.21.15.40:0/2358485354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9558003f30 0x7f9558004800 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9554007fd0 tx=0x7f9558005830 comp rx=0 tx=0).stop 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f956b7fe640 1 -- 172.21.15.40:0/2358485354 reap_dead start 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 -- 172.21.15.40:0/2358485354 shutdown_connections 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 -- 172.21.15.40:0/2358485354 >> 172.21.15.40:0/2358485354 conn(0x7f956c08ec70 msgr2=0x7f956c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 -- 172.21.15.40:0/2358485354 shutdown_connections 2024-04-26T21:19:02.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.526+0000 7f9571589640 1 -- 172.21.15.40:0/2358485354 wait complete. 2024-04-26T21:19:02.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.527+0000 7f9571589640 1 Processor -- start 2024-04-26T21:19:02.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.527+0000 7f9571589640 1 -- start start 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9558003f30 0x7f956c1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 0x7f956c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d40d0 con 0x7f956c0ce700 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d3f50 con 0x7f956c1607c0 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f9571589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f956c0b35b0 con 0x7f9558003f30 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f956b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f956b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57090/0 (socket says 172.21.15.40:57090) 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f956b7fe640 1 -- 172.21.15.40:0/3295573572 learned_addr learned my addr 172.21.15.40:0/3295573572 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f956a7fc640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 0x7f956c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.528+0000 7f956affd640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9558003f30 0x7f956c1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 -- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9558003f30 msgr2=0x7f956c1f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9558003f30 0x7f956c1f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 -- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 msgr2=0x7f956c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 0x7f956c0ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 -- 172.21.15.40:0/3295573572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0cd270 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956a7fc640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 0x7f956c0ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f956b7fe640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f955c009fd0 tx=0x7f955c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c019070 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955c00fdb0 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c15cbb0 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c003bd0 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.529+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c15d170 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.530+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956c06a290 con 0x7f956c0ce700 2024-04-26T21:19:02.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.530+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f955c01c070 con 0x7f956c0ce700 2024-04-26T21:19:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.531+0000 7f95637fe640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 0x7f954c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.531+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f955c08c310 con 0x7f956c0ce700 2024-04-26T21:19:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.531+0000 7f956affd640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 0x7f954c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.532+0000 7f956affd640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 0x7f954c06ede0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9564009b00 tx=0x7f9564008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.535+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f955c010040 con 0x7f956c0ce700 2024-04-26T21:19:02.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.757+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f956c0d3020 con 0x7f956c0ce700 2024-04-26T21:19:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.758+0000 7f95637fe640 1 -- 172.21.15.40:0/3295573572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7f955c05a7d0 con 0x7f956c0ce700 2024-04-26T21:19:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:02.759 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:19:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:19:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 msgr2=0x7f954c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 0x7f954c06ede0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9564009b00 tx=0x7f9564008040 comp rx=0 tx=0).stop 2024-04-26T21:19:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 msgr2=0x7f956c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f955c009fd0 tx=0x7f955c00f580 comp rx=0 tx=0).stop 2024-04-26T21:19:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 shutdown_connections 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f954c06c920 0x7f954c06ede0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f956c0ce700 0x7f956c0ceb80 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c1607c0 0x7f956c0ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 --2- 172.21.15.40:0/3295573572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9558003f30 0x7f956c1f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 >> 172.21.15.40:0/3295573572 conn(0x7f956c08ec70 msgr2=0x7f956c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 shutdown_connections 2024-04-26T21:19:02.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.760+0000 7f9571589640 1 -- 172.21.15.40:0/3295573572 wait complete. 2024-04-26T21:19:02.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.872+0000 7fc948d79640 1 Processor -- start 2024-04-26T21:19:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.872+0000 7fc948d79640 1 -- start start 2024-04-26T21:19:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc948d79640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9440d11b0 0x7fc94406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc948d79640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc94406e2c0 0x7fc94415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc948d79640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9440d1740 con 0x7fc9440cf2e0 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc948d79640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc94406e710 con 0x7fc94406e2c0 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc948d79640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc9440d1a20 con 0x7fc9440d11b0 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc942d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc94406e2c0 0x7fc94415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc94259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc9440cf2e0 0x7fc94406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47534/0 (socket says 172.21.15.40:47534) 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc942d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc94406e2c0 0x7fc94415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57100/0 (socket says 172.21.15.40:57100) 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc94259c640 1 -- 172.21.15.40:0/2549734198 learned_addr learned my addr 172.21.15.40:0/2549734198 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc941d9b640 1 --2- 172.21.15.40:0/2549734198 >> v2:172.21.15.184:3300/0 conn(0x7fc9440d11b0 0x7fc94406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc942d9d640 1 -- 172.21.15.40:0/2549734198 >> v2:172.21.15.184:3300/0 conn(0x7fc9440d11b0 msgr2=0x7fc94406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.873+0000 7fc942d9d640 1 --2- 172.21.15.40:0/2549734198 >> v2:172.21.15.184:3300/0 conn(0x7fc9440d11b0 0x7fc94406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc942d9d640 1 -- 172.21.15.40:0/2549734198 >> v1:172.21.15.40:6789/0 conn(0x7fc9440cf2e0 legacy=0x7fc94406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc942d9d640 1 -- 172.21.15.40:0/2549734198 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9440d1d00 con 0x7fc94406e2c0 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc942d9d640 1 --2- 172.21.15.40:0/2549734198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc94406e2c0 0x7fc94415ebe0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0097a0 tx=0x7fc93c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2030cbce469ddf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc94159a640 1 -- 172.21.15.40:0/2549734198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c037070 con 0x7fc94406e2c0 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc94159a640 1 -- 172.21.15.40:0/2549734198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c032440 con 0x7fc94406e2c0 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.874+0000 7fc94159a640 1 -- 172.21.15.40:0/2549734198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c032810 con 0x7fc94406e2c0 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 -- 172.21.15.40:0/2549734198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc94406e2c0 msgr2=0x7fc94415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 --2- 172.21.15.40:0/2549734198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc94406e2c0 0x7fc94415ebe0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0097a0 tx=0x7fc93c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 -- 172.21.15.40:0/2549734198 shutdown_connections 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 --2- 172.21.15.40:0/2549734198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc94406e2c0 0x7fc94415ebe0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 --2- 172.21.15.40:0/2549734198 >> v2:172.21.15.184:3300/0 conn(0x7fc9440d11b0 0x7fc94406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 -- 172.21.15.40:0/2549734198 >> 172.21.15.40:0/2549734198 conn(0x7fc9440b4a60 msgr2=0x7fc9440b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 -- 172.21.15.40:0/2549734198 shutdown_connections 2024-04-26T21:19:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.875+0000 7fc948d79640 1 -- 172.21.15.40:0/2549734198 wait complete. 2024-04-26T21:19:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.876+0000 7fc948d79640 1 Processor -- start 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.876+0000 7fc948d79640 1 -- start start 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc94406e2c0 0x7fc944160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9440cf2e0 0x7fc9440a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9440cf6c0 con 0x7fc9440d11b0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9440d22c0 con 0x7fc9440cf2e0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc948d79640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc9440cf840 con 0x7fc94406e2c0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc942d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc942d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57106/0 (socket says 172.21.15.40:57106) 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.877+0000 7fc942d9d640 1 -- 172.21.15.40:0/1412727950 learned_addr learned my addr 172.21.15.40:0/1412727950 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc941d9b640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9440cf2e0 0x7fc9440a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc94259c640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc94406e2c0 0x7fc944160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 -- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc94406e2c0 msgr2=0x7fc944160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc94406e2c0 0x7fc944160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 -- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9440cf2e0 msgr2=0x7fc9440a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9440cf2e0 0x7fc9440a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 -- 172.21.15.40:0/1412727950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9440b3f50 con 0x7fc9440d11b0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc942d9d640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098c0 tx=0x7fc93c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c037040 con 0x7fc9440d11b0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c042070 con 0x7fc9440d11b0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9440b4230 con 0x7fc9440d11b0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c030b50 con 0x7fc9440d11b0 2024-04-26T21:19:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.878+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9441f6b10 con 0x7fc9440d11b0 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.879+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc944072000 con 0x7fc9440d11b0 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.880+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc93c030cf0 con 0x7fc9440d11b0 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.880+0000 7fc93affd640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 0x7fc92406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.880+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fc93c0ac690 con 0x7fc9440d11b0 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.880+0000 7fc94259c640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 0x7fc92406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.881+0000 7fc94259c640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 0x7fc92406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc9440c9f30 tx=0x7fc934009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:02.884+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc944072000 con 0x7fc9440d11b0 2024-04-26T21:19:03.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.111+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc94406a290 con 0x7fc9440d11b0 2024-04-26T21:19:03.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.111+0000 7fc93affd640 1 -- 172.21.15.40:0/1412727950 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5177 (secure 0 0 0) 0x7fc93c032e20 con 0x7fc9440d11b0 2024-04-26T21:19:03.112 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:03.112 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:18:44.382283+0000","modified":"2024-04-26T21:18:46.380682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/2864100946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2864100946},{"type":"v1","addr":"172.21.15.184:6837","nonce":2864100946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":7}]} 2024-04-26T21:19:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2024-04-26T21:19:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 msgr2=0x7fc92406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 0x7fc92406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc9440c9f30 tx=0x7fc934009040 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 msgr2=0x7fc9440c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098c0 tx=0x7fc93c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 shutdown_connections 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc92406c8f0 0x7fc92406edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9440d11b0 0x7fc9440c8810 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9440cf2e0 0x7fc9440a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 --2- 172.21.15.40:0/1412727950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc94406e2c0 0x7fc944160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 >> 172.21.15.40:0/1412727950 conn(0x7fc9440b4a60 msgr2=0x7fc944162ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 shutdown_connections 2024-04-26T21:19:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.113+0000 7fc948d79640 1 -- 172.21.15.40:0/1412727950 wait complete. 2024-04-26T21:19:03.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:03.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.224+0000 7f5683a93640 1 Processor -- start 2024-04-26T21:19:03.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 -- start start 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 0x7f567c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f567c15f750 0x7f567c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567c0d03a0 con 0x7f567c15f370 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f567c0b4d10 con 0x7f567c15f750 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.225+0000 7f5683a93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f567c0d0680 con 0x7f567c0d3ee0 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f567c15f750 0x7f567c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f567c15f750 0x7f567c0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57122/0 (socket says 172.21.15.40:57122) 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 -- 172.21.15.40:0/2551073300 learned_addr learned my addr 172.21.15.40:0/2551073300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f568182f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f567c15f370 0x7f567c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47548/0 (socket says 172.21.15.40:47548) 2024-04-26T21:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f568102e640 1 --2- 172.21.15.40:0/2551073300 >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 0x7f567c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 -- 172.21.15.40:0/2551073300 >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 msgr2=0x7f567c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 --2- 172.21.15.40:0/2551073300 >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 0x7f567c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 -- 172.21.15.40:0/2551073300 >> v1:172.21.15.40:6789/0 conn(0x7f567c15f370 legacy=0x7f567c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f568082d640 1 -- 172.21.15.40:0/2551073300 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531047692 0 0) 0x7f567c0d03a0 con 0x7f567c15f370 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f5682030640 1 -- 172.21.15.40:0/2551073300 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c0d0960 con 0x7f567c15f750 2024-04-26T21:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.226+0000 7f568102e640 1 --2- 172.21.15.40:0/2551073300 >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 0x7f567c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5682030640 1 --2- 172.21.15.40:0/2551073300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c15f750 0x7f567c0b47d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5674009e50 tx=0x7f567402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf34f743ffdaaab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f568082d640 1 -- 172.21.15.40:0/2551073300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674036070 con 0x7f567c15f750 2024-04-26T21:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f568082d640 1 -- 172.21.15.40:0/2551073300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5674031600 con 0x7f567c15f750 2024-04-26T21:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f568082d640 1 -- 172.21.15.40:0/2551073300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740319d0 con 0x7f567c15f750 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 -- 172.21.15.40:0/2551073300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c15f750 msgr2=0x7f567c0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 --2- 172.21.15.40:0/2551073300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c15f750 0x7f567c0b47d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5674009e50 tx=0x7f567402a760 comp rx=0 tx=0).stop 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 -- 172.21.15.40:0/2551073300 shutdown_connections 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 --2- 172.21.15.40:0/2551073300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c15f750 0x7f567c0b47d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 --2- 172.21.15.40:0/2551073300 >> v2:172.21.15.184:3300/0 conn(0x7f567c0d3ee0 0x7f567c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 -- 172.21.15.40:0/2551073300 >> 172.21.15.40:0/2551073300 conn(0x7f567c08ec10 msgr2=0x7f567c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 -- 172.21.15.40:0/2551073300 shutdown_connections 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.227+0000 7f5683a93640 1 -- 172.21.15.40:0/2551073300 wait complete. 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 Processor -- start 2024-04-26T21:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 -- start start 2024-04-26T21:19:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f567c15f370 0x7f567c0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f567c15f750 0x7f567c0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0b51c0 con 0x7f567c0d3ee0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0d0f20 con 0x7f567c15f750 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.229+0000 7f5683a93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0b5340 con 0x7f567c15f370 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57124/0 (socket says 172.21.15.40:57124) 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 -- 172.21.15.40:0/549787438 learned_addr learned my addr 172.21.15.40:0/549787438 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 -- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f567c15f370 msgr2=0x7f567c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f567c15f370 0x7f567c0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 -- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f567c15f750 msgr2=0x7f567c0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f567c15f750 0x7f567c0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 -- 172.21.15.40:0/549787438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c0b9cd0 con 0x7f567c0d3ee0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f568182f640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f567800c4d0 tx=0x7f567800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678018070 con 0x7f567c0d3ee0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5678007500 con 0x7f567c0d3ee0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c0b9fb0 con 0x7f567c0d3ee0 2024-04-26T21:19:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678004a20 con 0x7f567c0d3ee0 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.230+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c0ba570 con 0x7f567c0d3ee0 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.231+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c06a290 con 0x7f567c0d3ee0 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.232+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f567801b070 con 0x7f567c0d3ee0 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.232+0000 7f56727fc640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 0x7f566006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.232+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f567808ad40 con 0x7f567c0d3ee0 2024-04-26T21:19:03.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.232+0000 7f568102e640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 0x7f566006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.233+0000 7f568102e640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 0x7f566006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f566c006fd0 tx=0x7f566c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.236+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5678059200 con 0x7f567c0d3ee0 2024-04-26T21:19:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.423+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f567c162e40 con 0x7f567c0d3ee0 2024-04-26T21:19:03.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.424+0000 7f56727fc640 1 -- 172.21.15.40:0/549787438 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16608 (secure 0 0 0) 0x7f5678058b00 con 0x7f567c0d3ee0 2024-04-26T21:19:03.426 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:03.426 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":97,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:18:46.017365+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:18:42.648529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-26T21:18:43.328674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 msgr2=0x7f566006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 0x7f566006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f566c006fd0 tx=0x7f566c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 msgr2=0x7f567c0b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f567800c4d0 tx=0x7f567800c990 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 shutdown_connections 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f566006c8f0 0x7f566006edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f567c15f750 0x7f567c0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f567c15f370 0x7f567c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 --2- 172.21.15.40:0/549787438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f567c0d3ee0 0x7f567c0b7ed0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 >> 172.21.15.40:0/549787438 conn(0x7f567c08ec10 msgr2=0x7f567c06db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 shutdown_connections 2024-04-26T21:19:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.426+0000 7f5683a93640 1 -- 172.21.15.40:0/549787438 wait complete. 2024-04-26T21:19:03.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:19:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.539+0000 7f5cf4304640 1 Processor -- start 2024-04-26T21:19:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.539+0000 7f5cf4304640 1 -- start start 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf4304640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cec15d380 0x7f5cec0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf4304640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cec0d0be0 con 0x7f5cec0b56d0 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf4304640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cec0d0ec0 con 0x7f5cec160d50 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf4304640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5cec0d11a0 con 0x7f5cec15d380 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf20a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cec15d380 0x7f5cec0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf189f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5cec0b56d0 0x7f5cec0d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47560/0 (socket says 172.21.15.40:47560) 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf20a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5cec15d380 0x7f5cec0ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57138/0 (socket says 172.21.15.40:57138) 2024-04-26T21:19:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf189f640 1 -- 172.21.15.40:0/2417708014 learned_addr learned my addr 172.21.15.40:0/2417708014 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf20a0640 1 -- 172.21.15.40:0/2417708014 >> v1:172.21.15.184:6789/0 conn(0x7f5cec160d50 legacy=0x7f5cec0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf20a0640 1 -- 172.21.15.40:0/2417708014 >> v1:172.21.15.40:6789/0 conn(0x7f5cec0b56d0 legacy=0x7f5cec0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.540+0000 7f5cf20a0640 1 -- 172.21.15.40:0/2417708014 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec0d1480 con 0x7f5cec15d380 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf20a0640 1 --2- 172.21.15.40:0/2417708014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec15d380 0x7f5cec0ce590 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097a0 tx=0x7f5ce802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e78642f4a129b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf109e640 1 -- 172.21.15.40:0/2417708014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037070 con 0x7f5cec15d380 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf109e640 1 -- 172.21.15.40:0/2417708014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce802ae50 con 0x7f5cec15d380 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf109e640 1 -- 172.21.15.40:0/2417708014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032660 con 0x7f5cec15d380 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf4304640 1 -- 172.21.15.40:0/2417708014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec15d380 msgr2=0x7f5cec0ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.541+0000 7f5cf4304640 1 --2- 172.21.15.40:0/2417708014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec15d380 0x7f5cec0ce590 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097a0 tx=0x7f5ce802a140 comp rx=0 tx=0).stop 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.542+0000 7f5cf4304640 1 -- 172.21.15.40:0/2417708014 shutdown_connections 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.542+0000 7f5cf4304640 1 --2- 172.21.15.40:0/2417708014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec15d380 0x7f5cec0ce590 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.542+0000 7f5cf4304640 1 -- 172.21.15.40:0/2417708014 >> 172.21.15.40:0/2417708014 conn(0x7f5cec0b4a60 msgr2=0x7f5cec0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.542+0000 7f5cf4304640 1 -- 172.21.15.40:0/2417708014 shutdown_connections 2024-04-26T21:19:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.542+0000 7f5cf4304640 1 -- 172.21.15.40:0/2417708014 wait complete. 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 Processor -- start 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 -- start start 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cec15d380 0x7f5cec0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec160d50 0x7f5cec156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec0b64d0 con 0x7f5cec0b56d0 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec0cedc0 con 0x7f5cec160d50 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.543+0000 7f5cf4304640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5cec0b6650 con 0x7f5cec15d380 2024-04-26T21:19:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57144/0 (socket says 172.21.15.40:57144) 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 -- 172.21.15.40:0/1342537930 learned_addr learned my addr 172.21.15.40:0/1342537930 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf189f640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cec15d380 0x7f5cec0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf28a1640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec160d50 0x7f5cec156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 -- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cec15d380 msgr2=0x7f5cec0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cec15d380 0x7f5cec0cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 -- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec160d50 msgr2=0x7f5cec156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec160d50 0x7f5cec156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 -- 172.21.15.40:0/1342537930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec0ccf20 con 0x7f5cec0b56d0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf20a0640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8042070 con 0x7f5cec0b56d0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce803e670 con 0x7f5cec0b56d0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec0cb940 con 0x7f5cec0b56d0 2024-04-26T21:19:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037040 con 0x7f5cec0b56d0 2024-04-26T21:19:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.544+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec0cbf00 con 0x7f5cec0b56d0 2024-04-26T21:19:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.545+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec06a290 con 0x7f5cec0b56d0 2024-04-26T21:19:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.546+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ce8045070 con 0x7f5cec0b56d0 2024-04-26T21:19:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.546+0000 7f5ce2ffd640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 0x7f5cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.546+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5ce80ac2c0 con 0x7f5cec0b56d0 2024-04-26T21:19:03.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.546+0000 7f5cf189f640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 0x7f5cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.547+0000 7f5cf189f640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 0x7f5cd006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5cec159f00 tx=0x7f5cdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.550+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ce807a780 con 0x7f5cec0b56d0 2024-04-26T21:19:03.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.779+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5cec075a70 con 0x7f5cec0b56d0 2024-04-26T21:19:03.780 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:19:03.779+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:19:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.781+0000 7f5ce2ffd640 1 -- 172.21.15.40:0/1342537930 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f5ce80ac0e0 con 0x7f5cec0b56d0 2024-04-26T21:19:03.782 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 msgr2=0x7f5cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 0x7f5cd006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5cec159f00 tx=0x7f5cdc009040 comp rx=0 tx=0).stop 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 msgr2=0x7f5cec15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce80046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 shutdown_connections 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5cd006c8f0 0x7f5cd006edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec160d50 0x7f5cec156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cec15d380 0x7f5cec0cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 --2- 172.21.15.40:0/1342537930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cec0b56d0 0x7f5cec15eaf0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 >> 172.21.15.40:0/1342537930 conn(0x7f5cec0b4a60 msgr2=0x7f5cec15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 shutdown_connections 2024-04-26T21:19:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.783+0000 7f5cf4304640 1 -- 172.21.15.40:0/1342537930 wait complete. 2024-04-26T21:19:03.786 INFO:tasks.ceph.mds.b.smithi184.stderr: -2156> 2024-04-26T21:18:50.668+0000 7fb2ef917640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:19:03.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:19:03.815 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:19:03.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.898+0000 7fa2775c1640 1 Processor -- start 2024-04-26T21:19:03.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.898+0000 7fa2775c1640 1 -- start start 2024-04-26T21:19:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa2775c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 0x7fa2700d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa2775c1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2700d4240 con 0x7fa2700d22f0 2024-04-26T21:19:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa2775c1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2700d31a0 con 0x7fa2700d0ff0 2024-04-26T21:19:03.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa2775c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700d3480 con 0x7fa27015f490 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa275b5e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2700d22f0 0x7fa2700ba3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47564/0 (socket says 172.21.15.40:47564) 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa275b5e640 1 -- 172.21.15.40:0/4294638196 learned_addr learned my addr 172.21.15.40:0/4294638196 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa27535d640 1 --2- 172.21.15.40:0/4294638196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 0x7fa2700d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272095139 0 0) 0x7fa2700d4240 con 0x7fa2700d22f0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa258003520 con 0x7fa2700d22f0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979318860 0 0) 0x7fa2700d31a0 con 0x7fa2700d0ff0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2700d4240 con 0x7fa2700d0ff0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa27535d640 1 -- 172.21.15.40:0/4294638196 >> v1:172.21.15.184:6789/0 conn(0x7fa2700d0ff0 legacy=0x7fa2700d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa27535d640 1 -- 172.21.15.40:0/4294638196 >> v1:172.21.15.40:6789/0 conn(0x7fa2700d22f0 legacy=0x7fa2700ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629377819 0 0) 0x7fa258003520 con 0x7fa2700d22f0 2024-04-26T21:19:03.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.899+0000 7fa27535d640 1 -- 172.21.15.40:0/4294638196 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700d3760 con 0x7fa27015f490 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.900+0000 7fa27535d640 1 --2- 172.21.15.40:0/4294638196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 0x7fa2700d0ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2680097d0 tx=0x7fa26802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1f3989677b5e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.900+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268037070 con 0x7fa27015f490 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.900+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26802ac30 con 0x7fa27015f490 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.900+0000 7fa26ffff640 1 -- 172.21.15.40:0/4294638196 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600019f0 con 0x7fa27015f490 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 -- 172.21.15.40:0/4294638196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 msgr2=0x7fa2700d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 --2- 172.21.15.40:0/4294638196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 0x7fa2700d0ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2680097d0 tx=0x7fa26802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 -- 172.21.15.40:0/4294638196 shutdown_connections 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 --2- 172.21.15.40:0/4294638196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa27015f490 0x7fa2700d0ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 -- 172.21.15.40:0/4294638196 >> 172.21.15.40:0/4294638196 conn(0x7fa27009e530 msgr2=0x7fa2700d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:03.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 -- 172.21.15.40:0/4294638196 shutdown_connections 2024-04-26T21:19:03.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.901+0000 7fa2775c1640 1 -- 172.21.15.40:0/4294638196 wait complete. 2024-04-26T21:19:03.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.902+0000 7fa2775c1640 1 Processor -- start 2024-04-26T21:19:03.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.902+0000 7fa2775c1640 1 -- start start 2024-04-26T21:19:03.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 0x7fa27014f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 0x7fa2700cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa27015f870 con 0x7fa27015f490 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700d3d20 con 0x7fa2700d22f0 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa2775c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700d2dc0 con 0x7fa2700d0ff0 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57158/0 (socket says 172.21.15.40:57158) 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 -- 172.21.15.40:0/1524694286 learned_addr learned my addr 172.21.15.40:0/1524694286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa274b5c640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 0x7fa2700cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa27535d640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 0x7fa27014f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 -- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 msgr2=0x7fa27014f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 0x7fa27014f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 -- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 msgr2=0x7fa2700cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 0x7fa2700cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 -- 172.21.15.40:0/1524694286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700cde10 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa27535d640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 0x7fa27014f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa274b5c640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 0x7fa2700cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.903+0000 7fa275b5e640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa26000cbe0 tx=0x7fa26000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.904+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260017070 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.904+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26000ad40 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.904+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2700ce0f0 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.904+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260012620 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.904+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2700ce6b0 con 0x7fa27015f490 2024-04-26T21:19:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.905+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27015fab0 con 0x7fa27015f490 2024-04-26T21:19:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.905+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa26001a070 con 0x7fa27015f490 2024-04-26T21:19:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.906+0000 7fa26dffb640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 0x7fa25c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.906+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fa26008ba80 con 0x7fa27015f490 2024-04-26T21:19:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.906+0000 7fa27535d640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 0x7fa25c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.907+0000 7fa27535d640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 0x7fa25c06ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa268034000 tx=0x7fa26802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:03.910+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa27015fab0 con 0x7fa27015f490 2024-04-26T21:19:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.135+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa27006a290 con 0x7fa27015f490 2024-04-26T21:19:04.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.782+0000 7fa26dffb640 1 -- 172.21.15.40:0/1524694286 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7fa260059e50 con 0x7fa27015f490 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 msgr2=0x7fa25c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 0x7fa25c06ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa268034000 tx=0x7fa26802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 msgr2=0x7fa27006cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa26000cbe0 tx=0x7fa26000a430 comp rx=0 tx=0).stop 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 shutdown_connections 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa25c06c920 0x7fa25c06ede0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa27015f490 0x7fa27006cf30 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2700d22f0 0x7fa2700cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 --2- 172.21.15.40:0/1524694286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2700d0ff0 0x7fa27014f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 >> 172.21.15.40:0/1524694286 conn(0x7fa27009e530 msgr2=0x7fa27015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 shutdown_connections 2024-04-26T21:19:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.785+0000 7fa2775c1640 1 -- 172.21.15.40:0/1524694286 wait complete. 2024-04-26T21:19:04.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:19:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.910+0000 7ff4b1ebd640 1 Processor -- start 2024-04-26T21:19:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.910+0000 7ff4b1ebd640 1 -- start start 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.911+0000 7ff4b1ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 0x7ff4ac0d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.911+0000 7ff4b1ebd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 0x7ff4ac0d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.911+0000 7ff4b1ebd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0d33a0 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.911+0000 7ff4b1ebd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac0b4970 con 0x7ff4ac0b6020 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.911+0000 7ff4b1ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d3680 con 0x7ff4ac162a10 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4aaffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 0x7ff4ac0d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 0x7ff4ac0d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4abfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff4ac0d1f70 0x7ff4ac163100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:58620/0 (socket says 172.21.15.40:58620) 2024-04-26T21:19:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4abfff640 1 -- 172.21.15.40:0/1088446537 learned_addr learned my addr 172.21.15.40:0/1088446537 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4ab7fe640 1 -- 172.21.15.40:0/1088446537 >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 msgr2=0x7ff4ac0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4ab7fe640 1 --2- 172.21.15.40:0/1088446537 >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 0x7ff4ac0d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4ab7fe640 1 -- 172.21.15.40:0/1088446537 >> v1:172.21.15.184:6789/0 conn(0x7ff4ac0d1f70 legacy=0x7ff4ac163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4ab7fe640 1 -- 172.21.15.40:0/1088446537 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0d3960 con 0x7ff4ac162a10 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.912+0000 7ff4aaffd640 1 --2- 172.21.15.40:0/1088446537 >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 0x7ff4ac0d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4ab7fe640 1 --2- 172.21.15.40:0/1088446537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 0x7ff4ac0d2dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ec468276643efb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4aa7fc640 1 -- 172.21.15.40:0/1088446537 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c037070 con 0x7ff4ac162a10 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4aa7fc640 1 -- 172.21.15.40:0/1088446537 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff49c032440 con 0x7ff4ac162a10 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4aa7fc640 1 -- 172.21.15.40:0/1088446537 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c032810 con 0x7ff4ac162a10 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/1088446537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 msgr2=0x7ff4ac0d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.913+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/1088446537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 0x7ff4ac0d2dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02a180 comp rx=0 tx=0).stop 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/1088446537 shutdown_connections 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/1088446537 >> v2:172.21.15.184:3300/0 conn(0x7ff4ac0b6020 0x7ff4ac0d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/1088446537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac162a10 0x7ff4ac0d2dc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/1088446537 >> 172.21.15.40:0/1088446537 conn(0x7ff4ac09e530 msgr2=0x7ff4ac0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/1088446537 shutdown_connections 2024-04-26T21:19:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.914+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/1088446537 wait complete. 2024-04-26T21:19:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.915+0000 7ff4b1ebd640 1 Processor -- start 2024-04-26T21:19:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.915+0000 7ff4b1ebd640 1 -- start start 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac0b6020 0x7ff4ac0b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4ac162a10 0x7ff4ac0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac162df0 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d3f20 con 0x7ff4ac162a10 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0b6600 con 0x7ff4ac0b6020 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57160/0 (socket says 172.21.15.40:57160) 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 -- 172.21.15.40:0/2937603859 learned_addr learned my addr 172.21.15.40:0/2937603859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4ab7fe640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac0b6020 0x7ff4ac0b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4abfff640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4ac162a10 0x7ff4ac0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 -- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac0b6020 msgr2=0x7ff4ac0b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac0b6020 0x7ff4ac0b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 -- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4ac162a10 msgr2=0x7ff4ac0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4ac162a10 0x7ff4ac0b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 -- 172.21.15.40:0/2937603859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0b9ca0 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4aaffd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000e990 tx=0x7ff4a000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0017070 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a000c9e0 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000cdb0 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.916+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac0b9f80 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.917+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac0ba540 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.918+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac06a290 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.918+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4a001a070 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.919+0000 7ff493fff640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 0x7ff49806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.919+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7ff4a008be10 con 0x7ff4ac0d1f70 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.919+0000 7ff4ab7fe640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 0x7ff49806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.920+0000 7ff4ab7fe640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 0x7ff49806edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff49c034000 tx=0x7ff49c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:04.923+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4a008f030 con 0x7ff4ac0d1f70 2024-04-26T21:19:05.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:05.111+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff4ac0d27a0 con 0x7ff4ac0d1f70 2024-04-26T21:19:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.095+0000 7ff493fff640 1 -- 172.21.15.40:0/2937603859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7ff4a0059af0 con 0x7ff4ac0d1f70 2024-04-26T21:19:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:19:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.099+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 msgr2=0x7ff49806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.099+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 0x7ff49806edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff49c034000 tx=0x7ff49c02e000 comp rx=0 tx=0).stop 2024-04-26T21:19:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.099+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 msgr2=0x7ff4ac1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.099+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000e990 tx=0x7ff4a000ee50 comp rx=0 tx=0).stop 2024-04-26T21:19:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 shutdown_connections 2024-04-26T21:19:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49806c8f0 0x7ff49806edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4ac162a10 0x7ff4ac0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4ac0d1f70 0x7ff4ac1507a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 --2- 172.21.15.40:0/2937603859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4ac0b6020 0x7ff4ac0b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.100+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 >> 172.21.15.40:0/2937603859 conn(0x7ff4ac09e530 msgr2=0x7ff4ac06d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.102+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 shutdown_connections 2024-04-26T21:19:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.102+0000 7ff4b1ebd640 1 -- 172.21.15.40:0/2937603859 wait complete. 2024-04-26T21:19:06.117 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:19:06.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.230+0000 7f7830124640 1 Processor -- start 2024-04-26T21:19:06.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 -- start start 2024-04-26T21:19:06.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7828162a10 0x7f78280d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280b6020 0x7f78280d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78280d1f70 0x7f7828163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f78280b4970 con 0x7f78280d1f70 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f78280d33a0 con 0x7f7828162a10 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f7830124640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78280d3680 con 0x7f78280b6020 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.231+0000 7f782e6c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78280d1f70 0x7f7828163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78280d1f70 0x7f7828163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57172/0 (socket says 172.21.15.40:57172) 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 -- 172.21.15.40:0/202646135 learned_addr learned my addr 172.21.15.40:0/202646135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782dec0640 1 --2- 172.21.15.40:0/202646135 >> v2:172.21.15.184:3300/0 conn(0x7f7828162a10 0x7f78280d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782d6bf640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280b6020 0x7f78280d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 -- 172.21.15.40:0/202646135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280b6020 msgr2=0x7f78280d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280b6020 0x7f78280d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 -- 172.21.15.40:0/202646135 >> v2:172.21.15.184:3300/0 conn(0x7f7828162a10 msgr2=0x7f78280d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 --2- 172.21.15.40:0/202646135 >> v2:172.21.15.184:3300/0 conn(0x7f7828162a10 0x7f78280d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.232+0000 7f782e6c1640 1 -- 172.21.15.40:0/202646135 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280d3960 con 0x7f78280d1f70 2024-04-26T21:19:06.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f782e6c1640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78280d1f70 0x7f7828163100 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f782808eda0 tx=0x7f781802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af67f79be7c52835 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:06.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f782cebe640 1 -- 172.21.15.40:0/202646135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818036070 con 0x7f78280d1f70 2024-04-26T21:19:06.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f782cebe640 1 -- 172.21.15.40:0/202646135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818031600 con 0x7f78280d1f70 2024-04-26T21:19:06.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f782cebe640 1 -- 172.21.15.40:0/202646135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78180319d0 con 0x7f78280d1f70 2024-04-26T21:19:06.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 -- 172.21.15.40:0/202646135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78280d1f70 msgr2=0x7f7828163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:06.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78280d1f70 0x7f7828163100 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f782808eda0 tx=0x7f781802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 -- 172.21.15.40:0/202646135 shutdown_connections 2024-04-26T21:19:06.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78280d1f70 0x7f7828163100 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 --2- 172.21.15.40:0/202646135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280b6020 0x7f78280d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 --2- 172.21.15.40:0/202646135 >> v2:172.21.15.184:3300/0 conn(0x7f7828162a10 0x7f78280d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 -- 172.21.15.40:0/202646135 >> 172.21.15.40:0/202646135 conn(0x7f782809e530 msgr2=0x7f78280d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 -- 172.21.15.40:0/202646135 shutdown_connections 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.233+0000 7f7830124640 1 -- 172.21.15.40:0/202646135 wait complete. 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 Processor -- start 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 -- start start 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78280b6020 0x7f78280cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280d1f70 0x7f782815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7828162df0 con 0x7f7828162a10 2024-04-26T21:19:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78280d3f20 con 0x7f78280b6020 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f7830124640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78280b6600 con 0x7f78280d1f70 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f782e6c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f782e6c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57174/0 (socket says 172.21.15.40:57174) 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.235+0000 7f782e6c1640 1 -- 172.21.15.40:0/1548976433 learned_addr learned my addr 172.21.15.40:0/1548976433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782dec0640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78280b6020 0x7f78280cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782d6bf640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280d1f70 0x7f782815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 -- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280d1f70 msgr2=0x7f782815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280d1f70 0x7f782815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 -- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78280b6020 msgr2=0x7f78280cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78280b6020 0x7f78280cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 -- 172.21.15.40:0/1548976433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280b8d60 con 0x7f7828162a10 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f782e6c1640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7818034040 tx=0x7f7818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818036050 con 0x7f7828162a10 2024-04-26T21:19:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818042070 con 0x7f7828162a10 2024-04-26T21:19:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78280b9070 con 0x7f7828162a10 2024-04-26T21:19:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818009510 con 0x7f7828162a10 2024-04-26T21:19:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.236+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78280b98b0 con 0x7f7828162a10 2024-04-26T21:19:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.237+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782806a290 con 0x7f7828162a10 2024-04-26T21:19:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.238+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7818007bd0 con 0x7f7828162a10 2024-04-26T21:19:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.238+0000 7f78267fc640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 0x7f781406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.238+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5357+0+0 (secure 0 0 0) 0x7f78180ac840 con 0x7f7828162a10 2024-04-26T21:19:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.238+0000 7f782dec0640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 0x7f781406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.239+0000 7f782dec0640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 0x7f781406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7820009a50 tx=0x7f7820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.242+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78280b9070 con 0x7f7828162a10 2024-04-26T21:19:06.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:06.429+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f78280d27a0 con 0x7f7828162a10 2024-04-26T21:19:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.103+0000 7f78267fc640 1 -- 172.21.15.40:0/1548976433 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f7818045360 con 0x7f7828162a10 2024-04-26T21:19:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 msgr2=0x7f781406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 0x7f781406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7820009a50 tx=0x7f7820008040 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 msgr2=0x7f78280b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7818034040 tx=0x7f7818004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 shutdown_connections 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f781406c8f0 0x7f781406edb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7828162a10 0x7f78280b2e70 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78280d1f70 0x7f782815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 --2- 172.21.15.40:0/1548976433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78280b6020 0x7f78280cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 >> 172.21.15.40:0/1548976433 conn(0x7f782809e530 msgr2=0x7f78280d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 shutdown_connections 2024-04-26T21:19:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.106+0000 7f7830124640 1 -- 172.21.15.40:0/1548976433 wait complete. 2024-04-26T21:19:07.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:19:07.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.238+0000 7fb7f7aa9640 1 Processor -- start 2024-04-26T21:19:07.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.238+0000 7fb7f7aa9640 1 -- start start 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f7aa9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7f016a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f7aa9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f009c700 con 0x7fb7f00d0010 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f7aa9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f00d1c50 con 0x7fb7f00b3c20 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f7aa9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f00b4480 con 0x7fb7f006db10 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f5845640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7f00d0010 0x7fb7f00b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47568/0 (socket says 172.21.15.40:47568) 2024-04-26T21:19:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f5845640 1 -- 172.21.15.40:0/3477183937 learned_addr learned my addr 172.21.15.40:0/3477183937 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 575826182 0 0) 0x7fb7f009c700 con 0x7fb7f00d0010 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.239+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7d8003520 con 0x7fb7f00d0010 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7f016a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473973001 0 0) 0x7fb7d8003520 con 0x7fb7f00d0010 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 msgr2=0x7fb7f016a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7f016a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 >> v1:172.21.15.184:6789/0 conn(0x7fb7f00b3c20 legacy=0x7fb7f00b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f00d1f30 con 0x7fb7f00d0010 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fb7e8001670 con 0x7fb7f00d0010 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 >> v1:172.21.15.40:6789/0 conn(0x7fb7f00d0010 legacy=0x7fb7f00b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7d80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d8003f60 0x7fb7d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00b4480 con 0x7fb7d8004d40 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d800baa0 con 0x7fb7d8003f60 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d800bca0 con 0x7fb7f006db10 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f5044640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d8003f60 0x7fb7d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f5845640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7d80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 -- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 msgr2=0x7fb7d80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7d80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 -- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d8003f60 msgr2=0x7fb7d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d8003f60 0x7fb7d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f6046640 1 -- 172.21.15.40:0/3477183937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8003520 con 0x7fb7d8004d40 2024-04-26T21:19:07.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.240+0000 7fb7f5845640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f006db10 0x7fb7d80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.241+0000 7fb7f6046640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009fc0 tx=0x7fb7d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6db7292c7137250 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.241+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0009c10 con 0x7fb7d8004d40 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.241+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e0036070 con 0x7fb7d8004d40 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.241+0000 7fb7f4843640 1 -- 172.21.15.40:0/3477183937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0031880 con 0x7fb7d8004d40 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/3477183937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 msgr2=0x7fb7d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/3477183937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009fc0 tx=0x7fb7d8005830 comp rx=0 tx=0).stop 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f6046640 1 -- 172.21.15.40:0/3477183937 reap_dead start 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/3477183937 shutdown_connections 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/3477183937 >> 172.21.15.40:0/3477183937 conn(0x7fb7f009ba40 msgr2=0x7fb7f009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/3477183937 shutdown_connections 2024-04-26T21:19:07.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.242+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/3477183937 wait complete. 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.243+0000 7fb7f7aa9640 1 Processor -- start 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.243+0000 7fb7f7aa9640 1 -- start start 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f0152bd0 0x7fb7f0163680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f015a770 0x7fb7f00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f0168720 con 0x7fb7d8004d40 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01685a0 con 0x7fb7f0152bd0 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f01688a0 con 0x7fb7f015a770 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57196/0 (socket says 172.21.15.40:57196) 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 -- 172.21.15.40:0/2058564875 learned_addr learned my addr 172.21.15.40:0/2058564875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5044640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f0152bd0 0x7fb7f0163680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 -- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f015a770 msgr2=0x7fb7f00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f015a770 0x7fb7f00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 -- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f0152bd0 msgr2=0x7fb7f0163680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f0152bd0 0x7fb7f0163680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 -- 172.21.15.40:0/2058564875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f01f6b70 con 0x7fb7d8004d40 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f5845640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e920 tx=0x7fb7e800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800c7f0 con 0x7fb7d8004d40 2024-04-26T21:19:07.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e8018070 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f01529d0 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.244+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800cb90 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.245+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01f70c0 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.246+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f006a290 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.246+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7e800cd30 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.246+0000 7fb7ee7fc640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 0x7fb7dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.246+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7fb7e808b3e0 con 0x7fb7d8004d40 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.247+0000 7fb7f5044640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 0x7fb7dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.247+0000 7fb7f5044640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 0x7fb7dc06ede0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00cf260 tx=0x7fb7e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.251+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb7f01529d0 con 0x7fb7d8004d40 2024-04-26T21:19:07.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.435+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb7f0068e00 con 0x7fb7d8004d40 2024-04-26T21:19:07.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.439+0000 7fb7ee7fc640 1 -- 172.21.15.40:0/2058564875 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7fb7e8059b00 con 0x7fb7d8004d40 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 msgr2=0x7fb7dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 0x7fb7dc06ede0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00cf260 tx=0x7fb7e4009040 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 msgr2=0x7fb7f0152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e920 tx=0x7fb7e800ede0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 shutdown_connections 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb7dc06c920 0x7fb7dc06ede0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f015a770 0x7fb7f00708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f0152bd0 0x7fb7f0163680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 --2- 172.21.15.40:0/2058564875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d8004d40 0x7fb7f0152490 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 >> 172.21.15.40:0/2058564875 conn(0x7fb7f009ba40 msgr2=0x7fb7f006ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 shutdown_connections 2024-04-26T21:19:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.441+0000 7fb7f7aa9640 1 -- 172.21.15.40:0/2058564875 wait complete. 2024-04-26T21:19:07.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-04-26T21:19:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.553+0000 7f453aacf640 1 Processor -- start 2024-04-26T21:19:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.553+0000 7f453aacf640 1 -- start start 2024-04-26T21:19:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453aacf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4534160970 0x7f45340d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453aacf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 0x7f45340d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453aacf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4534070e10 con 0x7f45340717c0 2024-04-26T21:19:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453aacf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f45340710f0 con 0x7f4534160970 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453aacf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f45340713d0 con 0x7f45340720d0 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453886b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4534160970 0x7f45340d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453906c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f45340717c0 0x7f453409ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47578/0 (socket says 172.21.15.40:47578) 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453906c640 1 -- 172.21.15.40:0/1791518236 learned_addr learned my addr 172.21.15.40:0/1791518236 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f452bfff640 1 --2- 172.21.15.40:0/1791518236 >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 0x7f45340d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453886b640 1 -- 172.21.15.40:0/1791518236 >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 msgr2=0x7f45340d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453886b640 1 --2- 172.21.15.40:0/1791518236 >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 0x7f45340d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453886b640 1 -- 172.21.15.40:0/1791518236 >> v1:172.21.15.40:6789/0 conn(0x7f45340717c0 legacy=0x7f453409ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f453886b640 1 -- 172.21.15.40:0/1791518236 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45340b58a0 con 0x7f4534160970 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f452b7fe640 1 -- 172.21.15.40:0/1791518236 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099618302 0 0) 0x7f4534070e10 con 0x7f45340717c0 2024-04-26T21:19:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.554+0000 7f452bfff640 1 --2- 172.21.15.40:0/1791518236 >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 0x7f45340d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.555+0000 7f453886b640 1 --2- 172.21.15.40:0/1791518236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f45340d12e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f45340a0fe0 tx=0x7f453002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664dc3975d95d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.555+0000 7f452b7fe640 1 -- 172.21.15.40:0/1791518236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530037070 con 0x7f4534160970 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.555+0000 7f452b7fe640 1 -- 172.21.15.40:0/1791518236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530032440 con 0x7f4534160970 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.555+0000 7f452b7fe640 1 -- 172.21.15.40:0/1791518236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530032810 con 0x7f4534160970 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 -- 172.21.15.40:0/1791518236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 msgr2=0x7f45340d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 --2- 172.21.15.40:0/1791518236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f45340d12e0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f45340a0fe0 tx=0x7f453002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 -- 172.21.15.40:0/1791518236 shutdown_connections 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 --2- 172.21.15.40:0/1791518236 >> v2:172.21.15.184:3300/0 conn(0x7f45340720d0 0x7f45340d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 --2- 172.21.15.40:0/1791518236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f45340d12e0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 -- 172.21.15.40:0/1791518236 >> 172.21.15.40:0/1791518236 conn(0x7f45340b4a10 msgr2=0x7f45340b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 -- 172.21.15.40:0/1791518236 shutdown_connections 2024-04-26T21:19:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.556+0000 7f453aacf640 1 -- 172.21.15.40:0/1791518236 wait complete. 2024-04-26T21:19:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.557+0000 7f453aacf640 1 Processor -- start 2024-04-26T21:19:07.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.557+0000 7f453aacf640 1 -- start start 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45340717c0 0x7f45340b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 0x7f453414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45340b5530 con 0x7f4534160970 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4534160d50 con 0x7f45340717c0 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453aacf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f45340ceef0 con 0x7f45340720d0 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57212/0 (socket says 172.21.15.40:57212) 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 -- 172.21.15.40:0/1426448003 learned_addr learned my addr 172.21.15.40:0/1426448003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453886b640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45340717c0 0x7f45340b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f452bfff640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 0x7f453414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 -- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 msgr2=0x7f453414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 0x7f453414eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 -- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45340717c0 msgr2=0x7f45340b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45340717c0 0x7f45340b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f453906c640 1 -- 172.21.15.40:0/1426448003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45340b6830 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.558+0000 7f452bfff640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 0x7f453414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f453906c640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c7d0 tx=0x7f452c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c018070 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c00a640 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45340b6b10 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c00aa70 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.559+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45340b70d0 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.560+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453406a290 con 0x7f4534160970 2024-04-26T21:19:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.560+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f452c01b070 con 0x7f4534160970 2024-04-26T21:19:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.561+0000 7f45297fa640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 0x7f452006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.561+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f452c08ae10 con 0x7f4534160970 2024-04-26T21:19:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.561+0000 7f453886b640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 0x7f452006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.562+0000 7f453886b640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 0x7f452006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4530034000 tx=0x7f453002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.565+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f452c08e030 con 0x7f4534160970 2024-04-26T21:19:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:07.820+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f4534160120 con 0x7f4534160970 2024-04-26T21:19:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.105+0000 7f45297fa640 1 -- 172.21.15.40:0/1426448003 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v363) ==== 150+0+0 (secure 0 0 0) 0x7f452c058ee0 con 0x7f4534160970 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 msgr2=0x7f452006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 0x7f452006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4530034000 tx=0x7f453002e000 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 msgr2=0x7f453414f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c7d0 tx=0x7f452c00cc90 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 shutdown_connections 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f452006c8f0 0x7f452006edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4534160970 0x7f453414f3e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45340720d0 0x7f453414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 --2- 172.21.15.40:0/1426448003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45340717c0 0x7f45340b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 >> 172.21.15.40:0/1426448003 conn(0x7f45340b4a10 msgr2=0x7f45340b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 shutdown_connections 2024-04-26T21:19:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.107+0000 7f453aacf640 1 -- 172.21.15.40:0/1426448003 wait complete. 2024-04-26T21:19:08.120 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... skipped 'Require FUSE client to inject client release failure' 2024-04-26T21:19:08.120 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:19:08.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.224+0000 7f496036a640 1 Processor -- start 2024-04-26T21:19:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.224+0000 7f496036a640 1 -- start start 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f496036a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4958145980 0x7f4958160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f496036a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49580b4260 con 0x7f49580b61a0 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f496036a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49580b4530 con 0x7f49580b3de0 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f496036a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f49580b5140 con 0x7f4958145980 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f495e907640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49580b61a0 0x7f49580720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47592/0 (socket says 172.21.15.40:47592) 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f495e907640 1 -- 172.21.15.40:0/3560380887 learned_addr learned my addr 172.21.15.40:0/3560380887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f495e106640 1 --2- 172.21.15.40:0/3560380887 >> v2:172.21.15.184:3300/0 conn(0x7f4958145980 0x7f4958160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312965520 0 0) 0x7f49580b4260 con 0x7f49580b61a0 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.225+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4940003520 con 0x7f49580b61a0 2024-04-26T21:19:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089518110 0 0) 0x7f49580b4530 con 0x7f49580b3de0 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49580b4260 con 0x7f49580b3de0 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2655546911 0 0) 0x7f4940003520 con 0x7f49580b61a0 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 >> v2:172.21.15.184:3300/0 conn(0x7f4958145980 msgr2=0x7f4958160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 --2- 172.21.15.40:0/3560380887 >> v2:172.21.15.184:3300/0 conn(0x7f4958145980 0x7f4958160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 >> v1:172.21.15.184:6789/0 conn(0x7f49580b3de0 legacy=0x7f49580b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49580b4810 con 0x7f49580b61a0 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f49480019f0 con 0x7f49580b61a0 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 >> v1:172.21.15.40:6789/0 conn(0x7f49580b61a0 legacy=0x7f49580720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 0x7f4940003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4940004800 0x7f4940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940005210 0x7f494000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b5140 con 0x7f4940003b90 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f494000c140 con 0x7f4940005210 2024-04-26T21:19:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f494000c340 con 0x7f4940004800 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 0x7f4940003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e907640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940005210 0x7f494000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d905640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4940004800 0x7f4940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 -- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4940004800 msgr2=0x7f4940004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4940004800 0x7f4940004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 -- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940005210 msgr2=0x7f494000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940005210 0x7f494000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e106640 1 -- 172.21.15.40:0/3560380887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4940003520 con 0x7f4940003b90 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495e907640 1 -- 172.21.15.40:0/3560380887 reap_dead start 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.226+0000 7f495d905640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4940004800 0x7f4940004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.227+0000 7f495e106640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 0x7f4940003f90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ac0 tx=0x7f4940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725140821f80e915 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.227+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950036070 con 0x7f4940003b90 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.227+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4950031400 con 0x7f4940003b90 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.227+0000 7f495d104640 1 -- 172.21.15.40:0/3560380887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950009690 con 0x7f4940003b90 2024-04-26T21:19:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 -- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 msgr2=0x7f4940003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 0x7f4940003f90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ac0 tx=0x7f4940005830 comp rx=0 tx=0).stop 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 -- 172.21.15.40:0/3560380887 shutdown_connections 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 --2- 172.21.15.40:0/3560380887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940003b90 0x7f4940003f90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 -- 172.21.15.40:0/3560380887 >> 172.21.15.40:0/3560380887 conn(0x7f49580d0d30 msgr2=0x7f49580d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 -- 172.21.15.40:0/3560380887 shutdown_connections 2024-04-26T21:19:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.228+0000 7f496036a640 1 -- 172.21.15.40:0/3560380887 wait complete. 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.229+0000 7f496036a640 1 Processor -- start 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.229+0000 7f496036a640 1 -- start start 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940003b90 0x7f495813e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f495813fcc0 0x7f49580cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4958160630 con 0x7f4958151ce0 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b4dd0 con 0x7f4940003b90 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f496036a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49581607b0 con 0x7f495813fcc0 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57228/0 (socket says 172.21.15.40:57228) 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 -- 172.21.15.40:0/4133032761 learned_addr learned my addr 172.21.15.40:0/4133032761 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495e106640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940003b90 0x7f495813e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495e907640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f495813fcc0 0x7f49580cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 -- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f495813fcc0 msgr2=0x7f49580cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f495813fcc0 0x7f49580cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 -- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940003b90 msgr2=0x7f495813e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940003b90 0x7f495813e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 -- 172.21.15.40:0/4133032761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495813ec20 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f495d905640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f494c0099a0 tx=0x7f494c001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.230+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c008110 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.231+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c0082b0 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.231+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495813ef00 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.231+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0067d0 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.231+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495813f4c0 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.231+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580b1c50 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.232+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f494c013070 con 0x7f4958151ce0 2024-04-26T21:19:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.232+0000 7f4956ffd640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 0x7f494406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.233+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f494c08b2e0 con 0x7f4958151ce0 2024-04-26T21:19:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.233+0000 7f495e106640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 0x7f494406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.233+0000 7f495e106640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 0x7f494406ede0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4950033000 tx=0x7f4950008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.237+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49580b1c50 con 0x7f4958151ce0 2024-04-26T21:19:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.422+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4958073cc0 con 0x7f4958151ce0 2024-04-26T21:19:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.423+0000 7f4956ffd640 1 -- 172.21.15.40:0/4133032761 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+13024 (secure 0 0 0) 0x7f494c0593c0 con 0x7f4958151ce0 2024-04-26T21:19:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:08.424 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":100,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:07.096464+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/2864100946":"2024-04-27T21:19:03.780479+0000","172.21.15.184:6835/2864100946":"2024-04-27T21:19:03.780479+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:08.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 msgr2=0x7f494406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 0x7f494406ede0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4950033000 tx=0x7f4950008030 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 msgr2=0x7f49581f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f494c0099a0 tx=0x7f494c001860 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 shutdown_connections 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f494406c920 0x7f494406ede0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f495813fcc0 0x7f49580cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4958151ce0 0x7f49581f5000 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 --2- 172.21.15.40:0/4133032761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4940003b90 0x7f495813e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 >> 172.21.15.40:0/4133032761 conn(0x7f49580d0d30 msgr2=0x7f49580d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 shutdown_connections 2024-04-26T21:19:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.425+0000 7f496036a640 1 -- 172.21.15.40:0/4133032761 wait complete. 2024-04-26T21:19:08.437 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:08.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.540+0000 7f934ceda640 1 Processor -- start 2024-04-26T21:19:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.540+0000 7f934ceda640 1 -- start start 2024-04-26T21:19:08.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934ceda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93481631f0 0x7f93480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934ceda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f93480d1170 0x7f93481635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934ceda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934809c790 con 0x7f93480b3300 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934ceda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f93480d15f0 con 0x7f93480d1170 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934ceda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9348163ec0 con 0x7f93481631f0 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f93480d1170 0x7f93481635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f93480d1170 0x7f93481635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57236/0 (socket says 172.21.15.40:57236) 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 -- 172.21.15.40:0/610244688 learned_addr learned my addr 172.21.15.40:0/610244688 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f934659c640 1 --2- 172.21.15.40:0/610244688 >> v2:172.21.15.184:3300/0 conn(0x7f93481631f0 0x7f93480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 -- 172.21.15.40:0/610244688 >> v2:172.21.15.184:3300/0 conn(0x7f93481631f0 msgr2=0x7f93480b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 --2- 172.21.15.40:0/610244688 >> v2:172.21.15.184:3300/0 conn(0x7f93481631f0 0x7f93480b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.541+0000 7f9345d9b640 1 -- 172.21.15.40:0/610244688 >> v1:172.21.15.184:6789/0 conn(0x7f93480b3300 legacy=0x7f93480b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.542+0000 7f9345d9b640 1 -- 172.21.15.40:0/610244688 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93480d1a40 con 0x7f93480d1170 2024-04-26T21:19:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.542+0000 7f9345d9b640 1 --2- 172.21.15.40:0/610244688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93480d1170 0x7f93481635d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f93480d2ef0 tx=0x7f933002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288d14b7ea62d215 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.542+0000 7f934559a640 1 -- 172.21.15.40:0/610244688 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330037070 con 0x7f93480d1170 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.542+0000 7f934559a640 1 -- 172.21.15.40:0/610244688 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330032440 con 0x7f93480d1170 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.542+0000 7f934559a640 1 -- 172.21.15.40:0/610244688 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330032810 con 0x7f93480d1170 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 -- 172.21.15.40:0/610244688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93480d1170 msgr2=0x7f93481635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 --2- 172.21.15.40:0/610244688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93480d1170 0x7f93481635d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f93480d2ef0 tx=0x7f933002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 -- 172.21.15.40:0/610244688 shutdown_connections 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 --2- 172.21.15.40:0/610244688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93480d1170 0x7f93481635d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 --2- 172.21.15.40:0/610244688 >> v2:172.21.15.184:3300/0 conn(0x7f93481631f0 0x7f93480b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 -- 172.21.15.40:0/610244688 >> 172.21.15.40:0/610244688 conn(0x7f934809ba40 msgr2=0x7f934809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 -- 172.21.15.40:0/610244688 shutdown_connections 2024-04-26T21:19:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.543+0000 7f934ceda640 1 -- 172.21.15.40:0/610244688 wait complete. 2024-04-26T21:19:08.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.544+0000 7f934ceda640 1 Processor -- start 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.544+0000 7f934ceda640 1 -- start start 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480b3300 0x7f934814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93480d1170 0x7f934814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93480d0230 con 0x7f93481631f0 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93480d2000 con 0x7f93480d1170 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f934ceda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f934814ba80 con 0x7f93480b3300 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57246/0 (socket says 172.21.15.40:57246) 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 -- 172.21.15.40:0/522995509 learned_addr learned my addr 172.21.15.40:0/522995509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 -- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480b3300 msgr2=0x7f934814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480b3300 0x7f934814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 -- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93480d1170 msgr2=0x7f934814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93480d1170 0x7f934814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.545+0000 7f9346d9d640 1 -- 172.21.15.40:0/522995509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934814bd60 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f9346d9d640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f934000c390 tx=0x7f934000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340017070 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93400035f0 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934814c040 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400039e0 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.546+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934814ad30 con 0x7f93481631f0 2024-04-26T21:19:08.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.547+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934806a290 con 0x7f93481631f0 2024-04-26T21:19:08.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.547+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f934001a070 con 0x7f93481631f0 2024-04-26T21:19:08.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.548+0000 7f933effd640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 0x7f932806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.548+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f934008ac60 con 0x7f93481631f0 2024-04-26T21:19:08.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.548+0000 7f934659c640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 0x7f932806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.548+0000 7f934659c640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 0x7f932806edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9338007dc0 tx=0x7f93380072f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.552+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9340059390 con 0x7f93481631f0 2024-04-26T21:19:08.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.734+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93480c9e20 con 0x7f93481631f0 2024-04-26T21:19:08.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.734+0000 7f933effd640 1 -- 172.21.15.40:0/522995509 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f9340058c90 con 0x7f93481631f0 2024-04-26T21:19:08.735 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 msgr2=0x7f932806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 0x7f932806edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9338007dc0 tx=0x7f93380072f0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 msgr2=0x7f934814b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f934000c390 tx=0x7f934000c850 comp rx=0 tx=0).stop 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 shutdown_connections 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f932806c8f0 0x7f932806edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93481631f0 0x7f934814b0a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93480d1170 0x7f934814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 --2- 172.21.15.40:0/522995509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93480b3300 0x7f934814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 >> 172.21.15.40:0/522995509 conn(0x7f934809ba40 msgr2=0x7f93480b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 shutdown_connections 2024-04-26T21:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.736+0000 7f934ceda640 1 -- 172.21.15.40:0/522995509 wait complete. 2024-04-26T21:19:08.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-04-26T21:19:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.848+0000 7f13373bf640 1 Processor -- start 2024-04-26T21:19:08.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.848+0000 7f13373bf640 1 -- start start 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 0x7f13300d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13300cfd70 0x7f13300d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13300d01f0 0x7f13300d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f13300d5980 con 0x7f13300d01f0 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13300d06c0 con 0x7f13300cfd70 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f13373bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13300d09a0 con 0x7f1330166e80 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f133595c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13300d01f0 0x7f13300d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f133595c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13300d01f0 0x7f13300d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57260/0 (socket says 172.21.15.40:57260) 2024-04-26T21:19:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.849+0000 7f133595c640 1 -- 172.21.15.40:0/1949664927 learned_addr learned my addr 172.21.15.40:0/1949664927 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133515b640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 0x7f13300d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 -- 172.21.15.40:0/1949664927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 msgr2=0x7f13300d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133495a640 1 --2- 172.21.15.40:0/1949664927 >> v2:172.21.15.184:3300/0 conn(0x7f13300cfd70 0x7f13300d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 0x7f13300d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 -- 172.21.15.40:0/1949664927 >> v2:172.21.15.184:3300/0 conn(0x7f13300cfd70 msgr2=0x7f13300d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 --2- 172.21.15.40:0/1949664927 >> v2:172.21.15.184:3300/0 conn(0x7f13300cfd70 0x7f13300d4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 -- 172.21.15.40:0/1949664927 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13300d0c80 con 0x7f13300d01f0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133515b640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 0x7f13300d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.850+0000 7f133595c640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300d01f0 0x7f13300d5440 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f133008d620 tx=0x7f132002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305acb53704b3589 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f132ffff640 1 -- 172.21.15.40:0/1949664927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320036070 con 0x7f13300d01f0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f132ffff640 1 -- 172.21.15.40:0/1949664927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320031600 con 0x7f13300d01f0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f132ffff640 1 -- 172.21.15.40:0/1949664927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13200319d0 con 0x7f13300d01f0 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 -- 172.21.15.40:0/1949664927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300d01f0 msgr2=0x7f13300d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300d01f0 0x7f13300d5440 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f133008d620 tx=0x7f132002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 -- 172.21.15.40:0/1949664927 shutdown_connections 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300d01f0 0x7f13300d5440 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 --2- 172.21.15.40:0/1949664927 >> v2:172.21.15.184:3300/0 conn(0x7f13300cfd70 0x7f13300d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 --2- 172.21.15.40:0/1949664927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1330166e80 0x7f13300d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 -- 172.21.15.40:0/1949664927 >> 172.21.15.40:0/1949664927 conn(0x7f133008ec10 msgr2=0x7f133009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 -- 172.21.15.40:0/1949664927 shutdown_connections 2024-04-26T21:19:08.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.851+0000 7f13373bf640 1 -- 172.21.15.40:0/1949664927 wait complete. 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.852+0000 7f13373bf640 1 Processor -- start 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.852+0000 7f13373bf640 1 -- start start 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13300d01f0 0x7f13300cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1330166e80 0x7f13300ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13300d1c00 con 0x7f13300cfd70 2024-04-26T21:19:08.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1330167260 con 0x7f1330166e80 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f13373bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13300d1d80 con 0x7f13300d01f0 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f133515b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f133515b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57268/0 (socket says 172.21.15.40:57268) 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f133515b640 1 -- 172.21.15.40:0/105127231 learned_addr learned my addr 172.21.15.40:0/105127231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f133515b640 1 -- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13300d01f0 msgr2=0x7f13300cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.853+0000 7f133515b640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13300d01f0 0x7f13300cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f133515b640 1 -- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1330166e80 msgr2=0x7f13300ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f133515b640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1330166e80 0x7f13300ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f133515b640 1 -- 172.21.15.40:0/105127231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301f96a0 con 0x7f13300cfd70 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f133515b640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f132800c7f0 tx=0x7f132800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328015070 con 0x7f13300cfd70 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1328007820 con 0x7f13300cfd70 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13300cd1d0 con 0x7f13300cfd70 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328007bd0 con 0x7f13300cfd70 2024-04-26T21:19:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.854+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13300cd790 con 0x7f13300cfd70 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.855+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133006a290 con 0x7f13300cfd70 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.855+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1328018070 con 0x7f13300cfd70 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.856+0000 7f132dffb640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.856+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f132808b970 con 0x7f13300cfd70 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.856+0000 7f133495a640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.857+0000 7f133495a640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 0x7f131c06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1324006b10 tx=0x7f1324008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:08.860+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f132805a0a0 con 0x7f13300cfd70 2024-04-26T21:19:09.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:09.125+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f13300d5020 con 0x7f13300cfd70 2024-04-26T21:19:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.111+0000 7f132dffb640 1 -- 172.21.15.40:0/105127231 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v365) ==== 146+0+0 (secure 0 0 0) 0x7f13280599a0 con 0x7f13300cfd70 2024-04-26T21:19:10.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 msgr2=0x7f131c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 0x7f131c06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1324006b10 tx=0x7f1324008040 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 msgr2=0x7f1330165340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f132800c7f0 tx=0x7f132800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 shutdown_connections 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1330166e80 0x7f13300ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13300d01f0 0x7f13300cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 --2- 172.21.15.40:0/105127231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13300cfd70 0x7f1330165340 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 >> 172.21.15.40:0/105127231 conn(0x7f133008ec10 msgr2=0x7f133006e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.114+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 shutdown_connections 2024-04-26T21:19:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.115+0000 7f13373bf640 1 -- 172.21.15.40:0/105127231 wait complete. 2024-04-26T21:19:10.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:19:10.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.238+0000 7fd673d24640 1 Processor -- start 2024-04-26T21:19:10.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.239+0000 7fd673d24640 1 -- start start 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.239+0000 7fd673d24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd66c0d1940 0x7fd66c09c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.239+0000 7fd673d24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66c0cee60 con 0x7fd66c0b2420 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.239+0000 7fd673d24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66c0d0ff0 con 0x7fd66c15d340 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.239+0000 7fd673d24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c0ce9d0 con 0x7fd66c0d1940 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6722c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd66c0d1940 0x7fd66c09c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6712bf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd66c0b2420 0x7fd66c0702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47594/0 (socket says 172.21.15.40:47594) 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6722c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd66c0d1940 0x7fd66c09c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57280/0 (socket says 172.21.15.40:57280) 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6712bf640 1 -- 172.21.15.40:0/4085332806 learned_addr learned my addr 172.21.15.40:0/4085332806 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6722c1640 1 -- 172.21.15.40:0/4085332806 >> v1:172.21.15.184:6789/0 conn(0x7fd66c15d340 legacy=0x7fd66c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:10.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6722c1640 1 -- 172.21.15.40:0/4085332806 >> v1:172.21.15.40:6789/0 conn(0x7fd66c0b2420 legacy=0x7fd66c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd6722c1640 1 -- 172.21.15.40:0/4085332806 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c0d0d10 con 0x7fd66c0d1940 2024-04-26T21:19:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.240+0000 7fd670abe640 1 -- 172.21.15.40:0/4085332806 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091723923 0 0) 0x7fd66c0cee60 con 0x7fd66c0b2420 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd6722c1640 1 --2- 172.21.15.40:0/4085332806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c0d1940 0x7fd66c09c560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009d90 tx=0x7fd65c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a2815be92ee438 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd670abe640 1 -- 172.21.15.40:0/4085332806 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c007740 con 0x7fd66c0d1940 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd670abe640 1 -- 172.21.15.40:0/4085332806 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c0078e0 con 0x7fd66c0d1940 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd670abe640 1 -- 172.21.15.40:0/4085332806 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c007cb0 con 0x7fd66c0d1940 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd673d24640 1 -- 172.21.15.40:0/4085332806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c0d1940 msgr2=0x7fd66c09c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd673d24640 1 --2- 172.21.15.40:0/4085332806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c0d1940 0x7fd66c09c560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009d90 tx=0x7fd65c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd673d24640 1 -- 172.21.15.40:0/4085332806 shutdown_connections 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd673d24640 1 --2- 172.21.15.40:0/4085332806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c0d1940 0x7fd66c09c560 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.241+0000 7fd673d24640 1 -- 172.21.15.40:0/4085332806 >> 172.21.15.40:0/4085332806 conn(0x7fd66c09ba40 msgr2=0x7fd66c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.242+0000 7fd673d24640 1 -- 172.21.15.40:0/4085332806 shutdown_connections 2024-04-26T21:19:10.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.242+0000 7fd673d24640 1 -- 172.21.15.40:0/4085332806 wait complete. 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.243+0000 7fd673d24640 1 Processor -- start 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.243+0000 7fd673d24640 1 -- start start 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 0x7fd66c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 0x7fd66c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0b2090 con 0x7fd66c15d340 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c15d720 con 0x7fd66c0d1940 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c0b2210 con 0x7fd66c071120 2024-04-26T21:19:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57292/0 (socket says 172.21.15.40:57292) 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 -- 172.21.15.40:0/229193628 learned_addr learned my addr 172.21.15.40:0/229193628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd671ac0640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 0x7fd66c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6722c1640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 0x7fd66c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 -- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 msgr2=0x7fd66c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 0x7fd66c0715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 -- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 msgr2=0x7fd66c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 0x7fd66c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 -- 172.21.15.40:0/229193628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c06e010 con 0x7fd66c15d340 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd671ac0640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 0x7fd66c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6722c1640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 0x7fd66c0715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd6712bf640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c500 tx=0x7fd66000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660018070 con 0x7fd66c15d340 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660007580 con 0x7fd66c15d340 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c06e2f0 con 0x7fd66c15d340 2024-04-26T21:19:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660004a20 con 0x7fd66c15d340 2024-04-26T21:19:10.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.244+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c06e8b0 con 0x7fd66c15d340 2024-04-26T21:19:10.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.245+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c0693d0 con 0x7fd66c15d340 2024-04-26T21:19:10.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.246+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd66001b070 con 0x7fd66c15d340 2024-04-26T21:19:10.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.246+0000 7fd66a7fc640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.246+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7fd66008ab00 con 0x7fd66c15d340 2024-04-26T21:19:10.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.247+0000 7fd671ac0640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.247+0000 7fd671ac0640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd66400a830 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.251+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6600592c0 con 0x7fd66c15d340 2024-04-26T21:19:10.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.434+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd66c0b6cc0 con 0x7fd66c15d340 2024-04-26T21:19:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.438+0000 7fd66a7fc640 1 -- 172.21.15.40:0/229193628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fd660058bc0 con 0x7fd66c15d340 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 msgr2=0x7fd65806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd66400a830 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 msgr2=0x7fd66c1f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c500 tx=0x7fd66000c9c0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 shutdown_connections 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd66c071120 0x7fd66c0715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66c15d340 0x7fd66c1f4940 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 --2- 172.21.15.40:0/229193628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd66c0d1940 0x7fd66c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 >> 172.21.15.40:0/229193628 conn(0x7fd66c09ba40 msgr2=0x7fd66c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 shutdown_connections 2024-04-26T21:19:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.440+0000 7fd673d24640 1 -- 172.21.15.40:0/229193628 wait complete. 2024-04-26T21:19:10.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:19:10.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:10.497 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:10.536 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:19:10.536 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:10.537 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:10.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:10.552 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:10.552 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:10.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:10.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:10.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:10.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:10.610 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:10.610 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:10.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:10.650 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:19:10.650 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.750+0000 7f5693ff7640 1 Processor -- start 2024-04-26T21:19:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.750+0000 7f5693ff7640 1 -- start start 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0cf2e0 0x7f568c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f568c0d11b0 0x7f568c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c06e2c0 0x7f568c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f568c06e750 con 0x7f568c0cf2e0 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f568c0d1740 con 0x7f568c0d11b0 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5693ff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0d1a20 con 0x7f568c06e2c0 2024-04-26T21:19:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0cf2e0 0x7f568c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0cf2e0 0x7f568c06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41576/0 (socket says 172.21.15.40:41576) 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 -- 172.21.15.40:0/2007072504 learned_addr learned my addr 172.21.15.40:0/2007072504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5692594640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c06e2c0 0x7f568c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691592640 1 --2- 172.21.15.40:0/2007072504 >> v2:172.21.15.184:3300/0 conn(0x7f568c0d11b0 0x7f568c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 -- 172.21.15.40:0/2007072504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c06e2c0 msgr2=0x7f568c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c06e2c0 0x7f568c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 -- 172.21.15.40:0/2007072504 >> v2:172.21.15.184:3300/0 conn(0x7f568c0d11b0 msgr2=0x7f568c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 --2- 172.21.15.40:0/2007072504 >> v2:172.21.15.184:3300/0 conn(0x7f568c0d11b0 0x7f568c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.751+0000 7f5691d93640 1 -- 172.21.15.40:0/2007072504 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0d1d00 con 0x7f568c0cf2e0 2024-04-26T21:19:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5691d93640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c06d4d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5688009870 tx=0x7f568802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c461c34fcb377d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5690d91640 1 -- 172.21.15.40:0/2007072504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688037070 con 0x7f568c0cf2e0 2024-04-26T21:19:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5690d91640 1 -- 172.21.15.40:0/2007072504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5688032440 con 0x7f568c0cf2e0 2024-04-26T21:19:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5690d91640 1 -- 172.21.15.40:0/2007072504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688032810 con 0x7f568c0cf2e0 2024-04-26T21:19:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5693ff7640 1 -- 172.21.15.40:0/2007072504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 msgr2=0x7f568c06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.752+0000 7f5693ff7640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c06d4d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5688009870 tx=0x7f568802a210 comp rx=0 tx=0).stop 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 -- 172.21.15.40:0/2007072504 shutdown_connections 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c06e2c0 0x7f568c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 --2- 172.21.15.40:0/2007072504 >> v2:172.21.15.184:3300/0 conn(0x7f568c0d11b0 0x7f568c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 --2- 172.21.15.40:0/2007072504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c06d4d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 -- 172.21.15.40:0/2007072504 >> 172.21.15.40:0/2007072504 conn(0x7f568c0b4a60 msgr2=0x7f568c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 -- 172.21.15.40:0/2007072504 shutdown_connections 2024-04-26T21:19:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.753+0000 7f5693ff7640 1 -- 172.21.15.40:0/2007072504 wait complete. 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.754+0000 7f5693ff7640 1 Processor -- start 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.754+0000 7f5693ff7640 1 -- start start 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568c06e2c0 0x7f568c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 0x7f568c0b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f568c06da10 con 0x7f568c0cf2e0 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0d22c0 con 0x7f568c06e2c0 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0cf6c0 con 0x7f568c0d11b0 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41586/0 (socket says 172.21.15.40:41586) 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 -- 172.21.15.40:0/3216197994 learned_addr learned my addr 172.21.15.40:0/3216197994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691d93640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568c06e2c0 0x7f568c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5692594640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 0x7f568c0b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 -- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 msgr2=0x7f568c0b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 0x7f568c0b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 -- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568c06e2c0 msgr2=0x7f568c15c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568c06e2c0 0x7f568c15c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 -- 172.21.15.40:0/3216197994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0a0a50 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5692594640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 0x7f568c0b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5691592640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d5f0 tx=0x7f567c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0125f0 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c019070 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0129d0 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c1e9e40 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.755+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1ea400 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.756+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c06a290 con 0x7f568c0cf2e0 2024-04-26T21:19:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.757+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f567c012b70 con 0x7f568c0cf2e0 2024-04-26T21:19:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.757+0000 7f56827fc640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 0x7f567006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.757+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f567c08bb80 con 0x7f568c0cf2e0 2024-04-26T21:19:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.758+0000 7f5691d93640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 0x7f567006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:10.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.758+0000 7f5691d93640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 0x7f567006edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5688034000 tx=0x7f568802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.762+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f567c05a2a0 con 0x7f568c0cf2e0 2024-04-26T21:19:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.990+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f568c0b8ef0 con 0x7f568c0cf2e0 2024-04-26T21:19:10.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.990+0000 7f56827fc640 1 -- 172.21.15.40:0/3216197994 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3863 (secure 0 0 0) 0x7f567c059ba0 con 0x7f568c0cf2e0 2024-04-26T21:19:10.992 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:10.992 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/947105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":947105},{"type":"v1","addr":"172.21.15.184:6836","nonce":947105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1005997189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1005997189},{"type":"v1","addr":"172.21.15.184:6837","nonce":1005997189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2167603244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2167603244},{"type":"v1","addr":"172.21.15.40:6834","nonce":2167603244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3515994584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3515994584},{"type":"v1","addr":"172.21.15.40:6835","nonce":3515994584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-04-26T21:19:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 59 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 msgr2=0x7f567006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 0x7f567006edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5688034000 tx=0x7f568802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 msgr2=0x7f568c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d5f0 tx=0x7f567c00dab0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 shutdown_connections 2024-04-26T21:19:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f567006c8f0 0x7f567006edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f568c0d11b0 0x7f568c0b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0cf2e0 0x7f568c0b85a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 --2- 172.21.15.40:0/3216197994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f568c06e2c0 0x7f568c15c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 >> 172.21.15.40:0/3216197994 conn(0x7f568c0b4a60 msgr2=0x7f568c0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 shutdown_connections 2024-04-26T21:19:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:10.993+0000 7f5693ff7640 1 -- 172.21.15.40:0/3216197994 wait complete. 2024-04-26T21:19:11.005 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:19:11.005 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:19:11.005 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:19:11.005 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:11.005 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:19:11.006 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:19:11.006 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:19:11.006 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:11.006 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:19:11.006 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:19:11.006 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:19:11.006 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:11.007 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:19:11.007 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:19:11.007 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:19:11.007 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:11.014 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:19:11.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:19:11.015 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:19:11.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:19:11.018 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:19:11.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:19:11.019 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:19:11.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:19:11.058 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:19:11.058 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:11.059 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:19:11.059 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:11.061 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:19:11.061 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:11.062 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:19:11.062 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:11.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:19:11.134 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:19:11.135 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:19:11.141 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:19:11.142 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:19:11.142 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-04-26T21:19:11.142+0000 7f368ccb9640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:19:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.167+0000 7f89c892c640 1 Processor -- start 2024-04-26T21:19:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.167+0000 7f89c892c640 1 -- start start 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c892c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89c415f370 0x7f89c40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c892c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f750 0x7f89c40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c892c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c40d03a0 con 0x7f89c40d37d0 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c892c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f89c40b4d10 con 0x7f89c415f370 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c892c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89c40d0680 con 0x7f89c415f750 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f89c40d37d0 0x7f89c40d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40690/0 (socket says 172.21.15.40:40690) 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c2d9d640 1 -- 172.21.15.40:0/3393249221 learned_addr learned my addr 172.21.15.40:0/3393249221 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c259c640 1 --2- 172.21.15.40:0/3393249221 >> v2:172.21.15.184:3300/0 conn(0x7f89c415f370 0x7f89c40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c1d9b640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f750 0x7f89c40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c259c640 1 -- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f750 msgr2=0x7f89c40b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c259c640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f750 0x7f89c40b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c259c640 1 -- 172.21.15.40:0/3393249221 >> v1:172.21.15.40:6789/0 conn(0x7f89c40d37d0 legacy=0x7f89c40d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.168+0000 7f89c259c640 1 -- 172.21.15.40:0/3393249221 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c40d0960 con 0x7f89c415f370 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.169+0000 7f89c259c640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f370 0x7f89c40cfd70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f89b80097a0 tx=0x7f89b802a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65bce10e8fc52a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.169+0000 7f89c159a640 1 -- 172.21.15.40:0/3393249221 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8037070 con 0x7f89c415f370 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.169+0000 7f89c159a640 1 -- 172.21.15.40:0/3393249221 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b802ae40 con 0x7f89c415f370 2024-04-26T21:19:11.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.169+0000 7f89c159a640 1 -- 172.21.15.40:0/3393249221 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8032660 con 0x7f89c415f370 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 -- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f370 msgr2=0x7f89c40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f370 0x7f89c40cfd70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f89b80097a0 tx=0x7f89b802a0f0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 -- 172.21.15.40:0/3393249221 shutdown_connections 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f750 0x7f89c40b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 --2- 172.21.15.40:0/3393249221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f370 0x7f89c40cfd70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 -- 172.21.15.40:0/3393249221 >> 172.21.15.40:0/3393249221 conn(0x7f89c408ec10 msgr2=0x7f89c409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 -- 172.21.15.40:0/3393249221 shutdown_connections 2024-04-26T21:19:11.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.170+0000 7f89c892c640 1 -- 172.21.15.40:0/3393249221 wait complete. 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.171+0000 7f89c892c640 1 Processor -- start 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.171+0000 7f89c892c640 1 -- start start 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f370 0x7f89c4151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f750 0x7f89c40ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89c40b51c0 con 0x7f89c40d37d0 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89c40d0f20 con 0x7f89c415f750 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c892c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89c40b5340 con 0x7f89c415f370 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41624/0 (socket says 172.21.15.40:41624) 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c259c640 1 -- 172.21.15.40:0/3409496819 learned_addr learned my addr 172.21.15.40:0/3409496819 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c2d9d640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f750 0x7f89c40ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.172+0000 7f89c1d9b640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f370 0x7f89c4151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 -- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f370 msgr2=0x7f89c4151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f370 0x7f89c4151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 -- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f750 msgr2=0x7f89c40ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f750 0x7f89c40ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 -- 172.21.15.40:0/3409496819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c4161fb0 con 0x7f89c40d37d0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c259c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f89b8035040 tx=0x7f89b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8041070 con 0x7f89c40d37d0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b803c460 con 0x7f89c40d37d0 2024-04-26T21:19:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8037040 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c4162290 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.173+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c4162850 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.174+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c4068460 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.174+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89b803a070 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.175+0000 7f89b2ffd640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 0x7f89a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.175+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4943+0+0 (secure 0 0 0) 0x7f89b80ab9d0 con 0x7f89c40d37d0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.175+0000 7f89c1d9b640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 0x7f89a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.176+0000 7f89c1d9b640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 0x7f89a406edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f89c40bac60 tx=0x7f89b4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.179+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89c4068460 con 0x7f89c40d37d0 2024-04-26T21:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:11.376+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f89c406a290 con 0x7f89c40d37d0 2024-04-26T21:19:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.457+0000 7f89b2ffd640 1 -- 172.21.15.40:0/3409496819 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) ==== 97+0+0 (secure 0 0 0) 0x7f89b8079ac0 con 0x7f89c40d37d0 2024-04-26T21:19:12.458 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 msgr2=0x7f89a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 0x7f89a406edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f89c40bac60 tx=0x7f89b4009190 comp rx=0 tx=0).stop 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 msgr2=0x7f89c4152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f89b8035040 tx=0x7f89b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 shutdown_connections 2024-04-26T21:19:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f89a406c8f0 0x7f89a406edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89c415f750 0x7f89c40ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89c415f370 0x7f89c4151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 --2- 172.21.15.40:0/3409496819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89c40d37d0 0x7f89c4152490 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 >> 172.21.15.40:0/3409496819 conn(0x7f89c408ec10 msgr2=0x7f89c40d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 shutdown_connections 2024-04-26T21:19:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.460+0000 7f89c892c640 1 -- 172.21.15.40:0/3409496819 wait complete. 2024-04-26T21:19:12.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:19:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.580+0000 7f78b715f640 1 Processor -- start 2024-04-26T21:19:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.581+0000 7f78b715f640 1 -- start start 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.581+0000 7f78b715f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78b00b61a0 0x7f78b0160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.581+0000 7f78b715f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b00b4270 con 0x7f78b0145980 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.581+0000 7f78b715f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b00b4530 con 0x7f78b00b3de0 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.581+0000 7f78b715f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f78b00b5140 con 0x7f78b00b61a0 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78affff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78b00b61a0 0x7f78b0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78b4efb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78b0145980 0x7f78b00b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40704/0 (socket says 172.21.15.40:40704) 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78b4efb640 1 -- 172.21.15.40:0/317751332 learned_addr learned my addr 172.21.15.40:0/317751332 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78af7fe640 1 -- 172.21.15.40:0/317751332 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161826040 0 0) 0x7f78b00b4270 con 0x7f78b0145980 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78af7fe640 1 -- 172.21.15.40:0/317751332 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78a0003520 con 0x7f78b0145980 2024-04-26T21:19:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78affff640 1 -- 172.21.15.40:0/317751332 >> v1:172.21.15.184:6789/0 conn(0x7f78b00b3de0 legacy=0x7f78b00720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78affff640 1 -- 172.21.15.40:0/317751332 >> v1:172.21.15.40:6789/0 conn(0x7f78b0145980 legacy=0x7f78b00b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.582+0000 7f78affff640 1 -- 172.21.15.40:0/317751332 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b00b4810 con 0x7f78b00b61a0 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78affff640 1 --2- 172.21.15.40:0/317751332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b0160630 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f789c0097a0 tx=0x7f789c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24796da32e34c2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78af7fe640 1 -- 172.21.15.40:0/317751332 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c037070 con 0x7f78b00b61a0 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78af7fe640 1 -- 172.21.15.40:0/317751332 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c02ae50 con 0x7f78b00b61a0 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78af7fe640 1 -- 172.21.15.40:0/317751332 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c032660 con 0x7f78b00b61a0 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 -- 172.21.15.40:0/317751332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 msgr2=0x7f78b0160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 --2- 172.21.15.40:0/317751332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b0160630 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f789c0097a0 tx=0x7f789c02a140 comp rx=0 tx=0).stop 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 -- 172.21.15.40:0/317751332 shutdown_connections 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 --2- 172.21.15.40:0/317751332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b0160630 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 -- 172.21.15.40:0/317751332 >> 172.21.15.40:0/317751332 conn(0x7f78b00d0d30 msgr2=0x7f78b00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 -- 172.21.15.40:0/317751332 shutdown_connections 2024-04-26T21:19:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.583+0000 7f78b715f640 1 -- 172.21.15.40:0/317751332 wait complete. 2024-04-26T21:19:12.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 Processor -- start 2024-04-26T21:19:12.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 -- start start 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 0x7f78b0140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78b0145980 0x7f78b015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78b00727e0 con 0x7f78b00b61a0 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78b00b4dd0 con 0x7f78b00b3de0 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.585+0000 7f78b715f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78b0072960 con 0x7f78b0145980 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41650/0 (socket says 172.21.15.40:41650) 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 -- 172.21.15.40:0/3947409202 learned_addr learned my addr 172.21.15.40:0/3947409202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78b56fc640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78b0145980 0x7f78b015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78b4efb640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 0x7f78b0140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 -- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78b0145980 msgr2=0x7f78b015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78b0145980 0x7f78b015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 -- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 msgr2=0x7f78b0140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 0x7f78b0140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 -- 172.21.15.40:0/3947409202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b0140b70 con 0x7f78b00b61a0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78b4efb640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 0x7f78b0140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78affff640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f789c02a180 tx=0x7f789c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c041020 con 0x7f78b00b61a0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c03c670 con 0x7f78b00b61a0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b0140e20 con 0x7f78b00b61a0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c037040 con 0x7f78b00b61a0 2024-04-26T21:19:12.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.586+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b01413e0 con 0x7f78b00b61a0 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.587+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b006a290 con 0x7f78b00b61a0 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.588+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f789c044020 con 0x7f78b00b61a0 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.588+0000 7f78ad7fa640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 0x7f789406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.588+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f789c0abba0 con 0x7f78b00b61a0 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.588+0000 7f78b4efb640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 0x7f789406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.589+0000 7f78b4efb640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 0x7f789406edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005880 tx=0x7f78a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.593+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f789c04b070 con 0x7f78b00b61a0 2024-04-26T21:19:12.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.847+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f78b00d1290 con 0x7f78b00b61a0 2024-04-26T21:19:12.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.847+0000 7f78ad7fa640 1 -- 172.21.15.40:0/3947409202 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f789c07a3b0 con 0x7f78b00b61a0 2024-04-26T21:19:12.848 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:12.848 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:19:12.848 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:19:12.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:19:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:12.850 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:19:12.851 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.854 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:19:12.856 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:19:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:12.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:19:12.861 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:19:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 msgr2=0x7f789406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 0x7f789406edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005880 tx=0x7f78a4009040 comp rx=0 tx=0).stop 2024-04-26T21:19:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 msgr2=0x7f78b013e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f789c02a180 tx=0x7f789c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 shutdown_connections 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f789406c8f0 0x7f789406edb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78b0145980 0x7f78b015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b61a0 0x7f78b013e510 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 --2- 172.21.15.40:0/3947409202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78b00b3de0 0x7f78b0140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 >> 172.21.15.40:0/3947409202 conn(0x7f78b00d0d30 msgr2=0x7f78b015d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 shutdown_connections 2024-04-26T21:19:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.849+0000 7f78b715f640 1 -- 172.21.15.40:0/3947409202 wait complete. 2024-04-26T21:19:12.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:12.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.963+0000 7f31e8af1640 1 Processor -- start 2024-04-26T21:19:12.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.963+0000 7f31e8af1640 1 -- start start 2024-04-26T21:19:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e8af1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31e40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e8af1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40cfa80 con 0x7f31e4160640 2024-04-26T21:19:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e8af1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40cfd60 con 0x7f31e4148a30 2024-04-26T21:19:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e8af1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40745e0 con 0x7f31e40b3af0 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31dbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f31e4160640 0x7f31e41636a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40718/0 (socket says 172.21.15.40:40718) 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31dbfff640 1 -- 172.21.15.40:0/2303131033 learned_addr learned my addr 172.21.15.40:0/2303131033 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31e40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737388424 0 0) 0x7f31e40cfa80 con 0x7f31e4160640 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d0003520 con 0x7f31e4160640 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496224243 0 0) 0x7f31e40cfd60 con 0x7f31e4148a30 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e40cfa80 con 0x7f31e4148a30 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2034491597 0 0) 0x7f31d0003520 con 0x7f31e4160640 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 msgr2=0x7f31e40b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31e40b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 >> v1:172.21.15.184:6789/0 conn(0x7f31e4148a30 legacy=0x7f31e4073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40d0040 con 0x7f31e4160640 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f31cc001670 con 0x7f31e4160640 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 >> v1:172.21.15.40:6789/0 conn(0x7f31e4160640 legacy=0x7f31e41636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31d0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31d00040d0 0x7f31d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.964+0000 7f31e1d9b640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d0004d40 0x7f31d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40745e0 con 0x7f31d0004d40 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31d000bb50 con 0x7f31d00040d0 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31d000bd20 con 0x7f31e40b3af0 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d0004d40 0x7f31d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e259c640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31d0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31dbfff640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31d00040d0 0x7f31d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 -- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 msgr2=0x7f31d0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b3af0 0x7f31d0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 -- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31d00040d0 msgr2=0x7f31d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31d00040d0 0x7f31d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31e2d9d640 1 -- 172.21.15.40:0/2303131033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d0003520 con 0x7f31d0004d40 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.965+0000 7f31dbfff640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31d00040d0 0x7f31d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d0004d40 0x7f31d00051c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009fc0 tx=0x7f31d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c5e5fd3de7a43c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc009c10 con 0x7f31d0004d40 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc036070 con 0x7f31d0004d40 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e1d9b640 1 -- 172.21.15.40:0/2303131033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc031880 con 0x7f31d0004d40 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 -- 172.21.15.40:0/2303131033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d0004d40 msgr2=0x7f31d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 --2- 172.21.15.40:0/2303131033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d0004d40 0x7f31d00051c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009fc0 tx=0x7f31d0005830 comp rx=0 tx=0).stop 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e2d9d640 1 -- 172.21.15.40:0/2303131033 reap_dead start 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 -- 172.21.15.40:0/2303131033 shutdown_connections 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 -- 172.21.15.40:0/2303131033 >> 172.21.15.40:0/2303131033 conn(0x7f31e40ce8b0 msgr2=0x7f31e40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 -- 172.21.15.40:0/2303131033 shutdown_connections 2024-04-26T21:19:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.966+0000 7f31e8af1640 1 -- 172.21.15.40:0/2303131033 wait complete. 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 Processor -- start 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 -- start start 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31e4143550 0x7f31e40af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b7440 0x7f31e4142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4163db0 con 0x7f31e40b7060 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4163180 con 0x7f31e4143550 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.968+0000 7f31e8af1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f31e4148e10 con 0x7f31e40b7440 2024-04-26T21:19:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41678/0 (socket says 172.21.15.40:41678) 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 -- 172.21.15.40:0/3583871452 learned_addr learned my addr 172.21.15.40:0/3583871452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e2d9d640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b7440 0x7f31e4142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31dbfff640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31e4143550 0x7f31e40af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 -- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b7440 msgr2=0x7f31e4142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b7440 0x7f31e4142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 -- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31e4143550 msgr2=0x7f31e40af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31e4143550 0x7f31e40af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 -- 172.21.15.40:0/3583871452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40b01c0 con 0x7f31e40b7060 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e259c640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f31d400f770 tx=0x7f31d400fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400fe80 con 0x7f31e40b7060 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4018070 con 0x7f31e40b7060 2024-04-26T21:19:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e40b04a0 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400d730 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.969+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e40b0a60 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.970+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e406a290 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.971+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31d400d8d0 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.971+0000 7f31daffd640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.971+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f31d408b690 con 0x7f31e40b7060 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.971+0000 7f31dbfff640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.972+0000 7f31dbfff640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f31e40b3ac0 tx=0x7f31cc007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:12.975+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31d40597c0 con 0x7f31e40b7060 2024-04-26T21:19:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.162+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31e40b4010 con 0x7f31e40b7060 2024-04-26T21:19:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.163+0000 7f31daffd640 1 -- 172.21.15.40:0/3583871452 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+12893 (secure 0 0 0) 0x7f31d405dac0 con 0x7f31e40b7060 2024-04-26T21:19:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:13.164 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":105,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:12.456295+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.164+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 msgr2=0x7f31c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.164+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f31e40b3ac0 tx=0x7f31cc007040 comp rx=0 tx=0).stop 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.164+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 msgr2=0x7f31e40b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.164+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f31d400f770 tx=0x7f31d400fc30 comp rx=0 tx=0).stop 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 shutdown_connections 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31e40b7440 0x7f31e4142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31e4143550 0x7f31e40af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 --2- 172.21.15.40:0/3583871452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40b7060 0x7f31e40b1260 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 >> 172.21.15.40:0/3583871452 conn(0x7f31e40ce8b0 msgr2=0x7f31e4147d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 shutdown_connections 2024-04-26T21:19:13.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.165+0000 7f31e8af1640 1 -- 172.21.15.40:0/3583871452 wait complete. 2024-04-26T21:19:13.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:13.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:13.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.280+0000 7f6ac9274640 1 Processor -- start 2024-04-26T21:19:13.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.280+0000 7f6ac9274640 1 -- start start 2024-04-26T21:19:13.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6ac9274640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6ac40d0010 0x7f6ac406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6ac9274640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6ac9274640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac409c700 con 0x7f6ac40b3210 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6ac9274640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac40b4480 con 0x7f6ac40b3c20 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6ac9274640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac406e050 con 0x7f6ac40d0010 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41686/0 (socket says 172.21.15.40:41686) 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 -- 172.21.15.40:0/3390092436 learned_addr learned my addr 172.21.15.40:0/3390092436 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 -- 172.21.15.40:0/3390092436 >> v2:172.21.15.184:3300/0 conn(0x7f6ac40d0010 msgr2=0x7f6ac406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 --2- 172.21.15.40:0/3390092436 >> v2:172.21.15.184:3300/0 conn(0x7f6ac40d0010 0x7f6ac406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 -- 172.21.15.40:0/3390092436 >> v1:172.21.15.184:6789/0 conn(0x7f6ac40b3210 legacy=0x7f6ac415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.281+0000 7f6aba7fc640 1 -- 172.21.15.40:0/3390092436 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40d1c50 con 0x7f6ac40b3c20 2024-04-26T21:19:13.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6aba7fc640 1 --2- 172.21.15.40:0/3390092436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4073530 tx=0x7f6aac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ca1dfb05c567b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6ac27fc640 1 -- 172.21.15.40:0/3390092436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac037070 con 0x7f6ac40b3c20 2024-04-26T21:19:13.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6ac27fc640 1 -- 172.21.15.40:0/3390092436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac032440 con 0x7f6ac40b3c20 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6ac27fc640 1 -- 172.21.15.40:0/3390092436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac032810 con 0x7f6ac40b3c20 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6ac9274640 1 -- 172.21.15.40:0/3390092436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40b3c20 msgr2=0x7f6ac40b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.282+0000 7f6ac9274640 1 --2- 172.21.15.40:0/3390092436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4073530 tx=0x7f6aac02a230 comp rx=0 tx=0).stop 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 -- 172.21.15.40:0/3390092436 shutdown_connections 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 --2- 172.21.15.40:0/3390092436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40b3c20 0x7f6ac40b2cd0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 --2- 172.21.15.40:0/3390092436 >> v2:172.21.15.184:3300/0 conn(0x7f6ac40d0010 0x7f6ac406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 -- 172.21.15.40:0/3390092436 >> 172.21.15.40:0/3390092436 conn(0x7f6ac409ba40 msgr2=0x7f6ac409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 -- 172.21.15.40:0/3390092436 shutdown_connections 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.283+0000 7f6ac9274640 1 -- 172.21.15.40:0/3390092436 wait complete. 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.284+0000 7f6ac9274640 1 Processor -- start 2024-04-26T21:19:13.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.284+0000 7f6ac9274640 1 -- start start 2024-04-26T21:19:13.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 0x7f6ac4071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac40b3c20 0x7f6ac414dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40b37c0 con 0x7f6ac40d0010 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40b3640 con 0x7f6ac40b3210 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac40b3940 con 0x7f6ac40b3c20 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41696/0 (socket says 172.21.15.40:41696) 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2855900890 learned_addr learned my addr 172.21.15.40:0/2855900890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 0x7f6ac4071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6aba7fc640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac40b3c20 0x7f6ac414dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac40b3c20 msgr2=0x7f6ac414dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac40b3c20 0x7f6ac414dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 msgr2=0x7f6ac4071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 0x7f6ac4071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2855900890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac415b5b0 con 0x7f6ac40d0010 2024-04-26T21:19:13.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 0x7f6ac4071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac37fe640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00b9c0 tx=0x7f6abc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc019070 con 0x7f6ac40d0010 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc00cba0 con 0x7f6ac40d0010 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac4162410 con 0x7f6ac40d0010 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.285+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc014440 con 0x7f6ac40d0010 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.286+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41629d0 con 0x7f6ac40d0010 2024-04-26T21:19:13.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.287+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac409e480 con 0x7f6ac40d0010 2024-04-26T21:19:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.287+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6abc01c070 con 0x7f6ac40d0010 2024-04-26T21:19:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.288+0000 7f6abbfff640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.288+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f6abc08b5e0 con 0x7f6ac40d0010 2024-04-26T21:19:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.288+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.288+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009890 tx=0x7f6ab4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.292+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6abc059cd0 con 0x7f6ac40d0010 2024-04-26T21:19:13.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.477+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ac4068e00 con 0x7f6ac40d0010 2024-04-26T21:19:13.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.478+0000 7f6abbfff640 1 -- 172.21.15.40:0/2855900890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f6abc0595d0 con 0x7f6ac40d0010 2024-04-26T21:19:13.479 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 msgr2=0x7f6aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009890 tx=0x7f6ab4008040 comp rx=0 tx=0).stop 2024-04-26T21:19:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 msgr2=0x7f6ac414e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00b9c0 tx=0x7f6abc00be80 comp rx=0 tx=0).stop 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 shutdown_connections 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d0010 0x7f6ac414e510 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac40b3c20 0x7f6ac414dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 --2- 172.21.15.40:0/2855900890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac40b3210 0x7f6ac4071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 >> 172.21.15.40:0/2855900890 conn(0x7f6ac409ba40 msgr2=0x7f6ac40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 shutdown_connections 2024-04-26T21:19:13.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.480+0000 7f6ac9274640 1 -- 172.21.15.40:0/2855900890 wait complete. 2024-04-26T21:19:13.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:19:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.593+0000 7f63b4064640 1 Processor -- start 2024-04-26T21:19:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.593+0000 7f63b4064640 1 -- start start 2024-04-26T21:19:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.594+0000 7f63b4064640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.594+0000 7f63b4064640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.594+0000 7f63b4064640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ac0b2510 con 0x7f63ac15f340 2024-04-26T21:19:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.594+0000 7f63b4064640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac0b27f0 con 0x7f63ac15f720 2024-04-26T21:19:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.594+0000 7f63b4064640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac0b2ad0 con 0x7f63ac0d5610 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b15ff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b1e00640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f63ac15f340 0x7f63ac0d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:44892/0 (socket says 172.21.15.40:44892) 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:44252/0 (socket says 172.21.15.40:44252) 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b1e00640 1 -- 172.21.15.40:0/3554257086 learned_addr learned my addr 172.21.15.40:0/3554257086 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b15ff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:47220/0 (socket says 172.21.15.40:47220) 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 -- 172.21.15.40:0/3554257086 >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 msgr2=0x7f63ac0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 --2- 172.21.15.40:0/3554257086 >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 -- 172.21.15.40:0/3554257086 >> v1:172.21.15.184:6789/0 conn(0x7f63ac15f340 legacy=0x7f63ac0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b2601640 1 -- 172.21.15.40:0/3554257086 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac0b2db0 con 0x7f63ac0d5610 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b0dfe640 1 -- 172.21.15.40:0/3554257086 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799739636 0 0) 0x7f63ac0b2510 con 0x7f63ac15f340 2024-04-26T21:19:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.595+0000 7f63b15ff640 1 --2- 172.21.15.40:0/3554257086 >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.596+0000 7f63b2601640 1 --2- 172.21.15.40:0/3554257086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097a0 tx=0x7f63a402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a61aebcfca9931f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.596+0000 7f63b0dfe640 1 -- 172.21.15.40:0/3554257086 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4037070 con 0x7f63ac0d5610 2024-04-26T21:19:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.596+0000 7f63b0dfe640 1 -- 172.21.15.40:0/3554257086 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a402ae50 con 0x7f63ac0d5610 2024-04-26T21:19:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.596+0000 7f63b0dfe640 1 -- 172.21.15.40:0/3554257086 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4032660 con 0x7f63ac0d5610 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 -- 172.21.15.40:0/3554257086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 msgr2=0x7f63ac0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 --2- 172.21.15.40:0/3554257086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097a0 tx=0x7f63a402a100 comp rx=0 tx=0).stop 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 -- 172.21.15.40:0/3554257086 shutdown_connections 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 --2- 172.21.15.40:0/3554257086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac0d5610 0x7f63ac0d3d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 --2- 172.21.15.40:0/3554257086 >> v2:172.21.15.184:3300/0 conn(0x7f63ac15f720 0x7f63ac0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 -- 172.21.15.40:0/3554257086 >> 172.21.15.40:0/3554257086 conn(0x7f63ac08ec70 msgr2=0x7f63ac09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 -- 172.21.15.40:0/3554257086 shutdown_connections 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.597+0000 7f63b4064640 1 -- 172.21.15.40:0/3554257086 wait complete. 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.598+0000 7f63b4064640 1 Processor -- start 2024-04-26T21:19:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.598+0000 7f63b4064640 1 -- start start 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 0x7f63ac15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 0x7f63ac160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0b3940 con 0x7f63ac0d5610 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0b37c0 con 0x7f63ac15f720 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b4064640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac0b3ac0 con 0x7f63ac15f340 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41702/0 (socket says 172.21.15.40:41702) 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 -- 172.21.15.40:0/2833285522 learned_addr learned my addr 172.21.15.40:0/2833285522 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b2601640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 0x7f63ac160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b15ff640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 0x7f63ac15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 -- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 msgr2=0x7f63ac15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 0x7f63ac15fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 -- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 msgr2=0x7f63ac160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 0x7f63ac160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 -- 172.21.15.40:0/2833285522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac0caa20 con 0x7f63ac0d5610 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b2601640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 0x7f63ac160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b15ff640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 0x7f63ac15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63b1e00640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c890 tx=0x7f63a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.599+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8018070 con 0x7f63ac0d5610 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.600+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a800a640 con 0x7f63ac0d5610 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.600+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac0cad00 con 0x7f63ac0d5610 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.600+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a800aa30 con 0x7f63ac0d5610 2024-04-26T21:19:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.600+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac0cb2c0 con 0x7f63ac0d5610 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.600+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac162020 con 0x7f63ac0d5610 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.601+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63a800abd0 con 0x7f63ac0d5610 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.601+0000 7f63a27fc640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.602+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f63a808aed0 con 0x7f63ac0d5610 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.602+0000 7f63b15ff640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.602+0000 7f63b15ff640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 0x7f639006edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cbab0 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.606+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63a80595d0 con 0x7f63ac0d5610 2024-04-26T21:19:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.792+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f63ac06a290 con 0x7f63ac0d5610 2024-04-26T21:19:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.793+0000 7f63a27fc640 1 -- 172.21.15.40:0/2833285522 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) ==== 84+0+65 (secure 0 0 0) 0x7f63a8058ed0 con 0x7f63ac0d5610 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:13.794 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 msgr2=0x7f639006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 0x7f639006edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cbab0 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 msgr2=0x7f63ac1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c890 tx=0x7f63a800cd50 comp rx=0 tx=0).stop 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 shutdown_connections 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac15f720 0x7f63ac160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac15f340 0x7f63ac15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 --2- 172.21.15.40:0/2833285522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63ac0d5610 0x7f63ac1562b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 >> 172.21.15.40:0/2833285522 conn(0x7f63ac08ec70 msgr2=0x7f63ac0d4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 shutdown_connections 2024-04-26T21:19:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.795+0000 7f63b4064640 1 -- 172.21.15.40:0/2833285522 wait complete. 2024-04-26T21:19:13.807 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:19:13.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:19:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.910+0000 7fe43d07b640 1 Processor -- start 2024-04-26T21:19:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.910+0000 7fe43d07b640 1 -- start start 2024-04-26T21:19:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43d07b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4380d3a30 0x7fe43806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43d07b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4380d1240 con 0x7fe4380cee20 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43d07b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4380d14e0 con 0x7fe43809c150 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43d07b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe4380d1e10 con 0x7fe4380d3a30 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe436d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe4380cee20 0x7fe43809ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40724/0 (socket says 172.21.15.40:40724) 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe436d9d640 1 -- 172.21.15.40:0/1864037620 learned_addr learned my addr 172.21.15.40:0/1864037620 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43759e640 1 --2- 172.21.15.40:0/1864037620 >> v2:172.21.15.40:3300/0 conn(0x7fe4380d3a30 0x7fe43806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe435d9b640 1 -- 172.21.15.40:0/1864037620 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2802434317 0 0) 0x7fe4380d1240 con 0x7fe4380cee20 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe435d9b640 1 -- 172.21.15.40:0/1864037620 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe424003520 con 0x7fe4380cee20 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43759e640 1 -- 172.21.15.40:0/1864037620 >> v1:172.21.15.184:6789/0 conn(0x7fe43809c150 legacy=0x7fe43815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.911+0000 7fe43759e640 1 -- 172.21.15.40:0/1864037620 >> v1:172.21.15.40:6789/0 conn(0x7fe4380cee20 legacy=0x7fe43809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.912+0000 7fe43759e640 1 -- 172.21.15.40:0/1864037620 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380d17c0 con 0x7fe4380d3a30 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.912+0000 7fe43759e640 1 --2- 172.21.15.40:0/1864037620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe43806d5c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe428009d90 tx=0x7fe42802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe20a273a8c97f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.912+0000 7fe435d9b640 1 -- 172.21.15.40:0/1864037620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428007740 con 0x7fe4380d3a30 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.912+0000 7fe435d9b640 1 -- 172.21.15.40:0/1864037620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4280078e0 con 0x7fe4380d3a30 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.912+0000 7fe435d9b640 1 -- 172.21.15.40:0/1864037620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428007cb0 con 0x7fe4380d3a30 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 -- 172.21.15.40:0/1864037620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 msgr2=0x7fe43806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 --2- 172.21.15.40:0/1864037620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe43806d5c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe428009d90 tx=0x7fe42802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 -- 172.21.15.40:0/1864037620 shutdown_connections 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 --2- 172.21.15.40:0/1864037620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe43806d5c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 -- 172.21.15.40:0/1864037620 >> 172.21.15.40:0/1864037620 conn(0x7fe4380d2f10 msgr2=0x7fe4380d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 -- 172.21.15.40:0/1864037620 shutdown_connections 2024-04-26T21:19:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.913+0000 7fe43d07b640 1 -- 172.21.15.40:0/1864037620 wait complete. 2024-04-26T21:19:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.914+0000 7fe43d07b640 1 Processor -- start 2024-04-26T21:19:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.914+0000 7fe43d07b640 1 -- start start 2024-04-26T21:19:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe43809c150 0x7fe4380c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4380cee20 0x7fe438154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4380cf200 con 0x7fe4380d3a30 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe43809c7a0 con 0x7fe43809c150 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43d07b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4380cf380 con 0x7fe4380cee20 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41726/0 (socket says 172.21.15.40:41726) 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43759e640 1 -- 172.21.15.40:0/2521619106 learned_addr learned my addr 172.21.15.40:0/2521619106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe43659c640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4380cee20 0x7fe438154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.915+0000 7fe436d9d640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe43809c150 0x7fe4380c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 -- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4380cee20 msgr2=0x7fe438154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4380cee20 0x7fe438154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 -- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe43809c150 msgr2=0x7fe4380c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe43809c150 0x7fe4380c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 -- 172.21.15.40:0/2521619106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380cbc80 con 0x7fe4380d3a30 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43759e640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe428009eb0 tx=0x7fe428004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42803f070 con 0x7fe4380d3a30 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428031bc0 con 0x7fe4380d3a30 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4380cbf00 con 0x7fe4380d3a30 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428009450 con 0x7fe4380d3a30 2024-04-26T21:19:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.916+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4380cc4c0 con 0x7fe4380d3a30 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.917+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43806a290 con 0x7fe4380d3a30 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.918+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe42803b410 con 0x7fe4380d3a30 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.918+0000 7fe42f7fe640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.918+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fe4280acac0 con 0x7fe4380d3a30 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.918+0000 7fe436d9d640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.919+0000 7fe436d9d640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe430005880 tx=0x7fe430009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:13.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:13.923+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4380cbc80 con 0x7fe4380d3a30 2024-04-26T21:19:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.106+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4380b8e90 con 0x7fe4380d3a30 2024-04-26T21:19:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.462+0000 7fe42f7fe640 1 -- 172.21.15.40:0/2521619106 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) ==== 138+0+0 (secure 0 0 0) 0x7fe42807abf0 con 0x7fe4380d3a30 2024-04-26T21:19:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:19:14.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 msgr2=0x7fe41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe430005880 tx=0x7fe430009040 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 msgr2=0x7fe4381f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe428009eb0 tx=0x7fe428004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 shutdown_connections 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4380d3a30 0x7fe4381f5230 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4380cee20 0x7fe438154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 --2- 172.21.15.40:0/2521619106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe43809c150 0x7fe4380c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 >> 172.21.15.40:0/2521619106 conn(0x7fe4380d2f10 msgr2=0x7fe4380b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 shutdown_connections 2024-04-26T21:19:14.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.465+0000 7fe43d07b640 1 -- 172.21.15.40:0/2521619106 wait complete. 2024-04-26T21:19:14.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:19:14.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.590+0000 7f17bf87e640 1 Processor -- start 2024-04-26T21:19:14.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.590+0000 7f17bf87e640 1 -- start start 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bf87e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17b81459a0 0x7f17b80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bf87e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17b815d030 0x7f17b80b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bf87e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b80b4480 con 0x7f17b80b6130 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bf87e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f17b80b65a0 con 0x7f17b81459a0 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bf87e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f17b80b4760 con 0x7f17b815d030 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17b81459a0 0x7f17b80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17b81459a0 0x7f17b80b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41728/0 (socket says 172.21.15.40:41728) 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 -- 172.21.15.40:0/2062105594 learned_addr learned my addr 172.21.15.40:0/2062105594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bce19640 1 --2- 172.21.15.40:0/2062105594 >> v2:172.21.15.184:3300/0 conn(0x7f17b815d030 0x7f17b80b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 -- 172.21.15.40:0/2062105594 >> v2:172.21.15.184:3300/0 conn(0x7f17b815d030 msgr2=0x7f17b80b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 --2- 172.21.15.40:0/2062105594 >> v2:172.21.15.184:3300/0 conn(0x7f17b815d030 0x7f17b80b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 -- 172.21.15.40:0/2062105594 >> v1:172.21.15.184:6789/0 conn(0x7f17b80b6130 legacy=0x7f17b815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.591+0000 7f17bd61a640 1 -- 172.21.15.40:0/2062105594 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b80b4a40 con 0x7f17b81459a0 2024-04-26T21:19:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.592+0000 7f17bd61a640 1 --2- 172.21.15.40:0/2062105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b80b3de0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f17b806eb70 tx=0x7f17b402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfc86bc59f229d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.592+0000 7f17affff640 1 -- 172.21.15.40:0/2062105594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4037070 con 0x7f17b81459a0 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.592+0000 7f17affff640 1 -- 172.21.15.40:0/2062105594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b4032440 con 0x7f17b81459a0 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.592+0000 7f17affff640 1 -- 172.21.15.40:0/2062105594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4032810 con 0x7f17b81459a0 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 -- 172.21.15.40:0/2062105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 msgr2=0x7f17b80b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 --2- 172.21.15.40:0/2062105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b80b3de0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f17b806eb70 tx=0x7f17b402a200 comp rx=0 tx=0).stop 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 -- 172.21.15.40:0/2062105594 shutdown_connections 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 --2- 172.21.15.40:0/2062105594 >> v2:172.21.15.184:3300/0 conn(0x7f17b815d030 0x7f17b80b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 --2- 172.21.15.40:0/2062105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b80b3de0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 -- 172.21.15.40:0/2062105594 >> 172.21.15.40:0/2062105594 conn(0x7f17b80d0d30 msgr2=0x7f17b80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 -- 172.21.15.40:0/2062105594 shutdown_connections 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.593+0000 7f17bf87e640 1 -- 172.21.15.40:0/2062105594 wait complete. 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.594+0000 7f17bf87e640 1 Processor -- start 2024-04-26T21:19:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.594+0000 7f17bf87e640 1 -- start start 2024-04-26T21:19:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b80b6130 0x7f17b815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17b815d030 0x7f17b81f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f17b815d470 con 0x7f17b81459a0 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17b80b5000 con 0x7f17b80b6130 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f17b8073100 con 0x7f17b815d030 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41734/0 (socket says 172.21.15.40:41734) 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 -- 172.21.15.40:0/3749472455 learned_addr learned my addr 172.21.15.40:0/3749472455 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bd61a640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b80b6130 0x7f17b815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bde1b640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17b815d030 0x7f17b81f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 -- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17b815d030 msgr2=0x7f17b81f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17b815d030 0x7f17b81f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 -- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b80b6130 msgr2=0x7f17b815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b80b6130 0x7f17b815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 -- 172.21.15.40:0/3749472455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b815f5d0 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bce19640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d5f0 tx=0x7f17a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a80125f0 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a8019070 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b815e640 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.595+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a80129d0 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.596+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b80af720 con 0x7f17b81459a0 2024-04-26T21:19:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.596+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b806a290 con 0x7f17b81459a0 2024-04-26T21:19:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.597+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17a8012b70 con 0x7f17b81459a0 2024-04-26T21:19:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.598+0000 7f17adffb640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 0x7f179c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.598+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5227+0+0 (secure 0 0 0) 0x7f17a808bc80 con 0x7f17b81459a0 2024-04-26T21:19:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.598+0000 7f17bd61a640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 0x7f179c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.598+0000 7f17bd61a640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 0x7f179c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f17b4034000 tx=0x7f17b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:14.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.602+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17a8059c40 con 0x7f17b81459a0 2024-04-26T21:19:14.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:14.786+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f17b815cda0 con 0x7f17b81459a0 2024-04-26T21:19:15.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.467+0000 7f17adffb640 1 -- 172.21.15.40:0/3749472455 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) ==== 170+0+0 (secure 0 0 0) 0x7f17a805df40 con 0x7f17b81459a0 2024-04-26T21:19:15.468 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:19:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 msgr2=0x7f179c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 0x7f179c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f17b4034000 tx=0x7f17b402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 msgr2=0x7f17b815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d5f0 tx=0x7f17a800dab0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 shutdown_connections 2024-04-26T21:19:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f179c06c8f0 0x7f179c06edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17b815d030 0x7f17b81f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17b81459a0 0x7f17b815b500 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 --2- 172.21.15.40:0/3749472455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b80b6130 0x7f17b815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 >> 172.21.15.40:0/3749472455 conn(0x7f17b80d0d30 msgr2=0x7f17b815fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:15.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 shutdown_connections 2024-04-26T21:19:15.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.471+0000 7f17bf87e640 1 -- 172.21.15.40:0/3749472455 wait complete. 2024-04-26T21:19:15.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:19:15.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 Processor -- start 2024-04-26T21:19:15.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 -- start start 2024-04-26T21:19:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86b415d340 0x7f86b409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b40b2420 0x7f86b40d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b40d1b80 con 0x7f86b40d1770 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40ce9d0 con 0x7f86b40b2420 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b9751640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40cee60 con 0x7f86b415d340 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.606+0000 7f86b27fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b40b2420 0x7f86b40d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b40b2420 0x7f86b40d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41748/0 (socket says 172.21.15.40:41748) 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 -- 172.21.15.40:0/4063331758 learned_addr learned my addr 172.21.15.40:0/4063331758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/4063331758 >> v2:172.21.15.184:3300/0 conn(0x7f86b415d340 0x7f86b409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 -- 172.21.15.40:0/4063331758 >> v2:172.21.15.184:3300/0 conn(0x7f86b415d340 msgr2=0x7f86b409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 --2- 172.21.15.40:0/4063331758 >> v2:172.21.15.184:3300/0 conn(0x7f86b415d340 0x7f86b409c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 -- 172.21.15.40:0/4063331758 >> v1:172.21.15.184:6789/0 conn(0x7f86b40d1770 legacy=0x7f86b40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.607+0000 7f86b27fc640 1 -- 172.21.15.40:0/4063331758 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40d0ff0 con 0x7f86b40b2420 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b27fc640 1 --2- 172.21.15.40:0/4063331758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40b2420 0x7f86b40d1230 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f86b4072ee0 tx=0x7f86a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5bd0544e25e514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b1ffb640 1 -- 172.21.15.40:0/4063331758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8037070 con 0x7f86b40b2420 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b1ffb640 1 -- 172.21.15.40:0/4063331758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8032440 con 0x7f86b40b2420 2024-04-26T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b1ffb640 1 -- 172.21.15.40:0/4063331758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8032810 con 0x7f86b40b2420 2024-04-26T21:19:15.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 -- 172.21.15.40:0/4063331758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40b2420 msgr2=0x7f86b40d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 --2- 172.21.15.40:0/4063331758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40b2420 0x7f86b40d1230 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f86b4072ee0 tx=0x7f86a802a230 comp rx=0 tx=0).stop 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 -- 172.21.15.40:0/4063331758 shutdown_connections 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 --2- 172.21.15.40:0/4063331758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40b2420 0x7f86b40d1230 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 --2- 172.21.15.40:0/4063331758 >> v2:172.21.15.184:3300/0 conn(0x7f86b415d340 0x7f86b409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 -- 172.21.15.40:0/4063331758 >> 172.21.15.40:0/4063331758 conn(0x7f86b409ba40 msgr2=0x7f86b409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 -- 172.21.15.40:0/4063331758 shutdown_connections 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.608+0000 7f86b9751640 1 -- 172.21.15.40:0/4063331758 wait complete. 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 Processor -- start 2024-04-26T21:19:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 -- start start 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d1770 0x7f86b41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 0x7f86b4159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40712b0 con 0x7f86b4150bb0 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40ce660 con 0x7f86b415d340 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.610+0000 7f86b9751640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b415d720 con 0x7f86b40d1770 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41764/0 (socket says 172.21.15.40:41764) 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 -- 172.21.15.40:0/2159479667 learned_addr learned my addr 172.21.15.40:0/2159479667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b27fc640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 0x7f86b4159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d1770 0x7f86b41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 -- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d1770 msgr2=0x7f86b41595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d1770 0x7f86b41595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 -- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 msgr2=0x7f86b4159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 0x7f86b4159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 -- 172.21.15.40:0/2159479667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b409f310 con 0x7f86b4150bb0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b27fc640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 0x7f86b4159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b37fe640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c780 tx=0x7f86ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac004b60 con 0x7f86b4150bb0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac004d00 con 0x7f86b4150bb0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b409f5f0 con 0x7f86b4150bb0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00aa90 con 0x7f86b4150bb0 2024-04-26T21:19:15.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.611+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b409fbb0 con 0x7f86b4150bb0 2024-04-26T21:19:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.612+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b40693d0 con 0x7f86b4150bb0 2024-04-26T21:19:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.613+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86ac00ac30 con 0x7f86b4150bb0 2024-04-26T21:19:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.613+0000 7f869b7fe640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:15.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.614+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5609+0+0 (secure 0 0 0) 0x7f86ac08c1b0 con 0x7f86b4150bb0 2024-04-26T21:19:15.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.614+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:15.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.614+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f86a4006fd0 tx=0x7f86a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:15.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.618+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86ac05a510 con 0x7f86b4150bb0 2024-04-26T21:19:15.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:15.843+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f86b415cad0 con 0x7f86b4150bb0 2024-04-26T21:19:16.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.469+0000 7f869b7fe640 1 -- 172.21.15.40:0/2159479667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) ==== 174+0+0 (secure 0 0 0) 0x7f86ac059e10 con 0x7f86b4150bb0 2024-04-26T21:19:16.470 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 16 and data pool 17 2024-04-26T21:19:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.472+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 msgr2=0x7f86a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.473+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f86a4006fd0 tx=0x7f86a4008040 comp rx=0 tx=0).stop 2024-04-26T21:19:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.473+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 msgr2=0x7f86b40a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.473+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c780 tx=0x7f86ac00cc40 comp rx=0 tx=0).stop 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 shutdown_connections 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b4150bb0 0x7f86b40a0690 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b415d340 0x7f86b4159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 --2- 172.21.15.40:0/2159479667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d1770 0x7f86b41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.476+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 >> 172.21.15.40:0/2159479667 conn(0x7f86b409ba40 msgr2=0x7f86b415a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.479+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 shutdown_connections 2024-04-26T21:19:16.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.482+0000 7f86b9751640 1 -- 172.21.15.40:0/2159479667 wait complete. 2024-04-26T21:19:16.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.615+0000 7f58a0693640 1 Processor -- start 2024-04-26T21:19:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.615+0000 7f58a0693640 1 -- start start 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.615+0000 7f58a0693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 0x7f58980b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f58a0693640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58980d5610 0x7f58980d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f58a0693640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58980b2510 con 0x7f589815f720 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f58a0693640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f58980b27f0 con 0x7f58980d5610 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f58a0693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58980b2ad0 con 0x7f589815f340 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58980d5610 0x7f58980d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58980d5610 0x7f58980d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41774/0 (socket says 172.21.15.40:41774) 2024-04-26T21:19:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 -- 172.21.15.40:0/3508975555 learned_addr learned my addr 172.21.15.40:0/3508975555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 -- 172.21.15.40:0/3508975555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 msgr2=0x7f58980b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589e42f640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 0x7f58980b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 0x7f58980b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 -- 172.21.15.40:0/3508975555 >> v1:172.21.15.184:6789/0 conn(0x7f589815f720 legacy=0x7f58980d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589ec30640 1 -- 172.21.15.40:0/3508975555 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58980b2db0 con 0x7f58980d5610 2024-04-26T21:19:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.616+0000 7f589e42f640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 0x7f58980b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f589ec30640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f58980d3d40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5890009870 tx=0x7f589002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f8d74492190c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f589d42d640 1 -- 172.21.15.40:0/3508975555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890037070 con 0x7f58980d5610 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f589d42d640 1 -- 172.21.15.40:0/3508975555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890032440 con 0x7f58980d5610 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f589d42d640 1 -- 172.21.15.40:0/3508975555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890032810 con 0x7f58980d5610 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f58a0693640 1 -- 172.21.15.40:0/3508975555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 msgr2=0x7f58980d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.617+0000 7f58a0693640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f58980d3d40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5890009870 tx=0x7f589002a180 comp rx=0 tx=0).stop 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 -- 172.21.15.40:0/3508975555 shutdown_connections 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f58980d3d40 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 --2- 172.21.15.40:0/3508975555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f340 0x7f58980b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 -- 172.21.15.40:0/3508975555 >> 172.21.15.40:0/3508975555 conn(0x7f589808ec70 msgr2=0x7f589809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 -- 172.21.15.40:0/3508975555 shutdown_connections 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.618+0000 7f58a0693640 1 -- 172.21.15.40:0/3508975555 wait complete. 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.619+0000 7f58a0693640 1 Processor -- start 2024-04-26T21:19:16.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.619+0000 7f58a0693640 1 -- start start 2024-04-26T21:19:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.619+0000 7f58a0693640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.619+0000 7f58a0693640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 0x7f58980bab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 0x7f589814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58980b3980 con 0x7f58980d5610 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58980b3800 con 0x7f589815f340 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f58980b3b00 con 0x7f589815f720 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41786/0 (socket says 172.21.15.40:41786) 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 -- 172.21.15.40:0/3314543456 learned_addr learned my addr 172.21.15.40:0/3314543456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589dc2e640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 0x7f58980bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589ec30640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 0x7f589814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 -- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 msgr2=0x7f589814e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 0x7f589814e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 -- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 msgr2=0x7f58980bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 0x7f58980bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 -- 172.21.15.40:0/3314543456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58980ca290 con 0x7f58980d5610 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589ec30640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 0x7f589814e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589dc2e640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 0x7f58980bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f589e42f640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f588800cf90 tx=0x7f588800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588801a070 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588800d840 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58980ca570 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588800dc10 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.620+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58980cab30 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.621+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589806a290 con 0x7f58980d5610 2024-04-26T21:19:16.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.622+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f588801d070 con 0x7f58980d5610 2024-04-26T21:19:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.622+0000 7f5896ffd640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 0x7f588406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.622+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f588808c320 con 0x7f58980d5610 2024-04-26T21:19:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.623+0000 7f589dc2e640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 0x7f588406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.623+0000 7f589dc2e640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 0x7f588406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f588c006fd0 tx=0x7f588c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.627+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f588805a100 con 0x7f58980d5610 2024-04-26T21:19:16.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.816+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5898162e00 con 0x7f58980d5610 2024-04-26T21:19:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.818+0000 7f5896ffd640 1 -- 172.21.15.40:0/3314543456 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16640 (secure 0 0 0) 0x7f5888089110 con 0x7f58980d5610 2024-04-26T21:19:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:16.819 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":108,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:15.844936+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.819+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 msgr2=0x7f588406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.819+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 0x7f588406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f588c006fd0 tx=0x7f588c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.819+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 msgr2=0x7f589815cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.819+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f588800cf90 tx=0x7f588800a7d0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 shutdown_connections 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f588406c8f0 0x7f588406edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f589815f720 0x7f589814e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f589815f340 0x7f58980bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 --2- 172.21.15.40:0/3314543456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58980d5610 0x7f589815cd50 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 >> 172.21.15.40:0/3314543456 conn(0x7f589808ec70 msgr2=0x7f58980d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:16.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 shutdown_connections 2024-04-26T21:19:16.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.820+0000 7f58a0693640 1 -- 172.21.15.40:0/3314543456 wait complete. 2024-04-26T21:19:16.832 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:16.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.931+0000 7f916ba52640 1 Processor -- start 2024-04-26T21:19:16.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.931+0000 7f916ba52640 1 -- start start 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f916ba52640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91640d11b0 0x7f916406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f916ba52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f916415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f916ba52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91640d1740 con 0x7f91640cf2e0 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f916ba52640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f916406e750 con 0x7f91640d11b0 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f916ba52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91640d1a20 con 0x7f916406e2c0 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f91697ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91640cf2e0 0x7f916406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40726/0 (socket says 172.21.15.40:40726) 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f91697ee640 1 -- 172.21.15.40:0/2983260589 learned_addr learned my addr 172.21.15.40:0/2983260589 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:16.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950341653 0 0) 0x7f91640d1740 con 0x7f91640cf2e0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.932+0000 7f9168fed640 1 --2- 172.21.15.40:0/2983260589 >> v2:172.21.15.184:3300/0 conn(0x7f91640d11b0 0x7f916406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f916415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9150003520 con 0x7f91640cf2e0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3969533877 0 0) 0x7f9150003520 con 0x7f91640cf2e0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 msgr2=0x7f916415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f916415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 >> v2:172.21.15.184:3300/0 conn(0x7f91640d11b0 msgr2=0x7f916406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 --2- 172.21.15.40:0/2983260589 >> v2:172.21.15.184:3300/0 conn(0x7f91640d11b0 0x7f916406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91640d1d00 con 0x7f91640cf2e0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9160001670 con 0x7f91640cf2e0 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 >> v1:172.21.15.40:6789/0 conn(0x7f91640cf2e0 legacy=0x7f916406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91500039b0 0x7f9150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f9150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9150004d40 0x7f91500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f916406e750 con 0x7f9150004d40 2024-04-26T21:19:16.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91640d1a20 con 0x7f91500039b0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f915000b9c0 con 0x7f916406e2c0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9150004d40 0x7f91500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f91697ee640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91500039b0 0x7f9150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9168fed640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f9150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 -- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 msgr2=0x7f9150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f916406e2c0 0x7f9150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 -- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91500039b0 msgr2=0x7f9150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91500039b0 0x7f9150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.933+0000 7f9169fef640 1 -- 172.21.15.40:0/2983260589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9150003520 con 0x7f9150004d40 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.934+0000 7f9169fef640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9150004d40 0x7f91500051c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f915c009fc0 tx=0x7f9150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa11d44b6904b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.934+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c009c10 con 0x7f9150004d40 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.934+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c036070 con 0x7f9150004d40 2024-04-26T21:19:16.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.934+0000 7f915bfff640 1 -- 172.21.15.40:0/2983260589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c031880 con 0x7f9150004d40 2024-04-26T21:19:16.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 -- 172.21.15.40:0/2983260589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9150004d40 msgr2=0x7f91500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 --2- 172.21.15.40:0/2983260589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9150004d40 0x7f91500051c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f915c009fc0 tx=0x7f9150005830 comp rx=0 tx=0).stop 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f9169fef640 1 -- 172.21.15.40:0/2983260589 reap_dead start 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 -- 172.21.15.40:0/2983260589 shutdown_connections 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 -- 172.21.15.40:0/2983260589 >> 172.21.15.40:0/2983260589 conn(0x7f91640b4a60 msgr2=0x7f91640b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 -- 172.21.15.40:0/2983260589 shutdown_connections 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.935+0000 7f916ba52640 1 -- 172.21.15.40:0/2983260589 wait complete. 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.936+0000 7f916ba52640 1 Processor -- start 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.936+0000 7f916ba52640 1 -- start start 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 0x7f91641ea260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 0x7f91640b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91640cf6c0 con 0x7f91640b85e0 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91640d22c0 con 0x7f91641599c0 2024-04-26T21:19:16.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f916ba52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91640cf840 con 0x7f91641ea7a0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41796/0 (socket says 172.21.15.40:41796) 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 -- 172.21.15.40:0/2167793838 learned_addr learned my addr 172.21.15.40:0/2167793838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f91697ee640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 0x7f91641ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9168fed640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 0x7f91640b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 -- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 msgr2=0x7f91640b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 0x7f91640b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 -- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 msgr2=0x7f91641ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 0x7f91641ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 -- 172.21.15.40:0/2167793838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916415e470 con 0x7f91640b85e0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f91697ee640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 0x7f91641ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9168fed640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 0x7f91640b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.937+0000 7f9169fef640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e230 tx=0x7f915c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.938+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c009c10 con 0x7f91640b85e0 2024-04-26T21:19:16.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.938+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c031d30 con 0x7f91640b85e0 2024-04-26T21:19:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.938+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916415e750 con 0x7f91640b85e0 2024-04-26T21:19:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.938+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c003bd0 con 0x7f91640b85e0 2024-04-26T21:19:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.938+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9164159da0 con 0x7f91640b85e0 2024-04-26T21:19:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.939+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916406a290 con 0x7f91640b85e0 2024-04-26T21:19:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.939+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f915c005d10 con 0x7f91640b85e0 2024-04-26T21:19:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.940+0000 7f9159ffb640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 0x7f914806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.940+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f915c0ad1d0 con 0x7f91640b85e0 2024-04-26T21:19:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.940+0000 7f91697ee640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 0x7f914806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:16.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.941+0000 7f91697ee640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 0x7f914806edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9160004840 tx=0x7f9160009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:16.944+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f915c04a060 con 0x7f91640b85e0 2024-04-26T21:19:17.123 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:19:17.122+0000 7f90f1fe4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:19:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.129+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91641eab80 con 0x7f91640b85e0 2024-04-26T21:19:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.130+0000 7f9159ffb640 1 -- 172.21.15.40:0/2167793838 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16640 (secure 0 0 0) 0x7f915c07b600 con 0x7f91640b85e0 2024-04-26T21:19:17.132 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:17.132 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":108,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:15.844936+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 msgr2=0x7f914806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 0x7f914806edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9160004840 tx=0x7f9160009040 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 msgr2=0x7f916415d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e230 tx=0x7f915c009750 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 shutdown_connections 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f914806c8f0 0x7f914806edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91640b85e0 0x7f916415d080 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.132+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91641ea7a0 0x7f91640b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.133+0000 7f916ba52640 1 --2- 172.21.15.40:0/2167793838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91641599c0 0x7f91641ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.133+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 >> 172.21.15.40:0/2167793838 conn(0x7f91640b4a60 msgr2=0x7f91640c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.133+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 shutdown_connections 2024-04-26T21:19:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.133+0000 7f916ba52640 1 -- 172.21.15.40:0/2167793838 wait complete. 2024-04-26T21:19:17.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:19:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.248+0000 7fcf678b0640 1 Processor -- start 2024-04-26T21:19:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.248+0000 7fcf678b0640 1 -- start start 2024-04-26T21:19:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 0x7fcf600af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf60069ae0 0x7fcf600b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf600b03e0 0x7fcf600b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600aeb50 con 0x7fcf600b03e0 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600aee30 con 0x7fcf60069ae0 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf678b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf600af110 con 0x7fcf6015f470 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf600b03e0 0x7fcf600b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf600b03e0 0x7fcf600b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41804/0 (socket says 172.21.15.40:41804) 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1031011268 learned_addr learned my addr 172.21.15.40:0/1031011268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf64e4b640 1 --2- 172.21.15.40:0/1031011268 >> v2:172.21.15.184:3300/0 conn(0x7fcf60069ae0 0x7fcf600b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 0x7fcf600af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1031011268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 msgr2=0x7fcf600af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 0x7fcf600af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1031011268 >> v2:172.21.15.184:3300/0 conn(0x7fcf60069ae0 msgr2=0x7fcf600b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1031011268 >> v2:172.21.15.184:3300/0 conn(0x7fcf60069ae0 0x7fcf600b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1031011268 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600af3f0 con 0x7fcf600b03e0 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.249+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 0x7fcf600af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.250+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf600b03e0 0x7fcf600b4600 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006de20 tx=0x7fcf5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e526b59f8c24dbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.250+0000 7fcf57fff640 1 -- 172.21.15.40:0/1031011268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58036070 con 0x7fcf600b03e0 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.250+0000 7fcf57fff640 1 -- 172.21.15.40:0/1031011268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58031600 con 0x7fcf600b03e0 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.250+0000 7fcf57fff640 1 -- 172.21.15.40:0/1031011268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580319d0 con 0x7fcf600b03e0 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 -- 172.21.15.40:0/1031011268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf600b03e0 msgr2=0x7fcf600b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf600b03e0 0x7fcf600b4600 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006de20 tx=0x7fcf5802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 -- 172.21.15.40:0/1031011268 shutdown_connections 2024-04-26T21:19:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf600b03e0 0x7fcf600b4600 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1031011268 >> v2:172.21.15.184:3300/0 conn(0x7fcf60069ae0 0x7fcf600b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1031011268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf6015f470 0x7fcf600af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 -- 172.21.15.40:0/1031011268 >> 172.21.15.40:0/1031011268 conn(0x7fcf60073910 msgr2=0x7fcf60073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 -- 172.21.15.40:0/1031011268 shutdown_connections 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.251+0000 7fcf678b0640 1 -- 172.21.15.40:0/1031011268 wait complete. 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.252+0000 7fcf678b0640 1 Processor -- start 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.252+0000 7fcf678b0640 1 -- start start 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 0x7fcf601589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 0x7fcf60157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600b63a0 con 0x7fcf6015f470 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600b6220 con 0x7fcf600b03e0 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf600afe50 con 0x7fcf60069ae0 2024-04-26T21:19:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41810/0 (socket says 172.21.15.40:41810) 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1739929525 learned_addr learned my addr 172.21.15.40:0/1739929525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf64e4b640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 0x7fcf60157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 0x7fcf601589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 msgr2=0x7fcf601589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 0x7fcf601589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 msgr2=0x7fcf60157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 0x7fcf60157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 -- 172.21.15.40:0/1739929525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60158270 con 0x7fcf6015f470 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 0x7fcf601589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf64e4b640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 0x7fcf60157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf65e4d640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009f40 tx=0x7fcf58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5803f020 con 0x7fcf6015f470 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58037040 con 0x7fcf6015f470 2024-04-26T21:19:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf60159030 con 0x7fcf6015f470 2024-04-26T21:19:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.253+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58036070 con 0x7fcf6015f470 2024-04-26T21:19:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.254+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf60157530 con 0x7fcf6015f470 2024-04-26T21:19:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.255+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6006a290 con 0x7fcf6015f470 2024-04-26T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.255+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf58042020 con 0x7fcf6015f470 2024-04-26T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.255+0000 7fcf55ffb640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 0x7fcf4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.255+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7fcf580ac030 con 0x7fcf6015f470 2024-04-26T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.256+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 0x7fcf4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.256+0000 7fcf6564c640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 0x7fcf4406edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0088b0 tx=0x7fcf5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.260+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf58049030 con 0x7fcf6015f470 2024-04-26T21:19:17.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:17.489+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcf60073f00 con 0x7fcf6015f470 2024-04-26T21:19:18.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.496+0000 7fcf55ffb640 1 -- 172.21.15.40:0/1739929525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7fcf58079ea0 con 0x7fcf6015f470 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 msgr2=0x7fcf4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 0x7fcf4406edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0088b0 tx=0x7fcf5c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 msgr2=0x7fcf6014e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009f40 tx=0x7fcf58004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 shutdown_connections 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4406c8f0 0x7fcf4406edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6015f470 0x7fcf6014e260 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf600b03e0 0x7fcf60157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 --2- 172.21.15.40:0/1739929525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf60069ae0 0x7fcf601589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 >> 172.21.15.40:0/1739929525 conn(0x7fcf60073910 msgr2=0x7fcf600b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:18.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 shutdown_connections 2024-04-26T21:19:18.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.499+0000 7fcf678b0640 1 -- 172.21.15.40:0/1739929525 wait complete. 2024-04-26T21:19:18.513 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:19:18.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:19:18.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.622+0000 7fbe282c8640 1 Processor -- start 2024-04-26T21:19:18.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.622+0000 7fbe282c8640 1 -- start start 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe2015f680 0x7fbe200d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbe200b60b0 0x7fbe200d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2015fa60 con 0x7fbe2015f680 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fbe200d3f50 con 0x7fbe200b60b0 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe282c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200b5090 con 0x7fbe20163aa0 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.623+0000 7fbe26064640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe2015f680 0x7fbe200d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe2015f680 0x7fbe200d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41814/0 (socket says 172.21.15.40:41814) 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 -- 172.21.15.40:0/944570602 learned_addr learned my addr 172.21.15.40:0/944570602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe25863640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26865640 1 --2- 172.21.15.40:0/944570602 >> v2:172.21.15.184:3300/0 conn(0x7fbe200b60b0 0x7fbe200d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 -- 172.21.15.40:0/944570602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 msgr2=0x7fbe200b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 -- 172.21.15.40:0/944570602 >> v2:172.21.15.184:3300/0 conn(0x7fbe200b60b0 msgr2=0x7fbe200d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 --2- 172.21.15.40:0/944570602 >> v2:172.21.15.184:3300/0 conn(0x7fbe200b60b0 0x7fbe200d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.624+0000 7fbe26064640 1 -- 172.21.15.40:0/944570602 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200b5370 con 0x7fbe2015f680 2024-04-26T21:19:18.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe26064640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe200d3820 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fbe200b4b10 tx=0x7fbe1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bd6cc60d6e8000 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:18.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe25062640 1 -- 172.21.15.40:0/944570602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18037070 con 0x7fbe2015f680 2024-04-26T21:19:18.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe25062640 1 -- 172.21.15.40:0/944570602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18032440 con 0x7fbe2015f680 2024-04-26T21:19:18.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe25062640 1 -- 172.21.15.40:0/944570602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18032810 con 0x7fbe2015f680 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 -- 172.21.15.40:0/944570602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 msgr2=0x7fbe200d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe200d3820 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fbe200b4b10 tx=0x7fbe1802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 -- 172.21.15.40:0/944570602 shutdown_connections 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 --2- 172.21.15.40:0/944570602 >> v2:172.21.15.184:3300/0 conn(0x7fbe200b60b0 0x7fbe200d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 --2- 172.21.15.40:0/944570602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe200d3820 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.625+0000 7fbe282c8640 1 -- 172.21.15.40:0/944570602 >> 172.21.15.40:0/944570602 conn(0x7fbe2008eca0 msgr2=0x7fbe2009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.626+0000 7fbe282c8640 1 -- 172.21.15.40:0/944570602 shutdown_connections 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.626+0000 7fbe282c8640 1 -- 172.21.15.40:0/944570602 wait complete. 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 Processor -- start 2024-04-26T21:19:18.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 -- start start 2024-04-26T21:19:18.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe200b60b0 0x7fbe200cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200d55e0 con 0x7fbe2015f680 2024-04-26T21:19:18.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200d5460 con 0x7fbe200b60b0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe282c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200d5760 con 0x7fbe20163aa0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.627+0000 7fbe25863640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41826/0 (socket says 172.21.15.40:41826) 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 -- 172.21.15.40:0/661102828 learned_addr learned my addr 172.21.15.40:0/661102828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe26064640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe200b60b0 0x7fbe200cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe26865640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 -- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 msgr2=0x7fbe200cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 -- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe200b60b0 msgr2=0x7fbe200cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe200b60b0 0x7fbe200cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 -- 172.21.15.40:0/661102828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2015fbe0 con 0x7fbe2015f680 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe25863640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e870 tx=0x7fbe1400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14017070 con 0x7fbe2015f680 2024-04-26T21:19:18.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1400c940 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400cd30 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe200ced70 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.628+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2015fe80 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.629+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2006a290 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.630+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe1401a070 con 0x7fbe2015f680 2024-04-26T21:19:18.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.630+0000 7fbe1effd640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 0x7fbe0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:18.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.630+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fbe1408b8e0 con 0x7fbe2015f680 2024-04-26T21:19:18.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.630+0000 7fbe26064640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 0x7fbe0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:18.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.631+0000 7fbe26064640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 0x7fbe0c06edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:18.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.634+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe14059750 con 0x7fbe2015f680 2024-04-26T21:19:18.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:18.864+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbe200b4cd0 con 0x7fbe2015f680 2024-04-26T21:19:19.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.502+0000 7fbe1effd640 1 -- 172.21.15.40:0/661102828 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7fbe1405da50 con 0x7fbe2015f680 2024-04-26T21:19:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 msgr2=0x7fbe0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 0x7fbe0c06edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe1802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 msgr2=0x7fbe20156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e870 tx=0x7fbe1400ed30 comp rx=0 tx=0).stop 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 shutdown_connections 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe0c06c8f0 0x7fbe0c06edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe20163aa0 0x7fbe200cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe2015f680 0x7fbe20156920 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 --2- 172.21.15.40:0/661102828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe200b60b0 0x7fbe200cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 >> 172.21.15.40:0/661102828 conn(0x7fbe2008eca0 msgr2=0x7fbe200b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.504+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 shutdown_connections 2024-04-26T21:19:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.505+0000 7fbe282c8640 1 -- 172.21.15.40:0/661102828 wait complete. 2024-04-26T21:19:19.519 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:19:19.520 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:19.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.626+0000 7ffa44d6a640 1 Processor -- start 2024-04-26T21:19:19.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.626+0000 7ffa44d6a640 1 -- start start 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa44d6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa40148a30 0x7ffa40073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa44d6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa400b3af0 0x7ffa400b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa44d6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa400cfa80 con 0x7ffa40160640 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa44d6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffa400cfd60 con 0x7ffa40148a30 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa44d6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa400d0040 con 0x7ffa400b3af0 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffa40160640 0x7ffa401636a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40736/0 (socket says 172.21.15.40:40736) 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa37fff640 1 -- 172.21.15.40:0/2346981797 learned_addr learned my addr 172.21.15.40:0/2346981797 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2346981797 >> v2:172.21.15.40:3300/0 conn(0x7ffa40148a30 0x7ffa40073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3ed9d640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa400b3af0 0x7ffa400b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3dd9b640 1 -- 172.21.15.40:0/2346981797 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893515050 0 0) 0x7ffa400cfa80 con 0x7ffa40160640 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3dd9b640 1 -- 172.21.15.40:0/2346981797 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa2c003520 con 0x7ffa40160640 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2346981797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa400b3af0 msgr2=0x7ffa400b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa400b3af0 0x7ffa400b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2346981797 >> v1:172.21.15.40:6789/0 conn(0x7ffa40160640 legacy=0x7ffa401636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.627+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2346981797 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa40162ea0 con 0x7ffa40148a30 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.628+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa40148a30 0x7ffa40073ed0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ffa300097a0 tx=0x7ffa3002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7925f77172cda99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.628+0000 7ffa3dd9b640 1 -- 172.21.15.40:0/2346981797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30037070 con 0x7ffa40148a30 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.628+0000 7ffa3dd9b640 1 -- 172.21.15.40:0/2346981797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3002ae50 con 0x7ffa40148a30 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.628+0000 7ffa3dd9b640 1 -- 172.21.15.40:0/2346981797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30032660 con 0x7ffa40148a30 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2346981797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa40148a30 msgr2=0x7ffa40073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa40148a30 0x7ffa40073ed0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ffa300097a0 tx=0x7ffa3002a100 comp rx=0 tx=0).stop 2024-04-26T21:19:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2346981797 shutdown_connections 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa400b3af0 0x7ffa400b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2346981797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa40148a30 0x7ffa40073ed0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2346981797 >> 172.21.15.40:0/2346981797 conn(0x7ffa400ce8b0 msgr2=0x7ffa400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2346981797 shutdown_connections 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.629+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2346981797 wait complete. 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.630+0000 7ffa44d6a640 1 Processor -- start 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.630+0000 7ffa44d6a640 1 -- start start 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa40148a30 0x7ffa401f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 0x7ffa400b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40074410 con 0x7ffa400b3af0 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40163db0 con 0x7ffa40160640 2024-04-26T21:19:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa44d6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ffa40074590 con 0x7ffa40148a30 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41852/0 (socket says 172.21.15.40:41852) 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2903419037 learned_addr learned my addr 172.21.15.40:0/2903419037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa37fff640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa40148a30 0x7ffa401f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3ed9d640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 0x7ffa400b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa40148a30 msgr2=0x7ffa401f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa40148a30 0x7ffa401f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 msgr2=0x7ffa400b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 0x7ffa400b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3e59c640 1 -- 172.21.15.40:0/2903419037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400b4e30 con 0x7ffa400b3af0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.631+0000 7ffa3ed9d640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 0x7ffa400b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa3e59c640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ffa30000c00 tx=0x7ffa300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30041070 con 0x7ffa400b3af0 2024-04-26T21:19:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3003c460 con 0x7ffa400b3af0 2024-04-26T21:19:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30037040 con 0x7ffa400b3af0 2024-04-26T21:19:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa400b5110 con 0x7ffa400b3af0 2024-04-26T21:19:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.632+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa400b56d0 con 0x7ffa400b3af0 2024-04-26T21:19:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.633+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4006a290 con 0x7ffa400b3af0 2024-04-26T21:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.633+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa30044070 con 0x7ffa400b3af0 2024-04-26T21:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.634+0000 7ffa36ffd640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 0x7ffa2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.634+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7ffa300ac140 con 0x7ffa400b3af0 2024-04-26T21:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.634+0000 7ffa37fff640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 0x7ffa2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.635+0000 7ffa37fff640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 0x7ffa2006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffa40146840 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.638+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa30079e80 con 0x7ffa400b3af0 2024-04-26T21:19:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.860+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa400b1b70 con 0x7ffa400b3af0 2024-04-26T21:19:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.861+0000 7ffa36ffd640 1 -- 172.21.15.40:0/2903419037 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5186 (secure 0 0 0) 0x7ffa3007e180 con 0x7ffa400b3af0 2024-04-26T21:19:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:19.862 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:18.865416+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 64 2024-04-26T21:19:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 msgr2=0x7ffa2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 0x7ffa2006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffa40146840 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-04-26T21:19:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 msgr2=0x7ffa4015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ffa30000c00 tx=0x7ffa300046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 shutdown_connections 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ffa2006c8f0 0x7ffa2006edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa40160640 0x7ffa400b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa40148a30 0x7ffa401f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 --2- 172.21.15.40:0/2903419037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa400b3af0 0x7ffa4015e720 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 >> 172.21.15.40:0/2903419037 conn(0x7ffa400ce8b0 msgr2=0x7ffa4015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 shutdown_connections 2024-04-26T21:19:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.863+0000 7ffa44d6a640 1 -- 172.21.15.40:0/2903419037 wait complete. 2024-04-26T21:19:19.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.975+0000 7fa3e03c1640 1 Processor -- start 2024-04-26T21:19:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.975+0000 7fa3e03c1640 1 -- start start 2024-04-26T21:19:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3e03c1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d8145980 0x7fa3d8160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3e03c1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 0x7fa3d80720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3e03c1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d80b4260 con 0x7fa3d80b3de0 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3e03c1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d80b5140 con 0x7fa3d8145980 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3e03c1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d80b4530 con 0x7fa3d80b61a0 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de15d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d8145980 0x7fa3d8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3dd95c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa3d80b3de0 0x7fa3d80b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40738/0 (socket says 172.21.15.40:40738) 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3dd95c640 1 -- 172.21.15.40:0/150051548 learned_addr learned my addr 172.21.15.40:0/150051548 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de95e640 1 --2- 172.21.15.40:0/150051548 >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 0x7fa3d80720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de15d640 1 -- 172.21.15.40:0/150051548 >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 msgr2=0x7fa3d80720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de15d640 1 --2- 172.21.15.40:0/150051548 >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 0x7fa3d80720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de15d640 1 -- 172.21.15.40:0/150051548 >> v1:172.21.15.40:6789/0 conn(0x7fa3d80b3de0 legacy=0x7fa3d80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de15d640 1 -- 172.21.15.40:0/150051548 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d80b4810 con 0x7fa3d8145980 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3dd15b640 1 -- 172.21.15.40:0/150051548 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936808460 0 0) 0x7fa3d80b4260 con 0x7fa3d80b3de0 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.976+0000 7fa3de95e640 1 --2- 172.21.15.40:0/150051548 >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 0x7fa3d80720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3de15d640 1 --2- 172.21.15.40:0/150051548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8160630 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80704b0 tx=0x7fa3d402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f6b027c973ce35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3dd15b640 1 -- 172.21.15.40:0/150051548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4037070 con 0x7fa3d8145980 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3dd15b640 1 -- 172.21.15.40:0/150051548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d4032440 con 0x7fa3d8145980 2024-04-26T21:19:19.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3dd15b640 1 -- 172.21.15.40:0/150051548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d402ae60 con 0x7fa3d8145980 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 -- 172.21.15.40:0/150051548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 msgr2=0x7fa3d8160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/150051548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8160630 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80704b0 tx=0x7fa3d402a110 comp rx=0 tx=0).stop 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 -- 172.21.15.40:0/150051548 shutdown_connections 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/150051548 >> v2:172.21.15.184:3300/0 conn(0x7fa3d80b61a0 0x7fa3d80720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/150051548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8160630 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.977+0000 7fa3e03c1640 1 -- 172.21.15.40:0/150051548 >> 172.21.15.40:0/150051548 conn(0x7fa3d80d0d30 msgr2=0x7fa3d80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.978+0000 7fa3e03c1640 1 -- 172.21.15.40:0/150051548 shutdown_connections 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.978+0000 7fa3e03c1640 1 -- 172.21.15.40:0/150051548 wait complete. 2024-04-26T21:19:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 Processor -- start 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 -- start start 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3d80b3de0 0x7fa3d813f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 0x7fa3d80b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8072610 con 0x7fa3d8145980 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d80b4dd0 con 0x7fa3d80b61a0 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.979+0000 7fa3e03c1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d8072790 con 0x7fa3d80b3de0 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41864/0 (socket says 172.21.15.40:41864) 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 -- 172.21.15.40:0/4162518376 learned_addr learned my addr 172.21.15.40:0/4162518376 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3dd95c640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 0x7fa3d80b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de15d640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3d80b3de0 0x7fa3d813f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 -- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3d80b3de0 msgr2=0x7fa3d813f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3d80b3de0 0x7fa3d813f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 -- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 msgr2=0x7fa3d80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 0x7fa3d80b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 -- 172.21.15.40:0/4162518376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d8142420 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3dd95c640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 0x7fa3d80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3de95e640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cef0 tx=0x7fa3d000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d001a070 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d000d860 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d80b1ff0 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000dc50 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.980+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d80b25b0 con 0x7fa3d8145980 2024-04-26T21:19:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.981+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d806a290 con 0x7fa3d8145980 2024-04-26T21:19:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.982+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3d001d070 con 0x7fa3d8145980 2024-04-26T21:19:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.982+0000 7fa3ceffd640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 0x7fa3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.982+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa3d008c420 con 0x7fa3d8145980 2024-04-26T21:19:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.982+0000 7fa3de15d640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 0x7fa3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.983+0000 7fa3de15d640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 0x7fa3bc06edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4034000 tx=0x7fa3d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:19.986+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3d005a910 con 0x7fa3d8145980 2024-04-26T21:19:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.171+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3d81677a0 con 0x7fa3d8145980 2024-04-26T21:19:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.172+0000 7fa3ceffd640 1 -- 172.21.15.40:0/4162518376 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16627 (secure 0 0 0) 0x7fa3d005a210 con 0x7fa3d8145980 2024-04-26T21:19:20.173 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:20.174 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":109,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:17.479995+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:20.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 msgr2=0x7fa3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 0x7fa3bc06edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4034000 tx=0x7fa3d402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 msgr2=0x7fa3d8156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cef0 tx=0x7fa3d000a710 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 shutdown_connections 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa3bc06c8f0 0x7fa3bc06edb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8145980 0x7fa3d8156b70 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d80b61a0 0x7fa3d80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 --2- 172.21.15.40:0/4162518376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3d80b3de0 0x7fa3d813f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 >> 172.21.15.40:0/4162518376 conn(0x7fa3d80d0d30 msgr2=0x7fa3d80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 shutdown_connections 2024-04-26T21:19:20.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.174+0000 7fa3e03c1640 1 -- 172.21.15.40:0/4162518376 wait complete. 2024-04-26T21:19:20.187 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:19:20.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:19:20.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.286+0000 7f9831f6a640 1 Processor -- start 2024-04-26T21:19:20.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 -- start start 2024-04-26T21:19:20.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 0x7f982c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f982c0d5610 0x7f982c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c0b2510 con 0x7f982c15f340 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f982c0b27f0 con 0x7f982c0d5610 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f9831f6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0b2ad0 con 0x7f982c15f720 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.287+0000 7f982bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f982c0d5610 0x7f982c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f982c0d5610 0x7f982c0d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41880/0 (socket says 172.21.15.40:41880) 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 -- 172.21.15.40:0/758979964 learned_addr learned my addr 172.21.15.40:0/758979964 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982affd640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 0x7f982c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 -- 172.21.15.40:0/758979964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 msgr2=0x7f982c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 0x7f982c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 -- 172.21.15.40:0/758979964 >> v1:172.21.15.184:6789/0 conn(0x7f982c15f340 legacy=0x7f982c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 -- 172.21.15.40:0/758979964 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0b2db0 con 0x7f982c0d5610 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982affd640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 0x7f982c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.288+0000 7f982bfff640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0d5610 0x7f982c0d3d40 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9824009e50 tx=0x7f982402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d132b82f6d06568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f982a7fc640 1 -- 172.21.15.40:0/758979964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824036070 con 0x7f982c0d5610 2024-04-26T21:19:20.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f982a7fc640 1 -- 172.21.15.40:0/758979964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9824031600 con 0x7f982c0d5610 2024-04-26T21:19:20.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f982a7fc640 1 -- 172.21.15.40:0/758979964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240319d0 con 0x7f982c0d5610 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 -- 172.21.15.40:0/758979964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0d5610 msgr2=0x7f982c0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0d5610 0x7f982c0d3d40 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9824009e50 tx=0x7f982402a630 comp rx=0 tx=0).stop 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 -- 172.21.15.40:0/758979964 shutdown_connections 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c0d5610 0x7f982c0d3d40 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 --2- 172.21.15.40:0/758979964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c15f720 0x7f982c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 -- 172.21.15.40:0/758979964 >> 172.21.15.40:0/758979964 conn(0x7f982c08ec70 msgr2=0x7f982c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 -- 172.21.15.40:0/758979964 shutdown_connections 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.289+0000 7f9831f6a640 1 -- 172.21.15.40:0/758979964 wait complete. 2024-04-26T21:19:20.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.290+0000 7f9831f6a640 1 Processor -- start 2024-04-26T21:19:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 -- start start 2024-04-26T21:19:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 0x7f982c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 0x7f982c15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0b3980 con 0x7f982c15f340 2024-04-26T21:19:20.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0b3800 con 0x7f982c15f720 2024-04-26T21:19:20.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f9831f6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0b3b00 con 0x7f982c0d5610 2024-04-26T21:19:20.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.291+0000 7f982affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41890/0 (socket says 172.21.15.40:41890) 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 -- 172.21.15.40:0/3707811083 learned_addr learned my addr 172.21.15.40:0/3707811083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982bfff640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 0x7f982c15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982b7fe640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 0x7f982c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 -- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 msgr2=0x7f982c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 0x7f982c0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 -- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 msgr2=0x7f982c15fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 0x7f982c15fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 -- 172.21.15.40:0/3707811083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0ca7f0 con 0x7f982c15f340 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982bfff640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 0x7f982c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:20.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982b7fe640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 0x7f982c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f982affd640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f982000c7f0 tx=0x7f982000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820015070 con 0x7f982c15f340 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9820007820 con 0x7f982c15f340 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c0caad0 con 0x7f982c15f340 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820007bd0 con 0x7f982c15f340 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.292+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c0cb090 con 0x7f982c15f340 2024-04-26T21:19:20.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.293+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c162e40 con 0x7f982c15f340 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.294+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9820018070 con 0x7f982c15f340 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.294+0000 7f9813fff640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.294+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f982008bd50 con 0x7f982c15f340 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.295+0000 7f982b7fe640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.295+0000 7f982b7fe640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 0x7f981806edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f981c003eb0 tx=0x7f981c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.298+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f982c162e40 con 0x7f982c15f340 2024-04-26T21:19:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.480+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f982c06a290 con 0x7f981806c8f0 2024-04-26T21:19:20.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.487+0000 7f9813fff640 1 -- 172.21.15.40:0/3707811083 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251120 (secure 0 0 0) 0x7f981c055500 con 0x7f981806c8f0 2024-04-26T21:19:20.491 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:20.492 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 msgr2=0x7f981806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 0x7f981806edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f981c003eb0 tx=0x7f981c009040 comp rx=0 tx=0).stop 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 msgr2=0x7f982c1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f982000c7f0 tx=0x7f982000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 shutdown_connections 2024-04-26T21:19:20.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f981806c8f0 0x7f981806edb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f982c15f720 0x7f982c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f982c15f340 0x7f982c1507a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 --2- 172.21.15.40:0/3707811083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f982c0d5610 0x7f982c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 >> 172.21.15.40:0/3707811083 conn(0x7f982c08ec70 msgr2=0x7f982c0b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 shutdown_connections 2024-04-26T21:19:20.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.490+0000 7f9831f6a640 1 -- 172.21.15.40:0/3707811083 wait complete. 2024-04-26T21:19:20.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-26T21:19:19.024771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265288,"kb_used_data":11648,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709432,"statfs":{"total":773094113280,"available":772822458368,"internally_reserved":0,"allocated":11927552,"data_stored":5745414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.560991"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498644+0000","last_change":"2024-04-26T21:19:16.496097+0000","last_active":"2024-04-26T21:19:16.498644+0000","last_peered":"2024-04-26T21:19:16.498644+0000","last_clean":"2024-04-26T21:19:16.498644+0000","last_became_active":"2024-04-26T21:19:16.494023+0000","last_became_peered":"2024-04-26T21:19:16.494023+0000","last_unstale":"2024-04-26T21:19:16.498644+0000","last_undegraded":"2024-04-26T21:19:16.498644+0000","last_fullsized":"2024-04-26T21:19:16.498644+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:34.681056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501847+0000","last_change":"2024-04-26T21:19:16.498688+0000","last_active":"2024-04-26T21:19:16.501847+0000","last_peered":"2024-04-26T21:19:16.501847+0000","last_clean":"2024-04-26T21:19:16.501847+0000","last_became_active":"2024-04-26T21:19:16.498359+0000","last_became_peered":"2024-04-26T21:19:16.498359+0000","last_unstale":"2024-04-26T21:19:16.501847+0000","last_undegraded":"2024-04-26T21:19:16.501847+0000","last_fullsized":"2024-04-26T21:19:16.501847+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538503+0000","last_change":"2024-04-26T21:19:16.496173+0000","last_active":"2024-04-26T21:19:17.538503+0000","last_peered":"2024-04-26T21:19:17.538503+0000","last_clean":"2024-04-26T21:19:17.538503+0000","last_became_active":"2024-04-26T21:19:16.495826+0000","last_became_peered":"2024-04-26T21:19:16.495826+0000","last_unstale":"2024-04-26T21:19:17.538503+0000","last_undegraded":"2024-04-26T21:19:17.538503+0000","last_fullsized":"2024-04-26T21:19:17.538503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:13.492714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498530+0000","last_change":"2024-04-26T21:19:16.495999+0000","last_active":"2024-04-26T21:19:16.498530+0000","last_peered":"2024-04-26T21:19:16.498530+0000","last_clean":"2024-04-26T21:19:16.498530+0000","last_became_active":"2024-04-26T21:19:16.495621+0000","last_became_peered":"2024-04-26T21:19:16.495621+0000","last_unstale":"2024-04-26T21:19:16.498530+0000","last_undegraded":"2024-04-26T21:19:16.498530+0000","last_fullsized":"2024-04-26T21:19:16.498530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492356+0000","last_change":"2024-04-26T21:19:16.482865+0000","last_active":"2024-04-26T21:19:17.492356+0000","last_peered":"2024-04-26T21:19:17.492356+0000","last_clean":"2024-04-26T21:19:17.492356+0000","last_became_active":"2024-04-26T21:19:16.482588+0000","last_became_peered":"2024-04-26T21:19:16.482588+0000","last_unstale":"2024-04-26T21:19:17.492356+0000","last_undegraded":"2024-04-26T21:19:17.492356+0000","last_fullsized":"2024-04-26T21:19:17.492356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:23.086160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672936+0000","last_change":"2024-04-26T21:19:16.488443+0000","last_active":"2024-04-26T21:19:17.672936+0000","last_peered":"2024-04-26T21:19:17.672936+0000","last_clean":"2024-04-26T21:19:17.672936+0000","last_became_active":"2024-04-26T21:19:16.488149+0000","last_became_peered":"2024-04-26T21:19:16.488149+0000","last_unstale":"2024-04-26T21:19:17.672936+0000","last_undegraded":"2024-04-26T21:19:17.672936+0000","last_fullsized":"2024-04-26T21:19:17.672936+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538580+0000","last_change":"2024-04-26T21:19:16.498705+0000","last_active":"2024-04-26T21:19:17.538580+0000","last_peered":"2024-04-26T21:19:17.538580+0000","last_clean":"2024-04-26T21:19:17.538580+0000","last_became_active":"2024-04-26T21:19:16.498371+0000","last_became_peered":"2024-04-26T21:19:16.498371+0000","last_unstale":"2024-04-26T21:19:17.538580+0000","last_undegraded":"2024-04-26T21:19:17.538580+0000","last_fullsized":"2024-04-26T21:19:17.538580+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:33.522331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496250+0000","last_change":"2024-04-26T21:19:16.489425+0000","last_active":"2024-04-26T21:19:16.496250+0000","last_peered":"2024-04-26T21:19:16.496250+0000","last_clean":"2024-04-26T21:19:16.496250+0000","last_became_active":"2024-04-26T21:19:16.489130+0000","last_became_peered":"2024-04-26T21:19:16.489130+0000","last_unstale":"2024-04-26T21:19:16.496250+0000","last_undegraded":"2024-04-26T21:19:16.496250+0000","last_fullsized":"2024-04-26T21:19:16.496250+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488375+0000","last_change":"2024-04-26T21:19:16.485878+0000","last_active":"2024-04-26T21:19:16.488375+0000","last_peered":"2024-04-26T21:19:16.488375+0000","last_clean":"2024-04-26T21:19:16.488375+0000","last_became_active":"2024-04-26T21:19:16.485592+0000","last_became_peered":"2024-04-26T21:19:16.485592+0000","last_unstale":"2024-04-26T21:19:16.488375+0000","last_undegraded":"2024-04-26T21:19:16.488375+0000","last_fullsized":"2024-04-26T21:19:16.488375+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496673+0000","last_change":"2024-04-26T21:19:16.492337+0000","last_active":"2024-04-26T21:19:16.496673+0000","last_peered":"2024-04-26T21:19:16.496673+0000","last_clean":"2024-04-26T21:19:16.496673+0000","last_became_active":"2024-04-26T21:19:16.492009+0000","last_became_peered":"2024-04-26T21:19:16.492009+0000","last_unstale":"2024-04-26T21:19:16.496673+0000","last_undegraded":"2024-04-26T21:19:16.496673+0000","last_fullsized":"2024-04-26T21:19:16.496673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:37.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488890+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:16.488890+0000","last_peered":"2024-04-26T21:19:16.488890+0000","last_clean":"2024-04-26T21:19:16.488890+0000","last_became_active":"2024-04-26T21:19:16.484183+0000","last_became_peered":"2024-04-26T21:19:16.484183+0000","last_unstale":"2024-04-26T21:19:16.488890+0000","last_undegraded":"2024-04-26T21:19:16.488890+0000","last_fullsized":"2024-04-26T21:19:16.488890+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538678+0000","last_change":"2024-04-26T21:19:16.495059+0000","last_active":"2024-04-26T21:19:17.538678+0000","last_peered":"2024-04-26T21:19:17.538678+0000","last_clean":"2024-04-26T21:19:17.538678+0000","last_became_active":"2024-04-26T21:19:16.494780+0000","last_became_peered":"2024-04-26T21:19:16.494780+0000","last_unstale":"2024-04-26T21:19:17.538678+0000","last_undegraded":"2024-04-26T21:19:17.538678+0000","last_fullsized":"2024-04-26T21:19:17.538678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538257+0000","last_change":"2024-04-26T21:19:16.490955+0000","last_active":"2024-04-26T21:19:17.538257+0000","last_peered":"2024-04-26T21:19:17.538257+0000","last_clean":"2024-04-26T21:19:17.538257+0000","last_became_active":"2024-04-26T21:19:16.490657+0000","last_became_peered":"2024-04-26T21:19:16.490657+0000","last_unstale":"2024-04-26T21:19:17.538257+0000","last_undegraded":"2024-04-26T21:19:17.538257+0000","last_fullsized":"2024-04-26T21:19:17.538257+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491077+0000","last_change":"2024-04-26T21:19:16.487462+0000","last_active":"2024-04-26T21:19:16.491077+0000","last_peered":"2024-04-26T21:19:16.491077+0000","last_clean":"2024-04-26T21:19:16.491077+0000","last_became_active":"2024-04-26T21:19:16.486946+0000","last_became_peered":"2024-04-26T21:19:16.486946+0000","last_unstale":"2024-04-26T21:19:16.491077+0000","last_undegraded":"2024-04-26T21:19:16.491077+0000","last_fullsized":"2024-04-26T21:19:16.491077+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496577+0000","last_change":"2024-04-26T21:19:16.491720+0000","last_active":"2024-04-26T21:19:16.496577+0000","last_peered":"2024-04-26T21:19:16.496577+0000","last_clean":"2024-04-26T21:19:16.496577+0000","last_became_active":"2024-04-26T21:19:16.491369+0000","last_became_peered":"2024-04-26T21:19:16.491369+0000","last_unstale":"2024-04-26T21:19:16.496577+0000","last_undegraded":"2024-04-26T21:19:16.496577+0000","last_fullsized":"2024-04-26T21:19:16.496577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:18.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499278+0000","last_change":"2024-04-26T21:19:16.496447+0000","last_active":"2024-04-26T21:19:16.499278+0000","last_peered":"2024-04-26T21:19:16.499278+0000","last_clean":"2024-04-26T21:19:16.499278+0000","last_became_active":"2024-04-26T21:19:16.496204+0000","last_became_peered":"2024-04-26T21:19:16.496204+0000","last_unstale":"2024-04-26T21:19:16.499278+0000","last_undegraded":"2024-04-26T21:19:16.499278+0000","last_fullsized":"2024-04-26T21:19:16.499278+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:50.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538653+0000","last_change":"2024-04-26T21:19:16.491438+0000","last_active":"2024-04-26T21:19:17.538653+0000","last_peered":"2024-04-26T21:19:17.538653+0000","last_clean":"2024-04-26T21:19:17.538653+0000","last_became_active":"2024-04-26T21:19:16.490909+0000","last_became_peered":"2024-04-26T21:19:16.490909+0000","last_unstale":"2024-04-26T21:19:17.538653+0000","last_undegraded":"2024-04-26T21:19:17.538653+0000","last_fullsized":"2024-04-26T21:19:17.538653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496630+0000","last_change":"2024-04-26T21:19:16.491473+0000","last_active":"2024-04-26T21:19:16.496630+0000","last_peered":"2024-04-26T21:19:16.496630+0000","last_clean":"2024-04-26T21:19:16.496630+0000","last_became_active":"2024-04-26T21:19:16.491070+0000","last_became_peered":"2024-04-26T21:19:16.491070+0000","last_unstale":"2024-04-26T21:19:16.496630+0000","last_undegraded":"2024-04-26T21:19:16.496630+0000","last_fullsized":"2024-04-26T21:19:16.496630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:44.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501985+0000","last_change":"2024-04-26T21:19:16.499382+0000","last_active":"2024-04-26T21:19:16.501985+0000","last_peered":"2024-04-26T21:19:16.501985+0000","last_clean":"2024-04-26T21:19:16.501985+0000","last_became_active":"2024-04-26T21:19:16.499064+0000","last_became_peered":"2024-04-26T21:19:16.499064+0000","last_unstale":"2024-04-26T21:19:16.501985+0000","last_undegraded":"2024-04-26T21:19:16.501985+0000","last_fullsized":"2024-04-26T21:19:16.501985+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491852+0000","last_change":"2024-04-26T21:19:16.485418+0000","last_active":"2024-04-26T21:19:17.491852+0000","last_peered":"2024-04-26T21:19:17.491852+0000","last_clean":"2024-04-26T21:19:17.491852+0000","last_became_active":"2024-04-26T21:19:16.484504+0000","last_became_peered":"2024-04-26T21:19:16.484504+0000","last_unstale":"2024-04-26T21:19:17.491852+0000","last_undegraded":"2024-04-26T21:19:17.491852+0000","last_fullsized":"2024-04-26T21:19:17.491852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492897+0000","last_change":"2024-04-26T21:19:16.487920+0000","last_active":"2024-04-26T21:19:16.492897+0000","last_peered":"2024-04-26T21:19:16.492897+0000","last_clean":"2024-04-26T21:19:16.492897+0000","last_became_active":"2024-04-26T21:19:16.487548+0000","last_became_peered":"2024-04-26T21:19:16.487548+0000","last_unstale":"2024-04-26T21:19:16.492897+0000","last_undegraded":"2024-04-26T21:19:16.492897+0000","last_fullsized":"2024-04-26T21:19:16.492897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491267+0000","last_change":"2024-04-26T21:19:16.485140+0000","last_active":"2024-04-26T21:19:17.491267+0000","last_peered":"2024-04-26T21:19:17.491267+0000","last_clean":"2024-04-26T21:19:17.491267+0000","last_became_active":"2024-04-26T21:19:16.484871+0000","last_became_peered":"2024-04-26T21:19:16.484871+0000","last_unstale":"2024-04-26T21:19:17.491267+0000","last_undegraded":"2024-04-26T21:19:17.491267+0000","last_fullsized":"2024-04-26T21:19:17.491267+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.494135+0000","last_change":"2024-04-26T21:19:16.491074+0000","last_active":"2024-04-26T21:19:16.494135+0000","last_peered":"2024-04-26T21:19:16.494135+0000","last_clean":"2024-04-26T21:19:16.494135+0000","last_became_active":"2024-04-26T21:19:16.490719+0000","last_became_peered":"2024-04-26T21:19:16.490719+0000","last_unstale":"2024-04-26T21:19:16.494135+0000","last_undegraded":"2024-04-26T21:19:16.494135+0000","last_fullsized":"2024-04-26T21:19:16.494135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:07.415274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499547+0000","last_change":"2024-04-26T21:19:16.496876+0000","last_active":"2024-04-26T21:19:16.499547+0000","last_peered":"2024-04-26T21:19:16.499547+0000","last_clean":"2024-04-26T21:19:16.499547+0000","last_became_active":"2024-04-26T21:19:16.496601+0000","last_became_peered":"2024-04-26T21:19:16.496601+0000","last_unstale":"2024-04-26T21:19:16.499547+0000","last_undegraded":"2024-04-26T21:19:16.499547+0000","last_fullsized":"2024-04-26T21:19:16.499547+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:31.534507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491768+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:17.491768+0000","last_peered":"2024-04-26T21:19:17.491768+0000","last_clean":"2024-04-26T21:19:17.491768+0000","last_became_active":"2024-04-26T21:19:16.484220+0000","last_became_peered":"2024-04-26T21:19:16.484220+0000","last_unstale":"2024-04-26T21:19:17.491768+0000","last_undegraded":"2024-04-26T21:19:17.491768+0000","last_fullsized":"2024-04-26T21:19:17.491768+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:25.928632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499556+0000","last_change":"2024-04-26T21:19:16.494852+0000","last_active":"2024-04-26T21:19:16.499556+0000","last_peered":"2024-04-26T21:19:16.499556+0000","last_clean":"2024-04-26T21:19:16.499556+0000","last_became_active":"2024-04-26T21:19:16.494590+0000","last_became_peered":"2024-04-26T21:19:16.494590+0000","last_unstale":"2024-04-26T21:19:16.499556+0000","last_undegraded":"2024-04-26T21:19:16.499556+0000","last_fullsized":"2024-04-26T21:19:16.499556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:15.415456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673032+0000","last_change":"2024-04-26T21:19:16.484845+0000","last_active":"2024-04-26T21:19:17.673032+0000","last_peered":"2024-04-26T21:19:17.673032+0000","last_clean":"2024-04-26T21:19:17.673032+0000","last_became_active":"2024-04-26T21:19:16.484525+0000","last_became_peered":"2024-04-26T21:19:16.484525+0000","last_unstale":"2024-04-26T21:19:17.673032+0000","last_undegraded":"2024-04-26T21:19:17.673032+0000","last_fullsized":"2024-04-26T21:19:17.673032+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:31.425192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499512+0000","last_change":"2024-04-26T21:19:16.497005+0000","last_active":"2024-04-26T21:19:16.499512+0000","last_peered":"2024-04-26T21:19:16.499512+0000","last_clean":"2024-04-26T21:19:16.499512+0000","last_became_active":"2024-04-26T21:19:16.496669+0000","last_became_peered":"2024-04-26T21:19:16.496669+0000","last_unstale":"2024-04-26T21:19:16.499512+0000","last_undegraded":"2024-04-26T21:19:16.499512+0000","last_fullsized":"2024-04-26T21:19:16.499512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.580462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673108+0000","last_change":"2024-04-26T21:19:15.478535+0000","last_active":"2024-04-26T21:19:17.673108+0000","last_peered":"2024-04-26T21:19:17.673108+0000","last_clean":"2024-04-26T21:19:17.673108+0000","last_became_active":"2024-04-26T21:19:15.478272+0000","last_became_peered":"2024-04-26T21:19:15.478272+0000","last_unstale":"2024-04-26T21:19:17.673108+0000","last_undegraded":"2024-04-26T21:19:17.673108+0000","last_fullsized":"2024-04-26T21:19:17.673108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492017+0000","last_change":"2024-04-26T21:19:16.488880+0000","last_active":"2024-04-26T21:19:16.492017+0000","last_peered":"2024-04-26T21:19:16.492017+0000","last_clean":"2024-04-26T21:19:16.492017+0000","last_became_active":"2024-04-26T21:19:16.488578+0000","last_became_peered":"2024-04-26T21:19:16.488578+0000","last_unstale":"2024-04-26T21:19:16.492017+0000","last_undegraded":"2024-04-26T21:19:16.492017+0000","last_fullsized":"2024-04-26T21:19:16.492017+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538442+0000","last_change":"2024-04-26T21:19:15.481825+0000","last_active":"2024-04-26T21:19:17.538442+0000","last_peered":"2024-04-26T21:19:17.538442+0000","last_clean":"2024-04-26T21:19:17.538442+0000","last_became_active":"2024-04-26T21:19:15.481529+0000","last_became_peered":"2024-04-26T21:19:15.481529+0000","last_unstale":"2024-04-26T21:19:17.538442+0000","last_undegraded":"2024-04-26T21:19:17.538442+0000","last_fullsized":"2024-04-26T21:19:17.538442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492403+0000","last_change":"2024-04-26T21:19:15.485103+0000","last_active":"2024-04-26T21:19:17.492403+0000","last_peered":"2024-04-26T21:19:17.492403+0000","last_clean":"2024-04-26T21:19:17.492403+0000","last_became_active":"2024-04-26T21:19:15.484770+0000","last_became_peered":"2024-04-26T21:19:15.484770+0000","last_unstale":"2024-04-26T21:19:17.492403+0000","last_undegraded":"2024-04-26T21:19:17.492403+0000","last_fullsized":"2024-04-26T21:19:17.492403+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:15.463835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538722+0000","last_change":"2024-04-26T21:19:16.491699+0000","last_active":"2024-04-26T21:19:17.538722+0000","last_peered":"2024-04-26T21:19:17.538722+0000","last_clean":"2024-04-26T21:19:17.538722+0000","last_became_active":"2024-04-26T21:19:16.491182+0000","last_became_peered":"2024-04-26T21:19:16.491182+0000","last_unstale":"2024-04-26T21:19:17.538722+0000","last_undegraded":"2024-04-26T21:19:17.538722+0000","last_fullsized":"2024-04-26T21:19:17.538722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492120+0000","last_change":"2024-04-26T21:19:15.483533+0000","last_active":"2024-04-26T21:19:17.492120+0000","last_peered":"2024-04-26T21:19:17.492120+0000","last_clean":"2024-04-26T21:19:17.492120+0000","last_became_active":"2024-04-26T21:19:15.483244+0000","last_became_peered":"2024-04-26T21:19:15.483244+0000","last_unstale":"2024-04-26T21:19:17.492120+0000","last_undegraded":"2024-04-26T21:19:17.492120+0000","last_fullsized":"2024-04-26T21:19:17.492120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:44.801378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502307+0000","last_change":"2024-04-26T21:19:16.499234+0000","last_active":"2024-04-26T21:19:16.502307+0000","last_peered":"2024-04-26T21:19:16.502307+0000","last_clean":"2024-04-26T21:19:16.502307+0000","last_became_active":"2024-04-26T21:19:16.498965+0000","last_became_peered":"2024-04-26T21:19:16.498965+0000","last_unstale":"2024-04-26T21:19:16.502307+0000","last_undegraded":"2024-04-26T21:19:16.502307+0000","last_fullsized":"2024-04-26T21:19:16.502307+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:38.866044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672911+0000","last_change":"2024-04-26T21:19:15.485012+0000","last_active":"2024-04-26T21:19:17.672911+0000","last_peered":"2024-04-26T21:19:17.672911+0000","last_clean":"2024-04-26T21:19:17.672911+0000","last_became_active":"2024-04-26T21:19:15.484711+0000","last_became_peered":"2024-04-26T21:19:15.484711+0000","last_unstale":"2024-04-26T21:19:17.672911+0000","last_undegraded":"2024-04-26T21:19:17.672911+0000","last_fullsized":"2024-04-26T21:19:17.672911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:58.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501835+0000","last_change":"2024-04-26T21:19:16.498702+0000","last_active":"2024-04-26T21:19:16.501835+0000","last_peered":"2024-04-26T21:19:16.501835+0000","last_clean":"2024-04-26T21:19:16.501835+0000","last_became_active":"2024-04-26T21:19:16.498408+0000","last_became_peered":"2024-04-26T21:19:16.498408+0000","last_unstale":"2024-04-26T21:19:16.501835+0000","last_undegraded":"2024-04-26T21:19:16.501835+0000","last_fullsized":"2024-04-26T21:19:16.501835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491957+0000","last_change":"2024-04-26T21:19:15.491055+0000","last_active":"2024-04-26T21:19:17.491957+0000","last_peered":"2024-04-26T21:19:17.491957+0000","last_clean":"2024-04-26T21:19:17.491957+0000","last_became_active":"2024-04-26T21:19:15.490783+0000","last_became_peered":"2024-04-26T21:19:15.490783+0000","last_unstale":"2024-04-26T21:19:17.491957+0000","last_undegraded":"2024-04-26T21:19:17.491957+0000","last_fullsized":"2024-04-26T21:19:17.491957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:05.607960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503155+0000","last_change":"2024-04-26T21:19:16.501447+0000","last_active":"2024-04-26T21:19:16.503155+0000","last_peered":"2024-04-26T21:19:16.503155+0000","last_clean":"2024-04-26T21:19:16.503155+0000","last_became_active":"2024-04-26T21:19:16.501149+0000","last_became_peered":"2024-04-26T21:19:16.501149+0000","last_unstale":"2024-04-26T21:19:16.503155+0000","last_undegraded":"2024-04-26T21:19:16.503155+0000","last_fullsized":"2024-04-26T21:19:16.503155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:20.124617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479420+0000","last_change":"2024-04-26T21:19:15.490438+0000","last_active":"2024-04-26T21:19:16.479420+0000","last_peered":"2024-04-26T21:19:16.479420+0000","last_clean":"2024-04-26T21:19:16.479420+0000","last_became_active":"2024-04-26T21:19:15.490210+0000","last_became_peered":"2024-04-26T21:19:15.490210+0000","last_unstale":"2024-04-26T21:19:16.479420+0000","last_undegraded":"2024-04-26T21:19:16.479420+0000","last_fullsized":"2024-04-26T21:19:16.479420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.422988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672795+0000","last_change":"2024-04-26T21:19:16.491479+0000","last_active":"2024-04-26T21:19:17.672795+0000","last_peered":"2024-04-26T21:19:17.672795+0000","last_clean":"2024-04-26T21:19:17.672795+0000","last_became_active":"2024-04-26T21:19:16.491224+0000","last_became_peered":"2024-04-26T21:19:16.491224+0000","last_unstale":"2024-04-26T21:19:17.672795+0000","last_undegraded":"2024-04-26T21:19:17.672795+0000","last_fullsized":"2024-04-26T21:19:17.672795+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:51.573831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478367+0000","last_change":"2024-04-26T21:19:15.488327+0000","last_active":"2024-04-26T21:19:16.478367+0000","last_peered":"2024-04-26T21:19:16.478367+0000","last_clean":"2024-04-26T21:19:16.478367+0000","last_became_active":"2024-04-26T21:19:15.488039+0000","last_became_peered":"2024-04-26T21:19:15.488039+0000","last_unstale":"2024-04-26T21:19:16.478367+0000","last_undegraded":"2024-04-26T21:19:16.478367+0000","last_fullsized":"2024-04-26T21:19:16.478367+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:08.686002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673153+0000","last_change":"2024-04-26T21:19:16.481276+0000","last_active":"2024-04-26T21:19:17.673153+0000","last_peered":"2024-04-26T21:19:17.673153+0000","last_clean":"2024-04-26T21:19:17.673153+0000","last_became_active":"2024-04-26T21:19:16.480986+0000","last_became_peered":"2024-04-26T21:19:16.480986+0000","last_unstale":"2024-04-26T21:19:17.673153+0000","last_undegraded":"2024-04-26T21:19:17.673153+0000","last_fullsized":"2024-04-26T21:19:17.673153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492469+0000","last_change":"2024-04-26T21:19:15.484302+0000","last_active":"2024-04-26T21:19:17.492469+0000","last_peered":"2024-04-26T21:19:17.492469+0000","last_clean":"2024-04-26T21:19:17.492469+0000","last_became_active":"2024-04-26T21:19:15.483990+0000","last_became_peered":"2024-04-26T21:19:15.483990+0000","last_unstale":"2024-04-26T21:19:17.492469+0000","last_undegraded":"2024-04-26T21:19:17.492469+0000","last_fullsized":"2024-04-26T21:19:17.492469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:02.063693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492042+0000","last_change":"2024-04-26T21:19:16.489119+0000","last_active":"2024-04-26T21:19:16.492042+0000","last_peered":"2024-04-26T21:19:16.492042+0000","last_clean":"2024-04-26T21:19:16.492042+0000","last_became_active":"2024-04-26T21:19:16.488816+0000","last_became_peered":"2024-04-26T21:19:16.488816+0000","last_unstale":"2024-04-26T21:19:16.492042+0000","last_undegraded":"2024-04-26T21:19:16.492042+0000","last_fullsized":"2024-04-26T21:19:16.492042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:04.053313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491477+0000","last_change":"2024-04-26T21:19:15.492213+0000","last_active":"2024-04-26T21:19:17.491477+0000","last_peered":"2024-04-26T21:19:17.491477+0000","last_clean":"2024-04-26T21:19:17.491477+0000","last_became_active":"2024-04-26T21:19:15.491792+0000","last_became_peered":"2024-04-26T21:19:15.491792+0000","last_unstale":"2024-04-26T21:19:17.491477+0000","last_undegraded":"2024-04-26T21:19:17.491477+0000","last_fullsized":"2024-04-26T21:19:17.491477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491775+0000","last_change":"2024-04-26T21:19:16.496675+0000","last_active":"2024-04-26T21:19:17.491775+0000","last_peered":"2024-04-26T21:19:17.491775+0000","last_clean":"2024-04-26T21:19:17.491775+0000","last_became_active":"2024-04-26T21:19:16.496382+0000","last_became_peered":"2024-04-26T21:19:16.496382+0000","last_unstale":"2024-04-26T21:19:17.491775+0000","last_undegraded":"2024-04-26T21:19:17.491775+0000","last_fullsized":"2024-04-26T21:19:17.491775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502058+0000","last_change":"2024-04-26T21:19:15.493890+0000","last_active":"2024-04-26T21:19:16.502058+0000","last_peered":"2024-04-26T21:19:16.502058+0000","last_clean":"2024-04-26T21:19:16.502058+0000","last_became_active":"2024-04-26T21:19:15.493428+0000","last_became_peered":"2024-04-26T21:19:15.493428+0000","last_unstale":"2024-04-26T21:19:16.502058+0000","last_undegraded":"2024-04-26T21:19:16.502058+0000","last_fullsized":"2024-04-26T21:19:16.502058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:44.796649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673102+0000","last_change":"2024-04-26T21:19:16.489093+0000","last_active":"2024-04-26T21:19:17.673102+0000","last_peered":"2024-04-26T21:19:17.673102+0000","last_clean":"2024-04-26T21:19:17.673102+0000","last_became_active":"2024-04-26T21:19:16.488815+0000","last_became_peered":"2024-04-26T21:19:16.488815+0000","last_unstale":"2024-04-26T21:19:17.673102+0000","last_undegraded":"2024-04-26T21:19:17.673102+0000","last_fullsized":"2024-04-26T21:19:17.673102+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:43.157084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.504928+0000","last_change":"2024-04-26T21:19:15.489147+0000","last_active":"2024-04-26T21:19:16.504928+0000","last_peered":"2024-04-26T21:19:16.504928+0000","last_clean":"2024-04-26T21:19:16.504928+0000","last_became_active":"2024-04-26T21:19:15.488782+0000","last_became_peered":"2024-04-26T21:19:15.488782+0000","last_unstale":"2024-04-26T21:19:16.504928+0000","last_undegraded":"2024-04-26T21:19:16.504928+0000","last_fullsized":"2024-04-26T21:19:16.504928+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:12.490510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491886+0000","last_change":"2024-04-26T21:19:16.494938+0000","last_active":"2024-04-26T21:19:17.491886+0000","last_peered":"2024-04-26T21:19:17.491886+0000","last_clean":"2024-04-26T21:19:17.491886+0000","last_became_active":"2024-04-26T21:19:16.494586+0000","last_became_peered":"2024-04-26T21:19:16.494586+0000","last_unstale":"2024-04-26T21:19:17.491886+0000","last_undegraded":"2024-04-26T21:19:17.491886+0000","last_fullsized":"2024-04-26T21:19:17.491886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.466438+0000","last_change":"2024-04-26T21:19:14.466438+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.466438+0000","last_undegraded":"2024-04-26T21:19:14.466438+0000","last_fullsized":"2024-04-26T21:19:14.466438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491584+0000","last_change":"2024-04-26T21:19:16.484940+0000","last_active":"2024-04-26T21:19:17.491584+0000","last_peered":"2024-04-26T21:19:17.491584+0000","last_clean":"2024-04-26T21:19:17.491584+0000","last_became_active":"2024-04-26T21:19:16.484650+0000","last_became_peered":"2024-04-26T21:19:16.484650+0000","last_unstale":"2024-04-26T21:19:17.491584+0000","last_undegraded":"2024-04-26T21:19:17.491584+0000","last_fullsized":"2024-04-26T21:19:17.491584+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:09.688206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479957+0000","last_change":"2024-04-26T21:19:15.486046+0000","last_active":"2024-04-26T21:19:16.479957+0000","last_peered":"2024-04-26T21:19:16.479957+0000","last_clean":"2024-04-26T21:19:16.479957+0000","last_became_active":"2024-04-26T21:19:15.485776+0000","last_became_peered":"2024-04-26T21:19:15.485776+0000","last_unstale":"2024-04-26T21:19:16.479957+0000","last_undegraded":"2024-04-26T21:19:16.479957+0000","last_fullsized":"2024-04-26T21:19:16.479957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.737449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491404+0000","last_change":"2024-04-26T21:19:16.497306+0000","last_active":"2024-04-26T21:19:17.491404+0000","last_peered":"2024-04-26T21:19:17.491404+0000","last_clean":"2024-04-26T21:19:17.491404+0000","last_became_active":"2024-04-26T21:19:16.497039+0000","last_became_peered":"2024-04-26T21:19:16.497039+0000","last_unstale":"2024-04-26T21:19:17.491404+0000","last_undegraded":"2024-04-26T21:19:17.491404+0000","last_fullsized":"2024-04-26T21:19:17.491404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:16.896268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492215+0000","last_change":"2024-04-26T21:19:15.490142+0000","last_active":"2024-04-26T21:19:17.492215+0000","last_peered":"2024-04-26T21:19:17.492215+0000","last_clean":"2024-04-26T21:19:17.492215+0000","last_became_active":"2024-04-26T21:19:15.489841+0000","last_became_peered":"2024-04-26T21:19:15.489841+0000","last_unstale":"2024-04-26T21:19:17.492215+0000","last_undegraded":"2024-04-26T21:19:17.492215+0000","last_fullsized":"2024-04-26T21:19:17.492215+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:06.856169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492590+0000","last_change":"2024-04-26T21:19:16.485067+0000","last_active":"2024-04-26T21:19:17.492590+0000","last_peered":"2024-04-26T21:19:17.492590+0000","last_clean":"2024-04-26T21:19:17.492590+0000","last_became_active":"2024-04-26T21:19:16.484588+0000","last_became_peered":"2024-04-26T21:19:16.484588+0000","last_unstale":"2024-04-26T21:19:17.492590+0000","last_undegraded":"2024-04-26T21:19:17.492590+0000","last_fullsized":"2024-04-26T21:19:17.492590+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:06.657912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496599+0000","last_change":"2024-04-26T21:19:15.488200+0000","last_active":"2024-04-26T21:19:16.496599+0000","last_peered":"2024-04-26T21:19:16.496599+0000","last_clean":"2024-04-26T21:19:16.496599+0000","last_became_active":"2024-04-26T21:19:15.487840+0000","last_became_peered":"2024-04-26T21:19:15.487840+0000","last_unstale":"2024-04-26T21:19:16.496599+0000","last_undegraded":"2024-04-26T21:19:16.496599+0000","last_fullsized":"2024-04-26T21:19:16.496599+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:32.520127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492522+0000","last_change":"2024-04-26T21:19:15.482948+0000","last_active":"2024-04-26T21:19:17.492522+0000","last_peered":"2024-04-26T21:19:17.492522+0000","last_clean":"2024-04-26T21:19:17.492522+0000","last_became_active":"2024-04-26T21:19:15.482636+0000","last_became_peered":"2024-04-26T21:19:15.482636+0000","last_unstale":"2024-04-26T21:19:17.492522+0000","last_undegraded":"2024-04-26T21:19:17.492522+0000","last_fullsized":"2024-04-26T21:19:17.492522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:04.356393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538613+0000","last_change":"2024-04-26T21:19:16.492696+0000","last_active":"2024-04-26T21:19:17.538613+0000","last_peered":"2024-04-26T21:19:17.538613+0000","last_clean":"2024-04-26T21:19:17.538613+0000","last_became_active":"2024-04-26T21:19:16.492396+0000","last_became_peered":"2024-04-26T21:19:16.492396+0000","last_unstale":"2024-04-26T21:19:17.538613+0000","last_undegraded":"2024-04-26T21:19:17.538613+0000","last_fullsized":"2024-04-26T21:19:17.538613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498743+0000","last_change":"2024-04-26T21:19:15.488486+0000","last_active":"2024-04-26T21:19:16.498743+0000","last_peered":"2024-04-26T21:19:16.498743+0000","last_clean":"2024-04-26T21:19:16.498743+0000","last_became_active":"2024-04-26T21:19:15.488221+0000","last_became_peered":"2024-04-26T21:19:15.488221+0000","last_unstale":"2024-04-26T21:19:16.498743+0000","last_undegraded":"2024-04-26T21:19:16.498743+0000","last_fullsized":"2024-04-26T21:19:16.498743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:30.760409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538790+0000","last_change":"2024-04-26T21:19:16.492285+0000","last_active":"2024-04-26T21:19:17.538790+0000","last_peered":"2024-04-26T21:19:17.538790+0000","last_clean":"2024-04-26T21:19:17.538790+0000","last_became_active":"2024-04-26T21:19:16.491992+0000","last_became_peered":"2024-04-26T21:19:16.491992+0000","last_unstale":"2024-04-26T21:19:17.538790+0000","last_undegraded":"2024-04-26T21:19:17.538790+0000","last_fullsized":"2024-04-26T21:19:17.538790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672950+0000","last_change":"2024-04-26T21:19:15.486458+0000","last_active":"2024-04-26T21:19:17.672950+0000","last_peered":"2024-04-26T21:19:17.672950+0000","last_clean":"2024-04-26T21:19:17.672950+0000","last_became_active":"2024-04-26T21:19:15.486127+0000","last_became_peered":"2024-04-26T21:19:15.486127+0000","last_unstale":"2024-04-26T21:19:17.672950+0000","last_undegraded":"2024-04-26T21:19:17.672950+0000","last_fullsized":"2024-04-26T21:19:17.672950+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:05.272243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538381+0000","last_change":"2024-04-26T21:19:16.495963+0000","last_active":"2024-04-26T21:19:17.538381+0000","last_peered":"2024-04-26T21:19:17.538381+0000","last_clean":"2024-04-26T21:19:17.538381+0000","last_became_active":"2024-04-26T21:19:16.495645+0000","last_became_peered":"2024-04-26T21:19:16.495645+0000","last_unstale":"2024-04-26T21:19:17.538381+0000","last_undegraded":"2024-04-26T21:19:17.538381+0000","last_fullsized":"2024-04-26T21:19:17.538381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.465226+0000","last_change":"2024-04-26T21:19:14.465226+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.465226+0000","last_undegraded":"2024-04-26T21:19:14.465226+0000","last_fullsized":"2024-04-26T21:19:14.465226+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491410+0000","last_change":"2024-04-26T21:19:16.487812+0000","last_active":"2024-04-26T21:19:16.491410+0000","last_peered":"2024-04-26T21:19:16.491410+0000","last_clean":"2024-04-26T21:19:16.491410+0000","last_became_active":"2024-04-26T21:19:16.487449+0000","last_became_peered":"2024-04-26T21:19:16.487449+0000","last_unstale":"2024-04-26T21:19:16.491410+0000","last_undegraded":"2024-04-26T21:19:16.491410+0000","last_fullsized":"2024-04-26T21:19:16.491410+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492009+0000","last_change":"2024-04-26T21:19:15.489410+0000","last_active":"2024-04-26T21:19:17.492009+0000","last_peered":"2024-04-26T21:19:17.492009+0000","last_clean":"2024-04-26T21:19:17.492009+0000","last_became_active":"2024-04-26T21:19:15.489083+0000","last_became_peered":"2024-04-26T21:19:15.489083+0000","last_unstale":"2024-04-26T21:19:17.492009+0000","last_undegraded":"2024-04-26T21:19:17.492009+0000","last_fullsized":"2024-04-26T21:19:17.492009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:50.451081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538376+0000","last_change":"2024-04-26T21:19:16.486905+0000","last_active":"2024-04-26T21:19:17.538376+0000","last_peered":"2024-04-26T21:19:17.538376+0000","last_clean":"2024-04-26T21:19:17.538376+0000","last_became_active":"2024-04-26T21:19:16.486575+0000","last_became_peered":"2024-04-26T21:19:16.486575+0000","last_unstale":"2024-04-26T21:19:17.538376+0000","last_undegraded":"2024-04-26T21:19:17.538376+0000","last_fullsized":"2024-04-26T21:19:17.538376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.465173+0000","last_change":"2024-04-26T21:19:14.465173+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.465173+0000","last_undegraded":"2024-04-26T21:19:14.465173+0000","last_fullsized":"2024-04-26T21:19:14.465173+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491418+0000","last_change":"2024-04-26T21:19:16.483829+0000","last_active":"2024-04-26T21:19:17.491418+0000","last_peered":"2024-04-26T21:19:17.491418+0000","last_clean":"2024-04-26T21:19:17.491418+0000","last_became_active":"2024-04-26T21:19:16.483486+0000","last_became_peered":"2024-04-26T21:19:16.483486+0000","last_unstale":"2024-04-26T21:19:17.491418+0000","last_undegraded":"2024-04-26T21:19:17.491418+0000","last_fullsized":"2024-04-26T21:19:17.491418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:34.802721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478470+0000","last_change":"2024-04-26T21:19:15.487580+0000","last_active":"2024-04-26T21:19:16.478470+0000","last_peered":"2024-04-26T21:19:16.478470+0000","last_clean":"2024-04-26T21:19:16.478470+0000","last_became_active":"2024-04-26T21:19:15.487307+0000","last_became_peered":"2024-04-26T21:19:15.487307+0000","last_unstale":"2024-04-26T21:19:16.478470+0000","last_undegraded":"2024-04-26T21:19:16.478470+0000","last_fullsized":"2024-04-26T21:19:16.478470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.926428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492041+0000","last_change":"2024-04-26T21:19:16.487487+0000","last_active":"2024-04-26T21:19:16.492041+0000","last_peered":"2024-04-26T21:19:16.492041+0000","last_clean":"2024-04-26T21:19:16.492041+0000","last_became_active":"2024-04-26T21:19:16.487194+0000","last_became_peered":"2024-04-26T21:19:16.487194+0000","last_unstale":"2024-04-26T21:19:16.492041+0000","last_undegraded":"2024-04-26T21:19:16.492041+0000","last_fullsized":"2024-04-26T21:19:16.492041+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491590+0000","last_change":"2024-04-26T21:19:15.490106+0000","last_active":"2024-04-26T21:19:17.491590+0000","last_peered":"2024-04-26T21:19:17.491590+0000","last_clean":"2024-04-26T21:19:17.491590+0000","last_became_active":"2024-04-26T21:19:15.489789+0000","last_became_peered":"2024-04-26T21:19:15.489789+0000","last_unstale":"2024-04-26T21:19:17.491590+0000","last_undegraded":"2024-04-26T21:19:17.491590+0000","last_fullsized":"2024-04-26T21:19:17.491590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.506686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498668+0000","last_change":"2024-04-26T21:19:16.496139+0000","last_active":"2024-04-26T21:19:16.498668+0000","last_peered":"2024-04-26T21:19:16.498668+0000","last_clean":"2024-04-26T21:19:16.498668+0000","last_became_active":"2024-04-26T21:19:16.495785+0000","last_became_peered":"2024-04-26T21:19:16.495785+0000","last_unstale":"2024-04-26T21:19:16.498668+0000","last_undegraded":"2024-04-26T21:19:16.498668+0000","last_fullsized":"2024-04-26T21:19:16.498668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:47.719083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.477196+0000","last_change":"2024-04-26T21:19:15.476421+0000","last_active":"2024-04-26T21:19:17.477196+0000","last_peered":"2024-04-26T21:19:17.477196+0000","last_clean":"2024-04-26T21:19:17.477196+0000","last_became_active":"2024-04-26T21:19:15.476081+0000","last_became_peered":"2024-04-26T21:19:15.476081+0000","last_unstale":"2024-04-26T21:19:17.477196+0000","last_undegraded":"2024-04-26T21:19:17.477196+0000","last_fullsized":"2024-04-26T21:19:17.477196+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502994+0000","last_change":"2024-04-26T21:19:16.502148+0000","last_active":"2024-04-26T21:19:16.502994+0000","last_peered":"2024-04-26T21:19:16.502994+0000","last_clean":"2024-04-26T21:19:16.502994+0000","last_became_active":"2024-04-26T21:19:16.500040+0000","last_became_peered":"2024-04-26T21:19:16.500040+0000","last_unstale":"2024-04-26T21:19:16.502994+0000","last_undegraded":"2024-04-26T21:19:16.502994+0000","last_fullsized":"2024-04-26T21:19:16.502994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:54.448349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479257+0000","last_change":"2024-04-26T21:19:15.481531+0000","last_active":"2024-04-26T21:19:16.479257+0000","last_peered":"2024-04-26T21:19:16.479257+0000","last_clean":"2024-04-26T21:19:16.479257+0000","last_became_active":"2024-04-26T21:19:15.481231+0000","last_became_peered":"2024-04-26T21:19:15.481231+0000","last_unstale":"2024-04-26T21:19:16.479257+0000","last_undegraded":"2024-04-26T21:19:16.479257+0000","last_fullsized":"2024-04-26T21:19:16.479257+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:04.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492095+0000","last_change":"2024-04-26T21:19:16.488965+0000","last_active":"2024-04-26T21:19:16.492095+0000","last_peered":"2024-04-26T21:19:16.492095+0000","last_clean":"2024-04-26T21:19:16.492095+0000","last_became_active":"2024-04-26T21:19:16.488655+0000","last_became_peered":"2024-04-26T21:19:16.488655+0000","last_unstale":"2024-04-26T21:19:16.492095+0000","last_undegraded":"2024-04-26T21:19:16.492095+0000","last_fullsized":"2024-04-26T21:19:16.492095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491897+0000","last_change":"2024-04-26T21:19:15.493244+0000","last_active":"2024-04-26T21:19:17.491897+0000","last_peered":"2024-04-26T21:19:17.491897+0000","last_clean":"2024-04-26T21:19:17.491897+0000","last_became_active":"2024-04-26T21:19:15.493019+0000","last_became_peered":"2024-04-26T21:19:15.493019+0000","last_unstale":"2024-04-26T21:19:17.491897+0000","last_undegraded":"2024-04-26T21:19:17.491897+0000","last_fullsized":"2024-04-26T21:19:17.491897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:15.360405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496729+0000","last_change":"2024-04-26T21:19:16.493217+0000","last_active":"2024-04-26T21:19:16.496729+0000","last_peered":"2024-04-26T21:19:16.496729+0000","last_clean":"2024-04-26T21:19:16.496729+0000","last_became_active":"2024-04-26T21:19:16.492937+0000","last_became_peered":"2024-04-26T21:19:16.492937+0000","last_unstale":"2024-04-26T21:19:16.496729+0000","last_undegraded":"2024-04-26T21:19:16.496729+0000","last_fullsized":"2024-04-26T21:19:16.496729+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:31.762614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673072+0000","last_change":"2024-04-26T21:19:15.482604+0000","last_active":"2024-04-26T21:19:17.673072+0000","last_peered":"2024-04-26T21:19:17.673072+0000","last_clean":"2024-04-26T21:19:17.673072+0000","last_became_active":"2024-04-26T21:19:15.482320+0000","last_became_peered":"2024-04-26T21:19:15.482320+0000","last_unstale":"2024-04-26T21:19:17.673072+0000","last_undegraded":"2024-04-26T21:19:17.673072+0000","last_fullsized":"2024-04-26T21:19:17.673072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:24.748462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496666+0000","last_change":"2024-04-26T21:19:16.493348+0000","last_active":"2024-04-26T21:19:16.496666+0000","last_peered":"2024-04-26T21:19:16.496666+0000","last_clean":"2024-04-26T21:19:16.496666+0000","last_became_active":"2024-04-26T21:19:16.492941+0000","last_became_peered":"2024-04-26T21:19:16.492941+0000","last_unstale":"2024-04-26T21:19:16.496666+0000","last_undegraded":"2024-04-26T21:19:16.496666+0000","last_fullsized":"2024-04-26T21:19:16.496666+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.739653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479467+0000","last_change":"2024-04-26T21:19:15.489228+0000","last_active":"2024-04-26T21:19:16.479467+0000","last_peered":"2024-04-26T21:19:16.479467+0000","last_clean":"2024-04-26T21:19:16.479467+0000","last_became_active":"2024-04-26T21:19:15.488843+0000","last_became_peered":"2024-04-26T21:19:15.488843+0000","last_unstale":"2024-04-26T21:19:16.479467+0000","last_undegraded":"2024-04-26T21:19:16.479467+0000","last_fullsized":"2024-04-26T21:19:16.479467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:42.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538707+0000","last_change":"2024-04-26T21:19:16.491546+0000","last_active":"2024-04-26T21:19:17.538707+0000","last_peered":"2024-04-26T21:19:17.538707+0000","last_clean":"2024-04-26T21:19:17.538707+0000","last_became_active":"2024-04-26T21:19:16.491305+0000","last_became_peered":"2024-04-26T21:19:16.491305+0000","last_unstale":"2024-04-26T21:19:17.538707+0000","last_undegraded":"2024-04-26T21:19:17.538707+0000","last_fullsized":"2024-04-26T21:19:17.538707+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492104+0000","last_change":"2024-04-26T21:19:15.490249+0000","last_active":"2024-04-26T21:19:17.492104+0000","last_peered":"2024-04-26T21:19:17.492104+0000","last_clean":"2024-04-26T21:19:17.492104+0000","last_became_active":"2024-04-26T21:19:15.489962+0000","last_became_peered":"2024-04-26T21:19:15.489962+0000","last_unstale":"2024-04-26T21:19:17.492104+0000","last_undegraded":"2024-04-26T21:19:17.492104+0000","last_fullsized":"2024-04-26T21:19:17.492104+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503309+0000","last_change":"2024-04-26T21:19:16.501636+0000","last_active":"2024-04-26T21:19:16.503309+0000","last_peered":"2024-04-26T21:19:16.503309+0000","last_clean":"2024-04-26T21:19:16.503309+0000","last_became_active":"2024-04-26T21:19:16.501326+0000","last_became_peered":"2024-04-26T21:19:16.501326+0000","last_unstale":"2024-04-26T21:19:16.503309+0000","last_undegraded":"2024-04-26T21:19:16.503309+0000","last_fullsized":"2024-04-26T21:19:16.503309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:10.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478523+0000","last_change":"2024-04-26T21:19:15.486884+0000","last_active":"2024-04-26T21:19:16.478523+0000","last_peered":"2024-04-26T21:19:16.478523+0000","last_clean":"2024-04-26T21:19:16.478523+0000","last_became_active":"2024-04-26T21:19:15.486597+0000","last_became_peered":"2024-04-26T21:19:15.486597+0000","last_unstale":"2024-04-26T21:19:16.478523+0000","last_undegraded":"2024-04-26T21:19:16.478523+0000","last_fullsized":"2024-04-26T21:19:16.478523+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:16.558118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491321+0000","last_change":"2024-04-26T21:19:16.485658+0000","last_active":"2024-04-26T21:19:17.491321+0000","last_peered":"2024-04-26T21:19:17.491321+0000","last_clean":"2024-04-26T21:19:17.491321+0000","last_became_active":"2024-04-26T21:19:16.485382+0000","last_became_peered":"2024-04-26T21:19:16.485382+0000","last_unstale":"2024-04-26T21:19:17.491321+0000","last_undegraded":"2024-04-26T21:19:17.491321+0000","last_fullsized":"2024-04-26T21:19:17.491321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672961+0000","last_change":"2024-04-26T21:19:15.480711+0000","last_active":"2024-04-26T21:19:17.672961+0000","last_peered":"2024-04-26T21:19:17.672961+0000","last_clean":"2024-04-26T21:19:17.672961+0000","last_became_active":"2024-04-26T21:19:15.480405+0000","last_became_peered":"2024-04-26T21:19:15.480405+0000","last_unstale":"2024-04-26T21:19:17.672961+0000","last_undegraded":"2024-04-26T21:19:17.672961+0000","last_fullsized":"2024-04-26T21:19:17.672961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:00.640043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538749+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:19:17.538749+0000","last_peered":"2024-04-26T21:19:17.538749+0000","last_clean":"2024-04-26T21:19:17.538749+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:19:17.538749+0000","last_undegraded":"2024-04-26T21:19:17.538749+0000","last_fullsized":"2024-04-26T21:19:17.538749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509534,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":1161038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":570124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":576958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":570054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":571736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1162146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1310720,"data_stored":557004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:19:20.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:19:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.644+0000 7feae542a640 1 Processor -- start 2024-04-26T21:19:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.644+0000 7feae542a640 1 -- start start 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feae542a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feae0160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feae542a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae00b4260 con 0x7feae00b61a0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feae542a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae00b4530 con 0x7feae00b3de0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feae542a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feae00b5140 con 0x7feae0145980 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feadf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feae00b61a0 0x7feae00720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55362/0 (socket says 172.21.15.40:55362) 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feadf7fe640 1 -- 172.21.15.40:0/3673766711 learned_addr learned my addr 172.21.15.40:0/3673766711 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feadeffd640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feae0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319731899 0 0) 0x7feae00b4260 con 0x7feae00b61a0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feacc003520 con 0x7feae00b61a0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496242588 0 0) 0x7feae00b4530 con 0x7feae00b3de0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae00b4260 con 0x7feae00b3de0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1975124745 0 0) 0x7feacc003520 con 0x7feae00b61a0 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 msgr2=0x7feae0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feae0160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 >> v1:172.21.15.184:6789/0 conn(0x7feae00b3de0 legacy=0x7feae00b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae00b4810 con 0x7feae00b61a0 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fead8001a10 con 0x7feae00b61a0 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feadeffd640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feae0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.645+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 >> v1:172.21.15.40:6789/0 conn(0x7feae00b61a0 legacy=0x7feae00720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feacc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feacc0040b0 0x7feacc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc004d40 0x7feacc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feae00b5140 con 0x7feacc0040b0 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feacc00bb00 con 0x7feacc004d40 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feacc00bd00 con 0x7feae0145980 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feacc0040b0 0x7feacc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feadeffd640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feacc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7feadf7fe640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc004d40 0x7feacc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 -- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 msgr2=0x7feacc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae0145980 0x7feacc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 -- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc004d40 msgr2=0x7feacc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc004d40 0x7feacc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.646+0000 7fead67fc640 1 -- 172.21.15.40:0/3673766711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc003520 con 0x7feacc0040b0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7fead67fc640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feacc0040b0 0x7feacc004800 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7feac80018d0 tx=0x7feacc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a95dbbc3ecb39a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac80014f0 con 0x7feacc0040b0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac80052d0 con 0x7feacc0040b0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feade7fc640 1 -- 172.21.15.40:0/3673766711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac802aca0 con 0x7feacc0040b0 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 -- 172.21.15.40:0/3673766711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feacc0040b0 msgr2=0x7feacc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 --2- 172.21.15.40:0/3673766711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feacc0040b0 0x7feacc004800 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7feac80018d0 tx=0x7feacc005830 comp rx=0 tx=0).stop 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feadf7fe640 1 -- 172.21.15.40:0/3673766711 reap_dead start 2024-04-26T21:19:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 -- 172.21.15.40:0/3673766711 shutdown_connections 2024-04-26T21:19:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 -- 172.21.15.40:0/3673766711 >> 172.21.15.40:0/3673766711 conn(0x7feae00d0d30 msgr2=0x7feae00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 -- 172.21.15.40:0/3673766711 shutdown_connections 2024-04-26T21:19:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.647+0000 7feae542a640 1 -- 172.21.15.40:0/3673766711 wait complete. 2024-04-26T21:19:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.649+0000 7feae542a640 1 Processor -- start 2024-04-26T21:19:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.649+0000 7feae542a640 1 -- start start 2024-04-26T21:19:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc0040b0 0x7feae013cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae013b180 0x7feae013d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feae0160630 con 0x7feae013ad00 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feae00b4dd0 con 0x7feacc0040b0 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feae01607b0 con 0x7feae013b180 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57162/0 (socket says 172.21.15.40:57162) 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 -- 172.21.15.40:0/145441676 learned_addr learned my addr 172.21.15.40:0/145441676 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feadeffd640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc0040b0 0x7feae013cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feadf7fe640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae013b180 0x7feae013d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 -- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae013b180 msgr2=0x7feae013d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae013b180 0x7feae013d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 -- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc0040b0 msgr2=0x7feae013cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc0040b0 0x7feae013cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 -- 172.21.15.40:0/145441676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae013baf0 con 0x7feae013ad00 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead67fc640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7feac80018d0 tx=0x7feac80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac803f020 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac8006e50 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae013bdd0 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8037040 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.650+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae013c390 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.651+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae006a290 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.652+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feac8042020 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.652+0000 7fead7fff640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 0x7feac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.653+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7feac80ab730 con 0x7feae013ad00 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.653+0000 7feadeffd640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 0x7feac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.653+0000 7feadeffd640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 0x7feac006edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ad0 tx=0x7fead0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:20.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.657+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feac8049030 con 0x7feae013ad00 2024-04-26T21:19:20.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.839+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feae0073cc0 con 0x7feac006c8f0 2024-04-26T21:19:20.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.845+0000 7fead7fff640 1 -- 172.21.15.40:0/145441676 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251120 (secure 0 0 0) 0x7feae0073cc0 con 0x7feac006c8f0 2024-04-26T21:19:20.849 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:19:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 msgr2=0x7feac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 0x7feac006edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ad0 tx=0x7fead0009190 comp rx=0 tx=0).stop 2024-04-26T21:19:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 msgr2=0x7feae00b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7feac80018d0 tx=0x7feac80012d0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 shutdown_connections 2024-04-26T21:19:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feac006c8f0 0x7feac006edb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feae013b180 0x7feae013d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae013ad00 0x7feae00b14f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 --2- 172.21.15.40:0/145441676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feacc0040b0 0x7feae013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:20.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 >> 172.21.15.40:0/145441676 conn(0x7feae00d0d30 msgr2=0x7feae015f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:20.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 shutdown_connections 2024-04-26T21:19:20.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:20.848+0000 7feae542a640 1 -- 172.21.15.40:0/145441676 wait complete. 2024-04-26T21:19:20.861 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-26T21:19:19.024771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265288,"kb_used_data":11648,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709432,"statfs":{"total":773094113280,"available":772822458368,"internally_reserved":0,"allocated":11927552,"data_stored":5745414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.560991"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498644+0000","last_change":"2024-04-26T21:19:16.496097+0000","last_active":"2024-04-26T21:19:16.498644+0000","last_peered":"2024-04-26T21:19:16.498644+0000","last_clean":"2024-04-26T21:19:16.498644+0000","last_became_active":"2024-04-26T21:19:16.494023+0000","last_became_peered":"2024-04-26T21:19:16.494023+0000","last_unstale":"2024-04-26T21:19:16.498644+0000","last_undegraded":"2024-04-26T21:19:16.498644+0000","last_fullsized":"2024-04-26T21:19:16.498644+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:34.681056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501847+0000","last_change":"2024-04-26T21:19:16.498688+0000","last_active":"2024-04-26T21:19:16.501847+0000","last_peered":"2024-04-26T21:19:16.501847+0000","last_clean":"2024-04-26T21:19:16.501847+0000","last_became_active":"2024-04-26T21:19:16.498359+0000","last_became_peered":"2024-04-26T21:19:16.498359+0000","last_unstale":"2024-04-26T21:19:16.501847+0000","last_undegraded":"2024-04-26T21:19:16.501847+0000","last_fullsized":"2024-04-26T21:19:16.501847+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538503+0000","last_change":"2024-04-26T21:19:16.496173+0000","last_active":"2024-04-26T21:19:17.538503+0000","last_peered":"2024-04-26T21:19:17.538503+0000","last_clean":"2024-04-26T21:19:17.538503+0000","last_became_active":"2024-04-26T21:19:16.495826+0000","last_became_peered":"2024-04-26T21:19:16.495826+0000","last_unstale":"2024-04-26T21:19:17.538503+0000","last_undegraded":"2024-04-26T21:19:17.538503+0000","last_fullsized":"2024-04-26T21:19:17.538503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:13.492714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498530+0000","last_change":"2024-04-26T21:19:16.495999+0000","last_active":"2024-04-26T21:19:16.498530+0000","last_peered":"2024-04-26T21:19:16.498530+0000","last_clean":"2024-04-26T21:19:16.498530+0000","last_became_active":"2024-04-26T21:19:16.495621+0000","last_became_peered":"2024-04-26T21:19:16.495621+0000","last_unstale":"2024-04-26T21:19:16.498530+0000","last_undegraded":"2024-04-26T21:19:16.498530+0000","last_fullsized":"2024-04-26T21:19:16.498530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492356+0000","last_change":"2024-04-26T21:19:16.482865+0000","last_active":"2024-04-26T21:19:17.492356+0000","last_peered":"2024-04-26T21:19:17.492356+0000","last_clean":"2024-04-26T21:19:17.492356+0000","last_became_active":"2024-04-26T21:19:16.482588+0000","last_became_peered":"2024-04-26T21:19:16.482588+0000","last_unstale":"2024-04-26T21:19:17.492356+0000","last_undegraded":"2024-04-26T21:19:17.492356+0000","last_fullsized":"2024-04-26T21:19:17.492356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:23.086160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672936+0000","last_change":"2024-04-26T21:19:16.488443+0000","last_active":"2024-04-26T21:19:17.672936+0000","last_peered":"2024-04-26T21:19:17.672936+0000","last_clean":"2024-04-26T21:19:17.672936+0000","last_became_active":"2024-04-26T21:19:16.488149+0000","last_became_peered":"2024-04-26T21:19:16.488149+0000","last_unstale":"2024-04-26T21:19:17.672936+0000","last_undegraded":"2024-04-26T21:19:17.672936+0000","last_fullsized":"2024-04-26T21:19:17.672936+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538580+0000","last_change":"2024-04-26T21:19:16.498705+0000","last_active":"2024-04-26T21:19:17.538580+0000","last_peered":"2024-04-26T21:19:17.538580+0000","last_clean":"2024-04-26T21:19:17.538580+0000","last_became_active":"2024-04-26T21:19:16.498371+0000","last_became_peered":"2024-04-26T21:19:16.498371+0000","last_unstale":"2024-04-26T21:19:17.538580+0000","last_undegraded":"2024-04-26T21:19:17.538580+0000","last_fullsized":"2024-04-26T21:19:17.538580+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:33.522331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496250+0000","last_change":"2024-04-26T21:19:16.489425+0000","last_active":"2024-04-26T21:19:16.496250+0000","last_peered":"2024-04-26T21:19:16.496250+0000","last_clean":"2024-04-26T21:19:16.496250+0000","last_became_active":"2024-04-26T21:19:16.489130+0000","last_became_peered":"2024-04-26T21:19:16.489130+0000","last_unstale":"2024-04-26T21:19:16.496250+0000","last_undegraded":"2024-04-26T21:19:16.496250+0000","last_fullsized":"2024-04-26T21:19:16.496250+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488375+0000","last_change":"2024-04-26T21:19:16.485878+0000","last_active":"2024-04-26T21:19:16.488375+0000","last_peered":"2024-04-26T21:19:16.488375+0000","last_clean":"2024-04-26T21:19:16.488375+0000","last_became_active":"2024-04-26T21:19:16.485592+0000","last_became_peered":"2024-04-26T21:19:16.485592+0000","last_unstale":"2024-04-26T21:19:16.488375+0000","last_undegraded":"2024-04-26T21:19:16.488375+0000","last_fullsized":"2024-04-26T21:19:16.488375+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496673+0000","last_change":"2024-04-26T21:19:16.492337+0000","last_active":"2024-04-26T21:19:16.496673+0000","last_peered":"2024-04-26T21:19:16.496673+0000","last_clean":"2024-04-26T21:19:16.496673+0000","last_became_active":"2024-04-26T21:19:16.492009+0000","last_became_peered":"2024-04-26T21:19:16.492009+0000","last_unstale":"2024-04-26T21:19:16.496673+0000","last_undegraded":"2024-04-26T21:19:16.496673+0000","last_fullsized":"2024-04-26T21:19:16.496673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:37.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488890+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:16.488890+0000","last_peered":"2024-04-26T21:19:16.488890+0000","last_clean":"2024-04-26T21:19:16.488890+0000","last_became_active":"2024-04-26T21:19:16.484183+0000","last_became_peered":"2024-04-26T21:19:16.484183+0000","last_unstale":"2024-04-26T21:19:16.488890+0000","last_undegraded":"2024-04-26T21:19:16.488890+0000","last_fullsized":"2024-04-26T21:19:16.488890+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538678+0000","last_change":"2024-04-26T21:19:16.495059+0000","last_active":"2024-04-26T21:19:17.538678+0000","last_peered":"2024-04-26T21:19:17.538678+0000","last_clean":"2024-04-26T21:19:17.538678+0000","last_became_active":"2024-04-26T21:19:16.494780+0000","last_became_peered":"2024-04-26T21:19:16.494780+0000","last_unstale":"2024-04-26T21:19:17.538678+0000","last_undegraded":"2024-04-26T21:19:17.538678+0000","last_fullsized":"2024-04-26T21:19:17.538678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538257+0000","last_change":"2024-04-26T21:19:16.490955+0000","last_active":"2024-04-26T21:19:17.538257+0000","last_peered":"2024-04-26T21:19:17.538257+0000","last_clean":"2024-04-26T21:19:17.538257+0000","last_became_active":"2024-04-26T21:19:16.490657+0000","last_became_peered":"2024-04-26T21:19:16.490657+0000","last_unstale":"2024-04-26T21:19:17.538257+0000","last_undegraded":"2024-04-26T21:19:17.538257+0000","last_fullsized":"2024-04-26T21:19:17.538257+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491077+0000","last_change":"2024-04-26T21:19:16.487462+0000","last_active":"2024-04-26T21:19:16.491077+0000","last_peered":"2024-04-26T21:19:16.491077+0000","last_clean":"2024-04-26T21:19:16.491077+0000","last_became_active":"2024-04-26T21:19:16.486946+0000","last_became_peered":"2024-04-26T21:19:16.486946+0000","last_unstale":"2024-04-26T21:19:16.491077+0000","last_undegraded":"2024-04-26T21:19:16.491077+0000","last_fullsized":"2024-04-26T21:19:16.491077+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496577+0000","last_change":"2024-04-26T21:19:16.491720+0000","last_active":"2024-04-26T21:19:16.496577+0000","last_peered":"2024-04-26T21:19:16.496577+0000","last_clean":"2024-04-26T21:19:16.496577+0000","last_became_active":"2024-04-26T21:19:16.491369+0000","last_became_peered":"2024-04-26T21:19:16.491369+0000","last_unstale":"2024-04-26T21:19:16.496577+0000","last_undegraded":"2024-04-26T21:19:16.496577+0000","last_fullsized":"2024-04-26T21:19:16.496577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:18.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499278+0000","last_change":"2024-04-26T21:19:16.496447+0000","last_active":"2024-04-26T21:19:16.499278+0000","last_peered":"2024-04-26T21:19:16.499278+0000","last_clean":"2024-04-26T21:19:16.499278+0000","last_became_active":"2024-04-26T21:19:16.496204+0000","last_became_peered":"2024-04-26T21:19:16.496204+0000","last_unstale":"2024-04-26T21:19:16.499278+0000","last_undegraded":"2024-04-26T21:19:16.499278+0000","last_fullsized":"2024-04-26T21:19:16.499278+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:50.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538653+0000","last_change":"2024-04-26T21:19:16.491438+0000","last_active":"2024-04-26T21:19:17.538653+0000","last_peered":"2024-04-26T21:19:17.538653+0000","last_clean":"2024-04-26T21:19:17.538653+0000","last_became_active":"2024-04-26T21:19:16.490909+0000","last_became_peered":"2024-04-26T21:19:16.490909+0000","last_unstale":"2024-04-26T21:19:17.538653+0000","last_undegraded":"2024-04-26T21:19:17.538653+0000","last_fullsized":"2024-04-26T21:19:17.538653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496630+0000","last_change":"2024-04-26T21:19:16.491473+0000","last_active":"2024-04-26T21:19:16.496630+0000","last_peered":"2024-04-26T21:19:16.496630+0000","last_clean":"2024-04-26T21:19:16.496630+0000","last_became_active":"2024-04-26T21:19:16.491070+0000","last_became_peered":"2024-04-26T21:19:16.491070+0000","last_unstale":"2024-04-26T21:19:16.496630+0000","last_undegraded":"2024-04-26T21:19:16.496630+0000","last_fullsized":"2024-04-26T21:19:16.496630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:44.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501985+0000","last_change":"2024-04-26T21:19:16.499382+0000","last_active":"2024-04-26T21:19:16.501985+0000","last_peered":"2024-04-26T21:19:16.501985+0000","last_clean":"2024-04-26T21:19:16.501985+0000","last_became_active":"2024-04-26T21:19:16.499064+0000","last_became_peered":"2024-04-26T21:19:16.499064+0000","last_unstale":"2024-04-26T21:19:16.501985+0000","last_undegraded":"2024-04-26T21:19:16.501985+0000","last_fullsized":"2024-04-26T21:19:16.501985+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491852+0000","last_change":"2024-04-26T21:19:16.485418+0000","last_active":"2024-04-26T21:19:17.491852+0000","last_peered":"2024-04-26T21:19:17.491852+0000","last_clean":"2024-04-26T21:19:17.491852+0000","last_became_active":"2024-04-26T21:19:16.484504+0000","last_became_peered":"2024-04-26T21:19:16.484504+0000","last_unstale":"2024-04-26T21:19:17.491852+0000","last_undegraded":"2024-04-26T21:19:17.491852+0000","last_fullsized":"2024-04-26T21:19:17.491852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492897+0000","last_change":"2024-04-26T21:19:16.487920+0000","last_active":"2024-04-26T21:19:16.492897+0000","last_peered":"2024-04-26T21:19:16.492897+0000","last_clean":"2024-04-26T21:19:16.492897+0000","last_became_active":"2024-04-26T21:19:16.487548+0000","last_became_peered":"2024-04-26T21:19:16.487548+0000","last_unstale":"2024-04-26T21:19:16.492897+0000","last_undegraded":"2024-04-26T21:19:16.492897+0000","last_fullsized":"2024-04-26T21:19:16.492897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491267+0000","last_change":"2024-04-26T21:19:16.485140+0000","last_active":"2024-04-26T21:19:17.491267+0000","last_peered":"2024-04-26T21:19:17.491267+0000","last_clean":"2024-04-26T21:19:17.491267+0000","last_became_active":"2024-04-26T21:19:16.484871+0000","last_became_peered":"2024-04-26T21:19:16.484871+0000","last_unstale":"2024-04-26T21:19:17.491267+0000","last_undegraded":"2024-04-26T21:19:17.491267+0000","last_fullsized":"2024-04-26T21:19:17.491267+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.494135+0000","last_change":"2024-04-26T21:19:16.491074+0000","last_active":"2024-04-26T21:19:16.494135+0000","last_peered":"2024-04-26T21:19:16.494135+0000","last_clean":"2024-04-26T21:19:16.494135+0000","last_became_active":"2024-04-26T21:19:16.490719+0000","last_became_peered":"2024-04-26T21:19:16.490719+0000","last_unstale":"2024-04-26T21:19:16.494135+0000","last_undegraded":"2024-04-26T21:19:16.494135+0000","last_fullsized":"2024-04-26T21:19:16.494135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:07.415274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499547+0000","last_change":"2024-04-26T21:19:16.496876+0000","last_active":"2024-04-26T21:19:16.499547+0000","last_peered":"2024-04-26T21:19:16.499547+0000","last_clean":"2024-04-26T21:19:16.499547+0000","last_became_active":"2024-04-26T21:19:16.496601+0000","last_became_peered":"2024-04-26T21:19:16.496601+0000","last_unstale":"2024-04-26T21:19:16.499547+0000","last_undegraded":"2024-04-26T21:19:16.499547+0000","last_fullsized":"2024-04-26T21:19:16.499547+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:31.534507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491768+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:17.491768+0000","last_peered":"2024-04-26T21:19:17.491768+0000","last_clean":"2024-04-26T21:19:17.491768+0000","last_became_active":"2024-04-26T21:19:16.484220+0000","last_became_peered":"2024-04-26T21:19:16.484220+0000","last_unstale":"2024-04-26T21:19:17.491768+0000","last_undegraded":"2024-04-26T21:19:17.491768+0000","last_fullsized":"2024-04-26T21:19:17.491768+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:25.928632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499556+0000","last_change":"2024-04-26T21:19:16.494852+0000","last_active":"2024-04-26T21:19:16.499556+0000","last_peered":"2024-04-26T21:19:16.499556+0000","last_clean":"2024-04-26T21:19:16.499556+0000","last_became_active":"2024-04-26T21:19:16.494590+0000","last_became_peered":"2024-04-26T21:19:16.494590+0000","last_unstale":"2024-04-26T21:19:16.499556+0000","last_undegraded":"2024-04-26T21:19:16.499556+0000","last_fullsized":"2024-04-26T21:19:16.499556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:15.415456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673032+0000","last_change":"2024-04-26T21:19:16.484845+0000","last_active":"2024-04-26T21:19:17.673032+0000","last_peered":"2024-04-26T21:19:17.673032+0000","last_clean":"2024-04-26T21:19:17.673032+0000","last_became_active":"2024-04-26T21:19:16.484525+0000","last_became_peered":"2024-04-26T21:19:16.484525+0000","last_unstale":"2024-04-26T21:19:17.673032+0000","last_undegraded":"2024-04-26T21:19:17.673032+0000","last_fullsized":"2024-04-26T21:19:17.673032+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:31.425192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499512+0000","last_change":"2024-04-26T21:19:16.497005+0000","last_active":"2024-04-26T21:19:16.499512+0000","last_peered":"2024-04-26T21:19:16.499512+0000","last_clean":"2024-04-26T21:19:16.499512+0000","last_became_active":"2024-04-26T21:19:16.496669+0000","last_became_peered":"2024-04-26T21:19:16.496669+0000","last_unstale":"2024-04-26T21:19:16.499512+0000","last_undegraded":"2024-04-26T21:19:16.499512+0000","last_fullsized":"2024-04-26T21:19:16.499512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.580462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673108+0000","last_change":"2024-04-26T21:19:15.478535+0000","last_active":"2024-04-26T21:19:17.673108+0000","last_peered":"2024-04-26T21:19:17.673108+0000","last_clean":"2024-04-26T21:19:17.673108+0000","last_became_active":"2024-04-26T21:19:15.478272+0000","last_became_peered":"2024-04-26T21:19:15.478272+0000","last_unstale":"2024-04-26T21:19:17.673108+0000","last_undegraded":"2024-04-26T21:19:17.673108+0000","last_fullsized":"2024-04-26T21:19:17.673108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492017+0000","last_change":"2024-04-26T21:19:16.488880+0000","last_active":"2024-04-26T21:19:16.492017+0000","last_peered":"2024-04-26T21:19:16.492017+0000","last_clean":"2024-04-26T21:19:16.492017+0000","last_became_active":"2024-04-26T21:19:16.488578+0000","last_became_peered":"2024-04-26T21:19:16.488578+0000","last_unstale":"2024-04-26T21:19:16.492017+0000","last_undegraded":"2024-04-26T21:19:16.492017+0000","last_fullsized":"2024-04-26T21:19:16.492017+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538442+0000","last_change":"2024-04-26T21:19:15.481825+0000","last_active":"2024-04-26T21:19:17.538442+0000","last_peered":"2024-04-26T21:19:17.538442+0000","last_clean":"2024-04-26T21:19:17.538442+0000","last_became_active":"2024-04-26T21:19:15.481529+0000","last_became_peered":"2024-04-26T21:19:15.481529+0000","last_unstale":"2024-04-26T21:19:17.538442+0000","last_undegraded":"2024-04-26T21:19:17.538442+0000","last_fullsized":"2024-04-26T21:19:17.538442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492403+0000","last_change":"2024-04-26T21:19:15.485103+0000","last_active":"2024-04-26T21:19:17.492403+0000","last_peered":"2024-04-26T21:19:17.492403+0000","last_clean":"2024-04-26T21:19:17.492403+0000","last_became_active":"2024-04-26T21:19:15.484770+0000","last_became_peered":"2024-04-26T21:19:15.484770+0000","last_unstale":"2024-04-26T21:19:17.492403+0000","last_undegraded":"2024-04-26T21:19:17.492403+0000","last_fullsized":"2024-04-26T21:19:17.492403+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:15.463835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538722+0000","last_change":"2024-04-26T21:19:16.491699+0000","last_active":"2024-04-26T21:19:17.538722+0000","last_peered":"2024-04-26T21:19:17.538722+0000","last_clean":"2024-04-26T21:19:17.538722+0000","last_became_active":"2024-04-26T21:19:16.491182+0000","last_became_peered":"2024-04-26T21:19:16.491182+0000","last_unstale":"2024-04-26T21:19:17.538722+0000","last_undegraded":"2024-04-26T21:19:17.538722+0000","last_fullsized":"2024-04-26T21:19:17.538722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492120+0000","last_change":"2024-04-26T21:19:15.483533+0000","last_active":"2024-04-26T21:19:17.492120+0000","last_peered":"2024-04-26T21:19:17.492120+0000","last_clean":"2024-04-26T21:19:17.492120+0000","last_became_active":"2024-04-26T21:19:15.483244+0000","last_became_peered":"2024-04-26T21:19:15.483244+0000","last_unstale":"2024-04-26T21:19:17.492120+0000","last_undegraded":"2024-04-26T21:19:17.492120+0000","last_fullsized":"2024-04-26T21:19:17.492120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:44.801378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502307+0000","last_change":"2024-04-26T21:19:16.499234+0000","last_active":"2024-04-26T21:19:16.502307+0000","last_peered":"2024-04-26T21:19:16.502307+0000","last_clean":"2024-04-26T21:19:16.502307+0000","last_became_active":"2024-04-26T21:19:16.498965+0000","last_became_peered":"2024-04-26T21:19:16.498965+0000","last_unstale":"2024-04-26T21:19:16.502307+0000","last_undegraded":"2024-04-26T21:19:16.502307+0000","last_fullsized":"2024-04-26T21:19:16.502307+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:38.866044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672911+0000","last_change":"2024-04-26T21:19:15.485012+0000","last_active":"2024-04-26T21:19:17.672911+0000","last_peered":"2024-04-26T21:19:17.672911+0000","last_clean":"2024-04-26T21:19:17.672911+0000","last_became_active":"2024-04-26T21:19:15.484711+0000","last_became_peered":"2024-04-26T21:19:15.484711+0000","last_unstale":"2024-04-26T21:19:17.672911+0000","last_undegraded":"2024-04-26T21:19:17.672911+0000","last_fullsized":"2024-04-26T21:19:17.672911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:58.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501835+0000","last_change":"2024-04-26T21:19:16.498702+0000","last_active":"2024-04-26T21:19:16.501835+0000","last_peered":"2024-04-26T21:19:16.501835+0000","last_clean":"2024-04-26T21:19:16.501835+0000","last_became_active":"2024-04-26T21:19:16.498408+0000","last_became_peered":"2024-04-26T21:19:16.498408+0000","last_unstale":"2024-04-26T21:19:16.501835+0000","last_undegraded":"2024-04-26T21:19:16.501835+0000","last_fullsized":"2024-04-26T21:19:16.501835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491957+0000","last_change":"2024-04-26T21:19:15.491055+0000","last_active":"2024-04-26T21:19:17.491957+0000","last_peered":"2024-04-26T21:19:17.491957+0000","last_clean":"2024-04-26T21:19:17.491957+0000","last_became_active":"2024-04-26T21:19:15.490783+0000","last_became_peered":"2024-04-26T21:19:15.490783+0000","last_unstale":"2024-04-26T21:19:17.491957+0000","last_undegraded":"2024-04-26T21:19:17.491957+0000","last_fullsized":"2024-04-26T21:19:17.491957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:05.607960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503155+0000","last_change":"2024-04-26T21:19:16.501447+0000","last_active":"2024-04-26T21:19:16.503155+0000","last_peered":"2024-04-26T21:19:16.503155+0000","last_clean":"2024-04-26T21:19:16.503155+0000","last_became_active":"2024-04-26T21:19:16.501149+0000","last_became_peered":"2024-04-26T21:19:16.501149+0000","last_unstale":"2024-04-26T21:19:16.503155+0000","last_undegraded":"2024-04-26T21:19:16.503155+0000","last_fullsized":"2024-04-26T21:19:16.503155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:20.124617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479420+0000","last_change":"2024-04-26T21:19:15.490438+0000","last_active":"2024-04-26T21:19:16.479420+0000","last_peered":"2024-04-26T21:19:16.479420+0000","last_clean":"2024-04-26T21:19:16.479420+0000","last_became_active":"2024-04-26T21:19:15.490210+0000","last_became_peered":"2024-04-26T21:19:15.490210+0000","last_unstale":"2024-04-26T21:19:16.479420+0000","last_undegraded":"2024-04-26T21:19:16.479420+0000","last_fullsized":"2024-04-26T21:19:16.479420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.422988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672795+0000","last_change":"2024-04-26T21:19:16.491479+0000","last_active":"2024-04-26T21:19:17.672795+0000","last_peered":"2024-04-26T21:19:17.672795+0000","last_clean":"2024-04-26T21:19:17.672795+0000","last_became_active":"2024-04-26T21:19:16.491224+0000","last_became_peered":"2024-04-26T21:19:16.491224+0000","last_unstale":"2024-04-26T21:19:17.672795+0000","last_undegraded":"2024-04-26T21:19:17.672795+0000","last_fullsized":"2024-04-26T21:19:17.672795+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:51.573831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478367+0000","last_change":"2024-04-26T21:19:15.488327+0000","last_active":"2024-04-26T21:19:16.478367+0000","last_peered":"2024-04-26T21:19:16.478367+0000","last_clean":"2024-04-26T21:19:16.478367+0000","last_became_active":"2024-04-26T21:19:15.488039+0000","last_became_peered":"2024-04-26T21:19:15.488039+0000","last_unstale":"2024-04-26T21:19:16.478367+0000","last_undegraded":"2024-04-26T21:19:16.478367+0000","last_fullsized":"2024-04-26T21:19:16.478367+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:08.686002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673153+0000","last_change":"2024-04-26T21:19:16.481276+0000","last_active":"2024-04-26T21:19:17.673153+0000","last_peered":"2024-04-26T21:19:17.673153+0000","last_clean":"2024-04-26T21:19:17.673153+0000","last_became_active":"2024-04-26T21:19:16.480986+0000","last_became_peered":"2024-04-26T21:19:16.480986+0000","last_unstale":"2024-04-26T21:19:17.673153+0000","last_undegraded":"2024-04-26T21:19:17.673153+0000","last_fullsized":"2024-04-26T21:19:17.673153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492469+0000","last_change":"2024-04-26T21:19:15.484302+0000","last_active":"2024-04-26T21:19:17.492469+0000","last_peered":"2024-04-26T21:19:17.492469+0000","last_clean":"2024-04-26T21:19:17.492469+0000","last_became_active":"2024-04-26T21:19:15.483990+0000","last_became_peered":"2024-04-26T21:19:15.483990+0000","last_unstale":"2024-04-26T21:19:17.492469+0000","last_undegraded":"2024-04-26T21:19:17.492469+0000","last_fullsized":"2024-04-26T21:19:17.492469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:02.063693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492042+0000","last_change":"2024-04-26T21:19:16.489119+0000","last_active":"2024-04-26T21:19:16.492042+0000","last_peered":"2024-04-26T21:19:16.492042+0000","last_clean":"2024-04-26T21:19:16.492042+0000","last_became_active":"2024-04-26T21:19:16.488816+0000","last_became_peered":"2024-04-26T21:19:16.488816+0000","last_unstale":"2024-04-26T21:19:16.492042+0000","last_undegraded":"2024-04-26T21:19:16.492042+0000","last_fullsized":"2024-04-26T21:19:16.492042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:04.053313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491477+0000","last_change":"2024-04-26T21:19:15.492213+0000","last_active":"2024-04-26T21:19:17.491477+0000","last_peered":"2024-04-26T21:19:17.491477+0000","last_clean":"2024-04-26T21:19:17.491477+0000","last_became_active":"2024-04-26T21:19:15.491792+0000","last_became_peered":"2024-04-26T21:19:15.491792+0000","last_unstale":"2024-04-26T21:19:17.491477+0000","last_undegraded":"2024-04-26T21:19:17.491477+0000","last_fullsized":"2024-04-26T21:19:17.491477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491775+0000","last_change":"2024-04-26T21:19:16.496675+0000","last_active":"2024-04-26T21:19:17.491775+0000","last_peered":"2024-04-26T21:19:17.491775+0000","last_clean":"2024-04-26T21:19:17.491775+0000","last_became_active":"2024-04-26T21:19:16.496382+0000","last_became_peered":"2024-04-26T21:19:16.496382+0000","last_unstale":"2024-04-26T21:19:17.491775+0000","last_undegraded":"2024-04-26T21:19:17.491775+0000","last_fullsized":"2024-04-26T21:19:17.491775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502058+0000","last_change":"2024-04-26T21:19:15.493890+0000","last_active":"2024-04-26T21:19:16.502058+0000","last_peered":"2024-04-26T21:19:16.502058+0000","last_clean":"2024-04-26T21:19:16.502058+0000","last_became_active":"2024-04-26T21:19:15.493428+0000","last_became_peered":"2024-04-26T21:19:15.493428+0000","last_unstale":"2024-04-26T21:19:16.502058+0000","last_undegraded":"2024-04-26T21:19:16.502058+0000","last_fullsized":"2024-04-26T21:19:16.502058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:44.796649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673102+0000","last_change":"2024-04-26T21:19:16.489093+0000","last_active":"2024-04-26T21:19:17.673102+0000","last_peered":"2024-04-26T21:19:17.673102+0000","last_clean":"2024-04-26T21:19:17.673102+0000","last_became_active":"2024-04-26T21:19:16.488815+0000","last_became_peered":"2024-04-26T21:19:16.488815+0000","last_unstale":"2024-04-26T21:19:17.673102+0000","last_undegraded":"2024-04-26T21:19:17.673102+0000","last_fullsized":"2024-04-26T21:19:17.673102+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:43.157084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.504928+0000","last_change":"2024-04-26T21:19:15.489147+0000","last_active":"2024-04-26T21:19:16.504928+0000","last_peered":"2024-04-26T21:19:16.504928+0000","last_clean":"2024-04-26T21:19:16.504928+0000","last_became_active":"2024-04-26T21:19:15.488782+0000","last_became_peered":"2024-04-26T21:19:15.488782+0000","last_unstale":"2024-04-26T21:19:16.504928+0000","last_undegraded":"2024-04-26T21:19:16.504928+0000","last_fullsized":"2024-04-26T21:19:16.504928+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:12.490510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491886+0000","last_change":"2024-04-26T21:19:16.494938+0000","last_active":"2024-04-26T21:19:17.491886+0000","last_peered":"2024-04-26T21:19:17.491886+0000","last_clean":"2024-04-26T21:19:17.491886+0000","last_became_active":"2024-04-26T21:19:16.494586+0000","last_became_peered":"2024-04-26T21:19:16.494586+0000","last_unstale":"2024-04-26T21:19:17.491886+0000","last_undegraded":"2024-04-26T21:19:17.491886+0000","last_fullsized":"2024-04-26T21:19:17.491886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.466438+0000","last_change":"2024-04-26T21:19:14.466438+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.466438+0000","last_undegraded":"2024-04-26T21:19:14.466438+0000","last_fullsized":"2024-04-26T21:19:14.466438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491584+0000","last_change":"2024-04-26T21:19:16.484940+0000","last_active":"2024-04-26T21:19:17.491584+0000","last_peered":"2024-04-26T21:19:17.491584+0000","last_clean":"2024-04-26T21:19:17.491584+0000","last_became_active":"2024-04-26T21:19:16.484650+0000","last_became_peered":"2024-04-26T21:19:16.484650+0000","last_unstale":"2024-04-26T21:19:17.491584+0000","last_undegraded":"2024-04-26T21:19:17.491584+0000","last_fullsized":"2024-04-26T21:19:17.491584+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:09.688206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479957+0000","last_change":"2024-04-26T21:19:15.486046+0000","last_active":"2024-04-26T21:19:16.479957+0000","last_peered":"2024-04-26T21:19:16.479957+0000","last_clean":"2024-04-26T21:19:16.479957+0000","last_became_active":"2024-04-26T21:19:15.485776+0000","last_became_peered":"2024-04-26T21:19:15.485776+0000","last_unstale":"2024-04-26T21:19:16.479957+0000","last_undegraded":"2024-04-26T21:19:16.479957+0000","last_fullsized":"2024-04-26T21:19:16.479957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.737449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491404+0000","last_change":"2024-04-26T21:19:16.497306+0000","last_active":"2024-04-26T21:19:17.491404+0000","last_peered":"2024-04-26T21:19:17.491404+0000","last_clean":"2024-04-26T21:19:17.491404+0000","last_became_active":"2024-04-26T21:19:16.497039+0000","last_became_peered":"2024-04-26T21:19:16.497039+0000","last_unstale":"2024-04-26T21:19:17.491404+0000","last_undegraded":"2024-04-26T21:19:17.491404+0000","last_fullsized":"2024-04-26T21:19:17.491404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:16.896268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492215+0000","last_change":"2024-04-26T21:19:15.490142+0000","last_active":"2024-04-26T21:19:17.492215+0000","last_peered":"2024-04-26T21:19:17.492215+0000","last_clean":"2024-04-26T21:19:17.492215+0000","last_became_active":"2024-04-26T21:19:15.489841+0000","last_became_peered":"2024-04-26T21:19:15.489841+0000","last_unstale":"2024-04-26T21:19:17.492215+0000","last_undegraded":"2024-04-26T21:19:17.492215+0000","last_fullsized":"2024-04-26T21:19:17.492215+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:06.856169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492590+0000","last_change":"2024-04-26T21:19:16.485067+0000","last_active":"2024-04-26T21:19:17.492590+0000","last_peered":"2024-04-26T21:19:17.492590+0000","last_clean":"2024-04-26T21:19:17.492590+0000","last_became_active":"2024-04-26T21:19:16.484588+0000","last_became_peered":"2024-04-26T21:19:16.484588+0000","last_unstale":"2024-04-26T21:19:17.492590+0000","last_undegraded":"2024-04-26T21:19:17.492590+0000","last_fullsized":"2024-04-26T21:19:17.492590+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:06.657912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496599+0000","last_change":"2024-04-26T21:19:15.488200+0000","last_active":"2024-04-26T21:19:16.496599+0000","last_peered":"2024-04-26T21:19:16.496599+0000","last_clean":"2024-04-26T21:19:16.496599+0000","last_became_active":"2024-04-26T21:19:15.487840+0000","last_became_peered":"2024-04-26T21:19:15.487840+0000","last_unstale":"2024-04-26T21:19:16.496599+0000","last_undegraded":"2024-04-26T21:19:16.496599+0000","last_fullsized":"2024-04-26T21:19:16.496599+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:32.520127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T21:19:15.463203+0000","last_change":"2024-04-26T21:19:15.463203+0000","last_active":"2024-04-26T21:19:15.463203+0000","last_peered":"2024-04-26T21:19:15.463203+0000","last_clean":"2024-04-26T21:19:15.463203+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:15.463203+0000","last_undegraded":"2024-04-26T21:19:15.463203+0000","last_fullsized":"2024-04-26T21:19:15.463203+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492522+0000","last_change":"2024-04-26T21:19:15.482948+0000","last_active":"2024-04-26T21:19:17.492522+0000","last_peered":"2024-04-26T21:19:17.492522+0000","last_clean":"2024-04-26T21:19:17.492522+0000","last_became_active":"2024-04-26T21:19:15.482636+0000","last_became_peered":"2024-04-26T21:19:15.482636+0000","last_unstale":"2024-04-26T21:19:17.492522+0000","last_undegraded":"2024-04-26T21:19:17.492522+0000","last_fullsized":"2024-04-26T21:19:17.492522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:04.356393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538613+0000","last_change":"2024-04-26T21:19:16.492696+0000","last_active":"2024-04-26T21:19:17.538613+0000","last_peered":"2024-04-26T21:19:17.538613+0000","last_clean":"2024-04-26T21:19:17.538613+0000","last_became_active":"2024-04-26T21:19:16.492396+0000","last_became_peered":"2024-04-26T21:19:16.492396+0000","last_unstale":"2024-04-26T21:19:17.538613+0000","last_undegraded":"2024-04-26T21:19:17.538613+0000","last_fullsized":"2024-04-26T21:19:17.538613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498743+0000","last_change":"2024-04-26T21:19:15.488486+0000","last_active":"2024-04-26T21:19:16.498743+0000","last_peered":"2024-04-26T21:19:16.498743+0000","last_clean":"2024-04-26T21:19:16.498743+0000","last_became_active":"2024-04-26T21:19:15.488221+0000","last_became_peered":"2024-04-26T21:19:15.488221+0000","last_unstale":"2024-04-26T21:19:16.498743+0000","last_undegraded":"2024-04-26T21:19:16.498743+0000","last_fullsized":"2024-04-26T21:19:16.498743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:30.760409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538790+0000","last_change":"2024-04-26T21:19:16.492285+0000","last_active":"2024-04-26T21:19:17.538790+0000","last_peered":"2024-04-26T21:19:17.538790+0000","last_clean":"2024-04-26T21:19:17.538790+0000","last_became_active":"2024-04-26T21:19:16.491992+0000","last_became_peered":"2024-04-26T21:19:16.491992+0000","last_unstale":"2024-04-26T21:19:17.538790+0000","last_undegraded":"2024-04-26T21:19:17.538790+0000","last_fullsized":"2024-04-26T21:19:17.538790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672950+0000","last_change":"2024-04-26T21:19:15.486458+0000","last_active":"2024-04-26T21:19:17.672950+0000","last_peered":"2024-04-26T21:19:17.672950+0000","last_clean":"2024-04-26T21:19:17.672950+0000","last_became_active":"2024-04-26T21:19:15.486127+0000","last_became_peered":"2024-04-26T21:19:15.486127+0000","last_unstale":"2024-04-26T21:19:17.672950+0000","last_undegraded":"2024-04-26T21:19:17.672950+0000","last_fullsized":"2024-04-26T21:19:17.672950+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:05.272243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538381+0000","last_change":"2024-04-26T21:19:16.495963+0000","last_active":"2024-04-26T21:19:17.538381+0000","last_peered":"2024-04-26T21:19:17.538381+0000","last_clean":"2024-04-26T21:19:17.538381+0000","last_became_active":"2024-04-26T21:19:16.495645+0000","last_became_peered":"2024-04-26T21:19:16.495645+0000","last_unstale":"2024-04-26T21:19:17.538381+0000","last_undegraded":"2024-04-26T21:19:17.538381+0000","last_fullsized":"2024-04-26T21:19:17.538381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.465226+0000","last_change":"2024-04-26T21:19:14.465226+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.465226+0000","last_undegraded":"2024-04-26T21:19:14.465226+0000","last_fullsized":"2024-04-26T21:19:14.465226+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491410+0000","last_change":"2024-04-26T21:19:16.487812+0000","last_active":"2024-04-26T21:19:16.491410+0000","last_peered":"2024-04-26T21:19:16.491410+0000","last_clean":"2024-04-26T21:19:16.491410+0000","last_became_active":"2024-04-26T21:19:16.487449+0000","last_became_peered":"2024-04-26T21:19:16.487449+0000","last_unstale":"2024-04-26T21:19:16.491410+0000","last_undegraded":"2024-04-26T21:19:16.491410+0000","last_fullsized":"2024-04-26T21:19:16.491410+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492009+0000","last_change":"2024-04-26T21:19:15.489410+0000","last_active":"2024-04-26T21:19:17.492009+0000","last_peered":"2024-04-26T21:19:17.492009+0000","last_clean":"2024-04-26T21:19:17.492009+0000","last_became_active":"2024-04-26T21:19:15.489083+0000","last_became_peered":"2024-04-26T21:19:15.489083+0000","last_unstale":"2024-04-26T21:19:17.492009+0000","last_undegraded":"2024-04-26T21:19:17.492009+0000","last_fullsized":"2024-04-26T21:19:17.492009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:50.451081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538376+0000","last_change":"2024-04-26T21:19:16.486905+0000","last_active":"2024-04-26T21:19:17.538376+0000","last_peered":"2024-04-26T21:19:17.538376+0000","last_clean":"2024-04-26T21:19:17.538376+0000","last_became_active":"2024-04-26T21:19:16.486575+0000","last_became_peered":"2024-04-26T21:19:16.486575+0000","last_unstale":"2024-04-26T21:19:17.538376+0000","last_undegraded":"2024-04-26T21:19:17.538376+0000","last_fullsized":"2024-04-26T21:19:17.538376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-26T21:19:14.465173+0000","last_change":"2024-04-26T21:19:14.465173+0000","last_active":"2024-04-26T21:19:14.460999+0000","last_peered":"2024-04-26T21:19:14.460999+0000","last_clean":"2024-04-26T21:19:14.460999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:19:14.465173+0000","last_undegraded":"2024-04-26T21:19:14.465173+0000","last_fullsized":"2024-04-26T21:19:14.465173+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491418+0000","last_change":"2024-04-26T21:19:16.483829+0000","last_active":"2024-04-26T21:19:17.491418+0000","last_peered":"2024-04-26T21:19:17.491418+0000","last_clean":"2024-04-26T21:19:17.491418+0000","last_became_active":"2024-04-26T21:19:16.483486+0000","last_became_peered":"2024-04-26T21:19:16.483486+0000","last_unstale":"2024-04-26T21:19:17.491418+0000","last_undegraded":"2024-04-26T21:19:17.491418+0000","last_fullsized":"2024-04-26T21:19:17.491418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:34.802721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478470+0000","last_change":"2024-04-26T21:19:15.487580+0000","last_active":"2024-04-26T21:19:16.478470+0000","last_peered":"2024-04-26T21:19:16.478470+0000","last_clean":"2024-04-26T21:19:16.478470+0000","last_became_active":"2024-04-26T21:19:15.487307+0000","last_became_peered":"2024-04-26T21:19:15.487307+0000","last_unstale":"2024-04-26T21:19:16.478470+0000","last_undegraded":"2024-04-26T21:19:16.478470+0000","last_fullsized":"2024-04-26T21:19:16.478470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.926428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492041+0000","last_change":"2024-04-26T21:19:16.487487+0000","last_active":"2024-04-26T21:19:16.492041+0000","last_peered":"2024-04-26T21:19:16.492041+0000","last_clean":"2024-04-26T21:19:16.492041+0000","last_became_active":"2024-04-26T21:19:16.487194+0000","last_became_peered":"2024-04-26T21:19:16.487194+0000","last_unstale":"2024-04-26T21:19:16.492041+0000","last_undegraded":"2024-04-26T21:19:16.492041+0000","last_fullsized":"2024-04-26T21:19:16.492041+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491590+0000","last_change":"2024-04-26T21:19:15.490106+0000","last_active":"2024-04-26T21:19:17.491590+0000","last_peered":"2024-04-26T21:19:17.491590+0000","last_clean":"2024-04-26T21:19:17.491590+0000","last_became_active":"2024-04-26T21:19:15.489789+0000","last_became_peered":"2024-04-26T21:19:15.489789+0000","last_unstale":"2024-04-26T21:19:17.491590+0000","last_undegraded":"2024-04-26T21:19:17.491590+0000","last_fullsized":"2024-04-26T21:19:17.491590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.506686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498668+0000","last_change":"2024-04-26T21:19:16.496139+0000","last_active":"2024-04-26T21:19:16.498668+0000","last_peered":"2024-04-26T21:19:16.498668+0000","last_clean":"2024-04-26T21:19:16.498668+0000","last_became_active":"2024-04-26T21:19:16.495785+0000","last_became_peered":"2024-04-26T21:19:16.495785+0000","last_unstale":"2024-04-26T21:19:16.498668+0000","last_undegraded":"2024-04-26T21:19:16.498668+0000","last_fullsized":"2024-04-26T21:19:16.498668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:47.719083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.477196+0000","last_change":"2024-04-26T21:19:15.476421+0000","last_active":"2024-04-26T21:19:17.477196+0000","last_peered":"2024-04-26T21:19:17.477196+0000","last_clean":"2024-04-26T21:19:17.477196+0000","last_became_active":"2024-04-26T21:19:15.476081+0000","last_became_peered":"2024-04-26T21:19:15.476081+0000","last_unstale":"2024-04-26T21:19:17.477196+0000","last_undegraded":"2024-04-26T21:19:17.477196+0000","last_fullsized":"2024-04-26T21:19:17.477196+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502994+0000","last_change":"2024-04-26T21:19:16.502148+0000","last_active":"2024-04-26T21:19:16.502994+0000","last_peered":"2024-04-26T21:19:16.502994+0000","last_clean":"2024-04-26T21:19:16.502994+0000","last_became_active":"2024-04-26T21:19:16.500040+0000","last_became_peered":"2024-04-26T21:19:16.500040+0000","last_unstale":"2024-04-26T21:19:16.502994+0000","last_undegraded":"2024-04-26T21:19:16.502994+0000","last_fullsized":"2024-04-26T21:19:16.502994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:54.448349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479257+0000","last_change":"2024-04-26T21:19:15.481531+0000","last_active":"2024-04-26T21:19:16.479257+0000","last_peered":"2024-04-26T21:19:16.479257+0000","last_clean":"2024-04-26T21:19:16.479257+0000","last_became_active":"2024-04-26T21:19:15.481231+0000","last_became_peered":"2024-04-26T21:19:15.481231+0000","last_unstale":"2024-04-26T21:19:16.479257+0000","last_undegraded":"2024-04-26T21:19:16.479257+0000","last_fullsized":"2024-04-26T21:19:16.479257+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:04.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492095+0000","last_change":"2024-04-26T21:19:16.488965+0000","last_active":"2024-04-26T21:19:16.492095+0000","last_peered":"2024-04-26T21:19:16.492095+0000","last_clean":"2024-04-26T21:19:16.492095+0000","last_became_active":"2024-04-26T21:19:16.488655+0000","last_became_peered":"2024-04-26T21:19:16.488655+0000","last_unstale":"2024-04-26T21:19:16.492095+0000","last_undegraded":"2024-04-26T21:19:16.492095+0000","last_fullsized":"2024-04-26T21:19:16.492095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491897+0000","last_change":"2024-04-26T21:19:15.493244+0000","last_active":"2024-04-26T21:19:17.491897+0000","last_peered":"2024-04-26T21:19:17.491897+0000","last_clean":"2024-04-26T21:19:17.491897+0000","last_became_active":"2024-04-26T21:19:15.493019+0000","last_became_peered":"2024-04-26T21:19:15.493019+0000","last_unstale":"2024-04-26T21:19:17.491897+0000","last_undegraded":"2024-04-26T21:19:17.491897+0000","last_fullsized":"2024-04-26T21:19:17.491897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:15.360405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496729+0000","last_change":"2024-04-26T21:19:16.493217+0000","last_active":"2024-04-26T21:19:16.496729+0000","last_peered":"2024-04-26T21:19:16.496729+0000","last_clean":"2024-04-26T21:19:16.496729+0000","last_became_active":"2024-04-26T21:19:16.492937+0000","last_became_peered":"2024-04-26T21:19:16.492937+0000","last_unstale":"2024-04-26T21:19:16.496729+0000","last_undegraded":"2024-04-26T21:19:16.496729+0000","last_fullsized":"2024-04-26T21:19:16.496729+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:31.762614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673072+0000","last_change":"2024-04-26T21:19:15.482604+0000","last_active":"2024-04-26T21:19:17.673072+0000","last_peered":"2024-04-26T21:19:17.673072+0000","last_clean":"2024-04-26T21:19:17.673072+0000","last_became_active":"2024-04-26T21:19:15.482320+0000","last_became_peered":"2024-04-26T21:19:15.482320+0000","last_unstale":"2024-04-26T21:19:17.673072+0000","last_undegraded":"2024-04-26T21:19:17.673072+0000","last_fullsized":"2024-04-26T21:19:17.673072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:24.748462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496666+0000","last_change":"2024-04-26T21:19:16.493348+0000","last_active":"2024-04-26T21:19:16.496666+0000","last_peered":"2024-04-26T21:19:16.496666+0000","last_clean":"2024-04-26T21:19:16.496666+0000","last_became_active":"2024-04-26T21:19:16.492941+0000","last_became_peered":"2024-04-26T21:19:16.492941+0000","last_unstale":"2024-04-26T21:19:16.496666+0000","last_undegraded":"2024-04-26T21:19:16.496666+0000","last_fullsized":"2024-04-26T21:19:16.496666+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.739653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479467+0000","last_change":"2024-04-26T21:19:15.489228+0000","last_active":"2024-04-26T21:19:16.479467+0000","last_peered":"2024-04-26T21:19:16.479467+0000","last_clean":"2024-04-26T21:19:16.479467+0000","last_became_active":"2024-04-26T21:19:15.488843+0000","last_became_peered":"2024-04-26T21:19:15.488843+0000","last_unstale":"2024-04-26T21:19:16.479467+0000","last_undegraded":"2024-04-26T21:19:16.479467+0000","last_fullsized":"2024-04-26T21:19:16.479467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:42.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538707+0000","last_change":"2024-04-26T21:19:16.491546+0000","last_active":"2024-04-26T21:19:17.538707+0000","last_peered":"2024-04-26T21:19:17.538707+0000","last_clean":"2024-04-26T21:19:17.538707+0000","last_became_active":"2024-04-26T21:19:16.491305+0000","last_became_peered":"2024-04-26T21:19:16.491305+0000","last_unstale":"2024-04-26T21:19:17.538707+0000","last_undegraded":"2024-04-26T21:19:17.538707+0000","last_fullsized":"2024-04-26T21:19:17.538707+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492104+0000","last_change":"2024-04-26T21:19:15.490249+0000","last_active":"2024-04-26T21:19:17.492104+0000","last_peered":"2024-04-26T21:19:17.492104+0000","last_clean":"2024-04-26T21:19:17.492104+0000","last_became_active":"2024-04-26T21:19:15.489962+0000","last_became_peered":"2024-04-26T21:19:15.489962+0000","last_unstale":"2024-04-26T21:19:17.492104+0000","last_undegraded":"2024-04-26T21:19:17.492104+0000","last_fullsized":"2024-04-26T21:19:17.492104+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503309+0000","last_change":"2024-04-26T21:19:16.501636+0000","last_active":"2024-04-26T21:19:16.503309+0000","last_peered":"2024-04-26T21:19:16.503309+0000","last_clean":"2024-04-26T21:19:16.503309+0000","last_became_active":"2024-04-26T21:19:16.501326+0000","last_became_peered":"2024-04-26T21:19:16.501326+0000","last_unstale":"2024-04-26T21:19:16.503309+0000","last_undegraded":"2024-04-26T21:19:16.503309+0000","last_fullsized":"2024-04-26T21:19:16.503309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:10.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478523+0000","last_change":"2024-04-26T21:19:15.486884+0000","last_active":"2024-04-26T21:19:16.478523+0000","last_peered":"2024-04-26T21:19:16.478523+0000","last_clean":"2024-04-26T21:19:16.478523+0000","last_became_active":"2024-04-26T21:19:15.486597+0000","last_became_peered":"2024-04-26T21:19:15.486597+0000","last_unstale":"2024-04-26T21:19:16.478523+0000","last_undegraded":"2024-04-26T21:19:16.478523+0000","last_fullsized":"2024-04-26T21:19:16.478523+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:16.558118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491321+0000","last_change":"2024-04-26T21:19:16.485658+0000","last_active":"2024-04-26T21:19:17.491321+0000","last_peered":"2024-04-26T21:19:17.491321+0000","last_clean":"2024-04-26T21:19:17.491321+0000","last_became_active":"2024-04-26T21:19:16.485382+0000","last_became_peered":"2024-04-26T21:19:16.485382+0000","last_unstale":"2024-04-26T21:19:17.491321+0000","last_undegraded":"2024-04-26T21:19:17.491321+0000","last_fullsized":"2024-04-26T21:19:17.491321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672961+0000","last_change":"2024-04-26T21:19:15.480711+0000","last_active":"2024-04-26T21:19:17.672961+0000","last_peered":"2024-04-26T21:19:17.672961+0000","last_clean":"2024-04-26T21:19:17.672961+0000","last_became_active":"2024-04-26T21:19:15.480405+0000","last_became_peered":"2024-04-26T21:19:15.480405+0000","last_unstale":"2024-04-26T21:19:17.672961+0000","last_undegraded":"2024-04-26T21:19:17.672961+0000","last_fullsized":"2024-04-26T21:19:17.672961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:00.640043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538749+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:19:17.538749+0000","last_peered":"2024-04-26T21:19:17.538749+0000","last_clean":"2024-04-26T21:19:17.538749+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:19:17.538749+0000","last_undegraded":"2024-04-26T21:19:17.538749+0000","last_fullsized":"2024-04-26T21:19:17.538749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509534,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":1161038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":570124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":576958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":570054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":571736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1162146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1310720,"data_stored":557004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:19:20.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:19:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.000+0000 7fcc40ff0640 1 Processor -- start 2024-04-26T21:19:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.000+0000 7fcc40ff0640 1 -- start start 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc40ff0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc3c0d3ee0 0x7fcc3c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc40ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc3c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc40ff0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3c0d03a0 con 0x7fcc3c15f370 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc40ff0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c0b4d10 con 0x7fcc3c0d3ee0 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc40ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3c0d0680 con 0x7fcc3c15f750 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc3a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcc3c15f370 0x7fcc3c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55368/0 (socket says 172.21.15.40:55368) 2024-04-26T21:19:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.001+0000 7fcc3a59c640 1 -- 172.21.15.40:0/1889449975 learned_addr learned my addr 172.21.15.40:0/1889449975 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3ad9d640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc3c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc39d9b640 1 --2- 172.21.15.40:0/1889449975 >> v2:172.21.15.184:3300/0 conn(0x7fcc3c0d3ee0 0x7fcc3c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082592645 0 0) 0x7fcc3c0d03a0 con 0x7fcc3c15f370 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc28003520 con 0x7fcc3c15f370 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3605924680 0 0) 0x7fcc28003520 con 0x7fcc3c15f370 2024-04-26T21:19:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 msgr2=0x7fcc3c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc3c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 >> v2:172.21.15.184:3300/0 conn(0x7fcc3c0d3ee0 msgr2=0x7fcc3c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 --2- 172.21.15.40:0/1889449975 >> v2:172.21.15.184:3300/0 conn(0x7fcc3c0d3ee0 0x7fcc3c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c0d0960 con 0x7fcc3c15f370 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fcc34001670 con 0x7fcc3c15f370 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 >> v1:172.21.15.40:6789/0 conn(0x7fcc3c15f370 legacy=0x7fcc3c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc28004d40 0x7fcc280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b4d10 con 0x7fcc280039b0 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0d0680 con 0x7fcc28004d40 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2800b9c0 con 0x7fcc3c15f750 2024-04-26T21:19:21.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc39d9b640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3ad9d640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc28004d40 0x7fcc280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 -- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 msgr2=0x7fcc28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c15f750 0x7fcc28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 -- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc28004d40 msgr2=0x7fcc280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc28004d40 0x7fcc280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.002+0000 7fcc3a59c640 1 -- 172.21.15.40:0/1889449975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28003520 con 0x7fcc280039b0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.003+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc28003dd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c270 tx=0x7fcc28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feba52fafa76d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.003+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34007d30 con 0x7fcc280039b0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.003+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc34005ad0 con 0x7fcc280039b0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.003+0000 7fcc3959a640 1 -- 172.21.15.40:0/1889449975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34007070 con 0x7fcc280039b0 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 -- 172.21.15.40:0/1889449975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 msgr2=0x7fcc28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/1889449975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc28003dd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c270 tx=0x7fcc28005830 comp rx=0 tx=0).stop 2024-04-26T21:19:21.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc3ad9d640 1 -- 172.21.15.40:0/1889449975 reap_dead start 2024-04-26T21:19:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 -- 172.21.15.40:0/1889449975 shutdown_connections 2024-04-26T21:19:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 -- 172.21.15.40:0/1889449975 >> 172.21.15.40:0/1889449975 conn(0x7fcc3c08ec10 msgr2=0x7fcc3c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 -- 172.21.15.40:0/1889449975 shutdown_connections 2024-04-26T21:19:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.004+0000 7fcc40ff0640 1 -- 172.21.15.40:0/1889449975 wait complete. 2024-04-26T21:19:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.005+0000 7fcc40ff0640 1 Processor -- start 2024-04-26T21:19:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.005+0000 7fcc40ff0640 1 -- start start 2024-04-26T21:19:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c0b7df0 0x7fcc3c1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc3c1f60a0 0x7fcc3c0b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b47d0 con 0x7fcc280039b0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0d0f20 con 0x7fcc3c1f60a0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc40ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b4950 con 0x7fcc3c0b7df0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57176/0 (socket says 172.21.15.40:57176) 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 -- 172.21.15.40:0/3955740530 learned_addr learned my addr 172.21.15.40:0/3955740530 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 -- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c0b7df0 msgr2=0x7fcc3c1f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c0b7df0 0x7fcc3c1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 -- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc3c1f60a0 msgr2=0x7fcc3c0b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc3c1f60a0 0x7fcc3c0b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.006+0000 7fcc3a59c640 1 -- 172.21.15.40:0/3955740530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c0b87d0 con 0x7fcc280039b0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc3a59c640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c270 tx=0x7fcc3c0d0b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34042070 con 0x7fcc280039b0 2024-04-26T21:19:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc34007480 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c0b8ab0 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34001c70 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.007+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1f5640 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.008+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c06a290 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.008+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc34045070 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.009+0000 7fcc32ffd640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.009+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fcc340ac820 con 0x7fcc280039b0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.009+0000 7fcc39d9b640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.010+0000 7fcc39d9b640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0b9320 tx=0x7fcc24006c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.013+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc3404c030 con 0x7fcc280039b0 2024-04-26T21:19:21.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.193+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc3c0bac70 con 0x7fcc1c06c8f0 2024-04-26T21:19:21.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.199+0000 7fcc32ffd640 1 -- 172.21.15.40:0/3955740530 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252451 (secure 0 0 0) 0x7fcc3c0bac70 con 0x7fcc1c06c8f0 2024-04-26T21:19:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:21.204 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:19:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 msgr2=0x7fcc1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0b9320 tx=0x7fcc24006c30 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 msgr2=0x7fcc3c0ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c270 tx=0x7fcc3c0d0b30 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 shutdown_connections 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcc1c06c8f0 0x7fcc1c06edb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc3c1f60a0 0x7fcc3c0b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc3c0b7df0 0x7fcc3c1f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 --2- 172.21.15.40:0/3955740530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc280039b0 0x7fcc3c0ba730 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 >> 172.21.15.40:0/3955740530 conn(0x7fcc3c08ec10 msgr2=0x7fcc3c0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 shutdown_connections 2024-04-26T21:19:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.202+0000 7fcc40ff0640 1 -- 172.21.15.40:0/3955740530 wait complete. 2024-04-26T21:19:21.214 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-26T21:19:21.025443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265332,"kb_used_data":11692,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709388,"statfs":{"total":773094113280,"available":772822413312,"internally_reserved":0,"allocated":11972608,"data_stored":5764888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001751"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498644+0000","last_change":"2024-04-26T21:19:16.496097+0000","last_active":"2024-04-26T21:19:16.498644+0000","last_peered":"2024-04-26T21:19:16.498644+0000","last_clean":"2024-04-26T21:19:16.498644+0000","last_became_active":"2024-04-26T21:19:16.494023+0000","last_became_peered":"2024-04-26T21:19:16.494023+0000","last_unstale":"2024-04-26T21:19:16.498644+0000","last_undegraded":"2024-04-26T21:19:16.498644+0000","last_fullsized":"2024-04-26T21:19:16.498644+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:34.681056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501847+0000","last_change":"2024-04-26T21:19:16.498688+0000","last_active":"2024-04-26T21:19:16.501847+0000","last_peered":"2024-04-26T21:19:16.501847+0000","last_clean":"2024-04-26T21:19:16.501847+0000","last_became_active":"2024-04-26T21:19:16.498359+0000","last_became_peered":"2024-04-26T21:19:16.498359+0000","last_unstale":"2024-04-26T21:19:16.501847+0000","last_undegraded":"2024-04-26T21:19:16.501847+0000","last_fullsized":"2024-04-26T21:19:16.501847+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538503+0000","last_change":"2024-04-26T21:19:16.496173+0000","last_active":"2024-04-26T21:19:17.538503+0000","last_peered":"2024-04-26T21:19:17.538503+0000","last_clean":"2024-04-26T21:19:17.538503+0000","last_became_active":"2024-04-26T21:19:16.495826+0000","last_became_peered":"2024-04-26T21:19:16.495826+0000","last_unstale":"2024-04-26T21:19:17.538503+0000","last_undegraded":"2024-04-26T21:19:17.538503+0000","last_fullsized":"2024-04-26T21:19:17.538503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:13.492714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498530+0000","last_change":"2024-04-26T21:19:16.495999+0000","last_active":"2024-04-26T21:19:16.498530+0000","last_peered":"2024-04-26T21:19:16.498530+0000","last_clean":"2024-04-26T21:19:16.498530+0000","last_became_active":"2024-04-26T21:19:16.495621+0000","last_became_peered":"2024-04-26T21:19:16.495621+0000","last_unstale":"2024-04-26T21:19:16.498530+0000","last_undegraded":"2024-04-26T21:19:16.498530+0000","last_fullsized":"2024-04-26T21:19:16.498530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492356+0000","last_change":"2024-04-26T21:19:16.482865+0000","last_active":"2024-04-26T21:19:17.492356+0000","last_peered":"2024-04-26T21:19:17.492356+0000","last_clean":"2024-04-26T21:19:17.492356+0000","last_became_active":"2024-04-26T21:19:16.482588+0000","last_became_peered":"2024-04-26T21:19:16.482588+0000","last_unstale":"2024-04-26T21:19:17.492356+0000","last_undegraded":"2024-04-26T21:19:17.492356+0000","last_fullsized":"2024-04-26T21:19:17.492356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:23.086160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490188+0000","last_change":"2024-04-26T21:19:16.500088+0000","last_active":"2024-04-26T21:19:17.490188+0000","last_peered":"2024-04-26T21:19:17.490188+0000","last_clean":"2024-04-26T21:19:17.490188+0000","last_became_active":"2024-04-26T21:19:16.499783+0000","last_became_peered":"2024-04-26T21:19:16.499783+0000","last_unstale":"2024-04-26T21:19:17.490188+0000","last_undegraded":"2024-04-26T21:19:17.490188+0000","last_fullsized":"2024-04-26T21:19:17.490188+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.343352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672936+0000","last_change":"2024-04-26T21:19:16.488443+0000","last_active":"2024-04-26T21:19:17.672936+0000","last_peered":"2024-04-26T21:19:17.672936+0000","last_clean":"2024-04-26T21:19:17.672936+0000","last_became_active":"2024-04-26T21:19:16.488149+0000","last_became_peered":"2024-04-26T21:19:16.488149+0000","last_unstale":"2024-04-26T21:19:17.672936+0000","last_undegraded":"2024-04-26T21:19:17.672936+0000","last_fullsized":"2024-04-26T21:19:17.672936+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538580+0000","last_change":"2024-04-26T21:19:16.498705+0000","last_active":"2024-04-26T21:19:17.538580+0000","last_peered":"2024-04-26T21:19:17.538580+0000","last_clean":"2024-04-26T21:19:17.538580+0000","last_became_active":"2024-04-26T21:19:16.498371+0000","last_became_peered":"2024-04-26T21:19:16.498371+0000","last_unstale":"2024-04-26T21:19:17.538580+0000","last_undegraded":"2024-04-26T21:19:17.538580+0000","last_fullsized":"2024-04-26T21:19:17.538580+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:33.522331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496250+0000","last_change":"2024-04-26T21:19:16.489425+0000","last_active":"2024-04-26T21:19:16.496250+0000","last_peered":"2024-04-26T21:19:16.496250+0000","last_clean":"2024-04-26T21:19:16.496250+0000","last_became_active":"2024-04-26T21:19:16.489130+0000","last_became_peered":"2024-04-26T21:19:16.489130+0000","last_unstale":"2024-04-26T21:19:16.496250+0000","last_undegraded":"2024-04-26T21:19:16.496250+0000","last_fullsized":"2024-04-26T21:19:16.496250+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488375+0000","last_change":"2024-04-26T21:19:16.485878+0000","last_active":"2024-04-26T21:19:16.488375+0000","last_peered":"2024-04-26T21:19:16.488375+0000","last_clean":"2024-04-26T21:19:16.488375+0000","last_became_active":"2024-04-26T21:19:16.485592+0000","last_became_peered":"2024-04-26T21:19:16.485592+0000","last_unstale":"2024-04-26T21:19:16.488375+0000","last_undegraded":"2024-04-26T21:19:16.488375+0000","last_fullsized":"2024-04-26T21:19:16.488375+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496673+0000","last_change":"2024-04-26T21:19:16.492337+0000","last_active":"2024-04-26T21:19:16.496673+0000","last_peered":"2024-04-26T21:19:16.496673+0000","last_clean":"2024-04-26T21:19:16.496673+0000","last_became_active":"2024-04-26T21:19:16.492009+0000","last_became_peered":"2024-04-26T21:19:16.492009+0000","last_unstale":"2024-04-26T21:19:16.496673+0000","last_undegraded":"2024-04-26T21:19:16.496673+0000","last_fullsized":"2024-04-26T21:19:16.496673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:37.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488890+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:16.488890+0000","last_peered":"2024-04-26T21:19:16.488890+0000","last_clean":"2024-04-26T21:19:16.488890+0000","last_became_active":"2024-04-26T21:19:16.484183+0000","last_became_peered":"2024-04-26T21:19:16.484183+0000","last_unstale":"2024-04-26T21:19:16.488890+0000","last_undegraded":"2024-04-26T21:19:16.488890+0000","last_fullsized":"2024-04-26T21:19:16.488890+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538678+0000","last_change":"2024-04-26T21:19:16.495059+0000","last_active":"2024-04-26T21:19:17.538678+0000","last_peered":"2024-04-26T21:19:17.538678+0000","last_clean":"2024-04-26T21:19:17.538678+0000","last_became_active":"2024-04-26T21:19:16.494780+0000","last_became_peered":"2024-04-26T21:19:16.494780+0000","last_unstale":"2024-04-26T21:19:17.538678+0000","last_undegraded":"2024-04-26T21:19:17.538678+0000","last_fullsized":"2024-04-26T21:19:17.538678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538257+0000","last_change":"2024-04-26T21:19:16.490955+0000","last_active":"2024-04-26T21:19:17.538257+0000","last_peered":"2024-04-26T21:19:17.538257+0000","last_clean":"2024-04-26T21:19:17.538257+0000","last_became_active":"2024-04-26T21:19:16.490657+0000","last_became_peered":"2024-04-26T21:19:16.490657+0000","last_unstale":"2024-04-26T21:19:17.538257+0000","last_undegraded":"2024-04-26T21:19:17.538257+0000","last_fullsized":"2024-04-26T21:19:17.538257+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490182+0000","last_change":"2024-04-26T21:19:16.500703+0000","last_active":"2024-04-26T21:19:17.490182+0000","last_peered":"2024-04-26T21:19:17.490182+0000","last_clean":"2024-04-26T21:19:17.490182+0000","last_became_active":"2024-04-26T21:19:16.500372+0000","last_became_peered":"2024-04-26T21:19:16.500372+0000","last_unstale":"2024-04-26T21:19:17.490182+0000","last_undegraded":"2024-04-26T21:19:17.490182+0000","last_fullsized":"2024-04-26T21:19:17.490182+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:28.187760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491077+0000","last_change":"2024-04-26T21:19:16.487462+0000","last_active":"2024-04-26T21:19:16.491077+0000","last_peered":"2024-04-26T21:19:16.491077+0000","last_clean":"2024-04-26T21:19:16.491077+0000","last_became_active":"2024-04-26T21:19:16.486946+0000","last_became_peered":"2024-04-26T21:19:16.486946+0000","last_unstale":"2024-04-26T21:19:16.491077+0000","last_undegraded":"2024-04-26T21:19:16.491077+0000","last_fullsized":"2024-04-26T21:19:16.491077+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496577+0000","last_change":"2024-04-26T21:19:16.491720+0000","last_active":"2024-04-26T21:19:16.496577+0000","last_peered":"2024-04-26T21:19:16.496577+0000","last_clean":"2024-04-26T21:19:16.496577+0000","last_became_active":"2024-04-26T21:19:16.491369+0000","last_became_peered":"2024-04-26T21:19:16.491369+0000","last_unstale":"2024-04-26T21:19:16.496577+0000","last_undegraded":"2024-04-26T21:19:16.496577+0000","last_fullsized":"2024-04-26T21:19:16.496577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:18.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499278+0000","last_change":"2024-04-26T21:19:16.496447+0000","last_active":"2024-04-26T21:19:16.499278+0000","last_peered":"2024-04-26T21:19:16.499278+0000","last_clean":"2024-04-26T21:19:16.499278+0000","last_became_active":"2024-04-26T21:19:16.496204+0000","last_became_peered":"2024-04-26T21:19:16.496204+0000","last_unstale":"2024-04-26T21:19:16.499278+0000","last_undegraded":"2024-04-26T21:19:16.499278+0000","last_fullsized":"2024-04-26T21:19:16.499278+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:50.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538653+0000","last_change":"2024-04-26T21:19:16.491438+0000","last_active":"2024-04-26T21:19:17.538653+0000","last_peered":"2024-04-26T21:19:17.538653+0000","last_clean":"2024-04-26T21:19:17.538653+0000","last_became_active":"2024-04-26T21:19:16.490909+0000","last_became_peered":"2024-04-26T21:19:16.490909+0000","last_unstale":"2024-04-26T21:19:17.538653+0000","last_undegraded":"2024-04-26T21:19:17.538653+0000","last_fullsized":"2024-04-26T21:19:17.538653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496630+0000","last_change":"2024-04-26T21:19:16.491473+0000","last_active":"2024-04-26T21:19:16.496630+0000","last_peered":"2024-04-26T21:19:16.496630+0000","last_clean":"2024-04-26T21:19:16.496630+0000","last_became_active":"2024-04-26T21:19:16.491070+0000","last_became_peered":"2024-04-26T21:19:16.491070+0000","last_unstale":"2024-04-26T21:19:16.496630+0000","last_undegraded":"2024-04-26T21:19:16.496630+0000","last_fullsized":"2024-04-26T21:19:16.496630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:44.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501985+0000","last_change":"2024-04-26T21:19:16.499382+0000","last_active":"2024-04-26T21:19:16.501985+0000","last_peered":"2024-04-26T21:19:16.501985+0000","last_clean":"2024-04-26T21:19:16.501985+0000","last_became_active":"2024-04-26T21:19:16.499064+0000","last_became_peered":"2024-04-26T21:19:16.499064+0000","last_unstale":"2024-04-26T21:19:16.501985+0000","last_undegraded":"2024-04-26T21:19:16.501985+0000","last_fullsized":"2024-04-26T21:19:16.501985+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491852+0000","last_change":"2024-04-26T21:19:16.485418+0000","last_active":"2024-04-26T21:19:17.491852+0000","last_peered":"2024-04-26T21:19:17.491852+0000","last_clean":"2024-04-26T21:19:17.491852+0000","last_became_active":"2024-04-26T21:19:16.484504+0000","last_became_peered":"2024-04-26T21:19:16.484504+0000","last_unstale":"2024-04-26T21:19:17.491852+0000","last_undegraded":"2024-04-26T21:19:17.491852+0000","last_fullsized":"2024-04-26T21:19:17.491852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492897+0000","last_change":"2024-04-26T21:19:16.487920+0000","last_active":"2024-04-26T21:19:16.492897+0000","last_peered":"2024-04-26T21:19:16.492897+0000","last_clean":"2024-04-26T21:19:16.492897+0000","last_became_active":"2024-04-26T21:19:16.487548+0000","last_became_peered":"2024-04-26T21:19:16.487548+0000","last_unstale":"2024-04-26T21:19:16.492897+0000","last_undegraded":"2024-04-26T21:19:16.492897+0000","last_fullsized":"2024-04-26T21:19:16.492897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491267+0000","last_change":"2024-04-26T21:19:16.485140+0000","last_active":"2024-04-26T21:19:17.491267+0000","last_peered":"2024-04-26T21:19:17.491267+0000","last_clean":"2024-04-26T21:19:17.491267+0000","last_became_active":"2024-04-26T21:19:16.484871+0000","last_became_peered":"2024-04-26T21:19:16.484871+0000","last_unstale":"2024-04-26T21:19:17.491267+0000","last_undegraded":"2024-04-26T21:19:17.491267+0000","last_fullsized":"2024-04-26T21:19:17.491267+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.494135+0000","last_change":"2024-04-26T21:19:16.491074+0000","last_active":"2024-04-26T21:19:16.494135+0000","last_peered":"2024-04-26T21:19:16.494135+0000","last_clean":"2024-04-26T21:19:16.494135+0000","last_became_active":"2024-04-26T21:19:16.490719+0000","last_became_peered":"2024-04-26T21:19:16.490719+0000","last_unstale":"2024-04-26T21:19:16.494135+0000","last_undegraded":"2024-04-26T21:19:16.494135+0000","last_fullsized":"2024-04-26T21:19:16.494135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:07.415274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499547+0000","last_change":"2024-04-26T21:19:16.496876+0000","last_active":"2024-04-26T21:19:16.499547+0000","last_peered":"2024-04-26T21:19:16.499547+0000","last_clean":"2024-04-26T21:19:16.499547+0000","last_became_active":"2024-04-26T21:19:16.496601+0000","last_became_peered":"2024-04-26T21:19:16.496601+0000","last_unstale":"2024-04-26T21:19:16.499547+0000","last_undegraded":"2024-04-26T21:19:16.499547+0000","last_fullsized":"2024-04-26T21:19:16.499547+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:31.534507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490487+0000","last_change":"2024-04-26T21:19:16.491979+0000","last_active":"2024-04-26T21:19:17.490487+0000","last_peered":"2024-04-26T21:19:17.490487+0000","last_clean":"2024-04-26T21:19:17.490487+0000","last_became_active":"2024-04-26T21:19:16.491072+0000","last_became_peered":"2024-04-26T21:19:16.491072+0000","last_unstale":"2024-04-26T21:19:17.490487+0000","last_undegraded":"2024-04-26T21:19:17.490487+0000","last_fullsized":"2024-04-26T21:19:17.490487+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:12.435136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491768+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:17.491768+0000","last_peered":"2024-04-26T21:19:17.491768+0000","last_clean":"2024-04-26T21:19:17.491768+0000","last_became_active":"2024-04-26T21:19:16.484220+0000","last_became_peered":"2024-04-26T21:19:16.484220+0000","last_unstale":"2024-04-26T21:19:17.491768+0000","last_undegraded":"2024-04-26T21:19:17.491768+0000","last_fullsized":"2024-04-26T21:19:17.491768+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:25.928632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499556+0000","last_change":"2024-04-26T21:19:16.494852+0000","last_active":"2024-04-26T21:19:16.499556+0000","last_peered":"2024-04-26T21:19:16.499556+0000","last_clean":"2024-04-26T21:19:16.499556+0000","last_became_active":"2024-04-26T21:19:16.494590+0000","last_became_peered":"2024-04-26T21:19:16.494590+0000","last_unstale":"2024-04-26T21:19:16.499556+0000","last_undegraded":"2024-04-26T21:19:16.499556+0000","last_fullsized":"2024-04-26T21:19:16.499556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:15.415456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673032+0000","last_change":"2024-04-26T21:19:16.484845+0000","last_active":"2024-04-26T21:19:17.673032+0000","last_peered":"2024-04-26T21:19:17.673032+0000","last_clean":"2024-04-26T21:19:17.673032+0000","last_became_active":"2024-04-26T21:19:16.484525+0000","last_became_peered":"2024-04-26T21:19:16.484525+0000","last_unstale":"2024-04-26T21:19:17.673032+0000","last_undegraded":"2024-04-26T21:19:17.673032+0000","last_fullsized":"2024-04-26T21:19:17.673032+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:31.425192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499512+0000","last_change":"2024-04-26T21:19:16.497005+0000","last_active":"2024-04-26T21:19:16.499512+0000","last_peered":"2024-04-26T21:19:16.499512+0000","last_clean":"2024-04-26T21:19:16.499512+0000","last_became_active":"2024-04-26T21:19:16.496669+0000","last_became_peered":"2024-04-26T21:19:16.496669+0000","last_unstale":"2024-04-26T21:19:16.499512+0000","last_undegraded":"2024-04-26T21:19:16.499512+0000","last_fullsized":"2024-04-26T21:19:16.499512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.580462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490149+0000","last_change":"2024-04-26T21:19:16.500355+0000","last_active":"2024-04-26T21:19:17.490149+0000","last_peered":"2024-04-26T21:19:17.490149+0000","last_clean":"2024-04-26T21:19:17.490149+0000","last_became_active":"2024-04-26T21:19:16.500021+0000","last_became_peered":"2024-04-26T21:19:16.500021+0000","last_unstale":"2024-04-26T21:19:17.490149+0000","last_undegraded":"2024-04-26T21:19:17.490149+0000","last_fullsized":"2024-04-26T21:19:17.490149+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:01.461710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490414+0000","last_change":"2024-04-26T21:19:16.501698+0000","last_active":"2024-04-26T21:19:17.490414+0000","last_peered":"2024-04-26T21:19:17.490414+0000","last_clean":"2024-04-26T21:19:17.490414+0000","last_became_active":"2024-04-26T21:19:16.501426+0000","last_became_peered":"2024-04-26T21:19:16.501426+0000","last_unstale":"2024-04-26T21:19:17.490414+0000","last_undegraded":"2024-04-26T21:19:17.490414+0000","last_fullsized":"2024-04-26T21:19:17.490414+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:57.290656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673108+0000","last_change":"2024-04-26T21:19:15.478535+0000","last_active":"2024-04-26T21:19:17.673108+0000","last_peered":"2024-04-26T21:19:17.673108+0000","last_clean":"2024-04-26T21:19:17.673108+0000","last_became_active":"2024-04-26T21:19:15.478272+0000","last_became_peered":"2024-04-26T21:19:15.478272+0000","last_unstale":"2024-04-26T21:19:17.673108+0000","last_undegraded":"2024-04-26T21:19:17.673108+0000","last_fullsized":"2024-04-26T21:19:17.673108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492017+0000","last_change":"2024-04-26T21:19:16.488880+0000","last_active":"2024-04-26T21:19:16.492017+0000","last_peered":"2024-04-26T21:19:16.492017+0000","last_clean":"2024-04-26T21:19:16.492017+0000","last_became_active":"2024-04-26T21:19:16.488578+0000","last_became_peered":"2024-04-26T21:19:16.488578+0000","last_unstale":"2024-04-26T21:19:16.492017+0000","last_undegraded":"2024-04-26T21:19:16.492017+0000","last_fullsized":"2024-04-26T21:19:16.492017+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538442+0000","last_change":"2024-04-26T21:19:15.481825+0000","last_active":"2024-04-26T21:19:17.538442+0000","last_peered":"2024-04-26T21:19:17.538442+0000","last_clean":"2024-04-26T21:19:17.538442+0000","last_became_active":"2024-04-26T21:19:15.481529+0000","last_became_peered":"2024-04-26T21:19:15.481529+0000","last_unstale":"2024-04-26T21:19:17.538442+0000","last_undegraded":"2024-04-26T21:19:17.538442+0000","last_fullsized":"2024-04-26T21:19:17.538442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490592+0000","last_change":"2024-04-26T21:19:16.501161+0000","last_active":"2024-04-26T21:19:17.490592+0000","last_peered":"2024-04-26T21:19:17.490592+0000","last_clean":"2024-04-26T21:19:17.490592+0000","last_became_active":"2024-04-26T21:19:16.500862+0000","last_became_peered":"2024-04-26T21:19:16.500862+0000","last_unstale":"2024-04-26T21:19:17.490592+0000","last_undegraded":"2024-04-26T21:19:17.490592+0000","last_fullsized":"2024-04-26T21:19:17.490592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:08.968317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492403+0000","last_change":"2024-04-26T21:19:15.485103+0000","last_active":"2024-04-26T21:19:17.492403+0000","last_peered":"2024-04-26T21:19:17.492403+0000","last_clean":"2024-04-26T21:19:17.492403+0000","last_became_active":"2024-04-26T21:19:15.484770+0000","last_became_peered":"2024-04-26T21:19:15.484770+0000","last_unstale":"2024-04-26T21:19:17.492403+0000","last_undegraded":"2024-04-26T21:19:17.492403+0000","last_fullsized":"2024-04-26T21:19:17.492403+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:15.463835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538722+0000","last_change":"2024-04-26T21:19:16.491699+0000","last_active":"2024-04-26T21:19:17.538722+0000","last_peered":"2024-04-26T21:19:17.538722+0000","last_clean":"2024-04-26T21:19:17.538722+0000","last_became_active":"2024-04-26T21:19:16.491182+0000","last_became_peered":"2024-04-26T21:19:16.491182+0000","last_unstale":"2024-04-26T21:19:17.538722+0000","last_undegraded":"2024-04-26T21:19:17.538722+0000","last_fullsized":"2024-04-26T21:19:17.538722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492120+0000","last_change":"2024-04-26T21:19:15.483533+0000","last_active":"2024-04-26T21:19:17.492120+0000","last_peered":"2024-04-26T21:19:17.492120+0000","last_clean":"2024-04-26T21:19:17.492120+0000","last_became_active":"2024-04-26T21:19:15.483244+0000","last_became_peered":"2024-04-26T21:19:15.483244+0000","last_unstale":"2024-04-26T21:19:17.492120+0000","last_undegraded":"2024-04-26T21:19:17.492120+0000","last_fullsized":"2024-04-26T21:19:17.492120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:44.801378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502307+0000","last_change":"2024-04-26T21:19:16.499234+0000","last_active":"2024-04-26T21:19:16.502307+0000","last_peered":"2024-04-26T21:19:16.502307+0000","last_clean":"2024-04-26T21:19:16.502307+0000","last_became_active":"2024-04-26T21:19:16.498965+0000","last_became_peered":"2024-04-26T21:19:16.498965+0000","last_unstale":"2024-04-26T21:19:16.502307+0000","last_undegraded":"2024-04-26T21:19:16.502307+0000","last_fullsized":"2024-04-26T21:19:16.502307+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:38.866044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672911+0000","last_change":"2024-04-26T21:19:15.485012+0000","last_active":"2024-04-26T21:19:17.672911+0000","last_peered":"2024-04-26T21:19:17.672911+0000","last_clean":"2024-04-26T21:19:17.672911+0000","last_became_active":"2024-04-26T21:19:15.484711+0000","last_became_peered":"2024-04-26T21:19:15.484711+0000","last_unstale":"2024-04-26T21:19:17.672911+0000","last_undegraded":"2024-04-26T21:19:17.672911+0000","last_fullsized":"2024-04-26T21:19:17.672911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:58.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501835+0000","last_change":"2024-04-26T21:19:16.498702+0000","last_active":"2024-04-26T21:19:16.501835+0000","last_peered":"2024-04-26T21:19:16.501835+0000","last_clean":"2024-04-26T21:19:16.501835+0000","last_became_active":"2024-04-26T21:19:16.498408+0000","last_became_peered":"2024-04-26T21:19:16.498408+0000","last_unstale":"2024-04-26T21:19:16.501835+0000","last_undegraded":"2024-04-26T21:19:16.501835+0000","last_fullsized":"2024-04-26T21:19:16.501835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491957+0000","last_change":"2024-04-26T21:19:15.491055+0000","last_active":"2024-04-26T21:19:17.491957+0000","last_peered":"2024-04-26T21:19:17.491957+0000","last_clean":"2024-04-26T21:19:17.491957+0000","last_became_active":"2024-04-26T21:19:15.490783+0000","last_became_peered":"2024-04-26T21:19:15.490783+0000","last_unstale":"2024-04-26T21:19:17.491957+0000","last_undegraded":"2024-04-26T21:19:17.491957+0000","last_fullsized":"2024-04-26T21:19:17.491957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:05.607960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503155+0000","last_change":"2024-04-26T21:19:16.501447+0000","last_active":"2024-04-26T21:19:16.503155+0000","last_peered":"2024-04-26T21:19:16.503155+0000","last_clean":"2024-04-26T21:19:16.503155+0000","last_became_active":"2024-04-26T21:19:16.501149+0000","last_became_peered":"2024-04-26T21:19:16.501149+0000","last_unstale":"2024-04-26T21:19:16.503155+0000","last_undegraded":"2024-04-26T21:19:16.503155+0000","last_fullsized":"2024-04-26T21:19:16.503155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:20.124617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479420+0000","last_change":"2024-04-26T21:19:15.490438+0000","last_active":"2024-04-26T21:19:16.479420+0000","last_peered":"2024-04-26T21:19:16.479420+0000","last_clean":"2024-04-26T21:19:16.479420+0000","last_became_active":"2024-04-26T21:19:15.490210+0000","last_became_peered":"2024-04-26T21:19:15.490210+0000","last_unstale":"2024-04-26T21:19:16.479420+0000","last_undegraded":"2024-04-26T21:19:16.479420+0000","last_fullsized":"2024-04-26T21:19:16.479420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.422988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672795+0000","last_change":"2024-04-26T21:19:16.491479+0000","last_active":"2024-04-26T21:19:17.672795+0000","last_peered":"2024-04-26T21:19:17.672795+0000","last_clean":"2024-04-26T21:19:17.672795+0000","last_became_active":"2024-04-26T21:19:16.491224+0000","last_became_peered":"2024-04-26T21:19:16.491224+0000","last_unstale":"2024-04-26T21:19:17.672795+0000","last_undegraded":"2024-04-26T21:19:17.672795+0000","last_fullsized":"2024-04-26T21:19:17.672795+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:51.573831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478367+0000","last_change":"2024-04-26T21:19:15.488327+0000","last_active":"2024-04-26T21:19:16.478367+0000","last_peered":"2024-04-26T21:19:16.478367+0000","last_clean":"2024-04-26T21:19:16.478367+0000","last_became_active":"2024-04-26T21:19:15.488039+0000","last_became_peered":"2024-04-26T21:19:15.488039+0000","last_unstale":"2024-04-26T21:19:16.478367+0000","last_undegraded":"2024-04-26T21:19:16.478367+0000","last_fullsized":"2024-04-26T21:19:16.478367+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:08.686002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673153+0000","last_change":"2024-04-26T21:19:16.481276+0000","last_active":"2024-04-26T21:19:17.673153+0000","last_peered":"2024-04-26T21:19:17.673153+0000","last_clean":"2024-04-26T21:19:17.673153+0000","last_became_active":"2024-04-26T21:19:16.480986+0000","last_became_peered":"2024-04-26T21:19:16.480986+0000","last_unstale":"2024-04-26T21:19:17.673153+0000","last_undegraded":"2024-04-26T21:19:17.673153+0000","last_fullsized":"2024-04-26T21:19:17.673153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492469+0000","last_change":"2024-04-26T21:19:15.484302+0000","last_active":"2024-04-26T21:19:17.492469+0000","last_peered":"2024-04-26T21:19:17.492469+0000","last_clean":"2024-04-26T21:19:17.492469+0000","last_became_active":"2024-04-26T21:19:15.483990+0000","last_became_peered":"2024-04-26T21:19:15.483990+0000","last_unstale":"2024-04-26T21:19:17.492469+0000","last_undegraded":"2024-04-26T21:19:17.492469+0000","last_fullsized":"2024-04-26T21:19:17.492469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:02.063693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492042+0000","last_change":"2024-04-26T21:19:16.489119+0000","last_active":"2024-04-26T21:19:16.492042+0000","last_peered":"2024-04-26T21:19:16.492042+0000","last_clean":"2024-04-26T21:19:16.492042+0000","last_became_active":"2024-04-26T21:19:16.488816+0000","last_became_peered":"2024-04-26T21:19:16.488816+0000","last_unstale":"2024-04-26T21:19:16.492042+0000","last_undegraded":"2024-04-26T21:19:16.492042+0000","last_fullsized":"2024-04-26T21:19:16.492042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:04.053313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491477+0000","last_change":"2024-04-26T21:19:15.492213+0000","last_active":"2024-04-26T21:19:17.491477+0000","last_peered":"2024-04-26T21:19:17.491477+0000","last_clean":"2024-04-26T21:19:17.491477+0000","last_became_active":"2024-04-26T21:19:15.491792+0000","last_became_peered":"2024-04-26T21:19:15.491792+0000","last_unstale":"2024-04-26T21:19:17.491477+0000","last_undegraded":"2024-04-26T21:19:17.491477+0000","last_fullsized":"2024-04-26T21:19:17.491477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491775+0000","last_change":"2024-04-26T21:19:16.496675+0000","last_active":"2024-04-26T21:19:17.491775+0000","last_peered":"2024-04-26T21:19:17.491775+0000","last_clean":"2024-04-26T21:19:17.491775+0000","last_became_active":"2024-04-26T21:19:16.496382+0000","last_became_peered":"2024-04-26T21:19:16.496382+0000","last_unstale":"2024-04-26T21:19:17.491775+0000","last_undegraded":"2024-04-26T21:19:17.491775+0000","last_fullsized":"2024-04-26T21:19:17.491775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502058+0000","last_change":"2024-04-26T21:19:15.493890+0000","last_active":"2024-04-26T21:19:16.502058+0000","last_peered":"2024-04-26T21:19:16.502058+0000","last_clean":"2024-04-26T21:19:16.502058+0000","last_became_active":"2024-04-26T21:19:15.493428+0000","last_became_peered":"2024-04-26T21:19:15.493428+0000","last_unstale":"2024-04-26T21:19:16.502058+0000","last_undegraded":"2024-04-26T21:19:16.502058+0000","last_fullsized":"2024-04-26T21:19:16.502058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:44.796649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673102+0000","last_change":"2024-04-26T21:19:16.489093+0000","last_active":"2024-04-26T21:19:17.673102+0000","last_peered":"2024-04-26T21:19:17.673102+0000","last_clean":"2024-04-26T21:19:17.673102+0000","last_became_active":"2024-04-26T21:19:16.488815+0000","last_became_peered":"2024-04-26T21:19:16.488815+0000","last_unstale":"2024-04-26T21:19:17.673102+0000","last_undegraded":"2024-04-26T21:19:17.673102+0000","last_fullsized":"2024-04-26T21:19:17.673102+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:43.157084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.504928+0000","last_change":"2024-04-26T21:19:15.489147+0000","last_active":"2024-04-26T21:19:16.504928+0000","last_peered":"2024-04-26T21:19:16.504928+0000","last_clean":"2024-04-26T21:19:16.504928+0000","last_became_active":"2024-04-26T21:19:15.488782+0000","last_became_peered":"2024-04-26T21:19:15.488782+0000","last_unstale":"2024-04-26T21:19:16.504928+0000","last_undegraded":"2024-04-26T21:19:16.504928+0000","last_fullsized":"2024-04-26T21:19:16.504928+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:12.490510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491886+0000","last_change":"2024-04-26T21:19:16.494938+0000","last_active":"2024-04-26T21:19:17.491886+0000","last_peered":"2024-04-26T21:19:17.491886+0000","last_clean":"2024-04-26T21:19:17.491886+0000","last_became_active":"2024-04-26T21:19:16.494586+0000","last_became_peered":"2024-04-26T21:19:16.494586+0000","last_unstale":"2024-04-26T21:19:17.491886+0000","last_undegraded":"2024-04-26T21:19:17.491886+0000","last_fullsized":"2024-04-26T21:19:17.491886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490242+0000","last_change":"2024-04-26T21:19:15.486297+0000","last_active":"2024-04-26T21:19:17.490242+0000","last_peered":"2024-04-26T21:19:17.490242+0000","last_clean":"2024-04-26T21:19:17.490242+0000","last_became_active":"2024-04-26T21:19:15.485985+0000","last_became_peered":"2024-04-26T21:19:15.485985+0000","last_unstale":"2024-04-26T21:19:17.490242+0000","last_undegraded":"2024-04-26T21:19:17.490242+0000","last_fullsized":"2024-04-26T21:19:17.490242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:34.344417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491584+0000","last_change":"2024-04-26T21:19:16.484940+0000","last_active":"2024-04-26T21:19:17.491584+0000","last_peered":"2024-04-26T21:19:17.491584+0000","last_clean":"2024-04-26T21:19:17.491584+0000","last_became_active":"2024-04-26T21:19:16.484650+0000","last_became_peered":"2024-04-26T21:19:16.484650+0000","last_unstale":"2024-04-26T21:19:17.491584+0000","last_undegraded":"2024-04-26T21:19:17.491584+0000","last_fullsized":"2024-04-26T21:19:17.491584+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:09.688206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479957+0000","last_change":"2024-04-26T21:19:15.486046+0000","last_active":"2024-04-26T21:19:16.479957+0000","last_peered":"2024-04-26T21:19:16.479957+0000","last_clean":"2024-04-26T21:19:16.479957+0000","last_became_active":"2024-04-26T21:19:15.485776+0000","last_became_peered":"2024-04-26T21:19:15.485776+0000","last_unstale":"2024-04-26T21:19:16.479957+0000","last_undegraded":"2024-04-26T21:19:16.479957+0000","last_fullsized":"2024-04-26T21:19:16.479957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.737449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491404+0000","last_change":"2024-04-26T21:19:16.497306+0000","last_active":"2024-04-26T21:19:17.491404+0000","last_peered":"2024-04-26T21:19:17.491404+0000","last_clean":"2024-04-26T21:19:17.491404+0000","last_became_active":"2024-04-26T21:19:16.497039+0000","last_became_peered":"2024-04-26T21:19:16.497039+0000","last_unstale":"2024-04-26T21:19:17.491404+0000","last_undegraded":"2024-04-26T21:19:17.491404+0000","last_fullsized":"2024-04-26T21:19:17.491404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:16.896268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492215+0000","last_change":"2024-04-26T21:19:15.490142+0000","last_active":"2024-04-26T21:19:17.492215+0000","last_peered":"2024-04-26T21:19:17.492215+0000","last_clean":"2024-04-26T21:19:17.492215+0000","last_became_active":"2024-04-26T21:19:15.489841+0000","last_became_peered":"2024-04-26T21:19:15.489841+0000","last_unstale":"2024-04-26T21:19:17.492215+0000","last_undegraded":"2024-04-26T21:19:17.492215+0000","last_fullsized":"2024-04-26T21:19:17.492215+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:06.856169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492590+0000","last_change":"2024-04-26T21:19:16.485067+0000","last_active":"2024-04-26T21:19:17.492590+0000","last_peered":"2024-04-26T21:19:17.492590+0000","last_clean":"2024-04-26T21:19:17.492590+0000","last_became_active":"2024-04-26T21:19:16.484588+0000","last_became_peered":"2024-04-26T21:19:16.484588+0000","last_unstale":"2024-04-26T21:19:17.492590+0000","last_undegraded":"2024-04-26T21:19:17.492590+0000","last_fullsized":"2024-04-26T21:19:17.492590+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:06.657912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496599+0000","last_change":"2024-04-26T21:19:15.488200+0000","last_active":"2024-04-26T21:19:16.496599+0000","last_peered":"2024-04-26T21:19:16.496599+0000","last_clean":"2024-04-26T21:19:16.496599+0000","last_became_active":"2024-04-26T21:19:15.487840+0000","last_became_peered":"2024-04-26T21:19:15.487840+0000","last_unstale":"2024-04-26T21:19:16.496599+0000","last_undegraded":"2024-04-26T21:19:16.496599+0000","last_fullsized":"2024-04-26T21:19:16.496599+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:32.520127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490417+0000","last_change":"2024-04-26T21:19:16.505313+0000","last_active":"2024-04-26T21:19:17.490417+0000","last_peered":"2024-04-26T21:19:17.490417+0000","last_clean":"2024-04-26T21:19:17.490417+0000","last_became_active":"2024-04-26T21:19:16.504970+0000","last_became_peered":"2024-04-26T21:19:16.504970+0000","last_unstale":"2024-04-26T21:19:17.490417+0000","last_undegraded":"2024-04-26T21:19:17.490417+0000","last_fullsized":"2024-04-26T21:19:17.490417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:16.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492522+0000","last_change":"2024-04-26T21:19:15.482948+0000","last_active":"2024-04-26T21:19:17.492522+0000","last_peered":"2024-04-26T21:19:17.492522+0000","last_clean":"2024-04-26T21:19:17.492522+0000","last_became_active":"2024-04-26T21:19:15.482636+0000","last_became_peered":"2024-04-26T21:19:15.482636+0000","last_unstale":"2024-04-26T21:19:17.492522+0000","last_undegraded":"2024-04-26T21:19:17.492522+0000","last_fullsized":"2024-04-26T21:19:17.492522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:04.356393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538613+0000","last_change":"2024-04-26T21:19:16.492696+0000","last_active":"2024-04-26T21:19:17.538613+0000","last_peered":"2024-04-26T21:19:17.538613+0000","last_clean":"2024-04-26T21:19:17.538613+0000","last_became_active":"2024-04-26T21:19:16.492396+0000","last_became_peered":"2024-04-26T21:19:16.492396+0000","last_unstale":"2024-04-26T21:19:17.538613+0000","last_undegraded":"2024-04-26T21:19:17.538613+0000","last_fullsized":"2024-04-26T21:19:17.538613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498743+0000","last_change":"2024-04-26T21:19:15.488486+0000","last_active":"2024-04-26T21:19:16.498743+0000","last_peered":"2024-04-26T21:19:16.498743+0000","last_clean":"2024-04-26T21:19:16.498743+0000","last_became_active":"2024-04-26T21:19:15.488221+0000","last_became_peered":"2024-04-26T21:19:15.488221+0000","last_unstale":"2024-04-26T21:19:16.498743+0000","last_undegraded":"2024-04-26T21:19:16.498743+0000","last_fullsized":"2024-04-26T21:19:16.498743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:30.760409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538790+0000","last_change":"2024-04-26T21:19:16.492285+0000","last_active":"2024-04-26T21:19:17.538790+0000","last_peered":"2024-04-26T21:19:17.538790+0000","last_clean":"2024-04-26T21:19:17.538790+0000","last_became_active":"2024-04-26T21:19:16.491992+0000","last_became_peered":"2024-04-26T21:19:16.491992+0000","last_unstale":"2024-04-26T21:19:17.538790+0000","last_undegraded":"2024-04-26T21:19:17.538790+0000","last_fullsized":"2024-04-26T21:19:17.538790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672950+0000","last_change":"2024-04-26T21:19:15.486458+0000","last_active":"2024-04-26T21:19:17.672950+0000","last_peered":"2024-04-26T21:19:17.672950+0000","last_clean":"2024-04-26T21:19:17.672950+0000","last_became_active":"2024-04-26T21:19:15.486127+0000","last_became_peered":"2024-04-26T21:19:15.486127+0000","last_unstale":"2024-04-26T21:19:17.672950+0000","last_undegraded":"2024-04-26T21:19:17.672950+0000","last_fullsized":"2024-04-26T21:19:17.672950+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:05.272243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538381+0000","last_change":"2024-04-26T21:19:16.495963+0000","last_active":"2024-04-26T21:19:17.538381+0000","last_peered":"2024-04-26T21:19:17.538381+0000","last_clean":"2024-04-26T21:19:17.538381+0000","last_became_active":"2024-04-26T21:19:16.495645+0000","last_became_peered":"2024-04-26T21:19:16.495645+0000","last_unstale":"2024-04-26T21:19:17.538381+0000","last_undegraded":"2024-04-26T21:19:17.538381+0000","last_fullsized":"2024-04-26T21:19:17.538381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490371+0000","last_change":"2024-04-26T21:19:15.487470+0000","last_active":"2024-04-26T21:19:17.490371+0000","last_peered":"2024-04-26T21:19:17.490371+0000","last_clean":"2024-04-26T21:19:17.490371+0000","last_became_active":"2024-04-26T21:19:15.487190+0000","last_became_peered":"2024-04-26T21:19:15.487190+0000","last_unstale":"2024-04-26T21:19:17.490371+0000","last_undegraded":"2024-04-26T21:19:17.490371+0000","last_fullsized":"2024-04-26T21:19:17.490371+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:52.279807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491410+0000","last_change":"2024-04-26T21:19:16.487812+0000","last_active":"2024-04-26T21:19:16.491410+0000","last_peered":"2024-04-26T21:19:16.491410+0000","last_clean":"2024-04-26T21:19:16.491410+0000","last_became_active":"2024-04-26T21:19:16.487449+0000","last_became_peered":"2024-04-26T21:19:16.487449+0000","last_unstale":"2024-04-26T21:19:16.491410+0000","last_undegraded":"2024-04-26T21:19:16.491410+0000","last_fullsized":"2024-04-26T21:19:16.491410+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492009+0000","last_change":"2024-04-26T21:19:15.489410+0000","last_active":"2024-04-26T21:19:17.492009+0000","last_peered":"2024-04-26T21:19:17.492009+0000","last_clean":"2024-04-26T21:19:17.492009+0000","last_became_active":"2024-04-26T21:19:15.489083+0000","last_became_peered":"2024-04-26T21:19:15.489083+0000","last_unstale":"2024-04-26T21:19:17.492009+0000","last_undegraded":"2024-04-26T21:19:17.492009+0000","last_fullsized":"2024-04-26T21:19:17.492009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:50.451081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538376+0000","last_change":"2024-04-26T21:19:16.486905+0000","last_active":"2024-04-26T21:19:17.538376+0000","last_peered":"2024-04-26T21:19:17.538376+0000","last_clean":"2024-04-26T21:19:17.538376+0000","last_became_active":"2024-04-26T21:19:16.486575+0000","last_became_peered":"2024-04-26T21:19:16.486575+0000","last_unstale":"2024-04-26T21:19:17.538376+0000","last_undegraded":"2024-04-26T21:19:17.538376+0000","last_fullsized":"2024-04-26T21:19:17.538376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490212+0000","last_change":"2024-04-26T21:19:15.488247+0000","last_active":"2024-04-26T21:19:17.490212+0000","last_peered":"2024-04-26T21:19:17.490212+0000","last_clean":"2024-04-26T21:19:17.490212+0000","last_became_active":"2024-04-26T21:19:15.487933+0000","last_became_peered":"2024-04-26T21:19:15.487933+0000","last_unstale":"2024-04-26T21:19:17.490212+0000","last_undegraded":"2024-04-26T21:19:17.490212+0000","last_fullsized":"2024-04-26T21:19:17.490212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:16.427539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491418+0000","last_change":"2024-04-26T21:19:16.483829+0000","last_active":"2024-04-26T21:19:17.491418+0000","last_peered":"2024-04-26T21:19:17.491418+0000","last_clean":"2024-04-26T21:19:17.491418+0000","last_became_active":"2024-04-26T21:19:16.483486+0000","last_became_peered":"2024-04-26T21:19:16.483486+0000","last_unstale":"2024-04-26T21:19:17.491418+0000","last_undegraded":"2024-04-26T21:19:17.491418+0000","last_fullsized":"2024-04-26T21:19:17.491418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:34.802721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478470+0000","last_change":"2024-04-26T21:19:15.487580+0000","last_active":"2024-04-26T21:19:16.478470+0000","last_peered":"2024-04-26T21:19:16.478470+0000","last_clean":"2024-04-26T21:19:16.478470+0000","last_became_active":"2024-04-26T21:19:15.487307+0000","last_became_peered":"2024-04-26T21:19:15.487307+0000","last_unstale":"2024-04-26T21:19:16.478470+0000","last_undegraded":"2024-04-26T21:19:16.478470+0000","last_fullsized":"2024-04-26T21:19:16.478470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.926428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492041+0000","last_change":"2024-04-26T21:19:16.487487+0000","last_active":"2024-04-26T21:19:16.492041+0000","last_peered":"2024-04-26T21:19:16.492041+0000","last_clean":"2024-04-26T21:19:16.492041+0000","last_became_active":"2024-04-26T21:19:16.487194+0000","last_became_peered":"2024-04-26T21:19:16.487194+0000","last_unstale":"2024-04-26T21:19:16.492041+0000","last_undegraded":"2024-04-26T21:19:16.492041+0000","last_fullsized":"2024-04-26T21:19:16.492041+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491590+0000","last_change":"2024-04-26T21:19:15.490106+0000","last_active":"2024-04-26T21:19:17.491590+0000","last_peered":"2024-04-26T21:19:17.491590+0000","last_clean":"2024-04-26T21:19:17.491590+0000","last_became_active":"2024-04-26T21:19:15.489789+0000","last_became_peered":"2024-04-26T21:19:15.489789+0000","last_unstale":"2024-04-26T21:19:17.491590+0000","last_undegraded":"2024-04-26T21:19:17.491590+0000","last_fullsized":"2024-04-26T21:19:17.491590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.506686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498668+0000","last_change":"2024-04-26T21:19:16.496139+0000","last_active":"2024-04-26T21:19:16.498668+0000","last_peered":"2024-04-26T21:19:16.498668+0000","last_clean":"2024-04-26T21:19:16.498668+0000","last_became_active":"2024-04-26T21:19:16.495785+0000","last_became_peered":"2024-04-26T21:19:16.495785+0000","last_unstale":"2024-04-26T21:19:16.498668+0000","last_undegraded":"2024-04-26T21:19:16.498668+0000","last_fullsized":"2024-04-26T21:19:16.498668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:47.719083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.477196+0000","last_change":"2024-04-26T21:19:15.476421+0000","last_active":"2024-04-26T21:19:17.477196+0000","last_peered":"2024-04-26T21:19:17.477196+0000","last_clean":"2024-04-26T21:19:17.477196+0000","last_became_active":"2024-04-26T21:19:15.476081+0000","last_became_peered":"2024-04-26T21:19:15.476081+0000","last_unstale":"2024-04-26T21:19:17.477196+0000","last_undegraded":"2024-04-26T21:19:17.477196+0000","last_fullsized":"2024-04-26T21:19:17.477196+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502994+0000","last_change":"2024-04-26T21:19:16.502148+0000","last_active":"2024-04-26T21:19:16.502994+0000","last_peered":"2024-04-26T21:19:16.502994+0000","last_clean":"2024-04-26T21:19:16.502994+0000","last_became_active":"2024-04-26T21:19:16.500040+0000","last_became_peered":"2024-04-26T21:19:16.500040+0000","last_unstale":"2024-04-26T21:19:16.502994+0000","last_undegraded":"2024-04-26T21:19:16.502994+0000","last_fullsized":"2024-04-26T21:19:16.502994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:54.448349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479257+0000","last_change":"2024-04-26T21:19:15.481531+0000","last_active":"2024-04-26T21:19:16.479257+0000","last_peered":"2024-04-26T21:19:16.479257+0000","last_clean":"2024-04-26T21:19:16.479257+0000","last_became_active":"2024-04-26T21:19:15.481231+0000","last_became_peered":"2024-04-26T21:19:15.481231+0000","last_unstale":"2024-04-26T21:19:16.479257+0000","last_undegraded":"2024-04-26T21:19:16.479257+0000","last_fullsized":"2024-04-26T21:19:16.479257+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:04.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492095+0000","last_change":"2024-04-26T21:19:16.488965+0000","last_active":"2024-04-26T21:19:16.492095+0000","last_peered":"2024-04-26T21:19:16.492095+0000","last_clean":"2024-04-26T21:19:16.492095+0000","last_became_active":"2024-04-26T21:19:16.488655+0000","last_became_peered":"2024-04-26T21:19:16.488655+0000","last_unstale":"2024-04-26T21:19:16.492095+0000","last_undegraded":"2024-04-26T21:19:16.492095+0000","last_fullsized":"2024-04-26T21:19:16.492095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491897+0000","last_change":"2024-04-26T21:19:15.493244+0000","last_active":"2024-04-26T21:19:17.491897+0000","last_peered":"2024-04-26T21:19:17.491897+0000","last_clean":"2024-04-26T21:19:17.491897+0000","last_became_active":"2024-04-26T21:19:15.493019+0000","last_became_peered":"2024-04-26T21:19:15.493019+0000","last_unstale":"2024-04-26T21:19:17.491897+0000","last_undegraded":"2024-04-26T21:19:17.491897+0000","last_fullsized":"2024-04-26T21:19:17.491897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:15.360405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496729+0000","last_change":"2024-04-26T21:19:16.493217+0000","last_active":"2024-04-26T21:19:16.496729+0000","last_peered":"2024-04-26T21:19:16.496729+0000","last_clean":"2024-04-26T21:19:16.496729+0000","last_became_active":"2024-04-26T21:19:16.492937+0000","last_became_peered":"2024-04-26T21:19:16.492937+0000","last_unstale":"2024-04-26T21:19:16.496729+0000","last_undegraded":"2024-04-26T21:19:16.496729+0000","last_fullsized":"2024-04-26T21:19:16.496729+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:31.762614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673072+0000","last_change":"2024-04-26T21:19:15.482604+0000","last_active":"2024-04-26T21:19:17.673072+0000","last_peered":"2024-04-26T21:19:17.673072+0000","last_clean":"2024-04-26T21:19:17.673072+0000","last_became_active":"2024-04-26T21:19:15.482320+0000","last_became_peered":"2024-04-26T21:19:15.482320+0000","last_unstale":"2024-04-26T21:19:17.673072+0000","last_undegraded":"2024-04-26T21:19:17.673072+0000","last_fullsized":"2024-04-26T21:19:17.673072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:24.748462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496666+0000","last_change":"2024-04-26T21:19:16.493348+0000","last_active":"2024-04-26T21:19:16.496666+0000","last_peered":"2024-04-26T21:19:16.496666+0000","last_clean":"2024-04-26T21:19:16.496666+0000","last_became_active":"2024-04-26T21:19:16.492941+0000","last_became_peered":"2024-04-26T21:19:16.492941+0000","last_unstale":"2024-04-26T21:19:16.496666+0000","last_undegraded":"2024-04-26T21:19:16.496666+0000","last_fullsized":"2024-04-26T21:19:16.496666+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.739653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479467+0000","last_change":"2024-04-26T21:19:15.489228+0000","last_active":"2024-04-26T21:19:16.479467+0000","last_peered":"2024-04-26T21:19:16.479467+0000","last_clean":"2024-04-26T21:19:16.479467+0000","last_became_active":"2024-04-26T21:19:15.488843+0000","last_became_peered":"2024-04-26T21:19:15.488843+0000","last_unstale":"2024-04-26T21:19:16.479467+0000","last_undegraded":"2024-04-26T21:19:16.479467+0000","last_fullsized":"2024-04-26T21:19:16.479467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:42.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538707+0000","last_change":"2024-04-26T21:19:16.491546+0000","last_active":"2024-04-26T21:19:17.538707+0000","last_peered":"2024-04-26T21:19:17.538707+0000","last_clean":"2024-04-26T21:19:17.538707+0000","last_became_active":"2024-04-26T21:19:16.491305+0000","last_became_peered":"2024-04-26T21:19:16.491305+0000","last_unstale":"2024-04-26T21:19:17.538707+0000","last_undegraded":"2024-04-26T21:19:17.538707+0000","last_fullsized":"2024-04-26T21:19:17.538707+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492104+0000","last_change":"2024-04-26T21:19:15.490249+0000","last_active":"2024-04-26T21:19:17.492104+0000","last_peered":"2024-04-26T21:19:17.492104+0000","last_clean":"2024-04-26T21:19:17.492104+0000","last_became_active":"2024-04-26T21:19:15.489962+0000","last_became_peered":"2024-04-26T21:19:15.489962+0000","last_unstale":"2024-04-26T21:19:17.492104+0000","last_undegraded":"2024-04-26T21:19:17.492104+0000","last_fullsized":"2024-04-26T21:19:17.492104+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503309+0000","last_change":"2024-04-26T21:19:16.501636+0000","last_active":"2024-04-26T21:19:16.503309+0000","last_peered":"2024-04-26T21:19:16.503309+0000","last_clean":"2024-04-26T21:19:16.503309+0000","last_became_active":"2024-04-26T21:19:16.501326+0000","last_became_peered":"2024-04-26T21:19:16.501326+0000","last_unstale":"2024-04-26T21:19:16.503309+0000","last_undegraded":"2024-04-26T21:19:16.503309+0000","last_fullsized":"2024-04-26T21:19:16.503309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:10.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478523+0000","last_change":"2024-04-26T21:19:15.486884+0000","last_active":"2024-04-26T21:19:16.478523+0000","last_peered":"2024-04-26T21:19:16.478523+0000","last_clean":"2024-04-26T21:19:16.478523+0000","last_became_active":"2024-04-26T21:19:15.486597+0000","last_became_peered":"2024-04-26T21:19:15.486597+0000","last_unstale":"2024-04-26T21:19:16.478523+0000","last_undegraded":"2024-04-26T21:19:16.478523+0000","last_fullsized":"2024-04-26T21:19:16.478523+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:16.558118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491321+0000","last_change":"2024-04-26T21:19:16.485658+0000","last_active":"2024-04-26T21:19:17.491321+0000","last_peered":"2024-04-26T21:19:17.491321+0000","last_clean":"2024-04-26T21:19:17.491321+0000","last_became_active":"2024-04-26T21:19:16.485382+0000","last_became_peered":"2024-04-26T21:19:16.485382+0000","last_unstale":"2024-04-26T21:19:17.491321+0000","last_undegraded":"2024-04-26T21:19:17.491321+0000","last_fullsized":"2024-04-26T21:19:17.491321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672961+0000","last_change":"2024-04-26T21:19:15.480711+0000","last_active":"2024-04-26T21:19:17.672961+0000","last_peered":"2024-04-26T21:19:17.672961+0000","last_clean":"2024-04-26T21:19:17.672961+0000","last_became_active":"2024-04-26T21:19:15.480405+0000","last_became_peered":"2024-04-26T21:19:15.480405+0000","last_unstale":"2024-04-26T21:19:17.672961+0000","last_undegraded":"2024-04-26T21:19:17.672961+0000","last_fullsized":"2024-04-26T21:19:17.672961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:00.640043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538749+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:19:17.538749+0000","last_peered":"2024-04-26T21:19:17.538749+0000","last_clean":"2024-04-26T21:19:17.538749+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:19:17.538749+0000","last_undegraded":"2024-04-26T21:19:17.538749+0000","last_fullsized":"2024-04-26T21:19:17.538749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509534,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":1161038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":570124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":576958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":570054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":571736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1162146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":576478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:19:21.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:19:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.353+0000 7f13e0e45640 1 Processor -- start 2024-04-26T21:19:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.353+0000 7f13e0e45640 1 -- start start 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13e0e45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13dc0d5810 0x7f13dc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13e0e45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13dc0d2e20 0x7f13dc0b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13e0e45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13dc0b2550 con 0x7f13dc15f4d0 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13e0e45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f13dc0b6180 con 0x7f13dc0d2e20 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13e0e45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f13dc0b2830 con 0x7f13dc0d5810 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13dc0d2e20 0x7f13dc0b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13dc0d2e20 0x7f13dc0b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57182/0 (socket says 172.21.15.40:57182) 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.354+0000 7f13dad9d640 1 -- 172.21.15.40:0/2628483755 learned_addr learned my addr 172.21.15.40:0/2628483755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13d9d9b640 1 --2- 172.21.15.40:0/2628483755 >> v2:172.21.15.184:3300/0 conn(0x7f13dc0d5810 0x7f13dc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13dad9d640 1 -- 172.21.15.40:0/2628483755 >> v2:172.21.15.184:3300/0 conn(0x7f13dc0d5810 msgr2=0x7f13dc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13dad9d640 1 --2- 172.21.15.40:0/2628483755 >> v2:172.21.15.184:3300/0 conn(0x7f13dc0d5810 0x7f13dc0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13dad9d640 1 -- 172.21.15.40:0/2628483755 >> v1:172.21.15.184:6789/0 conn(0x7f13dc15f4d0 legacy=0x7f13dc0d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13dad9d640 1 -- 172.21.15.40:0/2628483755 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc0b2b10 con 0x7f13dc0d2e20 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.355+0000 7f13dad9d640 1 --2- 172.21.15.40:0/2628483755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0b35f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f13dc08d5e0 tx=0x7f13cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a931b3e4d76ea84a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13d959a640 1 -- 172.21.15.40:0/2628483755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc036070 con 0x7f13dc0d2e20 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13d959a640 1 -- 172.21.15.40:0/2628483755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc031600 con 0x7f13dc0d2e20 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13d959a640 1 -- 172.21.15.40:0/2628483755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0319d0 con 0x7f13dc0d2e20 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 -- 172.21.15.40:0/2628483755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 msgr2=0x7f13dc0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 --2- 172.21.15.40:0/2628483755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0b35f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f13dc08d5e0 tx=0x7f13cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 -- 172.21.15.40:0/2628483755 shutdown_connections 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 --2- 172.21.15.40:0/2628483755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0b35f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 --2- 172.21.15.40:0/2628483755 >> v2:172.21.15.184:3300/0 conn(0x7f13dc0d5810 0x7f13dc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 -- 172.21.15.40:0/2628483755 >> 172.21.15.40:0/2628483755 conn(0x7f13dc08ebf0 msgr2=0x7f13dc09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 -- 172.21.15.40:0/2628483755 shutdown_connections 2024-04-26T21:19:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.356+0000 7f13e0e45640 1 -- 172.21.15.40:0/2628483755 wait complete. 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 Processor -- start 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 -- start start 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc0d5810 0x7f13dc0ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 0x7f13dc1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0b3b30 con 0x7f13dc0d2e20 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0b30d0 con 0x7f13dc0d5810 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13e0e45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc0b3cb0 con 0x7f13dc15f4d0 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57192/0 (socket says 172.21.15.40:57192) 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.358+0000 7f13da59c640 1 -- 172.21.15.40:0/578805336 learned_addr learned my addr 172.21.15.40:0/578805336 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13dad9d640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 0x7f13dc1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13d9d9b640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc0d5810 0x7f13dc0ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 -- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 msgr2=0x7f13dc1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 0x7f13dc1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 -- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc0d5810 msgr2=0x7f13dc0ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc0d5810 0x7f13dc0ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 -- 172.21.15.40:0/578805336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc159d20 con 0x7f13dc0d2e20 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13dad9d640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 0x7f13dc1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13da59c640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13d400c3f0 tx=0x7f13d400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40052d0 con 0x7f13dc0d2e20 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d4005490 con 0x7f13dc0d2e20 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc0b96c0 con 0x7f13dc0d2e20 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4007260 con 0x7f13dc0d2e20 2024-04-26T21:19:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.359+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc0b9c80 con 0x7f13dc0d2e20 2024-04-26T21:19:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.360+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13dc06a290 con 0x7f13dc0d2e20 2024-04-26T21:19:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.361+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d4007400 con 0x7f13dc0d2e20 2024-04-26T21:19:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.361+0000 7f13d2ffd640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 0x7f13bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.361+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f13d408af80 con 0x7f13dc0d2e20 2024-04-26T21:19:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.361+0000 7f13d9d9b640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 0x7f13bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.362+0000 7f13d9d9b640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 0x7f13bc06edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0ce670 tx=0x7f13c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.365+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13dc159d20 con 0x7f13dc0d2e20 2024-04-26T21:19:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.622+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f13dc0c88e0 con 0x7f13dc0d2e20 2024-04-26T21:19:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.622+0000 7f13d2ffd640 1 -- 172.21.15.40:0/578805336 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1015 (secure 0 0 0) 0x7f13d4059440 con 0x7f13dc0d2e20 2024-04-26T21:19:21.623 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:21.624 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":464,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":271699968,"bytes_avail":772822413312,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":64,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":7962}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-26T21:17:04.990352+0000","services":{}},"progress_events":{}} 2024-04-26T21:19:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 msgr2=0x7f13bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 0x7f13bc06edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0ce670 tx=0x7f13c4008040 comp rx=0 tx=0).stop 2024-04-26T21:19:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 msgr2=0x7f13dc0c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13d400c3f0 tx=0x7f13d400c8b0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 shutdown_connections 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13bc06c8f0 0x7f13bc06edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13dc15f4d0 0x7f13dc1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc0d5810 0x7f13dc0ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 --2- 172.21.15.40:0/578805336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13dc0d2e20 0x7f13dc0c8d80 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 >> 172.21.15.40:0/578805336 conn(0x7f13dc08ebf0 msgr2=0x7f13dc06d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 shutdown_connections 2024-04-26T21:19:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.624+0000 7f13e0e45640 1 -- 172.21.15.40:0/578805336 wait complete. 2024-04-26T21:19:21.636 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:19:21.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:19:21.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.736+0000 7f19686f8640 1 Processor -- start 2024-04-26T21:19:21.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.736+0000 7f19686f8640 1 -- start start 2024-04-26T21:19:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f19686f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19600d3ee0 0x7f19600cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f19686f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f19600b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f19686f8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19600d03a0 con 0x7f196015f370 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f19686f8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19600b4d10 con 0x7f19600d3ee0 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f19686f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19600d0680 con 0x7f196015f750 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f1965c93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19600d3ee0 0x7f19600cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f1966494640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f196015f370 0x7f19600d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55378/0 (socket says 172.21.15.40:55378) 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f1966494640 1 -- 172.21.15.40:0/2538750530 learned_addr learned my addr 172.21.15.40:0/2538750530 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.737+0000 7f1966c95640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f19600b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965c93640 1 -- 172.21.15.40:0/2538750530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 msgr2=0x7f19600b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965c93640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f19600b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965c93640 1 -- 172.21.15.40:0/2538750530 >> v1:172.21.15.40:6789/0 conn(0x7f196015f370 legacy=0x7f19600d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965c93640 1 -- 172.21.15.40:0/2538750530 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600d0960 con 0x7f19600d3ee0 2024-04-26T21:19:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965492640 1 -- 172.21.15.40:0/2538750530 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758742475 0 0) 0x7f19600d03a0 con 0x7f196015f370 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.738+0000 7f1965c93640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600cfd70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1025fc5e1be410 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f1965492640 1 -- 172.21.15.40:0/2538750530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954037070 con 0x7f19600d3ee0 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f1965492640 1 -- 172.21.15.40:0/2538750530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954032440 con 0x7f19600d3ee0 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f1965492640 1 -- 172.21.15.40:0/2538750530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954032810 con 0x7f19600d3ee0 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 -- 172.21.15.40:0/2538750530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 msgr2=0x7f19600cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600cfd70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402a180 comp rx=0 tx=0).stop 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 -- 172.21.15.40:0/2538750530 shutdown_connections 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f19600b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 --2- 172.21.15.40:0/2538750530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600cfd70 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 -- 172.21.15.40:0/2538750530 >> 172.21.15.40:0/2538750530 conn(0x7f196008ec10 msgr2=0x7f196009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 -- 172.21.15.40:0/2538750530 shutdown_connections 2024-04-26T21:19:21.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.739+0000 7f19686f8640 1 -- 172.21.15.40:0/2538750530 wait complete. 2024-04-26T21:19:21.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.740+0000 7f19686f8640 1 Processor -- start 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.740+0000 7f19686f8640 1 -- start start 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 0x7f196006de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f1960152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19600b51c0 con 0x7f19600d3ee0 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19600d0f20 con 0x7f196015f370 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f19686f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19600b5340 con 0x7f196015f750 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f1966494640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f1966494640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57214/0 (socket says 172.21.15.40:57214) 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f1966494640 1 -- 172.21.15.40:0/2592097373 learned_addr learned my addr 172.21.15.40:0/2592097373 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f1966c95640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f1960152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.741+0000 7f1965c93640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 0x7f196006de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 -- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 msgr2=0x7f1960152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f1960152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 -- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 msgr2=0x7f196006de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 0x7f196006de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 -- 172.21.15.40:0/2592097373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600cc780 con 0x7f19600d3ee0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1965c93640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 0x7f196006de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f1966494640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195000c890 tx=0x7f195000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950018070 con 0x7f19600d3ee0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195000a640 con 0x7f19600d3ee0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000aa30 con 0x7f19600d3ee0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19600cca60 con 0x7f19600d3ee0 2024-04-26T21:19:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.742+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19600cd020 con 0x7f19600d3ee0 2024-04-26T21:19:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.743+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196006a290 con 0x7f19600d3ee0 2024-04-26T21:19:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.743+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f195001b070 con 0x7f19600d3ee0 2024-04-26T21:19:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.744+0000 7f195effd640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 0x7f194c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.744+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f195008b260 con 0x7f19600d3ee0 2024-04-26T21:19:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.744+0000 7f1965c93640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 0x7f194c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.745+0000 7f1965c93640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 0x7f194c06edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1954034000 tx=0x7f195402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:21.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.748+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1950059690 con 0x7f19600d3ee0 2024-04-26T21:19:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.931+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f196006d1a0 con 0x7f194c06c8f0 2024-04-26T21:19:21.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.938+0000 7f195effd640 1 -- 172.21.15.40:0/2592097373 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252451 (secure 0 0 0) 0x7f196006d1a0 con 0x7f194c06c8f0 2024-04-26T21:19:21.941 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 msgr2=0x7f194c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 0x7f194c06edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1954034000 tx=0x7f195402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 msgr2=0x7f19600ba8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195000c890 tx=0x7f195000cd50 comp rx=0 tx=0).stop 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 shutdown_connections 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f194c06c8f0 0x7f194c06edb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f196015f750 0x7f1960152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f196015f370 0x7f196006de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 --2- 172.21.15.40:0/2592097373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19600d3ee0 0x7f19600ba8a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 >> 172.21.15.40:0/2592097373 conn(0x7f196008ec10 msgr2=0x7f19600d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 shutdown_connections 2024-04-26T21:19:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:21.941+0000 7f19686f8640 1 -- 172.21.15.40:0/2592097373 wait complete. 2024-04-26T21:19:21.952 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-26T21:19:21.025443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265332,"kb_used_data":11692,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709388,"statfs":{"total":773094113280,"available":772822413312,"internally_reserved":0,"allocated":11972608,"data_stored":5764888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001751"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498644+0000","last_change":"2024-04-26T21:19:16.496097+0000","last_active":"2024-04-26T21:19:16.498644+0000","last_peered":"2024-04-26T21:19:16.498644+0000","last_clean":"2024-04-26T21:19:16.498644+0000","last_became_active":"2024-04-26T21:19:16.494023+0000","last_became_peered":"2024-04-26T21:19:16.494023+0000","last_unstale":"2024-04-26T21:19:16.498644+0000","last_undegraded":"2024-04-26T21:19:16.498644+0000","last_fullsized":"2024-04-26T21:19:16.498644+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:34.681056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501847+0000","last_change":"2024-04-26T21:19:16.498688+0000","last_active":"2024-04-26T21:19:16.501847+0000","last_peered":"2024-04-26T21:19:16.501847+0000","last_clean":"2024-04-26T21:19:16.501847+0000","last_became_active":"2024-04-26T21:19:16.498359+0000","last_became_peered":"2024-04-26T21:19:16.498359+0000","last_unstale":"2024-04-26T21:19:16.501847+0000","last_undegraded":"2024-04-26T21:19:16.501847+0000","last_fullsized":"2024-04-26T21:19:16.501847+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538503+0000","last_change":"2024-04-26T21:19:16.496173+0000","last_active":"2024-04-26T21:19:17.538503+0000","last_peered":"2024-04-26T21:19:17.538503+0000","last_clean":"2024-04-26T21:19:17.538503+0000","last_became_active":"2024-04-26T21:19:16.495826+0000","last_became_peered":"2024-04-26T21:19:16.495826+0000","last_unstale":"2024-04-26T21:19:17.538503+0000","last_undegraded":"2024-04-26T21:19:17.538503+0000","last_fullsized":"2024-04-26T21:19:17.538503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:13.492714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498530+0000","last_change":"2024-04-26T21:19:16.495999+0000","last_active":"2024-04-26T21:19:16.498530+0000","last_peered":"2024-04-26T21:19:16.498530+0000","last_clean":"2024-04-26T21:19:16.498530+0000","last_became_active":"2024-04-26T21:19:16.495621+0000","last_became_peered":"2024-04-26T21:19:16.495621+0000","last_unstale":"2024-04-26T21:19:16.498530+0000","last_undegraded":"2024-04-26T21:19:16.498530+0000","last_fullsized":"2024-04-26T21:19:16.498530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492356+0000","last_change":"2024-04-26T21:19:16.482865+0000","last_active":"2024-04-26T21:19:17.492356+0000","last_peered":"2024-04-26T21:19:17.492356+0000","last_clean":"2024-04-26T21:19:17.492356+0000","last_became_active":"2024-04-26T21:19:16.482588+0000","last_became_peered":"2024-04-26T21:19:16.482588+0000","last_unstale":"2024-04-26T21:19:17.492356+0000","last_undegraded":"2024-04-26T21:19:17.492356+0000","last_fullsized":"2024-04-26T21:19:17.492356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:23.086160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490188+0000","last_change":"2024-04-26T21:19:16.500088+0000","last_active":"2024-04-26T21:19:17.490188+0000","last_peered":"2024-04-26T21:19:17.490188+0000","last_clean":"2024-04-26T21:19:17.490188+0000","last_became_active":"2024-04-26T21:19:16.499783+0000","last_became_peered":"2024-04-26T21:19:16.499783+0000","last_unstale":"2024-04-26T21:19:17.490188+0000","last_undegraded":"2024-04-26T21:19:17.490188+0000","last_fullsized":"2024-04-26T21:19:17.490188+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.343352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672936+0000","last_change":"2024-04-26T21:19:16.488443+0000","last_active":"2024-04-26T21:19:17.672936+0000","last_peered":"2024-04-26T21:19:17.672936+0000","last_clean":"2024-04-26T21:19:17.672936+0000","last_became_active":"2024-04-26T21:19:16.488149+0000","last_became_peered":"2024-04-26T21:19:16.488149+0000","last_unstale":"2024-04-26T21:19:17.672936+0000","last_undegraded":"2024-04-26T21:19:17.672936+0000","last_fullsized":"2024-04-26T21:19:17.672936+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538580+0000","last_change":"2024-04-26T21:19:16.498705+0000","last_active":"2024-04-26T21:19:17.538580+0000","last_peered":"2024-04-26T21:19:17.538580+0000","last_clean":"2024-04-26T21:19:17.538580+0000","last_became_active":"2024-04-26T21:19:16.498371+0000","last_became_peered":"2024-04-26T21:19:16.498371+0000","last_unstale":"2024-04-26T21:19:17.538580+0000","last_undegraded":"2024-04-26T21:19:17.538580+0000","last_fullsized":"2024-04-26T21:19:17.538580+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:33.522331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496250+0000","last_change":"2024-04-26T21:19:16.489425+0000","last_active":"2024-04-26T21:19:16.496250+0000","last_peered":"2024-04-26T21:19:16.496250+0000","last_clean":"2024-04-26T21:19:16.496250+0000","last_became_active":"2024-04-26T21:19:16.489130+0000","last_became_peered":"2024-04-26T21:19:16.489130+0000","last_unstale":"2024-04-26T21:19:16.496250+0000","last_undegraded":"2024-04-26T21:19:16.496250+0000","last_fullsized":"2024-04-26T21:19:16.496250+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488375+0000","last_change":"2024-04-26T21:19:16.485878+0000","last_active":"2024-04-26T21:19:16.488375+0000","last_peered":"2024-04-26T21:19:16.488375+0000","last_clean":"2024-04-26T21:19:16.488375+0000","last_became_active":"2024-04-26T21:19:16.485592+0000","last_became_peered":"2024-04-26T21:19:16.485592+0000","last_unstale":"2024-04-26T21:19:16.488375+0000","last_undegraded":"2024-04-26T21:19:16.488375+0000","last_fullsized":"2024-04-26T21:19:16.488375+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496673+0000","last_change":"2024-04-26T21:19:16.492337+0000","last_active":"2024-04-26T21:19:16.496673+0000","last_peered":"2024-04-26T21:19:16.496673+0000","last_clean":"2024-04-26T21:19:16.496673+0000","last_became_active":"2024-04-26T21:19:16.492009+0000","last_became_peered":"2024-04-26T21:19:16.492009+0000","last_unstale":"2024-04-26T21:19:16.496673+0000","last_undegraded":"2024-04-26T21:19:16.496673+0000","last_fullsized":"2024-04-26T21:19:16.496673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:37.755632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.488890+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:16.488890+0000","last_peered":"2024-04-26T21:19:16.488890+0000","last_clean":"2024-04-26T21:19:16.488890+0000","last_became_active":"2024-04-26T21:19:16.484183+0000","last_became_peered":"2024-04-26T21:19:16.484183+0000","last_unstale":"2024-04-26T21:19:16.488890+0000","last_undegraded":"2024-04-26T21:19:16.488890+0000","last_fullsized":"2024-04-26T21:19:16.488890+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538678+0000","last_change":"2024-04-26T21:19:16.495059+0000","last_active":"2024-04-26T21:19:17.538678+0000","last_peered":"2024-04-26T21:19:17.538678+0000","last_clean":"2024-04-26T21:19:17.538678+0000","last_became_active":"2024-04-26T21:19:16.494780+0000","last_became_peered":"2024-04-26T21:19:16.494780+0000","last_unstale":"2024-04-26T21:19:17.538678+0000","last_undegraded":"2024-04-26T21:19:17.538678+0000","last_fullsized":"2024-04-26T21:19:17.538678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538257+0000","last_change":"2024-04-26T21:19:16.490955+0000","last_active":"2024-04-26T21:19:17.538257+0000","last_peered":"2024-04-26T21:19:17.538257+0000","last_clean":"2024-04-26T21:19:17.538257+0000","last_became_active":"2024-04-26T21:19:16.490657+0000","last_became_peered":"2024-04-26T21:19:16.490657+0000","last_unstale":"2024-04-26T21:19:17.538257+0000","last_undegraded":"2024-04-26T21:19:17.538257+0000","last_fullsized":"2024-04-26T21:19:17.538257+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490182+0000","last_change":"2024-04-26T21:19:16.500703+0000","last_active":"2024-04-26T21:19:17.490182+0000","last_peered":"2024-04-26T21:19:17.490182+0000","last_clean":"2024-04-26T21:19:17.490182+0000","last_became_active":"2024-04-26T21:19:16.500372+0000","last_became_peered":"2024-04-26T21:19:16.500372+0000","last_unstale":"2024-04-26T21:19:17.490182+0000","last_undegraded":"2024-04-26T21:19:17.490182+0000","last_fullsized":"2024-04-26T21:19:17.490182+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:28.187760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491077+0000","last_change":"2024-04-26T21:19:16.487462+0000","last_active":"2024-04-26T21:19:16.491077+0000","last_peered":"2024-04-26T21:19:16.491077+0000","last_clean":"2024-04-26T21:19:16.491077+0000","last_became_active":"2024-04-26T21:19:16.486946+0000","last_became_peered":"2024-04-26T21:19:16.486946+0000","last_unstale":"2024-04-26T21:19:16.491077+0000","last_undegraded":"2024-04-26T21:19:16.491077+0000","last_fullsized":"2024-04-26T21:19:16.491077+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496577+0000","last_change":"2024-04-26T21:19:16.491720+0000","last_active":"2024-04-26T21:19:16.496577+0000","last_peered":"2024-04-26T21:19:16.496577+0000","last_clean":"2024-04-26T21:19:16.496577+0000","last_became_active":"2024-04-26T21:19:16.491369+0000","last_became_peered":"2024-04-26T21:19:16.491369+0000","last_unstale":"2024-04-26T21:19:16.496577+0000","last_undegraded":"2024-04-26T21:19:16.496577+0000","last_fullsized":"2024-04-26T21:19:16.496577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:18.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499278+0000","last_change":"2024-04-26T21:19:16.496447+0000","last_active":"2024-04-26T21:19:16.499278+0000","last_peered":"2024-04-26T21:19:16.499278+0000","last_clean":"2024-04-26T21:19:16.499278+0000","last_became_active":"2024-04-26T21:19:16.496204+0000","last_became_peered":"2024-04-26T21:19:16.496204+0000","last_unstale":"2024-04-26T21:19:16.499278+0000","last_undegraded":"2024-04-26T21:19:16.499278+0000","last_fullsized":"2024-04-26T21:19:16.499278+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:50.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538653+0000","last_change":"2024-04-26T21:19:16.491438+0000","last_active":"2024-04-26T21:19:17.538653+0000","last_peered":"2024-04-26T21:19:17.538653+0000","last_clean":"2024-04-26T21:19:17.538653+0000","last_became_active":"2024-04-26T21:19:16.490909+0000","last_became_peered":"2024-04-26T21:19:16.490909+0000","last_unstale":"2024-04-26T21:19:17.538653+0000","last_undegraded":"2024-04-26T21:19:17.538653+0000","last_fullsized":"2024-04-26T21:19:17.538653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496630+0000","last_change":"2024-04-26T21:19:16.491473+0000","last_active":"2024-04-26T21:19:16.496630+0000","last_peered":"2024-04-26T21:19:16.496630+0000","last_clean":"2024-04-26T21:19:16.496630+0000","last_became_active":"2024-04-26T21:19:16.491070+0000","last_became_peered":"2024-04-26T21:19:16.491070+0000","last_unstale":"2024-04-26T21:19:16.496630+0000","last_undegraded":"2024-04-26T21:19:16.496630+0000","last_fullsized":"2024-04-26T21:19:16.496630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:44.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501985+0000","last_change":"2024-04-26T21:19:16.499382+0000","last_active":"2024-04-26T21:19:16.501985+0000","last_peered":"2024-04-26T21:19:16.501985+0000","last_clean":"2024-04-26T21:19:16.501985+0000","last_became_active":"2024-04-26T21:19:16.499064+0000","last_became_peered":"2024-04-26T21:19:16.499064+0000","last_unstale":"2024-04-26T21:19:16.501985+0000","last_undegraded":"2024-04-26T21:19:16.501985+0000","last_fullsized":"2024-04-26T21:19:16.501985+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491852+0000","last_change":"2024-04-26T21:19:16.485418+0000","last_active":"2024-04-26T21:19:17.491852+0000","last_peered":"2024-04-26T21:19:17.491852+0000","last_clean":"2024-04-26T21:19:17.491852+0000","last_became_active":"2024-04-26T21:19:16.484504+0000","last_became_peered":"2024-04-26T21:19:16.484504+0000","last_unstale":"2024-04-26T21:19:17.491852+0000","last_undegraded":"2024-04-26T21:19:17.491852+0000","last_fullsized":"2024-04-26T21:19:17.491852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492897+0000","last_change":"2024-04-26T21:19:16.487920+0000","last_active":"2024-04-26T21:19:16.492897+0000","last_peered":"2024-04-26T21:19:16.492897+0000","last_clean":"2024-04-26T21:19:16.492897+0000","last_became_active":"2024-04-26T21:19:16.487548+0000","last_became_peered":"2024-04-26T21:19:16.487548+0000","last_unstale":"2024-04-26T21:19:16.492897+0000","last_undegraded":"2024-04-26T21:19:16.492897+0000","last_fullsized":"2024-04-26T21:19:16.492897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491267+0000","last_change":"2024-04-26T21:19:16.485140+0000","last_active":"2024-04-26T21:19:17.491267+0000","last_peered":"2024-04-26T21:19:17.491267+0000","last_clean":"2024-04-26T21:19:17.491267+0000","last_became_active":"2024-04-26T21:19:16.484871+0000","last_became_peered":"2024-04-26T21:19:16.484871+0000","last_unstale":"2024-04-26T21:19:17.491267+0000","last_undegraded":"2024-04-26T21:19:17.491267+0000","last_fullsized":"2024-04-26T21:19:17.491267+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.494135+0000","last_change":"2024-04-26T21:19:16.491074+0000","last_active":"2024-04-26T21:19:16.494135+0000","last_peered":"2024-04-26T21:19:16.494135+0000","last_clean":"2024-04-26T21:19:16.494135+0000","last_became_active":"2024-04-26T21:19:16.490719+0000","last_became_peered":"2024-04-26T21:19:16.490719+0000","last_unstale":"2024-04-26T21:19:16.494135+0000","last_undegraded":"2024-04-26T21:19:16.494135+0000","last_fullsized":"2024-04-26T21:19:16.494135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:07.415274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499547+0000","last_change":"2024-04-26T21:19:16.496876+0000","last_active":"2024-04-26T21:19:16.499547+0000","last_peered":"2024-04-26T21:19:16.499547+0000","last_clean":"2024-04-26T21:19:16.499547+0000","last_became_active":"2024-04-26T21:19:16.496601+0000","last_became_peered":"2024-04-26T21:19:16.496601+0000","last_unstale":"2024-04-26T21:19:16.499547+0000","last_undegraded":"2024-04-26T21:19:16.499547+0000","last_fullsized":"2024-04-26T21:19:16.499547+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:31.534507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490487+0000","last_change":"2024-04-26T21:19:16.491979+0000","last_active":"2024-04-26T21:19:17.490487+0000","last_peered":"2024-04-26T21:19:17.490487+0000","last_clean":"2024-04-26T21:19:17.490487+0000","last_became_active":"2024-04-26T21:19:16.491072+0000","last_became_peered":"2024-04-26T21:19:16.491072+0000","last_unstale":"2024-04-26T21:19:17.490487+0000","last_undegraded":"2024-04-26T21:19:17.490487+0000","last_fullsized":"2024-04-26T21:19:17.490487+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:12.435136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491768+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:17.491768+0000","last_peered":"2024-04-26T21:19:17.491768+0000","last_clean":"2024-04-26T21:19:17.491768+0000","last_became_active":"2024-04-26T21:19:16.484220+0000","last_became_peered":"2024-04-26T21:19:16.484220+0000","last_unstale":"2024-04-26T21:19:17.491768+0000","last_undegraded":"2024-04-26T21:19:17.491768+0000","last_fullsized":"2024-04-26T21:19:17.491768+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:25.928632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499556+0000","last_change":"2024-04-26T21:19:16.494852+0000","last_active":"2024-04-26T21:19:16.499556+0000","last_peered":"2024-04-26T21:19:16.499556+0000","last_clean":"2024-04-26T21:19:16.499556+0000","last_became_active":"2024-04-26T21:19:16.494590+0000","last_became_peered":"2024-04-26T21:19:16.494590+0000","last_unstale":"2024-04-26T21:19:16.499556+0000","last_undegraded":"2024-04-26T21:19:16.499556+0000","last_fullsized":"2024-04-26T21:19:16.499556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:15.415456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673032+0000","last_change":"2024-04-26T21:19:16.484845+0000","last_active":"2024-04-26T21:19:17.673032+0000","last_peered":"2024-04-26T21:19:17.673032+0000","last_clean":"2024-04-26T21:19:17.673032+0000","last_became_active":"2024-04-26T21:19:16.484525+0000","last_became_peered":"2024-04-26T21:19:16.484525+0000","last_unstale":"2024-04-26T21:19:17.673032+0000","last_undegraded":"2024-04-26T21:19:17.673032+0000","last_fullsized":"2024-04-26T21:19:17.673032+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:31.425192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.499512+0000","last_change":"2024-04-26T21:19:16.497005+0000","last_active":"2024-04-26T21:19:16.499512+0000","last_peered":"2024-04-26T21:19:16.499512+0000","last_clean":"2024-04-26T21:19:16.499512+0000","last_became_active":"2024-04-26T21:19:16.496669+0000","last_became_peered":"2024-04-26T21:19:16.496669+0000","last_unstale":"2024-04-26T21:19:16.499512+0000","last_undegraded":"2024-04-26T21:19:16.499512+0000","last_fullsized":"2024-04-26T21:19:16.499512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.580462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490149+0000","last_change":"2024-04-26T21:19:16.500355+0000","last_active":"2024-04-26T21:19:17.490149+0000","last_peered":"2024-04-26T21:19:17.490149+0000","last_clean":"2024-04-26T21:19:17.490149+0000","last_became_active":"2024-04-26T21:19:16.500021+0000","last_became_peered":"2024-04-26T21:19:16.500021+0000","last_unstale":"2024-04-26T21:19:17.490149+0000","last_undegraded":"2024-04-26T21:19:17.490149+0000","last_fullsized":"2024-04-26T21:19:17.490149+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:01.461710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490414+0000","last_change":"2024-04-26T21:19:16.501698+0000","last_active":"2024-04-26T21:19:17.490414+0000","last_peered":"2024-04-26T21:19:17.490414+0000","last_clean":"2024-04-26T21:19:17.490414+0000","last_became_active":"2024-04-26T21:19:16.501426+0000","last_became_peered":"2024-04-26T21:19:16.501426+0000","last_unstale":"2024-04-26T21:19:17.490414+0000","last_undegraded":"2024-04-26T21:19:17.490414+0000","last_fullsized":"2024-04-26T21:19:17.490414+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:57.290656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673108+0000","last_change":"2024-04-26T21:19:15.478535+0000","last_active":"2024-04-26T21:19:17.673108+0000","last_peered":"2024-04-26T21:19:17.673108+0000","last_clean":"2024-04-26T21:19:17.673108+0000","last_became_active":"2024-04-26T21:19:15.478272+0000","last_became_peered":"2024-04-26T21:19:15.478272+0000","last_unstale":"2024-04-26T21:19:17.673108+0000","last_undegraded":"2024-04-26T21:19:17.673108+0000","last_fullsized":"2024-04-26T21:19:17.673108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492017+0000","last_change":"2024-04-26T21:19:16.488880+0000","last_active":"2024-04-26T21:19:16.492017+0000","last_peered":"2024-04-26T21:19:16.492017+0000","last_clean":"2024-04-26T21:19:16.492017+0000","last_became_active":"2024-04-26T21:19:16.488578+0000","last_became_peered":"2024-04-26T21:19:16.488578+0000","last_unstale":"2024-04-26T21:19:16.492017+0000","last_undegraded":"2024-04-26T21:19:16.492017+0000","last_fullsized":"2024-04-26T21:19:16.492017+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538442+0000","last_change":"2024-04-26T21:19:15.481825+0000","last_active":"2024-04-26T21:19:17.538442+0000","last_peered":"2024-04-26T21:19:17.538442+0000","last_clean":"2024-04-26T21:19:17.538442+0000","last_became_active":"2024-04-26T21:19:15.481529+0000","last_became_peered":"2024-04-26T21:19:15.481529+0000","last_unstale":"2024-04-26T21:19:17.538442+0000","last_undegraded":"2024-04-26T21:19:17.538442+0000","last_fullsized":"2024-04-26T21:19:17.538442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490592+0000","last_change":"2024-04-26T21:19:16.501161+0000","last_active":"2024-04-26T21:19:17.490592+0000","last_peered":"2024-04-26T21:19:17.490592+0000","last_clean":"2024-04-26T21:19:17.490592+0000","last_became_active":"2024-04-26T21:19:16.500862+0000","last_became_peered":"2024-04-26T21:19:16.500862+0000","last_unstale":"2024-04-26T21:19:17.490592+0000","last_undegraded":"2024-04-26T21:19:17.490592+0000","last_fullsized":"2024-04-26T21:19:17.490592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:08.968317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492403+0000","last_change":"2024-04-26T21:19:15.485103+0000","last_active":"2024-04-26T21:19:17.492403+0000","last_peered":"2024-04-26T21:19:17.492403+0000","last_clean":"2024-04-26T21:19:17.492403+0000","last_became_active":"2024-04-26T21:19:15.484770+0000","last_became_peered":"2024-04-26T21:19:15.484770+0000","last_unstale":"2024-04-26T21:19:17.492403+0000","last_undegraded":"2024-04-26T21:19:17.492403+0000","last_fullsized":"2024-04-26T21:19:17.492403+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:15.463835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538722+0000","last_change":"2024-04-26T21:19:16.491699+0000","last_active":"2024-04-26T21:19:17.538722+0000","last_peered":"2024-04-26T21:19:17.538722+0000","last_clean":"2024-04-26T21:19:17.538722+0000","last_became_active":"2024-04-26T21:19:16.491182+0000","last_became_peered":"2024-04-26T21:19:16.491182+0000","last_unstale":"2024-04-26T21:19:17.538722+0000","last_undegraded":"2024-04-26T21:19:17.538722+0000","last_fullsized":"2024-04-26T21:19:17.538722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492120+0000","last_change":"2024-04-26T21:19:15.483533+0000","last_active":"2024-04-26T21:19:17.492120+0000","last_peered":"2024-04-26T21:19:17.492120+0000","last_clean":"2024-04-26T21:19:17.492120+0000","last_became_active":"2024-04-26T21:19:15.483244+0000","last_became_peered":"2024-04-26T21:19:15.483244+0000","last_unstale":"2024-04-26T21:19:17.492120+0000","last_undegraded":"2024-04-26T21:19:17.492120+0000","last_fullsized":"2024-04-26T21:19:17.492120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:44.801378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502307+0000","last_change":"2024-04-26T21:19:16.499234+0000","last_active":"2024-04-26T21:19:16.502307+0000","last_peered":"2024-04-26T21:19:16.502307+0000","last_clean":"2024-04-26T21:19:16.502307+0000","last_became_active":"2024-04-26T21:19:16.498965+0000","last_became_peered":"2024-04-26T21:19:16.498965+0000","last_unstale":"2024-04-26T21:19:16.502307+0000","last_undegraded":"2024-04-26T21:19:16.502307+0000","last_fullsized":"2024-04-26T21:19:16.502307+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:38.866044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672911+0000","last_change":"2024-04-26T21:19:15.485012+0000","last_active":"2024-04-26T21:19:17.672911+0000","last_peered":"2024-04-26T21:19:17.672911+0000","last_clean":"2024-04-26T21:19:17.672911+0000","last_became_active":"2024-04-26T21:19:15.484711+0000","last_became_peered":"2024-04-26T21:19:15.484711+0000","last_unstale":"2024-04-26T21:19:17.672911+0000","last_undegraded":"2024-04-26T21:19:17.672911+0000","last_fullsized":"2024-04-26T21:19:17.672911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:58.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.501835+0000","last_change":"2024-04-26T21:19:16.498702+0000","last_active":"2024-04-26T21:19:16.501835+0000","last_peered":"2024-04-26T21:19:16.501835+0000","last_clean":"2024-04-26T21:19:16.501835+0000","last_became_active":"2024-04-26T21:19:16.498408+0000","last_became_peered":"2024-04-26T21:19:16.498408+0000","last_unstale":"2024-04-26T21:19:16.501835+0000","last_undegraded":"2024-04-26T21:19:16.501835+0000","last_fullsized":"2024-04-26T21:19:16.501835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:16.362609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491957+0000","last_change":"2024-04-26T21:19:15.491055+0000","last_active":"2024-04-26T21:19:17.491957+0000","last_peered":"2024-04-26T21:19:17.491957+0000","last_clean":"2024-04-26T21:19:17.491957+0000","last_became_active":"2024-04-26T21:19:15.490783+0000","last_became_peered":"2024-04-26T21:19:15.490783+0000","last_unstale":"2024-04-26T21:19:17.491957+0000","last_undegraded":"2024-04-26T21:19:17.491957+0000","last_fullsized":"2024-04-26T21:19:17.491957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:05.607960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503155+0000","last_change":"2024-04-26T21:19:16.501447+0000","last_active":"2024-04-26T21:19:16.503155+0000","last_peered":"2024-04-26T21:19:16.503155+0000","last_clean":"2024-04-26T21:19:16.503155+0000","last_became_active":"2024-04-26T21:19:16.501149+0000","last_became_peered":"2024-04-26T21:19:16.501149+0000","last_unstale":"2024-04-26T21:19:16.503155+0000","last_undegraded":"2024-04-26T21:19:16.503155+0000","last_fullsized":"2024-04-26T21:19:16.503155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:05:20.124617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479420+0000","last_change":"2024-04-26T21:19:15.490438+0000","last_active":"2024-04-26T21:19:16.479420+0000","last_peered":"2024-04-26T21:19:16.479420+0000","last_clean":"2024-04-26T21:19:16.479420+0000","last_became_active":"2024-04-26T21:19:15.490210+0000","last_became_peered":"2024-04-26T21:19:15.490210+0000","last_unstale":"2024-04-26T21:19:16.479420+0000","last_undegraded":"2024-04-26T21:19:16.479420+0000","last_fullsized":"2024-04-26T21:19:16.479420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.422988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672795+0000","last_change":"2024-04-26T21:19:16.491479+0000","last_active":"2024-04-26T21:19:17.672795+0000","last_peered":"2024-04-26T21:19:17.672795+0000","last_clean":"2024-04-26T21:19:17.672795+0000","last_became_active":"2024-04-26T21:19:16.491224+0000","last_became_peered":"2024-04-26T21:19:16.491224+0000","last_unstale":"2024-04-26T21:19:17.672795+0000","last_undegraded":"2024-04-26T21:19:17.672795+0000","last_fullsized":"2024-04-26T21:19:17.672795+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:51.573831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478367+0000","last_change":"2024-04-26T21:19:15.488327+0000","last_active":"2024-04-26T21:19:16.478367+0000","last_peered":"2024-04-26T21:19:16.478367+0000","last_clean":"2024-04-26T21:19:16.478367+0000","last_became_active":"2024-04-26T21:19:15.488039+0000","last_became_peered":"2024-04-26T21:19:15.488039+0000","last_unstale":"2024-04-26T21:19:16.478367+0000","last_undegraded":"2024-04-26T21:19:16.478367+0000","last_fullsized":"2024-04-26T21:19:16.478367+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:08.686002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673153+0000","last_change":"2024-04-26T21:19:16.481276+0000","last_active":"2024-04-26T21:19:17.673153+0000","last_peered":"2024-04-26T21:19:17.673153+0000","last_clean":"2024-04-26T21:19:17.673153+0000","last_became_active":"2024-04-26T21:19:16.480986+0000","last_became_peered":"2024-04-26T21:19:16.480986+0000","last_unstale":"2024-04-26T21:19:17.673153+0000","last_undegraded":"2024-04-26T21:19:17.673153+0000","last_fullsized":"2024-04-26T21:19:17.673153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492469+0000","last_change":"2024-04-26T21:19:15.484302+0000","last_active":"2024-04-26T21:19:17.492469+0000","last_peered":"2024-04-26T21:19:17.492469+0000","last_clean":"2024-04-26T21:19:17.492469+0000","last_became_active":"2024-04-26T21:19:15.483990+0000","last_became_peered":"2024-04-26T21:19:15.483990+0000","last_unstale":"2024-04-26T21:19:17.492469+0000","last_undegraded":"2024-04-26T21:19:17.492469+0000","last_fullsized":"2024-04-26T21:19:17.492469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:02.063693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492042+0000","last_change":"2024-04-26T21:19:16.489119+0000","last_active":"2024-04-26T21:19:16.492042+0000","last_peered":"2024-04-26T21:19:16.492042+0000","last_clean":"2024-04-26T21:19:16.492042+0000","last_became_active":"2024-04-26T21:19:16.488816+0000","last_became_peered":"2024-04-26T21:19:16.488816+0000","last_unstale":"2024-04-26T21:19:16.492042+0000","last_undegraded":"2024-04-26T21:19:16.492042+0000","last_fullsized":"2024-04-26T21:19:16.492042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:04.053313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491477+0000","last_change":"2024-04-26T21:19:15.492213+0000","last_active":"2024-04-26T21:19:17.491477+0000","last_peered":"2024-04-26T21:19:17.491477+0000","last_clean":"2024-04-26T21:19:17.491477+0000","last_became_active":"2024-04-26T21:19:15.491792+0000","last_became_peered":"2024-04-26T21:19:15.491792+0000","last_unstale":"2024-04-26T21:19:17.491477+0000","last_undegraded":"2024-04-26T21:19:17.491477+0000","last_fullsized":"2024-04-26T21:19:17.491477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491775+0000","last_change":"2024-04-26T21:19:16.496675+0000","last_active":"2024-04-26T21:19:17.491775+0000","last_peered":"2024-04-26T21:19:17.491775+0000","last_clean":"2024-04-26T21:19:17.491775+0000","last_became_active":"2024-04-26T21:19:16.496382+0000","last_became_peered":"2024-04-26T21:19:16.496382+0000","last_unstale":"2024-04-26T21:19:17.491775+0000","last_undegraded":"2024-04-26T21:19:17.491775+0000","last_fullsized":"2024-04-26T21:19:17.491775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502058+0000","last_change":"2024-04-26T21:19:15.493890+0000","last_active":"2024-04-26T21:19:16.502058+0000","last_peered":"2024-04-26T21:19:16.502058+0000","last_clean":"2024-04-26T21:19:16.502058+0000","last_became_active":"2024-04-26T21:19:15.493428+0000","last_became_peered":"2024-04-26T21:19:15.493428+0000","last_unstale":"2024-04-26T21:19:16.502058+0000","last_undegraded":"2024-04-26T21:19:16.502058+0000","last_fullsized":"2024-04-26T21:19:16.502058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:44.796649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673102+0000","last_change":"2024-04-26T21:19:16.489093+0000","last_active":"2024-04-26T21:19:17.673102+0000","last_peered":"2024-04-26T21:19:17.673102+0000","last_clean":"2024-04-26T21:19:17.673102+0000","last_became_active":"2024-04-26T21:19:16.488815+0000","last_became_peered":"2024-04-26T21:19:16.488815+0000","last_unstale":"2024-04-26T21:19:17.673102+0000","last_undegraded":"2024-04-26T21:19:17.673102+0000","last_fullsized":"2024-04-26T21:19:17.673102+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:43.157084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.504928+0000","last_change":"2024-04-26T21:19:15.489147+0000","last_active":"2024-04-26T21:19:16.504928+0000","last_peered":"2024-04-26T21:19:16.504928+0000","last_clean":"2024-04-26T21:19:16.504928+0000","last_became_active":"2024-04-26T21:19:15.488782+0000","last_became_peered":"2024-04-26T21:19:15.488782+0000","last_unstale":"2024-04-26T21:19:16.504928+0000","last_undegraded":"2024-04-26T21:19:16.504928+0000","last_fullsized":"2024-04-26T21:19:16.504928+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:12.490510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491886+0000","last_change":"2024-04-26T21:19:16.494938+0000","last_active":"2024-04-26T21:19:17.491886+0000","last_peered":"2024-04-26T21:19:17.491886+0000","last_clean":"2024-04-26T21:19:17.491886+0000","last_became_active":"2024-04-26T21:19:16.494586+0000","last_became_peered":"2024-04-26T21:19:16.494586+0000","last_unstale":"2024-04-26T21:19:17.491886+0000","last_undegraded":"2024-04-26T21:19:17.491886+0000","last_fullsized":"2024-04-26T21:19:17.491886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490242+0000","last_change":"2024-04-26T21:19:15.486297+0000","last_active":"2024-04-26T21:19:17.490242+0000","last_peered":"2024-04-26T21:19:17.490242+0000","last_clean":"2024-04-26T21:19:17.490242+0000","last_became_active":"2024-04-26T21:19:15.485985+0000","last_became_peered":"2024-04-26T21:19:15.485985+0000","last_unstale":"2024-04-26T21:19:17.490242+0000","last_undegraded":"2024-04-26T21:19:17.490242+0000","last_fullsized":"2024-04-26T21:19:17.490242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:34.344417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491584+0000","last_change":"2024-04-26T21:19:16.484940+0000","last_active":"2024-04-26T21:19:17.491584+0000","last_peered":"2024-04-26T21:19:17.491584+0000","last_clean":"2024-04-26T21:19:17.491584+0000","last_became_active":"2024-04-26T21:19:16.484650+0000","last_became_peered":"2024-04-26T21:19:16.484650+0000","last_unstale":"2024-04-26T21:19:17.491584+0000","last_undegraded":"2024-04-26T21:19:17.491584+0000","last_fullsized":"2024-04-26T21:19:17.491584+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:09.688206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479957+0000","last_change":"2024-04-26T21:19:15.486046+0000","last_active":"2024-04-26T21:19:16.479957+0000","last_peered":"2024-04-26T21:19:16.479957+0000","last_clean":"2024-04-26T21:19:16.479957+0000","last_became_active":"2024-04-26T21:19:15.485776+0000","last_became_peered":"2024-04-26T21:19:15.485776+0000","last_unstale":"2024-04-26T21:19:16.479957+0000","last_undegraded":"2024-04-26T21:19:16.479957+0000","last_fullsized":"2024-04-26T21:19:16.479957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.737449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491404+0000","last_change":"2024-04-26T21:19:16.497306+0000","last_active":"2024-04-26T21:19:17.491404+0000","last_peered":"2024-04-26T21:19:17.491404+0000","last_clean":"2024-04-26T21:19:17.491404+0000","last_became_active":"2024-04-26T21:19:16.497039+0000","last_became_peered":"2024-04-26T21:19:16.497039+0000","last_unstale":"2024-04-26T21:19:17.491404+0000","last_undegraded":"2024-04-26T21:19:17.491404+0000","last_fullsized":"2024-04-26T21:19:17.491404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:16.896268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492215+0000","last_change":"2024-04-26T21:19:15.490142+0000","last_active":"2024-04-26T21:19:17.492215+0000","last_peered":"2024-04-26T21:19:17.492215+0000","last_clean":"2024-04-26T21:19:17.492215+0000","last_became_active":"2024-04-26T21:19:15.489841+0000","last_became_peered":"2024-04-26T21:19:15.489841+0000","last_unstale":"2024-04-26T21:19:17.492215+0000","last_undegraded":"2024-04-26T21:19:17.492215+0000","last_fullsized":"2024-04-26T21:19:17.492215+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:06.856169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492590+0000","last_change":"2024-04-26T21:19:16.485067+0000","last_active":"2024-04-26T21:19:17.492590+0000","last_peered":"2024-04-26T21:19:17.492590+0000","last_clean":"2024-04-26T21:19:17.492590+0000","last_became_active":"2024-04-26T21:19:16.484588+0000","last_became_peered":"2024-04-26T21:19:16.484588+0000","last_unstale":"2024-04-26T21:19:17.492590+0000","last_undegraded":"2024-04-26T21:19:17.492590+0000","last_fullsized":"2024-04-26T21:19:17.492590+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:06.657912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496599+0000","last_change":"2024-04-26T21:19:15.488200+0000","last_active":"2024-04-26T21:19:16.496599+0000","last_peered":"2024-04-26T21:19:16.496599+0000","last_clean":"2024-04-26T21:19:16.496599+0000","last_became_active":"2024-04-26T21:19:15.487840+0000","last_became_peered":"2024-04-26T21:19:15.487840+0000","last_unstale":"2024-04-26T21:19:16.496599+0000","last_undegraded":"2024-04-26T21:19:16.496599+0000","last_fullsized":"2024-04-26T21:19:16.496599+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:32.520127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490417+0000","last_change":"2024-04-26T21:19:16.505313+0000","last_active":"2024-04-26T21:19:17.490417+0000","last_peered":"2024-04-26T21:19:17.490417+0000","last_clean":"2024-04-26T21:19:17.490417+0000","last_became_active":"2024-04-26T21:19:16.504970+0000","last_became_peered":"2024-04-26T21:19:16.504970+0000","last_unstale":"2024-04-26T21:19:17.490417+0000","last_undegraded":"2024-04-26T21:19:17.490417+0000","last_fullsized":"2024-04-26T21:19:17.490417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:16.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492522+0000","last_change":"2024-04-26T21:19:15.482948+0000","last_active":"2024-04-26T21:19:17.492522+0000","last_peered":"2024-04-26T21:19:17.492522+0000","last_clean":"2024-04-26T21:19:17.492522+0000","last_became_active":"2024-04-26T21:19:15.482636+0000","last_became_peered":"2024-04-26T21:19:15.482636+0000","last_unstale":"2024-04-26T21:19:17.492522+0000","last_undegraded":"2024-04-26T21:19:17.492522+0000","last_fullsized":"2024-04-26T21:19:17.492522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:04.356393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538613+0000","last_change":"2024-04-26T21:19:16.492696+0000","last_active":"2024-04-26T21:19:17.538613+0000","last_peered":"2024-04-26T21:19:17.538613+0000","last_clean":"2024-04-26T21:19:17.538613+0000","last_became_active":"2024-04-26T21:19:16.492396+0000","last_became_peered":"2024-04-26T21:19:16.492396+0000","last_unstale":"2024-04-26T21:19:17.538613+0000","last_undegraded":"2024-04-26T21:19:17.538613+0000","last_fullsized":"2024-04-26T21:19:17.538613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498743+0000","last_change":"2024-04-26T21:19:15.488486+0000","last_active":"2024-04-26T21:19:16.498743+0000","last_peered":"2024-04-26T21:19:16.498743+0000","last_clean":"2024-04-26T21:19:16.498743+0000","last_became_active":"2024-04-26T21:19:15.488221+0000","last_became_peered":"2024-04-26T21:19:15.488221+0000","last_unstale":"2024-04-26T21:19:16.498743+0000","last_undegraded":"2024-04-26T21:19:16.498743+0000","last_fullsized":"2024-04-26T21:19:16.498743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:30.760409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538790+0000","last_change":"2024-04-26T21:19:16.492285+0000","last_active":"2024-04-26T21:19:17.538790+0000","last_peered":"2024-04-26T21:19:17.538790+0000","last_clean":"2024-04-26T21:19:17.538790+0000","last_became_active":"2024-04-26T21:19:16.491992+0000","last_became_peered":"2024-04-26T21:19:16.491992+0000","last_unstale":"2024-04-26T21:19:17.538790+0000","last_undegraded":"2024-04-26T21:19:17.538790+0000","last_fullsized":"2024-04-26T21:19:17.538790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672950+0000","last_change":"2024-04-26T21:19:15.486458+0000","last_active":"2024-04-26T21:19:17.672950+0000","last_peered":"2024-04-26T21:19:17.672950+0000","last_clean":"2024-04-26T21:19:17.672950+0000","last_became_active":"2024-04-26T21:19:15.486127+0000","last_became_peered":"2024-04-26T21:19:15.486127+0000","last_unstale":"2024-04-26T21:19:17.672950+0000","last_undegraded":"2024-04-26T21:19:17.672950+0000","last_fullsized":"2024-04-26T21:19:17.672950+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:05.272243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538381+0000","last_change":"2024-04-26T21:19:16.495963+0000","last_active":"2024-04-26T21:19:17.538381+0000","last_peered":"2024-04-26T21:19:17.538381+0000","last_clean":"2024-04-26T21:19:17.538381+0000","last_became_active":"2024-04-26T21:19:16.495645+0000","last_became_peered":"2024-04-26T21:19:16.495645+0000","last_unstale":"2024-04-26T21:19:17.538381+0000","last_undegraded":"2024-04-26T21:19:17.538381+0000","last_fullsized":"2024-04-26T21:19:17.538381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490371+0000","last_change":"2024-04-26T21:19:15.487470+0000","last_active":"2024-04-26T21:19:17.490371+0000","last_peered":"2024-04-26T21:19:17.490371+0000","last_clean":"2024-04-26T21:19:17.490371+0000","last_became_active":"2024-04-26T21:19:15.487190+0000","last_became_peered":"2024-04-26T21:19:15.487190+0000","last_unstale":"2024-04-26T21:19:17.490371+0000","last_undegraded":"2024-04-26T21:19:17.490371+0000","last_fullsized":"2024-04-26T21:19:17.490371+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:52.279807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.491410+0000","last_change":"2024-04-26T21:19:16.487812+0000","last_active":"2024-04-26T21:19:16.491410+0000","last_peered":"2024-04-26T21:19:16.491410+0000","last_clean":"2024-04-26T21:19:16.491410+0000","last_became_active":"2024-04-26T21:19:16.487449+0000","last_became_peered":"2024-04-26T21:19:16.487449+0000","last_unstale":"2024-04-26T21:19:16.491410+0000","last_undegraded":"2024-04-26T21:19:16.491410+0000","last_fullsized":"2024-04-26T21:19:16.491410+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492009+0000","last_change":"2024-04-26T21:19:15.489410+0000","last_active":"2024-04-26T21:19:17.492009+0000","last_peered":"2024-04-26T21:19:17.492009+0000","last_clean":"2024-04-26T21:19:17.492009+0000","last_became_active":"2024-04-26T21:19:15.489083+0000","last_became_peered":"2024-04-26T21:19:15.489083+0000","last_unstale":"2024-04-26T21:19:17.492009+0000","last_undegraded":"2024-04-26T21:19:17.492009+0000","last_fullsized":"2024-04-26T21:19:17.492009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:50.451081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538376+0000","last_change":"2024-04-26T21:19:16.486905+0000","last_active":"2024-04-26T21:19:17.538376+0000","last_peered":"2024-04-26T21:19:17.538376+0000","last_clean":"2024-04-26T21:19:17.538376+0000","last_became_active":"2024-04-26T21:19:16.486575+0000","last_became_peered":"2024-04-26T21:19:16.486575+0000","last_unstale":"2024-04-26T21:19:17.538376+0000","last_undegraded":"2024-04-26T21:19:17.538376+0000","last_fullsized":"2024-04-26T21:19:17.538376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490212+0000","last_change":"2024-04-26T21:19:15.488247+0000","last_active":"2024-04-26T21:19:17.490212+0000","last_peered":"2024-04-26T21:19:17.490212+0000","last_clean":"2024-04-26T21:19:17.490212+0000","last_became_active":"2024-04-26T21:19:15.487933+0000","last_became_peered":"2024-04-26T21:19:15.487933+0000","last_unstale":"2024-04-26T21:19:17.490212+0000","last_undegraded":"2024-04-26T21:19:17.490212+0000","last_fullsized":"2024-04-26T21:19:17.490212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:16.427539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491418+0000","last_change":"2024-04-26T21:19:16.483829+0000","last_active":"2024-04-26T21:19:17.491418+0000","last_peered":"2024-04-26T21:19:17.491418+0000","last_clean":"2024-04-26T21:19:17.491418+0000","last_became_active":"2024-04-26T21:19:16.483486+0000","last_became_peered":"2024-04-26T21:19:16.483486+0000","last_unstale":"2024-04-26T21:19:17.491418+0000","last_undegraded":"2024-04-26T21:19:17.491418+0000","last_fullsized":"2024-04-26T21:19:17.491418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:34.802721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478470+0000","last_change":"2024-04-26T21:19:15.487580+0000","last_active":"2024-04-26T21:19:16.478470+0000","last_peered":"2024-04-26T21:19:16.478470+0000","last_clean":"2024-04-26T21:19:16.478470+0000","last_became_active":"2024-04-26T21:19:15.487307+0000","last_became_peered":"2024-04-26T21:19:15.487307+0000","last_unstale":"2024-04-26T21:19:16.478470+0000","last_undegraded":"2024-04-26T21:19:16.478470+0000","last_fullsized":"2024-04-26T21:19:16.478470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.926428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492041+0000","last_change":"2024-04-26T21:19:16.487487+0000","last_active":"2024-04-26T21:19:16.492041+0000","last_peered":"2024-04-26T21:19:16.492041+0000","last_clean":"2024-04-26T21:19:16.492041+0000","last_became_active":"2024-04-26T21:19:16.487194+0000","last_became_peered":"2024-04-26T21:19:16.487194+0000","last_unstale":"2024-04-26T21:19:16.492041+0000","last_undegraded":"2024-04-26T21:19:16.492041+0000","last_fullsized":"2024-04-26T21:19:16.492041+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.508890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491590+0000","last_change":"2024-04-26T21:19:15.490106+0000","last_active":"2024-04-26T21:19:17.491590+0000","last_peered":"2024-04-26T21:19:17.491590+0000","last_clean":"2024-04-26T21:19:17.491590+0000","last_became_active":"2024-04-26T21:19:15.489789+0000","last_became_peered":"2024-04-26T21:19:15.489789+0000","last_unstale":"2024-04-26T21:19:17.491590+0000","last_undegraded":"2024-04-26T21:19:17.491590+0000","last_fullsized":"2024-04-26T21:19:17.491590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.506686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.498668+0000","last_change":"2024-04-26T21:19:16.496139+0000","last_active":"2024-04-26T21:19:16.498668+0000","last_peered":"2024-04-26T21:19:16.498668+0000","last_clean":"2024-04-26T21:19:16.498668+0000","last_became_active":"2024-04-26T21:19:16.495785+0000","last_became_peered":"2024-04-26T21:19:16.495785+0000","last_unstale":"2024-04-26T21:19:16.498668+0000","last_undegraded":"2024-04-26T21:19:16.498668+0000","last_fullsized":"2024-04-26T21:19:16.498668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:47.719083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.477196+0000","last_change":"2024-04-26T21:19:15.476421+0000","last_active":"2024-04-26T21:19:17.477196+0000","last_peered":"2024-04-26T21:19:17.477196+0000","last_clean":"2024-04-26T21:19:17.477196+0000","last_became_active":"2024-04-26T21:19:15.476081+0000","last_became_peered":"2024-04-26T21:19:15.476081+0000","last_unstale":"2024-04-26T21:19:17.477196+0000","last_undegraded":"2024-04-26T21:19:17.477196+0000","last_fullsized":"2024-04-26T21:19:17.477196+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.502994+0000","last_change":"2024-04-26T21:19:16.502148+0000","last_active":"2024-04-26T21:19:16.502994+0000","last_peered":"2024-04-26T21:19:16.502994+0000","last_clean":"2024-04-26T21:19:16.502994+0000","last_became_active":"2024-04-26T21:19:16.500040+0000","last_became_peered":"2024-04-26T21:19:16.500040+0000","last_unstale":"2024-04-26T21:19:16.502994+0000","last_undegraded":"2024-04-26T21:19:16.502994+0000","last_fullsized":"2024-04-26T21:19:16.502994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:54.448349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479257+0000","last_change":"2024-04-26T21:19:15.481531+0000","last_active":"2024-04-26T21:19:16.479257+0000","last_peered":"2024-04-26T21:19:16.479257+0000","last_clean":"2024-04-26T21:19:16.479257+0000","last_became_active":"2024-04-26T21:19:15.481231+0000","last_became_peered":"2024-04-26T21:19:15.481231+0000","last_unstale":"2024-04-26T21:19:16.479257+0000","last_undegraded":"2024-04-26T21:19:16.479257+0000","last_fullsized":"2024-04-26T21:19:16.479257+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:04.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.492095+0000","last_change":"2024-04-26T21:19:16.488965+0000","last_active":"2024-04-26T21:19:16.492095+0000","last_peered":"2024-04-26T21:19:16.492095+0000","last_clean":"2024-04-26T21:19:16.492095+0000","last_became_active":"2024-04-26T21:19:16.488655+0000","last_became_peered":"2024-04-26T21:19:16.488655+0000","last_unstale":"2024-04-26T21:19:16.492095+0000","last_undegraded":"2024-04-26T21:19:16.492095+0000","last_fullsized":"2024-04-26T21:19:16.492095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:06.610164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491897+0000","last_change":"2024-04-26T21:19:15.493244+0000","last_active":"2024-04-26T21:19:17.491897+0000","last_peered":"2024-04-26T21:19:17.491897+0000","last_clean":"2024-04-26T21:19:17.491897+0000","last_became_active":"2024-04-26T21:19:15.493019+0000","last_became_peered":"2024-04-26T21:19:15.493019+0000","last_unstale":"2024-04-26T21:19:17.491897+0000","last_undegraded":"2024-04-26T21:19:17.491897+0000","last_fullsized":"2024-04-26T21:19:17.491897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:15.360405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496729+0000","last_change":"2024-04-26T21:19:16.493217+0000","last_active":"2024-04-26T21:19:16.496729+0000","last_peered":"2024-04-26T21:19:16.496729+0000","last_clean":"2024-04-26T21:19:16.496729+0000","last_became_active":"2024-04-26T21:19:16.492937+0000","last_became_peered":"2024-04-26T21:19:16.492937+0000","last_unstale":"2024-04-26T21:19:16.496729+0000","last_undegraded":"2024-04-26T21:19:16.496729+0000","last_fullsized":"2024-04-26T21:19:16.496729+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:31.762614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673072+0000","last_change":"2024-04-26T21:19:15.482604+0000","last_active":"2024-04-26T21:19:17.673072+0000","last_peered":"2024-04-26T21:19:17.673072+0000","last_clean":"2024-04-26T21:19:17.673072+0000","last_became_active":"2024-04-26T21:19:15.482320+0000","last_became_peered":"2024-04-26T21:19:15.482320+0000","last_unstale":"2024-04-26T21:19:17.673072+0000","last_undegraded":"2024-04-26T21:19:17.673072+0000","last_fullsized":"2024-04-26T21:19:17.673072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:24.748462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.496666+0000","last_change":"2024-04-26T21:19:16.493348+0000","last_active":"2024-04-26T21:19:16.496666+0000","last_peered":"2024-04-26T21:19:16.496666+0000","last_clean":"2024-04-26T21:19:16.496666+0000","last_became_active":"2024-04-26T21:19:16.492941+0000","last_became_peered":"2024-04-26T21:19:16.492941+0000","last_unstale":"2024-04-26T21:19:16.496666+0000","last_undegraded":"2024-04-26T21:19:16.496666+0000","last_fullsized":"2024-04-26T21:19:16.496666+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:08.739653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.479467+0000","last_change":"2024-04-26T21:19:15.489228+0000","last_active":"2024-04-26T21:19:16.479467+0000","last_peered":"2024-04-26T21:19:16.479467+0000","last_clean":"2024-04-26T21:19:16.479467+0000","last_became_active":"2024-04-26T21:19:15.488843+0000","last_became_peered":"2024-04-26T21:19:15.488843+0000","last_unstale":"2024-04-26T21:19:16.479467+0000","last_undegraded":"2024-04-26T21:19:16.479467+0000","last_fullsized":"2024-04-26T21:19:16.479467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:42.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538707+0000","last_change":"2024-04-26T21:19:16.491546+0000","last_active":"2024-04-26T21:19:17.538707+0000","last_peered":"2024-04-26T21:19:17.538707+0000","last_clean":"2024-04-26T21:19:17.538707+0000","last_became_active":"2024-04-26T21:19:16.491305+0000","last_became_peered":"2024-04-26T21:19:16.491305+0000","last_unstale":"2024-04-26T21:19:17.538707+0000","last_undegraded":"2024-04-26T21:19:17.538707+0000","last_fullsized":"2024-04-26T21:19:17.538707+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492104+0000","last_change":"2024-04-26T21:19:15.490249+0000","last_active":"2024-04-26T21:19:17.492104+0000","last_peered":"2024-04-26T21:19:17.492104+0000","last_clean":"2024-04-26T21:19:17.492104+0000","last_became_active":"2024-04-26T21:19:15.489962+0000","last_became_peered":"2024-04-26T21:19:15.489962+0000","last_unstale":"2024-04-26T21:19:17.492104+0000","last_undegraded":"2024-04-26T21:19:17.492104+0000","last_fullsized":"2024-04-26T21:19:17.492104+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.503309+0000","last_change":"2024-04-26T21:19:16.501636+0000","last_active":"2024-04-26T21:19:16.503309+0000","last_peered":"2024-04-26T21:19:16.503309+0000","last_clean":"2024-04-26T21:19:16.503309+0000","last_became_active":"2024-04-26T21:19:16.501326+0000","last_became_peered":"2024-04-26T21:19:16.501326+0000","last_unstale":"2024-04-26T21:19:16.503309+0000","last_undegraded":"2024-04-26T21:19:16.503309+0000","last_fullsized":"2024-04-26T21:19:16.503309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:10.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-26T21:19:16.478523+0000","last_change":"2024-04-26T21:19:15.486884+0000","last_active":"2024-04-26T21:19:16.478523+0000","last_peered":"2024-04-26T21:19:16.478523+0000","last_clean":"2024-04-26T21:19:16.478523+0000","last_became_active":"2024-04-26T21:19:15.486597+0000","last_became_peered":"2024-04-26T21:19:15.486597+0000","last_unstale":"2024-04-26T21:19:16.478523+0000","last_undegraded":"2024-04-26T21:19:16.478523+0000","last_fullsized":"2024-04-26T21:19:16.478523+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:16.558118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491321+0000","last_change":"2024-04-26T21:19:16.485658+0000","last_active":"2024-04-26T21:19:17.491321+0000","last_peered":"2024-04-26T21:19:17.491321+0000","last_clean":"2024-04-26T21:19:17.491321+0000","last_became_active":"2024-04-26T21:19:16.485382+0000","last_became_peered":"2024-04-26T21:19:16.485382+0000","last_unstale":"2024-04-26T21:19:17.491321+0000","last_undegraded":"2024-04-26T21:19:17.491321+0000","last_fullsized":"2024-04-26T21:19:17.491321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672961+0000","last_change":"2024-04-26T21:19:15.480711+0000","last_active":"2024-04-26T21:19:17.672961+0000","last_peered":"2024-04-26T21:19:17.672961+0000","last_clean":"2024-04-26T21:19:17.672961+0000","last_became_active":"2024-04-26T21:19:15.480405+0000","last_became_peered":"2024-04-26T21:19:15.480405+0000","last_unstale":"2024-04-26T21:19:17.672961+0000","last_undegraded":"2024-04-26T21:19:17.672961+0000","last_fullsized":"2024-04-26T21:19:17.672961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:00.640043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538749+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:19:17.538749+0000","last_peered":"2024-04-26T21:19:17.538749+0000","last_clean":"2024-04-26T21:19:17.538749+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:19:17.538749+0000","last_undegraded":"2024-04-26T21:19:17.538749+0000","last_fullsized":"2024-04-26T21:19:17.538749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509534,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":1161038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":570124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":576958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":570054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":571736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1162146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":576478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:19:24.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:19:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.071+0000 7f57813bf640 1 Processor -- start 2024-04-26T21:19:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.071+0000 7f57813bf640 1 -- start start 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.071+0000 7f57813bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f577c15f370 0x7f577c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f57813bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f57813bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f577c0d03a0 con 0x7f577c15f750 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f57813bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f577c0b4d10 con 0x7f577c15f370 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f57813bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f577c0d0680 con 0x7f577c0d3ee0 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f577c15f370 0x7f577c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f577c15f370 0x7f577c0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57230/0 (socket says 172.21.15.40:57230) 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 -- 172.21.15.40:0/3688728924 learned_addr learned my addr 172.21.15.40:0/3688728924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577b7fe640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 -- 172.21.15.40:0/3688728924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 msgr2=0x7f577c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 -- 172.21.15.40:0/3688728924 >> v1:172.21.15.184:6789/0 conn(0x7f577c15f750 legacy=0x7f577c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577affd640 1 -- 172.21.15.40:0/3688728924 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c0d0960 con 0x7f577c15f370 2024-04-26T21:19:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.072+0000 7f577b7fe640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f577affd640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c0cfd70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f576c009870 tx=0x7f576c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db590525f697ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f577a7fc640 1 -- 172.21.15.40:0/3688728924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c037070 con 0x7f577c15f370 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f577a7fc640 1 -- 172.21.15.40:0/3688728924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576c032440 con 0x7f577c15f370 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f577a7fc640 1 -- 172.21.15.40:0/3688728924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c032810 con 0x7f577c15f370 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f57813bf640 1 -- 172.21.15.40:0/3688728924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 msgr2=0x7f577c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.073+0000 7f57813bf640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c0cfd70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f576c009870 tx=0x7f576c02a210 comp rx=0 tx=0).stop 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 -- 172.21.15.40:0/3688728924 shutdown_connections 2024-04-26T21:19:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 --2- 172.21.15.40:0/3688728924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c0cfd70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 -- 172.21.15.40:0/3688728924 >> 172.21.15.40:0/3688728924 conn(0x7f577c08ec10 msgr2=0x7f577c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 -- 172.21.15.40:0/3688728924 shutdown_connections 2024-04-26T21:19:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.074+0000 7f57813bf640 1 -- 172.21.15.40:0/3688728924 wait complete. 2024-04-26T21:19:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 Processor -- start 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 -- start start 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f577c15f750 0x7f577c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0b51c0 con 0x7f577c15f370 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0d0f20 con 0x7f577c15f750 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.075+0000 7f57813bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f577c0b5340 con 0x7f577c0d3ee0 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57242/0 (socket says 172.21.15.40:57242) 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 -- 172.21.15.40:0/3811951843 learned_addr learned my addr 172.21.15.40:0/3811951843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f577affd640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f577b7fe640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f577c15f750 0x7f577c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 -- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 msgr2=0x7f577c0c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 -- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f577c15f750 msgr2=0x7f577c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f577c15f750 0x7f577c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 -- 172.21.15.40:0/3811951843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c160900 con 0x7f577c15f370 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f5773fff640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f576400b3f0 tx=0x7f576400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764015020 con 0x7f577c15f370 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5764011830 con 0x7f577c15f370 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c0c9260 con 0x7f577c15f370 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764011c20 con 0x7f577c15f370 2024-04-26T21:19:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.076+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c0c97e0 con 0x7f577c15f370 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.078+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c06a290 con 0x7f577c15f370 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.078+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5764018020 con 0x7f577c15f370 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.078+0000 7f57737fe640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.079+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f576408b6c0 con 0x7f577c15f370 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.079+0000 7f577affd640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.079+0000 7f577affd640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 0x7f575c06edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f576c034000 tx=0x7f576c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.083+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f576401f030 con 0x7f577c15f370 2024-04-26T21:19:25.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.263+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f577c0d14e0 con 0x7f575c06c8f0 2024-04-26T21:19:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.270+0000 7f57737fe640 1 -- 172.21.15.40:0/3811951843 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252407 (secure 0 0 0) 0x7f576c07fa10 con 0x7f575c06c8f0 2024-04-26T21:19:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:25.275 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:19:25.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 msgr2=0x7f575c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 0x7f575c06edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f576c034000 tx=0x7f576c02e000 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 msgr2=0x7f577c162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f576400b3f0 tx=0x7f576400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 shutdown_connections 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f577c15f750 0x7f577c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f577c15f370 0x7f577c162350 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 --2- 172.21.15.40:0/3811951843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f577c0d3ee0 0x7f577c0c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 >> 172.21.15.40:0/3811951843 conn(0x7f577c08ec10 msgr2=0x7f577c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 shutdown_connections 2024-04-26T21:19:25.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.273+0000 7f57813bf640 1 -- 172.21.15.40:0/3811951843 wait complete. 2024-04-26T21:19:25.285 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-26T21:19:25.026579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265352,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709368,"statfs":{"total":773094113280,"available":772822392832,"internally_reserved":0,"allocated":12034048,"data_stored":5796388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.555123"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226780+0000","last_change":"2024-04-26T21:19:16.496097+0000","last_active":"2024-04-26T21:19:18.226780+0000","last_peered":"2024-04-26T21:19:18.226780+0000","last_clean":"2024-04-26T21:19:18.226780+0000","last_became_active":"2024-04-26T21:19:16.494023+0000","last_became_peered":"2024-04-26T21:19:16.494023+0000","last_unstale":"2024-04-26T21:19:18.226780+0000","last_undegraded":"2024-04-26T21:19:18.226780+0000","last_fullsized":"2024-04-26T21:19:18.226780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:52.906467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226931+0000","last_change":"2024-04-26T21:19:16.498688+0000","last_active":"2024-04-26T21:19:18.226931+0000","last_peered":"2024-04-26T21:19:18.226931+0000","last_clean":"2024-04-26T21:19:18.226931+0000","last_became_active":"2024-04-26T21:19:16.498359+0000","last_became_peered":"2024-04-26T21:19:16.498359+0000","last_unstale":"2024-04-26T21:19:18.226931+0000","last_undegraded":"2024-04-26T21:19:18.226931+0000","last_fullsized":"2024-04-26T21:19:18.226931+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538503+0000","last_change":"2024-04-26T21:19:16.496173+0000","last_active":"2024-04-26T21:19:17.538503+0000","last_peered":"2024-04-26T21:19:17.538503+0000","last_clean":"2024-04-26T21:19:17.538503+0000","last_became_active":"2024-04-26T21:19:16.495826+0000","last_became_peered":"2024-04-26T21:19:16.495826+0000","last_unstale":"2024-04-26T21:19:17.538503+0000","last_undegraded":"2024-04-26T21:19:17.538503+0000","last_fullsized":"2024-04-26T21:19:17.538503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:13.492714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226394+0000","last_change":"2024-04-26T21:19:16.495999+0000","last_active":"2024-04-26T21:19:18.226394+0000","last_peered":"2024-04-26T21:19:18.226394+0000","last_clean":"2024-04-26T21:19:18.226394+0000","last_became_active":"2024-04-26T21:19:16.495621+0000","last_became_peered":"2024-04-26T21:19:16.495621+0000","last_unstale":"2024-04-26T21:19:18.226394+0000","last_undegraded":"2024-04-26T21:19:18.226394+0000","last_fullsized":"2024-04-26T21:19:18.226394+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:28.345835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492356+0000","last_change":"2024-04-26T21:19:16.482865+0000","last_active":"2024-04-26T21:19:17.492356+0000","last_peered":"2024-04-26T21:19:17.492356+0000","last_clean":"2024-04-26T21:19:17.492356+0000","last_became_active":"2024-04-26T21:19:16.482588+0000","last_became_peered":"2024-04-26T21:19:16.482588+0000","last_unstale":"2024-04-26T21:19:17.492356+0000","last_undegraded":"2024-04-26T21:19:17.492356+0000","last_fullsized":"2024-04-26T21:19:17.492356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:23.086160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490188+0000","last_change":"2024-04-26T21:19:16.500088+0000","last_active":"2024-04-26T21:19:17.490188+0000","last_peered":"2024-04-26T21:19:17.490188+0000","last_clean":"2024-04-26T21:19:17.490188+0000","last_became_active":"2024-04-26T21:19:16.499783+0000","last_became_peered":"2024-04-26T21:19:16.499783+0000","last_unstale":"2024-04-26T21:19:17.490188+0000","last_undegraded":"2024-04-26T21:19:17.490188+0000","last_fullsized":"2024-04-26T21:19:17.490188+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.343352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672936+0000","last_change":"2024-04-26T21:19:16.488443+0000","last_active":"2024-04-26T21:19:17.672936+0000","last_peered":"2024-04-26T21:19:17.672936+0000","last_clean":"2024-04-26T21:19:17.672936+0000","last_became_active":"2024-04-26T21:19:16.488149+0000","last_became_peered":"2024-04-26T21:19:16.488149+0000","last_unstale":"2024-04-26T21:19:17.672936+0000","last_undegraded":"2024-04-26T21:19:17.672936+0000","last_fullsized":"2024-04-26T21:19:17.672936+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:55.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538580+0000","last_change":"2024-04-26T21:19:16.498705+0000","last_active":"2024-04-26T21:19:17.538580+0000","last_peered":"2024-04-26T21:19:17.538580+0000","last_clean":"2024-04-26T21:19:17.538580+0000","last_became_active":"2024-04-26T21:19:16.498371+0000","last_became_peered":"2024-04-26T21:19:16.498371+0000","last_unstale":"2024-04-26T21:19:17.538580+0000","last_undegraded":"2024-04-26T21:19:17.538580+0000","last_fullsized":"2024-04-26T21:19:17.538580+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:33.522331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226750+0000","last_change":"2024-04-26T21:19:16.489425+0000","last_active":"2024-04-26T21:19:18.226750+0000","last_peered":"2024-04-26T21:19:18.226750+0000","last_clean":"2024-04-26T21:19:18.226750+0000","last_became_active":"2024-04-26T21:19:16.489130+0000","last_became_peered":"2024-04-26T21:19:16.489130+0000","last_unstale":"2024-04-26T21:19:18.226750+0000","last_undegraded":"2024-04-26T21:19:18.226750+0000","last_fullsized":"2024-04-26T21:19:18.226750+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:05.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226573+0000","last_change":"2024-04-26T21:19:16.485878+0000","last_active":"2024-04-26T21:19:18.226573+0000","last_peered":"2024-04-26T21:19:18.226573+0000","last_clean":"2024-04-26T21:19:18.226573+0000","last_became_active":"2024-04-26T21:19:16.485592+0000","last_became_peered":"2024-04-26T21:19:16.485592+0000","last_unstale":"2024-04-26T21:19:18.226573+0000","last_undegraded":"2024-04-26T21:19:18.226573+0000","last_fullsized":"2024-04-26T21:19:18.226573+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:54.847553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226799+0000","last_change":"2024-04-26T21:19:16.492337+0000","last_active":"2024-04-26T21:19:18.226799+0000","last_peered":"2024-04-26T21:19:18.226799+0000","last_clean":"2024-04-26T21:19:18.226799+0000","last_became_active":"2024-04-26T21:19:16.492009+0000","last_became_peered":"2024-04-26T21:19:16.492009+0000","last_unstale":"2024-04-26T21:19:18.226799+0000","last_undegraded":"2024-04-26T21:19:18.226799+0000","last_fullsized":"2024-04-26T21:19:18.226799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:40.818286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226751+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:18.226751+0000","last_peered":"2024-04-26T21:19:18.226751+0000","last_clean":"2024-04-26T21:19:18.226751+0000","last_became_active":"2024-04-26T21:19:16.484183+0000","last_became_peered":"2024-04-26T21:19:16.484183+0000","last_unstale":"2024-04-26T21:19:18.226751+0000","last_undegraded":"2024-04-26T21:19:18.226751+0000","last_fullsized":"2024-04-26T21:19:18.226751+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:42.843495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538678+0000","last_change":"2024-04-26T21:19:16.495059+0000","last_active":"2024-04-26T21:19:17.538678+0000","last_peered":"2024-04-26T21:19:17.538678+0000","last_clean":"2024-04-26T21:19:17.538678+0000","last_became_active":"2024-04-26T21:19:16.494780+0000","last_became_peered":"2024-04-26T21:19:16.494780+0000","last_unstale":"2024-04-26T21:19:17.538678+0000","last_undegraded":"2024-04-26T21:19:17.538678+0000","last_fullsized":"2024-04-26T21:19:17.538678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:42.189431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538257+0000","last_change":"2024-04-26T21:19:16.490955+0000","last_active":"2024-04-26T21:19:17.538257+0000","last_peered":"2024-04-26T21:19:17.538257+0000","last_clean":"2024-04-26T21:19:17.538257+0000","last_became_active":"2024-04-26T21:19:16.490657+0000","last_became_peered":"2024-04-26T21:19:16.490657+0000","last_unstale":"2024-04-26T21:19:17.538257+0000","last_undegraded":"2024-04-26T21:19:17.538257+0000","last_fullsized":"2024-04-26T21:19:17.538257+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490182+0000","last_change":"2024-04-26T21:19:16.500703+0000","last_active":"2024-04-26T21:19:17.490182+0000","last_peered":"2024-04-26T21:19:17.490182+0000","last_clean":"2024-04-26T21:19:17.490182+0000","last_became_active":"2024-04-26T21:19:16.500372+0000","last_became_peered":"2024-04-26T21:19:16.500372+0000","last_unstale":"2024-04-26T21:19:17.490182+0000","last_undegraded":"2024-04-26T21:19:17.490182+0000","last_fullsized":"2024-04-26T21:19:17.490182+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:28.187760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.225851+0000","last_change":"2024-04-26T21:19:16.487462+0000","last_active":"2024-04-26T21:19:18.225851+0000","last_peered":"2024-04-26T21:19:18.225851+0000","last_clean":"2024-04-26T21:19:18.225851+0000","last_became_active":"2024-04-26T21:19:16.486946+0000","last_became_peered":"2024-04-26T21:19:16.486946+0000","last_unstale":"2024-04-26T21:19:18.225851+0000","last_undegraded":"2024-04-26T21:19:18.225851+0000","last_fullsized":"2024-04-26T21:19:18.225851+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:00.841897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226845+0000","last_change":"2024-04-26T21:19:16.491720+0000","last_active":"2024-04-26T21:19:18.226845+0000","last_peered":"2024-04-26T21:19:18.226845+0000","last_clean":"2024-04-26T21:19:18.226845+0000","last_became_active":"2024-04-26T21:19:16.491369+0000","last_became_peered":"2024-04-26T21:19:16.491369+0000","last_unstale":"2024-04-26T21:19:18.226845+0000","last_undegraded":"2024-04-26T21:19:18.226845+0000","last_fullsized":"2024-04-26T21:19:18.226845+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:08.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226284+0000","last_change":"2024-04-26T21:19:16.496447+0000","last_active":"2024-04-26T21:19:18.226284+0000","last_peered":"2024-04-26T21:19:18.226284+0000","last_clean":"2024-04-26T21:19:18.226284+0000","last_became_active":"2024-04-26T21:19:16.496204+0000","last_became_peered":"2024-04-26T21:19:16.496204+0000","last_unstale":"2024-04-26T21:19:18.226284+0000","last_undegraded":"2024-04-26T21:19:18.226284+0000","last_fullsized":"2024-04-26T21:19:18.226284+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:30.530823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538653+0000","last_change":"2024-04-26T21:19:16.491438+0000","last_active":"2024-04-26T21:19:17.538653+0000","last_peered":"2024-04-26T21:19:17.538653+0000","last_clean":"2024-04-26T21:19:17.538653+0000","last_became_active":"2024-04-26T21:19:16.490909+0000","last_became_peered":"2024-04-26T21:19:16.490909+0000","last_unstale":"2024-04-26T21:19:17.538653+0000","last_undegraded":"2024-04-26T21:19:17.538653+0000","last_fullsized":"2024-04-26T21:19:17.538653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:25.474401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226454+0000","last_change":"2024-04-26T21:19:16.491473+0000","last_active":"2024-04-26T21:19:18.226454+0000","last_peered":"2024-04-26T21:19:18.226454+0000","last_clean":"2024-04-26T21:19:18.226454+0000","last_became_active":"2024-04-26T21:19:16.491070+0000","last_became_peered":"2024-04-26T21:19:16.491070+0000","last_unstale":"2024-04-26T21:19:18.226454+0000","last_undegraded":"2024-04-26T21:19:18.226454+0000","last_fullsized":"2024-04-26T21:19:18.226454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:30.530823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226495+0000","last_change":"2024-04-26T21:19:16.499382+0000","last_active":"2024-04-26T21:19:18.226495+0000","last_peered":"2024-04-26T21:19:18.226495+0000","last_clean":"2024-04-26T21:19:18.226495+0000","last_became_active":"2024-04-26T21:19:16.499064+0000","last_became_peered":"2024-04-26T21:19:16.499064+0000","last_unstale":"2024-04-26T21:19:18.226495+0000","last_undegraded":"2024-04-26T21:19:18.226495+0000","last_fullsized":"2024-04-26T21:19:18.226495+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:21.856772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491852+0000","last_change":"2024-04-26T21:19:16.485418+0000","last_active":"2024-04-26T21:19:17.491852+0000","last_peered":"2024-04-26T21:19:17.491852+0000","last_clean":"2024-04-26T21:19:17.491852+0000","last_became_active":"2024-04-26T21:19:16.484504+0000","last_became_peered":"2024-04-26T21:19:16.484504+0000","last_unstale":"2024-04-26T21:19:17.491852+0000","last_undegraded":"2024-04-26T21:19:17.491852+0000","last_fullsized":"2024-04-26T21:19:17.491852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226007+0000","last_change":"2024-04-26T21:19:16.487920+0000","last_active":"2024-04-26T21:19:18.226007+0000","last_peered":"2024-04-26T21:19:18.226007+0000","last_clean":"2024-04-26T21:19:18.226007+0000","last_became_active":"2024-04-26T21:19:16.487548+0000","last_became_peered":"2024-04-26T21:19:16.487548+0000","last_unstale":"2024-04-26T21:19:18.226007+0000","last_undegraded":"2024-04-26T21:19:18.226007+0000","last_fullsized":"2024-04-26T21:19:18.226007+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:04.053313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491267+0000","last_change":"2024-04-26T21:19:16.485140+0000","last_active":"2024-04-26T21:19:17.491267+0000","last_peered":"2024-04-26T21:19:17.491267+0000","last_clean":"2024-04-26T21:19:17.491267+0000","last_became_active":"2024-04-26T21:19:16.484871+0000","last_became_peered":"2024-04-26T21:19:16.484871+0000","last_unstale":"2024-04-26T21:19:17.491267+0000","last_undegraded":"2024-04-26T21:19:17.491267+0000","last_fullsized":"2024-04-26T21:19:17.491267+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226914+0000","last_change":"2024-04-26T21:19:16.491074+0000","last_active":"2024-04-26T21:19:18.226914+0000","last_peered":"2024-04-26T21:19:18.226914+0000","last_clean":"2024-04-26T21:19:18.226914+0000","last_became_active":"2024-04-26T21:19:16.490719+0000","last_became_peered":"2024-04-26T21:19:16.490719+0000","last_unstale":"2024-04-26T21:19:18.226914+0000","last_undegraded":"2024-04-26T21:19:18.226914+0000","last_fullsized":"2024-04-26T21:19:18.226914+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:48.682264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226708+0000","last_change":"2024-04-26T21:19:16.496876+0000","last_active":"2024-04-26T21:19:18.226708+0000","last_peered":"2024-04-26T21:19:18.226708+0000","last_clean":"2024-04-26T21:19:18.226708+0000","last_became_active":"2024-04-26T21:19:16.496601+0000","last_became_peered":"2024-04-26T21:19:16.496601+0000","last_unstale":"2024-04-26T21:19:18.226708+0000","last_undegraded":"2024-04-26T21:19:18.226708+0000","last_fullsized":"2024-04-26T21:19:18.226708+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:40.746376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490487+0000","last_change":"2024-04-26T21:19:16.491979+0000","last_active":"2024-04-26T21:19:17.490487+0000","last_peered":"2024-04-26T21:19:17.490487+0000","last_clean":"2024-04-26T21:19:17.490487+0000","last_became_active":"2024-04-26T21:19:16.491072+0000","last_became_peered":"2024-04-26T21:19:16.491072+0000","last_unstale":"2024-04-26T21:19:17.490487+0000","last_undegraded":"2024-04-26T21:19:17.490487+0000","last_fullsized":"2024-04-26T21:19:17.490487+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:12.435136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491768+0000","last_change":"2024-04-26T21:19:16.484472+0000","last_active":"2024-04-26T21:19:17.491768+0000","last_peered":"2024-04-26T21:19:17.491768+0000","last_clean":"2024-04-26T21:19:17.491768+0000","last_became_active":"2024-04-26T21:19:16.484220+0000","last_became_peered":"2024-04-26T21:19:16.484220+0000","last_unstale":"2024-04-26T21:19:17.491768+0000","last_undegraded":"2024-04-26T21:19:17.491768+0000","last_fullsized":"2024-04-26T21:19:17.491768+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:25.928632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226758+0000","last_change":"2024-04-26T21:19:16.494852+0000","last_active":"2024-04-26T21:19:18.226758+0000","last_peered":"2024-04-26T21:19:18.226758+0000","last_clean":"2024-04-26T21:19:18.226758+0000","last_became_active":"2024-04-26T21:19:16.494590+0000","last_became_peered":"2024-04-26T21:19:16.494590+0000","last_unstale":"2024-04-26T21:19:18.226758+0000","last_undegraded":"2024-04-26T21:19:18.226758+0000","last_fullsized":"2024-04-26T21:19:18.226758+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:21.647183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673032+0000","last_change":"2024-04-26T21:19:16.484845+0000","last_active":"2024-04-26T21:19:17.673032+0000","last_peered":"2024-04-26T21:19:17.673032+0000","last_clean":"2024-04-26T21:19:17.673032+0000","last_became_active":"2024-04-26T21:19:16.484525+0000","last_became_peered":"2024-04-26T21:19:16.484525+0000","last_unstale":"2024-04-26T21:19:17.673032+0000","last_undegraded":"2024-04-26T21:19:17.673032+0000","last_fullsized":"2024-04-26T21:19:17.673032+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:31.425192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.225988+0000","last_change":"2024-04-26T21:19:16.497005+0000","last_active":"2024-04-26T21:19:18.225988+0000","last_peered":"2024-04-26T21:19:18.225988+0000","last_clean":"2024-04-26T21:19:18.225988+0000","last_became_active":"2024-04-26T21:19:16.496669+0000","last_became_peered":"2024-04-26T21:19:16.496669+0000","last_unstale":"2024-04-26T21:19:18.225988+0000","last_undegraded":"2024-04-26T21:19:18.225988+0000","last_fullsized":"2024-04-26T21:19:18.225988+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:48.187818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490149+0000","last_change":"2024-04-26T21:19:16.500355+0000","last_active":"2024-04-26T21:19:17.490149+0000","last_peered":"2024-04-26T21:19:17.490149+0000","last_clean":"2024-04-26T21:19:17.490149+0000","last_became_active":"2024-04-26T21:19:16.500021+0000","last_became_peered":"2024-04-26T21:19:16.500021+0000","last_unstale":"2024-04-26T21:19:17.490149+0000","last_undegraded":"2024-04-26T21:19:17.490149+0000","last_fullsized":"2024-04-26T21:19:17.490149+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:01.461710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490414+0000","last_change":"2024-04-26T21:19:16.501698+0000","last_active":"2024-04-26T21:19:17.490414+0000","last_peered":"2024-04-26T21:19:17.490414+0000","last_clean":"2024-04-26T21:19:17.490414+0000","last_became_active":"2024-04-26T21:19:16.501426+0000","last_became_peered":"2024-04-26T21:19:16.501426+0000","last_unstale":"2024-04-26T21:19:17.490414+0000","last_undegraded":"2024-04-26T21:19:17.490414+0000","last_fullsized":"2024-04-26T21:19:17.490414+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:57.290656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673108+0000","last_change":"2024-04-26T21:19:15.478535+0000","last_active":"2024-04-26T21:19:17.673108+0000","last_peered":"2024-04-26T21:19:17.673108+0000","last_clean":"2024-04-26T21:19:17.673108+0000","last_became_active":"2024-04-26T21:19:15.478272+0000","last_became_peered":"2024-04-26T21:19:15.478272+0000","last_unstale":"2024-04-26T21:19:17.673108+0000","last_undegraded":"2024-04-26T21:19:17.673108+0000","last_fullsized":"2024-04-26T21:19:17.673108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226782+0000","last_change":"2024-04-26T21:19:16.488880+0000","last_active":"2024-04-26T21:19:18.226782+0000","last_peered":"2024-04-26T21:19:18.226782+0000","last_clean":"2024-04-26T21:19:18.226782+0000","last_became_active":"2024-04-26T21:19:16.488578+0000","last_became_peered":"2024-04-26T21:19:16.488578+0000","last_unstale":"2024-04-26T21:19:18.226782+0000","last_undegraded":"2024-04-26T21:19:18.226782+0000","last_fullsized":"2024-04-26T21:19:18.226782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:21.647183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538442+0000","last_change":"2024-04-26T21:19:15.481825+0000","last_active":"2024-04-26T21:19:17.538442+0000","last_peered":"2024-04-26T21:19:17.538442+0000","last_clean":"2024-04-26T21:19:17.538442+0000","last_became_active":"2024-04-26T21:19:15.481529+0000","last_became_peered":"2024-04-26T21:19:15.481529+0000","last_unstale":"2024-04-26T21:19:17.538442+0000","last_undegraded":"2024-04-26T21:19:17.538442+0000","last_fullsized":"2024-04-26T21:19:17.538442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490592+0000","last_change":"2024-04-26T21:19:16.501161+0000","last_active":"2024-04-26T21:19:17.490592+0000","last_peered":"2024-04-26T21:19:17.490592+0000","last_clean":"2024-04-26T21:19:17.490592+0000","last_became_active":"2024-04-26T21:19:16.500862+0000","last_became_peered":"2024-04-26T21:19:16.500862+0000","last_unstale":"2024-04-26T21:19:17.490592+0000","last_undegraded":"2024-04-26T21:19:17.490592+0000","last_fullsized":"2024-04-26T21:19:17.490592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:08.968317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492403+0000","last_change":"2024-04-26T21:19:15.485103+0000","last_active":"2024-04-26T21:19:17.492403+0000","last_peered":"2024-04-26T21:19:17.492403+0000","last_clean":"2024-04-26T21:19:17.492403+0000","last_became_active":"2024-04-26T21:19:15.484770+0000","last_became_peered":"2024-04-26T21:19:15.484770+0000","last_unstale":"2024-04-26T21:19:17.492403+0000","last_undegraded":"2024-04-26T21:19:17.492403+0000","last_fullsized":"2024-04-26T21:19:17.492403+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:15.463835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538722+0000","last_change":"2024-04-26T21:19:16.491699+0000","last_active":"2024-04-26T21:19:17.538722+0000","last_peered":"2024-04-26T21:19:17.538722+0000","last_clean":"2024-04-26T21:19:17.538722+0000","last_became_active":"2024-04-26T21:19:16.491182+0000","last_became_peered":"2024-04-26T21:19:16.491182+0000","last_unstale":"2024-04-26T21:19:17.538722+0000","last_undegraded":"2024-04-26T21:19:17.538722+0000","last_fullsized":"2024-04-26T21:19:17.538722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:47.766830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492120+0000","last_change":"2024-04-26T21:19:15.483533+0000","last_active":"2024-04-26T21:19:17.492120+0000","last_peered":"2024-04-26T21:19:17.492120+0000","last_clean":"2024-04-26T21:19:17.492120+0000","last_became_active":"2024-04-26T21:19:15.483244+0000","last_became_peered":"2024-04-26T21:19:15.483244+0000","last_unstale":"2024-04-26T21:19:17.492120+0000","last_undegraded":"2024-04-26T21:19:17.492120+0000","last_fullsized":"2024-04-26T21:19:17.492120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:44.801378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226265+0000","last_change":"2024-04-26T21:19:16.499234+0000","last_active":"2024-04-26T21:19:18.226265+0000","last_peered":"2024-04-26T21:19:18.226265+0000","last_clean":"2024-04-26T21:19:18.226265+0000","last_became_active":"2024-04-26T21:19:16.498965+0000","last_became_peered":"2024-04-26T21:19:16.498965+0000","last_unstale":"2024-04-26T21:19:18.226265+0000","last_undegraded":"2024-04-26T21:19:18.226265+0000","last_fullsized":"2024-04-26T21:19:18.226265+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:21.406859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672911+0000","last_change":"2024-04-26T21:19:15.485012+0000","last_active":"2024-04-26T21:19:17.672911+0000","last_peered":"2024-04-26T21:19:17.672911+0000","last_clean":"2024-04-26T21:19:17.672911+0000","last_became_active":"2024-04-26T21:19:15.484711+0000","last_became_peered":"2024-04-26T21:19:15.484711+0000","last_unstale":"2024-04-26T21:19:17.672911+0000","last_undegraded":"2024-04-26T21:19:17.672911+0000","last_fullsized":"2024-04-26T21:19:17.672911+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:58.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226155+0000","last_change":"2024-04-26T21:19:16.498702+0000","last_active":"2024-04-26T21:19:18.226155+0000","last_peered":"2024-04-26T21:19:18.226155+0000","last_clean":"2024-04-26T21:19:18.226155+0000","last_became_active":"2024-04-26T21:19:16.498408+0000","last_became_peered":"2024-04-26T21:19:16.498408+0000","last_unstale":"2024-04-26T21:19:18.226155+0000","last_undegraded":"2024-04-26T21:19:18.226155+0000","last_fullsized":"2024-04-26T21:19:18.226155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:50.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491957+0000","last_change":"2024-04-26T21:19:15.491055+0000","last_active":"2024-04-26T21:19:17.491957+0000","last_peered":"2024-04-26T21:19:17.491957+0000","last_clean":"2024-04-26T21:19:17.491957+0000","last_became_active":"2024-04-26T21:19:15.490783+0000","last_became_peered":"2024-04-26T21:19:15.490783+0000","last_unstale":"2024-04-26T21:19:17.491957+0000","last_undegraded":"2024-04-26T21:19:17.491957+0000","last_fullsized":"2024-04-26T21:19:17.491957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:05.607960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.225865+0000","last_change":"2024-04-26T21:19:16.501447+0000","last_active":"2024-04-26T21:19:18.225865+0000","last_peered":"2024-04-26T21:19:18.225865+0000","last_clean":"2024-04-26T21:19:18.225865+0000","last_became_active":"2024-04-26T21:19:16.501149+0000","last_became_peered":"2024-04-26T21:19:16.501149+0000","last_unstale":"2024-04-26T21:19:18.225865+0000","last_undegraded":"2024-04-26T21:19:18.225865+0000","last_fullsized":"2024-04-26T21:19:18.225865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:25.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.225920+0000","last_change":"2024-04-26T21:19:15.490438+0000","last_active":"2024-04-26T21:19:18.225920+0000","last_peered":"2024-04-26T21:19:18.225920+0000","last_clean":"2024-04-26T21:19:18.225920+0000","last_became_active":"2024-04-26T21:19:15.490210+0000","last_became_peered":"2024-04-26T21:19:15.490210+0000","last_unstale":"2024-04-26T21:19:18.225920+0000","last_undegraded":"2024-04-26T21:19:18.225920+0000","last_fullsized":"2024-04-26T21:19:18.225920+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:30.422988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672795+0000","last_change":"2024-04-26T21:19:16.491479+0000","last_active":"2024-04-26T21:19:17.672795+0000","last_peered":"2024-04-26T21:19:17.672795+0000","last_clean":"2024-04-26T21:19:17.672795+0000","last_became_active":"2024-04-26T21:19:16.491224+0000","last_became_peered":"2024-04-26T21:19:16.491224+0000","last_unstale":"2024-04-26T21:19:17.672795+0000","last_undegraded":"2024-04-26T21:19:17.672795+0000","last_fullsized":"2024-04-26T21:19:17.672795+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:51.573831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226021+0000","last_change":"2024-04-26T21:19:15.488327+0000","last_active":"2024-04-26T21:19:18.226021+0000","last_peered":"2024-04-26T21:19:18.226021+0000","last_clean":"2024-04-26T21:19:18.226021+0000","last_became_active":"2024-04-26T21:19:15.488039+0000","last_became_peered":"2024-04-26T21:19:15.488039+0000","last_unstale":"2024-04-26T21:19:18.226021+0000","last_undegraded":"2024-04-26T21:19:18.226021+0000","last_fullsized":"2024-04-26T21:19:18.226021+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:08.686002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673153+0000","last_change":"2024-04-26T21:19:16.481276+0000","last_active":"2024-04-26T21:19:17.673153+0000","last_peered":"2024-04-26T21:19:17.673153+0000","last_clean":"2024-04-26T21:19:17.673153+0000","last_became_active":"2024-04-26T21:19:16.480986+0000","last_became_peered":"2024-04-26T21:19:16.480986+0000","last_unstale":"2024-04-26T21:19:17.673153+0000","last_undegraded":"2024-04-26T21:19:17.673153+0000","last_fullsized":"2024-04-26T21:19:17.673153+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:47.708248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492469+0000","last_change":"2024-04-26T21:19:15.484302+0000","last_active":"2024-04-26T21:19:17.492469+0000","last_peered":"2024-04-26T21:19:17.492469+0000","last_clean":"2024-04-26T21:19:17.492469+0000","last_became_active":"2024-04-26T21:19:15.483990+0000","last_became_peered":"2024-04-26T21:19:15.483990+0000","last_unstale":"2024-04-26T21:19:17.492469+0000","last_undegraded":"2024-04-26T21:19:17.492469+0000","last_fullsized":"2024-04-26T21:19:17.492469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:02.063693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226972+0000","last_change":"2024-04-26T21:19:16.489119+0000","last_active":"2024-04-26T21:19:18.226972+0000","last_peered":"2024-04-26T21:19:18.226972+0000","last_clean":"2024-04-26T21:19:18.226972+0000","last_became_active":"2024-04-26T21:19:16.488816+0000","last_became_peered":"2024-04-26T21:19:16.488816+0000","last_unstale":"2024-04-26T21:19:18.226972+0000","last_undegraded":"2024-04-26T21:19:18.226972+0000","last_fullsized":"2024-04-26T21:19:18.226972+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:40.818286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491477+0000","last_change":"2024-04-26T21:19:15.492213+0000","last_active":"2024-04-26T21:19:17.491477+0000","last_peered":"2024-04-26T21:19:17.491477+0000","last_clean":"2024-04-26T21:19:17.491477+0000","last_became_active":"2024-04-26T21:19:15.491792+0000","last_became_peered":"2024-04-26T21:19:15.491792+0000","last_unstale":"2024-04-26T21:19:17.491477+0000","last_undegraded":"2024-04-26T21:19:17.491477+0000","last_fullsized":"2024-04-26T21:19:17.491477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491775+0000","last_change":"2024-04-26T21:19:16.496675+0000","last_active":"2024-04-26T21:19:17.491775+0000","last_peered":"2024-04-26T21:19:17.491775+0000","last_clean":"2024-04-26T21:19:17.491775+0000","last_became_active":"2024-04-26T21:19:16.496382+0000","last_became_peered":"2024-04-26T21:19:16.496382+0000","last_unstale":"2024-04-26T21:19:17.491775+0000","last_undegraded":"2024-04-26T21:19:17.491775+0000","last_fullsized":"2024-04-26T21:19:17.491775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:17.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226477+0000","last_change":"2024-04-26T21:19:15.493890+0000","last_active":"2024-04-26T21:19:18.226477+0000","last_peered":"2024-04-26T21:19:18.226477+0000","last_clean":"2024-04-26T21:19:18.226477+0000","last_became_active":"2024-04-26T21:19:15.493428+0000","last_became_peered":"2024-04-26T21:19:15.493428+0000","last_unstale":"2024-04-26T21:19:18.226477+0000","last_undegraded":"2024-04-26T21:19:18.226477+0000","last_fullsized":"2024-04-26T21:19:18.226477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:44.796649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673102+0000","last_change":"2024-04-26T21:19:16.489093+0000","last_active":"2024-04-26T21:19:17.673102+0000","last_peered":"2024-04-26T21:19:17.673102+0000","last_clean":"2024-04-26T21:19:17.673102+0000","last_became_active":"2024-04-26T21:19:16.488815+0000","last_became_peered":"2024-04-26T21:19:16.488815+0000","last_unstale":"2024-04-26T21:19:17.673102+0000","last_undegraded":"2024-04-26T21:19:17.673102+0000","last_fullsized":"2024-04-26T21:19:17.673102+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:43.157084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226835+0000","last_change":"2024-04-26T21:19:15.489147+0000","last_active":"2024-04-26T21:19:18.226835+0000","last_peered":"2024-04-26T21:19:18.226835+0000","last_clean":"2024-04-26T21:19:18.226835+0000","last_became_active":"2024-04-26T21:19:15.488782+0000","last_became_peered":"2024-04-26T21:19:15.488782+0000","last_unstale":"2024-04-26T21:19:18.226835+0000","last_undegraded":"2024-04-26T21:19:18.226835+0000","last_fullsized":"2024-04-26T21:19:18.226835+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:12.490510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491886+0000","last_change":"2024-04-26T21:19:16.494938+0000","last_active":"2024-04-26T21:19:17.491886+0000","last_peered":"2024-04-26T21:19:17.491886+0000","last_clean":"2024-04-26T21:19:17.491886+0000","last_became_active":"2024-04-26T21:19:16.494586+0000","last_became_peered":"2024-04-26T21:19:16.494586+0000","last_unstale":"2024-04-26T21:19:17.491886+0000","last_undegraded":"2024-04-26T21:19:17.491886+0000","last_fullsized":"2024-04-26T21:19:17.491886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490242+0000","last_change":"2024-04-26T21:19:15.486297+0000","last_active":"2024-04-26T21:19:17.490242+0000","last_peered":"2024-04-26T21:19:17.490242+0000","last_clean":"2024-04-26T21:19:17.490242+0000","last_became_active":"2024-04-26T21:19:15.485985+0000","last_became_peered":"2024-04-26T21:19:15.485985+0000","last_unstale":"2024-04-26T21:19:17.490242+0000","last_undegraded":"2024-04-26T21:19:17.490242+0000","last_fullsized":"2024-04-26T21:19:17.490242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:34.344417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491584+0000","last_change":"2024-04-26T21:19:16.484940+0000","last_active":"2024-04-26T21:19:17.491584+0000","last_peered":"2024-04-26T21:19:17.491584+0000","last_clean":"2024-04-26T21:19:17.491584+0000","last_became_active":"2024-04-26T21:19:16.484650+0000","last_became_peered":"2024-04-26T21:19:16.484650+0000","last_unstale":"2024-04-26T21:19:17.491584+0000","last_undegraded":"2024-04-26T21:19:17.491584+0000","last_fullsized":"2024-04-26T21:19:17.491584+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:09.688206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226416+0000","last_change":"2024-04-26T21:19:15.486046+0000","last_active":"2024-04-26T21:19:18.226416+0000","last_peered":"2024-04-26T21:19:18.226416+0000","last_clean":"2024-04-26T21:19:18.226416+0000","last_became_active":"2024-04-26T21:19:15.485776+0000","last_became_peered":"2024-04-26T21:19:15.485776+0000","last_unstale":"2024-04-26T21:19:18.226416+0000","last_undegraded":"2024-04-26T21:19:18.226416+0000","last_fullsized":"2024-04-26T21:19:18.226416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:07.737449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491404+0000","last_change":"2024-04-26T21:19:16.497306+0000","last_active":"2024-04-26T21:19:17.491404+0000","last_peered":"2024-04-26T21:19:17.491404+0000","last_clean":"2024-04-26T21:19:17.491404+0000","last_became_active":"2024-04-26T21:19:16.497039+0000","last_became_peered":"2024-04-26T21:19:16.497039+0000","last_unstale":"2024-04-26T21:19:17.491404+0000","last_undegraded":"2024-04-26T21:19:17.491404+0000","last_fullsized":"2024-04-26T21:19:17.491404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:16.896268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492215+0000","last_change":"2024-04-26T21:19:15.490142+0000","last_active":"2024-04-26T21:19:17.492215+0000","last_peered":"2024-04-26T21:19:17.492215+0000","last_clean":"2024-04-26T21:19:17.492215+0000","last_became_active":"2024-04-26T21:19:15.489841+0000","last_became_peered":"2024-04-26T21:19:15.489841+0000","last_unstale":"2024-04-26T21:19:17.492215+0000","last_undegraded":"2024-04-26T21:19:17.492215+0000","last_fullsized":"2024-04-26T21:19:17.492215+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:06.856169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492590+0000","last_change":"2024-04-26T21:19:16.485067+0000","last_active":"2024-04-26T21:19:17.492590+0000","last_peered":"2024-04-26T21:19:17.492590+0000","last_clean":"2024-04-26T21:19:17.492590+0000","last_became_active":"2024-04-26T21:19:16.484588+0000","last_became_peered":"2024-04-26T21:19:16.484588+0000","last_unstale":"2024-04-26T21:19:17.492590+0000","last_undegraded":"2024-04-26T21:19:17.492590+0000","last_fullsized":"2024-04-26T21:19:17.492590+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:06.657912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226890+0000","last_change":"2024-04-26T21:19:15.488200+0000","last_active":"2024-04-26T21:19:18.226890+0000","last_peered":"2024-04-26T21:19:18.226890+0000","last_clean":"2024-04-26T21:19:18.226890+0000","last_became_active":"2024-04-26T21:19:15.487840+0000","last_became_peered":"2024-04-26T21:19:15.487840+0000","last_unstale":"2024-04-26T21:19:18.226890+0000","last_undegraded":"2024-04-26T21:19:18.226890+0000","last_fullsized":"2024-04-26T21:19:18.226890+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:32.520127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490417+0000","last_change":"2024-04-26T21:19:16.505313+0000","last_active":"2024-04-26T21:19:17.490417+0000","last_peered":"2024-04-26T21:19:17.490417+0000","last_clean":"2024-04-26T21:19:17.490417+0000","last_became_active":"2024-04-26T21:19:16.504970+0000","last_became_peered":"2024-04-26T21:19:16.504970+0000","last_unstale":"2024-04-26T21:19:17.490417+0000","last_undegraded":"2024-04-26T21:19:17.490417+0000","last_fullsized":"2024-04-26T21:19:17.490417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:16.505006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492522+0000","last_change":"2024-04-26T21:19:15.482948+0000","last_active":"2024-04-26T21:19:17.492522+0000","last_peered":"2024-04-26T21:19:17.492522+0000","last_clean":"2024-04-26T21:19:17.492522+0000","last_became_active":"2024-04-26T21:19:15.482636+0000","last_became_peered":"2024-04-26T21:19:15.482636+0000","last_unstale":"2024-04-26T21:19:17.492522+0000","last_undegraded":"2024-04-26T21:19:17.492522+0000","last_fullsized":"2024-04-26T21:19:17.492522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:04.356393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538613+0000","last_change":"2024-04-26T21:19:16.492696+0000","last_active":"2024-04-26T21:19:17.538613+0000","last_peered":"2024-04-26T21:19:17.538613+0000","last_clean":"2024-04-26T21:19:17.538613+0000","last_became_active":"2024-04-26T21:19:16.492396+0000","last_became_peered":"2024-04-26T21:19:16.492396+0000","last_unstale":"2024-04-26T21:19:17.538613+0000","last_undegraded":"2024-04-26T21:19:17.538613+0000","last_fullsized":"2024-04-26T21:19:17.538613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:38.023551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226235+0000","last_change":"2024-04-26T21:19:15.488486+0000","last_active":"2024-04-26T21:19:18.226235+0000","last_peered":"2024-04-26T21:19:18.226235+0000","last_clean":"2024-04-26T21:19:18.226235+0000","last_became_active":"2024-04-26T21:19:15.488221+0000","last_became_peered":"2024-04-26T21:19:15.488221+0000","last_unstale":"2024-04-26T21:19:18.226235+0000","last_undegraded":"2024-04-26T21:19:18.226235+0000","last_fullsized":"2024-04-26T21:19:18.226235+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:30.760409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538790+0000","last_change":"2024-04-26T21:19:16.492285+0000","last_active":"2024-04-26T21:19:17.538790+0000","last_peered":"2024-04-26T21:19:17.538790+0000","last_clean":"2024-04-26T21:19:17.538790+0000","last_became_active":"2024-04-26T21:19:16.491992+0000","last_became_peered":"2024-04-26T21:19:16.491992+0000","last_unstale":"2024-04-26T21:19:17.538790+0000","last_undegraded":"2024-04-26T21:19:17.538790+0000","last_fullsized":"2024-04-26T21:19:17.538790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:58.488841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672950+0000","last_change":"2024-04-26T21:19:15.486458+0000","last_active":"2024-04-26T21:19:17.672950+0000","last_peered":"2024-04-26T21:19:17.672950+0000","last_clean":"2024-04-26T21:19:17.672950+0000","last_became_active":"2024-04-26T21:19:15.486127+0000","last_became_peered":"2024-04-26T21:19:15.486127+0000","last_unstale":"2024-04-26T21:19:17.672950+0000","last_undegraded":"2024-04-26T21:19:17.672950+0000","last_fullsized":"2024-04-26T21:19:17.672950+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:05.272243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538381+0000","last_change":"2024-04-26T21:19:16.495963+0000","last_active":"2024-04-26T21:19:17.538381+0000","last_peered":"2024-04-26T21:19:17.538381+0000","last_clean":"2024-04-26T21:19:17.538381+0000","last_became_active":"2024-04-26T21:19:16.495645+0000","last_became_peered":"2024-04-26T21:19:16.495645+0000","last_unstale":"2024-04-26T21:19:17.538381+0000","last_undegraded":"2024-04-26T21:19:17.538381+0000","last_fullsized":"2024-04-26T21:19:17.538381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:05.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490371+0000","last_change":"2024-04-26T21:19:15.487470+0000","last_active":"2024-04-26T21:19:17.490371+0000","last_peered":"2024-04-26T21:19:17.490371+0000","last_clean":"2024-04-26T21:19:17.490371+0000","last_became_active":"2024-04-26T21:19:15.487190+0000","last_became_peered":"2024-04-26T21:19:15.487190+0000","last_unstale":"2024-04-26T21:19:17.490371+0000","last_undegraded":"2024-04-26T21:19:17.490371+0000","last_fullsized":"2024-04-26T21:19:17.490371+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:52.279807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226111+0000","last_change":"2024-04-26T21:19:16.487812+0000","last_active":"2024-04-26T21:19:18.226111+0000","last_peered":"2024-04-26T21:19:18.226111+0000","last_clean":"2024-04-26T21:19:18.226111+0000","last_became_active":"2024-04-26T21:19:16.487449+0000","last_became_peered":"2024-04-26T21:19:16.487449+0000","last_unstale":"2024-04-26T21:19:18.226111+0000","last_undegraded":"2024-04-26T21:19:18.226111+0000","last_fullsized":"2024-04-26T21:19:18.226111+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:50.605659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492009+0000","last_change":"2024-04-26T21:19:15.489410+0000","last_active":"2024-04-26T21:19:17.492009+0000","last_peered":"2024-04-26T21:19:17.492009+0000","last_clean":"2024-04-26T21:19:17.492009+0000","last_became_active":"2024-04-26T21:19:15.489083+0000","last_became_peered":"2024-04-26T21:19:15.489083+0000","last_unstale":"2024-04-26T21:19:17.492009+0000","last_undegraded":"2024-04-26T21:19:17.492009+0000","last_fullsized":"2024-04-26T21:19:17.492009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:50.451081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538376+0000","last_change":"2024-04-26T21:19:16.486905+0000","last_active":"2024-04-26T21:19:17.538376+0000","last_peered":"2024-04-26T21:19:17.538376+0000","last_clean":"2024-04-26T21:19:17.538376+0000","last_became_active":"2024-04-26T21:19:16.486575+0000","last_became_peered":"2024-04-26T21:19:16.486575+0000","last_unstale":"2024-04-26T21:19:17.538376+0000","last_undegraded":"2024-04-26T21:19:17.538376+0000","last_fullsized":"2024-04-26T21:19:17.538376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:45.798853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.490212+0000","last_change":"2024-04-26T21:19:15.488247+0000","last_active":"2024-04-26T21:19:17.490212+0000","last_peered":"2024-04-26T21:19:17.490212+0000","last_clean":"2024-04-26T21:19:17.490212+0000","last_became_active":"2024-04-26T21:19:15.487933+0000","last_became_peered":"2024-04-26T21:19:15.487933+0000","last_unstale":"2024-04-26T21:19:17.490212+0000","last_undegraded":"2024-04-26T21:19:17.490212+0000","last_fullsized":"2024-04-26T21:19:17.490212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:16.427539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491418+0000","last_change":"2024-04-26T21:19:16.483829+0000","last_active":"2024-04-26T21:19:17.491418+0000","last_peered":"2024-04-26T21:19:17.491418+0000","last_clean":"2024-04-26T21:19:17.491418+0000","last_became_active":"2024-04-26T21:19:16.483486+0000","last_became_peered":"2024-04-26T21:19:16.483486+0000","last_unstale":"2024-04-26T21:19:17.491418+0000","last_undegraded":"2024-04-26T21:19:17.491418+0000","last_fullsized":"2024-04-26T21:19:17.491418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:34.802721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226156+0000","last_change":"2024-04-26T21:19:15.487580+0000","last_active":"2024-04-26T21:19:18.226156+0000","last_peered":"2024-04-26T21:19:18.226156+0000","last_clean":"2024-04-26T21:19:18.226156+0000","last_became_active":"2024-04-26T21:19:15.487307+0000","last_became_peered":"2024-04-26T21:19:15.487307+0000","last_unstale":"2024-04-26T21:19:18.226156+0000","last_undegraded":"2024-04-26T21:19:18.226156+0000","last_fullsized":"2024-04-26T21:19:18.226156+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.926428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226353+0000","last_change":"2024-04-26T21:19:16.487487+0000","last_active":"2024-04-26T21:19:18.226353+0000","last_peered":"2024-04-26T21:19:18.226353+0000","last_clean":"2024-04-26T21:19:18.226353+0000","last_became_active":"2024-04-26T21:19:16.487194+0000","last_became_peered":"2024-04-26T21:19:16.487194+0000","last_unstale":"2024-04-26T21:19:18.226353+0000","last_undegraded":"2024-04-26T21:19:18.226353+0000","last_fullsized":"2024-04-26T21:19:18.226353+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:21.856772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491590+0000","last_change":"2024-04-26T21:19:15.490106+0000","last_active":"2024-04-26T21:19:17.491590+0000","last_peered":"2024-04-26T21:19:17.491590+0000","last_clean":"2024-04-26T21:19:17.491590+0000","last_became_active":"2024-04-26T21:19:15.489789+0000","last_became_peered":"2024-04-26T21:19:15.489789+0000","last_unstale":"2024-04-26T21:19:17.491590+0000","last_undegraded":"2024-04-26T21:19:17.491590+0000","last_fullsized":"2024-04-26T21:19:17.491590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:47.506686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226939+0000","last_change":"2024-04-26T21:19:16.496139+0000","last_active":"2024-04-26T21:19:18.226939+0000","last_peered":"2024-04-26T21:19:18.226939+0000","last_clean":"2024-04-26T21:19:18.226939+0000","last_became_active":"2024-04-26T21:19:16.495785+0000","last_became_peered":"2024-04-26T21:19:16.495785+0000","last_unstale":"2024-04-26T21:19:18.226939+0000","last_undegraded":"2024-04-26T21:19:18.226939+0000","last_fullsized":"2024-04-26T21:19:18.226939+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:55.628305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226398+0000","last_change":"2024-04-26T21:19:15.476421+0000","last_active":"2024-04-26T21:19:18.226398+0000","last_peered":"2024-04-26T21:19:18.226398+0000","last_clean":"2024-04-26T21:19:18.226398+0000","last_became_active":"2024-04-26T21:19:15.476081+0000","last_became_peered":"2024-04-26T21:19:15.476081+0000","last_unstale":"2024-04-26T21:19:18.226398+0000","last_undegraded":"2024-04-26T21:19:18.226398+0000","last_fullsized":"2024-04-26T21:19:18.226398+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:14.413252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226723+0000","last_change":"2024-04-26T21:19:16.502148+0000","last_active":"2024-04-26T21:19:18.226723+0000","last_peered":"2024-04-26T21:19:18.226723+0000","last_clean":"2024-04-26T21:19:18.226723+0000","last_became_active":"2024-04-26T21:19:16.500040+0000","last_became_peered":"2024-04-26T21:19:16.500040+0000","last_unstale":"2024-04-26T21:19:18.226723+0000","last_undegraded":"2024-04-26T21:19:18.226723+0000","last_fullsized":"2024-04-26T21:19:18.226723+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:22.570837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226448+0000","last_change":"2024-04-26T21:19:15.481531+0000","last_active":"2024-04-26T21:19:18.226448+0000","last_peered":"2024-04-26T21:19:18.226448+0000","last_clean":"2024-04-26T21:19:18.226448+0000","last_became_active":"2024-04-26T21:19:15.481231+0000","last_became_peered":"2024-04-26T21:19:15.481231+0000","last_unstale":"2024-04-26T21:19:18.226448+0000","last_undegraded":"2024-04-26T21:19:18.226448+0000","last_fullsized":"2024-04-26T21:19:18.226448+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:04.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226227+0000","last_change":"2024-04-26T21:19:16.488965+0000","last_active":"2024-04-26T21:19:18.226227+0000","last_peered":"2024-04-26T21:19:18.226227+0000","last_clean":"2024-04-26T21:19:18.226227+0000","last_became_active":"2024-04-26T21:19:16.488655+0000","last_became_peered":"2024-04-26T21:19:16.488655+0000","last_unstale":"2024-04-26T21:19:18.226227+0000","last_undegraded":"2024-04-26T21:19:18.226227+0000","last_fullsized":"2024-04-26T21:19:18.226227+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:37.342425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491897+0000","last_change":"2024-04-26T21:19:15.493244+0000","last_active":"2024-04-26T21:19:17.491897+0000","last_peered":"2024-04-26T21:19:17.491897+0000","last_clean":"2024-04-26T21:19:17.491897+0000","last_became_active":"2024-04-26T21:19:15.493019+0000","last_became_peered":"2024-04-26T21:19:15.493019+0000","last_unstale":"2024-04-26T21:19:17.491897+0000","last_undegraded":"2024-04-26T21:19:17.491897+0000","last_fullsized":"2024-04-26T21:19:17.491897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:15.360405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226208+0000","last_change":"2024-04-26T21:19:16.493217+0000","last_active":"2024-04-26T21:19:18.226208+0000","last_peered":"2024-04-26T21:19:18.226208+0000","last_clean":"2024-04-26T21:19:18.226208+0000","last_became_active":"2024-04-26T21:19:16.492937+0000","last_became_peered":"2024-04-26T21:19:16.492937+0000","last_unstale":"2024-04-26T21:19:18.226208+0000","last_undegraded":"2024-04-26T21:19:18.226208+0000","last_fullsized":"2024-04-26T21:19:18.226208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:31.045507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.673072+0000","last_change":"2024-04-26T21:19:15.482604+0000","last_active":"2024-04-26T21:19:17.673072+0000","last_peered":"2024-04-26T21:19:17.673072+0000","last_clean":"2024-04-26T21:19:17.673072+0000","last_became_active":"2024-04-26T21:19:15.482320+0000","last_became_peered":"2024-04-26T21:19:15.482320+0000","last_unstale":"2024-04-26T21:19:17.673072+0000","last_undegraded":"2024-04-26T21:19:17.673072+0000","last_fullsized":"2024-04-26T21:19:17.673072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:24.748462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226323+0000","last_change":"2024-04-26T21:19:16.493348+0000","last_active":"2024-04-26T21:19:18.226323+0000","last_peered":"2024-04-26T21:19:18.226323+0000","last_clean":"2024-04-26T21:19:18.226323+0000","last_became_active":"2024-04-26T21:19:16.492941+0000","last_became_peered":"2024-04-26T21:19:16.492941+0000","last_unstale":"2024-04-26T21:19:18.226323+0000","last_undegraded":"2024-04-26T21:19:18.226323+0000","last_fullsized":"2024-04-26T21:19:18.226323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:37.342425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226317+0000","last_change":"2024-04-26T21:19:15.489228+0000","last_active":"2024-04-26T21:19:18.226317+0000","last_peered":"2024-04-26T21:19:18.226317+0000","last_clean":"2024-04-26T21:19:18.226317+0000","last_became_active":"2024-04-26T21:19:15.488843+0000","last_became_peered":"2024-04-26T21:19:15.488843+0000","last_unstale":"2024-04-26T21:19:18.226317+0000","last_undegraded":"2024-04-26T21:19:18.226317+0000","last_fullsized":"2024-04-26T21:19:18.226317+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:42.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538707+0000","last_change":"2024-04-26T21:19:16.491546+0000","last_active":"2024-04-26T21:19:17.538707+0000","last_peered":"2024-04-26T21:19:17.538707+0000","last_clean":"2024-04-26T21:19:17.538707+0000","last_became_active":"2024-04-26T21:19:16.491305+0000","last_became_peered":"2024-04-26T21:19:16.491305+0000","last_unstale":"2024-04-26T21:19:17.538707+0000","last_undegraded":"2024-04-26T21:19:17.538707+0000","last_fullsized":"2024-04-26T21:19:17.538707+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:41.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.492104+0000","last_change":"2024-04-26T21:19:15.490249+0000","last_active":"2024-04-26T21:19:17.492104+0000","last_peered":"2024-04-26T21:19:17.492104+0000","last_clean":"2024-04-26T21:19:17.492104+0000","last_became_active":"2024-04-26T21:19:15.489962+0000","last_became_peered":"2024-04-26T21:19:15.489962+0000","last_unstale":"2024-04-26T21:19:17.492104+0000","last_undegraded":"2024-04-26T21:19:17.492104+0000","last_fullsized":"2024-04-26T21:19:17.492104+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226172+0000","last_change":"2024-04-26T21:19:16.501636+0000","last_active":"2024-04-26T21:19:18.226172+0000","last_peered":"2024-04-26T21:19:18.226172+0000","last_clean":"2024-04-26T21:19:18.226172+0000","last_became_active":"2024-04-26T21:19:16.501326+0000","last_became_peered":"2024-04-26T21:19:16.501326+0000","last_unstale":"2024-04-26T21:19:18.226172+0000","last_undegraded":"2024-04-26T21:19:18.226172+0000","last_fullsized":"2024-04-26T21:19:18.226172+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:53.414509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:18.226212+0000","last_change":"2024-04-26T21:19:15.486884+0000","last_active":"2024-04-26T21:19:18.226212+0000","last_peered":"2024-04-26T21:19:18.226212+0000","last_clean":"2024-04-26T21:19:18.226212+0000","last_became_active":"2024-04-26T21:19:15.486597+0000","last_became_peered":"2024-04-26T21:19:15.486597+0000","last_unstale":"2024-04-26T21:19:18.226212+0000","last_undegraded":"2024-04-26T21:19:18.226212+0000","last_fullsized":"2024-04-26T21:19:18.226212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:03:16.558118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.491321+0000","last_change":"2024-04-26T21:19:16.485658+0000","last_active":"2024-04-26T21:19:17.491321+0000","last_peered":"2024-04-26T21:19:17.491321+0000","last_clean":"2024-04-26T21:19:17.491321+0000","last_became_active":"2024-04-26T21:19:16.485382+0000","last_became_peered":"2024-04-26T21:19:16.485382+0000","last_unstale":"2024-04-26T21:19:17.491321+0000","last_undegraded":"2024-04-26T21:19:17.491321+0000","last_fullsized":"2024-04-26T21:19:17.491321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:15.463203+0000","last_clean_scrub_stamp":"2024-04-26T21:19:15.463203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:53.203359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.672961+0000","last_change":"2024-04-26T21:19:15.480711+0000","last_active":"2024-04-26T21:19:17.672961+0000","last_peered":"2024-04-26T21:19:17.672961+0000","last_clean":"2024-04-26T21:19:17.672961+0000","last_became_active":"2024-04-26T21:19:15.480405+0000","last_became_peered":"2024-04-26T21:19:15.480405+0000","last_unstale":"2024-04-26T21:19:17.672961+0000","last_undegraded":"2024-04-26T21:19:17.672961+0000","last_fullsized":"2024-04-26T21:19:17.672961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:14.460999+0000","last_clean_scrub_stamp":"2024-04-26T21:19:14.460999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:00.640043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-26T21:19:17.538749+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:19:17.538749+0000","last_peered":"2024-04-26T21:19:17.538749+0000","last_clean":"2024-04-26T21:19:17.538749+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:19:17.538749+0000","last_undegraded":"2024-04-26T21:19:17.538749+0000","last_fullsized":"2024-04-26T21:19:17.538749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509535,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":1167338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":576424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509534,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":576958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1363968,"data_stored":578036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":576354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":1168446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":576478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:19:25.289 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:19:25.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:19:25.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.432+0000 7f029bff2640 1 Processor -- start 2024-04-26T21:19:25.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 -- start start 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f029415f3c0 0x7f02940d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02940d1fe0 0x7f0294163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02940b6ab0 con 0x7f02940d1b80 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f02940d1260 con 0x7f02940d1fe0 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.433+0000 7f029bff2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02940d4200 con 0x7f029415f3c0 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029a58f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02940d1fe0 0x7f0294163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029958d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f02940d1b80 0x7f0294163020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55382/0 (socket says 172.21.15.40:55382) 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029958d640 1 -- 172.21.15.40:0/1352836937 learned_addr learned my addr 172.21.15.40:0/1352836937 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f0299d8e640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f029415f3c0 0x7f02940d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f0298d8c640 1 -- 172.21.15.40:0/1352836937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209350216 0 0) 0x7f02940b6ab0 con 0x7f02940d1b80 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f0298d8c640 1 -- 172.21.15.40:0/1352836937 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0280003520 con 0x7f02940d1b80 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029a58f640 1 -- 172.21.15.40:0/1352836937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f029415f3c0 msgr2=0x7f02940d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029a58f640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f029415f3c0 0x7f02940d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029a58f640 1 -- 172.21.15.40:0/1352836937 >> v1:172.21.15.40:6789/0 conn(0x7f02940d1b80 legacy=0x7f0294163020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.434+0000 7f029a58f640 1 -- 172.21.15.40:0/1352836937 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02940d3160 con 0x7f02940d1fe0 2024-04-26T21:19:25.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029a58f640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02940d1fe0 0x7f0294163730 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f028c009d90 tx=0x7f028c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7114bd8c50cb54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f0298d8c640 1 -- 172.21.15.40:0/1352836937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c007740 con 0x7f02940d1fe0 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f0298d8c640 1 -- 172.21.15.40:0/1352836937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0078e0 con 0x7f02940d1fe0 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f0298d8c640 1 -- 172.21.15.40:0/1352836937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c007cb0 con 0x7f02940d1fe0 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 -- 172.21.15.40:0/1352836937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02940d1fe0 msgr2=0x7f0294163730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02940d1fe0 0x7f0294163730 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f028c009d90 tx=0x7f028c02a660 comp rx=0 tx=0).stop 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 -- 172.21.15.40:0/1352836937 shutdown_connections 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02940d1fe0 0x7f0294163730 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 --2- 172.21.15.40:0/1352836937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f029415f3c0 0x7f02940d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 -- 172.21.15.40:0/1352836937 >> 172.21.15.40:0/1352836937 conn(0x7f029409e4f0 msgr2=0x7f02940d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 -- 172.21.15.40:0/1352836937 shutdown_connections 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.435+0000 7f029bff2640 1 -- 172.21.15.40:0/1352836937 wait complete. 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 Processor -- start 2024-04-26T21:19:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 -- start start 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02940d1b80 0x7f02940b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02940d1fe0 0x7f02940ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d3a00 con 0x7f029415f3c0 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d3720 con 0x7f02940d1fe0 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029bff2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02940d3ce0 con 0x7f02940d1b80 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029a58f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.437+0000 7f029a58f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57262/0 (socket says 172.21.15.40:57262) 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 -- 172.21.15.40:0/221563723 learned_addr learned my addr 172.21.15.40:0/221563723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 -- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02940d1b80 msgr2=0x7f02940b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02940d1b80 0x7f02940b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 -- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02940d1fe0 msgr2=0x7f02940ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02940d1fe0 0x7f02940ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 -- 172.21.15.40:0/221563723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941f5090 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029a58f640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f028c009eb0 tx=0x7f028c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c040070 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c031bc0 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f5370 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c009450 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.438+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941f5930 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.439+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02940693d0 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.440+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f028c039040 con 0x7f029415f3c0 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.440+0000 7f028a7fc640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.440+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f028c0ad3a0 con 0x7f029415f3c0 2024-04-26T21:19:25.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.440+0000 7f0299d8e640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.441+0000 7f0299d8e640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 0x7f027806edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0290006a90 tx=0x7f0290006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.444+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f028c07b8f0 con 0x7f029415f3c0 2024-04-26T21:19:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.703+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f02940d58c0 con 0x7f029415f3c0 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.705+0000 7f028a7fc640 1 -- 172.21.15.40:0/221563723 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f028c043e40 con 0x7f029415f3c0 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:19:25.706 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:19:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:19:25.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 msgr2=0x7f027806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 0x7f027806edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0290006a90 tx=0x7f0290006a20 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 msgr2=0x7f02941f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f028c009eb0 tx=0x7f028c004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 shutdown_connections 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f027806c8f0 0x7f027806edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029415f3c0 0x7f02941f4100 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02940d1fe0 0x7f02940ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 --2- 172.21.15.40:0/221563723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02940d1b80 0x7f02940b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 >> 172.21.15.40:0/221563723 conn(0x7f029409e4f0 msgr2=0x7f029415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 shutdown_connections 2024-04-26T21:19:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.707+0000 7f029bff2640 1 -- 172.21.15.40:0/221563723 wait complete. 2024-04-26T21:19:25.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.819+0000 7f064d19d640 1 Processor -- start 2024-04-26T21:19:25.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.819+0000 7f064d19d640 1 -- start start 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0648145980 0x7f0648160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 0x7f06480b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06480b5fd0 0x7f064815c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f06480b6440 con 0x7f06480b5fd0 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06480b5140 con 0x7f0648145980 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064d19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06480b4260 con 0x7f06480b3de0 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06480b5fd0 0x7f064815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06480b5fd0 0x7f064815c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57270/0 (socket says 172.21.15.40:57270) 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 -- 172.21.15.40:0/711137098 learned_addr learned my addr 172.21.15.40:0/711137098 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f063ffff640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 0x7f06480b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f0646d9d640 1 --2- 172.21.15.40:0/711137098 >> v2:172.21.15.184:3300/0 conn(0x7f0648145980 0x7f0648160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 -- 172.21.15.40:0/711137098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 msgr2=0x7f06480b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 0x7f06480b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 -- 172.21.15.40:0/711137098 >> v2:172.21.15.184:3300/0 conn(0x7f0648145980 msgr2=0x7f0648160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 --2- 172.21.15.40:0/711137098 >> v2:172.21.15.184:3300/0 conn(0x7f0648145980 0x7f0648160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f064759e640 1 -- 172.21.15.40:0/711137098 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06480b4530 con 0x7f06480b5fd0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.820+0000 7f063ffff640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 0x7f06480b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.821+0000 7f064759e640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815c510 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f064806ff10 tx=0x7f064002c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2e1e442ad33067 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.821+0000 7f064659c640 1 -- 172.21.15.40:0/711137098 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640039070 con 0x7f06480b5fd0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.821+0000 7f064659c640 1 -- 172.21.15.40:0/711137098 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0640009600 con 0x7f06480b5fd0 2024-04-26T21:19:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.821+0000 7f064659c640 1 -- 172.21.15.40:0/711137098 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400099f0 con 0x7f06480b5fd0 2024-04-26T21:19:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 -- 172.21.15.40:0/711137098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 msgr2=0x7f064815c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815c510 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f064806ff10 tx=0x7f064002c810 comp rx=0 tx=0).stop 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 -- 172.21.15.40:0/711137098 shutdown_connections 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815c510 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 --2- 172.21.15.40:0/711137098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06480b3de0 0x7f06480b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 --2- 172.21.15.40:0/711137098 >> v2:172.21.15.184:3300/0 conn(0x7f0648145980 0x7f0648160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 -- 172.21.15.40:0/711137098 >> 172.21.15.40:0/711137098 conn(0x7f06480d0d30 msgr2=0x7f06480d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 -- 172.21.15.40:0/711137098 shutdown_connections 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.822+0000 7f064d19d640 1 -- 172.21.15.40:0/711137098 wait complete. 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.823+0000 7f064d19d640 1 Processor -- start 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.823+0000 7f064d19d640 1 -- start start 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06480b3de0 0x7f06480afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0648145980 0x7f06480ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06480b4dd0 con 0x7f06480b5fd0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06480b4af0 con 0x7f06480b3de0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f064815ca50 con 0x7f0648145980 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57286/0 (socket says 172.21.15.40:57286) 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 -- 172.21.15.40:0/3275984141 learned_addr learned my addr 172.21.15.40:0/3275984141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f0646d9d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06480b3de0 0x7f06480afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064759e640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0648145980 0x7f06480ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 -- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0648145980 msgr2=0x7f06480ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0648145980 0x7f06480ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 -- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06480b3de0 msgr2=0x7f06480afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06480b3de0 0x7f06480afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 -- 172.21.15.40:0/3275984141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0648141b60 con 0x7f06480b5fd0 2024-04-26T21:19:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063ffff640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f063000e810 tx=0x7f063000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063000f040 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630004030 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0648133830 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.824+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630004400 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.825+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0648133df0 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.826+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06481603e0 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.826+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f063000c6d0 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.826+0000 7f063f7fe640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 0x7f062806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.826+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f063008b7c0 con 0x7f06480b5fd0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.827+0000 7f0646d9d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 0x7f062806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.827+0000 7f0646d9d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 0x7f062806edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0648073420 tx=0x7f0638008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:25.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:25.831+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06481603e0 con 0x7f06480b5fd0 2024-04-26T21:19:26.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.055+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f064815c830 con 0x7f06480b5fd0 2024-04-26T21:19:26.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.055+0000 7f063f7fe640 1 -- 172.21.15.40:0/3275984141 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5186 (secure 0 0 0) 0x7f0630017020 con 0x7f06480b5fd0 2024-04-26T21:19:26.057 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:26.057 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:18.865416+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:26.057 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 64 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 msgr2=0x7f062806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 0x7f062806edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0648073420 tx=0x7f0638008040 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 msgr2=0x7f064815e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f063000e810 tx=0x7f063000ecd0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 shutdown_connections 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f062806c8f0 0x7f062806edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0648145980 0x7f06480ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06480b5fd0 0x7f064815e140 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 --2- 172.21.15.40:0/3275984141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06480b3de0 0x7f06480afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 >> 172.21.15.40:0/3275984141 conn(0x7f06480d0d30 msgr2=0x7f064815f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 shutdown_connections 2024-04-26T21:19:26.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.058+0000 7f064d19d640 1 -- 172.21.15.40:0/3275984141 wait complete. 2024-04-26T21:19:26.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:19:15.844900+0000', 'modified': '2024-04-26T21:19:18.865416+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7962}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7962': {'gid': 7962, 'name': 'b', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6836/798777741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 798777741}, {'type': 'v1', 'addr': '172.21.15.184:6836', 'nonce': 798777741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7962, 'qdb_cluster': [7962]} 2024-04-26T21:19:26.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:19:26.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:19:26.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.581+0000 7f61938c4640 1 Processor -- start 2024-04-26T21:19:26.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.581+0000 7f61938c4640 1 -- start start 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f61938c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f618c0d11b0 0x7f618c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f61938c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 0x7f618c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f61938c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618c0d1740 con 0x7f618c0cf2e0 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f61938c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f618c06e710 con 0x7f618c0d11b0 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f61938c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f618c0d1a20 con 0x7f618c06e2c0 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6190e5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f618c0d11b0 0x7f618c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6191660640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f618c0cf2e0 0x7f618c06d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55390/0 (socket says 172.21.15.40:55390) 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6190e5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f618c0d11b0 0x7f618c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57294/0 (socket says 172.21.15.40:57294) 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6191660640 1 -- 172.21.15.40:0/56300448 learned_addr learned my addr 172.21.15.40:0/56300448 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6191e61640 1 --2- 172.21.15.40:0/56300448 >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 0x7f618c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6190e5f640 1 -- 172.21.15.40:0/56300448 >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 msgr2=0x7f618c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6190e5f640 1 --2- 172.21.15.40:0/56300448 >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 0x7f618c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.582+0000 7f6190e5f640 1 -- 172.21.15.40:0/56300448 >> v1:172.21.15.40:6789/0 conn(0x7f618c0cf2e0 legacy=0x7f618c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f6190e5f640 1 -- 172.21.15.40:0/56300448 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c0d1d00 con 0x7f618c0d11b0 2024-04-26T21:19:26.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f618bfff640 1 -- 172.21.15.40:0/56300448 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443944548 0 0) 0x7f618c0d1740 con 0x7f618c0cf2e0 2024-04-26T21:19:26.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f6191e61640 1 --2- 172.21.15.40:0/56300448 >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 0x7f618c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f6190e5f640 1 --2- 172.21.15.40:0/56300448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0d11b0 0x7f618c06dd80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f61800097a0 tx=0x7f618002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e27bed60e2654af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f618bfff640 1 -- 172.21.15.40:0/56300448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180037070 con 0x7f618c0d11b0 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f618bfff640 1 -- 172.21.15.40:0/56300448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6180032440 con 0x7f618c0d11b0 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.583+0000 7f618bfff640 1 -- 172.21.15.40:0/56300448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180032810 con 0x7f618c0d11b0 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 -- 172.21.15.40:0/56300448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0d11b0 msgr2=0x7f618c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 --2- 172.21.15.40:0/56300448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0d11b0 0x7f618c06dd80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f61800097a0 tx=0x7f618002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 -- 172.21.15.40:0/56300448 shutdown_connections 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 --2- 172.21.15.40:0/56300448 >> v2:172.21.15.184:3300/0 conn(0x7f618c06e2c0 0x7f618c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 --2- 172.21.15.40:0/56300448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0d11b0 0x7f618c06dd80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 -- 172.21.15.40:0/56300448 >> 172.21.15.40:0/56300448 conn(0x7f618c0b4a60 msgr2=0x7f618c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 -- 172.21.15.40:0/56300448 shutdown_connections 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.584+0000 7f61938c4640 1 -- 172.21.15.40:0/56300448 wait complete. 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.585+0000 7f61938c4640 1 Processor -- start 2024-04-26T21:19:26.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.585+0000 7f61938c4640 1 -- start start 2024-04-26T21:19:26.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618c06e2c0 0x7f618c14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618c0d11b0 0x7f618c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f618c0cf6c0 con 0x7f618c0cf2e0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f618c0d22c0 con 0x7f618c0d11b0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f61938c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f618c0cf840 con 0x7f618c06e2c0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57300/0 (socket says 172.21.15.40:57300) 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 -- 172.21.15.40:0/752299636 learned_addr learned my addr 172.21.15.40:0/752299636 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6191660640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618c06e2c0 0x7f618c14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6191e61640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618c0d11b0 0x7f618c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 -- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618c06e2c0 msgr2=0x7f618c14e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618c06e2c0 0x7f618c14e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 -- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618c0d11b0 msgr2=0x7f618c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618c0d11b0 0x7f618c0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.586+0000 7f6190e5f640 1 -- 172.21.15.40:0/752299636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c0b6b10 con 0x7f618c0cf2e0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f6190e5f640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f61800098c0 tx=0x7f61800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f6189ffb640 1 -- 172.21.15.40:0/752299636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180037040 con 0x7f618c0cf2e0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f6189ffb640 1 -- 172.21.15.40:0/752299636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6180042070 con 0x7f618c0cf2e0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c0b6df0 con 0x7f618c0cf2e0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f6189ffb640 1 -- 172.21.15.40:0/752299636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180030bd0 con 0x7f618c0cf2e0 2024-04-26T21:19:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.587+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c0b73b0 con 0x7f618c0cf2e0 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.589+0000 7f6189ffb640 1 -- 172.21.15.40:0/752299636 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6180047460 con 0x7f618c0cf2e0 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.589+0000 7f6189ffb640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 0x7f617806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.589+0000 7f6189ffb640 1 -- 172.21.15.40:0/752299636 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f61800ac760 con 0x7f618c0cf2e0 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.589+0000 7f6191660640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 0x7f617806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.590+0000 7f6191660640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 0x7f617806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f617c0012e0 tx=0x7f617c005eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.590+0000 7f61965d9740 1 Processor -- start 2024-04-26T21:19:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.590+0000 7f61965d9740 1 -- start start 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cedba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5ced90120 0x55c5cedd63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5cecf4740 con 0x55c5cedd7ce0 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5cec903a0 con 0x55c5cedba440 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c5cecf0700 con 0x55c5ced90120 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6191660640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cedba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57310/0 (socket says 172.21.15.40:57310) 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 -- 172.21.15.40:0/3809057854 learned_addr learned my addr 172.21.15.40:0/3809057854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6191e61640 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5ced90120 0x55c5cedd63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 -- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5ced90120 msgr2=0x55c5cedd63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5ced90120 0x55c5cedd63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 -- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 msgr2=0x55c5cedba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cedba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.591+0000 7f6190e5f640 1 -- 172.21.15.40:0/3809057854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cecac750 con 0x55c5cedd7ce0 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f6190e5f640 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6180076980 tx=0x7f6180079f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc20b2a71ce32fad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f6172ffd640 1 -- 172.21.15.40:0/3809057854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61800a9070 con 0x55c5cedd7ce0 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f6172ffd640 1 -- 172.21.15.40:0/3809057854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618007a520 con 0x55c5cedd7ce0 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f6172ffd640 1 -- 172.21.15.40:0/3809057854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618007e8d0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 -- 172.21.15.40:0/3809057854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 msgr2=0x55c5cedd8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6180076980 tx=0x7f6180079f40 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 -- 172.21.15.40:0/3809057854 shutdown_connections 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5ced90120 0x55c5cedd63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedd8230 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 --2- 172.21.15.40:0/3809057854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cedba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 -- 172.21.15.40:0/3809057854 >> 172.21.15.40:0/3809057854 conn(0x55c5ced8ca50 msgr2=0x55c5cedd5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 -- 172.21.15.40:0/3809057854 shutdown_connections 2024-04-26T21:19:26.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.592+0000 7f61965d9740 1 -- 172.21.15.40:0/3809057854 wait complete. 2024-04-26T21:19:26.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.593+0000 7f61965d9740 1 -- start start 2024-04-26T21:19:26.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.593+0000 7f61965d9740 1 Processor -- start 2024-04-26T21:19:26.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5ceda4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57322/0 (socket says 172.21.15.40:57322) 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5cecf4740 con 0x55c5cedd7ce0 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 -- 172.21.15.40:0/1068501752 learned_addr learned my addr 172.21.15.40:0/1068501752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5cec7bfe0 con 0x55c5cedba440 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c5ced90ec0 con 0x55c5cedd4d40 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f61965d9740 10 client.0 _ll_register_callbacks cb 0x55c5cec92150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6191660640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5ceda4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6191e61640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 -- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 msgr2=0x55c5cedd61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedd61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 -- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 msgr2=0x55c5ceda4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5ceda4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cea062d0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6191e61640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedd61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6191660640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5ceda4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f6190e5f640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6180009890 tx=0x7f618007c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f590b5580a231f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.594+0000 7f616bfff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180097060 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f6180076b90 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618007a520 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f61800a9070 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 10 client.8127 fetch_fsmap learned FSMap version 64 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55c5cead06b0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618007a520 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f618003a030 con 0x55c5cedd7ce0 2024-04-26T21:19:26.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 10 client.8127 fetch_fsmap finished waiting for FSMap version 64 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 10 client.8127 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 20 client.8127 populate_metadata read hostname 'smithi040' 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ceda3590 0x55c5ceda3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 4 client.8127 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 --> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ced02350 con 0x55c5ceda3590 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.595+0000 7f6191660640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ceda3590 0x55c5ceda3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.596+0000 7f6191660640 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ceda3590 0x55c5ceda3970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d8a2062d8a397e server_cookie=259e7b2478d3eff0 in_seq=0 out_seq=0 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.596+0000 7f616bfff640 10 client.8127 ms_handle_connect on v2:172.21.15.184:6834/798777741 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f616bfff640 1 -- 172.21.15.40:0/1068501752 <== mds.0 v2:172.21.15.184:6834/798777741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5ced02350 con 0x55c5ceda3590 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f616bfff640 10 client.8127 handle_command_reply: tid=0 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 client.8127 shutdown 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 msgr2=0x55c5ceda4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6180009890 tx=0x7f618007c000 comp rx=0 tx=0).stop 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 shutdown_connections 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ceda3590 0x55c5ceda3970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedd61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5ceda4e60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f616bfff640 0 client.8127 ms_handle_reset on v2:172.21.15.184:6834/798777741 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 --2- 172.21.15.40:0/1068501752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5ceda4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 >> 172.21.15.40:0/1068501752 conn(0x55c5ced90120 msgr2=0x55c5cedd87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 shutdown_connections 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 1 -- 172.21.15.40:0/1068501752 wait complete. 2024-04-26T21:19:26.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.597+0000 7f61965d9740 20 client.8127 trim_cache size 0 max 16384 2024-04-26T21:19:26.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.615+0000 7f61965d9740 1 Processor -- start 2024-04-26T21:19:26.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.615+0000 7f61965d9740 1 -- start start 2024-04-26T21:19:26.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5ced90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedd7ce0 0x55c5ced90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f618c200950 con 0x55c5cedba440 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f618c15bae0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f618c15c130 con 0x55c5cedd4d40 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57332/0 (socket says 172.21.15.40:57332) 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 -- 172.21.15.40:0/160431728 learned_addr learned my addr 172.21.15.40:0/160431728 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 -- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 msgr2=0x55c5ced90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5ced90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 -- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedd7ce0 msgr2=0x55c5ced90a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedd7ce0 0x55c5ced90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 -- 172.21.15.40:0/160431728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cedbab90 con 0x55c5cedba440 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.616+0000 7f6191660640 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f618c1f8660 tx=0x7f617c009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61a13a65b9a578d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61727fc640 1 -- 172.21.15.40:0/160431728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c02c070 con 0x55c5cedba440 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61727fc640 1 -- 172.21.15.40:0/160431728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c001ce0 con 0x55c5cedba440 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61727fc640 1 -- 172.21.15.40:0/160431728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c013db0 con 0x55c5cedba440 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 -- 172.21.15.40:0/160431728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 msgr2=0x55c5cedb8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f618c1f8660 tx=0x7f617c009030 comp rx=0 tx=0).stop 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 -- 172.21.15.40:0/160431728 shutdown_connections 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedd7ce0 0x55c5ced90a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5ced90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 --2- 172.21.15.40:0/160431728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedba440 0x55c5cedb8b50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 -- 172.21.15.40:0/160431728 >> 172.21.15.40:0/160431728 conn(0x55c5ced90120 msgr2=0x55c5cedcbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 -- 172.21.15.40:0/160431728 shutdown_connections 2024-04-26T21:19:26.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.617+0000 7f61965d9740 1 -- 172.21.15.40:0/160431728 wait complete. 2024-04-26T21:19:26.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.618+0000 7f61965d9740 1 -- start start 2024-04-26T21:19:26.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.618+0000 7f61965d9740 1 Processor -- start 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cee4f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedb47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55c5cecf4740 con 0x55c5cedd7ce0 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5cec7bfe0 con 0x55c5cedba440 2024-04-26T21:19:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x55c5ced896c0 con 0x55c5cedd4d40 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 10 client.0 _ll_register_callbacks cb 0x7f61800a7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57348/0 (socket says 172.21.15.40:57348) 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 -- 172.21.15.40:0/1653864376 learned_addr learned my addr 172.21.15.40:0/1653864376 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191660640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cee4f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6190e5f640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedb47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 -- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 msgr2=0x55c5cedb47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedb47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 -- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 msgr2=0x55c5cee4f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cee4f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 -- 172.21.15.40:0/1653864376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ced8ef20 con 0x55c5cedd7ce0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f6191e61640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6184004700 tx=0x7f618400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf84bad64837d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f616bfff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:19:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184019070 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f6184015450 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.619+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5cecfc1c0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6184024050 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 10 client.8133 fetch_fsmap learned FSMap version 64 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55c5ced8ebe0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c5cecfc1c0 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f6184027020 con 0x55c5cedd7ce0 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 10 client.8133 fetch_fsmap finished waiting for FSMap version 64 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 10 client.8133 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 20 client.8133 populate_metadata read hostname 'smithi040' 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ced90c20 0x55c5cedb72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 4 client.8133 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 --> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] -- command(tid 0: {"prefix": "status"}) -- 0x55c5cec779a0 con 0x55c5ced90c20 2024-04-26T21:19:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.620+0000 7f6191660640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ced90c20 0x55c5cedb72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f6191660640 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ced90c20 0x55c5cedb72e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b78ba3cd833bc86 server_cookie=924eec822dd94532 in_seq=0 out_seq=0 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f616bfff640 10 client.8133 ms_handle_connect on v2:172.21.15.184:6834/798777741 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f616bfff640 1 -- 172.21.15.40:0/1653864376 <== mds.0 v2:172.21.15.184:6834/798777741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55c5cec779a0 con 0x55c5ced90c20 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f616bfff640 10 client.8133 handle_command_reply: tid=0 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 client.8133 shutdown 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 msgr2=0x55c5cedb4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6184004700 tx=0x7f618400c850 comp rx=0 tx=0).stop 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 shutdown_connections 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:6834/798777741,v1:172.21.15.184:6836/798777741] conn(0x55c5ced90c20 0x55c5cedb72e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5cedd7ce0 0x55c5cedb4cf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c5cedd4d40 0x55c5cedb47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f616bfff640 0 client.8133 ms_handle_reset on v2:172.21.15.184:6834/798777741 2024-04-26T21:19:26.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 --2- 172.21.15.40:0/1653864376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5cedba440 0x55c5cee4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 >> 172.21.15.40:0/1653864376 conn(0x55c5ced90120 msgr2=0x55c5ceda4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 shutdown_connections 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 1 -- 172.21.15.40:0/1653864376 wait complete. 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.621+0000 7f61965d9740 20 client.8133 trim_cache size 0 max 16384 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7962, 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:19:26.628 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.151286877, 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 64, 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 109, 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 109, 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.478513459 2024-04-26T21:19:26.629 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 msgr2=0x7f617806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 0x7f617806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f617c0012e0 tx=0x7f617c005eb0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 msgr2=0x7f618c0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f61800098c0 tx=0x7f61800046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 shutdown_connections 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f617806c8f0 0x7f617806edb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f618c0d11b0 0x7f618c0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f618c0cf2e0 0x7f618c0b61a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 --2- 172.21.15.40:0/752299636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f618c06e2c0 0x7f618c14e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 >> 172.21.15.40:0/752299636 conn(0x7f618c0b4a60 msgr2=0x7f618c075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 shutdown_connections 2024-04-26T21:19:26.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.622+0000 7f61938c4640 1 -- 172.21.15.40:0/752299636 wait complete. 2024-04-26T21:19:26.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:19:26.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.733+0000 7fa50c0c8640 1 Processor -- start 2024-04-26T21:19:26.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.733+0000 7fa50c0c8640 1 -- start start 2024-04-26T21:19:26.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50c0c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 0x7fa5040b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50c0c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa50415f720 0x7fa5040d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50c0c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5040b2510 con 0x7fa5040d4f00 2024-04-26T21:19:26.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50c0c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa5040b27f0 con 0x7fa50415f720 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50c0c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5040b2ad0 con 0x7fa50415f340 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509663640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa50415f720 0x7fa5040d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50a665640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa5040d4f00 0x7fa5040d5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55394/0 (socket says 172.21.15.40:55394) 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa50a665640 1 -- 172.21.15.40:0/408410385 learned_addr learned my addr 172.21.15.40:0/408410385 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509e64640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 0x7fa5040b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509663640 1 -- 172.21.15.40:0/408410385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 msgr2=0x7fa5040b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509663640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 0x7fa5040b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509663640 1 -- 172.21.15.40:0/408410385 >> v1:172.21.15.40:6789/0 conn(0x7fa5040d4f00 legacy=0x7fa5040d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa508e62640 1 -- 172.21.15.40:0/408410385 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3206215293 0 0) 0x7fa5040b2510 con 0x7fa5040d4f00 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509663640 1 -- 172.21.15.40:0/408410385 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5040b2db0 con 0x7fa50415f720 2024-04-26T21:19:26.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.734+0000 7fa509e64640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 0x7fa5040b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:26.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.735+0000 7fa509663640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa5040d3d40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40097a0 tx=0x7fa4f402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d198bd83bf24bba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.735+0000 7fa508e62640 1 -- 172.21.15.40:0/408410385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4037070 con 0x7fa50415f720 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.735+0000 7fa508e62640 1 -- 172.21.15.40:0/408410385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f402ae50 con 0x7fa50415f720 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.735+0000 7fa508e62640 1 -- 172.21.15.40:0/408410385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4032660 con 0x7fa50415f720 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 -- 172.21.15.40:0/408410385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 msgr2=0x7fa5040d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa5040d3d40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40097a0 tx=0x7fa4f402a100 comp rx=0 tx=0).stop 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 -- 172.21.15.40:0/408410385 shutdown_connections 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa5040d3d40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/408410385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa50415f340 0x7fa5040b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 -- 172.21.15.40:0/408410385 >> 172.21.15.40:0/408410385 conn(0x7fa50408ec70 msgr2=0x7fa50409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 -- 172.21.15.40:0/408410385 shutdown_connections 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.736+0000 7fa50c0c8640 1 -- 172.21.15.40:0/408410385 wait complete. 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.737+0000 7fa50c0c8640 1 Processor -- start 2024-04-26T21:19:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.737+0000 7fa50c0c8640 1 -- start start 2024-04-26T21:19:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5040d4f00 0x7fa5041537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 0x7fa5040b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5040b3980 con 0x7fa50415f720 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5040b3800 con 0x7fa50415f340 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50c0c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa5040b3b00 con 0x7fa5040d4f00 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57360/0 (socket says 172.21.15.40:57360) 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 -- 172.21.15.40:0/2077149386 learned_addr learned my addr 172.21.15.40:0/2077149386 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa509663640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 0x7fa5040b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa509e64640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5040d4f00 0x7fa5041537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 -- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5040d4f00 msgr2=0x7fa5041537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5040d4f00 0x7fa5041537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 -- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 msgr2=0x7fa5040b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 0x7fa5040b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 -- 172.21.15.40:0/2077149386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5040b92f0 con 0x7fa50415f720 2024-04-26T21:19:26.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa509663640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 0x7fa5040b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa50a665640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c7a0 tx=0x7fa4fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.738+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018070 con 0x7fa50415f720 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.739+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc00a640 con 0x7fa50415f720 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.739+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5040b95d0 con 0x7fa50415f720 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.739+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00aa70 con 0x7fa50415f720 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.739+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5040b9b90 con 0x7fa50415f720 2024-04-26T21:19:26.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.739+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50406a290 con 0x7fa50415f720 2024-04-26T21:19:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.740+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4fc01b070 con 0x7fa50415f720 2024-04-26T21:19:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.740+0000 7fa4fa7fc640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 0x7fa4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.741+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa4fc08b180 con 0x7fa50415f720 2024-04-26T21:19:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.741+0000 7fa509e64640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 0x7fa4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:26.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.741+0000 7fa509e64640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 0x7fa4e806edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa50000a990 tx=0x7fa500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:26.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.745+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4fc0595b0 con 0x7fa50415f720 2024-04-26T21:19:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.970+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa5040d3020 con 0x7fa50415f720 2024-04-26T21:19:26.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.971+0000 7fa4fa7fc640 1 -- 172.21.15.40:0/2077149386 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7fa4fc058eb0 con 0x7fa50415f720 2024-04-26T21:19:26.972 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:19:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 msgr2=0x7fa4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 0x7fa4e806edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa50000a990 tx=0x7fa500008040 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 msgr2=0x7fa50415fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c7a0 tx=0x7fa4fc00cc60 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 shutdown_connections 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4e806c8f0 0x7fa4e806edb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa50415f720 0x7fa50415fb20 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa50415f340 0x7fa5040b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 --2- 172.21.15.40:0/2077149386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5040d4f00 0x7fa5041537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 >> 172.21.15.40:0/2077149386 conn(0x7fa50408ec70 msgr2=0x7fa5040ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 shutdown_connections 2024-04-26T21:19:26.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:26.972+0000 7fa50c0c8640 1 -- 172.21.15.40:0/2077149386 wait complete. 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:19:26.984 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:19:26.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:19:26.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:19:26.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:19:26.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:26.985 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:19:27.000 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:19:27.000 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:19:27.001 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42303sec preferred_lft 42303sec 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:19:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.003 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:19:27.003 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:19:27.003 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:19:27.003 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:27.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.003 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:19:27.063 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:19:27.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.065 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:19:27.121 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:19:27.122 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:19:27.122 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:19:27.122 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:19:27.122 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:19:27.122 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:19:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:19:27.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:27.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.248 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:19:27.264 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:19:27.264 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:19:27.264 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:19:27.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:19:27.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:19:27.328 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:19:27.383 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:19:27.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:19:27.384 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:19:27.438 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:19:27.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:19:27.522 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:19:27.522 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:19:27.522 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:19:27.531 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:19:27.531 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:19:27.531 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:19:27.531 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:19:27.532 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:19:27.550 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:19:27.551 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:19:27.553 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:19:27.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:27.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:19:27.619 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:19:27.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:19:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:27.723 INFO:teuthology.orchestra.run.smithi040.stdout:client6997 2024-04-26T21:19:27.723 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:27.723 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:27.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:19:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.878+0000 7f98fe912640 1 Processor -- start 2024-04-26T21:19:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.878+0000 7f98fe912640 1 -- start start 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98fe912640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98f815f340 0x7f98f80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98fe912640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f720 0x7f98f80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98fe912640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f80b2510 con 0x7f98f80d4f00 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98fe912640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f98f80b27f0 con 0x7f98f815f340 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98fe912640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98f80b2ad0 con 0x7f98f815f720 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98f815f340 0x7f98f80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98f815f340 0x7f98f80b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57362/0 (socket says 172.21.15.40:57362) 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 -- 172.21.15.40:0/3834604613 learned_addr learned my addr 172.21.15.40:0/3834604613 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f77fe640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f720 0x7f98f80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 -- 172.21.15.40:0/3834604613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f720 msgr2=0x7f98f80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f720 0x7f98f80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.879+0000 7f98f7fff640 1 -- 172.21.15.40:0/3834604613 >> v1:172.21.15.184:6789/0 conn(0x7f98f80d4f00 legacy=0x7f98f80d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.880+0000 7f98f7fff640 1 -- 172.21.15.40:0/3834604613 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f80b2db0 con 0x7f98f815f340 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.880+0000 7f98f7fff640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f815f340 0x7f98f80b1e80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009870 tx=0x7f98e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732aa2981b8b588 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.880+0000 7f98f6ffd640 1 -- 172.21.15.40:0/3834604613 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0037070 con 0x7f98f815f340 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.880+0000 7f98f6ffd640 1 -- 172.21.15.40:0/3834604613 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e0032440 con 0x7f98f815f340 2024-04-26T21:19:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.880+0000 7f98f6ffd640 1 -- 172.21.15.40:0/3834604613 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0032810 con 0x7f98f815f340 2024-04-26T21:19:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 -- 172.21.15.40:0/3834604613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f815f340 msgr2=0x7f98f80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f815f340 0x7f98f80b1e80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009870 tx=0x7f98e002a180 comp rx=0 tx=0).stop 2024-04-26T21:19:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 -- 172.21.15.40:0/3834604613 shutdown_connections 2024-04-26T21:19:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f720 0x7f98f80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 --2- 172.21.15.40:0/3834604613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f815f340 0x7f98f80b1e80 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 -- 172.21.15.40:0/3834604613 >> 172.21.15.40:0/3834604613 conn(0x7f98f808ec70 msgr2=0x7f98f809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 -- 172.21.15.40:0/3834604613 shutdown_connections 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.881+0000 7f98fe912640 1 -- 172.21.15.40:0/3834604613 wait complete. 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.882+0000 7f98fe912640 1 Processor -- start 2024-04-26T21:19:27.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.882+0000 7f98fe912640 1 -- start start 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f340 0x7f98f80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98f815f720 0x7f98f80cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80b3980 con 0x7f98f80d4f00 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80b3800 con 0x7f98f815f720 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fe912640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98f80b3b00 con 0x7f98f815f340 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57376/0 (socket says 172.21.15.40:57376) 2024-04-26T21:19:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98f7fff640 1 -- 172.21.15.40:0/1637125283 learned_addr learned my addr 172.21.15.40:0/1637125283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98f77fe640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f340 0x7f98f80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.883+0000 7f98fceaf640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98f815f720 0x7f98f80cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 -- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f340 msgr2=0x7f98f80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f340 0x7f98f80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 -- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98f815f720 msgr2=0x7f98f80cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98f815f720 0x7f98f80cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 -- 172.21.15.40:0/1637125283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f80cdbd0 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f7fff640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f98e0000c00 tx=0x7f98e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0041020 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e00306c0 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f80cdeb0 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0037040 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.884+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f80ce470 con 0x7f98f80d4f00 2024-04-26T21:19:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.885+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f806a290 con 0x7f98f80d4f00 2024-04-26T21:19:27.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.885+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98e0044020 con 0x7f98f80d4f00 2024-04-26T21:19:27.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.886+0000 7f98f4ff9640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 0x7f98d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:27.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.886+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f98e00ac2c0 con 0x7f98f80d4f00 2024-04-26T21:19:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.886+0000 7f98f77fe640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 0x7f98d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.887+0000 7f98f77fe640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 0x7f98d806edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:27.890+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98e004b030 con 0x7f98f80d4f00 2024-04-26T21:19:28.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.113+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98f815fe10 con 0x7f98f80d4f00 2024-04-26T21:19:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.114+0000 7f98f4ff9640 1 -- 172.21.15.40:0/1637125283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f98e007a780 con 0x7f98f80d4f00 2024-04-26T21:19:28.115 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.115+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 msgr2=0x7f98d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.115+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 0x7f98d806edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8008040 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.115+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 msgr2=0x7f98f814f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.115+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f98e0000c00 tx=0x7f98e00046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 shutdown_connections 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f98d806c8f0 0x7f98d806edb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98f815f720 0x7f98f80cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98f815f340 0x7f98f80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 --2- 172.21.15.40:0/1637125283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f80d4f00 0x7f98f814f640 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 >> 172.21.15.40:0/1637125283 conn(0x7f98f808ec70 msgr2=0x7f98f80b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 shutdown_connections 2024-04-26T21:19:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.116+0000 7f98fe912640 1 -- 172.21.15.40:0/1637125283 wait complete. 2024-04-26T21:19:28.127 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:19:28.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:19:28.128 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:19:28.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.128 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:19:28.147 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:19:28.147 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:28.148 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42305sec preferred_lft 42305sec 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:19:28.149 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:19:28.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.150 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:19:28.214 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:19:28.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.216 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:19:28.275 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:19:28.275 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:19:28.275 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:19:28.275 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:19:28.275 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:19:28.276 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:19:28.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:19:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:19:28.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:19:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:28.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.410 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:19:28.424 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:19:28.424 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:19:28.425 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:19:28.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.425 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:19:28.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:19:28.489 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:19:28.544 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:19:28.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:19:28.544 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:19:28.598 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:19:28.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.599 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:19:28.681 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:19:28.681 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:19:28.681 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:19:28.689 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:19:28.689 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:19:28.690 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:19:28.690 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:19:28.690 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:19:28.694 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:19:28.694 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:19:28.694 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:19:28.694 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:19:28.694 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:19:28.695 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:19:28.695 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:19:28.695 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:19:28.698 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:19:28.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:28.699 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:19:28.769 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:19:28.857 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:19:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:19:28.858 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:19:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:28.864 INFO:teuthology.orchestra.run.smithi184.stdout:client6284 2024-04-26T21:19:28.864 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:28.864 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:28.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-26T21:19:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.994+0000 7f416fa45640 1 Processor -- start 2024-04-26T21:19:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 -- start start 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f416815f680 0x7f41680d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4168163aa0 0x7f41680b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41680d3f50 con 0x7f41680b60b0 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f416815fa60 con 0x7f4168163aa0 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.995+0000 7f416fa45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f41680b5090 con 0x7f416815f680 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416cfe0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4168163aa0 0x7f41680b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416dfe2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f41680b60b0 0x7f41680d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55410/0 (socket says 172.21.15.40:55410) 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416dfe2640 1 -- 172.21.15.40:0/2537056201 learned_addr learned my addr 172.21.15.40:0/2537056201 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416d7e1640 1 --2- 172.21.15.40:0/2537056201 >> v2:172.21.15.184:3300/0 conn(0x7f416815f680 0x7f41680d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416cfe0640 1 -- 172.21.15.40:0/2537056201 >> v2:172.21.15.184:3300/0 conn(0x7f416815f680 msgr2=0x7f41680d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416cfe0640 1 --2- 172.21.15.40:0/2537056201 >> v2:172.21.15.184:3300/0 conn(0x7f416815f680 0x7f41680d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416cfe0640 1 -- 172.21.15.40:0/2537056201 >> v1:172.21.15.40:6789/0 conn(0x7f41680b60b0 legacy=0x7f41680d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.996+0000 7f416cfe0640 1 -- 172.21.15.40:0/2537056201 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41680b5370 con 0x7f4168163aa0 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f416cfe0640 1 --2- 172.21.15.40:0/2537056201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41680b5b70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f41680d6370 tx=0x7f415802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51d7b6170ac5c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f415ffff640 1 -- 172.21.15.40:0/2537056201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158037070 con 0x7f4168163aa0 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f415ffff640 1 -- 172.21.15.40:0/2537056201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4158032440 con 0x7f4168163aa0 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f415ffff640 1 -- 172.21.15.40:0/2537056201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158032810 con 0x7f4168163aa0 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f416fa45640 1 -- 172.21.15.40:0/2537056201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 msgr2=0x7f41680b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f416fa45640 1 --2- 172.21.15.40:0/2537056201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41680b5b70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f41680d6370 tx=0x7f415802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.997+0000 7f416fa45640 1 -- 172.21.15.40:0/2537056201 shutdown_connections 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.998+0000 7f416fa45640 1 --2- 172.21.15.40:0/2537056201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41680b5b70 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.998+0000 7f416fa45640 1 --2- 172.21.15.40:0/2537056201 >> v2:172.21.15.184:3300/0 conn(0x7f416815f680 0x7f41680d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.998+0000 7f416fa45640 1 -- 172.21.15.40:0/2537056201 >> 172.21.15.40:0/2537056201 conn(0x7f416808eca0 msgr2=0x7f416809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.998+0000 7f416fa45640 1 -- 172.21.15.40:0/2537056201 shutdown_connections 2024-04-26T21:19:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.998+0000 7f416fa45640 1 -- 172.21.15.40:0/2537056201 wait complete. 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.999+0000 7f416fa45640 1 Processor -- start 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:28.999+0000 7f416fa45640 1 -- start start 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41680b60b0 0x7f41680d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f416815f680 0x7f41681f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41680d57b0 con 0x7f4168163aa0 2024-04-26T21:19:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41680d5630 con 0x7f416815f680 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41680d5930 con 0x7f41680b60b0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57390/0 (socket says 172.21.15.40:57390) 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 -- 172.21.15.40:0/2397124579 learned_addr learned my addr 172.21.15.40:0/2397124579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416cfe0640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f416815f680 0x7f41681f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416d7e1640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41680b60b0 0x7f41680d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 -- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41680b60b0 msgr2=0x7f41680d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41680b60b0 0x7f41680d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 -- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f416815f680 msgr2=0x7f41681f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f416815f680 0x7f41681f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 -- 172.21.15.40:0/2397124579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41680cbb90 con 0x7f4168163aa0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416dfe2640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f416000caf0 tx=0x7f416000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160017070 con 0x7f4168163aa0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416000aca0 con 0x7f4168163aa0 2024-04-26T21:19:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681f68b0 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.000+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160012600 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.001+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681f9360 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.002+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416806a290 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.002+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f416001a070 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.002+0000 7f415dffb640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.002+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f416008b800 con 0x7f4168163aa0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.003+0000 7f416d7e1640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.003+0000 7f416d7e1640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 0x7f414c06ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4164009b40 tx=0x7f4164009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.007+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4160059540 con 0x7f4168163aa0 2024-04-26T21:19:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.195+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f41680b26d0 con 0x7f4168163aa0 2024-04-26T21:19:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.198+0000 7f415dffb640 1 -- 172.21.15.40:0/2397124579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v37)=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7f416005d840 con 0x7f4168163aa0 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 msgr2=0x7f414c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 0x7f414c06ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4164009b40 tx=0x7f4164009190 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 msgr2=0x7f41681f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f416000caf0 tx=0x7f416000a060 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 shutdown_connections 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f414c06c920 0x7f414c06ede0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168163aa0 0x7f41681f5f50 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f416815f680 0x7f41681f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.200+0000 7f416fa45640 1 --2- 172.21.15.40:0/2397124579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41680b60b0 0x7f41680d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.201+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 >> 172.21.15.40:0/2397124579 conn(0x7f416808eca0 msgr2=0x7f416815db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.201+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 shutdown_connections 2024-04-26T21:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.201+0000 7f416fa45640 1 -- 172.21.15.40:0/2397124579 wait complete. 2024-04-26T21:19:29.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2024-04-26T21:19:29.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.315+0000 7efdf0c52640 1 Processor -- start 2024-04-26T21:19:29.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 -- start start 2024-04-26T21:19:29.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdec15f680 0x7efdec0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 0x7efdec0d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec0d58c0 con 0x7efdec163aa0 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdec15fa60 con 0x7efdec15f680 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdf0c52640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efdec0d3f50 con 0x7efdec0d4f20 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.316+0000 7efdea59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdec15f680 0x7efdec0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdec15f680 0x7efdec0d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57392/0 (socket says 172.21.15.40:57392) 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 -- 172.21.15.40:0/3746091967 learned_addr learned my addr 172.21.15.40:0/3746091967 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdead9d640 1 --2- 172.21.15.40:0/3746091967 >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 0x7efdec0d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 -- 172.21.15.40:0/3746091967 >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 msgr2=0x7efdec0d5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 --2- 172.21.15.40:0/3746091967 >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 0x7efdec0d5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 -- 172.21.15.40:0/3746091967 >> v1:172.21.15.184:6789/0 conn(0x7efdec163aa0 legacy=0x7efdec0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 -- 172.21.15.40:0/3746091967 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec0b5090 con 0x7efdec15f680 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdead9d640 1 --2- 172.21.15.40:0/3746091967 >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 0x7efdec0d5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.317+0000 7efdea59c640 1 --2- 172.21.15.40:0/3746091967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec0d3820 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efdec0cf840 tx=0x7efddc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81af468e2e0e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efde9d9b640 1 -- 172.21.15.40:0/3746091967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc037070 con 0x7efdec15f680 2024-04-26T21:19:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efde9d9b640 1 -- 172.21.15.40:0/3746091967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc032440 con 0x7efdec15f680 2024-04-26T21:19:29.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efde9d9b640 1 -- 172.21.15.40:0/3746091967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc032810 con 0x7efdec15f680 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 -- 172.21.15.40:0/3746091967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 msgr2=0x7efdec0d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 --2- 172.21.15.40:0/3746091967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec0d3820 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efdec0cf840 tx=0x7efddc02a270 comp rx=0 tx=0).stop 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 -- 172.21.15.40:0/3746091967 shutdown_connections 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 --2- 172.21.15.40:0/3746091967 >> v2:172.21.15.184:3300/0 conn(0x7efdec0d4f20 0x7efdec0d5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 --2- 172.21.15.40:0/3746091967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec0d3820 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 -- 172.21.15.40:0/3746091967 >> 172.21.15.40:0/3746091967 conn(0x7efdec08eca0 msgr2=0x7efdec09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 -- 172.21.15.40:0/3746091967 shutdown_connections 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.318+0000 7efdf0c52640 1 -- 172.21.15.40:0/3746091967 wait complete. 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 Processor -- start 2024-04-26T21:19:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 -- start start 2024-04-26T21:19:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdec0d4f20 0x7efdec0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec163aa0 0x7efdec0b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdec163f30 con 0x7efdec15f680 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0b6280 con 0x7efdec0d4f20 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.320+0000 7efdf0c52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0cfd80 con 0x7efdec163aa0 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57400/0 (socket says 172.21.15.40:57400) 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 -- 172.21.15.40:0/2450724575 learned_addr learned my addr 172.21.15.40:0/2450724575 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efdea59c640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdec0d4f20 0x7efdec0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efdead9d640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec163aa0 0x7efdec0b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 -- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec163aa0 msgr2=0x7efdec0b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec163aa0 0x7efdec0b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 -- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdec0d4f20 msgr2=0x7efdec0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdec0d4f20 0x7efdec0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 -- 172.21.15.40:0/2450724575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec15ad10 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde3fff640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c3f0 tx=0x7efdd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40052d0 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd4005490 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec160fc0 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4007260 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.321+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec161580 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.322+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec0d54a0 con 0x7efdec15f680 2024-04-26T21:19:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.323+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdd4007400 con 0x7efdec15f680 2024-04-26T21:19:29.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.323+0000 7efde2ffd640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 0x7efdcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.323+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7efdd408af90 con 0x7efdec15f680 2024-04-26T21:19:29.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.323+0000 7efdea59c640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 0x7efdcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.324+0000 7efdea59c640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 0x7efdcc06edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efddc034000 tx=0x7efddc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.328+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdec0d54a0 con 0x7efdec15f680 2024-04-26T21:19:29.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.514+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7efdec0b4cd0 con 0x7efdec15f680 2024-04-26T21:19:29.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.517+0000 7efde2ffd640 1 -- 172.21.15.40:0/2450724575 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v38)=0 v38) ==== 119+0+0 (secure 0 0 0) 0x7efdec15ad10 con 0x7efdec15f680 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 msgr2=0x7efdcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 0x7efdcc06edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efddc034000 tx=0x7efddc02e000 comp rx=0 tx=0).stop 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 msgr2=0x7efdec15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c3f0 tx=0x7efdd400c8b0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 shutdown_connections 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdcc06c8f0 0x7efdcc06edb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdec163aa0 0x7efdec0b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec15f680 0x7efdec15a7d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 --2- 172.21.15.40:0/2450724575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdec0d4f20 0x7efdec0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 >> 172.21.15.40:0/2450724575 conn(0x7efdec08eca0 msgr2=0x7efdec163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 shutdown_connections 2024-04-26T21:19:29.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.519+0000 7efdf0c52640 1 -- 172.21.15.40:0/2450724575 wait complete. 2024-04-26T21:19:29.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2024-04-26T21:19:29.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.634+0000 7f3d74ca9640 1 Processor -- start 2024-04-26T21:19:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.634+0000 7f3d74ca9640 1 -- start start 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d74ca9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700b4eb0 0x7f3d700b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d74ca9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d700ceff0 0x7f3d700cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d74ca9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7015dca0 con 0x7f3d70143840 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d74ca9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3d70143c20 con 0x7f3d700ceff0 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d74ca9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7015df80 con 0x7f3d700b4eb0 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d700ceff0 0x7f3d700cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d70143840 0x7f3d700b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55414/0 (socket says 172.21.15.40:55414) 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6e59c640 1 -- 172.21.15.40:0/3374542469 learned_addr learned my addr 172.21.15.40:0/3374542469 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6dd9b640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700b4eb0 0x7f3d700b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6ed9d640 1 -- 172.21.15.40:0/3374542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700b4eb0 msgr2=0x7f3d700b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6ed9d640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700b4eb0 0x7f3d700b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6ed9d640 1 -- 172.21.15.40:0/3374542469 >> v1:172.21.15.40:6789/0 conn(0x7f3d70143840 legacy=0x7f3d700b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.635+0000 7f3d6ed9d640 1 -- 172.21.15.40:0/3374542469 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7015e260 con 0x7f3d700ceff0 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d6ed9d640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700ceff0 0x7f3d700cf450 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d67a7d98add8005 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d6d59a640 1 -- 172.21.15.40:0/3374542469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007740 con 0x7f3d700ceff0 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d6d59a640 1 -- 172.21.15.40:0/3374542469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d680078e0 con 0x7f3d700ceff0 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d6d59a640 1 -- 172.21.15.40:0/3374542469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007cb0 con 0x7f3d700ceff0 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d74ca9640 1 -- 172.21.15.40:0/3374542469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700ceff0 msgr2=0x7f3d700cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.636+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700ceff0 0x7f3d700cf450 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6802a660 comp rx=0 tx=0).stop 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 -- 172.21.15.40:0/3374542469 shutdown_connections 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700ceff0 0x7f3d700cf450 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/3374542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700b4eb0 0x7f3d700b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 -- 172.21.15.40:0/3374542469 >> 172.21.15.40:0/3374542469 conn(0x7f3d700cd8a0 msgr2=0x7f3d700cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 -- 172.21.15.40:0/3374542469 shutdown_connections 2024-04-26T21:19:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.637+0000 7f3d74ca9640 1 -- 172.21.15.40:0/3374542469 wait complete. 2024-04-26T21:19:29.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.638+0000 7f3d74ca9640 1 Processor -- start 2024-04-26T21:19:29.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.638+0000 7f3d74ca9640 1 -- start start 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700ceff0 0x7f3d700ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d70143840 0x7f3d700afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7015d8c0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700b46f0 con 0x7f3d70143840 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7015da40 con 0x7f3d700ceff0 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57422/0 (socket says 172.21.15.40:57422) 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 -- 172.21.15.40:0/510906508 learned_addr learned my addr 172.21.15.40:0/510906508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6ed9d640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d70143840 0x7f3d700afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6dd9b640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700ceff0 0x7f3d700ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 -- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700ceff0 msgr2=0x7f3d700ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700ceff0 0x7f3d700ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 -- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d70143840 msgr2=0x7f3d700afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d70143840 0x7f3d700afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 -- 172.21.15.40:0/510906508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7012f6c0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d6e59c640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c3c0 tx=0x7f3d6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60017070 con 0x7f3d700b4eb0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60005ce0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600072f0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.639+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d700ae200 con 0x7f3d700b4eb0 2024-04-26T21:19:29.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.640+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7012fc40 con 0x7f3d700b4eb0 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.641+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7006a290 con 0x7f3d700b4eb0 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.641+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d6001a070 con 0x7f3d700b4eb0 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.641+0000 7f3d5effd640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 0x7f3d5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.641+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f3d6008adf0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.642+0000 7f3d6dd9b640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 0x7f3d5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.642+0000 7f3d6dd9b640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 0x7f3d5006ede0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3d70132770 tx=0x7f3d64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.646+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d600592b0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.829+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f3d70142c10 con 0x7f3d700b4eb0 2024-04-26T21:19:29.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.832+0000 7f3d5effd640 1 -- 172.21.15.40:0/510906508 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v39)=0 v39) ==== 128+0+0 (secure 0 0 0) 0x7f3d60058bb0 con 0x7f3d700b4eb0 2024-04-26T21:19:29.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 msgr2=0x7f3d5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 0x7f3d5006ede0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3d70132770 tx=0x7f3d64008040 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 msgr2=0x7f3d7015a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c3c0 tx=0x7f3d6000c880 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 shutdown_connections 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d5006c920 0x7f3d5006ede0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d70143840 0x7f3d700afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d700ceff0 0x7f3d700ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 --2- 172.21.15.40:0/510906508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d700b4eb0 0x7f3d7015a920 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.834+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 >> 172.21.15.40:0/510906508 conn(0x7f3d700cd8a0 msgr2=0x7f3d700b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.835+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 shutdown_connections 2024-04-26T21:19:29.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.835+0000 7f3d74ca9640 1 -- 172.21.15.40:0/510906508 wait complete. 2024-04-26T21:19:29.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-26T21:19:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.948+0000 7f43e2a09640 1 Processor -- start 2024-04-26T21:19:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e2a09640 1 -- start start 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e2a09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43dc1631f0 0x7f43dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e2a09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc09c790 con 0x7f43dc06ea30 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e2a09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc163ec0 con 0x7f43dc0d1170 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e2a09640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f43dc06eec0 con 0x7f43dc1631f0 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e0fa6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43dc06ea30 0x7f43dc0b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55418/0 (socket says 172.21.15.40:55418) 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.949+0000 7f43e0fa6640 1 -- 172.21.15.40:0/1819316419 learned_addr learned my addr 172.21.15.40:0/1819316419 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1819316419 >> v2:172.21.15.184:3300/0 conn(0x7f43dc1631f0 0x7f43dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218388412 0 0) 0x7f43dc09c790 con 0x7f43dc06ea30 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43c4003520 con 0x7f43dc06ea30 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2804462885 0 0) 0x7f43c4003520 con 0x7f43dc06ea30 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 >> v2:172.21.15.184:3300/0 conn(0x7f43dc1631f0 msgr2=0x7f43dc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 --2- 172.21.15.40:0/1819316419 >> v2:172.21.15.184:3300/0 conn(0x7f43dc1631f0 0x7f43dc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 >> v1:172.21.15.184:6789/0 conn(0x7f43dc0d1170 legacy=0x7f43dc1635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc0d1a40 con 0x7f43dc06ea30 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f43d40019f0 con 0x7f43dc06ea30 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 >> v1:172.21.15.40:6789/0 conn(0x7f43dc06ea30 legacy=0x7f43dc0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40039b0 0x7f43c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40043b0 0x7f43c4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 0x7f43c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43c400c030 con 0x7f43c4005120 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43c400c1e0 con 0x7f43c40039b0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43c400c3e0 con 0x7f43c40043b0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 0x7f43c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43db7fe640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40043b0 0x7f43c4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.950+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40039b0 0x7f43c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 -- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40043b0 msgr2=0x7f43c4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c40043b0 0x7f43c4004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 -- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40039b0 msgr2=0x7f43c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c40039b0 0x7f43c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 -- 172.21.15.40:0/1819316419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c4003520 con 0x7f43c4005120 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 -- 172.21.15.40:0/1819316419 reap_dead start 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 0x7f43c400b9c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f43d400a920 tx=0x7f43c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8813b1cd69e6c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d400a4c0 con 0x7f43c4005120 2024-04-26T21:19:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d402f070 con 0x7f43c4005120 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.951+0000 7f43daffd640 1 -- 172.21.15.40:0/1819316419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d402bce0 con 0x7f43c4005120 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 -- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 msgr2=0x7f43c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 0x7f43c400b9c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f43d400a920 tx=0x7f43c4005830 comp rx=0 tx=0).stop 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 -- 172.21.15.40:0/1819316419 shutdown_connections 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1819316419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c4005120 0x7f43c400b9c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 -- 172.21.15.40:0/1819316419 >> 172.21.15.40:0/1819316419 conn(0x7f43dc09ba40 msgr2=0x7f43dc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 -- 172.21.15.40:0/1819316419 shutdown_connections 2024-04-26T21:19:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.952+0000 7f43e2a09640 1 -- 172.21.15.40:0/1819316419 wait complete. 2024-04-26T21:19:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.953+0000 7f43e2a09640 1 Processor -- start 2024-04-26T21:19:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.953+0000 7f43e2a09640 1 -- start start 2024-04-26T21:19:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 0x7f43dc157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 0x7f43dc161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0d2000 con 0x7f43dc155640 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0d15f0 con 0x7f43dc1f56a0 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e2a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc0d0230 con 0x7f43c4005120 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57430/0 (socket says 172.21.15.40:57430) 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 -- 172.21.15.40:0/1474017980 learned_addr learned my addr 172.21.15.40:0/1474017980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 0x7f43dc157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 0x7f43dc161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 -- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 msgr2=0x7f43dc157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 0x7f43dc157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 -- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 msgr2=0x7f43dc161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 0x7f43dc161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43db7fe640 1 -- 172.21.15.40:0/1474017980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc1f5c50 con 0x7f43dc155640 2024-04-26T21:19:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 0x7f43dc157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.954+0000 7f43e0fa6640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 0x7f43dc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43db7fe640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43d000f7c0 tx=0x7f43d000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d000fe70 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d0018070 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc1f5f30 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d000d730 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.955+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc155120 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.956+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43dc1f6100 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.956+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43d000d8d0 con 0x7f43dc155640 2024-04-26T21:19:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.957+0000 7f43d8ff9640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 0x7f43c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.957+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f43d008ba80 con 0x7f43dc155640 2024-04-26T21:19:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.957+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 0x7f43c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.958+0000 7f43dbfff640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 0x7f43c806edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009a40 tx=0x7f43cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:29.961+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43d001a310 con 0x7f43dc155640 2024-04-26T21:19:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.145+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f43dc06a290 con 0x7f43dc155640 2024-04-26T21:19:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.145+0000 7f43d8ff9640 1 -- 172.21.15.40:0/1474017980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v39) ==== 106+0+4 (secure 0 0 0) 0x7f43d0059f40 con 0x7f43dc155640 2024-04-26T21:19:30.146 INFO:teuthology.orchestra.run.smithi040.stdout:100 2024-04-26T21:19:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 msgr2=0x7f43c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 0x7f43c806edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009a40 tx=0x7f43cc008040 comp rx=0 tx=0).stop 2024-04-26T21:19:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 msgr2=0x7f43dc1f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43d000f7c0 tx=0x7f43d000fc80 comp rx=0 tx=0).stop 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 shutdown_connections 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43c806c8f0 0x7f43c806edb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43dc1f56a0 0x7f43dc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43dc155640 0x7f43dc1f5160 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 --2- 172.21.15.40:0/1474017980 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4005120 0x7f43dc157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 >> 172.21.15.40:0/1474017980 conn(0x7f43dc09ba40 msgr2=0x7f43dc0716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 shutdown_connections 2024-04-26T21:19:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.147+0000 7f43e2a09640 1 -- 172.21.15.40:0/1474017980 wait complete. 2024-04-26T21:19:30.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-26T21:19:30.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.262+0000 7f6ac9625640 1 Processor -- start 2024-04-26T21:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.262+0000 7f6ac9625640 1 -- start start 2024-04-26T21:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac9625640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ac40d1940 0x7f6ac409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac9625640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac40cee60 con 0x7f6ac40b2420 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac9625640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac40d0ff0 con 0x7f6ac415d340 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac9625640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac40ce9d0 con 0x7f6ac40d1940 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ac40d1940 0x7f6ac409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac27fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ac40b2420 0x7f6ac40702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55422/0 (socket says 172.21.15.40:55422) 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.263+0000 7f6ac27fc640 1 -- 172.21.15.40:0/2833107175 learned_addr learned my addr 172.21.15.40:0/2833107175 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.264+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2833107175 >> v1:172.21.15.184:6789/0 conn(0x7f6ac415d340 legacy=0x7f6ac40d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.264+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2833107175 >> v1:172.21.15.40:6789/0 conn(0x7f6ac40b2420 legacy=0x7f6ac40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.264+0000 7f6ac37fe640 1 -- 172.21.15.40:0/2833107175 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40d0d10 con 0x7f6ac40d1940 2024-04-26T21:19:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.264+0000 7f6ac37fe640 1 --2- 172.21.15.40:0/2833107175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409c560 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009d90 tx=0x7f6ab402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3236e3ccd409c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac1ffb640 1 -- 172.21.15.40:0/2833107175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4007740 con 0x7f6ac40d1940 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac1ffb640 1 -- 172.21.15.40:0/2833107175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab40078e0 con 0x7f6ac40d1940 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac1ffb640 1 -- 172.21.15.40:0/2833107175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4007cb0 con 0x7f6ac40d1940 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 -- 172.21.15.40:0/2833107175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 msgr2=0x7f6ac409c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 --2- 172.21.15.40:0/2833107175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409c560 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009d90 tx=0x7f6ab402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 -- 172.21.15.40:0/2833107175 shutdown_connections 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 --2- 172.21.15.40:0/2833107175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409c560 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 -- 172.21.15.40:0/2833107175 >> 172.21.15.40:0/2833107175 conn(0x7f6ac409ba40 msgr2=0x7f6ac409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 -- 172.21.15.40:0/2833107175 shutdown_connections 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.265+0000 7f6ac9625640 1 -- 172.21.15.40:0/2833107175 wait complete. 2024-04-26T21:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.266+0000 7f6ac9625640 1 Processor -- start 2024-04-26T21:19:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 -- start start 2024-04-26T21:19:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac415d340 0x7f6ac409de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac41591a0 0x7f6ac4159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40b2090 con 0x7f6ac40d1940 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac415d720 con 0x7f6ac415d340 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac9625640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac40b2210 con 0x7f6ac41591a0 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57454/0 (socket says 172.21.15.40:57454) 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.267+0000 7f6ac2ffd640 1 -- 172.21.15.40:0/1266312432 learned_addr learned my addr 172.21.15.40:0/1266312432 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac27fc640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac415d340 0x7f6ac409de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 -- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac41591a0 msgr2=0x7f6ac4159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac37fe640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac41591a0 0x7f6ac4159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac41591a0 0x7f6ac4159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 -- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac415d340 msgr2=0x7f6ac409de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac415d340 0x7f6ac409de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 -- 172.21.15.40:0/1266312432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac409e7b0 con 0x7f6ac40d1940 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac2ffd640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00b3f0 tx=0x7f6abc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc015020 con 0x7f6ac40d1940 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc011830 con 0x7f6ac40d1940 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc011c20 con 0x7f6ac40d1940 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac409ea90 con 0x7f6ac40d1940 2024-04-26T21:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.268+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41f3940 con 0x7f6ac40d1940 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.269+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac41f3730 con 0x7f6ac40d1940 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.270+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6abc018020 con 0x7f6ac40d1940 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.270+0000 7f6abb7fe640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.270+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f6abc08b7f0 con 0x7f6ac40d1940 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.270+0000 7f6ac27fc640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.271+0000 7f6ac27fc640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40a0fd0 tx=0x7f6aac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.274+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6abc01f030 con 0x7f6ac40d1940 2024-04-26T21:19:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.459+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f6ac40693d0 con 0x7f6ac40d1940 2024-04-26T21:19:30.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.464+0000 7f6abb7fe640 1 -- 172.21.15.40:0/1266312432 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v40)=0 v40) ==== 124+0+0 (secure 0 0 0) 0x7f6abc0172c0 con 0x7f6ac40d1940 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 msgr2=0x7f6aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40a0fd0 tx=0x7f6aac009040 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 msgr2=0x7f6ac409fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00b3f0 tx=0x7f6abc00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 shutdown_connections 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6aa406c8f0 0x7f6aa406edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ac41591a0 0x7f6ac4159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ac415d340 0x7f6ac409de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 --2- 172.21.15.40:0/1266312432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ac40d1940 0x7f6ac409fe40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 >> 172.21.15.40:0/1266312432 conn(0x7f6ac409ba40 msgr2=0x7f6ac40c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 shutdown_connections 2024-04-26T21:19:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.467+0000 7f6ac9625640 1 -- 172.21.15.40:0/1266312432 wait complete. 2024-04-26T21:19:30.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-26T21:19:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.580+0000 7f0d767bd640 1 Processor -- start 2024-04-26T21:19:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.580+0000 7f0d767bd640 1 -- start start 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d767bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d767bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d70163aa0 0x7f0d700b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d767bd640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d700d3f50 con 0x7f0d700b60b0 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d767bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7015fa60 con 0x7f0d70163aa0 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d767bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700b5090 con 0x7f0d7015f680 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d6ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d6f7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d70163aa0 0x7f0d700b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d6ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:43432/0 (socket says 172.21.15.40:43432) 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.581+0000 7f0d6ffff640 1 -- 172.21.15.40:0/951105613 learned_addr learned my addr 172.21.15.40:0/951105613 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.582+0000 7f0d6ffff640 1 -- 172.21.15.40:0/951105613 >> v2:172.21.15.184:3300/0 conn(0x7f0d70163aa0 msgr2=0x7f0d700b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.582+0000 7f0d6ffff640 1 --2- 172.21.15.40:0/951105613 >> v2:172.21.15.184:3300/0 conn(0x7f0d70163aa0 0x7f0d700b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.582+0000 7f0d6ffff640 1 -- 172.21.15.40:0/951105613 >> v1:172.21.15.184:6789/0 conn(0x7f0d700b60b0 legacy=0x7f0d700d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.582+0000 7f0d6ffff640 1 -- 172.21.15.40:0/951105613 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d700b5370 con 0x7f0d7015f680 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d6ffff640 1 --2- 172.21.15.40:0/951105613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009870 tx=0x7f0d6002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e684341248e001ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d6effd640 1 -- 172.21.15.40:0/951105613 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60037070 con 0x7f0d7015f680 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d6effd640 1 -- 172.21.15.40:0/951105613 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d60032440 con 0x7f0d7015f680 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d6effd640 1 -- 172.21.15.40:0/951105613 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60032810 con 0x7f0d7015f680 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 -- 172.21.15.40:0/951105613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 msgr2=0x7f0d700d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 --2- 172.21.15.40:0/951105613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009870 tx=0x7f0d6002a250 comp rx=0 tx=0).stop 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 -- 172.21.15.40:0/951105613 shutdown_connections 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 --2- 172.21.15.40:0/951105613 >> v2:172.21.15.184:3300/0 conn(0x7f0d70163aa0 0x7f0d700b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 --2- 172.21.15.40:0/951105613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d7015f680 0x7f0d700d3820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 -- 172.21.15.40:0/951105613 >> 172.21.15.40:0/951105613 conn(0x7f0d7008eca0 msgr2=0x7f0d7009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 -- 172.21.15.40:0/951105613 shutdown_connections 2024-04-26T21:19:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.583+0000 7f0d767bd640 1 -- 172.21.15.40:0/951105613 wait complete. 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.584+0000 7f0d767bd640 1 Processor -- start 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 -- start start 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7015f680 0x7f0d70160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d70163aa0 0x7f0d70151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700d57b0 con 0x7f0d700b60b0 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700d5630 con 0x7f0d7015f680 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d767bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700d5930 con 0x7f0d70163aa0 2024-04-26T21:19:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d6ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d6ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55848/0 (socket says 172.21.15.40:55848) 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.585+0000 7f0d6ffff640 1 -- 172.21.15.40:0/3952896871 learned_addr learned my addr 172.21.15.40:0/3952896871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6f7fe640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7015f680 0x7f0d70160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d74d5a640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d70163aa0 0x7f0d70151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 -- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d70163aa0 msgr2=0x7f0d70151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d70163aa0 0x7f0d70151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 -- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7015f680 msgr2=0x7f0d70160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7015f680 0x7f0d70160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 -- 172.21.15.40:0/3952896871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d700c9f10 con 0x7f0d700b60b0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6ffff640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60037040 con 0x7f0d700b60b0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d60042070 con 0x7f0d700b60b0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d700ca1f0 con 0x7f0d700b60b0 2024-04-26T21:19:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60030880 con 0x7f0d700b60b0 2024-04-26T21:19:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.586+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701618e0 con 0x7f0d700b60b0 2024-04-26T21:19:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.587+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7006a290 con 0x7f0d700b60b0 2024-04-26T21:19:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.587+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d60030a20 con 0x7f0d700b60b0 2024-04-26T21:19:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.588+0000 7f0d6cff9640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.588+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f0d600ac390 con 0x7f0d700b60b0 2024-04-26T21:19:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.588+0000 7f0d6f7fe640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.589+0000 7f0d6f7fe640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d701617a0 tx=0x7f0d64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.592+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d6007a7c0 con 0x7f0d700b60b0 2024-04-26T21:19:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.776+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f0d700b4cd0 con 0x7f0d700b60b0 2024-04-26T21:19:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.777+0000 7f0d6cff9640 1 -- 172.21.15.40:0/3952896871 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v40) ==== 106+0+8 (secure 0 0 0) 0x7f0d6007a0c0 con 0x7f0d700b60b0 2024-04-26T21:19:30.778 INFO:teuthology.orchestra.run.smithi040.stdout:1048576 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.778+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 msgr2=0x7f0d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.778+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d701617a0 tx=0x7f0d64008040 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.778+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 msgr2=0x7f0d70150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.778+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d600046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 shutdown_connections 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d70163aa0 0x7f0d70151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d7015f680 0x7f0d70160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 --2- 172.21.15.40:0/3952896871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d700b60b0 0x7f0d70150650 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 >> 172.21.15.40:0/3952896871 conn(0x7f0d7008eca0 msgr2=0x7f0d70163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 shutdown_connections 2024-04-26T21:19:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.779+0000 7f0d767bd640 1 -- 172.21.15.40:0/3952896871 wait complete. 2024-04-26T21:19:30.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.893+0000 7f9b06197640 1 Processor -- start 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.893+0000 7f9b06197640 1 -- start start 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 0x7f9b000b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b0015f990 0x7f9b000d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b000b55b0 0x7f9b000d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9b000b6aa0 con 0x7f9b000b55b0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9b000d3eb0 con 0x7f9b0015f5b0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9b06197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b000d4190 con 0x7f9b0015f990 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b000b55b0 0x7f9b000d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9b000b55b0 0x7f9b000d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55860/0 (socket says 172.21.15.40:55860) 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 -- 172.21.15.40:0/2863639219 learned_addr learned my addr 172.21.15.40:0/2863639219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9aff7fe640 1 --2- 172.21.15.40:0/2863639219 >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 0x7f9b000b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afeffd640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b0015f990 0x7f9b000d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 -- 172.21.15.40:0/2863639219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b0015f990 msgr2=0x7f9b000d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b0015f990 0x7f9b000d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 -- 172.21.15.40:0/2863639219 >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 msgr2=0x7f9b000b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 --2- 172.21.15.40:0/2863639219 >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 0x7f9b000b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9afffff640 1 -- 172.21.15.40:0/2863639219 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b000d0200 con 0x7f9b000b55b0 2024-04-26T21:19:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.894+0000 7f9aff7fe640 1 --2- 172.21.15.40:0/2863639219 >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 0x7f9b000b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9afffff640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b000b55b0 0x7f9b000d0f40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9b0008f0f0 tx=0x7f9af802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03cad40fe362059 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9afe7fc640 1 -- 172.21.15.40:0/2863639219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8036070 con 0x7f9b000b55b0 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9afe7fc640 1 -- 172.21.15.40:0/2863639219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af8031600 con 0x7f9b000b55b0 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9afe7fc640 1 -- 172.21.15.40:0/2863639219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af80319d0 con 0x7f9b000b55b0 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 -- 172.21.15.40:0/2863639219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b000b55b0 msgr2=0x7f9b000d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b000b55b0 0x7f9b000d0f40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9b0008f0f0 tx=0x7f9af802a610 comp rx=0 tx=0).stop 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 -- 172.21.15.40:0/2863639219 shutdown_connections 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b000b55b0 0x7f9b000d0f40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 --2- 172.21.15.40:0/2863639219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b0015f990 0x7f9b000d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 --2- 172.21.15.40:0/2863639219 >> v2:172.21.15.184:3300/0 conn(0x7f9b0015f5b0 0x7f9b000b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.895+0000 7f9b06197640 1 -- 172.21.15.40:0/2863639219 >> 172.21.15.40:0/2863639219 conn(0x7f9b0008ec40 msgr2=0x7f9b0009e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.896+0000 7f9b06197640 1 -- 172.21.15.40:0/2863639219 shutdown_connections 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.896+0000 7f9b06197640 1 -- 172.21.15.40:0/2863639219 wait complete. 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.897+0000 7f9b06197640 1 Processor -- start 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.897+0000 7f9b06197640 1 -- start start 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.897+0000 7f9b06197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b000b55b0 0x7f9b001627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 0x7f9b00153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9b000d34c0 con 0x7f9b0015f990 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b000d3340 con 0x7f9b0015f5b0 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9b000d3640 con 0x7f9b000b55b0 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55864/0 (socket says 172.21.15.40:55864) 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 -- 172.21.15.40:0/931129726 learned_addr learned my addr 172.21.15.40:0/931129726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9aff7fe640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b000b55b0 0x7f9b001627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afeffd640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 0x7f9b00153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 -- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b000b55b0 msgr2=0x7f9b001627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b000b55b0 0x7f9b001627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 -- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 msgr2=0x7f9b00153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 0x7f9b00153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 -- 172.21.15.40:0/931129726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b000b8870 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afeffd640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 0x7f9b00153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9afffff640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034040 tx=0x7f9af8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8036050 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af8042070 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8009510 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b000b8b50 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.898+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001612a0 con 0x7f9b0015f990 2024-04-26T21:19:30.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.899+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0006a290 con 0x7f9b0015f990 2024-04-26T21:19:30.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.900+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9af8007bd0 con 0x7f9b0015f990 2024-04-26T21:19:30.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.900+0000 7f9ae7fff640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:30.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.901+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f9af80ac9f0 con 0x7f9b0015f990 2024-04-26T21:19:30.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.901+0000 7f9aff7fe640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.901+0000 7f9aff7fe640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 0x7f9aec06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9af0006fd0 tx=0x7f9af0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:30.905+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9af807ae20 con 0x7f9b0015f990 2024-04-26T21:19:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.092+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f9b000d60c0 con 0x7f9b0015f990 2024-04-26T21:19:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.092+0000 7f9ae7fff640 1 -- 172.21.15.40:0/931129726 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v40) ==== 112+0+10 (secure 0 0 0) 0x7f9af807a720 con 0x7f9b0015f990 2024-04-26T21:19:31.093 INFO:teuthology.orchestra.run.smithi040.stdout:60.000000 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 msgr2=0x7f9aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 0x7f9aec06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9af0006fd0 tx=0x7f9af0008040 comp rx=0 tx=0).stop 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 msgr2=0x7f9b000c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034040 tx=0x7f9af8004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 shutdown_connections 2024-04-26T21:19:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b0015f990 0x7f9b000c8d60 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b0015f5b0 0x7f9b00153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 --2- 172.21.15.40:0/931129726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b000b55b0 0x7f9b001627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 >> 172.21.15.40:0/931129726 conn(0x7f9b0008ec40 msgr2=0x7f9b000d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 shutdown_connections 2024-04-26T21:19:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.094+0000 7f9b06197640 1 -- 172.21.15.40:0/931129726 wait complete. 2024-04-26T21:19:31.106 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:31.106 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:31.122 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:19:31.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:19:31.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:31.220 INFO:teuthology.orchestra.run.smithi040.stdout:client6997 2024-04-26T21:19:31.220 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:31.220 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> import fcntl 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> import time 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:19:31.236 DEBUG:teuthology.orchestra.run.smithi040:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> n = 800 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> if not os.path.exists(abs_path): 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(abs_path) 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> handles = [] 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(0, n): 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> fname = "file_"+str(i) 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> path = os.path.join(abs_path, fname) 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> handles.append(open(path, '"'"'w'"'"')) 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:19:31.237 DEBUG:teuthology.orchestra.run.smithi040:> print("waiting with handles open", file=sys.stderr) 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> while True: 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> if os.read(0, 4096) == b"": 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> break 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> except BlockingIOError: 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> pass 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(2) 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-26T21:19:31.238 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:19:31.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.379+0000 7fe4edc93640 1 Processor -- start 2024-04-26T21:19:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.379+0000 7fe4edc93640 1 -- start start 2024-04-26T21:19:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f340 0x7fe4e80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe4e815f720 0x7fe4e80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4e80d4f00 0x7fe4e80d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e80b2510 con 0x7fe4e80d4f00 2024-04-26T21:19:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e80b27f0 con 0x7fe4e815f720 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4edc93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e80b2ad0 con 0x7fe4e815f340 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4e80d4f00 0x7fe4e80d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4e80d4f00 0x7fe4e80d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55872/0 (socket says 172.21.15.40:55872) 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 -- 172.21.15.40:0/2196068722 learned_addr learned my addr 172.21.15.40:0/2196068722 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e77fe640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f340 0x7fe4e80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e6ffd640 1 --2- 172.21.15.40:0/2196068722 >> v2:172.21.15.184:3300/0 conn(0x7fe4e815f720 0x7fe4e80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 -- 172.21.15.40:0/2196068722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f340 msgr2=0x7fe4e80b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f340 0x7fe4e80b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 -- 172.21.15.40:0/2196068722 >> v2:172.21.15.184:3300/0 conn(0x7fe4e815f720 msgr2=0x7fe4e80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 --2- 172.21.15.40:0/2196068722 >> v2:172.21.15.184:3300/0 conn(0x7fe4e815f720 0x7fe4e80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.380+0000 7fe4e7fff640 1 -- 172.21.15.40:0/2196068722 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e80b2db0 con 0x7fe4e80d4f00 2024-04-26T21:19:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.381+0000 7fe4e7fff640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e80d4f00 0x7fe4e80d5350 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe4e808f530 tx=0x7fe4e002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624ec9c2bc187a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.381+0000 7fe4e67fc640 1 -- 172.21.15.40:0/2196068722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0036070 con 0x7fe4e80d4f00 2024-04-26T21:19:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.381+0000 7fe4e67fc640 1 -- 172.21.15.40:0/2196068722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e0031600 con 0x7fe4e80d4f00 2024-04-26T21:19:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.381+0000 7fe4e67fc640 1 -- 172.21.15.40:0/2196068722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00319d0 con 0x7fe4e80d4f00 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 -- 172.21.15.40:0/2196068722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e80d4f00 msgr2=0x7fe4e80d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e80d4f00 0x7fe4e80d5350 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe4e808f530 tx=0x7fe4e002a610 comp rx=0 tx=0).stop 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 -- 172.21.15.40:0/2196068722 shutdown_connections 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e80d4f00 0x7fe4e80d5350 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 --2- 172.21.15.40:0/2196068722 >> v2:172.21.15.184:3300/0 conn(0x7fe4e815f720 0x7fe4e80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 --2- 172.21.15.40:0/2196068722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f340 0x7fe4e80b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 -- 172.21.15.40:0/2196068722 >> 172.21.15.40:0/2196068722 conn(0x7fe4e808ec70 msgr2=0x7fe4e809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 -- 172.21.15.40:0/2196068722 shutdown_connections 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.382+0000 7fe4edc93640 1 -- 172.21.15.40:0/2196068722 wait complete. 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.383+0000 7fe4edc93640 1 Processor -- start 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.383+0000 7fe4edc93640 1 -- start start 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e80d4f00 0x7fe4e8150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 0x7fe4e80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80b5040 con 0x7fe4e815f340 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80b4ec0 con 0x7fe4e80d4f00 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4edc93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e80b51c0 con 0x7fe4e815f720 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55884/0 (socket says 172.21.15.40:55884) 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 -- 172.21.15.40:0/4160934927 learned_addr learned my addr 172.21.15.40:0/4160934927 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e77fe640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e80d4f00 0x7fe4e8150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e7fff640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 0x7fe4e80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 -- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 msgr2=0x7fe4e80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 0x7fe4e80cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 -- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e80d4f00 msgr2=0x7fe4e8150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e80d4f00 0x7fe4e8150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e6ffd640 1 -- 172.21.15.40:0/4160934927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e8151610 con 0x7fe4e815f340 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.384+0000 7fe4e7fff640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 0x7fe4e80cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4e6ffd640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00e870 tx=0x7fe4dc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc017070 con 0x7fe4e815f340 2024-04-26T21:19:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4dc00c940 con 0x7fe4e815f340 2024-04-26T21:19:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e81518f0 con 0x7fe4e815f340 2024-04-26T21:19:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00cd30 con 0x7fe4e815f340 2024-04-26T21:19:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.385+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e814efd0 con 0x7fe4e815f340 2024-04-26T21:19:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.386+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e806a290 con 0x7fe4e815f340 2024-04-26T21:19:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.386+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4dc01a070 con 0x7fe4e815f340 2024-04-26T21:19:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.387+0000 7fe4cffff640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.387+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fe4dc08bc60 con 0x7fe4e815f340 2024-04-26T21:19:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.387+0000 7fe4e77fe640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.388+0000 7fe4e77fe640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 0x7fe4d406edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009ac0 tx=0x7fe4d8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.391+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4dc059ad0 con 0x7fe4e815f340 2024-04-26T21:19:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.617+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e80d3020 con 0x7fe4e815f340 2024-04-26T21:19:31.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.618+0000 7fe4cffff640 1 -- 172.21.15.40:0/4160934927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5186 (secure 0 0 0) 0x7fe4dc05ddd0 con 0x7fe4e815f340 2024-04-26T21:19:31.619 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:31.619 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:18.865416+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:31.619 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 64 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 msgr2=0x7fe4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 0x7fe4d406edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009ac0 tx=0x7fe4d8009190 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 msgr2=0x7fe4e81f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00e870 tx=0x7fe4dc00ed30 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 shutdown_connections 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4e815f720 0x7fe4e80cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4e815f340 0x7fe4e81f58f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 --2- 172.21.15.40:0/4160934927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e80d4f00 0x7fe4e8150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 >> 172.21.15.40:0/4160934927 conn(0x7fe4e808ec70 msgr2=0x7fe4e806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 shutdown_connections 2024-04-26T21:19:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:31.620+0000 7fe4edc93640 1 -- 172.21.15.40:0/4160934927 wait complete. 2024-04-26T21:19:31.632 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-26T21:19:31.751 INFO:teuthology.orchestra.run.smithi184.stdout:[{"id":6284,"entity":{"name":{"type":"client","num":6284},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1731993216}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.0489377809999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9911511499747654,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6284 v1:192.168.0.1:0/1731993216","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":6997,"entity":{"name":{"type":"client","num":6997},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1396555110}},"state":"open","num_leases":1,"num_caps":479,"request_load_avg":502,"uptime":4.1949143959999997,"requests_in_flight":477,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":71.784896210572313,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":63.878647334951935,"halflife":1.5},"recall_caps_throttle2o":{"value":50.280795553028817,"halflife":0.5},"session_cache_liveness":{"value":958.52701700970147,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6997 v1:172.21.15.40:0/1396555110","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:19:31.751 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1731993216, "type": "v1" }, "name": { "num": 6284, "type": "client" } }, "id": 6284, "inst": "client.6284 v1:192.168.0.1:0/1731993216", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9911511499747654 }, "state": "open", "uptime": 3.048937781 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1396555110, "type": "v1" }, "name": { "num": 6997, "type": "client" } }, "id": 6997, "inst": "client.6997 v1:172.21.15.40:0/1396555110", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 479, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 71.78489621057231 }, "recall_caps_throttle": { "halflife": 1.5, "value": 63.878647334951935 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 50.28079555302882 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 502, "requests_in_flight": 477, "session_cache_liveness": { "halflife": 300, "value": 958.5270170097015 }, "state": "open", "uptime": 4.194914396 } ] 2024-04-26T21:19:31.752 DEBUG:tasks.ceph_test_case:wait_until_equal: 479 != 802, waiting (timeout=600)... 2024-04-26T21:19:32.166 INFO:teuthology.orchestra.run.smithi040.stderr:waiting with handles open 2024-04-26T21:19:32.166 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:19:32.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:19:32.366 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:19:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:19:36.170 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:19:36.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.865+0000 7f16edb89640 1 Processor -- start 2024-04-26T21:19:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.865+0000 7f16edb89640 1 -- start start 2024-04-26T21:19:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16edb89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16e815f1e0 0x7f16e806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16edb89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16e806d2b0 0x7f16e806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16edb89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e80b4a10 con 0x7f16e80b44e0 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16edb89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f16e806dc60 con 0x7f16e815f1e0 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16edb89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f16e806eeb0 con 0x7f16e806d2b0 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16e77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16e815f1e0 0x7f16e806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16e77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16e815f1e0 0x7f16e806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55896/0 (socket says 172.21.15.40:55896) 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16e77fe640 1 -- 172.21.15.40:0/1026258742 learned_addr learned my addr 172.21.15.40:0/1026258742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.866+0000 7f16e7fff640 1 --2- 172.21.15.40:0/1026258742 >> v2:172.21.15.184:3300/0 conn(0x7f16e806d2b0 0x7f16e806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e77fe640 1 -- 172.21.15.40:0/1026258742 >> v2:172.21.15.184:3300/0 conn(0x7f16e806d2b0 msgr2=0x7f16e806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e77fe640 1 --2- 172.21.15.40:0/1026258742 >> v2:172.21.15.184:3300/0 conn(0x7f16e806d2b0 0x7f16e806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e77fe640 1 -- 172.21.15.40:0/1026258742 >> v1:172.21.15.184:6789/0 conn(0x7f16e80b44e0 legacy=0x7f16e80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e77fe640 1 -- 172.21.15.40:0/1026258742 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e80b4cf0 con 0x7f16e815f1e0 2024-04-26T21:19:36.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e77fe640 1 --2- 172.21.15.40:0/1026258742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e815f1e0 0x7f16e806e970 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f16e80b0ba0 tx=0x7f16d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf714c38f1fc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:36.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e67fc640 1 -- 172.21.15.40:0/1026258742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8037070 con 0x7f16e815f1e0 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e67fc640 1 -- 172.21.15.40:0/1026258742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d8032440 con 0x7f16e815f1e0 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.867+0000 7f16e67fc640 1 -- 172.21.15.40:0/1026258742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8032810 con 0x7f16e815f1e0 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 -- 172.21.15.40:0/1026258742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e815f1e0 msgr2=0x7f16e806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 --2- 172.21.15.40:0/1026258742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e815f1e0 0x7f16e806e970 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f16e80b0ba0 tx=0x7f16d802a270 comp rx=0 tx=0).stop 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 -- 172.21.15.40:0/1026258742 shutdown_connections 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 --2- 172.21.15.40:0/1026258742 >> v2:172.21.15.184:3300/0 conn(0x7f16e806d2b0 0x7f16e806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 --2- 172.21.15.40:0/1026258742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e815f1e0 0x7f16e806e970 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 -- 172.21.15.40:0/1026258742 >> 172.21.15.40:0/1026258742 conn(0x7f16e80728a0 msgr2=0x7f16e8072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 -- 172.21.15.40:0/1026258742 shutdown_connections 2024-04-26T21:19:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.868+0000 7f16edb89640 1 -- 172.21.15.40:0/1026258742 wait complete. 2024-04-26T21:19:36.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.869+0000 7f16edb89640 1 Processor -- start 2024-04-26T21:19:36.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 -- start start 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 0x7f16e8161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16e815f1e0 0x7f16e815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80b5590 con 0x7f16e806d2b0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16e80b52b0 con 0x7f16e80b44e0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16edb89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f16e815f5c0 con 0x7f16e815f1e0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55900/0 (socket says 172.21.15.40:55900) 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.870+0000 7f16e77fe640 1 -- 172.21.15.40:0/3540715550 learned_addr learned my addr 172.21.15.40:0/3540715550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e7fff640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16e815f1e0 0x7f16e815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e6ffd640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 0x7f16e8161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 -- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16e815f1e0 msgr2=0x7f16e815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16e815f1e0 0x7f16e815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 -- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 msgr2=0x7f16e8161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 0x7f16e8161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 -- 172.21.15.40:0/3540715550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e815d370 con 0x7f16e806d2b0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e6ffd640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 0x7f16e8161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16e77fe640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f16d802a2b0 tx=0x7f16d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8037040 con 0x7f16e806d2b0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d8042070 con 0x7f16e806d2b0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81601c0 con 0x7f16e806d2b0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8030880 con 0x7f16e806d2b0 2024-04-26T21:19:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.871+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e8160780 con 0x7f16e806d2b0 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.872+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e806a290 con 0x7f16e806d2b0 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.873+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16d8030a20 con 0x7f16e806d2b0 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.873+0000 7f16cffff640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.873+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f16d80ac260 con 0x7f16e806d2b0 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.874+0000 7f16e6ffd640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.874+0000 7f16e6ffd640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f16e806dad0 tx=0x7f16dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:36.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:36.878+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16d807a720 con 0x7f16e806d2b0 2024-04-26T21:19:37.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.109+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16e806e930 con 0x7f16e806d2b0 2024-04-26T21:19:37.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.110+0000 7f16cffff640 1 -- 172.21.15.40:0/3540715550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5186 (secure 0 0 0) 0x7f16d807a020 con 0x7f16e806d2b0 2024-04-26T21:19:37.111 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:37.111 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:36.505939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:37.111 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 msgr2=0x7f16d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 0x7f16d406edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f16e806dad0 tx=0x7f16dc009040 comp rx=0 tx=0).stop 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 msgr2=0x7f16e8150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f16d802a2b0 tx=0x7f16d80046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 shutdown_connections 2024-04-26T21:19:37.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d406c8f0 0x7f16d406edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16e815f1e0 0x7f16e815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16e80b44e0 0x7f16e8161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 --2- 172.21.15.40:0/3540715550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16e806d2b0 0x7f16e8150d40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 >> 172.21.15.40:0/3540715550 conn(0x7f16e80728a0 msgr2=0x7f16e806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 shutdown_connections 2024-04-26T21:19:37.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.112+0000 7f16edb89640 1 -- 172.21.15.40:0/3540715550 wait complete. 2024-04-26T21:19:37.124 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-26T21:19:37.252 INFO:teuthology.orchestra.run.smithi184.stdout:[{"id":6284,"entity":{"name":{"type":"client","num":6284},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1731993216}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":8.5498255249999993,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9533749399423468,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6284 v1:192.168.0.1:0/1731993216","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":6997,"entity":{"name":{"type":"client","num":6997},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1396555110}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":775,"uptime":9.6948021610000001,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2004.9070174465451,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":691.18563714814411,"halflife":1.5},"recall_caps_throttle2o":{"value":186.56906052065466,"halflife":0.5},"session_cache_liveness":{"value":1584.6481715501245,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6997 v1:172.21.15.40:0/1396555110","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:19:37.254 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1731993216, "type": "v1" }, "name": { "num": 6284, "type": "client" } }, "id": 6284, "inst": "client.6284 v1:192.168.0.1:0/1731993216", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.953374939942347 }, "state": "open", "uptime": 8.549825525 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1396555110, "type": "v1" }, "name": { "num": 6997, "type": "client" } }, "id": 6997, "inst": "client.6997 v1:172.21.15.40:0/1396555110", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2004.907017446545 }, "recall_caps_throttle": { "halflife": 1.5, "value": 691.1856371481441 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 186.56906052065466 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 775, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1584.6481715501245 }, "state": "open", "uptime": 9.694802161 } ] 2024-04-26T21:19:37.254 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-26T21:19:37.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:19:37.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.353+0000 7f628d4d3640 1 Processor -- start 2024-04-26T21:19:37.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.353+0000 7f628d4d3640 1 -- start start 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f628d4d3640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6288160610 0x7f62880b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f628d4d3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f628815fb50 con 0x7f62880b41f0 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f628d4d3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62880b5da0 con 0x7f62881598a0 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f628d4d3640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f62880b6080 con 0x7f6288160610 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f62877fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f62880b41f0 0x7f6288162f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41418/0 (socket says 172.21.15.40:41418) 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f62877fe640 1 -- 172.21.15.40:0/865195045 learned_addr learned my addr 172.21.15.40:0/865195045 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.354+0000 7f62867fc640 1 --2- 172.21.15.40:0/865195045 >> v2:172.21.15.184:3300/0 conn(0x7f6288160610 0x7f62880b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757228267 0 0) 0x7f628815fb50 con 0x7f62880b41f0 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6274003520 con 0x7f62880b41f0 2024-04-26T21:19:37.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2390827370 0 0) 0x7f6274003520 con 0x7f62880b41f0 2024-04-26T21:19:37.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f62867fc640 1 -- 172.21.15.40:0/865195045 >> v1:172.21.15.184:6789/0 conn(0x7f62881598a0 legacy=0x7f62880b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f62867fc640 1 -- 172.21.15.40:0/865195045 >> v1:172.21.15.40:6789/0 conn(0x7f62880b41f0 legacy=0x7f6288162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.355+0000 7f62867fc640 1 -- 172.21.15.40:0/865195045 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628815fe10 con 0x7f6288160610 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f62867fc640 1 --2- 172.21.15.40:0/865195045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288160610 0x7f62880b3cb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f62700097a0 tx=0x7f627002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec9a2eb84a7d6e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270037070 con 0x7f6288160610 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627002aad0 con 0x7f6288160610 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f6285ffb640 1 -- 172.21.15.40:0/865195045 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62780019f0 con 0x7f6288160610 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 -- 172.21.15.40:0/865195045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288160610 msgr2=0x7f62880b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 --2- 172.21.15.40:0/865195045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288160610 0x7f62880b3cb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f62700097a0 tx=0x7f627002a040 comp rx=0 tx=0).stop 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 -- 172.21.15.40:0/865195045 shutdown_connections 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 --2- 172.21.15.40:0/865195045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288160610 0x7f62880b3cb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 -- 172.21.15.40:0/865195045 >> 172.21.15.40:0/865195045 conn(0x7f628806d3b0 msgr2=0x7f628806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:37.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 -- 172.21.15.40:0/865195045 shutdown_connections 2024-04-26T21:19:37.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.356+0000 7f628d4d3640 1 -- 172.21.15.40:0/865195045 wait complete. 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 Processor -- start 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 -- start start 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 0x7f6288148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6288160610 0x7f62880af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6288159c80 con 0x7f62880b41f0 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62880b5a60 con 0x7f62881598a0 2024-04-26T21:19:37.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f628d4d3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f628815cf00 con 0x7f6288160610 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f6286ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f6286ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55906/0 (socket says 172.21.15.40:55906) 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.358+0000 7f6286ffd640 1 -- 172.21.15.40:0/344234683 learned_addr learned my addr 172.21.15.40:0/344234683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f62867fc640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 0x7f6288148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 -- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6288160610 msgr2=0x7f62880af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f62877fe640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6288160610 0x7f62880af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6288160610 0x7f62880af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 -- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 msgr2=0x7f6288148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 0x7f6288148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 -- 172.21.15.40:0/344234683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628815b920 con 0x7f62880b41f0 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f62867fc640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 0x7f6288148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:37.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f6286ffd640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f628000c570 tx=0x7f628000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280005480 con 0x7f62880b41f0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62800035f0 con 0x7f62880b41f0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628815bc00 con 0x7f62880b41f0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800039c0 con 0x7f62880b41f0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.359+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628815c1c0 con 0x7f62880b41f0 2024-04-26T21:19:37.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.360+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62880b8340 con 0x7f62880b41f0 2024-04-26T21:19:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.361+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6280003b60 con 0x7f62880b41f0 2024-04-26T21:19:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.361+0000 7f627f7fe640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 0x7f626806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.361+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f628008b950 con 0x7f62880b41f0 2024-04-26T21:19:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.361+0000 7f62867fc640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 0x7f626806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.362+0000 7f62867fc640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 0x7f626806ef50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6270034000 tx=0x7f627002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.365+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62880b8340 con 0x7f62880b41f0 2024-04-26T21:19:37.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.590+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f62880b6240 con 0x7f62880b41f0 2024-04-26T21:19:37.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.590+0000 7f627f7fe640 1 -- 172.21.15.40:0/344234683 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f62880b6240 con 0x7f62880b41f0 2024-04-26T21:19:37.591 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:37.591 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 msgr2=0x7f626806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 0x7f626806ef50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6270034000 tx=0x7f627002e000 comp rx=0 tx=0).stop 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 msgr2=0x7f628815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f628000c570 tx=0x7f628000ca30 comp rx=0 tx=0).stop 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 shutdown_connections 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f626806ca90 0x7f626806ef50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6288160610 0x7f62880af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62881598a0 0x7f6288148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 --2- 172.21.15.40:0/344234683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62880b41f0 0x7f628815c9c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 >> 172.21.15.40:0/344234683 conn(0x7f628806d3b0 msgr2=0x7f62880b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:37.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 shutdown_connections 2024-04-26T21:19:37.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.593+0000 7f628d4d3640 1 -- 172.21.15.40:0/344234683 wait complete. 2024-04-26T21:19:37.604 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:19:37.604 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-26T21:19:37.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:19:37.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.703+0000 7f5c83ba4640 1 Processor -- start 2024-04-26T21:19:37.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.703+0000 7f5c83ba4640 1 -- start start 2024-04-26T21:19:37.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c83ba4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c83ba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c0b5530 0x7f5c7c0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c83ba4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c15fce0 con 0x7f5c7c070570 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c83ba4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7c0b59c0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c83ba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c160610 con 0x7f5c7c0b5530 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c81940640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c81940640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55908/0 (socket says 172.21.15.40:55908) 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.704+0000 7f5c81940640 1 -- 172.21.15.40:0/1230538193 learned_addr learned my addr 172.21.15.40:0/1230538193 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c82141640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c0b5530 0x7f5c7c0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c81940640 1 -- 172.21.15.40:0/1230538193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c0b5530 msgr2=0x7f5c7c0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c81940640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c0b5530 0x7f5c7c0d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c81940640 1 -- 172.21.15.40:0/1230538193 >> v1:172.21.15.184:6789/0 conn(0x7f5c7c070570 legacy=0x7f5c7c09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c81940640 1 -- 172.21.15.40:0/1230538193 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c15ffc0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c81940640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666396670ec1e00b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.705+0000 7f5c8093e640 1 -- 172.21.15.40:0/1230538193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c036070 con 0x7f5c7c09ba40 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c8093e640 1 -- 172.21.15.40:0/1230538193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c031600 con 0x7f5c7c09ba40 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c8093e640 1 -- 172.21.15.40:0/1230538193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0319d0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 -- 172.21.15.40:0/1230538193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 msgr2=0x7f5c7c15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a710 comp rx=0 tx=0).stop 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 -- 172.21.15.40:0/1230538193 shutdown_connections 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c0b5530 0x7f5c7c0d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/1230538193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c15f6a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 -- 172.21.15.40:0/1230538193 >> 172.21.15.40:0/1230538193 conn(0x7f5c7c0b4a10 msgr2=0x7f5c7c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 -- 172.21.15.40:0/1230538193 shutdown_connections 2024-04-26T21:19:37.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.706+0000 7f5c83ba4640 1 -- 172.21.15.40:0/1230538193 wait complete. 2024-04-26T21:19:37.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.707+0000 7f5c83ba4640 1 Processor -- start 2024-04-26T21:19:37.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.707+0000 7f5c83ba4640 1 -- start start 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 0x7f5c7c0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c7c0b5530 0x7f5c7c161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0d1580 con 0x7f5c7c09ba40 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0709b0 con 0x7f5c7c0b5530 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c83ba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c0d1700 con 0x7f5c7c070570 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c8113f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c8113f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55910/0 (socket says 172.21.15.40:55910) 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c8113f640 1 -- 172.21.15.40:0/3281876897 learned_addr learned my addr 172.21.15.40:0/3281876897 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c82141640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c7c0b5530 0x7f5c7c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.708+0000 7f5c81940640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 0x7f5c7c0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 -- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 msgr2=0x7f5c7c0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 0x7f5c7c0c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 -- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c7c0b5530 msgr2=0x7f5c7c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c7c0b5530 0x7f5c7c161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 -- 172.21.15.40:0/3281876897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c163a80 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c81940640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 0x7f5c7c0c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c8113f640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c400 tx=0x7f5c7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700052d0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c70005490 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1629e0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007260 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.709+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c162fa0 con 0x7f5c7c09ba40 2024-04-26T21:19:37.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.710+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c06a290 con 0x7f5c7c09ba40 2024-04-26T21:19:37.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.710+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c70007400 con 0x7f5c7c09ba40 2024-04-26T21:19:37.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.711+0000 7f5c7a7fc640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:37.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.711+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f5c7008b030 con 0x7f5c7c09ba40 2024-04-26T21:19:37.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.711+0000 7f5c81940640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:37.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.712+0000 7f5c81940640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 0x7f5c6806edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c033000 tx=0x7f5c6c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:37.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.715+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c70058e10 con 0x7f5c7c09ba40 2024-04-26T21:19:37.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.940+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5c7c0b1890 con 0x7f5c7c09ba40 2024-04-26T21:19:37.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.941+0000 7f5c7a7fc640 1 -- 172.21.15.40:0/3281876897 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f5c7005d110 con 0x7f5c7c09ba40 2024-04-26T21:19:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:37.942 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 msgr2=0x7f5c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 0x7f5c6806edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c033000 tx=0x7f5c6c009000 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 msgr2=0x7f5c7c1f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c400 tx=0x7f5c7000c8c0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 shutdown_connections 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c7c0b5530 0x7f5c7c161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c7c09ba40 0x7f5c7c1f5830 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 --2- 172.21.15.40:0/3281876897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c7c070570 0x7f5c7c0c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 >> 172.21.15.40:0/3281876897 conn(0x7f5c7c0b4a10 msgr2=0x7f5c7c06dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 shutdown_connections 2024-04-26T21:19:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37.942+0000 7f5c83ba4640 1 -- 172.21.15.40:0/3281876897 wait complete. 2024-04-26T21:19:37.954 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:19:37.954 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-26T21:19:37.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-26T21:19:38.172 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:19:38.173 INFO:teuthology.orchestra.run.smithi040.stderr:stdin closed, goodbye! 2024-04-26T21:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:19:38.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:38.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.334+0000 7fa6f2408640 1 Processor -- start 2024-04-26T21:19:38.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.334+0000 7fa6f2408640 1 -- start start 2024-04-26T21:19:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 0x7fa6ec15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec06e750 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec0d1740 con 0x7fa6ec06e2c0 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f2408640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0d1a20 con 0x7fa6ec0cf2e0 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55924/0 (socket says 172.21.15.40:55924) 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 -- 172.21.15.40:0/2726607042 learned_addr learned my addr 172.21.15.40:0/2726607042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:38.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f09a5640 1 --2- 172.21.15.40:0/2726607042 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 0x7fa6ec15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6ebfff640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 -- 172.21.15.40:0/2726607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 msgr2=0x7fa6ec06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 -- 172.21.15.40:0/2726607042 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 msgr2=0x7fa6ec15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 --2- 172.21.15.40:0/2726607042 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 0x7fa6ec15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6eb7fe640 1 -- 172.21.15.40:0/2726607042 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec0d1d00 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.335+0000 7fa6f09a5640 1 --2- 172.21.15.40:0/2726607042 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 0x7fa6ec15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.336+0000 7fa6eb7fe640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec163ff0 tx=0x7fa6dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ebe02a0be59562 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.336+0000 7fa6eaffd640 1 -- 172.21.15.40:0/2726607042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc037070 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.336+0000 7fa6eaffd640 1 -- 172.21.15.40:0/2726607042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc032440 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.336+0000 7fa6eaffd640 1 -- 172.21.15.40:0/2726607042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc032810 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 -- 172.21.15.40:0/2726607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 msgr2=0x7fa6ec06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec163ff0 tx=0x7fa6dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 -- 172.21.15.40:0/2726607042 shutdown_connections 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 --2- 172.21.15.40:0/2726607042 >> v2:172.21.15.184:3300/0 conn(0x7fa6ec06e2c0 0x7fa6ec15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec06dd80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 --2- 172.21.15.40:0/2726607042 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 -- 172.21.15.40:0/2726607042 >> 172.21.15.40:0/2726607042 conn(0x7fa6ec0b4a60 msgr2=0x7fa6ec0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 -- 172.21.15.40:0/2726607042 shutdown_connections 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.337+0000 7fa6f2408640 1 -- 172.21.15.40:0/2726607042 wait complete. 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.338+0000 7fa6f2408640 1 Processor -- start 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.338+0000 7fa6f2408640 1 -- start start 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 0x7fa6ec1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec09ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec06da10 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec0d22c0 con 0x7fa6ec06e2c0 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f2408640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0cf6c0 con 0x7fa6ec0cf2e0 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55930/0 (socket says 172.21.15.40:55930) 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 -- 172.21.15.40:0/27420274 learned_addr learned my addr 172.21.15.40:0/27420274 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6ebfff640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 0x7fa6ec1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6eb7fe640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec09ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 -- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 msgr2=0x7fa6ec09ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec09ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 -- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 msgr2=0x7fa6ec1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 0x7fa6ec1602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 -- 172.21.15.40:0/27420274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec09f650 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6ebfff640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 0x7fa6ec1602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6eb7fe640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec09ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.339+0000 7fa6f09a5640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007f20 tx=0x7fa6e400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.340+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e401a070 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.340+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e4003b90 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.340+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec09f930 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.340+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004030 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.340+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec09fef0 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.341+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec06a290 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.341+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e401d070 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.342+0000 7fa6e8ff9640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.342+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e408bf80 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.342+0000 7fa6ebfff640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.343+0000 7fa6ebfff640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 0x7fa6d806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0006fd0 tx=0x7fa6e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.346+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6e4010070 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.572+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6ec1625d0 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.573+0000 7fa6e8ff9640 1 -- 172.21.15.40:0/27420274 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5186 (secure 0 0 0) 0x7fa6e405a290 con 0x7fa6ec0d11b0 2024-04-26T21:19:38.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:38.574 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:36.505939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:38.575 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 msgr2=0x7fa6d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 0x7fa6d806edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0006fd0 tx=0x7fa6e0008040 comp rx=0 tx=0).stop 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 msgr2=0x7fa6ec156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007f20 tx=0x7fa6e400d560 comp rx=0 tx=0).stop 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 shutdown_connections 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0d11b0 0x7fa6ec156310 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6ec0cf2e0 0x7fa6ec09ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 --2- 172.21.15.40:0/27420274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6ec06e2c0 0x7fa6ec1602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 >> 172.21.15.40:0/27420274 conn(0x7fa6ec0b4a60 msgr2=0x7fa6ec1611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 shutdown_connections 2024-04-26T21:19:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.575+0000 7fa6f2408640 1 -- 172.21.15.40:0/27420274 wait complete. 2024-04-26T21:19:38.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-26T21:19:38.705 INFO:teuthology.orchestra.run.smithi184.stdout:[{"id":6284,"entity":{"name":{"type":"client","num":6284},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1731993216}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":10.002795875,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9434768643033604,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6284 v1:192.168.0.1:0/1731993216","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":6997,"entity":{"name":{"type":"client","num":6997},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":1396555110}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":762,"uptime":11.147772510999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2365.5890328989703,"halflife":60},"release_caps":{"value":399.43332100992887,"halflife":60},"recall_caps_throttle":{"value":945.15051942133721,"halflife":1.5},"recall_caps_throttle2o":{"value":399.12733513002428,"halflife":0.5},"session_cache_liveness":{"value":1579.3373093053565,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6997 v1:172.21.15.40:0/1396555110","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:19:38.707 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1731993216, "type": "v1" }, "name": { "num": 6284, "type": "client" } }, "id": 6284, "inst": "client.6284 v1:192.168.0.1:0/1731993216", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9434768643033604 }, "state": "open", "uptime": 10.002795875 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 1396555110, "type": "v1" }, "name": { "num": 6997, "type": "client" } }, "id": 6997, "inst": "client.6997 v1:172.21.15.40:0/1396555110", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2365.5890328989703 }, "recall_caps_throttle": { "halflife": 1.5, "value": 945.1505194213372 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 399.1273351300243 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 399.4333210099289 }, "request_load_avg": 762, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1579.3373093053565 }, "state": "open", "uptime": 11.147772511 } ] 2024-04-26T21:19:38.707 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:19:38.708 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:19:38.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:38.743 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:38.780 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:19:38.780 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:38.780 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 800\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi040') 2024-04-26T21:19:38.781 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:38.781 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:38.799 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:38.799 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:38.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:38.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.995+0000 7f9422620640 1 Processor -- start 2024-04-26T21:19:38.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.995+0000 7f9422620640 1 -- start start 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f9422620640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f9422620640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f941c0d3ee0 0x7f941c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f9422620640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f941c0d03a0 con 0x7f941c15f750 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f9422620640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f941c0b4d10 con 0x7f941c0d3ee0 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f9422620640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f941c0d0680 con 0x7f941c15f370 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f941b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f941c15f750 0x7f941c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41424/0 (socket says 172.21.15.40:41424) 2024-04-26T21:19:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.996+0000 7f941b7fe640 1 -- 172.21.15.40:0/3098506495 learned_addr learned my addr 172.21.15.40:0/3098506495 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f9420bbd640 1 --2- 172.21.15.40:0/3098506495 >> v2:172.21.15.184:3300/0 conn(0x7f941c0d3ee0 0x7f941c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f941bfff640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f941affd640 1 -- 172.21.15.40:0/3098506495 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004090849 0 0) 0x7f941c0d03a0 con 0x7f941c15f750 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f941affd640 1 -- 172.21.15.40:0/3098506495 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9404003520 con 0x7f941c15f750 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f9420bbd640 1 -- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 msgr2=0x7f941c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f9420bbd640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f9420bbd640 1 -- 172.21.15.40:0/3098506495 >> v1:172.21.15.40:6789/0 conn(0x7f941c15f750 legacy=0x7f941c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.997+0000 7f9420bbd640 1 -- 172.21.15.40:0/3098506495 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c0d0960 con 0x7f941c0d3ee0 2024-04-26T21:19:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9420bbd640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c0d3ee0 0x7f941c0b47d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9414009d90 tx=0x7f941402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f14949183fc16ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f941affd640 1 -- 172.21.15.40:0/3098506495 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414031600 con 0x7f941c0d3ee0 2024-04-26T21:19:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f941affd640 1 -- 172.21.15.40:0/3098506495 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94140317a0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f941affd640 1 -- 172.21.15.40:0/3098506495 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414031b50 con 0x7f941c0d3ee0 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 -- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c0d3ee0 msgr2=0x7f941c0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c0d3ee0 0x7f941c0b47d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9414009d90 tx=0x7f941402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 -- 172.21.15.40:0/3098506495 shutdown_connections 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c0d3ee0 0x7f941c0b47d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 --2- 172.21.15.40:0/3098506495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 -- 172.21.15.40:0/3098506495 >> 172.21.15.40:0/3098506495 conn(0x7f941c08ec10 msgr2=0x7f941c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 -- 172.21.15.40:0/3098506495 shutdown_connections 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:38.998+0000 7f9422620640 1 -- 172.21.15.40:0/3098506495 wait complete. 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 Processor -- start 2024-04-26T21:19:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 -- start start 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 0x7f941c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f941c0b51c0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f941c0d0f20 con 0x7f941c15f750 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.000+0000 7f9422620640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f941c0b5340 con 0x7f941c15f370 2024-04-26T21:19:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55942/0 (socket says 172.21.15.40:55942) 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 -- 172.21.15.40:0/1979603886 learned_addr learned my addr 172.21.15.40:0/1979603886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9420bbd640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 0x7f941c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941b7fe640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 -- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 msgr2=0x7f941c06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c06db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 -- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 msgr2=0x7f941c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 0x7f941c15cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 -- 172.21.15.40:0/1979603886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c09f1f0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9420bbd640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 0x7f941c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941b7fe640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c06db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f941bfff640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9b0 tx=0x7f940c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c017070 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c00c9a0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00cd90 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c09f4d0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.001+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c09fa90 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.002+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c06a290 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.003+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f940c01a070 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.003+0000 7f9418ff9640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.003+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f940c08be90 con 0x7f941c0d3ee0 2024-04-26T21:19:39.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.003+0000 7f941b7fe640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.004+0000 7f941b7fe640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 0x7f940806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f941c0baac0 tx=0x7f9410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.007+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f940c05a350 con 0x7f941c0d3ee0 2024-04-26T21:19:39.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.196+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f941c0695e0 con 0x7f941c0d3ee0 2024-04-26T21:19:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.197+0000 7f9418ff9640 1 -- 172.21.15.40:0/1979603886 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16627 (secure 0 0 0) 0x7f940c059c50 con 0x7f941c0d3ee0 2024-04-26T21:19:39.199 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:39.199 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":109,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:17.479995+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 msgr2=0x7f940806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 0x7f940806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f941c0baac0 tx=0x7f9410009040 comp rx=0 tx=0).stop 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 msgr2=0x7f941c1f4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9b0 tx=0x7f940c00ee70 comp rx=0 tx=0).stop 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 shutdown_connections 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f941c15f750 0x7f941c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f941c15f370 0x7f941c06db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 --2- 172.21.15.40:0/1979603886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f941c0d3ee0 0x7f941c1f4df0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 >> 172.21.15.40:0/1979603886 conn(0x7f941c08ec10 msgr2=0x7f941c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 shutdown_connections 2024-04-26T21:19:39.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.200+0000 7f9422620640 1 -- 172.21.15.40:0/1979603886 wait complete. 2024-04-26T21:19:39.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:14.110950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:14.791024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:39.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.315+0000 7fae06e8c640 1 Processor -- start 2024-04-26T21:19:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.315+0000 7fae06e8c640 1 -- start start 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae06e8c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae000d3820 0x7fae000b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae06e8c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000d3c90 con 0x7fae0015f680 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae06e8c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000d3f50 con 0x7fae000b60b0 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae06e8c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fae0015fa60 con 0x7fae000d3820 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae04c28640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae0015f680 0x7fae00163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41428/0 (socket says 172.21.15.40:41428) 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fae04c28640 1 -- 172.21.15.40:0/3799443365 learned_addr learned my addr 172.21.15.40:0/3799443365 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fadfffff640 1 --2- 172.21.15.40:0/3799443365 >> v2:172.21.15.184:3300/0 conn(0x7fae000d3820 0x7fae000b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.316+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758862889 0 0) 0x7fae000d3c90 con 0x7fae0015f680 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.317+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade8003520 con 0x7fae0015f680 2024-04-26T21:19:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.317+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1070316173 0 0) 0x7fade8003520 con 0x7fae0015f680 2024-04-26T21:19:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.317+0000 7fadfffff640 1 -- 172.21.15.40:0/3799443365 >> v1:172.21.15.184:6789/0 conn(0x7fae000b60b0 legacy=0x7fae000d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.317+0000 7fadfffff640 1 -- 172.21.15.40:0/3799443365 >> v1:172.21.15.40:6789/0 conn(0x7fae0015f680 legacy=0x7fae00163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.317+0000 7fadfffff640 1 -- 172.21.15.40:0/3799443365 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000b5090 con 0x7fae000d3820 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fadfffff640 1 --2- 172.21.15.40:0/3799443365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000d3820 0x7fae000b5b70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7977bd28e26a6db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4037070 con 0x7fae000d3820 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf402ac50 con 0x7fae000d3820 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fadff7fe640 1 -- 172.21.15.40:0/3799443365 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80019f0 con 0x7fae000d3820 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 -- 172.21.15.40:0/3799443365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000d3820 msgr2=0x7fae000b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 --2- 172.21.15.40:0/3799443365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000d3820 0x7fae000b5b70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 -- 172.21.15.40:0/3799443365 shutdown_connections 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 --2- 172.21.15.40:0/3799443365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000d3820 0x7fae000b5b70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 -- 172.21.15.40:0/3799443365 >> 172.21.15.40:0/3799443365 conn(0x7fae0008eca0 msgr2=0x7fae0009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 -- 172.21.15.40:0/3799443365 shutdown_connections 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.318+0000 7fae06e8c640 1 -- 172.21.15.40:0/3799443365 wait complete. 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.319+0000 7fae06e8c640 1 Processor -- start 2024-04-26T21:19:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.319+0000 7fae06e8c640 1 -- start start 2024-04-26T21:19:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000b60b0 0x7fae001f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae000d3820 0x7fae00162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d57b0 con 0x7fae0015f680 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d5630 con 0x7fae000b60b0 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae06e8c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fae000d5930 con 0x7fae000d3820 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae05429640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae05429640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55958/0 (socket says 172.21.15.40:55958) 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.320+0000 7fae05429640 1 -- 172.21.15.40:0/2725561688 learned_addr learned my addr 172.21.15.40:0/2725561688 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fadfffff640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae000d3820 0x7fae00162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae04c28640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000b60b0 0x7fae001f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 -- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae000d3820 msgr2=0x7fae00162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae000d3820 0x7fae00162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 -- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000b60b0 msgr2=0x7fae001f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000b60b0 0x7fae001f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 -- 172.21.15.40:0/2725561688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae001f4da0 con 0x7fae0015f680 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae05429640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c3a0 tx=0x7fadf000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0017070 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf00052d0 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae000ce1f0 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003830 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.321+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae000ce7b0 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.322+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0006a290 con 0x7fae0015f680 2024-04-26T21:19:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.323+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadf001a070 con 0x7fae0015f680 2024-04-26T21:19:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.323+0000 7fadfd7fa640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.323+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fadf008ad60 con 0x7fae0015f680 2024-04-26T21:19:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.323+0000 7fae04c28640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.324+0000 7fae04c28640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005c00 tx=0x7fadf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.328+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadf0058c90 con 0x7fae0015f680 2024-04-26T21:19:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.512+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae000757c0 con 0x7fae0015f680 2024-04-26T21:19:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.513+0000 7fadfd7fa640 1 -- 172.21.15.40:0/2725561688 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fadf005cf90 con 0x7fae0015f680 2024-04-26T21:19:39.514 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.514+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 msgr2=0x7fadec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.514+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005c00 tx=0x7fadf8009040 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.514+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 msgr2=0x7fae000d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.514+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c3a0 tx=0x7fadf000c860 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 shutdown_connections 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0015f680 0x7fae000d1f10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae000d3820 0x7fae00162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 --2- 172.21.15.40:0/2725561688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae000b60b0 0x7fae001f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 >> 172.21.15.40:0/2725561688 conn(0x7fae0008eca0 msgr2=0x7fae000b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 shutdown_connections 2024-04-26T21:19:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.515+0000 7fae06e8c640 1 -- 172.21.15.40:0/2725561688 wait complete. 2024-04-26T21:19:39.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:19:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.629+0000 7f082d62a640 1 Processor -- start 2024-04-26T21:19:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.629+0000 7f082d62a640 1 -- start start 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f082d62a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f08280b2420 0x7f082809c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f082d62a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280cee60 con 0x7f08280d1940 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f082d62a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280d0ff0 con 0x7f082815d340 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f082d62a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f08280ce9d0 con 0x7f08280b2420 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f08277fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f08280d1940 0x7f08280702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41442/0 (socket says 172.21.15.40:41442) 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f08277fe640 1 -- 172.21.15.40:0/2298434675 learned_addr learned my addr 172.21.15.40:0/2298434675 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f08267fc640 1 --2- 172.21.15.40:0/2298434675 >> v2:172.21.15.184:3300/0 conn(0x7f08280b2420 0x7f082809c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.630+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581561469 0 0) 0x7f08280cee60 con 0x7f08280d1940 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0814003520 con 0x7f08280d1940 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3651068334 0 0) 0x7f08280d0ff0 con 0x7f082815d340 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08280cee60 con 0x7f082815d340 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720052221 0 0) 0x7f0814003520 con 0x7f08280d1940 2024-04-26T21:19:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f08267fc640 1 -- 172.21.15.40:0/2298434675 >> v1:172.21.15.184:6789/0 conn(0x7f082815d340 legacy=0x7f08280d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f08267fc640 1 -- 172.21.15.40:0/2298434675 >> v1:172.21.15.40:6789/0 conn(0x7f08280d1940 legacy=0x7f08280702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.631+0000 7f08267fc640 1 -- 172.21.15.40:0/2298434675 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08280d0d10 con 0x7f08280b2420 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f08267fc640 1 --2- 172.21.15.40:0/2298434675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08280b2420 0x7f082809c560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f08100097a0 tx=0x7f081002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aeb26fcbced29cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810037070 con 0x7f08280b2420 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081002ac50 con 0x7f08280b2420 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f0825ffb640 1 -- 172.21.15.40:0/2298434675 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180019f0 con 0x7f08280b2420 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 -- 172.21.15.40:0/2298434675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08280b2420 msgr2=0x7f082809c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 --2- 172.21.15.40:0/2298434675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08280b2420 0x7f082809c560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f08100097a0 tx=0x7f081002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 -- 172.21.15.40:0/2298434675 shutdown_connections 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 --2- 172.21.15.40:0/2298434675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08280b2420 0x7f082809c560 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 -- 172.21.15.40:0/2298434675 >> 172.21.15.40:0/2298434675 conn(0x7f082809ba40 msgr2=0x7f082809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 -- 172.21.15.40:0/2298434675 shutdown_connections 2024-04-26T21:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.632+0000 7f082d62a640 1 -- 172.21.15.40:0/2298434675 wait complete. 2024-04-26T21:19:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.633+0000 7f082d62a640 1 Processor -- start 2024-04-26T21:19:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 -- start start 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08280d1940 0x7f08281f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082806e940 0x7f08280a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08280b2090 con 0x7f082815d340 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f082815d720 con 0x7f082806e940 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f082d62a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f08280b2210 con 0x7f08280d1940 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f08267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f08267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55960/0 (socket says 172.21.15.40:55960) 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.634+0000 7f08267fc640 1 -- 172.21.15.40:0/3894529563 learned_addr learned my addr 172.21.15.40:0/3894529563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08277fe640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082806e940 0x7f08280a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f0826ffd640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08280d1940 0x7f08281f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 -- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08280d1940 msgr2=0x7f08281f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08280d1940 0x7f08281f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 -- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082806e940 msgr2=0x7f08280a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082806e940 0x7f08280a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 -- 172.21.15.40:0/3894529563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828154c70 con 0x7f082815d340 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f08267fc640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f0810035040 tx=0x7f08100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810037050 con 0x7f082815d340 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810043070 con 0x7f082815d340 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810006e10 con 0x7f082815d340 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082806ee50 con 0x7f082815d340 2024-04-26T21:19:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.635+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082809d490 con 0x7f082815d340 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.636+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082809cfe0 con 0x7f082815d340 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.637+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f081003c810 con 0x7f082815d340 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.637+0000 7f081f7fe640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 0x7f080806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.637+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f08100ac280 con 0x7f082815d340 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.637+0000 7f0826ffd640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 0x7f080806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.638+0000 7f0826ffd640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 0x7f080806ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0820007fd0 tx=0x7f0820009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.641+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f081007a060 con 0x7f082815d340 2024-04-26T21:19:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.828+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f082815cad0 con 0x7f082815d340 2024-04-26T21:19:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.829+0000 7f081f7fe640 1 -- 172.21.15.40:0/3894529563 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f081007e360 con 0x7f082815d340 2024-04-26T21:19:39.830 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:19:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 msgr2=0x7f080806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 0x7f080806ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0820007fd0 tx=0x7f0820009040 comp rx=0 tx=0).stop 2024-04-26T21:19:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 msgr2=0x7f082806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f0810035040 tx=0x7f08100046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 shutdown_connections 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f080806c920 0x7f080806ede0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082806e940 0x7f08280a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f082815d340 0x7f082806e400 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 --2- 172.21.15.40:0/3894529563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08280d1940 0x7f08281f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 >> 172.21.15.40:0/3894529563 conn(0x7f082809ba40 msgr2=0x7f08280c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 shutdown_connections 2024-04-26T21:19:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.831+0000 7f082d62a640 1 -- 172.21.15.40:0/3894529563 wait complete. 2024-04-26T21:19:39.843 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:19:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.946+0000 7f7a87de7640 1 Processor -- start 2024-04-26T21:19:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 -- start start 2024-04-26T21:19:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8015f490 0x7f7a800d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 0x7f7a800d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a800d2120 0x7f7a800ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7a800d2570 con 0x7f7a800d2120 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7a800d4240 con 0x7f7a800d0ff0 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.947+0000 7f7a87de7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800d31a0 con 0x7f7a8015f490 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a800d2120 0x7f7a800ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a800d2120 0x7f7a800ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55976/0 (socket says 172.21.15.40:55976) 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 -- 172.21.15.40:0/1184330266 learned_addr learned my addr 172.21.15.40:0/1184330266 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a85b83640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8015f490 0x7f7a800d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a85382640 1 --2- 172.21.15.40:0/1184330266 >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 0x7f7a800d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 -- 172.21.15.40:0/1184330266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8015f490 msgr2=0x7f7a800d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8015f490 0x7f7a800d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 -- 172.21.15.40:0/1184330266 >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 msgr2=0x7f7a800d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 --2- 172.21.15.40:0/1184330266 >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 0x7f7a800d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a86384640 1 -- 172.21.15.40:0/1184330266 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800d3480 con 0x7f7a800d2120 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.948+0000 7f7a85382640 1 --2- 172.21.15.40:0/1184330266 >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 0x7f7a800d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a86384640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d2120 0x7f7a800ba3a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7a8008d620 tx=0x7f7a7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6187dee8bbc38b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a84b81640 1 -- 172.21.15.40:0/1184330266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70036070 con 0x7f7a800d2120 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a84b81640 1 -- 172.21.15.40:0/1184330266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a70031600 con 0x7f7a800d2120 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a84b81640 1 -- 172.21.15.40:0/1184330266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a700319d0 con 0x7f7a800d2120 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 -- 172.21.15.40:0/1184330266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d2120 msgr2=0x7f7a800ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d2120 0x7f7a800ba3a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7a8008d620 tx=0x7f7a7002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 -- 172.21.15.40:0/1184330266 shutdown_connections 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d2120 0x7f7a800ba3a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1184330266 >> v2:172.21.15.184:3300/0 conn(0x7f7a800d0ff0 0x7f7a800d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1184330266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a8015f490 0x7f7a800d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 -- 172.21.15.40:0/1184330266 >> 172.21.15.40:0/1184330266 conn(0x7f7a8009e530 msgr2=0x7f7a800d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 -- 172.21.15.40:0/1184330266 shutdown_connections 2024-04-26T21:19:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.949+0000 7f7a87de7640 1 -- 172.21.15.40:0/1184330266 wait complete. 2024-04-26T21:19:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 Processor -- start 2024-04-26T21:19:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 -- start start 2024-04-26T21:19:39.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 0x7f7a800c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8015f490 0x7f7a800ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800d3d20 con 0x7f7a800d0ff0 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800d3a40 con 0x7f7a8015f490 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a87de7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8015f870 con 0x7f7a800d2120 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a85b83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a85b83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55986/0 (socket says 172.21.15.40:55986) 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.951+0000 7f7a85b83640 1 -- 172.21.15.40:0/1439256825 learned_addr learned my addr 172.21.15.40:0/1439256825 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a86384640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8015f490 0x7f7a800ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85382640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 0x7f7a800c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 -- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 msgr2=0x7f7a800c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 0x7f7a800c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 -- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8015f490 msgr2=0x7f7a800ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8015f490 0x7f7a800ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 -- 172.21.15.40:0/1439256825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800cbb40 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85382640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 0x7f7a800c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a85b83640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d830 tx=0x7f7a7800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800b6c0 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78019070 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a800cbe20 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800ba80 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.952+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8015e720 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.953+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8006a290 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.954+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a7801e460 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.954+0000 7f7a7e7fc640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.954+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f7a7808c0f0 con 0x7f7a800d0ff0 2024-04-26T21:19:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.954+0000 7f7a85382640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.955+0000 7f7a85382640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a800b6950 tx=0x7f7a74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:39.958+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a7805a520 con 0x7f7a800d0ff0 2024-04-26T21:19:40.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.142+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a800d5900 con 0x7f7a800d0ff0 2024-04-26T21:19:40.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.142+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/1439256825 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f7a78059e20 con 0x7f7a800d0ff0 2024-04-26T21:19:40.143 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:19:40.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 msgr2=0x7f7a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a800b6950 tx=0x7f7a74008040 comp rx=0 tx=0).stop 2024-04-26T21:19:40.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 msgr2=0x7f7a801eb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d830 tx=0x7f7a7800dcf0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 shutdown_connections 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a8015f490 0x7f7a800ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a800d2120 0x7f7a800c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 --2- 172.21.15.40:0/1439256825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800d0ff0 0x7f7a801eb260 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 >> 172.21.15.40:0/1439256825 conn(0x7f7a8009e530 msgr2=0x7f7a80075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 shutdown_connections 2024-04-26T21:19:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.144+0000 7f7a87de7640 1 -- 172.21.15.40:0/1439256825 wait complete. 2024-04-26T21:19:40.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:19:40.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.257+0000 7f43cfe83640 1 Processor -- start 2024-04-26T21:19:40.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.257+0000 7f43cfe83640 1 -- start start 2024-04-26T21:19:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cfe83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43c806fa50 0x7f43c806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cfe83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c80731f0 0x7f43c8073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cfe83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c80ae8b0 con 0x7f43c81485b0 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cfe83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43c8073ba0 con 0x7f43c806fa50 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cfe83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80aeb90 con 0x7f43c80731f0 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cd41e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43c806fa50 0x7f43c806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cd41e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43c806fa50 0x7f43c806fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55996/0 (socket says 172.21.15.40:55996) 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.258+0000 7f43cd41e640 1 -- 172.21.15.40:0/81393355 learned_addr learned my addr 172.21.15.40:0/81393355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43ce420640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c80731f0 0x7f43c8073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43cd41e640 1 -- 172.21.15.40:0/81393355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c80731f0 msgr2=0x7f43c8073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43cd41e640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c80731f0 0x7f43c8073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43cd41e640 1 -- 172.21.15.40:0/81393355 >> v1:172.21.15.184:6789/0 conn(0x7f43c81485b0 legacy=0x7f43c80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43cd41e640 1 -- 172.21.15.40:0/81393355 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c80aee70 con 0x7f43c806fa50 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.259+0000 7f43cd41e640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c806fa50 0x7f43c806fee0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009870 tx=0x7f43bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f52d89a330985f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43ccc1d640 1 -- 172.21.15.40:0/81393355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc037070 con 0x7f43c806fa50 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43ccc1d640 1 -- 172.21.15.40:0/81393355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc032440 con 0x7f43c806fa50 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43ccc1d640 1 -- 172.21.15.40:0/81393355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc032810 con 0x7f43c806fa50 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 -- 172.21.15.40:0/81393355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c806fa50 msgr2=0x7f43c806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c806fa50 0x7f43c806fee0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009870 tx=0x7f43bc02a270 comp rx=0 tx=0).stop 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 -- 172.21.15.40:0/81393355 shutdown_connections 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c80731f0 0x7f43c8073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 --2- 172.21.15.40:0/81393355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c806fa50 0x7f43c806fee0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 -- 172.21.15.40:0/81393355 >> 172.21.15.40:0/81393355 conn(0x7f43c80d0d30 msgr2=0x7f43c80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:40.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 -- 172.21.15.40:0/81393355 shutdown_connections 2024-04-26T21:19:40.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.260+0000 7f43cfe83640 1 -- 172.21.15.40:0/81393355 wait complete. 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.261+0000 7f43cfe83640 1 Processor -- start 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.261+0000 7f43cfe83640 1 -- start start 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 0x7f43c81f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 0x7f43c8142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80af710 con 0x7f43c80731f0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80af430 con 0x7f43c81485b0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cfe83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80b4cd0 con 0x7f43c806fa50 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cd41e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cd41e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56012/0 (socket says 172.21.15.40:56012) 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.262+0000 7f43cd41e640 1 -- 172.21.15.40:0/3116606751 learned_addr learned my addr 172.21.15.40:0/3116606751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43ce420640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 0x7f43c8142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cdc1f640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 0x7f43c81f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 -- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 msgr2=0x7f43c81f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 0x7f43c81f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 -- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 msgr2=0x7f43c8142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 0x7f43c8142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 -- 172.21.15.40:0/3116606751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c8134d10 con 0x7f43c80731f0 2024-04-26T21:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43ce420640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 0x7f43c8142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cdc1f640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 0x7f43c81f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cd41e640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc041020 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc0306c0 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc037040 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c815feb0 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.263+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c8160470 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.264+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c806a290 con 0x7f43c80731f0 2024-04-26T21:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.265+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43bc044020 con 0x7f43c80731f0 2024-04-26T21:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.265+0000 7f43c67fc640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.265+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f43bc0ac310 con 0x7f43c80731f0 2024-04-26T21:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.265+0000 7f43cdc1f640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.266+0000 7f43cdc1f640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 0x7f43b406ede0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f43c00015e0 tx=0x7f43c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.270+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43bc04b030 con 0x7f43c80731f0 2024-04-26T21:19:40.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.468+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43c8072ae0 con 0x7f43c80731f0 2024-04-26T21:19:40.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.468+0000 7f43c67fc640 1 -- 172.21.15.40:0/3116606751 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f43bc07a740 con 0x7f43c80731f0 2024-04-26T21:19:40.469 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 msgr2=0x7f43b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 0x7f43b406ede0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f43c00015e0 tx=0x7f43c0009040 comp rx=0 tx=0).stop 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 msgr2=0x7f43c8142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 shutdown_connections 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c81485b0 0x7f43c8142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43c80731f0 0x7f43c8142450 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 --2- 172.21.15.40:0/3116606751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c806fa50 0x7f43c81f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 >> 172.21.15.40:0/3116606751 conn(0x7f43c80d0d30 msgr2=0x7f43c80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 shutdown_connections 2024-04-26T21:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.470+0000 7f43cfe83640 1 -- 172.21.15.40:0/3116606751 wait complete. 2024-04-26T21:19:40.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:19:40.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:40.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:19:40.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:19:40.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:40.657 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:19:40.672 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:40.672 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:40.672 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:40.688 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:40.688 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:40.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:40.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.844+0000 7f92d0178640 1 Processor -- start 2024-04-26T21:19:40.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.844+0000 7f92d0178640 1 -- start start 2024-04-26T21:19:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c815f340 0x7f92c80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 0x7f92c80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 0x7f92c80d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f92c80d58d0 con 0x7f92c815f340 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f92c80b2510 con 0x7f92c815f720 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92d0178640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92c80b27f0 con 0x7f92c80d4f00 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c815f340 0x7f92c80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92c815f340 0x7f92c80b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45508/0 (socket says 172.21.15.40:45508) 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 -- 172.21.15.40:0/1931860854 learned_addr learned my addr 172.21.15.40:0/1931860854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cd713640 1 --2- 172.21.15.40:0/1931860854 >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 0x7f92c80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92ce715640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 0x7f92c80d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 -- 172.21.15.40:0/1931860854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 msgr2=0x7f92c80d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 0x7f92c80d5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 -- 172.21.15.40:0/1931860854 >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 msgr2=0x7f92c80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 --2- 172.21.15.40:0/1931860854 >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 0x7f92c80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cdf14640 1 -- 172.21.15.40:0/1931860854 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c80b2ad0 con 0x7f92c815f340 2024-04-26T21:19:40.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92ce715640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 0x7f92c80d5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:40.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.845+0000 7f92cd713640 1 --2- 172.21.15.40:0/1931860854 >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 0x7f92c80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.846+0000 7f92cdf14640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f340 0x7f92c80b1e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f92c80cf8a0 tx=0x7f92c402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa81e657426dd63c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.846+0000 7f92ccf12640 1 -- 172.21.15.40:0/1931860854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4037070 con 0x7f92c815f340 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.846+0000 7f92ccf12640 1 -- 172.21.15.40:0/1931860854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c4032440 con 0x7f92c815f340 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.846+0000 7f92ccf12640 1 -- 172.21.15.40:0/1931860854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4032810 con 0x7f92c815f340 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 -- 172.21.15.40:0/1931860854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f340 msgr2=0x7f92c80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f340 0x7f92c80b1e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f92c80cf8a0 tx=0x7f92c402a140 comp rx=0 tx=0).stop 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 -- 172.21.15.40:0/1931860854 shutdown_connections 2024-04-26T21:19:40.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c80d4f00 0x7f92c80d5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 --2- 172.21.15.40:0/1931860854 >> v2:172.21.15.184:3300/0 conn(0x7f92c815f720 0x7f92c80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 --2- 172.21.15.40:0/1931860854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f340 0x7f92c80b1e80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 -- 172.21.15.40:0/1931860854 >> 172.21.15.40:0/1931860854 conn(0x7f92c808ec70 msgr2=0x7f92c809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 -- 172.21.15.40:0/1931860854 shutdown_connections 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.847+0000 7f92d0178640 1 -- 172.21.15.40:0/1931860854 wait complete. 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.848+0000 7f92d0178640 1 Processor -- start 2024-04-26T21:19:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.848+0000 7f92d0178640 1 -- start start 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c80d4f00 0x7f92c80a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c815f340 0x7f92c809f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92c80b5040 con 0x7f92c815f720 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92c80b4ec0 con 0x7f92c80d4f00 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92d0178640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92c80b51c0 con 0x7f92c815f340 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45520/0 (socket says 172.21.15.40:45520) 2024-04-26T21:19:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 -- 172.21.15.40:0/3197624869 learned_addr learned my addr 172.21.15.40:0/3197624869 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92cd713640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c815f340 0x7f92c809f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92cdf14640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c80d4f00 0x7f92c80a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 -- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c815f340 msgr2=0x7f92c809f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c815f340 0x7f92c809f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 -- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c80d4f00 msgr2=0x7f92c80a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c80d4f00 0x7f92c80a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 -- 172.21.15.40:0/3197624869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c80a0ff0 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92ce715640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f92c0007e10 tx=0x7f92c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c001a070 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.849+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c0003b90 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.850+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c80b81f0 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.850+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004030 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.850+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c80b87b0 con 0x7f92c815f720 2024-04-26T21:19:40.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.851+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c806a290 con 0x7f92c815f720 2024-04-26T21:19:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.851+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92c001d070 con 0x7f92c815f720 2024-04-26T21:19:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.851+0000 7f92be7fc640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 0x7f92ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.852+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f92c008bc80 con 0x7f92c815f720 2024-04-26T21:19:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.852+0000 7f92cdf14640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 0x7f92ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:40.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.852+0000 7f92cdf14640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 0x7f92ac06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f92c4034000 tx=0x7f92c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:40.856+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92c0010070 con 0x7f92c815f720 2024-04-26T21:19:41.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.041+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92c80b4b20 con 0x7f92c815f720 2024-04-26T21:19:41.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.042+0000 7f92be7fc640 1 -- 172.21.15.40:0/3197624869 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16627 (secure 0 0 0) 0x7f92c0059af0 con 0x7f92c815f720 2024-04-26T21:19:41.043 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:41.043 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":109,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:17.479995+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 msgr2=0x7f92ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 0x7f92ac06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f92c4034000 tx=0x7f92c402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 msgr2=0x7f92c80b9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f92c0007e10 tx=0x7f92c000d4a0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 shutdown_connections 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92ac06c8f0 0x7f92ac06edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92c815f720 0x7f92c80b9570 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92c815f340 0x7f92c809f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 --2- 172.21.15.40:0/3197624869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c80d4f00 0x7f92c80a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 >> 172.21.15.40:0/3197624869 conn(0x7f92c808ec70 msgr2=0x7f92c806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 shutdown_connections 2024-04-26T21:19:41.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.044+0000 7f92d0178640 1 -- 172.21.15.40:0/3197624869 wait complete. 2024-04-26T21:19:41.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:14.110950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:14.791024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:41.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:41.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.159+0000 7fe839346640 1 Processor -- start 2024-04-26T21:19:41.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.159+0000 7fe839346640 1 -- start start 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe839346640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe83415f260 0x7fe83406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe839346640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe834070560 0x7fe8340b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe839346640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83415f640 con 0x7fe83406ebe0 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe839346640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe83406c2a0 con 0x7fe834070560 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe839346640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe83406c580 con 0x7fe83415f260 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe82a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe834070560 0x7fe8340b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe8337fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe83406ebe0 0x7fe8340ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60838/0 (socket says 172.21.15.40:60838) 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe8337fe640 1 -- 172.21.15.40:0/1990450258 learned_addr learned my addr 172.21.15.40:0/1990450258 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe832ffd640 1 --2- 172.21.15.40:0/1990450258 >> v2:172.21.15.184:3300/0 conn(0x7fe83415f260 0x7fe83406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe82a7fc640 1 -- 172.21.15.40:0/1990450258 >> v2:172.21.15.184:3300/0 conn(0x7fe83415f260 msgr2=0x7fe83406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe82a7fc640 1 --2- 172.21.15.40:0/1990450258 >> v2:172.21.15.184:3300/0 conn(0x7fe83415f260 0x7fe83406d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe82a7fc640 1 -- 172.21.15.40:0/1990450258 >> v1:172.21.15.40:6789/0 conn(0x7fe83406ebe0 legacy=0x7fe8340ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.160+0000 7fe82a7fc640 1 -- 172.21.15.40:0/1990450258 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83406c860 con 0x7fe834070560 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe82a7fc640 1 --2- 172.21.15.40:0/1990450258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834070560 0x7fe8340b46f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe834068a40 tx=0x7fe81c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59569577a184feed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe8327fc640 1 -- 172.21.15.40:0/1990450258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c037070 con 0x7fe834070560 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe8327fc640 1 -- 172.21.15.40:0/1990450258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c032440 con 0x7fe834070560 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe8327fc640 1 -- 172.21.15.40:0/1990450258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c02ae60 con 0x7fe834070560 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe839346640 1 -- 172.21.15.40:0/1990450258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834070560 msgr2=0x7fe8340b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.161+0000 7fe839346640 1 --2- 172.21.15.40:0/1990450258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834070560 0x7fe8340b46f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe834068a40 tx=0x7fe81c02a110 comp rx=0 tx=0).stop 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 -- 172.21.15.40:0/1990450258 shutdown_connections 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 --2- 172.21.15.40:0/1990450258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe834070560 0x7fe8340b46f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 --2- 172.21.15.40:0/1990450258 >> v2:172.21.15.184:3300/0 conn(0x7fe83415f260 0x7fe83406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 -- 172.21.15.40:0/1990450258 >> 172.21.15.40:0/1990450258 conn(0x7fe8340d1660 msgr2=0x7fe8340d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 -- 172.21.15.40:0/1990450258 shutdown_connections 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.162+0000 7fe839346640 1 -- 172.21.15.40:0/1990450258 wait complete. 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.163+0000 7fe839346640 1 Processor -- start 2024-04-26T21:19:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.163+0000 7fe839346640 1 -- start start 2024-04-26T21:19:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.163+0000 7fe839346640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83406ebe0 0x7fe834154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 0x7fe8340cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe83406d100 con 0x7fe83415f260 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe83406ce20 con 0x7fe83406ebe0 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe83406db60 con 0x7fe834070560 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45542/0 (socket says 172.21.15.40:45542) 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 -- 172.21.15.40:0/645447403 learned_addr learned my addr 172.21.15.40:0/645447403 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe832ffd640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83406ebe0 0x7fe834154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe82a7fc640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 0x7fe8340cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 -- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 msgr2=0x7fe8340cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 0x7fe8340cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 -- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83406ebe0 msgr2=0x7fe834154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83406ebe0 0x7fe834154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 -- 172.21.15.40:0/645447403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83415f8b0 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe82a7fc640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 0x7fe8340cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe8337fe640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ca80 tx=0x7fe82c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c017070 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82c00ac30 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83415fb90 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c012600 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.164+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe834162470 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.165+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83406a290 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.166+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe82c01a070 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.166+0000 7fe82bfff640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.166+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fe82c08b700 con 0x7fe83415f260 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.167+0000 7fe832ffd640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.167+0000 7fe832ffd640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 0x7fe81406edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe824006fd0 tx=0x7fe824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.171+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe82c00ae20 con 0x7fe83415f260 2024-04-26T21:19:41.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.355+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8341621d0 con 0x7fe83415f260 2024-04-26T21:19:41.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.356+0000 7fe82bfff640 1 -- 172.21.15.40:0/645447403 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fe82c0594f0 con 0x7fe83415f260 2024-04-26T21:19:41.357 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 msgr2=0x7fe81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 0x7fe81406edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe824006fd0 tx=0x7fe824008040 comp rx=0 tx=0).stop 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 msgr2=0x7fe834161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ca80 tx=0x7fe82c00cf40 comp rx=0 tx=0).stop 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 shutdown_connections 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83415f260 0x7fe834161840 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe834070560 0x7fe8340cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 --2- 172.21.15.40:0/645447403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83406ebe0 0x7fe834154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 >> 172.21.15.40:0/645447403 conn(0x7fe8340d1660 msgr2=0x7fe8340d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 shutdown_connections 2024-04-26T21:19:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.358+0000 7fe839346640 1 -- 172.21.15.40:0/645447403 wait complete. 2024-04-26T21:19:41.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:19:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 Processor -- start 2024-04-26T21:19:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 -- start start 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faba81468a0 0x7faba80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba8070740 con 0x7faba806ffd0 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faba8070a00 con 0x7faba81468a0 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.472+0000 7fabae70a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faba8070ce0 con 0x7faba815d6f0 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba77fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faba81468a0 0x7faba80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7fabacca7640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7faba806ffd0 0x7faba806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:45088/0 (socket says 172.21.15.40:45088) 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faba81468a0 0x7faba80b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33334/0 (socket says 172.21.15.40:33334) 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7fabacca7640 1 -- 172.21.15.40:0/430381644 learned_addr learned my addr 172.21.15.40:0/430381644 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba77fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38496/0 (socket says 172.21.15.40:38496) 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 -- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 msgr2=0x7faba80b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 -- 172.21.15.40:0/430381644 >> v1:172.21.15.184:6789/0 conn(0x7faba806ffd0 legacy=0x7faba806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba7fff640 1 -- 172.21.15.40:0/430381644 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba8070fc0 con 0x7faba81468a0 2024-04-26T21:19:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.473+0000 7faba77fe640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.474+0000 7faba7fff640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba81468a0 0x7faba80b7250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fab980097a0 tx=0x7fab9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458d8dc30a77114f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.474+0000 7faba6ffd640 1 -- 172.21.15.40:0/430381644 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98037070 con 0x7faba81468a0 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.474+0000 7faba6ffd640 1 -- 172.21.15.40:0/430381644 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9802ae50 con 0x7faba81468a0 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.474+0000 7faba6ffd640 1 -- 172.21.15.40:0/430381644 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98032660 con 0x7faba81468a0 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 -- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba81468a0 msgr2=0x7faba80b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba81468a0 0x7faba80b7250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fab980097a0 tx=0x7fab9802a100 comp rx=0 tx=0).stop 2024-04-26T21:19:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 -- 172.21.15.40:0/430381644 shutdown_connections 2024-04-26T21:19:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba815d6f0 0x7faba80b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 --2- 172.21.15.40:0/430381644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba81468a0 0x7faba80b7250 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 -- 172.21.15.40:0/430381644 >> 172.21.15.40:0/430381644 conn(0x7faba806f050 msgr2=0x7faba806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 -- 172.21.15.40:0/430381644 shutdown_connections 2024-04-26T21:19:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.475+0000 7fabae70a640 1 -- 172.21.15.40:0/430381644 wait complete. 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.476+0000 7fabae70a640 1 Processor -- start 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.476+0000 7fabae70a640 1 -- start start 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 0x7faba815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba81468a0 0x7faba815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faba8146e00 con 0x7faba815d6f0 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faba8146c80 con 0x7faba806ffd0 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabae70a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faba80ce920 con 0x7faba81468a0 2024-04-26T21:19:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45556/0 (socket says 172.21.15.40:45556) 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 -- 172.21.15.40:0/2679664796 learned_addr learned my addr 172.21.15.40:0/2679664796 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7faba7fff640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 0x7faba815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7faba77fe640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba81468a0 0x7faba815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 -- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba81468a0 msgr2=0x7faba815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba81468a0 0x7faba815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 -- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 msgr2=0x7faba815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 0x7faba815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 -- 172.21.15.40:0/2679664796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba80cb470 con 0x7faba815d6f0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7faba7fff640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 0x7faba815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.477+0000 7fabacca7640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7faba000cbc0 tx=0x7faba000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.478+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7faba815d6f0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.478+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba000ad40 con 0x7faba815d6f0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.478+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba8160540 con 0x7faba815d6f0 2024-04-26T21:19:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.478+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0012600 con 0x7faba815d6f0 2024-04-26T21:19:41.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.478+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba80cb730 con 0x7faba815d6f0 2024-04-26T21:19:41.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.479+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba806a290 con 0x7faba815d6f0 2024-04-26T21:19:41.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.479+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faba001a070 con 0x7faba815d6f0 2024-04-26T21:19:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.480+0000 7faba4ff9640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 0x7fab9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.480+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7faba008b8b0 con 0x7faba815d6f0 2024-04-26T21:19:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.480+0000 7faba7fff640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 0x7fab9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.480+0000 7faba7fff640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 0x7fab9406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab98034000 tx=0x7fab9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.484+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faba0059600 con 0x7faba815d6f0 2024-04-26T21:19:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.670+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faba8073b10 con 0x7faba815d6f0 2024-04-26T21:19:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.670+0000 7faba4ff9640 1 -- 172.21.15.40:0/2679664796 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7faba005d900 con 0x7faba815d6f0 2024-04-26T21:19:41.671 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 msgr2=0x7fab9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 0x7fab9406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab98034000 tx=0x7fab9802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 msgr2=0x7faba813a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7faba000cbc0 tx=0x7faba000a430 comp rx=0 tx=0).stop 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 shutdown_connections 2024-04-26T21:19:41.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab9406c8f0 0x7fab9406edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba815d6f0 0x7faba813a230 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faba81468a0 0x7faba815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 --2- 172.21.15.40:0/2679664796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faba806ffd0 0x7faba815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.672+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 >> 172.21.15.40:0/2679664796 conn(0x7faba806f050 msgr2=0x7faba80b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.673+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 shutdown_connections 2024-04-26T21:19:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.673+0000 7fabae70a640 1 -- 172.21.15.40:0/2679664796 wait complete. 2024-04-26T21:19:41.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:19:41.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.787+0000 7f41f0499640 1 Processor -- start 2024-04-26T21:19:41.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.787+0000 7f41f0499640 1 -- start start 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.787+0000 7f41f0499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e815f490 0x7f41e80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.787+0000 7f41f0499640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41e80d22f0 0x7f41e80ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41f0499640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e80d4240 con 0x7f41e80d0ff0 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41f0499640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f41e80d31a0 con 0x7f41e80d22f0 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41f0499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41e80d3480 con 0x7f41e815f490 2024-04-26T21:19:41.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41e80d22f0 0x7f41e80ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41e80d22f0 0x7f41e80ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45560/0 (socket says 172.21.15.40:45560) 2024-04-26T21:19:41.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 -- 172.21.15.40:0/510882283 learned_addr learned my addr 172.21.15.40:0/510882283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41ee235640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e815f490 0x7f41e80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 -- 172.21.15.40:0/510882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e815f490 msgr2=0x7f41e80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e815f490 0x7f41e80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 -- 172.21.15.40:0/510882283 >> v1:172.21.15.184:6789/0 conn(0x7f41e80d0ff0 legacy=0x7f41e80d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.788+0000 7f41eea36640 1 -- 172.21.15.40:0/510882283 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80d3760 con 0x7f41e80d22f0 2024-04-26T21:19:41.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41eea36640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d22f0 0x7f41e80ba3a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009870 tx=0x7f41e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f544e1b8256e34e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41ed233640 1 -- 172.21.15.40:0/510882283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0037070 con 0x7f41e80d22f0 2024-04-26T21:19:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41ed233640 1 -- 172.21.15.40:0/510882283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e0032440 con 0x7f41e80d22f0 2024-04-26T21:19:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41ed233640 1 -- 172.21.15.40:0/510882283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0032810 con 0x7f41e80d22f0 2024-04-26T21:19:41.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 -- 172.21.15.40:0/510882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d22f0 msgr2=0x7f41e80ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d22f0 0x7f41e80ba3a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009870 tx=0x7f41e002a210 comp rx=0 tx=0).stop 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 -- 172.21.15.40:0/510882283 shutdown_connections 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d22f0 0x7f41e80ba3a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 --2- 172.21.15.40:0/510882283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e815f490 0x7f41e80d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 -- 172.21.15.40:0/510882283 >> 172.21.15.40:0/510882283 conn(0x7f41e809e530 msgr2=0x7f41e80d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 -- 172.21.15.40:0/510882283 shutdown_connections 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.789+0000 7f41f0499640 1 -- 172.21.15.40:0/510882283 wait complete. 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 Processor -- start 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 -- start start 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 0x7f41e80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e815f490 0x7f41e80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41e815f870 con 0x7f41e80d0ff0 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41e80d3d20 con 0x7f41e815f490 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41f0499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f41e80d2dc0 con 0x7f41e80d22f0 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41ee235640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41ee235640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45572/0 (socket says 172.21.15.40:45572) 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.791+0000 7f41ee235640 1 -- 172.21.15.40:0/2635134149 learned_addr learned my addr 172.21.15.40:0/2635134149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41eea36640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e815f490 0x7f41e80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41eda34640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 0x7f41e80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 -- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 msgr2=0x7f41e80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 0x7f41e80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 -- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e815f490 msgr2=0x7f41e80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e815f490 0x7f41e80cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 -- 172.21.15.40:0/2635134149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e80ba2b0 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41eda34640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 0x7f41e80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41ee235640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ce40 tx=0x7f41d800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8003b90 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d8003d30 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e80ba590 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8004240 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.792+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e80bab50 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.793+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e806a290 con 0x7f41e80d0ff0 2024-04-26T21:19:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.793+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d80043e0 con 0x7f41e80d0ff0 2024-04-26T21:19:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.794+0000 7f41e6ffd640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.794+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f41d808c650 con 0x7f41e80d0ff0 2024-04-26T21:19:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.794+0000 7f41eda34640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.795+0000 7f41eda34640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 0x7f41d406ede0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f41e8161fb0 tx=0x7f41dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.799+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41d801d340 con 0x7f41e80d0ff0 2024-04-26T21:19:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.983+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41e80cc030 con 0x7f41e80d0ff0 2024-04-26T21:19:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.983+0000 7f41e6ffd640 1 -- 172.21.15.40:0/2635134149 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f41d805aa80 con 0x7f41e80d0ff0 2024-04-26T21:19:41.984 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:19:41.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 msgr2=0x7f41d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 0x7f41d406ede0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f41e8161fb0 tx=0x7f41dc008040 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 msgr2=0x7f41e81f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ce40 tx=0x7f41d800a660 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 shutdown_connections 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f41d406c920 0x7f41d406ede0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e815f490 0x7f41e80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e80d22f0 0x7f41e80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 --2- 172.21.15.40:0/2635134149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41e80d0ff0 0x7f41e81f4d90 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.985+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 >> 172.21.15.40:0/2635134149 conn(0x7f41e809e530 msgr2=0x7f41e80cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.986+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 shutdown_connections 2024-04-26T21:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:41.986+0000 7f41f0499640 1 -- 172.21.15.40:0/2635134149 wait complete. 2024-04-26T21:19:41.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:19:42.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.097+0000 7f0230662640 1 Processor -- start 2024-04-26T21:19:42.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.098+0000 7f0230662640 1 -- start start 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.098+0000 7f0230662640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f0230662640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02280b3de0 0x7f02280720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f0230662640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02280b4530 con 0x7f0228145980 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f0230662640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f02280b5140 con 0x7f02280b3de0 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f0230662640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f02280b4810 con 0x7f02280b61a0 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022ebff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f02280b3de0 0x7f02280720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022e3fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0228145980 0x7f02280b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60840/0 (socket says 172.21.15.40:60840) 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022e3fe640 1 -- 172.21.15.40:0/2460523034 learned_addr learned my addr 172.21.15.40:0/2460523034 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022dbfd640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022ebff640 1 -- 172.21.15.40:0/2460523034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 msgr2=0x7f0228160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022ebff640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022ebff640 1 -- 172.21.15.40:0/2460523034 >> v1:172.21.15.40:6789/0 conn(0x7f0228145980 legacy=0x7f02280b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022ebff640 1 -- 172.21.15.40:0/2460523034 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02280b4af0 con 0x7f02280b3de0 2024-04-26T21:19:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.099+0000 7f022d3fc640 1 -- 172.21.15.40:0/2460523034 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7609032 0 0) 0x7f02280b4530 con 0x7f0228145980 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f022ebff640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f02280720d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0220009d90 tx=0x7f022002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c587cc4d96d2e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f022d3fc640 1 -- 172.21.15.40:0/2460523034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220031600 con 0x7f02280b3de0 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f022d3fc640 1 -- 172.21.15.40:0/2460523034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02200317a0 con 0x7f02280b3de0 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f022d3fc640 1 -- 172.21.15.40:0/2460523034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220031b70 con 0x7f02280b3de0 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f0230662640 1 -- 172.21.15.40:0/2460523034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 msgr2=0x7f02280720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.100+0000 7f0230662640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f02280720d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0220009d90 tx=0x7f022002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 -- 172.21.15.40:0/2460523034 shutdown_connections 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f02280720d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 --2- 172.21.15.40:0/2460523034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 -- 172.21.15.40:0/2460523034 >> 172.21.15.40:0/2460523034 conn(0x7f02280d0d30 msgr2=0x7f02280d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 -- 172.21.15.40:0/2460523034 shutdown_connections 2024-04-26T21:19:42.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.101+0000 7f0230662640 1 -- 172.21.15.40:0/2460523034 wait complete. 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.102+0000 7f0230662640 1 Processor -- start 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.102+0000 7f0230662640 1 -- start start 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0228145980 0x7f02280cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0228072610 con 0x7f02280b3de0 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02280b4240 con 0x7f0228145980 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0228072790 con 0x7f02280b61a0 2024-04-26T21:19:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45588/0 (socket says 172.21.15.40:45588) 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 -- 172.21.15.40:0/3053035323 learned_addr learned my addr 172.21.15.40:0/3053035323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022ebff640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0228145980 0x7f02280cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 -- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 msgr2=0x7f0228158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 -- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0228145980 msgr2=0x7f02280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0228145980 0x7f02280cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 -- 172.21.15.40:0/3053035323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02280cf7c0 con 0x7f02280b3de0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f022e3fe640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f022400c460 tx=0x7f022400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224018070 con 0x7f02280b3de0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0224007500 con 0x7f02280b3de0 2024-04-26T21:19:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022813e150 con 0x7f02280b3de0 2024-04-26T21:19:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224004a20 con 0x7f02280b3de0 2024-04-26T21:19:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.103+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022813e710 con 0x7f02280b3de0 2024-04-26T21:19:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.104+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022806a290 con 0x7f02280b3de0 2024-04-26T21:19:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.105+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f022401b070 con 0x7f02280b3de0 2024-04-26T21:19:42.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.105+0000 7f021effd640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 0x7f020c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.105+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f022408ace0 con 0x7f02280b3de0 2024-04-26T21:19:42.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.106+0000 7f022dbfd640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 0x7f020c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.106+0000 7f022dbfd640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 0x7f020c06ede0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0218006a90 tx=0x7f0218006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.110+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02280cf7c0 con 0x7f02280b3de0 2024-04-26T21:19:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.308+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0228073cc0 con 0x7f02280b3de0 2024-04-26T21:19:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.308+0000 7f021effd640 1 -- 172.21.15.40:0/3053035323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f02240591a0 con 0x7f02280b3de0 2024-04-26T21:19:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 msgr2=0x7f020c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 0x7f020c06ede0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0218006a90 tx=0x7f0218006a20 comp rx=0 tx=0).stop 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 msgr2=0x7f022815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f022400c460 tx=0x7f022400c920 comp rx=0 tx=0).stop 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 shutdown_connections 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f020c06c920 0x7f020c06ede0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0228145980 0x7f02280cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02280b61a0 0x7f0228158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 --2- 172.21.15.40:0/3053035323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02280b3de0 0x7f022815ba80 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 >> 172.21.15.40:0/3053035323 conn(0x7f02280d0d30 msgr2=0x7f0228144250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 shutdown_connections 2024-04-26T21:19:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.310+0000 7f0230662640 1 -- 172.21.15.40:0/3053035323 wait complete. 2024-04-26T21:19:42.322 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:19:42.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:42.322 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:19:42.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:19:42.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:19:42.354 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:19:42.413 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:42.413 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:42.413 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:42.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:42.429 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:19:42.429 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:42.430 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:42.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:42.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.571+0000 7f7a8f146640 1 Processor -- start 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.571+0000 7f7a8f146640 1 -- start start 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8f146640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7a8813ff70 0x7f7a88146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8f146640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a880d05b0 con 0x7f7a880cff20 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8f146640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a880d0890 con 0x7f7a88140350 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8f146640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8806e450 con 0x7f7a8813ff70 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8d6e3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a880cff20 0x7f7a8806ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60844/0 (socket says 172.21.15.40:60844) 2024-04-26T21:19:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8d6e3640 1 -- 172.21.15.40:0/2111779647 learned_addr learned my addr 172.21.15.40:0/2111779647 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4048991870 0 0) 0x7f7a880d05b0 con 0x7f7a880cff20 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a70003520 con 0x7f7a880cff20 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.572+0000 7f7a8cee2640 1 --2- 172.21.15.40:0/2111779647 >> v2:172.21.15.184:3300/0 conn(0x7f7a8813ff70 0x7f7a88146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715538425 0 0) 0x7f7a70003520 con 0x7f7a880cff20 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 >> v2:172.21.15.184:3300/0 conn(0x7f7a8813ff70 msgr2=0x7f7a88146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 --2- 172.21.15.40:0/2111779647 >> v2:172.21.15.184:3300/0 conn(0x7f7a8813ff70 0x7f7a88146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 >> v1:172.21.15.184:6789/0 conn(0x7f7a88140350 legacy=0x7f7a880cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8806e730 con 0x7f7a880cff20 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7a78001670 con 0x7f7a880cff20 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 >> v1:172.21.15.40:6789/0 conn(0x7f7a880cff20 legacy=0x7f7a8806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 0x7f7a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700052e0 0x7f7a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8806e450 con 0x7f7a70004800 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7000c200 con 0x7f7a70003a00 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7000c400 con 0x7f7a700052e0 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 0x7f7a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a8cee2640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a8d6e3640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700052e0 0x7f7a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 -- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700052e0 msgr2=0x7f7a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700052e0 0x7f7a7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 -- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a70003a00 msgr2=0x7f7a70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a87fff640 1 -- 172.21.15.40:0/2111779647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70003520 con 0x7f7a70004800 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a8d6e3640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700052e0 0x7f7a7000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.573+0000 7f7a8d6e3640 1 -- 172.21.15.40:0/2111779647 reap_dead start 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 0x7f7a70004c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c001750 tx=0x7f7a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb291568000174d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c001370 con 0x7f7a70004800 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c0052d0 con 0x7f7a70004800 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a877fe640 1 -- 172.21.15.40:0/2111779647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c02abc0 con 0x7f7a70004800 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a8f146640 1 -- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 msgr2=0x7f7a70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.574+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 0x7f7a70004c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c001750 tx=0x7f7a70005830 comp rx=0 tx=0).stop 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.575+0000 7f7a8f146640 1 -- 172.21.15.40:0/2111779647 shutdown_connections 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.575+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2111779647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a70004800 0x7f7a70004c80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.575+0000 7f7a8f146640 1 -- 172.21.15.40:0/2111779647 >> 172.21.15.40:0/2111779647 conn(0x7f7a880d0d30 msgr2=0x7f7a880d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.575+0000 7f7a8f146640 1 -- 172.21.15.40:0/2111779647 shutdown_connections 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.575+0000 7f7a8f146640 1 -- 172.21.15.40:0/2111779647 wait complete. 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.576+0000 7f7a8f146640 1 Processor -- start 2024-04-26T21:19:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.576+0000 7f7a8f146640 1 -- start start 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a70004800 0x7f7a8813f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b00f0 0x7f7a88144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880cfba0 con 0x7f7a88072620 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880cfa20 con 0x7f7a880b00f0 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7a880cfd20 con 0x7f7a70004800 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45596/0 (socket says 172.21.15.40:45596) 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 -- 172.21.15.40:0/2812199779 learned_addr learned my addr 172.21.15.40:0/2812199779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8d6e3640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b00f0 0x7f7a88144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8cee2640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a70004800 0x7f7a8813f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 -- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a70004800 msgr2=0x7f7a8813f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a70004800 0x7f7a8813f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 -- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b00f0 msgr2=0x7f7a88144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b00f0 0x7f7a88144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 -- 172.21.15.40:0/2812199779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881f9050 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a87fff640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c03f030 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c005480 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c037040 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a881f9330 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.577+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881f98f0 con 0x7f7a88072620 2024-04-26T21:19:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.578+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a880b6ca0 con 0x7f7a88072620 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.579+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a7c042070 con 0x7f7a88072620 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.579+0000 7f7a857fa640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 0x7f7a7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.579+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f7a7c0ab350 con 0x7f7a88072620 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.579+0000 7f7a8cee2640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 0x7f7a7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.580+0000 7f7a8cee2640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 0x7f7a7406edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7a8806f7c0 tx=0x7f7a80009760 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.583+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a7c049070 con 0x7f7a88072620 2024-04-26T21:19:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.808+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a8806a290 con 0x7f7a88072620 2024-04-26T21:19:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.808+0000 7f7a857fa640 1 -- 172.21.15.40:0/2812199779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5186 (secure 0 0 0) 0x7f7a7c0796e0 con 0x7f7a88072620 2024-04-26T21:19:42.810 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:42.810 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:36.505939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:42.810 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2024-04-26T21:19:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.810+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 msgr2=0x7f7a7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.810+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 0x7f7a7406edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7a8806f7c0 tx=0x7f7a80009760 comp rx=0 tx=0).stop 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.810+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 msgr2=0x7f7a880afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.810+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c001120 comp rx=0 tx=0).stop 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 shutdown_connections 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7a7406c8f0 0x7f7a7406edb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a880b00f0 0x7f7a88144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a88072620 0x7f7a880afbb0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 --2- 172.21.15.40:0/2812199779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a70004800 0x7f7a8813f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 >> 172.21.15.40:0/2812199779 conn(0x7f7a880d0d30 msgr2=0x7f7a8815cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 shutdown_connections 2024-04-26T21:19:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.811+0000 7f7a8f146640 1 -- 172.21.15.40:0/2812199779 wait complete. 2024-04-26T21:19:42.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.923+0000 7f2b74c74640 1 Processor -- start 2024-04-26T21:19:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.923+0000 7f2b74c74640 1 -- start start 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b74c74640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b7015f720 0x7f2b700b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b74c74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b700d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b74c74640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b700b2510 con 0x7f2b7015f340 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b74c74640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2b700b27f0 con 0x7f2b7015f720 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b74c74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b700b2ad0 con 0x7f2b700d5610 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b7015f340 0x7f2b700d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60854/0 (socket says 172.21.15.40:60854) 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6e59c640 1 -- 172.21.15.40:0/2678753491 learned_addr learned my addr 172.21.15.40:0/2678753491 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/2678753491 >> v2:172.21.15.184:3300/0 conn(0x7f2b7015f720 0x7f2b700b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b700d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837818671 0 0) 0x7f2b700b2510 con 0x7f2b7015f340 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.924+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b5c003520 con 0x7f2b7015f340 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2621695906 0 0) 0x7f2b5c003520 con 0x7f2b7015f340 2024-04-26T21:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 msgr2=0x7f2b700d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b700d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 >> v2:172.21.15.184:3300/0 conn(0x7f2b7015f720 msgr2=0x7f2b700b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 --2- 172.21.15.40:0/2678753491 >> v2:172.21.15.184:3300/0 conn(0x7f2b7015f720 0x7f2b700b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b700b2db0 con 0x7f2b7015f340 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f2b68001670 con 0x7f2b7015f340 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/2678753491 >> v2:172.21.15.184:3300/0 conn(0x7f2b7015f720 0x7f2b700b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 >> v1:172.21.15.40:6789/0 conn(0x7f2b7015f340 legacy=0x7f2b700d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b5c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5c003f30 0x7f2b5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700b27f0 con 0x7f2b5c004d40 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700b2ad0 con 0x7f2b5c003f30 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5c00ba10 con 0x7f2b700d5610 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6e59c640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b5c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5c003f30 0x7f2b5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 -- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 msgr2=0x7f2b5c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700d5610 0x7f2b5c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 -- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5c003f30 msgr2=0x7f2b5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b5c003f30 0x7f2b5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.925+0000 7f2b6ed9d640 1 -- 172.21.15.40:0/2678753491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c003520 con 0x7f2b5c004d40 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009fc0 tx=0x7f2b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d327763e3af6091 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60009c10 con 0x7f2b5c004d40 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b60036070 con 0x7f2b5c004d40 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b6d59a640 1 -- 172.21.15.40:0/2678753491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60031840 con 0x7f2b5c004d40 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b74c74640 1 -- 172.21.15.40:0/2678753491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b5c004d40 msgr2=0x7f2b5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b74c74640 1 --2- 172.21.15.40:0/2678753491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b5c004d40 0x7f2b5c0051c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009fc0 tx=0x7f2b5c005830 comp rx=0 tx=0).stop 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.926+0000 7f2b6ed9d640 1 -- 172.21.15.40:0/2678753491 reap_dead start 2024-04-26T21:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.927+0000 7f2b74c74640 1 -- 172.21.15.40:0/2678753491 shutdown_connections 2024-04-26T21:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.927+0000 7f2b74c74640 1 -- 172.21.15.40:0/2678753491 >> 172.21.15.40:0/2678753491 conn(0x7f2b7008ec70 msgr2=0x7f2b7009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.927+0000 7f2b74c74640 1 -- 172.21.15.40:0/2678753491 shutdown_connections 2024-04-26T21:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.927+0000 7f2b74c74640 1 -- 172.21.15.40:0/2678753491 wait complete. 2024-04-26T21:19:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.928+0000 7f2b74c74640 1 Processor -- start 2024-04-26T21:19:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.928+0000 7f2b74c74640 1 -- start start 2024-04-26T21:19:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b5c004d40 0x7f2b701565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b701f5460 0x7f2b700a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700d40d0 con 0x7f2b7015d9a0 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700d3f50 con 0x7f2b701f5460 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2b700b35b0 con 0x7f2b5c004d40 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45618/0 (socket says 172.21.15.40:45618) 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 -- 172.21.15.40:0/149573382 learned_addr learned my addr 172.21.15.40:0/149573382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6e59c640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b5c004d40 0x7f2b701565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 -- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b5c004d40 msgr2=0x7f2b701565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b5c004d40 0x7f2b701565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 -- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b701f5460 msgr2=0x7f2b700a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6ed9d640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b701f5460 0x7f2b700a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b701f5460 0x7f2b700a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 -- 172.21.15.40:0/149573382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b700a0050 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b6dd9b640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011ca0 tx=0x7f2b5800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800fd20 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b58004030 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004400 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.929+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b700a0330 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.930+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b700a08f0 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.930+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7006a290 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.931+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b58012040 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.931+0000 7f2b66ffd640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 0x7f2b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.932+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5767+0+0 (secure 0 0 0) 0x7f2b5808c220 con 0x7f2b7015d9a0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.932+0000 7f2b6e59c640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 0x7f2b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.932+0000 7f2b6e59c640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 0x7f2b5006edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2b68007db0 tx=0x7f2b68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:42.936+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b58019070 con 0x7f2b7015d9a0 2024-04-26T21:19:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.124+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b700d3020 con 0x7f2b7015d9a0 2024-04-26T21:19:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.125+0000 7f2b66ffd640 1 -- 172.21.15.40:0/149573382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16781 (secure 0 0 0) 0x7f2b5805a570 con 0x7f2b7015d9a0 2024-04-26T21:19:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:43.127 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":110,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:42.637062+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 msgr2=0x7f2b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 0x7f2b5006edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2b68007db0 tx=0x7f2b68009040 comp rx=0 tx=0).stop 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 msgr2=0x7f2b701f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011ca0 tx=0x7f2b5800f610 comp rx=0 tx=0).stop 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 shutdown_connections 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2b5006c8f0 0x7f2b5006edb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b701f5460 0x7f2b700a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7015d9a0 0x7f2b701f4f20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.127+0000 7f2b74c74640 1 --2- 172.21.15.40:0/149573382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b5c004d40 0x7f2b701565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.128+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 >> 172.21.15.40:0/149573382 conn(0x7f2b7008ec70 msgr2=0x7f2b7006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.128+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 shutdown_connections 2024-04-26T21:19:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.128+0000 7f2b74c74640 1 -- 172.21.15.40:0/149573382 wait complete. 2024-04-26T21:19:43.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:14.110950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:14.791024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:43.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:43.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.243+0000 7f4330d9e640 1 Processor -- start 2024-04-26T21:19:43.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.243+0000 7f4330d9e640 1 -- start start 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4330d9e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f432c0d3c10 0x7f432c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4330d9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f432c0b8370 0x7f432c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4330d9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c0cfb80 con 0x7f432c0d3830 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4330d9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d2ee0 con 0x7f432c0b8370 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4330d9e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d31c0 con 0x7f432c0d3c10 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f432c0d3830 0x7f432c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60858/0 (socket says 172.21.15.40:60858) 2024-04-26T21:19:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432b7fe640 1 -- 172.21.15.40:0/3970157704 learned_addr learned my addr 172.21.15.40:0/3970157704 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432bfff640 1 --2- 172.21.15.40:0/3970157704 >> v2:172.21.15.40:3300/0 conn(0x7f432c0b8370 0x7f432c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f4323fff640 1 --2- 172.21.15.40:0/3970157704 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d3c10 0x7f432c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432affd640 1 -- 172.21.15.40:0/3970157704 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277921288 0 0) 0x7f431c001110 con 0x7f432c0d3830 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432affd640 1 -- 172.21.15.40:0/3970157704 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4318003520 con 0x7f432c0d3830 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432bfff640 1 -- 172.21.15.40:0/3970157704 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d3c10 msgr2=0x7f432c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432bfff640 1 --2- 172.21.15.40:0/3970157704 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d3c10 0x7f432c0b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432bfff640 1 -- 172.21.15.40:0/3970157704 >> v1:172.21.15.40:6789/0 conn(0x7f432c0d3830 legacy=0x7f432c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.244+0000 7f432bfff640 1 -- 172.21.15.40:0/3970157704 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0cfe00 con 0x7f432c0b8370 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.245+0000 7f432bfff640 1 --2- 172.21.15.40:0/3970157704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0b8370 0x7f432c0d0600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f432c06f510 tx=0x7f432402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235c963819a2bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.245+0000 7f432affd640 1 -- 172.21.15.40:0/3970157704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324007740 con 0x7f432c0b8370 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.245+0000 7f432affd640 1 -- 172.21.15.40:0/3970157704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43240078e0 con 0x7f432c0b8370 2024-04-26T21:19:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.245+0000 7f432affd640 1 -- 172.21.15.40:0/3970157704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324007cb0 con 0x7f432c0b8370 2024-04-26T21:19:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 -- 172.21.15.40:0/3970157704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0b8370 msgr2=0x7f432c0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3970157704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0b8370 0x7f432c0d0600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f432c06f510 tx=0x7f432402a680 comp rx=0 tx=0).stop 2024-04-26T21:19:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 -- 172.21.15.40:0/3970157704 shutdown_connections 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3970157704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0b8370 0x7f432c0d0600 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3970157704 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d3c10 0x7f432c0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 -- 172.21.15.40:0/3970157704 >> 172.21.15.40:0/3970157704 conn(0x7f432c0d4fe0 msgr2=0x7f432c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 -- 172.21.15.40:0/3970157704 shutdown_connections 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.246+0000 7f4330d9e640 1 -- 172.21.15.40:0/3970157704 wait complete. 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.247+0000 7f4330d9e640 1 Processor -- start 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.247+0000 7f4330d9e640 1 -- start start 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0b8370 0x7f432c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c0d3830 0x7f432c0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d2bb0 con 0x7f432c0d3c10 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d5d90 con 0x7f432c0d3830 2024-04-26T21:19:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4330d9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0d0b40 con 0x7f432c0b8370 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45636/0 (socket says 172.21.15.40:45636) 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 -- 172.21.15.40:0/3559483799 learned_addr learned my addr 172.21.15.40:0/3559483799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f4323fff640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c0d3830 0x7f432c0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432b7fe640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0b8370 0x7f432c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 -- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0b8370 msgr2=0x7f432c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0b8370 0x7f432c15b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 -- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c0d3830 msgr2=0x7f432c0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c0d3830 0x7f432c0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 -- 172.21.15.40:0/3559483799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0cdc30 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.248+0000 7f432bfff640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f432402ac70 tx=0x7f4324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.249+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324040070 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.249+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324031bc0 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.249+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c0cdf10 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.249+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324009450 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.249+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c0ca180 con 0x7f432c0d3c10 2024-04-26T21:19:43.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.250+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c0ce300 con 0x7f432c0d3c10 2024-04-26T21:19:43.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.250+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4324039040 con 0x7f432c0d3c10 2024-04-26T21:19:43.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.250+0000 7f4328ff9640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.251+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5767+0+0 (secure 0 0 0) 0x7f43240ad310 con 0x7f432c0d3c10 2024-04-26T21:19:43.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.251+0000 7f432b7fe640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.251+0000 7f432b7fe640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f431c0012b0 tx=0x7f431c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.255+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4324043360 con 0x7f432c0d3c10 2024-04-26T21:19:43.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.442+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f432c050730 con 0x7f432c0d3c10 2024-04-26T21:19:43.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.443+0000 7f4328ff9640 1 -- 172.21.15.40:0/3559483799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f432407b8b0 con 0x7f432c0d3c10 2024-04-26T21:19:43.444 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 msgr2=0x7f430c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f431c0012b0 tx=0x7f431c009040 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 msgr2=0x7f432c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f432402ac70 tx=0x7f4324004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 shutdown_connections 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3c10 0x7f432c156010 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c0d3830 0x7f432c0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 --2- 172.21.15.40:0/3559483799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0b8370 0x7f432c15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 >> 172.21.15.40:0/3559483799 conn(0x7f432c0d4fe0 msgr2=0x7f432c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 shutdown_connections 2024-04-26T21:19:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.445+0000 7f4330d9e640 1 -- 172.21.15.40:0/3559483799 wait complete. 2024-04-26T21:19:43.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:19:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.560+0000 7ff494ad1640 1 Processor -- start 2024-04-26T21:19:43.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.560+0000 7ff494ad1640 1 -- start start 2024-04-26T21:19:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff494ad1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4900d3340 0x7ff4900d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff494ad1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900b6aa0 con 0x7ff49015f5b0 2024-04-26T21:19:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff494ad1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900d3eb0 con 0x7ff49015f990 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff494ad1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900d4190 con 0x7ff4900d3340 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff49015f5b0 0x7ff4900b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60864/0 (socket says 172.21.15.40:60864) 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48e59c640 1 -- 172.21.15.40:0/1007984946 learned_addr learned my addr 172.21.15.40:0/1007984946 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48ed9d640 1 --2- 172.21.15.40:0/1007984946 >> v2:172.21.15.184:3300/0 conn(0x7ff4900d3340 0x7ff4900d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48dd9b640 1 -- 172.21.15.40:0/1007984946 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1703615814 0 0) 0x7ff4900b6aa0 con 0x7ff49015f5b0 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48dd9b640 1 -- 172.21.15.40:0/1007984946 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff47c003520 con 0x7ff49015f5b0 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48ed9d640 1 -- 172.21.15.40:0/1007984946 >> v1:172.21.15.184:6789/0 conn(0x7ff49015f990 legacy=0x7ff4900d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48ed9d640 1 -- 172.21.15.40:0/1007984946 >> v1:172.21.15.40:6789/0 conn(0x7ff49015f5b0 legacy=0x7ff4900b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.561+0000 7ff48ed9d640 1 -- 172.21.15.40:0/1007984946 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900d0200 con 0x7ff4900d3340 2024-04-26T21:19:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.562+0000 7ff48ed9d640 1 --2- 172.21.15.40:0/1007984946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4900d3340 0x7ff4900d3790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff48802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f96581830c29d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.562+0000 7ff48dd9b640 1 -- 172.21.15.40:0/1007984946 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488007740 con 0x7ff4900d3340 2024-04-26T21:19:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.562+0000 7ff48dd9b640 1 -- 172.21.15.40:0/1007984946 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4880078e0 con 0x7ff4900d3340 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.562+0000 7ff48dd9b640 1 -- 172.21.15.40:0/1007984946 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488007c90 con 0x7ff4900d3340 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 -- 172.21.15.40:0/1007984946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4900d3340 msgr2=0x7ff4900d3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 --2- 172.21.15.40:0/1007984946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4900d3340 0x7ff4900d3790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff48802a660 comp rx=0 tx=0).stop 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 -- 172.21.15.40:0/1007984946 shutdown_connections 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 --2- 172.21.15.40:0/1007984946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4900d3340 0x7ff4900d3790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 -- 172.21.15.40:0/1007984946 >> 172.21.15.40:0/1007984946 conn(0x7ff49008ec40 msgr2=0x7ff49009e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 -- 172.21.15.40:0/1007984946 shutdown_connections 2024-04-26T21:19:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.563+0000 7ff494ad1640 1 -- 172.21.15.40:0/1007984946 wait complete. 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.564+0000 7ff494ad1640 1 Processor -- start 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.564+0000 7ff494ad1640 1 -- start start 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900d3340 0x7ff49014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff49015f5b0 0x7ff4900cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900b5900 con 0x7ff49015f990 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900b5780 con 0x7ff49015f5b0 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff494ad1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900d1110 con 0x7ff4900d3340 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45650/0 (socket says 172.21.15.40:45650) 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 -- 172.21.15.40:0/2388101812 learned_addr learned my addr 172.21.15.40:0/2388101812 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff487fff640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff49015f5b0 0x7ff4900cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48e59c640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900d3340 0x7ff49014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 -- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900d3340 msgr2=0x7ff49014f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900d3340 0x7ff49014f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 -- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff49015f5b0 msgr2=0x7ff4900cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.565+0000 7ff48ed9d640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff49015f5b0 0x7ff4900cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff48ed9d640 1 -- 172.21.15.40:0/2388101812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900cb9e0 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff48ed9d640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff488004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488040070 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488031c40 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488009450 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900b9060 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.566+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49014fb00 con 0x7ff49015f990 2024-04-26T21:19:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.567+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49006a290 con 0x7ff49015f990 2024-04-26T21:19:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.567+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff488039040 con 0x7ff49015f990 2024-04-26T21:19:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.568+0000 7ff486ffd640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.568+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5767+0+0 (secure 0 0 0) 0x7ff4880accb0 con 0x7ff49015f990 2024-04-26T21:19:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.568+0000 7ff48e59c640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.568+0000 7ff48e59c640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 0x7ff47006edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff4800011e0 tx=0x7ff480007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.572+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff48807aa50 con 0x7ff49015f990 2024-04-26T21:19:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.761+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4900d0b60 con 0x7ff49015f990 2024-04-26T21:19:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.762+0000 7ff486ffd640 1 -- 172.21.15.40:0/2388101812 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) ==== 104+0+11 (secure 0 0 0) 0x7ff48807ed50 con 0x7ff49015f990 2024-04-26T21:19:43.763 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:19:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 msgr2=0x7ff47006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 0x7ff47006edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff4800011e0 tx=0x7ff480007d30 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 msgr2=0x7ff4900c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff488004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 shutdown_connections 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff49015f990 0x7ff4900c91d0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff49015f5b0 0x7ff4900cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 --2- 172.21.15.40:0/2388101812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4900d3340 0x7ff49014f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 >> 172.21.15.40:0/2388101812 conn(0x7ff49008ec40 msgr2=0x7ff49006dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.763+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 shutdown_connections 2024-04-26T21:19:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.764+0000 7ff494ad1640 1 -- 172.21.15.40:0/2388101812 wait complete. 2024-04-26T21:19:43.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:19:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.879+0000 7ff0a2ac8640 1 Processor -- start 2024-04-26T21:19:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.879+0000 7ff0a2ac8640 1 -- start start 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a2ac8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff09c1631f0 0x7ff09c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a2ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a2ac8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c09c790 con 0x7ff09c0b3300 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a2ac8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c0d15f0 con 0x7ff09c1631f0 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a2ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c163ec0 con 0x7ff09c0d1170 2024-04-26T21:19:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a1065640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff09c0b3300 0x7ff09c0b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:45124/0 (socket says 172.21.15.40:45124) 2024-04-26T21:19:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff09bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a1065640 1 -- 172.21.15.40:0/704818129 learned_addr learned my addr 172.21.15.40:0/704818129 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.880+0000 7ff0a0864640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff09c1631f0 0x7ff09c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.881+0000 7ff0a0864640 1 -- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 msgr2=0x7ff09c1635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.881+0000 7ff0a0864640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.881+0000 7ff0a0864640 1 -- 172.21.15.40:0/704818129 >> v1:172.21.15.184:6789/0 conn(0x7ff09c0b3300 legacy=0x7ff09c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.881+0000 7ff0a0864640 1 -- 172.21.15.40:0/704818129 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0d1a40 con 0x7ff09c1631f0 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.881+0000 7ff09bfff640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a0864640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c1631f0 0x7ff09c0b2000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff094009d90 tx=0x7ff09402a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9592e7f1692a0547 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff09b7fe640 1 -- 172.21.15.40:0/704818129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094007740 con 0x7ff09c1631f0 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff09b7fe640 1 -- 172.21.15.40:0/704818129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0940078e0 con 0x7ff09c1631f0 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff09b7fe640 1 -- 172.21.15.40:0/704818129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094007cb0 con 0x7ff09c1631f0 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c1631f0 msgr2=0x7ff09c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c1631f0 0x7ff09c0b2000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff094009d90 tx=0x7ff09402a4d0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/704818129 shutdown_connections 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/704818129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c1631f0 0x7ff09c0b2000 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/704818129 >> 172.21.15.40:0/704818129 conn(0x7ff09c09ba40 msgr2=0x7ff09c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/704818129 shutdown_connections 2024-04-26T21:19:43.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.882+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/704818129 wait complete. 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.883+0000 7ff0a2ac8640 1 Processor -- start 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 -- start start 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c0b3300 0x7ff09c160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d0230 con 0x7ff09c1631f0 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d2000 con 0x7ff09c0b3300 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.884+0000 7ff0a2ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c161080 con 0x7ff09c0d1170 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45652/0 (socket says 172.21.15.40:45652) 2024-04-26T21:19:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 -- 172.21.15.40:0/3587239965 learned_addr learned my addr 172.21.15.40:0/3587239965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a0864640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c0b3300 0x7ff09c160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff09bfff640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 -- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 msgr2=0x7ff09c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 -- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c0b3300 msgr2=0x7ff09c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c0b3300 0x7ff09c160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 -- 172.21.15.40:0/3587239965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c15faa0 con 0x7ff09c1631f0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a1065640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c780 tx=0x7ff08c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c018070 con 0x7ff09c1631f0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c0072d0 con 0x7ff09c1631f0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c15fd80 con 0x7ff09c1631f0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0047e0 con 0x7ff09c1631f0 2024-04-26T21:19:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.885+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c160340 con 0x7ff09c1631f0 2024-04-26T21:19:43.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.886+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c06a290 con 0x7ff09c1631f0 2024-04-26T21:19:43.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.887+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff08c01b070 con 0x7ff09c1631f0 2024-04-26T21:19:43.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.887+0000 7ff0997fa640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:43.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.887+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7ff08c08afa0 con 0x7ff09c1631f0 2024-04-26T21:19:43.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.888+0000 7ff0a0864640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:43.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.888+0000 7ff0a0864640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff094034000 tx=0x7ff09402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:43.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:43.892+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff08c058cb0 con 0x7ff09c1631f0 2024-04-26T21:19:44.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.079+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff09c0b8f10 con 0x7ff09c1631f0 2024-04-26T21:19:44.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.080+0000 7ff0997fa640 1 -- 172.21.15.40:0/3587239965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) ==== 100+0+11 (secure 0 0 0) 0x7ff08c05cfb0 con 0x7ff09c1631f0 2024-04-26T21:19:44.081 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:19:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 msgr2=0x7ff08806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff094034000 tx=0x7ff09402f000 comp rx=0 tx=0).stop 2024-04-26T21:19:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 msgr2=0x7ff09c162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c780 tx=0x7ff08c00cc40 comp rx=0 tx=0).stop 2024-04-26T21:19:44.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 shutdown_connections 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff09c1631f0 0x7ff09c162280 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c0d1170 0x7ff09c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 --2- 172.21.15.40:0/3587239965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c0b3300 0x7ff09c160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 >> 172.21.15.40:0/3587239965 conn(0x7ff09c09ba40 msgr2=0x7ff09c071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 shutdown_connections 2024-04-26T21:19:44.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.082+0000 7ff0a2ac8640 1 -- 172.21.15.40:0/3587239965 wait complete. 2024-04-26T21:19:44.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.193+0000 7f912ab2b640 1 Processor -- start 2024-04-26T21:19:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f912ab2b640 1 -- start start 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f912ab2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9124160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f912ab2b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91240b4530 con 0x7f91240b3de0 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f912ab2b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91240b4810 con 0x7f9124145980 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f912ab2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91240b5140 con 0x7f91240b61a0 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.194+0000 7f91290c8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91240b3de0 0x7f91240720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60880/0 (socket says 172.21.15.40:60880) 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91290c8640 1 -- 172.21.15.40:0/676409287 learned_addr learned my addr 172.21.15.40:0/676409287 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f9123fff640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9124160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613307458 0 0) 0x7f91240b4530 con 0x7f91240b3de0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9114003520 con 0x7f91240b3de0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3811267503 0 0) 0x7f9114003520 con 0x7f91240b3de0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 msgr2=0x7f9124160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9124160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 >> v1:172.21.15.184:6789/0 conn(0x7f9124145980 legacy=0x7f91240b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91240b4af0 con 0x7f91240b3de0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f911c0019f0 con 0x7f91240b3de0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 >> v1:172.21.15.40:6789/0 conn(0x7f91240b3de0 legacy=0x7f91240720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140039e0 0x7f9114003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9114004800 0x7f9114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9114005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91240b5140 con 0x7f91140039e0 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f911400baf0 con 0x7f9114004800 2024-04-26T21:19:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f911400bcf0 con 0x7f91240b61a0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91288c7640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140039e0 0x7f9114003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f9123fff640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9114004800 0x7f9114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91290c8640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9114005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.195+0000 7f91288c7640 1 -- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 msgr2=0x7f9114005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91288c7640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91240b61a0 0x7f9114005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91288c7640 1 -- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9114004800 msgr2=0x7f9114004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91288c7640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9114004800 0x7f9114004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91288c7640 1 -- 172.21.15.40:0/676409287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9114003520 con 0x7f91140039e0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91288c7640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140039e0 0x7f9114003e00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9118001750 tx=0x7f9114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4264205e6db91d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118001370 con 0x7f91140039e0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91180052d0 con 0x7f91140039e0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.196+0000 7f91237fe640 1 -- 172.21.15.40:0/676409287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911802abe0 con 0x7f91140039e0 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 -- 172.21.15.40:0/676409287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140039e0 msgr2=0x7f9114003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 --2- 172.21.15.40:0/676409287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91140039e0 0x7f9114003e00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9118001750 tx=0x7f9114005830 comp rx=0 tx=0).stop 2024-04-26T21:19:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f91290c8640 1 -- 172.21.15.40:0/676409287 reap_dead start 2024-04-26T21:19:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 -- 172.21.15.40:0/676409287 shutdown_connections 2024-04-26T21:19:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 -- 172.21.15.40:0/676409287 >> 172.21.15.40:0/676409287 conn(0x7f91240d0d30 msgr2=0x7f91240d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 -- 172.21.15.40:0/676409287 shutdown_connections 2024-04-26T21:19:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.197+0000 7f912ab2b640 1 -- 172.21.15.40:0/676409287 wait complete. 2024-04-26T21:19:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.198+0000 7f912ab2b640 1 Processor -- start 2024-04-26T21:19:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.198+0000 7f912ab2b640 1 -- start start 2024-04-26T21:19:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 0x7f91241394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 0x7f912415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9124160630 con 0x7f912413be30 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91240b4240 con 0x7f912415d5b0 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f912ab2b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91241607b0 con 0x7f91140039e0 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45674/0 (socket says 172.21.15.40:45674) 2024-04-26T21:19:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 -- 172.21.15.40:0/3041864442 learned_addr learned my addr 172.21.15.40:0/3041864442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f9123fff640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 0x7f912415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91288c7640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 0x7f91241394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 -- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 msgr2=0x7f91241394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 0x7f91241394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 -- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 msgr2=0x7f912415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 0x7f912415da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91290c8640 1 -- 172.21.15.40:0/3041864442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124150260 con 0x7f912413be30 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f9123fff640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 0x7f912415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.199+0000 7f91288c7640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 0x7f91241394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f91290c8640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f9114004780 tx=0x7f911c00e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c00ec30 con 0x7f912413be30 2024-04-26T21:19:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911c00edf0 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912413d170 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c013420 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.200+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912413ce90 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.201+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912406a290 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.201+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f911c01a020 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.202+0000 7f91217fa640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 0x7f910806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.202+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f911c08b660 con 0x7f912413be30 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.202+0000 7f91288c7640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 0x7f910806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.202+0000 7f91288c7640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 0x7f910806edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9118032000 tx=0x7f911802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.206+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f911c021020 con 0x7f912413be30 2024-04-26T21:19:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.431+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f912415fc90 con 0x7f912413be30 2024-04-26T21:19:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.432+0000 7f91217fa640 1 -- 172.21.15.40:0/3041864442 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5186 (secure 0 0 0) 0x7f911c059a40 con 0x7f912413be30 2024-04-26T21:19:44.433 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:44.433 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:36.505939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 msgr2=0x7f910806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 0x7f910806edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9118032000 tx=0x7f911802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 msgr2=0x7f912413c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f9114004780 tx=0x7f911c00e160 comp rx=0 tx=0).stop 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 shutdown_connections 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f910806c8f0 0x7f910806edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912413be30 0x7f912413c2b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912415d5b0 0x7f912415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 --2- 172.21.15.40:0/3041864442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91140039e0 0x7f91241394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 >> 172.21.15.40:0/3041864442 conn(0x7f91240d0d30 msgr2=0x7f91240d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 shutdown_connections 2024-04-26T21:19:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.434+0000 7f912ab2b640 1 -- 172.21.15.40:0/3041864442 wait complete. 2024-04-26T21:19:44.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.546+0000 7f2dbaf46640 1 Processor -- start 2024-04-26T21:19:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.546+0000 7f2dbaf46640 1 -- start start 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2db415f1e0 0x7f2db406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2db40b44e0 0x7f2db40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db406d2b0 0x7f2db406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2db406eeb0 con 0x7f2db40b44e0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2db40b4a10 con 0x7f2db415f1e0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2dbaf46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40b4cf0 con 0x7f2db406d2b0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2db40b44e0 0x7f2db40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2db40b44e0 0x7f2db40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45686/0 (socket says 172.21.15.40:45686) 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 -- 172.21.15.40:0/2955613839 learned_addr learned my addr 172.21.15.40:0/2955613839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db8ce2640 1 --2- 172.21.15.40:0/2955613839 >> v2:172.21.15.184:3300/0 conn(0x7f2db415f1e0 0x7f2db406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db94e3640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db406d2b0 0x7f2db406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 -- 172.21.15.40:0/2955613839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db406d2b0 msgr2=0x7f2db406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db406d2b0 0x7f2db406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 -- 172.21.15.40:0/2955613839 >> v2:172.21.15.184:3300/0 conn(0x7f2db415f1e0 msgr2=0x7f2db406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 --2- 172.21.15.40:0/2955613839 >> v2:172.21.15.184:3300/0 conn(0x7f2db415f1e0 0x7f2db406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.547+0000 7f2db3fff640 1 -- 172.21.15.40:0/2955613839 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db40b4fd0 con 0x7f2db40b44e0 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.548+0000 7f2db3fff640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db40ae8b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2db406e610 tx=0x7f2da002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5e050defc971e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.548+0000 7f2db37fe640 1 -- 172.21.15.40:0/2955613839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0037070 con 0x7f2db40b44e0 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.548+0000 7f2db37fe640 1 -- 172.21.15.40:0/2955613839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da0032440 con 0x7f2db40b44e0 2024-04-26T21:19:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.548+0000 7f2db37fe640 1 -- 172.21.15.40:0/2955613839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0032810 con 0x7f2db40b44e0 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 -- 172.21.15.40:0/2955613839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 msgr2=0x7f2db40ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db40ae8b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2db406e610 tx=0x7f2da002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 -- 172.21.15.40:0/2955613839 shutdown_connections 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db406d2b0 0x7f2db406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/2955613839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db40ae8b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/2955613839 >> v2:172.21.15.184:3300/0 conn(0x7f2db415f1e0 0x7f2db406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 -- 172.21.15.40:0/2955613839 >> 172.21.15.40:0/2955613839 conn(0x7f2db40728a0 msgr2=0x7f2db4072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 -- 172.21.15.40:0/2955613839 shutdown_connections 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.549+0000 7f2dbaf46640 1 -- 172.21.15.40:0/2955613839 wait complete. 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.550+0000 7f2dbaf46640 1 Processor -- start 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.550+0000 7f2dbaf46640 1 -- start start 2024-04-26T21:19:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2db406d2b0 0x7f2db41588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db415f1e0 0x7f2db41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40aedf0 con 0x7f2db40b44e0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40b5590 con 0x7f2db406d2b0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2dbaf46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2db415f5c0 con 0x7f2db415f1e0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45696/0 (socket says 172.21.15.40:45696) 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 -- 172.21.15.40:0/3271089088 learned_addr learned my addr 172.21.15.40:0/3271089088 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db8ce2640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2db406d2b0 0x7f2db41588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db94e3640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db415f1e0 0x7f2db41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 -- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db415f1e0 msgr2=0x7f2db41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db415f1e0 0x7f2db41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 -- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2db406d2b0 msgr2=0x7f2db41588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2db406d2b0 0x7f2db41588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 -- 172.21.15.40:0/3271089088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4158e30 con 0x7f2db40b44e0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db3fff640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2da002a220 tx=0x7f2da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.551+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0041020 con 0x7f2db40b44e0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.552+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da00306c0 con 0x7f2db40b44e0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.552+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db415d620 con 0x7f2db40b44e0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.552+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0037040 con 0x7f2db40b44e0 2024-04-26T21:19:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.552+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db415dbe0 con 0x7f2db40b44e0 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.553+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db406a290 con 0x7f2db40b44e0 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.553+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2da0044020 con 0x7f2db40b44e0 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.554+0000 7f2db17fa640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.554+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f2da00ac280 con 0x7f2db40b44e0 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.554+0000 7f2db8ce2640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.554+0000 7f2db8ce2640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2da8007920 tx=0x7f2da8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.558+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2da004b030 con 0x7f2db40b44e0 2024-04-26T21:19:44.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.746+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2db4070b40 con 0x7f2db40b44e0 2024-04-26T21:19:44.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.747+0000 7f2db17fa640 1 -- 172.21.15.40:0/3271089088 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16781 (secure 0 0 0) 0x7f2da007a020 con 0x7f2db40b44e0 2024-04-26T21:19:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:44.749 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":111,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:43.638451+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:44.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 msgr2=0x7f2d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2da8007920 tx=0x7f2da8008040 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 msgr2=0x7f2db4142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2da002a220 tx=0x7f2da00046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 shutdown_connections 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2db415f1e0 0x7f2db41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db40b44e0 0x7f2db4142da0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 --2- 172.21.15.40:0/3271089088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2db406d2b0 0x7f2db41588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 >> 172.21.15.40:0/3271089088 conn(0x7f2db40728a0 msgr2=0x7f2db406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 shutdown_connections 2024-04-26T21:19:44.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.749+0000 7f2dbaf46640 1 -- 172.21.15.40:0/3271089088 wait complete. 2024-04-26T21:19:44.761 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:19:44.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:44.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.861+0000 7ff2bb646640 1 Processor -- start 2024-04-26T21:19:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.861+0000 7ff2bb646640 1 -- start start 2024-04-26T21:19:44.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2bb646640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2b406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2bb646640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b40d1740 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2bb646640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b40d1a20 con 0x7ff2b406e2c0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2bb646640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b406e750 con 0x7ff2b40d11b0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2b93e2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2b40cf2e0 0x7ff2b406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60896/0 (socket says 172.21.15.40:60896) 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2b93e2640 1 -- 172.21.15.40:0/1012810781 learned_addr learned my addr 172.21.15.40:0/1012810781 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244157328 0 0) 0x7ff2b40d1740 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2b406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2a0003520 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3380456120 0 0) 0x7ff2a0003520 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 msgr2=0x7ff2b406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2b406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 >> v1:172.21.15.184:6789/0 conn(0x7ff2b406e2c0 legacy=0x7ff2b415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b40d1d00 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7ff2b0001670 con 0x7ff2b40cf2e0 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 >> v1:172.21.15.40:6789/0 conn(0x7ff2b40cf2e0 legacy=0x7ff2b406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00039e0 0x7ff2a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a0004390 0x7ff2a0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2a00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b406e750 con 0x7ff2a0004390 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a000b9f0 con 0x7ff2a00039e0 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.862+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a000bbf0 con 0x7ff2b40d11b0 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a0004390 0x7ff2a0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00039e0 0x7ff2a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b9be3640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2a00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 -- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 msgr2=0x7ff2a00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b40d11b0 0x7ff2a00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 -- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00039e0 msgr2=0x7ff2a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00039e0 0x7ff2a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 -- 172.21.15.40:0/1012810781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a0003520 con 0x7ff2a0004390 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00039e0 0x7ff2a0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.863+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a0004390 0x7ff2a0004860 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009ac0 tx=0x7ff2a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24267916a3162c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4036070 con 0x7ff2a0004390 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a4031400 con 0x7ff2a0004390 2024-04-26T21:19:44.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2abfff640 1 -- 172.21.15.40:0/1012810781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4009690 con 0x7ff2a0004390 2024-04-26T21:19:44.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 -- 172.21.15.40:0/1012810781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a0004390 msgr2=0x7ff2a0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 --2- 172.21.15.40:0/1012810781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a0004390 0x7ff2a0004860 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009ac0 tx=0x7ff2a0005830 comp rx=0 tx=0).stop 2024-04-26T21:19:44.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2b9be3640 1 -- 172.21.15.40:0/1012810781 reap_dead start 2024-04-26T21:19:44.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 -- 172.21.15.40:0/1012810781 shutdown_connections 2024-04-26T21:19:44.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 -- 172.21.15.40:0/1012810781 >> 172.21.15.40:0/1012810781 conn(0x7ff2b40b4a60 msgr2=0x7ff2b40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 -- 172.21.15.40:0/1012810781 shutdown_connections 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.864+0000 7ff2bb646640 1 -- 172.21.15.40:0/1012810781 wait complete. 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.865+0000 7ff2bb646640 1 Processor -- start 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 -- start start 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 0x7ff2b4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b4150060 0x7ff2b4151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b40cf6c0 con 0x7ff2b40b5b20 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b40d22c0 con 0x7ff2a0004390 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2bb646640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b40cf840 con 0x7ff2b4150060 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2b9be3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2b9be3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45720/0 (socket says 172.21.15.40:45720) 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.866+0000 7ff2b9be3640 1 -- 172.21.15.40:0/183096328 learned_addr learned my addr 172.21.15.40:0/183096328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 0x7ff2b4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 -- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b4150060 msgr2=0x7ff2b4151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b8be1640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b4150060 0x7ff2b4151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b4150060 0x7ff2b4151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 -- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 msgr2=0x7ff2b4152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 0x7ff2b4152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 -- 172.21.15.40:0/183096328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41507d0 con 0x7ff2b40b5b20 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 0x7ff2b4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2b9be3640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4073840 tx=0x7ff2ac00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac003970 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac018070 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac003d50 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b4150ab0 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.867+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b4151070 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.868+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b406a290 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.869+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2ac0050d0 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.869+0000 7ff2a9ffb640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.869+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7ff2ac08b8a0 con 0x7ff2b40b5b20 2024-04-26T21:19:44.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.869+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.870+0000 7ff2b93e2640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 0x7ff29806ede0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00017e0 tx=0x7ff2b0001c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:44.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:44.873+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2ac059c90 con 0x7ff2b40b5b20 2024-04-26T21:19:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.095+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2b40b4770 con 0x7ff2b40b5b20 2024-04-26T21:19:45.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.095+0000 7ff2a9ffb640 1 -- 172.21.15.40:0/183096328 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5187 (secure 0 0 0) 0x7ff2ac08b070 con 0x7ff2b40b5b20 2024-04-26T21:19:45.096 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:45.096 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:44.507950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":10,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:45.097 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 67 2024-04-26T21:19:45.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 msgr2=0x7ff29806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 0x7ff29806ede0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00017e0 tx=0x7ff2b0001c60 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 msgr2=0x7ff2b415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4073840 tx=0x7ff2ac00eb90 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 shutdown_connections 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b40b5b20 0x7ff2b415fa40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b4150060 0x7ff2b4151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 --2- 172.21.15.40:0/183096328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a0004390 0x7ff2b4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 >> 172.21.15.40:0/183096328 conn(0x7ff2b40b4a60 msgr2=0x7ff2b40b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 shutdown_connections 2024-04-26T21:19:45.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.097+0000 7ff2bb646640 1 -- 172.21.15.40:0/183096328 wait complete. 2024-04-26T21:19:45.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.209+0000 7fda3b2da640 1 Processor -- start 2024-04-26T21:19:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.209+0000 7fda3b2da640 1 -- start start 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda3b2da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda34148a30 0x7fda34073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda3b2da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda340b3af0 0x7fda340b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda3b2da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda340cfa80 con 0x7fda34160640 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda3b2da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fda340cfd60 con 0x7fda34148a30 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda3b2da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fda340d0040 con 0x7fda340b3af0 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda39076640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda34148a30 0x7fda34073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda38875640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda34160640 0x7fda341636a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60912/0 (socket says 172.21.15.40:60912) 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.210+0000 7fda38875640 1 -- 172.21.15.40:0/334581689 learned_addr learned my addr 172.21.15.40:0/334581689 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda39877640 1 --2- 172.21.15.40:0/334581689 >> v2:172.21.15.184:3300/0 conn(0x7fda340b3af0 0x7fda340b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda39076640 1 -- 172.21.15.40:0/334581689 >> v2:172.21.15.184:3300/0 conn(0x7fda340b3af0 msgr2=0x7fda340b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda39076640 1 --2- 172.21.15.40:0/334581689 >> v2:172.21.15.184:3300/0 conn(0x7fda340b3af0 0x7fda340b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda39076640 1 -- 172.21.15.40:0/334581689 >> v1:172.21.15.40:6789/0 conn(0x7fda34160640 legacy=0x7fda341636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda39076640 1 -- 172.21.15.40:0/334581689 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda34162ea0 con 0x7fda34148a30 2024-04-26T21:19:45.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.211+0000 7fda2bfff640 1 -- 172.21.15.40:0/334581689 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930957203 0 0) 0x7fda340cfa80 con 0x7fda34160640 2024-04-26T21:19:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda39076640 1 --2- 172.21.15.40:0/334581689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34148a30 0x7fda34073ed0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fda3415f8b0 tx=0x7fda3002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46e8f739b6047be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda2bfff640 1 -- 172.21.15.40:0/334581689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30037070 con 0x7fda34148a30 2024-04-26T21:19:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda2bfff640 1 -- 172.21.15.40:0/334581689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30032440 con 0x7fda34148a30 2024-04-26T21:19:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda2bfff640 1 -- 172.21.15.40:0/334581689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30032810 con 0x7fda34148a30 2024-04-26T21:19:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 -- 172.21.15.40:0/334581689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34148a30 msgr2=0x7fda34073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 --2- 172.21.15.40:0/334581689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34148a30 0x7fda34073ed0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fda3415f8b0 tx=0x7fda3002a130 comp rx=0 tx=0).stop 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 -- 172.21.15.40:0/334581689 shutdown_connections 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 --2- 172.21.15.40:0/334581689 >> v2:172.21.15.184:3300/0 conn(0x7fda340b3af0 0x7fda340b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 --2- 172.21.15.40:0/334581689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34148a30 0x7fda34073ed0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 -- 172.21.15.40:0/334581689 >> 172.21.15.40:0/334581689 conn(0x7fda340ce8b0 msgr2=0x7fda340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 -- 172.21.15.40:0/334581689 shutdown_connections 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.212+0000 7fda3b2da640 1 -- 172.21.15.40:0/334581689 wait complete. 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.213+0000 7fda3b2da640 1 Processor -- start 2024-04-26T21:19:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 -- start start 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 0x7fda341f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda34148a30 0x7fda340d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda34074410 con 0x7fda34160640 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda34163db0 con 0x7fda340b3af0 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda3b2da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda34074590 con 0x7fda34148a30 2024-04-26T21:19:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda39877640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda39877640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45734/0 (socket says 172.21.15.40:45734) 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda39877640 1 -- 172.21.15.40:0/251160043 learned_addr learned my addr 172.21.15.40:0/251160043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.214+0000 7fda39076640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 0x7fda341f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda38875640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda34148a30 0x7fda340d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 -- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda34148a30 msgr2=0x7fda340d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda34148a30 0x7fda340d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 -- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 msgr2=0x7fda341f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 0x7fda341f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 -- 172.21.15.40:0/251160043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda340aea50 con 0x7fda34160640 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39076640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 0x7fda341f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda39877640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007e60 tx=0x7fda2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c014420 con 0x7fda34160640 2024-04-26T21:19:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c019070 con 0x7fda34160640 2024-04-26T21:19:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c014820 con 0x7fda34160640 2024-04-26T21:19:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda340aed30 con 0x7fda34160640 2024-04-26T21:19:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.215+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda340af2f0 con 0x7fda34160640 2024-04-26T21:19:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.216+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3406a290 con 0x7fda34160640 2024-04-26T21:19:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.217+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda2c0149c0 con 0x7fda34160640 2024-04-26T21:19:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.217+0000 7fda29ffb640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.217+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fda2c08c410 con 0x7fda34160640 2024-04-26T21:19:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.217+0000 7fda39076640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.218+0000 7fda39076640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 0x7fda1806ede0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda30034000 tx=0x7fda3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.221+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda2c05a800 con 0x7fda34160640 2024-04-26T21:19:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.445+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda340739f0 con 0x7fda34160640 2024-04-26T21:19:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.445+0000 7fda29ffb640 1 -- 172.21.15.40:0/251160043 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5187 (secure 0 0 0) 0x7fda2c05a100 con 0x7fda34160640 2024-04-26T21:19:45.446 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:15.844900+0000","modified":"2024-04-26T21:19:44.507950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":10,"addr":"172.21.15.184:6836/798777741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":798777741},{"type":"v1","addr":"172.21.15.184:6836","nonce":798777741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7962,"qdb_cluster":[7962]},"id":8}]} 2024-04-26T21:19:45.447 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 67 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 msgr2=0x7fda1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 0x7fda1806ede0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda30034000 tx=0x7fda3002e000 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 msgr2=0x7fda341f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007e60 tx=0x7fda2c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 shutdown_connections 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34160640 0x7fda341f4d90 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fda34148a30 0x7fda340d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 --2- 172.21.15.40:0/251160043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda340b3af0 0x7fda341f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.447+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 >> 172.21.15.40:0/251160043 conn(0x7fda340ce8b0 msgr2=0x7fda341636a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.448+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 shutdown_connections 2024-04-26T21:19:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.448+0000 7fda3b2da640 1 -- 172.21.15.40:0/251160043 wait complete. 2024-04-26T21:19:45.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.559+0000 7fdc18a17640 1 Processor -- start 2024-04-26T21:19:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.559+0000 7fdc18a17640 1 -- start start 2024-04-26T21:19:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc18a17640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 0x7fdc1406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc18a17640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc1406d2b0 0x7fdc1406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc18a17640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc140b4a10 con 0x7fdc1415f1e0 2024-04-26T21:19:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc18a17640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1406dc60 con 0x7fdc1406d2b0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc18a17640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1406eeb0 con 0x7fdc140b44e0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc1406d2b0 0x7fdc1406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc1406d2b0 0x7fdc1406d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45740/0 (socket says 172.21.15.40:45740) 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 -- 172.21.15.40:0/1553235901 learned_addr learned my addr 172.21.15.40:0/1553235901 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 -- 172.21.15.40:0/1553235901 >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 msgr2=0x7fdc1406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc11d9b640 1 --2- 172.21.15.40:0/1553235901 >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 0x7fdc1406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 --2- 172.21.15.40:0/1553235901 >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 0x7fdc1406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 -- 172.21.15.40:0/1553235901 >> v1:172.21.15.184:6789/0 conn(0x7fdc1415f1e0 legacy=0x7fdc140ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc12d9d640 1 -- 172.21.15.40:0/1553235901 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc140b4cf0 con 0x7fdc1406d2b0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.560+0000 7fdc11d9b640 1 --2- 172.21.15.40:0/1553235901 >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 0x7fdc1406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc12d9d640 1 --2- 172.21.15.40:0/1553235901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1406d2b0 0x7fdc1406d720 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fdc1406e5c0 tx=0x7fdc0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b256f7380dcbc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc1159a640 1 -- 172.21.15.40:0/1553235901 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04036070 con 0x7fdc1406d2b0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc1159a640 1 -- 172.21.15.40:0/1553235901 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc04031600 con 0x7fdc1406d2b0 2024-04-26T21:19:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc1159a640 1 -- 172.21.15.40:0/1553235901 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc040319d0 con 0x7fdc1406d2b0 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc18a17640 1 -- 172.21.15.40:0/1553235901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1406d2b0 msgr2=0x7fdc1406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.561+0000 7fdc18a17640 1 --2- 172.21.15.40:0/1553235901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1406d2b0 0x7fdc1406d720 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fdc1406e5c0 tx=0x7fdc0402a6f0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 -- 172.21.15.40:0/1553235901 shutdown_connections 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 --2- 172.21.15.40:0/1553235901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1406d2b0 0x7fdc1406d720 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 --2- 172.21.15.40:0/1553235901 >> v2:172.21.15.184:3300/0 conn(0x7fdc140b44e0 0x7fdc1406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 -- 172.21.15.40:0/1553235901 >> 172.21.15.40:0/1553235901 conn(0x7fdc140728a0 msgr2=0x7fdc14072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 -- 172.21.15.40:0/1553235901 shutdown_connections 2024-04-26T21:19:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.562+0000 7fdc18a17640 1 -- 172.21.15.40:0/1553235901 wait complete. 2024-04-26T21:19:45.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.563+0000 7fdc18a17640 1 Processor -- start 2024-04-26T21:19:45.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.563+0000 7fdc18a17640 1 -- start start 2024-04-26T21:19:45.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc1406d2b0 0x7fdc14142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc1415f1e0 0x7fdc1413f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140b5590 con 0x7fdc140b44e0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140b52b0 con 0x7fdc1406d2b0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1415f5c0 con 0x7fdc1415f1e0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45752/0 (socket says 172.21.15.40:45752) 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 -- 172.21.15.40:0/2641977056 learned_addr learned my addr 172.21.15.40:0/2641977056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc1259c640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc1406d2b0 0x7fdc14142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc12d9d640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc1415f1e0 0x7fdc1413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 -- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc1415f1e0 msgr2=0x7fdc1413f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc1415f1e0 0x7fdc1413f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 -- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc1406d2b0 msgr2=0x7fdc14142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc1406d2b0 0x7fdc14142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 -- 172.21.15.40:0/2641977056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc14161d30 con 0x7fdc140b44e0 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc11d9b640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c820 tx=0x7fdbfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc015070 con 0x7fdc140b44e0 2024-04-26T21:19:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc007a30 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc007e00 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc14162010 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.564+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc141625d0 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.565+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1406a290 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.566+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbfc018070 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.567+0000 7fdc0affd640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 0x7fdbf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.567+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fdbfc08bb60 con 0x7fdc140b44e0 2024-04-26T21:19:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.567+0000 7fdc1259c640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 0x7fdbf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.567+0000 7fdc1259c640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 0x7fdbf406edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.571+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdbfc059f50 con 0x7fdc140b44e0 2024-04-26T21:19:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.756+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc14068470 con 0x7fdc140b44e0 2024-04-26T21:19:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.757+0000 7fdc0affd640 1 -- 172.21.15.40:0/2641977056 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16781 (secure 0 0 0) 0x7fdbfc059850 con 0x7fdc140b44e0 2024-04-26T21:19:45.758 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:45.758 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":111,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:43.638451+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:14.110950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:14.791024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 msgr2=0x7fdbf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 0x7fdbf406edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 msgr2=0x7fdc14161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c820 tx=0x7fdbfc00cc20 comp rx=0 tx=0).stop 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 shutdown_connections 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdbf406c8f0 0x7fdbf406edb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc1415f1e0 0x7fdc1413f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc140b44e0 0x7fdc14161500 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 --2- 172.21.15.40:0/2641977056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc1406d2b0 0x7fdc14142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 >> 172.21.15.40:0/2641977056 conn(0x7fdc140728a0 msgr2=0x7fdc1406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 shutdown_connections 2024-04-26T21:19:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.759+0000 7fdc18a17640 1 -- 172.21.15.40:0/2641977056 wait complete. 2024-04-26T21:19:45.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:19:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.871+0000 7f3a966f8640 1 Processor -- start 2024-04-26T21:19:45.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.871+0000 7f3a966f8640 1 -- start start 2024-04-26T21:19:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a966f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a9015b4e0 0x7f3a90071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a966f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a900cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a966f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a900733e0 con 0x7f3a9015b8c0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a966f8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a900ce230 con 0x7f3a9015b4e0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a966f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a90072f00 con 0x7f3a9006f2c0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a8ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a9015b4e0 0x7f3a90071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a8ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a9015b4e0 0x7f3a90071fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45766/0 (socket says 172.21.15.40:45766) 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.872+0000 7f3a8ffff640 1 -- 172.21.15.40:0/62482153 learned_addr learned my addr 172.21.15.40:0/62482153 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a94c95640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a900cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a8ffff640 1 -- 172.21.15.40:0/62482153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 msgr2=0x7f3a900cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a900cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a8ffff640 1 -- 172.21.15.40:0/62482153 >> v1:172.21.15.184:6789/0 conn(0x7f3a9015b8c0 legacy=0x7f3a9006ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a8ffff640 1 -- 172.21.15.40:0/62482153 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90072660 con 0x7f3a9015b4e0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a94c95640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a900cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.873+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a90071fd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb862ce3be1bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a8effd640 1 -- 172.21.15.40:0/62482153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037070 con 0x7f3a9015b4e0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a8effd640 1 -- 172.21.15.40:0/62482153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84032440 con 0x7f3a9015b4e0 2024-04-26T21:19:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a8effd640 1 -- 172.21.15.40:0/62482153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84032810 con 0x7f3a9015b4e0 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 -- 172.21.15.40:0/62482153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 msgr2=0x7f3a90071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a90071fd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8402a270 comp rx=0 tx=0).stop 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 -- 172.21.15.40:0/62482153 shutdown_connections 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a900cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 --2- 172.21.15.40:0/62482153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a90071fd0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 -- 172.21.15.40:0/62482153 >> 172.21.15.40:0/62482153 conn(0x7f3a900d0d30 msgr2=0x7f3a900d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 -- 172.21.15.40:0/62482153 shutdown_connections 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.874+0000 7f3a966f8640 1 -- 172.21.15.40:0/62482153 wait complete. 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.875+0000 7f3a966f8640 1 Processor -- start 2024-04-26T21:19:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 -- start start 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a9015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a9014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a9015b8c0 0x7f3a9014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a900b3e80 con 0x7f3a9015b4e0 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90072c20 con 0x7f3a9015b8c0 2024-04-26T21:19:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a966f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a90073820 con 0x7f3a9006f2c0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a9014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a8f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a9014e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45780/0 (socket says 172.21.15.40:45780) 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.876+0000 7f3a8f7fe640 1 -- 172.21.15.40:0/3277350339 learned_addr learned my addr 172.21.15.40:0/3277350339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a94c95640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a9015b8c0 0x7f3a9014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a9015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 -- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 msgr2=0x7f3a9015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a9015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 -- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a9015b8c0 msgr2=0x7f3a9014ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a9015b8c0 0x7f3a9014ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 -- 172.21.15.40:0/3277350339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9015ed20 con 0x7f3a9015b4e0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a9006f2c0 0x7f3a9015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8f7fe640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9015b4e0 0x7f3a9014e7a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c440 tx=0x7f3a8000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800052d0 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a80005470 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80007260 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a966f8640 1 -- 172.21.15.40:0/3277350339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a900af690 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.877+0000 7f3a966f8640 1 -- 172.21.15.40:0/3277350339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a900afc50 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.878+0000 7f3a966f8640 1 -- 172.21.15.40:0/3277350339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9006a290 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.879+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a80007400 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.879+0000 7f3a8cff9640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.879+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f3a8008af60 con 0x7f3a9015b4e0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.879+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.880+0000 7f3a8ffff640 1 --2- 172.21.15.40:0/3277350339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3a84034000 tx=0x7f3a8402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:45.883+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a800593e0 con 0x7f3a9015b4e0 2024-04-26T21:19:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.109+0000 7f3a966f8640 1 -- 172.21.15.40:0/3277350339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3a900afe20 con 0x7f3a9015b4e0 2024-04-26T21:19:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.110+0000 7f3a8cff9640 1 -- 172.21.15.40:0/3277350339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v67) ==== 316+0+0 (secure 0 0 0) 0x7f3a80058ce0 con 0x7f3a9015b4e0 2024-04-26T21:19:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-04-26T21:19:46.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:19:46.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-04-26T21:19:46.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.217+0000 7f502d833640 1 Processor -- start 2024-04-26T21:19:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 -- start start 2024-04-26T21:19:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f502815f490 0x7f50280d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50280d4240 con 0x7f50280d0ff0 2024-04-26T21:19:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f50280d31a0 con 0x7f502815f490 2024-04-26T21:19:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.218+0000 7f502d833640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50280d3480 con 0x7f50280d22f0 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f50277fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f502815f490 0x7f50280d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f50277fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:38812/0 (socket says 172.21.15.40:38812) 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f502815f490 0x7f50280d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:33538/0 (socket says 172.21.15.40:33538) 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f50277fe640 1 -- 172.21.15.40:0/2157360933 learned_addr learned my addr 172.21.15.40:0/2157360933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5025ffb640 1 -- 172.21.15.40:0/2157360933 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022702021 0 0) 0x7f50280d4240 con 0x7f50280d0ff0 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5025ffb640 1 -- 172.21.15.40:0/2157360933 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5010003520 con 0x7f50280d0ff0 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 -- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 msgr2=0x7f50280ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 -- 172.21.15.40:0/2157360933 >> v1:172.21.15.184:6789/0 conn(0x7f50280d0ff0 legacy=0x7f50280d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f5026ffd640 1 -- 172.21.15.40:0/2157360933 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50280d3760 con 0x7f502815f490 2024-04-26T21:19:46.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.219+0000 7f50277fe640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.220+0000 7f5026ffd640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502815f490 0x7f50280d0ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6731cf5bd183750b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.220+0000 7f5025ffb640 1 -- 172.21.15.40:0/2157360933 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020037070 con 0x7f502815f490 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.220+0000 7f5025ffb640 1 -- 172.21.15.40:0/2157360933 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5020032440 con 0x7f502815f490 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.220+0000 7f5025ffb640 1 -- 172.21.15.40:0/2157360933 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50200327f0 con 0x7f502815f490 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 -- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502815f490 msgr2=0x7f50280d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502815f490 0x7f50280d0ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002a180 comp rx=0 tx=0).stop 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 -- 172.21.15.40:0/2157360933 shutdown_connections 2024-04-26T21:19:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50280d22f0 0x7f50280ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 --2- 172.21.15.40:0/2157360933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502815f490 0x7f50280d0ab0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 -- 172.21.15.40:0/2157360933 >> 172.21.15.40:0/2157360933 conn(0x7f502809e530 msgr2=0x7f50280d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 -- 172.21.15.40:0/2157360933 shutdown_connections 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.221+0000 7f502d833640 1 -- 172.21.15.40:0/2157360933 wait complete. 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.222+0000 7f502d833640 1 Processor -- start 2024-04-26T21:19:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.222+0000 7f502d833640 1 -- start start 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280d0ff0 0x7f5028152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 0x7f50280b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f502815f870 con 0x7f50280d22f0 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50280d3d20 con 0x7f50280d0ff0 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f502d833640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f50280d2dc0 con 0x7f502815f490 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45788/0 (socket says 172.21.15.40:45788) 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 -- 172.21.15.40:0/1248128741 learned_addr learned my addr 172.21.15.40:0/1248128741 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f5026ffd640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280d0ff0 0x7f5028152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50277fe640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 0x7f50280b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 -- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 msgr2=0x7f50280b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 0x7f50280b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 -- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280d0ff0 msgr2=0x7f5028152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280d0ff0 0x7f5028152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50267fc640 1 -- 172.21.15.40:0/1248128741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50280cce70 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.223+0000 7f50277fe640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 0x7f50280b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f50267fc640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f501c00c430 tx=0x7f501c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c017070 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501c0052f0 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50280cd150 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c003830 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.224+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50280cdc20 con 0x7f50280d22f0 2024-04-26T21:19:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.225+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502806a290 con 0x7f50280d22f0 2024-04-26T21:19:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.225+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f501c01a070 con 0x7f50280d22f0 2024-04-26T21:19:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.226+0000 7f500ffff640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 0x7f501406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.226+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f501c08ade0 con 0x7f50280d22f0 2024-04-26T21:19:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.226+0000 7f5026ffd640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 0x7f501406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.227+0000 7f5026ffd640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 0x7f501406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5020034000 tx=0x7f502002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.230+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f501c058c10 con 0x7f50280d22f0 2024-04-26T21:19:46.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.456+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f50280cd320 con 0x7f50280d22f0 2024-04-26T21:19:46.457 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:19:46.457+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:19:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.458+0000 7f500ffff640 1 -- 172.21.15.40:0/1248128741 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 191+0+0 (secure 0 0 0) 0x7f501c05cf10 con 0x7f50280d22f0 2024-04-26T21:19:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 msgr2=0x7f501406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 0x7f501406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5020034000 tx=0x7f502002e000 comp rx=0 tx=0).stop 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 msgr2=0x7f50280cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f501c00c430 tx=0x7f501c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 shutdown_connections 2024-04-26T21:19:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f501406c8f0 0x7f501406edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502815f490 0x7f50280b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50280d22f0 0x7f50280cb8d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 --2- 172.21.15.40:0/1248128741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50280d0ff0 0x7f5028152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.461+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 >> 172.21.15.40:0/1248128741 conn(0x7f502809e530 msgr2=0x7f502815fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.462+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 shutdown_connections 2024-04-26T21:19:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.462+0000 7f502d833640 1 -- 172.21.15.40:0/1248128741 wait complete. 2024-04-26T21:19:46.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:19:46.501 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:19:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.576+0000 7f4f2a007640 1 Processor -- start 2024-04-26T21:19:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.576+0000 7f4f2a007640 1 -- start start 2024-04-26T21:19:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f2a007640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 0x7f4f240b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f2a007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 0x7f4f240cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f2a007640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f2415dca0 con 0x7f4f24143840 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f2a007640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4f24143c20 con 0x7f4f240b4eb0 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f2a007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2415df80 con 0x7f4f240ceff0 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f24143840 0x7f4f240b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60924/0 (socket says 172.21.15.40:60924) 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f237fe640 1 -- 172.21.15.40:0/1501425031 learned_addr learned my addr 172.21.15.40:0/1501425031 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f23fff640 1 --2- 172.21.15.40:0/1501425031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 0x7f4f240cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f22ffd640 1 --2- 172.21.15.40:0/1501425031 >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 0x7f4f240b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483868545 0 0) 0x7f4f2415dca0 con 0x7f4f24143840 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f0c003520 con 0x7f4f24143840 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f23fff640 1 -- 172.21.15.40:0/1501425031 >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 msgr2=0x7f4f240b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f23fff640 1 --2- 172.21.15.40:0/1501425031 >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 0x7f4f240b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2968937085 0 0) 0x7f4f0c003520 con 0x7f4f24143840 2024-04-26T21:19:46.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f23fff640 1 -- 172.21.15.40:0/1501425031 >> v1:172.21.15.40:6789/0 conn(0x7f4f24143840 legacy=0x7f4f240b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f23fff640 1 -- 172.21.15.40:0/1501425031 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2415e260 con 0x7f4f240ceff0 2024-04-26T21:19:46.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.577+0000 7f4f22ffd640 1 --2- 172.21.15.40:0/1501425031 >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 0x7f4f240b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.578+0000 7f4f23fff640 1 --2- 172.21.15.40:0/1501425031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 0x7f4f240cf480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009d90 tx=0x7f4f1c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f68f7c0cb54d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.578+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18001670 con 0x7f4f240ceff0 2024-04-26T21:19:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.578+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c007540 con 0x7f4f240ceff0 2024-04-26T21:19:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.578+0000 7f4f227fc640 1 -- 172.21.15.40:0/1501425031 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c007910 con 0x7f4f240ceff0 2024-04-26T21:19:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 -- 172.21.15.40:0/1501425031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 msgr2=0x7f4f240cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 --2- 172.21.15.40:0/1501425031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 0x7f4f240cf480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009d90 tx=0x7f4f1c02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 -- 172.21.15.40:0/1501425031 shutdown_connections 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 --2- 172.21.15.40:0/1501425031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f240ceff0 0x7f4f240cf480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 --2- 172.21.15.40:0/1501425031 >> v2:172.21.15.184:3300/0 conn(0x7f4f240b4eb0 0x7f4f240b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 -- 172.21.15.40:0/1501425031 >> 172.21.15.40:0/1501425031 conn(0x7f4f240cd8a0 msgr2=0x7f4f240cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 -- 172.21.15.40:0/1501425031 shutdown_connections 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.579+0000 7f4f2a007640 1 -- 172.21.15.40:0/1501425031 wait complete. 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.580+0000 7f4f2a007640 1 Processor -- start 2024-04-26T21:19:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.580+0000 7f4f2a007640 1 -- start start 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f240ceff0 0x7f4f2413d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 0x7f4f241410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2415d8c0 con 0x7f4f240b4eb0 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f240b46f0 con 0x7f4f240ceff0 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f2a007640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2415da40 con 0x7f4f24143840 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45794/0 (socket says 172.21.15.40:45794) 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 -- 172.21.15.40:0/2714695726 learned_addr learned my addr 172.21.15.40:0/2714695726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f23fff640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 0x7f4f241410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f22ffd640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f240ceff0 0x7f4f2413d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 -- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 msgr2=0x7f4f241410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 0x7f4f241410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 -- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f240ceff0 msgr2=0x7f4f2413d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f240ceff0 0x7f4f2413d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f237fe640 1 -- 172.21.15.40:0/2714695726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f24144020 con 0x7f4f240b4eb0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.581+0000 7f4f23fff640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 0x7f4f241410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f237fe640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800c730 tx=0x7f4f1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18017070 con 0x7f4f240b4eb0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f18005ad0 con 0x7f4f240b4eb0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f24144300 con 0x7f4f240b4eb0 2024-04-26T21:19:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18007070 con 0x7f4f240b4eb0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.582+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2415af90 con 0x7f4f240b4eb0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.583+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2415d480 con 0x7f4f240b4eb0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.583+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f1801a070 con 0x7f4f240b4eb0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.584+0000 7f4f0bfff640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 0x7f4f1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.584+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f4f1808af70 con 0x7f4f240b4eb0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.584+0000 7f4f22ffd640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 0x7f4f1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.585+0000 7f4f22ffd640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 0x7f4f1006ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009560 tx=0x7f4f14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.589+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f2415d480 con 0x7f4f240b4eb0 2024-04-26T21:19:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:46.817+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4f2406a290 con 0x7f4f240b4eb0 2024-04-26T21:19:47.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.459+0000 7f4f0bfff640 1 -- 172.21.15.40:0/2714695726 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f4f18058cb0 con 0x7f4f240b4eb0 2024-04-26T21:19:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 msgr2=0x7f4f1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 0x7f4f1006ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009560 tx=0x7f4f14008040 comp rx=0 tx=0).stop 2024-04-26T21:19:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 msgr2=0x7f4f2415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800c730 tx=0x7f4f1800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 shutdown_connections 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4f1006c920 0x7f4f1006ede0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f24143840 0x7f4f241410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f240ceff0 0x7f4f2413d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 --2- 172.21.15.40:0/2714695726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f240b4eb0 0x7f4f2415aa50 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 >> 172.21.15.40:0/2714695726 conn(0x7f4f240cd8a0 msgr2=0x7f4f240b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 shutdown_connections 2024-04-26T21:19:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.462+0000 7f4f2a007640 1 -- 172.21.15.40:0/2714695726 wait complete. 2024-04-26T21:19:47.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:19:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.583+0000 7f2e323f7640 1 Processor -- start 2024-04-26T21:19:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.583+0000 7f2e323f7640 1 -- start start 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e323f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e2c148050 0x7f2e2c0d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e323f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e2c148cb0 0x7f2e2c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e323f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e2c0b5590 con 0x7f2e2c0b5190 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e323f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e2c163c90 con 0x7f2e2c148cb0 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e323f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e2c163080 con 0x7f2e2c148050 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e2b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e2c148cb0 0x7f2e2c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e30994640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2e2c0b5190 0x7f2e2c070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60934/0 (socket says 172.21.15.40:60934) 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.584+0000 7f2e30994640 1 -- 172.21.15.40:0/1750058185 learned_addr learned my addr 172.21.15.40:0/1750058185 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2bfff640 1 --2- 172.21.15.40:0/1750058185 >> v2:172.21.15.184:3300/0 conn(0x7f2e2c148050 0x7f2e2c0d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/1750058185 >> v2:172.21.15.184:3300/0 conn(0x7f2e2c148050 msgr2=0x7f2e2c0d38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2b7fe640 1 --2- 172.21.15.40:0/1750058185 >> v2:172.21.15.184:3300/0 conn(0x7f2e2c148050 0x7f2e2c0d38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/1750058185 >> v1:172.21.15.40:6789/0 conn(0x7f2e2c0b5190 legacy=0x7f2e2c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/1750058185 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c163360 con 0x7f2e2c148cb0 2024-04-26T21:19:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2affd640 1 -- 172.21.15.40:0/1750058185 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525084618 0 0) 0x7f2e2c0b5590 con 0x7f2e2c0b5190 2024-04-26T21:19:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.585+0000 7f2e2b7fe640 1 --2- 172.21.15.40:0/1750058185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148cb0 0x7f2e2c0b4c50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0b5c60 tx=0x7f2e2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaca8411a05fbba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e2affd640 1 -- 172.21.15.40:0/1750058185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037070 con 0x7f2e2c148cb0 2024-04-26T21:19:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e2affd640 1 -- 172.21.15.40:0/1750058185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20032440 con 0x7f2e2c148cb0 2024-04-26T21:19:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e2affd640 1 -- 172.21.15.40:0/1750058185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20032810 con 0x7f2e2c148cb0 2024-04-26T21:19:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 -- 172.21.15.40:0/1750058185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148cb0 msgr2=0x7f2e2c0b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 --2- 172.21.15.40:0/1750058185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148cb0 0x7f2e2c0b4c50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0b5c60 tx=0x7f2e2002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 -- 172.21.15.40:0/1750058185 shutdown_connections 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 --2- 172.21.15.40:0/1750058185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148cb0 0x7f2e2c0b4c50 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 --2- 172.21.15.40:0/1750058185 >> v2:172.21.15.184:3300/0 conn(0x7f2e2c148050 0x7f2e2c0d38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 -- 172.21.15.40:0/1750058185 >> 172.21.15.40:0/1750058185 conn(0x7f2e2c06c960 msgr2=0x7f2e2c0b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 -- 172.21.15.40:0/1750058185 shutdown_connections 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.586+0000 7f2e323f7640 1 -- 172.21.15.40:0/1750058185 wait complete. 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.587+0000 7f2e323f7640 1 Processor -- start 2024-04-26T21:19:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 -- start start 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 0x7f2e2c0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e2c148cb0 0x7f2e2c13d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c0d3e30 con 0x7f2e2c148050 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c163920 con 0x7f2e2c0b5190 2024-04-26T21:19:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e323f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2c148430 con 0x7f2e2c148cb0 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e2b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e2b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45812/0 (socket says 172.21.15.40:45812) 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.588+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/2632319790 learned_addr learned my addr 172.21.15.40:0/2632319790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2bfff640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 0x7f2e2c0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e30994640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e2c148cb0 0x7f2e2c13d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e2c148cb0 msgr2=0x7f2e2c13d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e2c148cb0 0x7f2e2c13d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 msgr2=0x7f2e2c0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 0x7f2e2c0cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 -- 172.21.15.40:0/2632319790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c151a40 con 0x7f2e2c148050 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2bfff640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 0x7f2e2c0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e2b7fe640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037040 con 0x7f2e2c148050 2024-04-26T21:19:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20042070 con 0x7f2e2c148050 2024-04-26T21:19:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2c151d20 con 0x7f2e2c148050 2024-04-26T21:19:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20030bd0 con 0x7f2e2c148050 2024-04-26T21:19:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.589+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2c1522e0 con 0x7f2e2c148050 2024-04-26T21:19:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.590+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2c0693d0 con 0x7f2e2c148050 2024-04-26T21:19:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.591+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e20047460 con 0x7f2e2c148050 2024-04-26T21:19:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.591+0000 7f2e28ff9640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 0x7f2e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.591+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f2e200ac800 con 0x7f2e2c148050 2024-04-26T21:19:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.591+0000 7f2e2bfff640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 0x7f2e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.592+0000 7f2e2bfff640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 0x7f2e1806edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006fd0 tx=0x7f2e1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:47.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.595+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e20032e20 con 0x7f2e2c148050 2024-04-26T21:19:47.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:47.783+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e2c06ea80 con 0x7f2e2c148050 2024-04-26T21:19:48.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.668+0000 7f2e28ff9640 1 -- 172.21.15.40:0/2632319790 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) ==== 181+0+0 (secure 0 0 0) 0x7f2e2007aba0 con 0x7f2e2c148050 2024-04-26T21:19:48.670 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:19:48.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 msgr2=0x7f2e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 0x7f2e1806edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006fd0 tx=0x7f2e1c008040 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 msgr2=0x7f2e2c13d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e200046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 shutdown_connections 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2e1806c8f0 0x7f2e1806edb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e2c148cb0 0x7f2e2c13d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e2c148050 0x7f2e2c13d3a0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 --2- 172.21.15.40:0/2632319790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e2c0b5190 0x7f2e2c0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.674+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 >> 172.21.15.40:0/2632319790 conn(0x7f2e2c06c960 msgr2=0x7f2e2c0b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:48.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.675+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 shutdown_connections 2024-04-26T21:19:48.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.675+0000 7f2e323f7640 1 -- 172.21.15.40:0/2632319790 wait complete. 2024-04-26T21:19:48.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:19:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.801+0000 7f563dc59640 1 Processor -- start 2024-04-26T21:19:48.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.801+0000 7f563dc59640 1 -- start start 2024-04-26T21:19:48.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f563dc59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56380cee20 0x7f563806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f563dc59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f563809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f563dc59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56380d1240 con 0x7f563809bf80 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f563dc59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f56380d1e10 con 0x7f56380cee20 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f563dc59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56380d14e0 con 0x7f56380d3a30 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f5637fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f563809bf80 0x7f56381630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60938/0 (socket says 172.21.15.40:60938) 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f5637fff640 1 -- 172.21.15.40:0/793575059 learned_addr learned my addr 172.21.15.40:0/793575059 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917877222 0 0) 0x7f56380d1240 con 0x7f563809bf80 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56377fe640 1 --2- 172.21.15.40:0/793575059 >> v2:172.21.15.184:3300/0 conn(0x7f56380cee20 0x7f563806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5620003520 con 0x7f563809bf80 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f5636ffd640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f563809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 890537374 0 0) 0x7f5620003520 con 0x7f563809bf80 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 msgr2=0x7f563809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:48.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f563809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 >> v2:172.21.15.184:3300/0 conn(0x7f56380cee20 msgr2=0x7f563806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 --2- 172.21.15.40:0/793575059 >> v2:172.21.15.184:3300/0 conn(0x7f56380cee20 0x7f563806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56380d17c0 con 0x7f563809bf80 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f56300019f0 con 0x7f563809bf80 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 >> v1:172.21.15.40:6789/0 conn(0x7f563809bf80 legacy=0x7f56381630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f5620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.802+0000 7f56367fc640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56367fc640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f56200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56380d1e10 con 0x7f56200039b0 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56380d14e0 con 0x7f5620004800 2024-04-26T21:19:48.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f562000b9c0 con 0x7f56380d3a30 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f5620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f5637fff640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f56200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f5636ffd640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 -- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 msgr2=0x7f56200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f56200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 -- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5620004800 msgr2=0x7f5620004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f56377fe640 1 -- 172.21.15.40:0/793575059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5620003520 con 0x7f56200039b0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.803+0000 7f5637fff640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56380d3a30 0x7f56200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f56377fe640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f5620003dd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ac0 tx=0x7f5620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2c7a6e7d834326 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c036070 con 0x7f56200039b0 2024-04-26T21:19:48.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c031400 con 0x7f56200039b0 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f56367fc640 1 -- 172.21.15.40:0/793575059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c009690 con 0x7f56200039b0 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 -- 172.21.15.40:0/793575059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 msgr2=0x7f5620003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 --2- 172.21.15.40:0/793575059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f5620003dd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ac0 tx=0x7f5620005830 comp rx=0 tx=0).stop 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f5637fff640 1 -- 172.21.15.40:0/793575059 reap_dead start 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 -- 172.21.15.40:0/793575059 shutdown_connections 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 -- 172.21.15.40:0/793575059 >> 172.21.15.40:0/793575059 conn(0x7f56380d2f10 msgr2=0x7f56380d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 -- 172.21.15.40:0/793575059 shutdown_connections 2024-04-26T21:19:48.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.804+0000 7f563dc59640 1 -- 172.21.15.40:0/793575059 wait complete. 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 Processor -- start 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 -- start start 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56381f4b10 0x7f563815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f563815dba0 0x7f563815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56380cf200 con 0x7f56200039b0 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f563809c7a0 con 0x7f56381f4b10 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.806+0000 7f563dc59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f56380cf380 con 0x7f563815dba0 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45838/0 (socket says 172.21.15.40:45838) 2024-04-26T21:19:48.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 -- 172.21.15.40:0/684549744 learned_addr learned my addr 172.21.15.40:0/684549744 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f5637fff640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f563815dba0 0x7f563815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 -- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f563815dba0 msgr2=0x7f563815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f5636ffd640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56381f4b10 0x7f563815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f563815dba0 0x7f563815e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 -- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56381f4b10 msgr2=0x7f563815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56381f4b10 0x7f563815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 -- 172.21.15.40:0/684549744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563815b4d0 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f56377fe640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f562c009be0 tx=0x7f562c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c040020 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c003c40 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c036040 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563815b750 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.807+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563815bd10 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.808+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563806a290 con 0x7f56200039b0 2024-04-26T21:19:48.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.809+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f562c043020 con 0x7f56200039b0 2024-04-26T21:19:48.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.809+0000 7f561ffff640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 0x7f562406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:48.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.809+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5415+0+0 (secure 0 0 0) 0x7f562c0abb50 con 0x7f56200039b0 2024-04-26T21:19:48.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.809+0000 7f5636ffd640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 0x7f562406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:48.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.810+0000 7f5636ffd640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 0x7f562406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f56381f4f30 tx=0x7f5628008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:48.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:48.813+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f562c04a070 con 0x7f56200039b0 2024-04-26T21:19:49.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.000+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f56380b8e90 con 0x7f56200039b0 2024-04-26T21:19:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.674+0000 7f561ffff640 1 -- 172.21.15.40:0/684549744 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) ==== 169+0+0 (secure 0 0 0) 0x7f563815b4d0 con 0x7f56200039b0 2024-04-26T21:19:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 msgr2=0x7f562406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 0x7f562406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f56381f4f30 tx=0x7f5628008040 comp rx=0 tx=0).stop 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 msgr2=0x7f563815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f562c009be0 tx=0x7f562c004730 comp rx=0 tx=0).stop 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 shutdown_connections 2024-04-26T21:19:49.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f562406c8f0 0x7f562406edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f563815dba0 0x7f563815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56381f4b10 0x7f563815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 --2- 172.21.15.40:0/684549744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56200039b0 0x7f563815af90 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 >> 172.21.15.40:0/684549744 conn(0x7f56380d2f10 msgr2=0x7f5638162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 shutdown_connections 2024-04-26T21:19:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.676+0000 7f563dc59640 1 -- 172.21.15.40:0/684549744 wait complete. 2024-04-26T21:19:49.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-26T21:19:49.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.809+0000 7f43b6df9640 1 Processor -- start 2024-04-26T21:19:49.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.809+0000 7f43b6df9640 1 -- start start 2024-04-26T21:19:49.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b6df9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 0x7f43b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b6df9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43b00d1170 0x7f43b01635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b6df9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b009c790 con 0x7f43b00b3300 2024-04-26T21:19:49.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b6df9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43b00d15f0 con 0x7f43b00d1170 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b6df9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f43b0163ec0 con 0x7f43b01631f0 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43b00d1170 0x7f43b01635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43b00d1170 0x7f43b01635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45840/0 (socket says 172.21.15.40:45840) 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 -- 172.21.15.40:0/3213792599 learned_addr learned my addr 172.21.15.40:0/3213792599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b4b95640 1 --2- 172.21.15.40:0/3213792599 >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 0x7f43b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 -- 172.21.15.40:0/3213792599 >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 msgr2=0x7f43b00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 --2- 172.21.15.40:0/3213792599 >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 0x7f43b00b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 -- 172.21.15.40:0/3213792599 >> v1:172.21.15.184:6789/0 conn(0x7f43b00b3300 legacy=0x7f43b00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43affff640 1 -- 172.21.15.40:0/3213792599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b00d1a40 con 0x7f43b00d1170 2024-04-26T21:19:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.810+0000 7f43b4b95640 1 --2- 172.21.15.40:0/3213792599 >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 0x7f43b00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:49.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.811+0000 7f43affff640 1 --2- 172.21.15.40:0/3213792599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b01635d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f43b00d2ef0 tx=0x7f43a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227f042cf7dae2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:49.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.811+0000 7f43af7fe640 1 -- 172.21.15.40:0/3213792599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4037070 con 0x7f43b00d1170 2024-04-26T21:19:49.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.811+0000 7f43af7fe640 1 -- 172.21.15.40:0/3213792599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a4032440 con 0x7f43b00d1170 2024-04-26T21:19:49.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.811+0000 7f43af7fe640 1 -- 172.21.15.40:0/3213792599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4032810 con 0x7f43b00d1170 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 -- 172.21.15.40:0/3213792599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 msgr2=0x7f43b01635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 --2- 172.21.15.40:0/3213792599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b01635d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f43b00d2ef0 tx=0x7f43a402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 -- 172.21.15.40:0/3213792599 shutdown_connections 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 --2- 172.21.15.40:0/3213792599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b01635d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 --2- 172.21.15.40:0/3213792599 >> v2:172.21.15.184:3300/0 conn(0x7f43b01631f0 0x7f43b00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 -- 172.21.15.40:0/3213792599 >> 172.21.15.40:0/3213792599 conn(0x7f43b009ba40 msgr2=0x7f43b009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 -- 172.21.15.40:0/3213792599 shutdown_connections 2024-04-26T21:19:49.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.812+0000 7f43b6df9640 1 -- 172.21.15.40:0/3213792599 wait complete. 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.813+0000 7f43b6df9640 1 Processor -- start 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.813+0000 7f43b6df9640 1 -- start start 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43b00b3300 0x7f43b00cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 0x7f43b01e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00d0230 con 0x7f43b00d1170 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00d2000 con 0x7f43b01631f0 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b6df9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43b0157db0 con 0x7f43b00b3300 2024-04-26T21:19:49.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45852/0 (socket says 172.21.15.40:45852) 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 -- 172.21.15.40:0/4083071627 learned_addr learned my addr 172.21.15.40:0/4083071627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b5396640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 0x7f43b01e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43b4b95640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43b00b3300 0x7f43b00cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 -- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43b00b3300 msgr2=0x7f43b00cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43b00b3300 0x7f43b00cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 -- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 msgr2=0x7f43b01e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.814+0000 7f43affff640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 0x7f43b01e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43affff640 1 -- 172.21.15.40:0/4083071627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b01e9c30 con 0x7f43b00d1170 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43b5396640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 0x7f43b01e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43affff640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f43a4000c00 tx=0x7f43a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4041070 con 0x7f43b00d1170 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a403c870 con 0x7f43b00d1170 2024-04-26T21:19:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b01e9f10 con 0x7f43b00d1170 2024-04-26T21:19:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4037040 con 0x7f43b00d1170 2024-04-26T21:19:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.815+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01ea4d0 con 0x7f43b00d1170 2024-04-26T21:19:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.816+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b006a290 con 0x7f43b00d1170 2024-04-26T21:19:49.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.817+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43a4044070 con 0x7f43b00d1170 2024-04-26T21:19:49.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.817+0000 7f43ad7fa640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 0x7f439c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:49.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.817+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f43a40ac530 con 0x7f43b00d1170 2024-04-26T21:19:49.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.817+0000 7f43b4b95640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 0x7f439c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:49.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.818+0000 7f43b4b95640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 0x7f439c06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43b00b8e10 tx=0x7f43a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:49.821+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43a407a730 con 0x7f43b00d1170 2024-04-26T21:19:50.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.008+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f43b00cbd40 con 0x7f43b00d1170 2024-04-26T21:19:50.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.011+0000 7f43ad7fa640 1 -- 172.21.15.40:0/4083071627 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v41) ==== 111+0+0 (secure 0 0 0) 0x7f43a407ea30 con 0x7f43b00d1170 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.013+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 msgr2=0x7f439c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.013+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 0x7f439c06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43b00b8e10 tx=0x7f43a0008040 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.013+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 msgr2=0x7f43b00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.013+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f43a4000c00 tx=0x7f43a40046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 shutdown_connections 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f439c06c8f0 0x7f439c06edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43b01631f0 0x7f43b01e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00d1170 0x7f43b00cc4a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 --2- 172.21.15.40:0/4083071627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43b00b3300 0x7f43b00cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 >> 172.21.15.40:0/4083071627 conn(0x7f43b009ba40 msgr2=0x7f43b0075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 shutdown_connections 2024-04-26T21:19:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.014+0000 7f43b6df9640 1 -- 172.21.15.40:0/4083071627 wait complete. 2024-04-26T21:19:50.025 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:19:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.127+0000 7f8b93b38640 1 Processor -- start 2024-04-26T21:19:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.127+0000 7f8b93b38640 1 -- start start 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b93b38640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b8c143840 0x7f8b8c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b93b38640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c15dca0 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b93b38640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c15df80 con 0x7f8b8c0b4520 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b93b38640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c143c20 con 0x7f8b8c143840 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b918d4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b8c143840 0x7f8b8c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b920d5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b8c0b55c0 0x7f8b8c0ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60946/0 (socket says 172.21.15.40:60946) 2024-04-26T21:19:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b920d5640 1 -- 172.21.15.40:0/3645470504 learned_addr learned my addr 172.21.15.40:0/3645470504 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b918d4640 1 -- 172.21.15.40:0/3645470504 >> v1:172.21.15.184:6789/0 conn(0x7f8b8c0b4520 legacy=0x7f8b8c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b918d4640 1 -- 172.21.15.40:0/3645470504 >> v1:172.21.15.40:6789/0 conn(0x7f8b8c0b55c0 legacy=0x7f8b8c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b918d4640 1 -- 172.21.15.40:0/3645470504 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c15e260 con 0x7f8b8c143840 2024-04-26T21:19:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.128+0000 7f8b908d2640 1 -- 172.21.15.40:0/3645470504 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614582923 0 0) 0x7f8b8c15dca0 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.129+0000 7f8b918d4640 1 --2- 172.21.15.40:0/3645470504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c143840 0x7f8b8c0b3fe0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097d0 tx=0x7f8b8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d763280600cec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.129+0000 7f8b908d2640 1 -- 172.21.15.40:0/3645470504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037070 con 0x7f8b8c143840 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.129+0000 7f8b908d2640 1 -- 172.21.15.40:0/3645470504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b88032440 con 0x7f8b8c143840 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.129+0000 7f8b908d2640 1 -- 172.21.15.40:0/3645470504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032810 con 0x7f8b8c143840 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 -- 172.21.15.40:0/3645470504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c143840 msgr2=0x7f8b8c0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3645470504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c143840 0x7f8b8c0b3fe0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097d0 tx=0x7f8b8802a210 comp rx=0 tx=0).stop 2024-04-26T21:19:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 -- 172.21.15.40:0/3645470504 shutdown_connections 2024-04-26T21:19:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3645470504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c143840 0x7f8b8c0b3fe0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 -- 172.21.15.40:0/3645470504 >> 172.21.15.40:0/3645470504 conn(0x7f8b8c0cd8a0 msgr2=0x7f8b8c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 -- 172.21.15.40:0/3645470504 shutdown_connections 2024-04-26T21:19:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.130+0000 7f8b93b38640 1 -- 172.21.15.40:0/3645470504 wait complete. 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.131+0000 7f8b93b38640 1 Processor -- start 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.131+0000 7f8b93b38640 1 -- start start 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b8c0b4520 0x7f8b8c073200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 0x7f8b8c0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c15da40 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c15d8c0 con 0x7f8b8c0b4520 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c0cf700 con 0x7f8b8c143840 2024-04-26T21:19:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45860/0 (socket says 172.21.15.40:45860) 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 -- 172.21.15.40:0/3613286381 learned_addr learned my addr 172.21.15.40:0/3613286381 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b918d4640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b8c0b4520 0x7f8b8c073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b920d5640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 0x7f8b8c0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 -- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 msgr2=0x7f8b8c0aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 0x7f8b8c0aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 -- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b8c0b4520 msgr2=0x7f8b8c073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b8c0b4520 0x7f8b8c073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 -- 172.21.15.40:0/3613286381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0cb400 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b920d5640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 0x7f8b8c0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b910d3640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c430 tx=0x7f8b7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c011790 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c017070 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c0cb6e0 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.132+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c011b70 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.133+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c0cbca0 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.134+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c06a290 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.134+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b7c011d10 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.134+0000 7f8b827fc640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 0x7f8b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.135+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f8b7c08b590 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.135+0000 7f8b918d4640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 0x7f8b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.135+0000 7f8b918d4640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 0x7f8b7006edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.139+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b7c059d50 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.324+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f8b8c0b6580 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.329+0000 7f8b827fc640 1 -- 172.21.15.40:0/3613286381 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f8b7c059650 con 0x7f8b8c0b55c0 2024-04-26T21:19:50.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 msgr2=0x7f8b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 0x7f8b7006edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 msgr2=0x7f8b8c073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c430 tx=0x7f8b7c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 shutdown_connections 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8b7006c8f0 0x7f8b7006edb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b8c143840 0x7f8b8c0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b8c0b55c0 0x7f8b8c073740 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 --2- 172.21.15.40:0/3613286381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b8c0b4520 0x7f8b8c073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 >> 172.21.15.40:0/3613286381 conn(0x7f8b8c0cd8a0 msgr2=0x7f8b8c0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 shutdown_connections 2024-04-26T21:19:50.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.331+0000 7f8b93b38640 1 -- 172.21.15.40:0/3613286381 wait complete. 2024-04-26T21:19:50.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-26T21:19:50.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.444+0000 7fecefaa0640 1 Processor -- start 2024-04-26T21:19:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 -- start start 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 0x7fece80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece80d3f50 con 0x7fece80b60b0 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fece815fa60 con 0x7fece8163aa0 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.445+0000 7fecefaa0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fece80b5090 con 0x7fece815f680 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced83c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced03b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 0x7fece80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7fecee03d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fece80b60b0 0x7fece80d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:45198/0 (socket says 172.21.15.40:45198) 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7fecee03d640 1 -- 172.21.15.40:0/450820543 learned_addr learned my addr 172.21.15.40:0/450820543 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7fecec83a640 1 -- 172.21.15.40:0/450820543 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3125281133 0 0) 0x7fece80d3f50 con 0x7fece80b60b0 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced83c640 1 -- 172.21.15.40:0/450820543 >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 msgr2=0x7fece80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced83c640 1 --2- 172.21.15.40:0/450820543 >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 0x7fece80b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced83c640 1 -- 172.21.15.40:0/450820543 >> v1:172.21.15.184:6789/0 conn(0x7fece80b60b0 legacy=0x7fece80d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced83c640 1 -- 172.21.15.40:0/450820543 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece80b5370 con 0x7fece815f680 2024-04-26T21:19:50.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.446+0000 7feced03b640 1 --2- 172.21.15.40:0/450820543 >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 0x7fece80b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.447+0000 7feced83c640 1 --2- 172.21.15.40:0/450820543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80d3820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4fe6dd663feb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.447+0000 7fecec83a640 1 -- 172.21.15.40:0/450820543 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4037070 con 0x7fece815f680 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.447+0000 7fecec83a640 1 -- 172.21.15.40:0/450820543 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece4032440 con 0x7fece815f680 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.447+0000 7fecec83a640 1 -- 172.21.15.40:0/450820543 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4032810 con 0x7fece815f680 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 -- 172.21.15.40:0/450820543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 msgr2=0x7fece80d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 --2- 172.21.15.40:0/450820543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80d3820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402a180 comp rx=0 tx=0).stop 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 -- 172.21.15.40:0/450820543 shutdown_connections 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 --2- 172.21.15.40:0/450820543 >> v2:172.21.15.184:3300/0 conn(0x7fece8163aa0 0x7fece80b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 --2- 172.21.15.40:0/450820543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80d3820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 -- 172.21.15.40:0/450820543 >> 172.21.15.40:0/450820543 conn(0x7fece808eca0 msgr2=0x7fece809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 -- 172.21.15.40:0/450820543 shutdown_connections 2024-04-26T21:19:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.448+0000 7fecefaa0640 1 -- 172.21.15.40:0/450820543 wait complete. 2024-04-26T21:19:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.449+0000 7fecefaa0640 1 Processor -- start 2024-04-26T21:19:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.449+0000 7fecefaa0640 1 -- start start 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece80b60b0 0x7fece80c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fece80d57b0 con 0x7fece8163aa0 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fece80d5630 con 0x7fece80b60b0 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecefaa0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fece80d5930 con 0x7fece815f680 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45874/0 (socket says 172.21.15.40:45874) 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 -- 172.21.15.40:0/144268649 learned_addr learned my addr 172.21.15.40:0/144268649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7feced83c640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece80b60b0 0x7fece80c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7feced03b640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 -- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 msgr2=0x7fece80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 -- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece80b60b0 msgr2=0x7fece80c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece80b60b0 0x7fece80c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.450+0000 7fecee03d640 1 -- 172.21.15.40:0/144268649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece80ca330 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecee03d640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fece000c7a0 tx=0x7fece000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0018070 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece000a640 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece80ca5e0 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece000aab0 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.451+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece80caba0 con 0x7fece8163aa0 2024-04-26T21:19:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.452+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece806a290 con 0x7fece8163aa0 2024-04-26T21:19:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.452+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fece000ac50 con 0x7fece8163aa0 2024-04-26T21:19:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.453+0000 7fecde7fc640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.453+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7fece008af40 con 0x7fece8163aa0 2024-04-26T21:19:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.453+0000 7feced83c640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.453+0000 7feced83c640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fece4034000 tx=0x7fece402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.457+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fece0058f10 con 0x7fece8163aa0 2024-04-26T21:19:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.640+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7fece8163700 con 0x7fece8163aa0 2024-04-26T21:19:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.645+0000 7fecde7fc640 1 -- 172.21.15.40:0/144268649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v43) ==== 102+0+0 (secure 0 0 0) 0x7fece005d210 con 0x7fece8163aa0 2024-04-26T21:19:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 msgr2=0x7feccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fece4034000 tx=0x7fece402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 msgr2=0x7fece80cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fece000c7a0 tx=0x7fece000cc60 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 shutdown_connections 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece8163aa0 0x7fece80cc340 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fece815f680 0x7fece80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 --2- 172.21.15.40:0/144268649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece80b60b0 0x7fece80c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 >> 172.21.15.40:0/144268649 conn(0x7fece808eca0 msgr2=0x7fece80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 shutdown_connections 2024-04-26T21:19:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.647+0000 7fecefaa0640 1 -- 172.21.15.40:0/144268649 wait complete. 2024-04-26T21:19:50.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-26T21:19:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.762+0000 7efe72a23640 1 Processor -- start 2024-04-26T21:19:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.762+0000 7efe72a23640 1 -- start start 2024-04-26T21:19:50.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe72a23640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efe6c06db10 0x7efe6c16a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe72a23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6c09c700 con 0x7efe6c0b3c20 2024-04-26T21:19:50.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe72a23640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6c0d1c50 con 0x7efe6c0d0010 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe72a23640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c0b4480 con 0x7efe6c06db10 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe6c0b3c20 0x7efe6c0b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55808/0 (socket says 172.21.15.40:55808) 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6b7fe640 1 -- 172.21.15.40:0/3647868705 learned_addr learned my addr 172.21.15.40:0/3647868705 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213601957 0 0) 0x7efe6c09c700 con 0x7efe6c0b3c20 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe54003520 con 0x7efe6c0b3c20 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe70fc0640 1 --2- 172.21.15.40:0/3647868705 >> v2:172.21.15.184:3300/0 conn(0x7efe6c06db10 0x7efe6c16a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1360303903 0 0) 0x7efe54003520 con 0x7efe6c0b3c20 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 >> v2:172.21.15.184:3300/0 conn(0x7efe6c06db10 msgr2=0x7efe6c16a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 --2- 172.21.15.40:0/3647868705 >> v2:172.21.15.184:3300/0 conn(0x7efe6c06db10 0x7efe6c16a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 >> v1:172.21.15.184:6789/0 conn(0x7efe6c0d0010 legacy=0x7efe6c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c0d1f30 con 0x7efe6c0b3c20 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7efe5c001670 con 0x7efe6c0b3c20 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 >> v1:172.21.15.40:6789/0 conn(0x7efe6c0b3c20 legacy=0x7efe6c0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.763+0000 7efe6affd640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540039b0 0x7efe54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6affd640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe54004800 0x7efe54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6affd640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 0x7efe5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0b4480 con 0x7efe540051c0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe5400c060 con 0x7efe54004800 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe5400c260 con 0x7efe540039b0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 0x7efe5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6bfff640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540039b0 0x7efe54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe6b7fe640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe54004800 0x7efe54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 -- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540039b0 msgr2=0x7efe54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540039b0 0x7efe54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 -- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe54004800 msgr2=0x7efe54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe54004800 0x7efe54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.764+0000 7efe70fc0640 1 -- 172.21.15.40:0/3647868705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe54003520 con 0x7efe540051c0 2024-04-26T21:19:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe70fc0640 1 -- 172.21.15.40:0/3647868705 reap_dead start 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe70fc0640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 0x7efe5400b9c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7efe64009fc0 tx=0x7efe54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db1e9d0a015f6b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64035020 con 0x7efe540051c0 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64003bd0 con 0x7efe540051c0 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe6affd640 1 -- 172.21.15.40:0/3647868705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64009c10 con 0x7efe540051c0 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 -- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 msgr2=0x7efe5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 0x7efe5400b9c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7efe64009fc0 tx=0x7efe54005830 comp rx=0 tx=0).stop 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 -- 172.21.15.40:0/3647868705 shutdown_connections 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 --2- 172.21.15.40:0/3647868705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe540051c0 0x7efe5400b9c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 -- 172.21.15.40:0/3647868705 >> 172.21.15.40:0/3647868705 conn(0x7efe6c09ba40 msgr2=0x7efe6c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 -- 172.21.15.40:0/3647868705 shutdown_connections 2024-04-26T21:19:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.765+0000 7efe72a23640 1 -- 172.21.15.40:0/3647868705 wait complete. 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.766+0000 7efe72a23640 1 Processor -- start 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 -- start start 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540051c0 0x7efe6c0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe6c1f5df0 0x7efe6c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0b2e50 con 0x7efe6c1eee40 2024-04-26T21:19:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0b2cd0 con 0x7efe6c1f5df0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.767+0000 7efe72a23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c0b2fd0 con 0x7efe540051c0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39384/0 (socket says 172.21.15.40:39384) 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 -- 172.21.15.40:0/165395898 learned_addr learned my addr 172.21.15.40:0/165395898 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe6bfff640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540051c0 0x7efe6c0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe6b7fe640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe6c1f5df0 0x7efe6c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 -- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540051c0 msgr2=0x7efe6c0b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540051c0 0x7efe6c0b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 -- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe6c1f5df0 msgr2=0x7efe6c0b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe6c1f5df0 0x7efe6c0b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 -- 172.21.15.40:0/165395898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1ef810 con 0x7efe6c1eee40 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe70fc0640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e230 tx=0x7efe64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64035030 con 0x7efe6c1eee40 2024-04-26T21:19:50.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64043070 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c0b4c40 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6402d070 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.768+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c0b5200 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.769+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c06a290 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.770+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe64005d10 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.770+0000 7efe68ff9640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 0x7efe5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.770+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7efe640ac480 con 0x7efe6c1eee40 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.770+0000 7efe6bfff640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 0x7efe5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.771+0000 7efe6bfff640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 0x7efe5806ede0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.775+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe6407a560 con 0x7efe6c1eee40 2024-04-26T21:19:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.960+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7efe6c0b8f10 con 0x7efe6c1eee40 2024-04-26T21:19:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.963+0000 7efe68ff9640 1 -- 172.21.15.40:0/165395898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v44) ==== 105+0+0 (secure 0 0 0) 0x7efe6407e860 con 0x7efe6c1eee40 2024-04-26T21:19:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 msgr2=0x7efe5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 0x7efe5806ede0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe60009040 comp rx=0 tx=0).stop 2024-04-26T21:19:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 msgr2=0x7efe6c1ef2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e230 tx=0x7efe64009750 comp rx=0 tx=0).stop 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 shutdown_connections 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe5806c920 0x7efe5806ede0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c1eee40 0x7efe6c1ef2d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe6c1f5df0 0x7efe6c0b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 --2- 172.21.15.40:0/165395898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540051c0 0x7efe6c0b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 >> 172.21.15.40:0/165395898 conn(0x7efe6c09ba40 msgr2=0x7efe6c0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 shutdown_connections 2024-04-26T21:19:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:50.965+0000 7efe72a23640 1 -- 172.21.15.40:0/165395898 wait complete. 2024-04-26T21:19:50.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-26T21:19:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.080+0000 7fdce8a9c640 1 Processor -- start 2024-04-26T21:19:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.080+0000 7fdce8a9c640 1 -- start start 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40d0010 0x7fdce406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 0x7fdce40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdce40b3210 0x7fdce40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdce40b4480 con 0x7fdce40b3210 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fdce406e050 con 0x7fdce40b3c20 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce8a9c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdce409c700 con 0x7fdce40d0010 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdce40b3210 0x7fdce40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdce40b3210 0x7fdce40b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39392/0 (socket says 172.21.15.40:39392) 2024-04-26T21:19:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 -- 172.21.15.40:0/263843032 learned_addr learned my addr 172.21.15.40:0/263843032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/263843032 >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 0x7fdce40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce259c640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40d0010 0x7fdce406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 -- 172.21.15.40:0/263843032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40d0010 msgr2=0x7fdce406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40d0010 0x7fdce406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 -- 172.21.15.40:0/263843032 >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 msgr2=0x7fdce40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/263843032 >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 0x7fdce40b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdce2d9d640 1 -- 172.21.15.40:0/263843032 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce40d1c50 con 0x7fdce40b3210 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.081+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/263843032 >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 0x7fdce40b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.082+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40b3210 0x7fdce40b36a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdce4073f10 tx=0x7fdcdc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a266c199903e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.082+0000 7fdce1d9b640 1 -- 172.21.15.40:0/263843032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc036070 con 0x7fdce40b3210 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.082+0000 7fdce1d9b640 1 -- 172.21.15.40:0/263843032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc031600 con 0x7fdce40b3210 2024-04-26T21:19:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.082+0000 7fdce1d9b640 1 -- 172.21.15.40:0/263843032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0319d0 con 0x7fdce40b3210 2024-04-26T21:19:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 -- 172.21.15.40:0/263843032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40b3210 msgr2=0x7fdce40b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40b3210 0x7fdce40b36a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdce4073f10 tx=0x7fdcdc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 -- 172.21.15.40:0/263843032 shutdown_connections 2024-04-26T21:19:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40b3210 0x7fdce40b36a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/263843032 >> v2:172.21.15.184:3300/0 conn(0x7fdce40b3c20 0x7fdce40b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/263843032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40d0010 0x7fdce406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 -- 172.21.15.40:0/263843032 >> 172.21.15.40:0/263843032 conn(0x7fdce409ba40 msgr2=0x7fdce409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 -- 172.21.15.40:0/263843032 shutdown_connections 2024-04-26T21:19:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.083+0000 7fdce8a9c640 1 -- 172.21.15.40:0/263843032 wait complete. 2024-04-26T21:19:51.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.084+0000 7fdce8a9c640 1 Processor -- start 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.084+0000 7fdce8a9c640 1 -- start start 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce40b3210 0x7fdce4157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40b3c20 0x7fdce40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdce415fe90 con 0x7fdce40d0010 2024-04-26T21:19:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdce415fd10 con 0x7fdce40b3210 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce8a9c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fdce4160010 con 0x7fdce40b3c20 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39404/0 (socket says 172.21.15.40:39404) 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 -- 172.21.15.40:0/3012547106 learned_addr learned my addr 172.21.15.40:0/3012547106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40b3c20 0x7fdce40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce259c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce40b3210 0x7fdce4157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 -- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40b3c20 msgr2=0x7fdce40cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40b3c20 0x7fdce40cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 -- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce40b3210 msgr2=0x7fdce4157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce40b3210 0x7fdce4157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 -- 172.21.15.40:0/3012547106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce40b65c0 con 0x7fdce40d0010 2024-04-26T21:19:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.085+0000 7fdce2d9d640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc034040 tx=0x7fdcdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.086+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc041020 con 0x7fdce40d0010 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.086+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0077a0 con 0x7fdce40d0010 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.086+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce40b68a0 con 0x7fdce40d0010 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.086+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc036040 con 0x7fdce40d0010 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.086+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce40b7820 con 0x7fdce40d0010 2024-04-26T21:19:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.087+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce406a290 con 0x7fdce40d0010 2024-04-26T21:19:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.087+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcdc044020 con 0x7fdce40d0010 2024-04-26T21:19:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.088+0000 7fdcdaffd640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 0x7fdcc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.088+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7fdcdc0ac490 con 0x7fdce40d0010 2024-04-26T21:19:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.088+0000 7fdce259c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 0x7fdcc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.089+0000 7fdce259c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 0x7fdcc406edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400a9a0 tx=0x7fdcd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.092+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcdc04b030 con 0x7fdce40d0010 2024-04-26T21:19:51.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.277+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7fdce4068e00 con 0x7fdce40d0010 2024-04-26T21:19:51.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.282+0000 7fdcdaffd640 1 -- 172.21.15.40:0/3012547106 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v45) ==== 107+0+0 (secure 0 0 0) 0x7fdcdc07a560 con 0x7fdce40d0010 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 msgr2=0x7fdcc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 0x7fdcc406edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400a9a0 tx=0x7fdcd4008040 comp rx=0 tx=0).stop 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 msgr2=0x7fdce40b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc034040 tx=0x7fdcdc004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 shutdown_connections 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fdcc406c8f0 0x7fdcc406edb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdce40d0010 0x7fdce40b16e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdce40b3c20 0x7fdce40cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 --2- 172.21.15.40:0/3012547106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdce40b3210 0x7fdce4157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.284+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 >> 172.21.15.40:0/3012547106 conn(0x7fdce409ba40 msgr2=0x7fdce41605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.285+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 shutdown_connections 2024-04-26T21:19:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.285+0000 7fdce8a9c640 1 -- 172.21.15.40:0/3012547106 wait complete. 2024-04-26T21:19:51.296 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-04-26T21:19:51.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.396+0000 7f2d0527e640 1 Processor -- start 2024-04-26T21:19:51.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.396+0000 7f2d0527e640 1 -- start start 2024-04-26T21:19:51.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2d0527e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 0x7f2d000b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2d0527e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000b4530 con 0x7f2d000b61a0 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2d0527e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000b4810 con 0x7f2d00145980 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2d0527e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d00160ac0 con 0x7f2d00160630 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cfe7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d000b61a0 0x7f2d000720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55816/0 (socket says 172.21.15.40:55816) 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cfe7fc640 1 -- 172.21.15.40:0/2859394073 learned_addr learned my addr 172.21.15.40:0/2859394073 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/2859394073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 0x7f2d000b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150741620 0 0) 0x7f2d000b4530 con 0x7f2d000b61a0 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ce8005730 con 0x7f2d000b61a0 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cff7fe640 1 -- 172.21.15.40:0/2859394073 >> v1:172.21.15.184:6789/0 conn(0x7f2d00145980 legacy=0x7f2d000b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cff7fe640 1 -- 172.21.15.40:0/2859394073 >> v1:172.21.15.40:6789/0 conn(0x7f2d000b61a0 legacy=0x7f2d000720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1376884141 0 0) 0x7f2ce8005730 con 0x7f2d000b61a0 2024-04-26T21:19:51.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.397+0000 7f2cff7fe640 1 -- 172.21.15.40:0/2859394073 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000b4af0 con 0x7f2d00160630 2024-04-26T21:19:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.398+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/2859394073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 0x7f2d000b3de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009dc0 tx=0x7f2cf802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b838609e19bf91d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.398+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4001690 con 0x7f2d00160630 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.398+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8007540 con 0x7f2d00160630 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.398+0000 7f2cfdffb640 1 -- 172.21.15.40:0/2859394073 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80078f0 con 0x7f2d00160630 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 -- 172.21.15.40:0/2859394073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 msgr2=0x7f2d000b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 --2- 172.21.15.40:0/2859394073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 0x7f2d000b3de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009dc0 tx=0x7f2cf802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 -- 172.21.15.40:0/2859394073 shutdown_connections 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 --2- 172.21.15.40:0/2859394073 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d00160630 0x7f2d000b3de0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 -- 172.21.15.40:0/2859394073 >> 172.21.15.40:0/2859394073 conn(0x7f2d000d0d30 msgr2=0x7f2d000d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 -- 172.21.15.40:0/2859394073 shutdown_connections 2024-04-26T21:19:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.399+0000 7f2d0527e640 1 -- 172.21.15.40:0/2859394073 wait complete. 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.400+0000 7f2d0527e640 1 Processor -- start 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.400+0000 7f2d0527e640 1 -- start start 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 0x7f2d0013f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d00160630 0x7f2d000b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000727e0 con 0x7f2d00145980 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000b5140 con 0x7f2d00160630 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2d0527e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d00072960 con 0x7f2d000b61a0 2024-04-26T21:19:51.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2cfe7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2cfe7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39410/0 (socket says 172.21.15.40:39410) 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2cfe7fc640 1 -- 172.21.15.40:0/3317684841 learned_addr learned my addr 172.21.15.40:0/3317684841 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d00160630 0x7f2d000b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.401+0000 7f2cfeffd640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 0x7f2d0013f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 -- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 msgr2=0x7f2d0013f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 0x7f2d0013f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 -- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d00160630 msgr2=0x7f2d000b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d00160630 0x7f2d000b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 -- 172.21.15.40:0/3317684841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0015bc60 con 0x7f2d00145980 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfeffd640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 0x7f2d0013f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2cfe7fc640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c7b0 tx=0x7f2cf400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4018070 con 0x7f2d00145980 2024-04-26T21:19:51.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf40072d0 con 0x7f2d00145980 2024-04-26T21:19:51.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d000b09d0 con 0x7f2d00145980 2024-04-26T21:19:51.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40047e0 con 0x7f2d00145980 2024-04-26T21:19:51.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.402+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d000b0f50 con 0x7f2d00145980 2024-04-26T21:19:51.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.403+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0006a290 con 0x7f2d00145980 2024-04-26T21:19:51.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.404+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf401b070 con 0x7f2d00145980 2024-04-26T21:19:51.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.404+0000 7f2ce77fe640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.404+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f2cf408abd0 con 0x7f2d00145980 2024-04-26T21:19:51.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.404+0000 7f2cfeffd640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.405+0000 7f2cfeffd640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d000b3d90 tx=0x7f2cf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.408+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d0015bc60 con 0x7f2d00145980 2024-04-26T21:19:51.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.663+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f2d00073cc0 con 0x7f2d00145980 2024-04-26T21:19:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.680+0000 7f2ce77fe640 1 -- 172.21.15.40:0/3317684841 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v403) ==== 143+0+0 (secure 0 0 0) 0x7f2cf4058cb0 con 0x7f2d00145980 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.681+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 msgr2=0x7f2cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.681+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d000b3d90 tx=0x7f2cf0009040 comp rx=0 tx=0).stop 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 msgr2=0x7f2d000ab920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c7b0 tx=0x7f2cf400cc70 comp rx=0 tx=0).stop 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 shutdown_connections 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d00160630 0x7f2d000b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d00145980 0x7f2d000ab920 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 --2- 172.21.15.40:0/3317684841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d000b61a0 0x7f2d0013f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 >> 172.21.15.40:0/3317684841 conn(0x7f2d000d0d30 msgr2=0x7f2d001436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 shutdown_connections 2024-04-26T21:19:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.682+0000 7f2d0527e640 1 -- 172.21.15.40:0/3317684841 wait complete. 2024-04-26T21:19:51.694 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-26T21:19:51.694 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:19:51.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:51.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.793+0000 7f2e14485640 1 Processor -- start 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.793+0000 7f2e14485640 1 -- start start 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.794+0000 7f2e14485640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e0c0d3ee0 0x7f2e0c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.794+0000 7f2e14485640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c15f750 0x7f2e0c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.794+0000 7f2e14485640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0c0d03a0 con 0x7f2e0c15f370 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.794+0000 7f2e14485640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c0b4d10 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.794+0000 7f2e14485640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c0d0680 con 0x7f2e0c15f750 2024-04-26T21:19:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e12a22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c15f750 0x7f2e0c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e11a20640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e0c0d3ee0 0x7f2e0c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e12221640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2e0c15f370 0x7f2e0c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2024-04-26T21:19:51.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e12221640 1 -- 172.21.15.40:0/756269356 learned_addr learned my addr 172.21.15.40:0/756269356 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e11a20640 1 -- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c15f750 msgr2=0x7f2e0c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e11a20640 1 --2- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c15f750 0x7f2e0c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e11a20640 1 -- 172.21.15.40:0/756269356 >> v1:172.21.15.184:6789/0 conn(0x7f2e0c15f370 legacy=0x7f2e0c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e11a20640 1 -- 172.21.15.40:0/756269356 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c0d0960 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.795+0000 7f2e1121f640 1 -- 172.21.15.40:0/756269356 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823150564 0 0) 0x7f2e0c0d03a0 con 0x7f2e0c15f370 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.796+0000 7f2e11a20640 1 --2- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c0d3ee0 0x7f2e0c0cfd70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0097a0 tx=0x7f2dfc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33fe682536d603f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.796+0000 7f2e1121f640 1 -- 172.21.15.40:0/756269356 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc037070 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.796+0000 7f2e1121f640 1 -- 172.21.15.40:0/756269356 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc032440 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.796+0000 7f2e1121f640 1 -- 172.21.15.40:0/756269356 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc032810 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 -- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c0d3ee0 msgr2=0x7f2e0c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 --2- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c0d3ee0 0x7f2e0c0cfd70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0097a0 tx=0x7f2dfc02a180 comp rx=0 tx=0).stop 2024-04-26T21:19:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 -- 172.21.15.40:0/756269356 shutdown_connections 2024-04-26T21:19:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 --2- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c15f750 0x7f2e0c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 --2- 172.21.15.40:0/756269356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c0d3ee0 0x7f2e0c0cfd70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 -- 172.21.15.40:0/756269356 >> 172.21.15.40:0/756269356 conn(0x7f2e0c08ec10 msgr2=0x7f2e0c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 -- 172.21.15.40:0/756269356 shutdown_connections 2024-04-26T21:19:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.797+0000 7f2e14485640 1 -- 172.21.15.40:0/756269356 wait complete. 2024-04-26T21:19:51.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.798+0000 7f2e14485640 1 Processor -- start 2024-04-26T21:19:51.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.798+0000 7f2e14485640 1 -- start start 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c0d3ee0 0x7f2e0c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c15f750 0x7f2e0c160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c0b51c0 con 0x7f2e0c15f370 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c0d0f20 con 0x7f2e0c15f750 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e14485640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c0b5340 con 0x7f2e0c0d3ee0 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39422/0 (socket says 172.21.15.40:39422) 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 -- 172.21.15.40:0/203907317 learned_addr learned my addr 172.21.15.40:0/203907317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e12221640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c0d3ee0 0x7f2e0c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e12a22640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c15f750 0x7f2e0c160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 -- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c0d3ee0 msgr2=0x7f2e0c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c0d3ee0 0x7f2e0c15fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 -- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c15f750 msgr2=0x7f2e0c160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c15f750 0x7f2e0c160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.799+0000 7f2e11a20640 1 -- 172.21.15.40:0/203907317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1531d0 con 0x7f2e0c15f370 2024-04-26T21:19:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e11a20640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0098c0 tx=0x7f2dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc037040 con 0x7f2e0c15f370 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc042070 con 0x7f2e0c15f370 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c0c9f30 con 0x7f2e0c15f370 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc030940 con 0x7f2e0c15f370 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.800+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c0ca4f0 con 0x7f2e0c15f370 2024-04-26T21:19:51.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.801+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c06a290 con 0x7f2e0c15f370 2024-04-26T21:19:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.801+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dfc030ae0 con 0x7f2e0c15f370 2024-04-26T21:19:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.802+0000 7f2e02ffd640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 0x7f2df006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.802+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f2dfc0ac080 con 0x7f2e0c15f370 2024-04-26T21:19:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.802+0000 7f2e12221640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 0x7f2df006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.802+0000 7f2e12221640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 0x7f2df006ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2e08000f60 tx=0x7f2e08007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:51.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.806+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2dfc07a0d0 con 0x7f2e0c15f370 2024-04-26T21:19:51.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.992+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e0c161280 con 0x7f2e0c15f370 2024-04-26T21:19:51.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.993+0000 7f2e02ffd640 1 -- 172.21.15.40:0/203907317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+13030 (secure 0 0 0) 0x7f2dfc07e3d0 con 0x7f2e0c15f370 2024-04-26T21:19:51.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:51.994 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":114,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:49.670145+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6836/798777741":"2024-04-27T21:19:46.457736+0000","172.21.15.184:6834/798777741":"2024-04-27T21:19:46.457736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:51.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 msgr2=0x7f2df006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 0x7f2df006ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2e08000f60 tx=0x7f2e08007d50 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 msgr2=0x7f2e0c0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0098c0 tx=0x7f2dfc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 shutdown_connections 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2df006c920 0x7f2df006ede0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e0c15f750 0x7f2e0c160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e0c15f370 0x7f2e0c0cc2c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 --2- 172.21.15.40:0/203907317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e0c0d3ee0 0x7f2e0c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 >> 172.21.15.40:0/203907317 conn(0x7f2e0c08ec10 msgr2=0x7f2e0c15e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 shutdown_connections 2024-04-26T21:19:51.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:51.995+0000 7f2e14485640 1 -- 172.21.15.40:0/203907317 wait complete. 2024-04-26T21:19:52.007 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:52.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:52.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.110+0000 7f0ef6a46640 1 Processor -- start 2024-04-26T21:19:52.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.110+0000 7f0ef6a46640 1 -- start start 2024-04-26T21:19:52.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef6a46640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ef00b6b70 0x7f0ef00720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef6a46640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef00b46a0 con 0x7f0ef006f340 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef6a46640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef00b5db0 con 0x7f0ef00d0300 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef6a46640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef00b6090 con 0x7f0ef00b6b70 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef4fe3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ef00b6b70 0x7f0ef00720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0eeffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0ef006f340 0x7f0ef006f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55822/0 (socket says 172.21.15.40:55822) 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef4fe3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ef00b6b70 0x7f0ef00720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39428/0 (socket says 172.21.15.40:39428) 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0eeffff640 1 -- 172.21.15.40:0/195827470 learned_addr learned my addr 172.21.15.40:0/195827470 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/195827470 >> v1:172.21.15.184:6789/0 conn(0x7f0ef00d0300 legacy=0x7f0ef00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.111+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/195827470 >> v1:172.21.15.40:6789/0 conn(0x7f0ef006f340 legacy=0x7f0ef006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/195827470 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef00b6370 con 0x7f0ef00b6b70 2024-04-26T21:19:52.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0eeeffd640 1 -- 172.21.15.40:0/195827470 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372359632 0 0) 0x7f0ef00b46a0 con 0x7f0ef006f340 2024-04-26T21:19:52.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0ef4fe3640 1 --2- 172.21.15.40:0/195827470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00b6b70 0x7f0ef00720f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009d90 tx=0x7f0ee802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a989e2d8910c7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0eeeffd640 1 -- 172.21.15.40:0/195827470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8007740 con 0x7f0ef00b6b70 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0eeeffd640 1 -- 172.21.15.40:0/195827470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee80078e0 con 0x7f0ef00b6b70 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.112+0000 7f0eeeffd640 1 -- 172.21.15.40:0/195827470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8007cb0 con 0x7f0ef00b6b70 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 -- 172.21.15.40:0/195827470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00b6b70 msgr2=0x7f0ef00720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/195827470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00b6b70 0x7f0ef00720f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009d90 tx=0x7f0ee802a660 comp rx=0 tx=0).stop 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 -- 172.21.15.40:0/195827470 shutdown_connections 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/195827470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00b6b70 0x7f0ef00720f0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 -- 172.21.15.40:0/195827470 >> 172.21.15.40:0/195827470 conn(0x7f0ef00cd8a0 msgr2=0x7f0ef00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 -- 172.21.15.40:0/195827470 shutdown_connections 2024-04-26T21:19:52.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.113+0000 7f0ef6a46640 1 -- 172.21.15.40:0/195827470 wait complete. 2024-04-26T21:19:52.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.114+0000 7f0ef6a46640 1 Processor -- start 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.114+0000 7f0ef6a46640 1 -- start start 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 0x7f0ef01f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 0x7f0ef0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0142bb0 con 0x7f0ef00d0300 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0142a30 con 0x7f0ef00b6b70 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef6a46640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef0072630 con 0x7f0ef006f340 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39434/0 (socket says 172.21.15.40:39434) 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/3742147297 learned_addr learned my addr 172.21.15.40:0/3742147297 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0eeffff640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 0x7f0ef01f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0eef7fe640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 0x7f0ef0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 msgr2=0x7f0ef01f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 0x7f0ef01f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.115+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 msgr2=0x7f0ef0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0ef4fe3640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 0x7f0ef0158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0ef4fe3640 1 -- 172.21.15.40:0/3742147297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01f95e0 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0eeffff640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 0x7f0ef01f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0eef7fe640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 0x7f0ef0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0ef4fe3640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009eb0 tx=0x7f0ee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8040070 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee8031c40 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8003990 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef015a890 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.116+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef015ab30 con 0x7f0ef00d0300 2024-04-26T21:19:52.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.117+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef006a290 con 0x7f0ef00d0300 2024-04-26T21:19:52.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.117+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ee8039040 con 0x7f0ef00d0300 2024-04-26T21:19:52.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.118+0000 7f0eecff9640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 0x7f0edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.118+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f0ee80acb40 con 0x7f0ef00d0300 2024-04-26T21:19:52.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.118+0000 7f0eeffff640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 0x7f0edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.119+0000 7f0eeffff640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 0x7f0edc06edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4000f60 tx=0x7f0ee4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.122+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ef01f95e0 con 0x7f0ef00d0300 2024-04-26T21:19:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.307+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ef00b46a0 con 0x7f0ef00d0300 2024-04-26T21:19:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.308+0000 7f0eecff9640 1 -- 172.21.15.40:0/3742147297 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f0ee807b300 con 0x7f0ef00d0300 2024-04-26T21:19:52.309 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.309+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 msgr2=0x7f0edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.309+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 0x7f0edc06edb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4000f60 tx=0x7f0ee4009040 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.309+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 msgr2=0x7f0ef00ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.309+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009eb0 tx=0x7f0ee8004a40 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 shutdown_connections 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0edc06c8f0 0x7f0edc06edb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef00d0300 0x7f0ef00ceca0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef00b6b70 0x7f0ef0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 --2- 172.21.15.40:0/3742147297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ef006f340 0x7f0ef01f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 >> 172.21.15.40:0/3742147297 conn(0x7f0ef00cd8a0 msgr2=0x7f0ef0140350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 shutdown_connections 2024-04-26T21:19:52.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.310+0000 7f0ef6a46640 1 -- 172.21.15.40:0/3742147297 wait complete. 2024-04-26T21:19:52.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-04-26T21:19:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.421+0000 7fb900a83640 1 Processor -- start 2024-04-26T21:19:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.421+0000 7fb900a83640 1 -- start start 2024-04-26T21:19:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb900a83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0cf2e0 0x7fb8fc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb900a83640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8fc0d11b0 0x7fb8fc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb900a83640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8fc0d1740 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb900a83640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8fc06e710 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb900a83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8fc0d1a20 con 0x7fb8fc0cf2e0 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8fc0d11b0 0x7fb8fc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8fad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8fc06e2c0 0x7fb8fc15ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55832/0 (socket says 172.21.15.40:55832) 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8fad9d640 1 -- 172.21.15.40:0/4041960073 learned_addr learned my addr 172.21.15.40:0/4041960073 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8fa59c640 1 --2- 172.21.15.40:0/4041960073 >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0cf2e0 0x7fb8fc06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f3fff640 1 -- 172.21.15.40:0/4041960073 >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0cf2e0 msgr2=0x7fb8fc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4041960073 >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0cf2e0 0x7fb8fc06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f3fff640 1 -- 172.21.15.40:0/4041960073 >> v1:172.21.15.40:6789/0 conn(0x7fb8fc06e2c0 legacy=0x7fb8fc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f3fff640 1 -- 172.21.15.40:0/4041960073 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc0d1d00 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.422+0000 7fb8f9d9b640 1 -- 172.21.15.40:0/4041960073 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277896451 0 0) 0x7fb8fc0d1740 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.423+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4041960073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc0d11b0 0x7fb8fc06dd80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40097a0 tx=0x7fb8e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34216da82d025e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.423+0000 7fb8f9d9b640 1 -- 172.21.15.40:0/4041960073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4037070 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.423+0000 7fb8f9d9b640 1 -- 172.21.15.40:0/4041960073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e4032440 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.423+0000 7fb8f9d9b640 1 -- 172.21.15.40:0/4041960073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4032810 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 -- 172.21.15.40:0/4041960073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc0d11b0 msgr2=0x7fb8fc06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 --2- 172.21.15.40:0/4041960073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc0d11b0 0x7fb8fc06dd80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40097a0 tx=0x7fb8e402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 -- 172.21.15.40:0/4041960073 shutdown_connections 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 --2- 172.21.15.40:0/4041960073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc0d11b0 0x7fb8fc06dd80 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 --2- 172.21.15.40:0/4041960073 >> v2:172.21.15.184:3300/0 conn(0x7fb8fc0cf2e0 0x7fb8fc06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 -- 172.21.15.40:0/4041960073 >> 172.21.15.40:0/4041960073 conn(0x7fb8fc0b4a60 msgr2=0x7fb8fc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 -- 172.21.15.40:0/4041960073 shutdown_connections 2024-04-26T21:19:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.424+0000 7fb900a83640 1 -- 172.21.15.40:0/4041960073 wait complete. 2024-04-26T21:19:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.425+0000 7fb900a83640 1 Processor -- start 2024-04-26T21:19:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.425+0000 7fb900a83640 1 -- start start 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 0x7fb8fc0a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc0d11b0 0x7fb8fc1f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc06da10 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0d22c0 con 0x7fb8fc0cf2e0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb900a83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc0cf6c0 con 0x7fb8fc0d11b0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39466/0 (socket says 172.21.15.40:39466) 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 -- 172.21.15.40:0/4181078683 learned_addr learned my addr 172.21.15.40:0/4181078683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 -- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc0d11b0 msgr2=0x7fb8fc1f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc0d11b0 0x7fb8fc1f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 -- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 msgr2=0x7fb8fc0a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 0x7fb8fc0a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 0x7fb8fc0a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 -- 172.21.15.40:0/4181078683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc1f8de0 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 0x7fb8fc0a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.426+0000 7fb8fa59c640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00d800 tx=0x7fb8ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.427+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00b6c0 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.427+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec019070 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.427+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00ba80 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.427+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc1f90c0 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.427+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc09fdb0 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.428+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc06a290 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.428+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8ec00bc20 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.428+0000 7fb8f2ffd640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 0x7fb8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.429+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7fb8ec08bcb0 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.429+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 0x7fb8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:52.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.429+0000 7fb8f3fff640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 0x7fb8dc06edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4034000 tx=0x7fb8e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:52.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.433+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8ec01c360 con 0x7fb8fc06e2c0 2024-04-26T21:19:52.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:52.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7fb8fc0b8ef0 con 0x7fb8fc06e2c0 2024-04-26T21:19:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.685+0000 7fb8f2ffd640 1 -- 172.21.15.40:0/4181078683 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v405) ==== 154+0+0 (secure 0 0 0) 0x7fb8ec05a470 con 0x7fb8fc06e2c0 2024-04-26T21:19:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 msgr2=0x7fb8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 0x7fb8dc06edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4034000 tx=0x7fb8e402e000 comp rx=0 tx=0).stop 2024-04-26T21:19:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 msgr2=0x7fb8fc15db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00d800 tx=0x7fb8ec00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 shutdown_connections 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8dc06c8f0 0x7fb8dc06edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8fc0d11b0 0x7fb8fc1f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8fc0cf2e0 0x7fb8fc0a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 --2- 172.21.15.40:0/4181078683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8fc06e2c0 0x7fb8fc15db00 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 >> 172.21.15.40:0/4181078683 conn(0x7fb8fc0b4a60 msgr2=0x7fb8fc071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 shutdown_connections 2024-04-26T21:19:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.688+0000 7fb900a83640 1 -- 172.21.15.40:0/4181078683 wait complete. 2024-04-26T21:19:53.703 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:19:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.812+0000 7f16fd82a640 1 Processor -- start 2024-04-26T21:19:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.812+0000 7f16fd82a640 1 -- start start 2024-04-26T21:19:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16fd82a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16f8143840 0x7f16f80b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16fd82a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 0x7f16f80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16fd82a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f815dca0 con 0x7f16f80b53f0 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16fd82a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f16f8143c20 con 0x7f16f8143840 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16fd82a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f16f815df80 con 0x7f16f80b4520 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16f8143840 0x7f16f80b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16f8143840 0x7f16f80b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39476/0 (socket says 172.21.15.40:39476) 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 -- 172.21.15.40:0/2107681428 learned_addr learned my addr 172.21.15.40:0/2107681428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 -- 172.21.15.40:0/2107681428 >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 msgr2=0x7f16f80b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f67fc640 1 --2- 172.21.15.40:0/2107681428 >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 0x7f16f80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 --2- 172.21.15.40:0/2107681428 >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 0x7f16f80b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 -- 172.21.15.40:0/2107681428 >> v1:172.21.15.184:6789/0 conn(0x7f16f80b53f0 legacy=0x7f16f8160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f6ffd640 1 -- 172.21.15.40:0/2107681428 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f815e260 con 0x7f16f8143840 2024-04-26T21:19:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.813+0000 7f16f67fc640 1 --2- 172.21.15.40:0/2107681428 >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 0x7f16f80b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.814+0000 7f16f6ffd640 1 --2- 172.21.15.40:0/2107681428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f8143840 0x7f16f80b3fe0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b0a90 tx=0x7f16f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973652af0205da3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.814+0000 7f16f5ffb640 1 -- 172.21.15.40:0/2107681428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0037070 con 0x7f16f8143840 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.814+0000 7f16f5ffb640 1 -- 172.21.15.40:0/2107681428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f0032440 con 0x7f16f8143840 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.814+0000 7f16f5ffb640 1 -- 172.21.15.40:0/2107681428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0032810 con 0x7f16f8143840 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 -- 172.21.15.40:0/2107681428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f8143840 msgr2=0x7f16f80b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 --2- 172.21.15.40:0/2107681428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f8143840 0x7f16f80b3fe0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b0a90 tx=0x7f16f002a230 comp rx=0 tx=0).stop 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 -- 172.21.15.40:0/2107681428 shutdown_connections 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 --2- 172.21.15.40:0/2107681428 >> v2:172.21.15.184:3300/0 conn(0x7f16f80b4520 0x7f16f80b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 --2- 172.21.15.40:0/2107681428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f8143840 0x7f16f80b3fe0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 -- 172.21.15.40:0/2107681428 >> 172.21.15.40:0/2107681428 conn(0x7f16f80cd8a0 msgr2=0x7f16f80cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 -- 172.21.15.40:0/2107681428 shutdown_connections 2024-04-26T21:19:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.815+0000 7f16fd82a640 1 -- 172.21.15.40:0/2107681428 wait complete. 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.816+0000 7f16fd82a640 1 Processor -- start 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.816+0000 7f16fd82a640 1 -- start start 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 0x7f16f813b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16f8143840 0x7f16f813bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f16f815da40 con 0x7f16f80b4520 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16f815d8c0 con 0x7f16f80b53f0 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16fd82a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f16f80d0000 con 0x7f16f8143840 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39478/0 (socket says 172.21.15.40:39478) 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 -- 172.21.15.40:0/379608931 learned_addr learned my addr 172.21.15.40:0/379608931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f67fc640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 0x7f16f813b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f77fe640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16f8143840 0x7f16f813bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 -- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16f8143840 msgr2=0x7f16f813bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16f8143840 0x7f16f813bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 -- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 msgr2=0x7f16f813b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 0x7f16f813b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f6ffd640 1 -- 172.21.15.40:0/379608931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f815ab90 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.817+0000 7f16f67fc640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 0x7f16f813b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16f6ffd640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f16f002a270 tx=0x7f16f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0037040 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f0042070 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f813c320 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0030bb0 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.818+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f813c8e0 con 0x7f16f80b4520 2024-04-26T21:19:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.819+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f806a290 con 0x7f16f80b4520 2024-04-26T21:19:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.819+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16f0047460 con 0x7f16f80b4520 2024-04-26T21:19:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.820+0000 7f16effff640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 0x7f16d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.820+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f16f00ac560 con 0x7f16f80b4520 2024-04-26T21:19:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.820+0000 7f16f67fc640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 0x7f16d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.821+0000 7f16f67fc640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 0x7f16d806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b42c0 tx=0x7f16e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:53.824+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16f007ac90 con 0x7f16f80b4520 2024-04-26T21:19:54.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.009+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f16f80ce420 con 0x7f16f80b4520 2024-04-26T21:19:54.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.012+0000 7f16effff640 1 -- 172.21.15.40:0/379608931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f16f00ac030 con 0x7f16f80b4520 2024-04-26T21:19:54.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 msgr2=0x7f16d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 0x7f16d806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b42c0 tx=0x7f16e0008040 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 msgr2=0x7f16f8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f16f002a270 tx=0x7f16f00046c0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 shutdown_connections 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f16d806c8f0 0x7f16d806edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16f8143840 0x7f16f813bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f80b53f0 0x7f16f813b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 --2- 172.21.15.40:0/379608931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16f80b4520 0x7f16f8159fb0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 >> 172.21.15.40:0/379608931 conn(0x7f16f80cd8a0 msgr2=0x7f16f80d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 shutdown_connections 2024-04-26T21:19:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.014+0000 7f16fd82a640 1 -- 172.21.15.40:0/379608931 wait complete. 2024-04-26T21:19:54.026 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:19:54.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:54.067 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:19:54.109 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:19:54.110 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:54.110 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:54.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:54.126 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:54.126 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:54.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:54.168 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:19:54.168 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:54.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:19:54.183 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:19:54.183 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:19:54.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:19:54.225 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:19:54.225 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:19:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.325+0000 7faa16445640 1 Processor -- start 2024-04-26T21:19:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.325+0000 7faa16445640 1 -- start start 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa16445640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7faa100cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa16445640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa100d3ee0 0x7faa100b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa16445640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa100d03a0 con 0x7faa1015f750 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa16445640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7faa100b4d10 con 0x7faa100d3ee0 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa16445640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa100d0680 con 0x7faa1015f370 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa0f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa1015f750 0x7faa100d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55836/0 (socket says 172.21.15.40:55836) 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.326+0000 7faa0f7fe640 1 -- 172.21.15.40:0/2367039769 learned_addr learned my addr 172.21.15.40:0/2367039769 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550632230 0 0) 0x7faa100d03a0 con 0x7faa1015f750 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9f8003520 con 0x7faa1015f750 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa149e2640 1 --2- 172.21.15.40:0/2367039769 >> v2:172.21.15.184:3300/0 conn(0x7faa100d3ee0 0x7faa100b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0ffff640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7faa100cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 500836565 0 0) 0x7fa9f8003520 con 0x7faa1015f750 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 msgr2=0x7faa100cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7faa100cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 >> v2:172.21.15.184:3300/0 conn(0x7faa100d3ee0 msgr2=0x7faa100b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 --2- 172.21.15.40:0/2367039769 >> v2:172.21.15.184:3300/0 conn(0x7faa100d3ee0 0x7faa100b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa100d0960 con 0x7faa1015f750 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7faa04001670 con 0x7faa1015f750 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 >> v1:172.21.15.40:6789/0 conn(0x7faa1015f750 legacy=0x7faa100d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9f80039b0 0x7fa9f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9f8004800 0x7fa9f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7fa9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa100b4d10 con 0x7fa9f8004800 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa100d0680 con 0x7fa9f80039b0 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f800ba10 con 0x7faa1015f370 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9f8004800 0x7fa9f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0ffff640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9f80039b0 0x7fa9f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa149e2640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7fa9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 -- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 msgr2=0x7fa9f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa1015f370 0x7fa9f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 -- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9f80039b0 msgr2=0x7fa9f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9f80039b0 0x7fa9f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.327+0000 7faa0f7fe640 1 -- 172.21.15.40:0/2367039769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f8003520 con 0x7fa9f8004800 2024-04-26T21:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.328+0000 7faa0f7fe640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9f8004800 0x7fa9f8004c80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c190 tx=0x7fa9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebbdcd024500b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.328+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04007c80 con 0x7fa9f8004800 2024-04-26T21:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.328+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa04005ad0 con 0x7fa9f8004800 2024-04-26T21:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.328+0000 7faa0effd640 1 -- 172.21.15.40:0/2367039769 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04007070 con 0x7fa9f8004800 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 -- 172.21.15.40:0/2367039769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9f8004800 msgr2=0x7fa9f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 --2- 172.21.15.40:0/2367039769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9f8004800 0x7fa9f8004c80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c190 tx=0x7fa9f8005830 comp rx=0 tx=0).stop 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa149e2640 1 -- 172.21.15.40:0/2367039769 reap_dead start 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 -- 172.21.15.40:0/2367039769 shutdown_connections 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 -- 172.21.15.40:0/2367039769 >> 172.21.15.40:0/2367039769 conn(0x7faa1008ec10 msgr2=0x7faa1009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 -- 172.21.15.40:0/2367039769 shutdown_connections 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.329+0000 7faa16445640 1 -- 172.21.15.40:0/2367039769 wait complete. 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.330+0000 7faa16445640 1 Processor -- start 2024-04-26T21:19:54.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.330+0000 7faa16445640 1 -- start start 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9f8004800 0x7faa10150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa101512a0 0x7faa100b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa100cfd70 con 0x7faa100b8760 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa100d0f20 con 0x7faa101512a0 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa16445640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7faa100cfef0 con 0x7fa9f8004800 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39486/0 (socket says 172.21.15.40:39486) 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 -- 172.21.15.40:0/4229731456 learned_addr learned my addr 172.21.15.40:0/4229731456 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa0f7fe640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa101512a0 0x7faa100b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa0ffff640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9f8004800 0x7faa10150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 -- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9f8004800 msgr2=0x7faa10150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9f8004800 0x7faa10150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 -- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa101512a0 msgr2=0x7faa100b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa101512a0 0x7faa100b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.331+0000 7faa149e2640 1 -- 172.21.15.40:0/4229731456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101e6fb0 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa149e2640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7faa100b7cf0 tx=0x7faa0800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08011e60 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08015430 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa101e7290 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08015800 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.332+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa101e8ee0 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.333+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1006a290 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.333+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa080159a0 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.334+0000 7faa0cff9640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.334+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7faa0808c5f0 con 0x7faa100b8760 2024-04-26T21:19:54.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.334+0000 7faa0ffff640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.335+0000 7faa0ffff640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa00009a60 tx=0x7faa00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.338+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa08019030 con 0x7faa100b8760 2024-04-26T21:19:54.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.564+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa100d24d0 con 0x7faa100b8760 2024-04-26T21:19:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.564+0000 7faa0cff9640 1 -- 172.21.15.40:0/4229731456 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3872 (secure 0 0 0) 0x7faa08010040 con 0x7faa100b8760 2024-04-26T21:19:54.565 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:54.565 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2123614289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2123614289},{"type":"v1","addr":"172.21.15.184:6837","nonce":2123614289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/153076255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":153076255},{"type":"v1","addr":"172.21.15.184:6836","nonce":153076255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1982166723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1982166723},{"type":"v1","addr":"172.21.15.40:6834","nonce":1982166723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1115205228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1115205228},{"type":"v1","addr":"172.21.15.40:6835","nonce":1115205228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-26T21:19:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 69 2024-04-26T21:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 msgr2=0x7fa9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa00009a60 tx=0x7faa00009190 comp rx=0 tx=0).stop 2024-04-26T21:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 msgr2=0x7faa100b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7faa100b7cf0 tx=0x7faa0800f580 comp rx=0 tx=0).stop 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 shutdown_connections 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa9fc06c8f0 0x7fa9fc06edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa100b8760 0x7faa100b4480 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa101512a0 0x7faa100b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 --2- 172.21.15.40:0/4229731456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9f8004800 0x7faa10150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 >> 172.21.15.40:0/4229731456 conn(0x7faa1008ec10 msgr2=0x7faa1015e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 shutdown_connections 2024-04-26T21:19:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.566+0000 7faa16445640 1 -- 172.21.15.40:0/4229731456 wait complete. 2024-04-26T21:19:54.578 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:19:54.578 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:19:54.578 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:19:54.579 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:54.579 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:19:54.579 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:19:54.579 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:19:54.579 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:54.579 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:19:54.579 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:19:54.580 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:19:54.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:54.580 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:19:54.580 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:19:54.580 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:19:54.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:19:54.588 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:19:54.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:19:54.589 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:19:54.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:19:54.591 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:19:54.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:19:54.592 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:19:54.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:19:54.630 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:19:54.630 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:54.631 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:19:54.631 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:54.634 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:19:54.634 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:54.635 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:19:54.635 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:54.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:19:54.705 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:19:54.706 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:19:54.714 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:19:54.715 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:19:54.715 INFO:tasks.ceph.mds.b.smithi184.stderr:2024-04-26T21:19:54.714+0000 7f2b644a4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:19:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.741+0000 7f62df196640 1 Processor -- start 2024-04-26T21:19:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.742+0000 7f62df196640 1 -- start start 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.742+0000 7f62df196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.742+0000 7f62df196640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80ceef0 con 0x7f62d80d1510 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62df196640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80cda60 con 0x7f62d809c450 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62df196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62d815d740 con 0x7f62d815d360 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dd733640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f62d80d1510 0x7f62d8070310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55866/0 (socket says 172.21.15.40:55866) 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62d7fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f62d809c450 0x7f62d80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:54052/0 (socket says 172.21.15.40:54052) 2024-04-26T21:19:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dd733640 1 -- 172.21.15.40:0/2536765629 learned_addr learned my addr 172.21.15.40:0/2536765629 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dcf32640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664719357 0 0) 0x7f62d80ceef0 con 0x7f62d80d1510 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62c8003520 con 0x7f62d80d1510 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1448800853 0 0) 0x7f62d80cda60 con 0x7f62d809c450 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d80ceef0 con 0x7f62d809c450 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dcf32640 1 -- 172.21.15.40:0/2536765629 >> v1:172.21.15.184:6789/0 conn(0x7f62d809c450 legacy=0x7f62d80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dcf32640 1 -- 172.21.15.40:0/2536765629 >> v1:172.21.15.40:6789/0 conn(0x7f62d80d1510 legacy=0x7f62d8070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.743+0000 7f62dcf32640 1 -- 172.21.15.40:0/2536765629 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d80d1100 con 0x7f62d815d360 2024-04-26T21:19:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62dcf32640 1 --2- 172.21.15.40:0/2536765629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d80d0b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0097d0 tx=0x7f62cc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc90d5611f180c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc037070 con 0x7f62d815d360 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc032440 con 0x7f62d815d360 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62d77fe640 1 -- 172.21.15.40:0/2536765629 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc032810 con 0x7f62d815d360 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62df196640 1 -- 172.21.15.40:0/2536765629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 msgr2=0x7f62d80d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.744+0000 7f62df196640 1 --2- 172.21.15.40:0/2536765629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d80d0b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0097d0 tx=0x7f62cc02a210 comp rx=0 tx=0).stop 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.745+0000 7f62df196640 1 -- 172.21.15.40:0/2536765629 shutdown_connections 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.745+0000 7f62df196640 1 --2- 172.21.15.40:0/2536765629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d80d0b20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.745+0000 7f62df196640 1 -- 172.21.15.40:0/2536765629 >> 172.21.15.40:0/2536765629 conn(0x7f62d80b4a40 msgr2=0x7f62d80b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.745+0000 7f62df196640 1 -- 172.21.15.40:0/2536765629 shutdown_connections 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.745+0000 7f62df196640 1 -- 172.21.15.40:0/2536765629 wait complete. 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.746+0000 7f62df196640 1 Processor -- start 2024-04-26T21:19:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.746+0000 7f62df196640 1 -- start start 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d809c450 0x7f62d80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f62d809c8b0 con 0x7f62d80d1510 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80cdfe0 con 0x7f62d809c450 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f62d8070a20 con 0x7f62d815d360 2024-04-26T21:19:54.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39546/0 (socket says 172.21.15.40:39546) 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 -- 172.21.15.40:0/2506145979 learned_addr learned my addr 172.21.15.40:0/2506145979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62dcf32640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d809c450 0x7f62d80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62dd733640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 -- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 msgr2=0x7f62d814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 -- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d809c450 msgr2=0x7f62d80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d809c450 0x7f62d80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 -- 172.21.15.40:0/2506145979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d814c350 con 0x7f62d80d1510 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d7fff640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c500 tx=0x7f62c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4018070 con 0x7f62d80d1510 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4007500 con 0x7f62d80d1510 2024-04-26T21:19:54.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004a20 con 0x7f62d80d1510 2024-04-26T21:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d814c630 con 0x7f62d80d1510 2024-04-26T21:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.747+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d814cbf0 con 0x7f62d80d1510 2024-04-26T21:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.748+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d80c6890 con 0x7f62d80d1510 2024-04-26T21:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.749+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62c4004bc0 con 0x7f62d80d1510 2024-04-26T21:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.749+0000 7f62d57fa640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.749+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f62c408aae0 con 0x7f62d80d1510 2024-04-26T21:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.749+0000 7f62dcf32640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.750+0000 7f62dcf32640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 0x7f62bc06edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f62cc034000 tx=0x7f62cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.754+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62c4058d10 con 0x7f62d80d1510 2024-04-26T21:19:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:54.948+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f62d80693d0 con 0x7f62d80d1510 2024-04-26T21:19:55.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.697+0000 7f62d57fa640 1 -- 172.21.15.40:0/2506145979 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) ==== 97+0+0 (secure 0 0 0) 0x7f62c405d010 con 0x7f62d80d1510 2024-04-26T21:19:55.698 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 msgr2=0x7f62bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 0x7f62bc06edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f62cc034000 tx=0x7f62cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 msgr2=0x7f62d815fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c500 tx=0x7f62c400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 shutdown_connections 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d815d360 0x7f62d814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62d80d1510 0x7f62d815fc70 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 --2- 172.21.15.40:0/2506145979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d809c450 0x7f62d80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 >> 172.21.15.40:0/2506145979 conn(0x7f62d80b4a40 msgr2=0x7f62d80c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.700+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 shutdown_connections 2024-04-26T21:19:55.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.701+0000 7f62df196640 1 -- 172.21.15.40:0/2506145979 wait complete. 2024-04-26T21:19:55.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:19:55.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.822+0000 7fcebc7a5640 1 Processor -- start 2024-04-26T21:19:55.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.822+0000 7fcebc7a5640 1 -- start start 2024-04-26T21:19:55.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fcebc7a5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fceb40cf2e0 0x7fceb406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fcebc7a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fceb40d11b0 0x7fceb406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fcebc7a5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb40d1740 con 0x7fceb406e2c0 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fcebc7a5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fceb406e710 con 0x7fceb40d11b0 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fcebc7a5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fceb40d1a20 con 0x7fceb40cf2e0 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fceb40d11b0 0x7fceb406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fceb40d11b0 0x7fceb406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39548/0 (socket says 172.21.15.40:39548) 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 -- 172.21.15.40:0/3251692682 learned_addr learned my addr 172.21.15.40:0/3251692682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceba541640 1 --2- 172.21.15.40:0/3251692682 >> v2:172.21.15.184:3300/0 conn(0x7fceb40cf2e0 0x7fceb406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 -- 172.21.15.40:0/3251692682 >> v2:172.21.15.184:3300/0 conn(0x7fceb40cf2e0 msgr2=0x7fceb406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 --2- 172.21.15.40:0/3251692682 >> v2:172.21.15.184:3300/0 conn(0x7fceb40cf2e0 0x7fceb406d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.823+0000 7fceb9d40640 1 -- 172.21.15.40:0/3251692682 >> v1:172.21.15.184:6789/0 conn(0x7fceb406e2c0 legacy=0x7fceb415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:55.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.824+0000 7fceb9d40640 1 -- 172.21.15.40:0/3251692682 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb40d1d00 con 0x7fceb40d11b0 2024-04-26T21:19:55.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.824+0000 7fceb9d40640 1 --2- 172.21.15.40:0/3251692682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb406dd80 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009870 tx=0x7fcea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83040308ecb88be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.824+0000 7fceb953f640 1 -- 172.21.15.40:0/3251692682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4037070 con 0x7fceb40d11b0 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.824+0000 7fceb953f640 1 -- 172.21.15.40:0/3251692682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea4032440 con 0x7fceb40d11b0 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.824+0000 7fceb953f640 1 -- 172.21.15.40:0/3251692682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4032810 con 0x7fceb40d11b0 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 -- 172.21.15.40:0/3251692682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 msgr2=0x7fceb406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/3251692682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb406dd80 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009870 tx=0x7fcea402a270 comp rx=0 tx=0).stop 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 -- 172.21.15.40:0/3251692682 shutdown_connections 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/3251692682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb406dd80 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/3251692682 >> v2:172.21.15.184:3300/0 conn(0x7fceb40cf2e0 0x7fceb406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 -- 172.21.15.40:0/3251692682 >> 172.21.15.40:0/3251692682 conn(0x7fceb40b4a60 msgr2=0x7fceb40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 -- 172.21.15.40:0/3251692682 shutdown_connections 2024-04-26T21:19:55.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.825+0000 7fcebc7a5640 1 -- 172.21.15.40:0/3251692682 wait complete. 2024-04-26T21:19:55.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.826+0000 7fcebc7a5640 1 Processor -- start 2024-04-26T21:19:55.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.826+0000 7fcebc7a5640 1 -- start start 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 0x7fceb41f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 0x7fceb41f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fceb406da10 con 0x7fceb40d11b0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fceb40d22c0 con 0x7fceb40cf2e0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebc7a5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fceb40cf6c0 con 0x7fceb406e2c0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39560/0 (socket says 172.21.15.40:39560) 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 -- 172.21.15.40:0/2899496305 learned_addr learned my addr 172.21.15.40:0/2899496305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fceb9d40640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 0x7fceb41f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fceba541640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 0x7fceb41f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 -- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 msgr2=0x7fceb41f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 0x7fceb41f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 -- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 msgr2=0x7fceb41f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 0x7fceb41f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fcebad42640 1 -- 172.21.15.40:0/2899496305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb409e290 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fceb9d40640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 0x7fceb41f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.827+0000 7fceba541640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 0x7fceb41f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fcebad42640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c780 tx=0x7fceac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac017070 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceac003da0 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb40ca1e0 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0052b0 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.828+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb40ca7a0 con 0x7fceb40d11b0 2024-04-26T21:19:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.829+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb406a290 con 0x7fceb40d11b0 2024-04-26T21:19:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.829+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceac01a070 con 0x7fceb40d11b0 2024-04-26T21:19:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.830+0000 7fceaaffd640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.830+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7fceac08adc0 con 0x7fceb40d11b0 2024-04-26T21:19:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.830+0000 7fceba541640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.830+0000 7fceba541640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 0x7fce9806ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fceb0008890 tx=0x7fceb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:55.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:55.834+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fceac0595d0 con 0x7fceb40d11b0 2024-04-26T21:19:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.091+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fceb40b47e0 con 0x7fceb40d11b0 2024-04-26T21:19:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.091+0000 7fceaaffd640 1 -- 172.21.15.40:0/2899496305 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7fceac058ed0 con 0x7fceb40d11b0 2024-04-26T21:19:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:19:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:19:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:19:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:19:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:19:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:19:56.106 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:19:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 msgr2=0x7fce9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 0x7fce9806ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fceb0008890 tx=0x7fceb0008040 comp rx=0 tx=0).stop 2024-04-26T21:19:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 msgr2=0x7fceb409d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c780 tx=0x7fceac00cc40 comp rx=0 tx=0).stop 2024-04-26T21:19:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 shutdown_connections 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fceb40d11b0 0x7fceb409d880 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fceb40cf2e0 0x7fceb41f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 --2- 172.21.15.40:0/2899496305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fceb406e2c0 0x7fceb41f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 >> 172.21.15.40:0/2899496305 conn(0x7fceb40b4a60 msgr2=0x7fceb40c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 shutdown_connections 2024-04-26T21:19:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.093+0000 7fcebc7a5640 1 -- 172.21.15.40:0/2899496305 wait complete. 2024-04-26T21:19:56.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.208+0000 7fab90924640 1 Processor -- start 2024-04-26T21:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.208+0000 7fab90924640 1 -- start start 2024-04-26T21:19:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab90924640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab8c148a30 0x7fab8c073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab90924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab8c1636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab90924640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8c0cfa80 con 0x7fab8c163be0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab90924640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c0cfd60 con 0x7fab8c148a30 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab90924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0d0040 con 0x7fab8c160640 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab8ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fab8c163be0 0x7fab8c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55878/0 (socket says 172.21.15.40:55878) 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab8ad9d640 1 -- 172.21.15.40:0/377462248 learned_addr learned my addr 172.21.15.40:0/377462248 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab8a59c640 1 --2- 172.21.15.40:0/377462248 >> v2:172.21.15.184:3300/0 conn(0x7fab8c148a30 0x7fab8c073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab83fff640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab8c1636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2246088005 0 0) 0x7fab8c0cfa80 con 0x7fab8c163be0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab78003520 con 0x7fab8c163be0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1454850013 0 0) 0x7fab78003520 con 0x7fab8c163be0 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 msgr2=0x7fab8c1636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab8c1636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 >> v2:172.21.15.184:3300/0 conn(0x7fab8c148a30 msgr2=0x7fab8c073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 --2- 172.21.15.40:0/377462248 >> v2:172.21.15.184:3300/0 conn(0x7fab8c148a30 0x7fab8c073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c162ea0 con 0x7fab8c163be0 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fab840019f0 con 0x7fab8c163be0 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.209+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 >> v1:172.21.15.40:6789/0 conn(0x7fab8c163be0 legacy=0x7fab8c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0cfd60 con 0x7fab78004d40 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0d0040 con 0x7fab780039b0 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab7800b9c0 con 0x7fab8c160640 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8a59c640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab83fff640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 -- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 msgr2=0x7fab78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8c160640 0x7fab78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 -- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab780039b0 msgr2=0x7fab78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.210+0000 7fab8ad9d640 1 -- 172.21.15.40:0/377462248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab78003520 con 0x7fab78004d40 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fab8400a920 tx=0x7fab78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb6bb8322c94f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8400a4c0 con 0x7fab78004d40 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8402f070 con 0x7fab78004d40 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab89d9b640 1 -- 172.21.15.40:0/377462248 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8402bd00 con 0x7fab78004d40 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 -- 172.21.15.40:0/377462248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab78004d40 msgr2=0x7fab780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 --2- 172.21.15.40:0/377462248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fab8400a920 tx=0x7fab78005830 comp rx=0 tx=0).stop 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab8ad9d640 1 -- 172.21.15.40:0/377462248 reap_dead start 2024-04-26T21:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 -- 172.21.15.40:0/377462248 shutdown_connections 2024-04-26T21:19:56.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 -- 172.21.15.40:0/377462248 >> 172.21.15.40:0/377462248 conn(0x7fab8c0ce8b0 msgr2=0x7fab8c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 -- 172.21.15.40:0/377462248 shutdown_connections 2024-04-26T21:19:56.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.211+0000 7fab90924640 1 -- 172.21.15.40:0/377462248 wait complete. 2024-04-26T21:19:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 Processor -- start 2024-04-26T21:19:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 -- start start 2024-04-26T21:19:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab78004d40 0x7fab8c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab8c0c6dd0 0x7fab8c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c074590 con 0x7fab8c0b6c40 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.213+0000 7fab90924640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c074410 con 0x7fab8c0c6dd0 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab90924640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c148e10 con 0x7fab78004d40 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39576/0 (socket says 172.21.15.40:39576) 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 -- 172.21.15.40:0/2062547055 learned_addr learned my addr 172.21.15.40:0/2062547055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8a59c640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab78004d40 0x7fab8c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab83fff640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab8c0c6dd0 0x7fab8c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 -- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab78004d40 msgr2=0x7fab8c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab78004d40 0x7fab8c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 -- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab8c0c6dd0 msgr2=0x7fab8c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab8c0c6dd0 0x7fab8c0c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 -- 172.21.15.40:0/2062547055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c0c9400 con 0x7fab8c0b6c40 2024-04-26T21:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab8ad9d640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fab8400aa40 tx=0x7fab8c163070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84044070 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab84034470 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8400a4c0 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c0c7e80 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.214+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c0c75e0 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.215+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c06a290 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.216+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab84047070 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.216+0000 7fab82ffd640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 0x7fab6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.217+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7fab840ab7b0 con 0x7fab8c0b6c40 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.217+0000 7fab8a59c640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 0x7fab6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.217+0000 7fab8a59c640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 0x7fab6c06ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0c82c0 tx=0x7fab7c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.221+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab84079f60 con 0x7fab8c0b6c40 2024-04-26T21:19:56.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.407+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab8c0b1300 con 0x7fab8c0b6c40 2024-04-26T21:19:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.408+0000 7fab82ffd640 1 -- 172.21.15.40:0/2062547055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+12901 (secure 0 0 0) 0x7fab84079860 con 0x7fab8c0b6c40 2024-04-26T21:19:56.409 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:56.409 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:55.705263+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 msgr2=0x7fab6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 0x7fab6c06ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0c82c0 tx=0x7fab7c009190 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 msgr2=0x7fab8c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fab8400aa40 tx=0x7fab8c163070 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 shutdown_connections 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.409+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab6c06c920 0x7fab6c06ede0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab8c0b6c40 0x7fab8c0b70c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab8c0c6dd0 0x7fab8c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 --2- 172.21.15.40:0/2062547055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab78004d40 0x7fab8c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 >> 172.21.15.40:0/2062547055 conn(0x7fab8c0ce8b0 msgr2=0x7fab8c160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 shutdown_connections 2024-04-26T21:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.410+0000 7fab90924640 1 -- 172.21.15.40:0/2062547055 wait complete. 2024-04-26T21:19:56.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:19:56.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:19:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.525+0000 7f8d1b452640 1 Processor -- start 2024-04-26T21:19:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.525+0000 7f8d1b452640 1 -- start start 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d1b452640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 0x7f8d1409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d1b452640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8d140b2420 0x7f8d140d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d1b452640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d140cee60 con 0x7f8d140d1770 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d1b452640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8d140d1be0 con 0x7f8d140b2420 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d1b452640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d140ce9d0 con 0x7f8d1415d340 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d199ef640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8d140d1770 0x7f8d140cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55890/0 (socket says 172.21.15.40:55890) 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.526+0000 7f8d199ef640 1 -- 172.21.15.40:0/4097432866 learned_addr learned my addr 172.21.15.40:0/4097432866 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d189ed640 1 --2- 172.21.15.40:0/4097432866 >> v2:172.21.15.184:3300/0 conn(0x7f8d140b2420 0x7f8d140d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d191ee640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 0x7f8d1409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258474043 0 0) 0x7f8d140cee60 con 0x7f8d140d1770 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d00003520 con 0x7f8d140d1770 2024-04-26T21:19:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d189ed640 1 -- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 msgr2=0x7f8d1409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d189ed640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 0x7f8d1409c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2448598630 0 0) 0x7f8d00003520 con 0x7f8d140d1770 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d189ed640 1 -- 172.21.15.40:0/4097432866 >> v1:172.21.15.40:6789/0 conn(0x7f8d140d1770 legacy=0x7f8d140cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d189ed640 1 -- 172.21.15.40:0/4097432866 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d140d0ff0 con 0x7f8d140b2420 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.527+0000 7f8d191ee640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 0x7f8d1409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.528+0000 7f8d189ed640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d140b2420 0x7f8d140d1230 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d040097a0 tx=0x7f8d0402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19fb682728f8ae39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.528+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04037070 con 0x7f8d140b2420 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.528+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0402aba0 con 0x7f8d140b2420 2024-04-26T21:19:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.528+0000 7f8d0bfff640 1 -- 172.21.15.40:0/4097432866 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0019f0 con 0x7f8d140b2420 2024-04-26T21:19:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.528+0000 7f8d1b452640 1 -- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d140b2420 msgr2=0x7f8d140d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d140b2420 0x7f8d140d1230 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d040097a0 tx=0x7f8d0402a110 comp rx=0 tx=0).stop 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 -- 172.21.15.40:0/4097432866 shutdown_connections 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d140b2420 0x7f8d140d1230 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 --2- 172.21.15.40:0/4097432866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d1415d340 0x7f8d1409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 -- 172.21.15.40:0/4097432866 >> 172.21.15.40:0/4097432866 conn(0x7f8d1409ba40 msgr2=0x7f8d1409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 -- 172.21.15.40:0/4097432866 shutdown_connections 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.529+0000 7f8d1b452640 1 -- 172.21.15.40:0/4097432866 wait complete. 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.530+0000 7f8d1b452640 1 Processor -- start 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.530+0000 7f8d1b452640 1 -- start start 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d140d1770 0x7f8d1414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1415d340 0x7f8d14158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140712b0 con 0x7f8d140cf8b0 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140ce660 con 0x7f8d1415d340 2024-04-26T21:19:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d1b452640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1415d720 con 0x7f8d140d1770 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39586/0 (socket says 172.21.15.40:39586) 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 -- 172.21.15.40:0/3429984840 learned_addr learned my addr 172.21.15.40:0/3429984840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d191ee640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d140d1770 0x7f8d1414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d189ed640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1415d340 0x7f8d14158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 -- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d140d1770 msgr2=0x7f8d1414c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d140d1770 0x7f8d1414c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 -- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1415d340 msgr2=0x7f8d14158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1415d340 0x7f8d14158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 -- 172.21.15.40:0/3429984840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d140cbfb0 con 0x7f8d140cf8b0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d199ef640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c001b60 tx=0x7f8d0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c017070 con 0x7f8d140cf8b0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.531+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c012400 con 0x7f8d140cf8b0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.532+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d141f4100 con 0x7f8d140cf8b0 2024-04-26T21:19:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.532+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0127b0 con 0x7f8d140cf8b0 2024-04-26T21:19:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.532+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1414be20 con 0x7f8d140cf8b0 2024-04-26T21:19:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.533+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1414bc50 con 0x7f8d140cf8b0 2024-04-26T21:19:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.533+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d0c01a070 con 0x7f8d140cf8b0 2024-04-26T21:19:56.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.533+0000 7f8d09ffb640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 0x7f8cf806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.533+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7f8d0c08b640 con 0x7f8d140cf8b0 2024-04-26T21:19:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.534+0000 7f8d191ee640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 0x7f8cf806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.534+0000 7f8d191ee640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 0x7f8cf806ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8d14155ca0 tx=0x7f8d10009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.538+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d0c059770 con 0x7f8d140cf8b0 2024-04-26T21:19:56.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.724+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d140693d0 con 0x7f8d140cf8b0 2024-04-26T21:19:56.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.725+0000 7f8d09ffb640 1 -- 172.21.15.40:0/3429984840 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f8d0c05da70 con 0x7f8d140cf8b0 2024-04-26T21:19:56.726 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:19:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 msgr2=0x7f8cf806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 0x7f8cf806ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8d14155ca0 tx=0x7f8d10009190 comp rx=0 tx=0).stop 2024-04-26T21:19:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 msgr2=0x7f8d140cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c001b60 tx=0x7f8d0c00a430 comp rx=0 tx=0).stop 2024-04-26T21:19:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 shutdown_connections 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8cf806c920 0x7f8cf806ede0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d140cf8b0 0x7f8d140cfd40 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1415d340 0x7f8d14158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 --2- 172.21.15.40:0/3429984840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d140d1770 0x7f8d1414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 >> 172.21.15.40:0/3429984840 conn(0x7f8d1409ba40 msgr2=0x7f8d140cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 shutdown_connections 2024-04-26T21:19:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.727+0000 7f8d1b452640 1 -- 172.21.15.40:0/3429984840 wait complete. 2024-04-26T21:19:56.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:19:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.839+0000 7fe8c1abb640 1 Processor -- start 2024-04-26T21:19:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.839+0000 7fe8c1abb640 1 -- start start 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8c1abb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8bc0b61a0 0x7fe8bc160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8c1abb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8bc0b4270 con 0x7fe8bc145980 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8c1abb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8bc0b4530 con 0x7fe8bc0b3de0 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8c1abb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe8bc0b5140 con 0x7fe8bc0b61a0 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8bb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe8bc145980 0x7fe8bc0b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55894/0 (socket says 172.21.15.40:55894) 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8bb7fe640 1 -- 172.21.15.40:0/3841684850 learned_addr learned my addr 172.21.15.40:0/3841684850 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064192035 0 0) 0x7fe8bc0b4270 con 0x7fe8bc145980 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8a4003520 con 0x7fe8bc145980 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.840+0000 7fe8baffd640 1 --2- 172.21.15.40:0/3841684850 >> v2:172.21.15.184:3300/0 conn(0x7fe8bc0b61a0 0x7fe8bc160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4037020889 0 0) 0x7fe8a4003520 con 0x7fe8bc145980 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 >> v2:172.21.15.184:3300/0 conn(0x7fe8bc0b61a0 msgr2=0x7fe8bc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 --2- 172.21.15.40:0/3841684850 >> v2:172.21.15.184:3300/0 conn(0x7fe8bc0b61a0 0x7fe8bc160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 >> v1:172.21.15.184:6789/0 conn(0x7fe8bc0b3de0 legacy=0x7fe8bc0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0b4810 con 0x7fe8bc145980 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe8ac001670 con 0x7fe8bc145980 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 >> v1:172.21.15.40:6789/0 conn(0x7fe8bc145980 legacy=0x7fe8bc0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a4004800 0x7fe8a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8a40051c0 0x7fe8a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0b5140 con 0x7fe8a40039b0 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a400c160 con 0x7fe8a4004800 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a400c360 con 0x7fe8a40051c0 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8baffd640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a4004800 0x7fe8a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bbfff640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8a40051c0 0x7fe8a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 -- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8a40051c0 msgr2=0x7fe8a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8a40051c0 0x7fe8a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 -- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a4004800 msgr2=0x7fe8a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a4004800 0x7fe8a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bb7fe640 1 -- 172.21.15.40:0/3841684850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a4003520 con 0x7fe8a40039b0 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8bbfff640 1 -- 172.21.15.40:0/3841684850 reap_dead start 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.841+0000 7fe8baffd640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a4004800 0x7fe8a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003dd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c240 tx=0x7fe8a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f6b465a1a3e54c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac007cf0 con 0x7fe8a40039b0 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ac0058d0 con 0x7fe8a40039b0 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8ba7fc640 1 -- 172.21.15.40:0/3841684850 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac005d00 con 0x7fe8a40039b0 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8c1abb640 1 -- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 msgr2=0x7fe8a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.842+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003dd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c240 tx=0x7fe8a4005830 comp rx=0 tx=0).stop 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.843+0000 7fe8c1abb640 1 -- 172.21.15.40:0/3841684850 shutdown_connections 2024-04-26T21:19:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.843+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/3841684850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003dd0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.843+0000 7fe8c1abb640 1 -- 172.21.15.40:0/3841684850 >> 172.21.15.40:0/3841684850 conn(0x7fe8bc0d0d30 msgr2=0x7fe8bc0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.843+0000 7fe8c1abb640 1 -- 172.21.15.40:0/3841684850 shutdown_connections 2024-04-26T21:19:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.843+0000 7fe8c1abb640 1 -- 172.21.15.40:0/3841684850 wait complete. 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.844+0000 7fe8c1abb640 1 Processor -- start 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.844+0000 7fe8c1abb640 1 -- start start 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a40039b0 0x7fe8bc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8bc1e8770 0x7fe8bc135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc160630 con 0x7fe8bc135b90 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0b4dd0 con 0x7fe8a40039b0 2024-04-26T21:19:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc1607b0 con 0x7fe8bc1e8770 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39596/0 (socket says 172.21.15.40:39596) 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 -- 172.21.15.40:0/4010994634 learned_addr learned my addr 172.21.15.40:0/4010994634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a40039b0 0x7fe8bc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8baffd640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8bc1e8770 0x7fe8bc135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 -- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8bc1e8770 msgr2=0x7fe8bc135650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8bc1e8770 0x7fe8bc135650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 -- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a40039b0 msgr2=0x7fe8bc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a40039b0 0x7fe8bc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 -- 172.21.15.40:0/4010994634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0af1e0 con 0x7fe8bc135b90 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8bbfff640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4007fd0 tx=0x7fe8b400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4003970 con 0x7fe8bc135b90 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b4018070 con 0x7fe8bc135b90 2024-04-26T21:19:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4003d70 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.845+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc0af4c0 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.846+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc0afa80 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.846+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc06a290 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.847+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8b40050d0 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.847+0000 7fe8a3fff640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 0x7fe8a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.848+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7fe8b408b320 con 0x7fe8bc135b90 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.848+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 0x7fe8a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.848+0000 7fe8bb7fe640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 0x7fe8a806edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac033000 tx=0x7fe8ac009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:56.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:56.852+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8b4059450 con 0x7fe8bc135b90 2024-04-26T21:19:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.037+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe8bc1e8e00 con 0x7fe8bc135b90 2024-04-26T21:19:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.037+0000 7fe8a3fff640 1 -- 172.21.15.40:0/4010994634 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) ==== 84+0+65 (secure 0 0 0) 0x7fe8b405d750 con 0x7fe8bc135b90 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:19:57.039 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:19:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 msgr2=0x7fe8a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 0x7fe8a806edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac033000 tx=0x7fe8ac009030 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 msgr2=0x7fe8bc136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4007fd0 tx=0x7fe8b400ec10 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 shutdown_connections 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe8a806c8f0 0x7fe8a806edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8bc135b90 0x7fe8bc136020 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8bc1e8770 0x7fe8bc135650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 --2- 172.21.15.40:0/4010994634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8a40039b0 0x7fe8bc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 >> 172.21.15.40:0/4010994634 conn(0x7fe8bc0d0d30 msgr2=0x7fe8bc06ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 shutdown_connections 2024-04-26T21:19:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.039+0000 7fe8c1abb640 1 -- 172.21.15.40:0/4010994634 wait complete. 2024-04-26T21:19:57.051 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:19:57.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:19:57.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.155+0000 7f0b2b893640 1 Processor -- start 2024-04-26T21:19:57.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b2b893640 1 -- start start 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b2b893640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b240d2140 0x7f0b240d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b2b893640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240d33a0 con 0x7f0b240b6020 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b2b893640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240d3680 con 0x7f0b24162a10 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b2b893640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240b4970 con 0x7f0b240d2140 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b28e2e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b240d2140 0x7f0b240d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b29e30640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0b240b6020 0x7f0b240cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55900/0 (socket says 172.21.15.40:55900) 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b28e2e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b240d2140 0x7f0b240d2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39598/0 (socket says 172.21.15.40:39598) 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.156+0000 7f0b29e30640 1 -- 172.21.15.40:0/4291181103 learned_addr learned my addr 172.21.15.40:0/4291181103 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.157+0000 7f0b28e2e640 1 -- 172.21.15.40:0/4291181103 >> v1:172.21.15.184:6789/0 conn(0x7f0b24162a10 legacy=0x7f0b240d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.157+0000 7f0b28e2e640 1 -- 172.21.15.40:0/4291181103 >> v1:172.21.15.40:6789/0 conn(0x7f0b240b6020 legacy=0x7f0b240cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.157+0000 7f0b28e2e640 1 -- 172.21.15.40:0/4291181103 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240d3960 con 0x7f0b240d2140 2024-04-26T21:19:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.157+0000 7f0b23fff640 1 -- 172.21.15.40:0/4291181103 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869882970 0 0) 0x7f0b240d33a0 con 0x7f0b240b6020 2024-04-26T21:19:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.157+0000 7f0b28e2e640 1 --2- 172.21.15.40:0/4291181103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240d2140 0x7f0b240d2dc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0b180097a0 tx=0x7f0b1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51465aa7febb8687 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b23fff640 1 -- 172.21.15.40:0/4291181103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b240d2140 2024-04-26T21:19:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b23fff640 1 -- 172.21.15.40:0/4291181103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b240d2140 2024-04-26T21:19:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b23fff640 1 -- 172.21.15.40:0/4291181103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b240d2140 2024-04-26T21:19:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 -- 172.21.15.40:0/4291181103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240d2140 msgr2=0x7f0b240d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4291181103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240d2140 0x7f0b240d2dc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0b180097a0 tx=0x7f0b1802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 -- 172.21.15.40:0/4291181103 shutdown_connections 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4291181103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240d2140 0x7f0b240d2dc0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 -- 172.21.15.40:0/4291181103 >> 172.21.15.40:0/4291181103 conn(0x7f0b2409e530 msgr2=0x7f0b240d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 -- 172.21.15.40:0/4291181103 shutdown_connections 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.158+0000 7f0b2b893640 1 -- 172.21.15.40:0/4291181103 wait complete. 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.159+0000 7f0b2b893640 1 Processor -- start 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.159+0000 7f0b2b893640 1 -- start start 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 0x7f0b240cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b24162a10 0x7f0b240cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24162df0 con 0x7f0b240b6020 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d3f20 con 0x7f0b240d2140 2024-04-26T21:19:57.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2b893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240b6600 con 0x7f0b24162a10 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2962f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2962f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39604/0 (socket says 172.21.15.40:39604) 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.160+0000 7f0b2962f640 1 -- 172.21.15.40:0/4095534305 learned_addr learned my addr 172.21.15.40:0/4095534305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 -- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b24162a10 msgr2=0x7f0b240cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b24162a10 0x7f0b240cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 -- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 msgr2=0x7f0b240cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b28e2e640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 0x7f0b240cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 0x7f0b240cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 -- 172.21.15.40:0/4095534305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240c8ec0 con 0x7f0b240b6020 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b28e2e640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 0x7f0b240cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2962f640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00b3f0 tx=0x7f0b1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c015020 con 0x7f0b240b6020 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1c011830 con 0x7f0b240b6020 2024-04-26T21:19:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b240c91a0 con 0x7f0b240b6020 2024-04-26T21:19:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c011c20 con 0x7f0b240b6020 2024-04-26T21:19:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.161+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b240c9760 con 0x7f0b240b6020 2024-04-26T21:19:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.162+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2406a290 con 0x7f0b240b6020 2024-04-26T21:19:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.163+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b1c018020 con 0x7f0b240b6020 2024-04-26T21:19:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.163+0000 7f0b21ffb640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 0x7f0b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.163+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7f0b1c08b440 con 0x7f0b240b6020 2024-04-26T21:19:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.163+0000 7f0b28e2e640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 0x7f0b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.164+0000 7f0b28e2e640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 0x7f0b1006edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.167+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b1c01f030 con 0x7f0b240b6020 2024-04-26T21:19:57.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.350+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0b240b6ac0 con 0x7f0b240b6020 2024-04-26T21:19:57.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.703+0000 7f0b21ffb640 1 -- 172.21.15.40:0/4095534305 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) ==== 138+0+0 (secure 0 0 0) 0x7f0b1c0172c0 con 0x7f0b240b6020 2024-04-26T21:19:57.704 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 msgr2=0x7f0b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 0x7f0b1006edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).stop 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 msgr2=0x7f0b24158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00b3f0 tx=0x7f0b1c00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 shutdown_connections 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b1006c8f0 0x7f0b1006edb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b24162a10 0x7f0b240cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b240d2140 0x7f0b240cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 --2- 172.21.15.40:0/4095534305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b240b6020 0x7f0b24158270 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 >> 172.21.15.40:0/4095534305 conn(0x7f0b2409e530 msgr2=0x7f0b2406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 shutdown_connections 2024-04-26T21:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.706+0000 7f0b2b893640 1 -- 172.21.15.40:0/4095534305 wait complete. 2024-04-26T21:19:57.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:19:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.830+0000 7fcbf850a640 1 Processor -- start 2024-04-26T21:19:57.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.830+0000 7fcbf850a640 1 -- start start 2024-04-26T21:19:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf850a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbf00b3620 0x7fcbf0160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf850a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbf009c150 0x7fcbf006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf850a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf00b46f0 con 0x7fcbf00b3ac0 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf850a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf006d3b0 con 0x7fcbf009c150 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf850a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf00d1f60 con 0x7fcbf00b3620 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbf009c150 0x7fcbf006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf5aa5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcbf00b3ac0 0x7fcbf009ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55906/0 (socket says 172.21.15.40:55906) 2024-04-26T21:19:57.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf5aa5640 1 -- 172.21.15.40:0/153882018 learned_addr learned my addr 172.21.15.40:0/153882018 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcbf009c150 0x7fcbf006db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39610/0 (socket says 172.21.15.40:39610) 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf62a6640 1 --2- 172.21.15.40:0/153882018 >> v2:172.21.15.184:3300/0 conn(0x7fcbf00b3620 0x7fcbf0160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 -- 172.21.15.40:0/153882018 >> v2:172.21.15.184:3300/0 conn(0x7fcbf00b3620 msgr2=0x7fcbf0160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 --2- 172.21.15.40:0/153882018 >> v2:172.21.15.184:3300/0 conn(0x7fcbf00b3620 0x7fcbf0160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 -- 172.21.15.40:0/153882018 >> v1:172.21.15.40:6789/0 conn(0x7fcbf00b3ac0 legacy=0x7fcbf009ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf52a4640 1 -- 172.21.15.40:0/153882018 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197211436 0 0) 0x7fcbe0001110 con 0x7fcbf00b3ac0 2024-04-26T21:19:57.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.831+0000 7fcbf6aa7640 1 -- 172.21.15.40:0/153882018 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf00d2240 con 0x7fcbf009c150 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.832+0000 7fcbf6aa7640 1 --2- 172.21.15.40:0/153882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf006db00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00736a0 tx=0x7fcbe802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7c80f471fcd03c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.832+0000 7fcbf52a4640 1 -- 172.21.15.40:0/153882018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8007740 con 0x7fcbf009c150 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.832+0000 7fcbf52a4640 1 -- 172.21.15.40:0/153882018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe80078e0 con 0x7fcbf009c150 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.832+0000 7fcbf52a4640 1 -- 172.21.15.40:0/153882018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8007cb0 con 0x7fcbf009c150 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 -- 172.21.15.40:0/153882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 msgr2=0x7fcbf006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 --2- 172.21.15.40:0/153882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf006db00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00736a0 tx=0x7fcbe802a680 comp rx=0 tx=0).stop 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 -- 172.21.15.40:0/153882018 shutdown_connections 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 --2- 172.21.15.40:0/153882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf006db00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 --2- 172.21.15.40:0/153882018 >> v2:172.21.15.184:3300/0 conn(0x7fcbf00b3620 0x7fcbf0160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 -- 172.21.15.40:0/153882018 >> 172.21.15.40:0/153882018 conn(0x7fcbf00b4ab0 msgr2=0x7fcbf00b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 -- 172.21.15.40:0/153882018 shutdown_connections 2024-04-26T21:19:57.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.833+0000 7fcbf850a640 1 -- 172.21.15.40:0/153882018 wait complete. 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.834+0000 7fcbf850a640 1 Processor -- start 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.834+0000 7fcbf850a640 1 -- start start 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf00b3620 0x7fcbf009cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf00b3ac0 0x7fcbf009d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf00b3ff0 con 0x7fcbf009c150 2024-04-26T21:19:57.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf006d600 con 0x7fcbf00b3ac0 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf00b4170 con 0x7fcbf00b3620 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39624/0 (socket says 172.21.15.40:39624) 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 -- 172.21.15.40:0/300246680 learned_addr learned my addr 172.21.15.40:0/300246680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf5aa5640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf00b3620 0x7fcbf009cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf6aa7640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf00b3ac0 0x7fcbf009d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 -- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf00b3620 msgr2=0x7fcbf009cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf00b3620 0x7fcbf009cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 -- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf00b3ac0 msgr2=0x7fcbf009d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf00b3ac0 0x7fcbf009d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 -- 172.21.15.40:0/300246680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf009ede0 con 0x7fcbf009c150 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf62a6640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00d5f0 tx=0x7fcbec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0125f0 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec019070 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf00c9200 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0129d0 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.835+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf009f360 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.836+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf006a290 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.837+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbec012b70 con 0x7fcbf009c150 2024-04-26T21:19:57.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.837+0000 7fcbe6ffd640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 0x7fcbd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:57.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.837+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5227+0+0 (secure 0 0 0) 0x7fcbec08be40 con 0x7fcbf009c150 2024-04-26T21:19:57.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.838+0000 7fcbf5aa5640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 0x7fcbd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:57.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.838+0000 7fcbf5aa5640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 0x7fcbd406edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcbf01f8c00 tx=0x7fcbe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:57.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:57.842+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbec05a3d0 con 0x7fcbf009c150 2024-04-26T21:19:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.028+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcbf00745c0 con 0x7fcbf009c150 2024-04-26T21:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.706+0000 7fcbe6ffd640 1 -- 172.21.15.40:0/300246680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) ==== 170+0+0 (secure 0 0 0) 0x7fcbec059cd0 con 0x7fcbf009c150 2024-04-26T21:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 msgr2=0x7fcbd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 0x7fcbd406edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcbf01f8c00 tx=0x7fcbe0009040 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 msgr2=0x7fcbf00b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00d5f0 tx=0x7fcbec00dab0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 shutdown_connections 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcbd406c8f0 0x7fcbd406edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf00b3ac0 0x7fcbf009d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf00b3620 0x7fcbf009cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 --2- 172.21.15.40:0/300246680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbf009c150 0x7fcbf00b26f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.709+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 >> 172.21.15.40:0/300246680 conn(0x7fcbf00b4ab0 msgr2=0x7fcbf0070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.710+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 shutdown_connections 2024-04-26T21:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.710+0000 7fcbf850a640 1 -- 172.21.15.40:0/300246680 wait complete. 2024-04-26T21:19:58.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:19:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.841+0000 7f2d802b8640 1 Processor -- start 2024-04-26T21:19:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.841+0000 7f2d802b8640 1 -- start start 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d802b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 0x7f2d780b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d802b8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d780b55b0 0x7f2d780d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d802b8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780b6aa0 con 0x7f2d7815f990 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d802b8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780d3eb0 con 0x7f2d780b55b0 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d802b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780d4190 con 0x7f2d7815f5b0 2024-04-26T21:19:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d7e855640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d780b55b0 0x7f2d780d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d7e855640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d780b55b0 0x7f2d780d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39632/0 (socket says 172.21.15.40:39632) 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d7e855640 1 -- 172.21.15.40:0/4233804285 learned_addr learned my addr 172.21.15.40:0/4233804285 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.842+0000 7f2d7e054640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 0x7f2d780b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e855640 1 -- 172.21.15.40:0/4233804285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 msgr2=0x7f2d780b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e855640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 0x7f2d780b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e855640 1 -- 172.21.15.40:0/4233804285 >> v1:172.21.15.184:6789/0 conn(0x7f2d7815f990 legacy=0x7f2d780d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e855640 1 -- 172.21.15.40:0/4233804285 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780d0200 con 0x7f2d780b55b0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e054640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 0x7f2d780b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7e855640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780b55b0 0x7f2d780d3340 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e50 tx=0x7f2d7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fee7ae4a40c36d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.843+0000 7f2d7d052640 1 -- 172.21.15.40:0/4233804285 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74036070 con 0x7f2d780b55b0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d7d052640 1 -- 172.21.15.40:0/4233804285 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74031600 con 0x7f2d780b55b0 2024-04-26T21:19:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d7d052640 1 -- 172.21.15.40:0/4233804285 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740319d0 con 0x7f2d780b55b0 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 -- 172.21.15.40:0/4233804285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780b55b0 msgr2=0x7f2d780d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780b55b0 0x7f2d780d3340 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e50 tx=0x7f2d7402a680 comp rx=0 tx=0).stop 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 -- 172.21.15.40:0/4233804285 shutdown_connections 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780b55b0 0x7f2d780d3340 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 --2- 172.21.15.40:0/4233804285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f5b0 0x7f2d780b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 -- 172.21.15.40:0/4233804285 >> 172.21.15.40:0/4233804285 conn(0x7f2d7808ec40 msgr2=0x7f2d7809e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 -- 172.21.15.40:0/4233804285 shutdown_connections 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.844+0000 7f2d802b8640 1 -- 172.21.15.40:0/4233804285 wait complete. 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.845+0000 7f2d802b8640 1 Processor -- start 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.845+0000 7f2d802b8640 1 -- start start 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d780b55b0 0x7f2d780cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f990 0x7f2d781533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d1290 con 0x7f2d7815f5b0 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d1110 con 0x7f2d780b55b0 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d802b8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780d3880 con 0x7f2d7815f990 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d7d853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d7d853640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39642/0 (socket says 172.21.15.40:39642) 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d7d853640 1 -- 172.21.15.40:0/3315838127 learned_addr learned my addr 172.21.15.40:0/3315838127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d7e855640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f990 0x7f2d781533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.846+0000 7f2d7e054640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d780b55b0 0x7f2d780cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 -- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f990 msgr2=0x7f2d781533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f990 0x7f2d781533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 -- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d780b55b0 msgr2=0x7f2d780cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d780b55b0 0x7f2d780cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 -- 172.21.15.40:0/3315838127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d78160de0 con 0x7f2d7815f5b0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d7d853640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c3c0 tx=0x7f2d6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680052d0 con 0x7f2d7815f5b0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68005490 con 0x7f2d7815f5b0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68007260 con 0x7f2d7815f5b0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d780c9990 con 0x7f2d7815f5b0 2024-04-26T21:19:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.847+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d780c9f50 con 0x7f2d7815f5b0 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.848+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7806a290 con 0x7f2d7815f5b0 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.849+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d68007400 con 0x7f2d7815f5b0 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.849+0000 7f2d6effd640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.849+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5609+0+0 (secure 0 0 0) 0x7f2d6808b470 con 0x7f2d7815f5b0 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.849+0000 7f2d7e054640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.850+0000 7f2d7e054640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2d780bb0c0 tx=0x7f2d70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:58.853+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d78160de0 con 0x7f2d7815f5b0 2024-04-26T21:19:59.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.076+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2d780d60c0 con 0x7f2d7815f5b0 2024-04-26T21:19:59.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.728+0000 7f2d6effd640 1 -- 172.21.15.40:0/3315838127 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v72) ==== 174+0+0 (secure 0 0 0) 0x7f2d68059290 con 0x7f2d7815f5b0 2024-04-26T21:19:59.730 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 18 and data pool 19 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.733+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 msgr2=0x7f2d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.733+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2d780bb0c0 tx=0x7f2d70008040 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.733+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 msgr2=0x7f2d7815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.733+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c3c0 tx=0x7f2d6800c880 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 shutdown_connections 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d7815f990 0x7f2d781533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d7815f5b0 0x7f2d7815d700 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 --2- 172.21.15.40:0/3315838127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d780b55b0 0x7f2d780cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.734+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 >> 172.21.15.40:0/3315838127 conn(0x7f2d7808ec40 msgr2=0x7f2d780b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.736+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 shutdown_connections 2024-04-26T21:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.736+0000 7f2d802b8640 1 -- 172.21.15.40:0/3315838127 wait complete. 2024-04-26T21:19:59.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:19:59.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.868+0000 7f0885bf0640 1 Processor -- start 2024-04-26T21:19:59.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.868+0000 7f0885bf0640 1 -- start start 2024-04-26T21:19:59.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f0885bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08800cf2e0 0x7f088006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f0885bf0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f088006e490 0x7f088015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f0885bf0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08800d1740 con 0x7f08800d11b0 2024-04-26T21:19:59.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f0885bf0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f08800d1a20 con 0x7f088006e490 2024-04-26T21:19:59.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f0885bf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f08800d1d00 con 0x7f08800cf2e0 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f088006e490 0x7f088015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f088006e490 0x7f088015ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39646/0 (socket says 172.21.15.40:39646) 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 -- 172.21.15.40:0/3124413836 learned_addr learned my addr 172.21.15.40:0/3124413836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087f7fe640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08800cf2e0 0x7f088006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 -- 172.21.15.40:0/3124413836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08800cf2e0 msgr2=0x7f088006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08800cf2e0 0x7f088006d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 -- 172.21.15.40:0/3124413836 >> v1:172.21.15.184:6789/0 conn(0x7f08800d11b0 legacy=0x7f088006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087ffff640 1 -- 172.21.15.40:0/3124413836 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800d1fe0 con 0x7f088006e490 2024-04-26T21:19:59.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.869+0000 7f087e7fc640 1 -- 172.21.15.40:0/3124413836 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577675412 0 0) 0x7f08800d1740 con 0x7f08800d11b0 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.870+0000 7f087ffff640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088006e490 0x7f088015ebe0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f0878009e50 tx=0x7f087802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefce7841f97917e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.870+0000 7f087e7fc640 1 -- 172.21.15.40:0/3124413836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878036070 con 0x7f088006e490 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.870+0000 7f087e7fc640 1 -- 172.21.15.40:0/3124413836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0878031600 con 0x7f088006e490 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.870+0000 7f087e7fc640 1 -- 172.21.15.40:0/3124413836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780319d0 con 0x7f088006e490 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 -- 172.21.15.40:0/3124413836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088006e490 msgr2=0x7f088015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088006e490 0x7f088015ebe0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f0878009e50 tx=0x7f087802a710 comp rx=0 tx=0).stop 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 -- 172.21.15.40:0/3124413836 shutdown_connections 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f088006e490 0x7f088015ebe0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 --2- 172.21.15.40:0/3124413836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f08800cf2e0 0x7f088006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 -- 172.21.15.40:0/3124413836 >> 172.21.15.40:0/3124413836 conn(0x7f08800b4a60 msgr2=0x7f08800b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 -- 172.21.15.40:0/3124413836 shutdown_connections 2024-04-26T21:19:59.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.871+0000 7f0885bf0640 1 -- 172.21.15.40:0/3124413836 wait complete. 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.872+0000 7f0885bf0640 1 Processor -- start 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.872+0000 7f0885bf0640 1 -- start start 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 0x7f08800c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08800d11b0 0x7f08800b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08800cf6c0 con 0x7f08800cf2e0 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f088006da10 con 0x7f08800d11b0 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f0885bf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f08800cf840 con 0x7f088006e490 2024-04-26T21:19:59.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39650/0 (socket says 172.21.15.40:39650) 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 -- 172.21.15.40:0/2001803628 learned_addr learned my addr 172.21.15.40:0/2001803628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087f7fe640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 0x7f08800c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087ffff640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08800d11b0 0x7f08800b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 -- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 msgr2=0x7f08800c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 0x7f08800c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 -- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08800d11b0 msgr2=0x7f08800b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08800d11b0 0x7f08800b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087effd640 1 -- 172.21.15.40:0/2001803628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801f8850 con 0x7f08800cf2e0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.873+0000 7f087f7fe640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 0x7f08800c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f087effd640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f087400c410 tx=0x7f087400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874017070 con 0x7f08800cf2e0 2024-04-26T21:19:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740035f0 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801f8b30 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740039e0 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.874+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801f90f0 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.875+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088006a290 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.875+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f087401a070 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.876+0000 7f0867fff640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 0x7f086c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.876+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7f087408afb0 con 0x7f08800cf2e0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.876+0000 7f087f7fe640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 0x7f086c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:19:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.876+0000 7f087f7fe640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 0x7f086c06ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0870009a20 tx=0x7f0870008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:19:59.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:19:59.880+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08740593d0 con 0x7f08800cf2e0 2024-04-26T21:20:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.065+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08800b8ef0 con 0x7f08800cf2e0 2024-04-26T21:20:00.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.066+0000 7f0867fff640 1 -- 172.21.15.40:0/2001803628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16692 (secure 0 0 0) 0x7f0874058cd0 con 0x7f08800cf2e0 2024-04-26T21:20:00.068 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:59.077894+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 msgr2=0x7f086c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 0x7f086c06ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0870009a20 tx=0x7f0870008040 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 msgr2=0x7f08800b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f087400c410 tx=0x7f087400c8d0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 shutdown_connections 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f086c06c920 0x7f086c06ede0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08800d11b0 0x7f08800b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08800cf2e0 0x7f08800b7b70 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 --2- 172.21.15.40:0/2001803628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f088006e490 0x7f08800c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.068+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 >> 172.21.15.40:0/2001803628 conn(0x7f08800b4a60 msgr2=0x7f0880071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.069+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 shutdown_connections 2024-04-26T21:20:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.069+0000 7f0885bf0640 1 -- 172.21.15.40:0/2001803628 wait complete. 2024-04-26T21:20:00.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.181+0000 7f3c9cf40640 1 Processor -- start 2024-04-26T21:20:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.181+0000 7f3c9cf40640 1 -- start start 2024-04-26T21:20:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9cf40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 0x7f3c980b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9cf40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c980d5610 0x7f3c980d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9cf40640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c980b2510 con 0x7f3c9815f720 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9cf40640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c980b27f0 con 0x7f3c980d5610 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9cf40640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3c980b2ad0 con 0x7f3c9815f340 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c980d5610 0x7f3c980d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c980d5610 0x7f3c980d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39666/0 (socket says 172.21.15.40:39666) 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 -- 172.21.15.40:0/2958517350 learned_addr learned my addr 172.21.15.40:0/2958517350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9659c640 1 --2- 172.21.15.40:0/2958517350 >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 0x7f3c980b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 -- 172.21.15.40:0/2958517350 >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 msgr2=0x7f3c980b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 --2- 172.21.15.40:0/2958517350 >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 0x7f3c980b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 -- 172.21.15.40:0/2958517350 >> v1:172.21.15.184:6789/0 conn(0x7f3c9815f720 legacy=0x7f3c980d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c96d9d640 1 -- 172.21.15.40:0/2958517350 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c980b2db0 con 0x7f3c980d5610 2024-04-26T21:20:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.182+0000 7f3c9659c640 1 --2- 172.21.15.40:0/2958517350 >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 0x7f3c980b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.183+0000 7f3c96d9d640 1 --2- 172.21.15.40:0/2958517350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c980d5610 0x7f3c980d3d40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3c980d6000 tx=0x7f3c8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66047055c2c5ea1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.183+0000 7f3c9559a640 1 -- 172.21.15.40:0/2958517350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88037070 con 0x7f3c980d5610 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.183+0000 7f3c9559a640 1 -- 172.21.15.40:0/2958517350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88032440 con 0x7f3c980d5610 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.183+0000 7f3c9559a640 1 -- 172.21.15.40:0/2958517350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88032810 con 0x7f3c980d5610 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 -- 172.21.15.40:0/2958517350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c980d5610 msgr2=0x7f3c980d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/2958517350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c980d5610 0x7f3c980d3d40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3c980d6000 tx=0x7f3c8802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 -- 172.21.15.40:0/2958517350 shutdown_connections 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/2958517350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c980d5610 0x7f3c980d3d40 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/2958517350 >> v2:172.21.15.184:3300/0 conn(0x7f3c9815f340 0x7f3c980b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 -- 172.21.15.40:0/2958517350 >> 172.21.15.40:0/2958517350 conn(0x7f3c9808ec70 msgr2=0x7f3c9809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 -- 172.21.15.40:0/2958517350 shutdown_connections 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.184+0000 7f3c9cf40640 1 -- 172.21.15.40:0/2958517350 wait complete. 2024-04-26T21:20:00.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.185+0000 7f3c9cf40640 1 Processor -- start 2024-04-26T21:20:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.185+0000 7f3c9cf40640 1 -- start start 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c980d5610 0x7f3c981555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9815f340 0x7f3c980a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980b3940 con 0x7f3c9815f720 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980b37c0 con 0x7f3c9815f340 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9cf40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3c980b3ac0 con 0x7f3c980d5610 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39670/0 (socket says 172.21.15.40:39670) 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 -- 172.21.15.40:0/953320817 learned_addr learned my addr 172.21.15.40:0/953320817 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c9659c640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c980d5610 0x7f3c981555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c95d9b640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9815f340 0x7f3c980a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 -- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c980d5610 msgr2=0x7f3c981555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c980d5610 0x7f3c981555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 -- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9815f340 msgr2=0x7f3c980a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9815f340 0x7f3c980a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.186+0000 7f3c96d9d640 1 -- 172.21.15.40:0/953320817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c980a22f0 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c96d9d640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3c880090b0 tx=0x7f3c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88037040 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88042070 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c980a25d0 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88030920 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.187+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c980b8540 con 0x7f3c9815f720 2024-04-26T21:20:00.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.188+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9806a290 con 0x7f3c9815f720 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.188+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c88030ac0 con 0x7f3c9815f720 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.189+0000 7f3c86ffd640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 0x7f3c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.189+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7f3c880ac310 con 0x7f3c9815f720 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.189+0000 7f3c9659c640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 0x7f3c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.189+0000 7f3c9659c640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 0x7f3c7806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3c90003b80 tx=0x7f3c90009890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.193+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c8807a0f0 con 0x7f3c9815f720 2024-04-26T21:20:00.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.377+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c980d3020 con 0x7f3c9815f720 2024-04-26T21:20:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.378+0000 7f3c86ffd640 1 -- 172.21.15.40:0/953320817 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16692 (secure 0 0 0) 0x7f3c8807e3f0 con 0x7f3c9815f720 2024-04-26T21:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:19:59.077894+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 msgr2=0x7f3c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 0x7f3c7806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3c90003b80 tx=0x7f3c90009890 comp rx=0 tx=0).stop 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 msgr2=0x7f3c980b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3c880090b0 tx=0x7f3c880046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 shutdown_connections 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3c7806c8f0 0x7f3c7806edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c9815f720 0x7f3c980b8a60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9815f340 0x7f3c980a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 --2- 172.21.15.40:0/953320817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c980d5610 0x7f3c981555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 >> 172.21.15.40:0/953320817 conn(0x7f3c9808ec70 msgr2=0x7f3c980b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 shutdown_connections 2024-04-26T21:20:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.380+0000 7f3c9cf40640 1 -- 172.21.15.40:0/953320817 wait complete. 2024-04-26T21:20:00.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:20:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.495+0000 7f286ddaf640 1 Processor -- start 2024-04-26T21:20:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.495+0000 7f286ddaf640 1 -- start start 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f286ddaf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 0x7f28680b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f286ddaf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28680b3e50 0x7f28680720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f286ddaf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28680b46a0 con 0x7f286806f340 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f286ddaf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f28680b5db0 con 0x7f28680b3e50 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f286ddaf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f28680b6090 con 0x7f28680d0300 2024-04-26T21:20:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2867fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28680b3e50 0x7f28680720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f28677fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f286806f340 0x7f286806f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55908/0 (socket says 172.21.15.40:55908) 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f28677fe640 1 -- 172.21.15.40:0/2683560344 learned_addr learned my addr 172.21.15.40:0/2683560344 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2866ffd640 1 --2- 172.21.15.40:0/2683560344 >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 0x7f28680b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2867fff640 1 -- 172.21.15.40:0/2683560344 >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 msgr2=0x7f28680b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2867fff640 1 --2- 172.21.15.40:0/2683560344 >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 0x7f28680b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2867fff640 1 -- 172.21.15.40:0/2683560344 >> v1:172.21.15.40:6789/0 conn(0x7f286806f340 legacy=0x7f286806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f28667fc640 1 -- 172.21.15.40:0/2683560344 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787117390 0 0) 0x7f28680b46a0 con 0x7f286806f340 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2867fff640 1 -- 172.21.15.40:0/2683560344 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680b6370 con 0x7f28680b3e50 2024-04-26T21:20:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.496+0000 7f2866ffd640 1 --2- 172.21.15.40:0/2683560344 >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 0x7f28680b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.497+0000 7f2867fff640 1 --2- 172.21.15.40:0/2683560344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28680720f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2868068b90 tx=0x7f286002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f110677b93b5e59b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.497+0000 7f28667fc640 1 -- 172.21.15.40:0/2683560344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860007740 con 0x7f28680b3e50 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.497+0000 7f28667fc640 1 -- 172.21.15.40:0/2683560344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28600078e0 con 0x7f28680b3e50 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.497+0000 7f28667fc640 1 -- 172.21.15.40:0/2683560344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860007cb0 con 0x7f28680b3e50 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 -- 172.21.15.40:0/2683560344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 msgr2=0x7f28680720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 --2- 172.21.15.40:0/2683560344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28680720f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2868068b90 tx=0x7f286002a560 comp rx=0 tx=0).stop 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 -- 172.21.15.40:0/2683560344 shutdown_connections 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 --2- 172.21.15.40:0/2683560344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28680720f0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 --2- 172.21.15.40:0/2683560344 >> v2:172.21.15.184:3300/0 conn(0x7f28680d0300 0x7f28680b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 -- 172.21.15.40:0/2683560344 >> 172.21.15.40:0/2683560344 conn(0x7f28680cd8a0 msgr2=0x7f28680cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 -- 172.21.15.40:0/2683560344 shutdown_connections 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.498+0000 7f286ddaf640 1 -- 172.21.15.40:0/2683560344 wait complete. 2024-04-26T21:20:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.499+0000 7f286ddaf640 1 Processor -- start 2024-04-26T21:20:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.499+0000 7f286ddaf640 1 -- start start 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 0x7f28681f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28680d0300 0x7f28680cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28680b4460 con 0x7f28680b3e50 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28680b42e0 con 0x7f286806f340 2024-04-26T21:20:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f286ddaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2868142a30 con 0x7f28680d0300 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39698/0 (socket says 172.21.15.40:39698) 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 -- 172.21.15.40:0/3869509943 learned_addr learned my addr 172.21.15.40:0/3869509943 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f28677fe640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 0x7f28681f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2867fff640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28680d0300 0x7f28680cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 -- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28680d0300 msgr2=0x7f28680cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28680d0300 0x7f28680cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 -- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 msgr2=0x7f28681f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 0x7f28681f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 -- 172.21.15.40:0/3869509943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28681f6010 con 0x7f28680b3e50 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f28677fe640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 0x7f28681f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f2866ffd640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d800 tx=0x7f285c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.500+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c004030 con 0x7f28680b3e50 2024-04-26T21:20:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.501+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c0041d0 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.501+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00b6c0 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.501+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28680c5f60 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.501+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28680c6200 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.502+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286806a290 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.502+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f285c00b860 con 0x7f28680b3e50 2024-04-26T21:20:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.503+0000 7f284ffff640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 0x7f285406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.503+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7f285c08bda0 con 0x7f28680b3e50 2024-04-26T21:20:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.503+0000 7f28677fe640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 0x7f285406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.503+0000 7f28677fe640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 0x7f285406ede0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2858000f60 tx=0x7f2858007c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:00.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.507+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28681f6010 con 0x7f28680b3e50 2024-04-26T21:20:00.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:00.736+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f28680b46a0 con 0x7f28680b3e50 2024-04-26T21:20:01.700 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:20:01.700+0000 7fad4e243640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:20:01.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.741+0000 7f284ffff640 1 -- 172.21.15.40:0/3869509943 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7f285c05a1d0 con 0x7f28680b3e50 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 msgr2=0x7f285406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 0x7f285406ede0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2858000f60 tx=0x7f2858007c30 comp rx=0 tx=0).stop 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 msgr2=0x7f28681f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d800 tx=0x7f285c00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 shutdown_connections 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f285406c920 0x7f285406ede0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f28680d0300 0x7f28680cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28680b3e50 0x7f28681f5ad0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 --2- 172.21.15.40:0/3869509943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f286806f340 0x7f28681f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 >> 172.21.15.40:0/3869509943 conn(0x7f28680cd8a0 msgr2=0x7f28680b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 shutdown_connections 2024-04-26T21:20:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.744+0000 7f286ddaf640 1 -- 172.21.15.40:0/3869509943 wait complete. 2024-04-26T21:20:01.758 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:20:01.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:20:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.868+0000 7f8a7a789640 1 Processor -- start 2024-04-26T21:20:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.868+0000 7f8a7a789640 1 -- start start 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a7a789640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a7415f5b0 0x7f8a740b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a7a789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 0x7f8a740d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a7a789640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a740b6aa0 con 0x7f8a740b55b0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a7a789640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8a740d3eb0 con 0x7f8a7415f5b0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a7a789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a740d4190 con 0x7f8a7415f990 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a78d26640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8a740b55b0 0x7f8a740d3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53276/0 (socket says 172.21.15.40:53276) 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a78d26640 1 -- 172.21.15.40:0/3652712769 learned_addr learned my addr 172.21.15.40:0/3652712769 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a737fe640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 0x7f8a740d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.869+0000 7f8a73fff640 1 --2- 172.21.15.40:0/3652712769 >> v2:172.21.15.184:3300/0 conn(0x7f8a7415f5b0 0x7f8a740b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a72ffd640 1 -- 172.21.15.40:0/3652712769 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731515545 0 0) 0x7f8a740b6aa0 con 0x7f8a740b55b0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a72ffd640 1 -- 172.21.15.40:0/3652712769 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a5c003520 con 0x7f8a740b55b0 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a73fff640 1 -- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 msgr2=0x7f8a740d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:01.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a73fff640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 0x7f8a740d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a73fff640 1 -- 172.21.15.40:0/3652712769 >> v1:172.21.15.40:6789/0 conn(0x7f8a740b55b0 legacy=0x7f8a740d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a73fff640 1 -- 172.21.15.40:0/3652712769 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a740d0200 con 0x7f8a7415f5b0 2024-04-26T21:20:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a737fe640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 0x7f8a740d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.870+0000 7f8a73fff640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f5b0 0x7f8a740b5070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8a680097a0 tx=0x7f8a6802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db686237d68577c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a72ffd640 1 -- 172.21.15.40:0/3652712769 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68037070 con 0x7f8a7415f5b0 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a72ffd640 1 -- 172.21.15.40:0/3652712769 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6802ae50 con 0x7f8a7415f5b0 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a72ffd640 1 -- 172.21.15.40:0/3652712769 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68032660 con 0x7f8a7415f5b0 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 -- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f5b0 msgr2=0x7f8a740b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f5b0 0x7f8a740b5070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8a680097a0 tx=0x7f8a6802a100 comp rx=0 tx=0).stop 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 -- 172.21.15.40:0/3652712769 shutdown_connections 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f990 0x7f8a740d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 --2- 172.21.15.40:0/3652712769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f5b0 0x7f8a740b5070 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 -- 172.21.15.40:0/3652712769 >> 172.21.15.40:0/3652712769 conn(0x7f8a7408ec40 msgr2=0x7f8a7409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 -- 172.21.15.40:0/3652712769 shutdown_connections 2024-04-26T21:20:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.871+0000 7f8a7a789640 1 -- 172.21.15.40:0/3652712769 wait complete. 2024-04-26T21:20:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.872+0000 7f8a7a789640 1 Processor -- start 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 -- start start 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f5b0 0x7f8a740c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 0x7f8a7415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740d3bd0 con 0x7f8a740b55b0 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740d3a50 con 0x7f8a7415f990 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a7a789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8a740d0f40 con 0x7f8a7415f5b0 2024-04-26T21:20:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.873+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59830/0 (socket says 172.21.15.40:59830) 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 -- 172.21.15.40:0/300315754 learned_addr learned my addr 172.21.15.40:0/300315754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a737fe640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f5b0 0x7f8a740c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a78d26640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 0x7f8a7415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 -- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f5b0 msgr2=0x7f8a740c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f5b0 0x7f8a740c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 -- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 msgr2=0x7f8a7415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 0x7f8a7415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 -- 172.21.15.40:0/300315754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a74161900 con 0x7f8a740b55b0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a78d26640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 0x7f8a7415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a73fff640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a68035040 tx=0x7f8a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68041070 con 0x7f8a740b55b0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6803c460 con 0x7f8a740b55b0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a74161be0 con 0x7f8a740b55b0 2024-04-26T21:20:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68037040 con 0x7f8a740b55b0 2024-04-26T21:20:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.874+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741621a0 con 0x7f8a740b55b0 2024-04-26T21:20:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.875+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74068580 con 0x7f8a740b55b0 2024-04-26T21:20:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.876+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a6803a070 con 0x7f8a740b55b0 2024-04-26T21:20:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.876+0000 7f8a70ff9640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 0x7f8a6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.876+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f8a680abc60 con 0x7f8a740b55b0 2024-04-26T21:20:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.876+0000 7f8a737fe640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 0x7f8a6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.877+0000 7f8a737fe640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 0x7f8a6006ede0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009560 tx=0x7f8a64009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:01.881+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a74068580 con 0x7f8a740b55b0 2024-04-26T21:20:02.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.106+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8a740d60c0 con 0x7f8a740b55b0 2024-04-26T21:20:02.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:20:02.429 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:20:02.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.743+0000 7f8a70ff9640 1 -- 172.21.15.40:0/300315754 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7f8a68079a40 con 0x7f8a740b55b0 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 msgr2=0x7f8a6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 0x7f8a6006ede0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009560 tx=0x7f8a64009190 comp rx=0 tx=0).stop 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 msgr2=0x7f8a7415c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a68035040 tx=0x7f8a680046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 shutdown_connections 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8a6006c920 0x7f8a6006ede0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a7415f990 0x7f8a7415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a7415f5b0 0x7f8a740c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 --2- 172.21.15.40:0/300315754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a740b55b0 0x7f8a7415c4e0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 >> 172.21.15.40:0/300315754 conn(0x7f8a7408ec40 msgr2=0x7f8a740d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:02.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 shutdown_connections 2024-04-26T21:20:02.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.746+0000 7f8a7a789640 1 -- 172.21.15.40:0/300315754 wait complete. 2024-04-26T21:20:02.760 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:20:02.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:02.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.863+0000 7f63370f1640 1 Processor -- start 2024-04-26T21:20:02.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.863+0000 7f63370f1640 1 -- start start 2024-04-26T21:20:02.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f63370f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f633013ff70 0x7f6330146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f63370f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6330140350 0x7f63301465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f63370f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63300d05b0 con 0x7f63300cf810 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f63370f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f63300d0390 con 0x7f6330140350 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f63370f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f63300d0890 con 0x7f633013ff70 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6330140350 0x7f63301465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f633568e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63300cf810 0x7f63300cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53286/0 (socket says 172.21.15.40:53286) 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f633568e640 1 -- 172.21.15.40:0/717650607 learned_addr learned my addr 172.21.15.40:0/717650607 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f6334e8d640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f633013ff70 0x7f6330146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632ffff640 1 -- 172.21.15.40:0/717650607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f633013ff70 msgr2=0x7f6330146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632ffff640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f633013ff70 0x7f6330146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632ffff640 1 -- 172.21.15.40:0/717650607 >> v1:172.21.15.40:6789/0 conn(0x7f63300cf810 legacy=0x7f63300cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632ffff640 1 -- 172.21.15.40:0/717650607 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633006e450 con 0x7f6330140350 2024-04-26T21:20:02.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.864+0000 7f632f7fe640 1 -- 172.21.15.40:0/717650607 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098816169 0 0) 0x7f63300d05b0 con 0x7f63300cf810 2024-04-26T21:20:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.865+0000 7f632ffff640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f63301465c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c09a5b5ea975d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.865+0000 7f632f7fe640 1 -- 172.21.15.40:0/717650607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320037070 con 0x7f6330140350 2024-04-26T21:20:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.865+0000 7f632f7fe640 1 -- 172.21.15.40:0/717650607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632002ae50 con 0x7f6330140350 2024-04-26T21:20:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.865+0000 7f632f7fe640 1 -- 172.21.15.40:0/717650607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320032660 con 0x7f6330140350 2024-04-26T21:20:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 -- 172.21.15.40:0/717650607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 msgr2=0x7f63301465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f63301465c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002a100 comp rx=0 tx=0).stop 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 -- 172.21.15.40:0/717650607 shutdown_connections 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f63301465c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 --2- 172.21.15.40:0/717650607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f633013ff70 0x7f6330146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 -- 172.21.15.40:0/717650607 >> 172.21.15.40:0/717650607 conn(0x7f63300d0d30 msgr2=0x7f63300d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 -- 172.21.15.40:0/717650607 shutdown_connections 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.866+0000 7f63370f1640 1 -- 172.21.15.40:0/717650607 wait complete. 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.867+0000 7f63370f1640 1 Processor -- start 2024-04-26T21:20:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.867+0000 7f63370f1640 1 -- start start 2024-04-26T21:20:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 0x7f63300730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f633013ff70 0x7f63301f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f633006edd0 con 0x7f6330140350 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f633006ec50 con 0x7f633013ff70 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f63370f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f633006ef50 con 0x7f63300cf810 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59844/0 (socket says 172.21.15.40:59844) 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 -- 172.21.15.40:0/698465339 learned_addr learned my addr 172.21.15.40:0/698465339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f6334e8d640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 0x7f63300730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f632ffff640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f633013ff70 0x7f63301f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 -- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 msgr2=0x7f63300730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 0x7f63300730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 -- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f633013ff70 msgr2=0x7f63301f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f633013ff70 0x7f63301f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f633568e640 1 -- 172.21.15.40:0/698465339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63300737b0 con 0x7f6330140350 2024-04-26T21:20:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.868+0000 7f6334e8d640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 0x7f63300730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f633568e640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f631800c450 tx=0x7f631800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318018070 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6318007500 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6330073a90 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318004a20 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.869+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633013a6b0 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.870+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6330073fb0 con 0x7f6330140350 2024-04-26T21:20:02.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.870+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f631801b070 con 0x7f6330140350 2024-04-26T21:20:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.871+0000 7f632d7fa640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 0x7f631006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.871+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f631808add0 con 0x7f6330140350 2024-04-26T21:20:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.871+0000 7f6334e8d640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 0x7f631006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:02.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.872+0000 7f6334e8d640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 0x7f631006edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f63300d16a0 tx=0x7f6328007650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:02.875+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6330073fb0 con 0x7f6330140350 2024-04-26T21:20:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.096+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f633006a290 con 0x7f6330140350 2024-04-26T21:20:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.097+0000 7f632d7fa640 1 -- 172.21.15.40:0/698465339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f631808c090 con 0x7f6330140350 2024-04-26T21:20:03.099 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:03.099 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:02.108019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 76 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 msgr2=0x7f631006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 0x7f631006edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f63300d16a0 tx=0x7f6328007650 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 msgr2=0x7f633015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f631800c450 tx=0x7f631800c910 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 shutdown_connections 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f631006c8f0 0x7f631006edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330140350 0x7f633015c830 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f633013ff70 0x7f63301f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 --2- 172.21.15.40:0/698465339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63300cf810 0x7f63300730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 >> 172.21.15.40:0/698465339 conn(0x7f63300d0d30 msgr2=0x7f63300b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 shutdown_connections 2024-04-26T21:20:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.099+0000 7f63370f1640 1 -- 172.21.15.40:0/698465339 wait complete. 2024-04-26T21:20:03.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:03.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.210+0000 7f79c53b4640 1 Processor -- start 2024-04-26T21:20:03.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.210+0000 7f79c53b4640 1 -- start start 2024-04-26T21:20:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79c53b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79c015f340 0x7f79c00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79c53b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 0x7f79c00d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79c53b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c00b2510 con 0x7f79c00d4f00 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79c53b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00b27f0 con 0x7f79c015f340 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79c53b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00b2ad0 con 0x7f79c015f720 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79c015f340 0x7f79c00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79c015f340 0x7f79c00b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59848/0 (socket says 172.21.15.40:59848) 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 -- 172.21.15.40:0/668349725 learned_addr learned my addr 172.21.15.40:0/668349725 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79be7fc640 1 --2- 172.21.15.40:0/668349725 >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 0x7f79c00d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 -- 172.21.15.40:0/668349725 >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 msgr2=0x7f79c00d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 --2- 172.21.15.40:0/668349725 >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 0x7f79c00d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 -- 172.21.15.40:0/668349725 >> v1:172.21.15.184:6789/0 conn(0x7f79c00d4f00 legacy=0x7f79c00d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79beffd640 1 -- 172.21.15.40:0/668349725 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00b2db0 con 0x7f79c015f340 2024-04-26T21:20:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.211+0000 7f79be7fc640 1 --2- 172.21.15.40:0/668349725 >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 0x7f79c00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.212+0000 7f79beffd640 1 --2- 172.21.15.40:0/668349725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c015f340 0x7f79c00b1e80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cf8a0 tx=0x7f79b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ddc5f562693a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.212+0000 7f79bdffb640 1 -- 172.21.15.40:0/668349725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037070 con 0x7f79c015f340 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.212+0000 7f79bdffb640 1 -- 172.21.15.40:0/668349725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8032440 con 0x7f79c015f340 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.212+0000 7f79bdffb640 1 -- 172.21.15.40:0/668349725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8032810 con 0x7f79c015f340 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 -- 172.21.15.40:0/668349725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c015f340 msgr2=0x7f79c00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 --2- 172.21.15.40:0/668349725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c015f340 0x7f79c00b1e80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cf8a0 tx=0x7f79b802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 -- 172.21.15.40:0/668349725 shutdown_connections 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 --2- 172.21.15.40:0/668349725 >> v2:172.21.15.184:3300/0 conn(0x7f79c015f720 0x7f79c00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 --2- 172.21.15.40:0/668349725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c015f340 0x7f79c00b1e80 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 -- 172.21.15.40:0/668349725 >> 172.21.15.40:0/668349725 conn(0x7f79c008ec70 msgr2=0x7f79c009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 -- 172.21.15.40:0/668349725 shutdown_connections 2024-04-26T21:20:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.213+0000 7f79c53b4640 1 -- 172.21.15.40:0/668349725 wait complete. 2024-04-26T21:20:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.214+0000 7f79c53b4640 1 Processor -- start 2024-04-26T21:20:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.214+0000 7f79c53b4640 1 -- start start 2024-04-26T21:20:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c015f340 0x7f79c00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 0x7f79c014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b3940 con 0x7f79c00d4f00 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b37c0 con 0x7f79c015f340 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79c53b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79c00b3ac0 con 0x7f79c015f720 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59860/0 (socket says 172.21.15.40:59860) 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 -- 172.21.15.40:0/1067597784 learned_addr learned my addr 172.21.15.40:0/1067597784 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79be7fc640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c015f340 0x7f79c00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79bf7fe640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 0x7f79c014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 -- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 msgr2=0x7f79c014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 0x7f79c014e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 -- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c015f340 msgr2=0x7f79c00cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c015f340 0x7f79c00cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79beffd640 1 -- 172.21.15.40:0/1067597784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00cca00 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.215+0000 7f79bf7fe640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 0x7f79c014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79beffd640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000c00 tx=0x7f79b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8041070 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b803c870 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037040 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c00ccce0 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.216+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c00cd2a0 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.217+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c006a290 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.217+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79b8044070 con 0x7f79c00d4f00 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.218+0000 7f79b77fe640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 0x7f79a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.218+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f79b80ac8e0 con 0x7f79c00d4f00 2024-04-26T21:20:03.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.218+0000 7f79be7fc640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 0x7f79a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.219+0000 7f79be7fc640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 0x7f79a006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ca4b0 tx=0x7f79a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.222+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79b807ad90 con 0x7f79c00d4f00 2024-04-26T21:20:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.407+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79c00d3020 con 0x7f79c00d4f00 2024-04-26T21:20:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.408+0000 7f79b77fe640 1 -- 172.21.15.40:0/1067597784 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7f79b807a690 con 0x7f79c00d4f00 2024-04-26T21:20:03.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:03.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 msgr2=0x7f79a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 0x7f79a006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ca4b0 tx=0x7f79a8008040 comp rx=0 tx=0).stop 2024-04-26T21:20:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 msgr2=0x7f79c00caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000c00 tx=0x7f79b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 shutdown_connections 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f79a006c8f0 0x7f79a006edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c015f720 0x7f79c014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c015f340 0x7f79c00cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 --2- 172.21.15.40:0/1067597784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00d4f00 0x7f79c00caf70 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 >> 172.21.15.40:0/1067597784 conn(0x7f79c008ec70 msgr2=0x7f79c00d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 shutdown_connections 2024-04-26T21:20:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.410+0000 7f79c53b4640 1 -- 172.21.15.40:0/1067597784 wait complete. 2024-04-26T21:20:03.423 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:20:03.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:03.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.522+0000 7f953b99d640 1 Processor -- start 2024-04-26T21:20:03.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.522+0000 7f953b99d640 1 -- start start 2024-04-26T21:20:03.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f953b99d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340cf2e0 0x7f953406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f953b99d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f953406e490 0x7f953415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f953b99d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95340d1740 con 0x7f95340d11b0 2024-04-26T21:20:03.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f953b99d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95340d1a20 con 0x7f953406e490 2024-04-26T21:20:03.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f953b99d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95340d1d00 con 0x7f95340cf2e0 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f953406e490 0x7f953415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f953406e490 0x7f953415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59874/0 (socket says 172.21.15.40:59874) 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 -- 172.21.15.40:0/988556216 learned_addr learned my addr 172.21.15.40:0/988556216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539739640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340cf2e0 0x7f953406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 -- 172.21.15.40:0/988556216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340cf2e0 msgr2=0x7f953406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340cf2e0 0x7f953406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 -- 172.21.15.40:0/988556216 >> v1:172.21.15.184:6789/0 conn(0x7f95340d11b0 legacy=0x7f953406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.523+0000 7f9539f3a640 1 -- 172.21.15.40:0/988556216 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340d1fe0 con 0x7f953406e490 2024-04-26T21:20:03.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.524+0000 7f9539f3a640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953415ebe0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e50 tx=0x7f952c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a465b81192a148a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.524+0000 7f952bfff640 1 -- 172.21.15.40:0/988556216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c036070 con 0x7f953406e490 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.524+0000 7f952bfff640 1 -- 172.21.15.40:0/988556216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c031600 con 0x7f953406e490 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.524+0000 7f952bfff640 1 -- 172.21.15.40:0/988556216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c0319d0 con 0x7f953406e490 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 -- 172.21.15.40:0/988556216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 msgr2=0x7f953415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953415ebe0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f952c009e50 tx=0x7f952c02a710 comp rx=0 tx=0).stop 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 -- 172.21.15.40:0/988556216 shutdown_connections 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953415ebe0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 --2- 172.21.15.40:0/988556216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340cf2e0 0x7f953406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 -- 172.21.15.40:0/988556216 >> 172.21.15.40:0/988556216 conn(0x7f95340b4a60 msgr2=0x7f95340b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 -- 172.21.15.40:0/988556216 shutdown_connections 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.525+0000 7f953b99d640 1 -- 172.21.15.40:0/988556216 wait complete. 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.526+0000 7f953b99d640 1 Processor -- start 2024-04-26T21:20:03.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.526+0000 7f953b99d640 1 -- start start 2024-04-26T21:20:03.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 0x7f95340b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d11b0 0x7f95340cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95340cf6c0 con 0x7f953406e490 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f953406da10 con 0x7f95340cf2e0 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f95340cf840 con 0x7f95340d11b0 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59876/0 (socket says 172.21.15.40:59876) 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 -- 172.21.15.40:0/3360706529 learned_addr learned my addr 172.21.15.40:0/3360706529 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539f3a640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d11b0 0x7f95340cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9538f38640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 0x7f95340b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 -- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d11b0 msgr2=0x7f95340cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d11b0 0x7f95340cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 -- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 msgr2=0x7f95340b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 0x7f95340b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 -- 172.21.15.40:0/3360706529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953414fb30 con 0x7f953406e490 2024-04-26T21:20:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9538f38640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 0x7f95340b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9539739640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f953000d5f0 tx=0x7f953000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300125f0 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530019070 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300129d0 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.527+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340b7040 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.528+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95340b7600 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.528+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953406a290 con 0x7f953406e490 2024-04-26T21:20:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.529+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9530012b70 con 0x7f953406e490 2024-04-26T21:20:03.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.529+0000 7f9529ffb640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.529+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f953008be20 con 0x7f953406e490 2024-04-26T21:20:03.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.529+0000 7f9538f38640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.530+0000 7f9538f38640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 0x7f951806edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95340ccca0 tx=0x7f9524009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.533+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f953005a2e0 con 0x7f953406e490 2024-04-26T21:20:03.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.718+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9534072140 con 0x7f951806c8f0 2024-04-26T21:20:03.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.725+0000 7f9529ffb640 1 -- 172.21.15.40:0/3360706529 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252471 (secure 0 0 0) 0x7f9534072140 con 0x7f951806c8f0 2024-04-26T21:20:03.730 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:03.732 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 msgr2=0x7f951806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 0x7f951806edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95340ccca0 tx=0x7f9524009040 comp rx=0 tx=0).stop 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 msgr2=0x7f953414f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f953000d5f0 tx=0x7f953000dab0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 shutdown_connections 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f951806c8f0 0x7f951806edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95340d11b0 0x7f95340cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95340cf2e0 0x7f95340b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 --2- 172.21.15.40:0/3360706529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953406e490 0x7f953414f5f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 >> 172.21.15.40:0/3360706529 conn(0x7f95340b4a60 msgr2=0x7f95340cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 shutdown_connections 2024-04-26T21:20:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.728+0000 7f953b99d640 1 -- 172.21.15.40:0/3360706529 wait complete. 2024-04-26T21:20:03.741 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-26T21:20:03.035382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270832,"kb_used_data":13040,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703888,"statfs":{"total":773094113280,"available":772816781312,"internally_reserved":0,"allocated":13352960,"data_stored":6386177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.329807"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731455+0000","last_change":"2024-04-26T21:19:59.733872+0000","last_active":"2024-04-26T21:20:00.731455+0000","last_peered":"2024-04-26T21:20:00.731455+0000","last_clean":"2024-04-26T21:20:00.731455+0000","last_became_active":"2024-04-26T21:19:59.733535+0000","last_became_peered":"2024-04-26T21:19:59.733535+0000","last_unstale":"2024-04-26T21:20:00.731455+0000","last_undegraded":"2024-04-26T21:20:00.731455+0000","last_fullsized":"2024-04-26T21:20:00.731455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:20.583522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.734709+0000","last_change":"2024-04-26T21:19:59.732087+0000","last_active":"2024-04-26T21:19:59.734709+0000","last_peered":"2024-04-26T21:19:59.734709+0000","last_clean":"2024-04-26T21:19:59.734709+0000","last_became_active":"2024-04-26T21:19:59.731700+0000","last_became_peered":"2024-04-26T21:19:59.731700+0000","last_unstale":"2024-04-26T21:19:59.734709+0000","last_undegraded":"2024-04-26T21:19:59.734709+0000","last_fullsized":"2024-04-26T21:19:59.734709+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:13.608672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733040+0000","last_change":"2024-04-26T21:19:59.738269+0000","last_active":"2024-04-26T21:20:00.733040+0000","last_peered":"2024-04-26T21:20:00.733040+0000","last_clean":"2024-04-26T21:20:00.733040+0000","last_became_active":"2024-04-26T21:19:59.737930+0000","last_became_peered":"2024-04-26T21:19:59.737930+0000","last_unstale":"2024-04-26T21:20:00.733040+0000","last_undegraded":"2024-04-26T21:20:00.733040+0000","last_fullsized":"2024-04-26T21:20:00.733040+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:37.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731284+0000","last_change":"2024-04-26T21:19:59.737753+0000","last_active":"2024-04-26T21:20:00.731284+0000","last_peered":"2024-04-26T21:20:00.731284+0000","last_clean":"2024-04-26T21:20:00.731284+0000","last_became_active":"2024-04-26T21:19:59.736072+0000","last_became_peered":"2024-04-26T21:19:59.736072+0000","last_unstale":"2024-04-26T21:20:00.731284+0000","last_undegraded":"2024-04-26T21:20:00.731284+0000","last_fullsized":"2024-04-26T21:20:00.731284+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:49.851261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331699+0000","last_change":"2024-04-26T21:19:59.739748+0000","last_active":"2024-04-26T21:20:01.331699+0000","last_peered":"2024-04-26T21:20:01.331699+0000","last_clean":"2024-04-26T21:20:01.331699+0000","last_became_active":"2024-04-26T21:19:59.739443+0000","last_became_peered":"2024-04-26T21:19:59.739443+0000","last_unstale":"2024-04-26T21:20:01.331699+0000","last_undegraded":"2024-04-26T21:20:01.331699+0000","last_fullsized":"2024-04-26T21:20:01.331699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:31.202273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331261+0000","last_change":"2024-04-26T21:19:59.742243+0000","last_active":"2024-04-26T21:20:01.331261+0000","last_peered":"2024-04-26T21:20:01.331261+0000","last_clean":"2024-04-26T21:20:01.331261+0000","last_became_active":"2024-04-26T21:19:59.741404+0000","last_became_peered":"2024-04-26T21:19:59.741404+0000","last_unstale":"2024-04-26T21:20:01.331261+0000","last_undegraded":"2024-04-26T21:20:01.331261+0000","last_fullsized":"2024-04-26T21:20:01.331261+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:33.846756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331449+0000","last_change":"2024-04-26T21:19:59.736465+0000","last_active":"2024-04-26T21:20:01.331449+0000","last_peered":"2024-04-26T21:20:01.331449+0000","last_clean":"2024-04-26T21:20:01.331449+0000","last_became_active":"2024-04-26T21:19:59.736076+0000","last_became_peered":"2024-04-26T21:19:59.736076+0000","last_unstale":"2024-04-26T21:20:01.331449+0000","last_undegraded":"2024-04-26T21:20:01.331449+0000","last_fullsized":"2024-04-26T21:20:01.331449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:04.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733267+0000","last_change":"2024-04-26T21:19:59.743604+0000","last_active":"2024-04-26T21:20:00.733267+0000","last_peered":"2024-04-26T21:20:00.733267+0000","last_clean":"2024-04-26T21:20:00.733267+0000","last_became_active":"2024-04-26T21:19:59.743102+0000","last_became_peered":"2024-04-26T21:19:59.743102+0000","last_unstale":"2024-04-26T21:20:00.733267+0000","last_undegraded":"2024-04-26T21:20:00.733267+0000","last_fullsized":"2024-04-26T21:20:00.733267+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:25.244936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733270+0000","last_change":"2024-04-26T21:19:59.736191+0000","last_active":"2024-04-26T21:20:00.733270+0000","last_peered":"2024-04-26T21:20:00.733270+0000","last_clean":"2024-04-26T21:20:00.733270+0000","last_became_active":"2024-04-26T21:19:59.735710+0000","last_became_peered":"2024-04-26T21:19:59.735710+0000","last_unstale":"2024-04-26T21:20:00.733270+0000","last_undegraded":"2024-04-26T21:20:00.733270+0000","last_fullsized":"2024-04-26T21:20:00.733270+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:33:08.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736494+0000","last_change":"2024-04-26T21:19:59.733848+0000","last_active":"2024-04-26T21:19:59.736494+0000","last_peered":"2024-04-26T21:19:59.736494+0000","last_clean":"2024-04-26T21:19:59.736494+0000","last_became_active":"2024-04-26T21:19:59.732101+0000","last_became_peered":"2024-04-26T21:19:59.732101+0000","last_unstale":"2024-04-26T21:19:59.736494+0000","last_undegraded":"2024-04-26T21:19:59.736494+0000","last_fullsized":"2024-04-26T21:19:59.736494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:06.854976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331482+0000","last_change":"2024-04-26T21:19:59.735448+0000","last_active":"2024-04-26T21:20:01.331482+0000","last_peered":"2024-04-26T21:20:01.331482+0000","last_clean":"2024-04-26T21:20:01.331482+0000","last_became_active":"2024-04-26T21:19:59.735133+0000","last_became_peered":"2024-04-26T21:19:59.735133+0000","last_unstale":"2024-04-26T21:20:01.331482+0000","last_undegraded":"2024-04-26T21:20:01.331482+0000","last_fullsized":"2024-04-26T21:20:01.331482+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:02.533988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733534+0000","last_change":"2024-04-26T21:19:59.736254+0000","last_active":"2024-04-26T21:20:00.733534+0000","last_peered":"2024-04-26T21:20:00.733534+0000","last_clean":"2024-04-26T21:20:00.733534+0000","last_became_active":"2024-04-26T21:19:59.735612+0000","last_became_peered":"2024-04-26T21:19:59.735612+0000","last_unstale":"2024-04-26T21:20:00.733534+0000","last_undegraded":"2024-04-26T21:20:00.733534+0000","last_fullsized":"2024-04-26T21:20:00.733534+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:33.846756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731347+0000","last_change":"2024-04-26T21:19:59.737724+0000","last_active":"2024-04-26T21:20:00.731347+0000","last_peered":"2024-04-26T21:20:00.731347+0000","last_clean":"2024-04-26T21:20:00.731347+0000","last_became_active":"2024-04-26T21:19:59.737404+0000","last_became_peered":"2024-04-26T21:19:59.737404+0000","last_unstale":"2024-04-26T21:20:00.731347+0000","last_undegraded":"2024-04-26T21:20:00.731347+0000","last_fullsized":"2024-04-26T21:20:00.731347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:33.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.747633+0000","last_change":"2024-04-26T21:19:59.741481+0000","last_active":"2024-04-26T21:19:59.747633+0000","last_peered":"2024-04-26T21:19:59.747633+0000","last_clean":"2024-04-26T21:19:59.747633+0000","last_became_active":"2024-04-26T21:19:59.741185+0000","last_became_peered":"2024-04-26T21:19:59.741185+0000","last_unstale":"2024-04-26T21:19:59.747633+0000","last_undegraded":"2024-04-26T21:19:59.747633+0000","last_fullsized":"2024-04-26T21:19:59.747633+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:22.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731304+0000","last_change":"2024-04-26T21:19:59.741512+0000","last_active":"2024-04-26T21:20:00.731304+0000","last_peered":"2024-04-26T21:20:00.731304+0000","last_clean":"2024-04-26T21:20:00.731304+0000","last_became_active":"2024-04-26T21:19:59.741227+0000","last_became_peered":"2024-04-26T21:19:59.741227+0000","last_unstale":"2024-04-26T21:20:00.731304+0000","last_undegraded":"2024-04-26T21:20:00.731304+0000","last_fullsized":"2024-04-26T21:20:00.731304+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:59.603706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331328+0000","last_change":"2024-04-26T21:19:59.743606+0000","last_active":"2024-04-26T21:20:01.331328+0000","last_peered":"2024-04-26T21:20:01.331328+0000","last_clean":"2024-04-26T21:20:01.331328+0000","last_became_active":"2024-04-26T21:19:59.743300+0000","last_became_peered":"2024-04-26T21:19:59.743300+0000","last_unstale":"2024-04-26T21:20:01.331328+0000","last_undegraded":"2024-04-26T21:20:01.331328+0000","last_fullsized":"2024-04-26T21:20:01.331328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:01.429730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331425+0000","last_change":"2024-04-26T21:19:59.737320+0000","last_active":"2024-04-26T21:20:01.331425+0000","last_peered":"2024-04-26T21:20:01.331425+0000","last_clean":"2024-04-26T21:20:01.331425+0000","last_became_active":"2024-04-26T21:19:59.736787+0000","last_became_peered":"2024-04-26T21:19:59.736787+0000","last_unstale":"2024-04-26T21:20:01.331425+0000","last_undegraded":"2024-04-26T21:20:01.331425+0000","last_fullsized":"2024-04-26T21:20:01.331425+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:11.586932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332033+0000","last_change":"2024-04-26T21:19:59.751536+0000","last_active":"2024-04-26T21:20:01.332033+0000","last_peered":"2024-04-26T21:20:01.332033+0000","last_clean":"2024-04-26T21:20:01.332033+0000","last_became_active":"2024-04-26T21:19:59.751211+0000","last_became_peered":"2024-04-26T21:19:59.751211+0000","last_unstale":"2024-04-26T21:20:01.332033+0000","last_undegraded":"2024-04-26T21:20:01.332033+0000","last_fullsized":"2024-04-26T21:20:01.332033+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:26.458504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331226+0000","last_change":"2024-04-26T21:19:59.734866+0000","last_active":"2024-04-26T21:20:01.331226+0000","last_peered":"2024-04-26T21:20:01.331226+0000","last_clean":"2024-04-26T21:20:01.331226+0000","last_became_active":"2024-04-26T21:19:59.734566+0000","last_became_peered":"2024-04-26T21:19:59.734566+0000","last_unstale":"2024-04-26T21:20:01.331226+0000","last_undegraded":"2024-04-26T21:20:01.331226+0000","last_fullsized":"2024-04-26T21:20:01.331226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:53.666517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332332+0000","last_change":"2024-04-26T21:19:59.748469+0000","last_active":"2024-04-26T21:20:01.332332+0000","last_peered":"2024-04-26T21:20:01.332332+0000","last_clean":"2024-04-26T21:20:01.332332+0000","last_became_active":"2024-04-26T21:19:59.748096+0000","last_became_peered":"2024-04-26T21:19:59.748096+0000","last_unstale":"2024-04-26T21:20:01.332332+0000","last_undegraded":"2024-04-26T21:20:01.332332+0000","last_fullsized":"2024-04-26T21:20:01.332332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:46.589006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331698+0000","last_change":"2024-04-26T21:19:59.739498+0000","last_active":"2024-04-26T21:20:01.331698+0000","last_peered":"2024-04-26T21:20:01.331698+0000","last_clean":"2024-04-26T21:20:01.331698+0000","last_became_active":"2024-04-26T21:19:59.739220+0000","last_became_peered":"2024-04-26T21:19:59.739220+0000","last_unstale":"2024-04-26T21:20:01.331698+0000","last_undegraded":"2024-04-26T21:20:01.331698+0000","last_fullsized":"2024-04-26T21:20:01.331698+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:01.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331579+0000","last_change":"2024-04-26T21:19:59.733660+0000","last_active":"2024-04-26T21:20:01.331579+0000","last_peered":"2024-04-26T21:20:01.331579+0000","last_clean":"2024-04-26T21:20:01.331579+0000","last_became_active":"2024-04-26T21:19:59.733325+0000","last_became_peered":"2024-04-26T21:19:59.733325+0000","last_unstale":"2024-04-26T21:20:01.331579+0000","last_undegraded":"2024-04-26T21:20:01.331579+0000","last_fullsized":"2024-04-26T21:20:01.331579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:43.789437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733803+0000","last_change":"2024-04-26T21:19:59.737585+0000","last_active":"2024-04-26T21:20:00.733803+0000","last_peered":"2024-04-26T21:20:00.733803+0000","last_clean":"2024-04-26T21:20:00.733803+0000","last_became_active":"2024-04-26T21:19:59.735626+0000","last_became_peered":"2024-04-26T21:19:59.735626+0000","last_unstale":"2024-04-26T21:20:00.733803+0000","last_undegraded":"2024-04-26T21:20:00.733803+0000","last_fullsized":"2024-04-26T21:20:00.733803+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:04.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330961+0000","last_change":"2024-04-26T21:19:59.736557+0000","last_active":"2024-04-26T21:20:01.330961+0000","last_peered":"2024-04-26T21:20:01.330961+0000","last_clean":"2024-04-26T21:20:01.330961+0000","last_became_active":"2024-04-26T21:19:59.735989+0000","last_became_peered":"2024-04-26T21:19:59.735989+0000","last_unstale":"2024-04-26T21:20:01.330961+0000","last_undegraded":"2024-04-26T21:20:01.330961+0000","last_fullsized":"2024-04-26T21:20:01.330961+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:06.346153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.734754+0000","last_change":"2024-04-26T21:19:59.732840+0000","last_active":"2024-04-26T21:19:59.734754+0000","last_peered":"2024-04-26T21:19:59.734754+0000","last_clean":"2024-04-26T21:19:59.734754+0000","last_became_active":"2024-04-26T21:19:59.732549+0000","last_became_peered":"2024-04-26T21:19:59.732549+0000","last_unstale":"2024-04-26T21:19:59.734754+0000","last_undegraded":"2024-04-26T21:19:59.734754+0000","last_fullsized":"2024-04-26T21:19:59.734754+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:47.410089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331969+0000","last_change":"2024-04-26T21:19:59.741217+0000","last_active":"2024-04-26T21:20:01.331969+0000","last_peered":"2024-04-26T21:20:01.331969+0000","last_clean":"2024-04-26T21:20:01.331969+0000","last_became_active":"2024-04-26T21:19:59.739898+0000","last_became_peered":"2024-04-26T21:19:59.739898+0000","last_unstale":"2024-04-26T21:20:01.331969+0000","last_undegraded":"2024-04-26T21:20:01.331969+0000","last_fullsized":"2024-04-26T21:20:01.331969+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:27.371133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731372+0000","last_change":"2024-04-26T21:19:59.734661+0000","last_active":"2024-04-26T21:20:00.731372+0000","last_peered":"2024-04-26T21:20:00.731372+0000","last_clean":"2024-04-26T21:20:00.731372+0000","last_became_active":"2024-04-26T21:19:59.734398+0000","last_became_peered":"2024-04-26T21:19:59.734398+0000","last_unstale":"2024-04-26T21:20:00.731372+0000","last_undegraded":"2024-04-26T21:20:00.731372+0000","last_fullsized":"2024-04-26T21:20:00.731372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:14.286604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732863+0000","last_change":"2024-04-26T21:19:59.746591+0000","last_active":"2024-04-26T21:20:00.732863+0000","last_peered":"2024-04-26T21:20:00.732863+0000","last_clean":"2024-04-26T21:20:00.732863+0000","last_became_active":"2024-04-26T21:19:59.746332+0000","last_became_peered":"2024-04-26T21:19:59.746332+0000","last_unstale":"2024-04-26T21:20:00.732863+0000","last_undegraded":"2024-04-26T21:20:00.732863+0000","last_fullsized":"2024-04-26T21:20:00.732863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:56.733811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731588+0000","last_change":"2024-04-26T21:19:59.736974+0000","last_active":"2024-04-26T21:20:00.731588+0000","last_peered":"2024-04-26T21:20:00.731588+0000","last_clean":"2024-04-26T21:20:00.731588+0000","last_became_active":"2024-04-26T21:19:59.736373+0000","last_became_peered":"2024-04-26T21:19:59.736373+0000","last_unstale":"2024-04-26T21:20:00.731588+0000","last_undegraded":"2024-04-26T21:20:00.731588+0000","last_fullsized":"2024-04-26T21:20:00.731588+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:04.888279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731341+0000","last_change":"2024-04-26T21:19:59.736643+0000","last_active":"2024-04-26T21:20:00.731341+0000","last_peered":"2024-04-26T21:20:00.731341+0000","last_clean":"2024-04-26T21:20:00.731341+0000","last_became_active":"2024-04-26T21:19:59.735982+0000","last_became_peered":"2024-04-26T21:19:59.735982+0000","last_unstale":"2024-04-26T21:20:00.731341+0000","last_undegraded":"2024-04-26T21:20:00.731341+0000","last_fullsized":"2024-04-26T21:20:00.731341+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:47.294410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331507+0000","last_change":"2024-04-26T21:19:59.733293+0000","last_active":"2024-04-26T21:20:01.331507+0000","last_peered":"2024-04-26T21:20:01.331507+0000","last_clean":"2024-04-26T21:20:01.331507+0000","last_became_active":"2024-04-26T21:19:59.733025+0000","last_became_peered":"2024-04-26T21:19:59.733025+0000","last_unstale":"2024-04-26T21:20:01.331507+0000","last_undegraded":"2024-04-26T21:20:01.331507+0000","last_fullsized":"2024-04-26T21:20:01.331507+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:01.426059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331363+0000","last_change":"2024-04-26T21:19:59.748579+0000","last_active":"2024-04-26T21:20:01.331363+0000","last_peered":"2024-04-26T21:20:01.331363+0000","last_clean":"2024-04-26T21:20:01.331363+0000","last_became_active":"2024-04-26T21:19:59.748315+0000","last_became_peered":"2024-04-26T21:19:59.748315+0000","last_unstale":"2024-04-26T21:20:01.331363+0000","last_undegraded":"2024-04-26T21:20:01.331363+0000","last_fullsized":"2024-04-26T21:20:01.331363+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:31.428915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332165+0000","last_change":"2024-04-26T21:19:59.743633+0000","last_active":"2024-04-26T21:20:01.332165+0000","last_peered":"2024-04-26T21:20:01.332165+0000","last_clean":"2024-04-26T21:20:01.332165+0000","last_became_active":"2024-04-26T21:19:59.743315+0000","last_became_peered":"2024-04-26T21:19:59.743315+0000","last_unstale":"2024-04-26T21:20:01.332165+0000","last_undegraded":"2024-04-26T21:20:01.332165+0000","last_fullsized":"2024-04-26T21:20:01.332165+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:43.928940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331944+0000","last_change":"2024-04-26T21:19:58.725284+0000","last_active":"2024-04-26T21:20:01.331944+0000","last_peered":"2024-04-26T21:20:01.331944+0000","last_clean":"2024-04-26T21:20:01.331944+0000","last_became_active":"2024-04-26T21:19:58.725042+0000","last_became_peered":"2024-04-26T21:19:58.725042+0000","last_unstale":"2024-04-26T21:20:01.331944+0000","last_undegraded":"2024-04-26T21:20:01.331944+0000","last_fullsized":"2024-04-26T21:20:01.331944+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:35.142245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733427+0000","last_change":"2024-04-26T21:19:59.741239+0000","last_active":"2024-04-26T21:20:00.733427+0000","last_peered":"2024-04-26T21:20:00.733427+0000","last_clean":"2024-04-26T21:20:00.733427+0000","last_became_active":"2024-04-26T21:19:59.740965+0000","last_became_peered":"2024-04-26T21:19:59.740965+0000","last_unstale":"2024-04-26T21:20:00.733427+0000","last_undegraded":"2024-04-26T21:20:00.733427+0000","last_fullsized":"2024-04-26T21:20:00.733427+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:44.082993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732919+0000","last_change":"2024-04-26T21:19:58.723316+0000","last_active":"2024-04-26T21:20:00.732919+0000","last_peered":"2024-04-26T21:20:00.732919+0000","last_clean":"2024-04-26T21:20:00.732919+0000","last_became_active":"2024-04-26T21:19:58.722959+0000","last_became_peered":"2024-04-26T21:19:58.722959+0000","last_unstale":"2024-04-26T21:20:00.732919+0000","last_undegraded":"2024-04-26T21:20:00.732919+0000","last_fullsized":"2024-04-26T21:20:00.732919+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.132046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331853+0000","last_change":"2024-04-26T21:19:59.737451+0000","last_active":"2024-04-26T21:20:01.331853+0000","last_peered":"2024-04-26T21:20:01.331853+0000","last_clean":"2024-04-26T21:20:01.331853+0000","last_became_active":"2024-04-26T21:19:59.737141+0000","last_became_peered":"2024-04-26T21:19:59.737141+0000","last_unstale":"2024-04-26T21:20:01.331853+0000","last_undegraded":"2024-04-26T21:20:01.331853+0000","last_fullsized":"2024-04-26T21:20:01.331853+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:11.213452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331302+0000","last_change":"2024-04-26T21:19:58.719431+0000","last_active":"2024-04-26T21:20:01.331302+0000","last_peered":"2024-04-26T21:20:01.331302+0000","last_clean":"2024-04-26T21:20:01.331302+0000","last_became_active":"2024-04-26T21:19:58.719106+0000","last_became_peered":"2024-04-26T21:19:58.719106+0000","last_unstale":"2024-04-26T21:20:01.331302+0000","last_undegraded":"2024-04-26T21:20:01.331302+0000","last_fullsized":"2024-04-26T21:20:01.331302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:23.427070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331151+0000","last_change":"2024-04-26T21:19:59.736357+0000","last_active":"2024-04-26T21:20:01.331151+0000","last_peered":"2024-04-26T21:20:01.331151+0000","last_clean":"2024-04-26T21:20:01.331151+0000","last_became_active":"2024-04-26T21:19:59.736017+0000","last_became_peered":"2024-04-26T21:19:59.736017+0000","last_unstale":"2024-04-26T21:20:01.331151+0000","last_undegraded":"2024-04-26T21:20:01.331151+0000","last_fullsized":"2024-04-26T21:20:01.331151+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.714404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.754755+0000","last_change":"2024-04-26T21:19:58.724703+0000","last_active":"2024-04-26T21:19:59.754755+0000","last_peered":"2024-04-26T21:19:59.754755+0000","last_clean":"2024-04-26T21:19:59.754755+0000","last_became_active":"2024-04-26T21:19:58.724402+0000","last_became_peered":"2024-04-26T21:19:58.724402+0000","last_unstale":"2024-04-26T21:19:59.754755+0000","last_undegraded":"2024-04-26T21:19:59.754755+0000","last_fullsized":"2024-04-26T21:19:59.754755+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.297668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331853+0000","last_change":"2024-04-26T21:19:59.752282+0000","last_active":"2024-04-26T21:20:01.331853+0000","last_peered":"2024-04-26T21:20:01.331853+0000","last_clean":"2024-04-26T21:20:01.331853+0000","last_became_active":"2024-04-26T21:19:59.751977+0000","last_became_peered":"2024-04-26T21:19:59.751977+0000","last_unstale":"2024-04-26T21:20:01.331853+0000","last_undegraded":"2024-04-26T21:20:01.331853+0000","last_fullsized":"2024-04-26T21:20:01.331853+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.903302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.756334+0000","last_change":"2024-04-26T21:19:58.729375+0000","last_active":"2024-04-26T21:19:59.756334+0000","last_peered":"2024-04-26T21:19:59.756334+0000","last_clean":"2024-04-26T21:19:59.756334+0000","last_became_active":"2024-04-26T21:19:58.729056+0000","last_became_peered":"2024-04-26T21:19:58.729056+0000","last_unstale":"2024-04-26T21:19:59.756334+0000","last_undegraded":"2024-04-26T21:19:59.756334+0000","last_fullsized":"2024-04-26T21:19:59.756334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:53.987276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331295+0000","last_change":"2024-04-26T21:19:59.738633+0000","last_active":"2024-04-26T21:20:01.331295+0000","last_peered":"2024-04-26T21:20:01.331295+0000","last_clean":"2024-04-26T21:20:01.331295+0000","last_became_active":"2024-04-26T21:19:59.736697+0000","last_became_peered":"2024-04-26T21:19:59.736697+0000","last_unstale":"2024-04-26T21:20:01.331295+0000","last_undegraded":"2024-04-26T21:20:01.331295+0000","last_fullsized":"2024-04-26T21:20:01.331295+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.655606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331929+0000","last_change":"2024-04-26T21:19:58.724843+0000","last_active":"2024-04-26T21:20:01.331929+0000","last_peered":"2024-04-26T21:20:01.331929+0000","last_clean":"2024-04-26T21:20:01.331929+0000","last_became_active":"2024-04-26T21:19:58.724555+0000","last_became_peered":"2024-04-26T21:19:58.724555+0000","last_unstale":"2024-04-26T21:20:01.331929+0000","last_undegraded":"2024-04-26T21:20:01.331929+0000","last_fullsized":"2024-04-26T21:20:01.331929+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:10.581613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733449+0000","last_change":"2024-04-26T21:19:59.743381+0000","last_active":"2024-04-26T21:20:00.733449+0000","last_peered":"2024-04-26T21:20:00.733449+0000","last_clean":"2024-04-26T21:20:00.733449+0000","last_became_active":"2024-04-26T21:19:59.742686+0000","last_became_peered":"2024-04-26T21:19:59.742686+0000","last_unstale":"2024-04-26T21:20:00.733449+0000","last_undegraded":"2024-04-26T21:20:00.733449+0000","last_fullsized":"2024-04-26T21:20:00.733449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:31.428915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330878+0000","last_change":"2024-04-26T21:19:58.727712+0000","last_active":"2024-04-26T21:20:01.330878+0000","last_peered":"2024-04-26T21:20:01.330878+0000","last_clean":"2024-04-26T21:20:01.330878+0000","last_became_active":"2024-04-26T21:19:58.727492+0000","last_became_peered":"2024-04-26T21:19:58.727492+0000","last_unstale":"2024-04-26T21:20:01.330878+0000","last_undegraded":"2024-04-26T21:20:01.330878+0000","last_fullsized":"2024-04-26T21:20:01.330878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:29.954861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.747310+0000","last_change":"2024-04-26T21:19:59.742882+0000","last_active":"2024-04-26T21:19:59.747310+0000","last_peered":"2024-04-26T21:19:59.747310+0000","last_clean":"2024-04-26T21:19:59.747310+0000","last_became_active":"2024-04-26T21:19:59.742590+0000","last_became_peered":"2024-04-26T21:19:59.742590+0000","last_unstale":"2024-04-26T21:19:59.747310+0000","last_undegraded":"2024-04-26T21:19:59.747310+0000","last_fullsized":"2024-04-26T21:19:59.747310+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:48.636497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.724352+0000","last_change":"2024-04-26T21:19:58.727584+0000","last_active":"2024-04-26T21:19:59.724352+0000","last_peered":"2024-04-26T21:19:59.724352+0000","last_clean":"2024-04-26T21:19:59.724352+0000","last_became_active":"2024-04-26T21:19:58.727319+0000","last_became_peered":"2024-04-26T21:19:58.727319+0000","last_unstale":"2024-04-26T21:19:59.724352+0000","last_undegraded":"2024-04-26T21:19:59.724352+0000","last_fullsized":"2024-04-26T21:19:59.724352+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:28.723204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731478+0000","last_change":"2024-04-26T21:19:59.738798+0000","last_active":"2024-04-26T21:20:00.731478+0000","last_peered":"2024-04-26T21:20:00.731478+0000","last_clean":"2024-04-26T21:20:00.731478+0000","last_became_active":"2024-04-26T21:19:59.738487+0000","last_became_peered":"2024-04-26T21:19:59.738487+0000","last_unstale":"2024-04-26T21:20:00.731478+0000","last_undegraded":"2024-04-26T21:20:00.731478+0000","last_fullsized":"2024-04-26T21:20:00.731478+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:13.771920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733346+0000","last_change":"2024-04-26T21:19:58.721714+0000","last_active":"2024-04-26T21:20:00.733346+0000","last_peered":"2024-04-26T21:20:00.733346+0000","last_clean":"2024-04-26T21:20:00.733346+0000","last_became_active":"2024-04-26T21:19:58.721465+0000","last_became_peered":"2024-04-26T21:19:58.721465+0000","last_unstale":"2024-04-26T21:20:00.733346+0000","last_undegraded":"2024-04-26T21:20:00.733346+0000","last_fullsized":"2024-04-26T21:20:00.733346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:17.038499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733202+0000","last_change":"2024-04-26T21:19:59.750126+0000","last_active":"2024-04-26T21:20:00.733202+0000","last_peered":"2024-04-26T21:20:00.733202+0000","last_clean":"2024-04-26T21:20:00.733202+0000","last_became_active":"2024-04-26T21:19:59.749826+0000","last_became_peered":"2024-04-26T21:19:59.749826+0000","last_unstale":"2024-04-26T21:20:00.733202+0000","last_undegraded":"2024-04-26T21:20:00.733202+0000","last_fullsized":"2024-04-26T21:20:00.733202+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:21.264648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332095+0000","last_change":"2024-04-26T21:19:58.725110+0000","last_active":"2024-04-26T21:20:01.332095+0000","last_peered":"2024-04-26T21:20:01.332095+0000","last_clean":"2024-04-26T21:20:01.332095+0000","last_became_active":"2024-04-26T21:19:58.724877+0000","last_became_peered":"2024-04-26T21:19:58.724877+0000","last_unstale":"2024-04-26T21:20:01.332095+0000","last_undegraded":"2024-04-26T21:20:01.332095+0000","last_fullsized":"2024-04-26T21:20:01.332095+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:32.841437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331905+0000","last_change":"2024-04-26T21:19:59.748929+0000","last_active":"2024-04-26T21:20:01.331905+0000","last_peered":"2024-04-26T21:20:01.331905+0000","last_clean":"2024-04-26T21:20:01.331905+0000","last_became_active":"2024-04-26T21:19:59.748496+0000","last_became_peered":"2024-04-26T21:19:59.748496+0000","last_unstale":"2024-04-26T21:20:01.331905+0000","last_undegraded":"2024-04-26T21:20:01.331905+0000","last_fullsized":"2024-04-26T21:20:01.331905+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:46.481083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733062+0000","last_change":"2024-04-26T21:19:58.719609+0000","last_active":"2024-04-26T21:20:00.733062+0000","last_peered":"2024-04-26T21:20:00.733062+0000","last_clean":"2024-04-26T21:20:00.733062+0000","last_became_active":"2024-04-26T21:19:58.719297+0000","last_became_peered":"2024-04-26T21:19:58.719297+0000","last_unstale":"2024-04-26T21:20:00.733062+0000","last_undegraded":"2024-04-26T21:20:00.733062+0000","last_fullsized":"2024-04-26T21:20:00.733062+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:50.271573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331326+0000","last_change":"2024-04-26T21:19:59.735869+0000","last_active":"2024-04-26T21:20:01.331326+0000","last_peered":"2024-04-26T21:20:01.331326+0000","last_clean":"2024-04-26T21:20:01.331326+0000","last_became_active":"2024-04-26T21:19:59.735547+0000","last_became_peered":"2024-04-26T21:19:59.735547+0000","last_unstale":"2024-04-26T21:20:01.331326+0000","last_undegraded":"2024-04-26T21:20:01.331326+0000","last_fullsized":"2024-04-26T21:20:01.331326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:00.000224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733586+0000","last_change":"2024-04-26T21:19:58.725217+0000","last_active":"2024-04-26T21:20:00.733586+0000","last_peered":"2024-04-26T21:20:00.733586+0000","last_clean":"2024-04-26T21:20:00.733586+0000","last_became_active":"2024-04-26T21:19:58.724942+0000","last_became_peered":"2024-04-26T21:19:58.724942+0000","last_unstale":"2024-04-26T21:20:00.733586+0000","last_undegraded":"2024-04-26T21:20:00.733586+0000","last_fullsized":"2024-04-26T21:20:00.733586+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:20.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731671+0000","last_change":"2024-04-26T21:19:59.738453+0000","last_active":"2024-04-26T21:20:00.731671+0000","last_peered":"2024-04-26T21:20:00.731671+0000","last_clean":"2024-04-26T21:20:00.731671+0000","last_became_active":"2024-04-26T21:19:59.738094+0000","last_became_peered":"2024-04-26T21:19:59.738094+0000","last_unstale":"2024-04-26T21:20:00.731671+0000","last_undegraded":"2024-04-26T21:20:00.731671+0000","last_fullsized":"2024-04-26T21:20:00.731671+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:51.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731546+0000","last_change":"2024-04-26T21:19:58.722720+0000","last_active":"2024-04-26T21:20:00.731546+0000","last_peered":"2024-04-26T21:20:00.731546+0000","last_clean":"2024-04-26T21:20:00.731546+0000","last_became_active":"2024-04-26T21:19:58.722374+0000","last_became_peered":"2024-04-26T21:19:58.722374+0000","last_unstale":"2024-04-26T21:20:00.731546+0000","last_undegraded":"2024-04-26T21:20:00.731546+0000","last_fullsized":"2024-04-26T21:20:00.731546+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:33.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.744172+0000","last_change":"2024-04-26T21:19:59.735713+0000","last_active":"2024-04-26T21:19:59.744172+0000","last_peered":"2024-04-26T21:19:59.744172+0000","last_clean":"2024-04-26T21:19:59.744172+0000","last_became_active":"2024-04-26T21:19:59.735447+0000","last_became_peered":"2024-04-26T21:19:59.735447+0000","last_unstale":"2024-04-26T21:19:59.744172+0000","last_undegraded":"2024-04-26T21:19:59.744172+0000","last_fullsized":"2024-04-26T21:19:59.744172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.161102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.724969+0000","last_change":"2024-04-26T21:19:58.727436+0000","last_active":"2024-04-26T21:19:59.724969+0000","last_peered":"2024-04-26T21:19:59.724969+0000","last_clean":"2024-04-26T21:19:59.724969+0000","last_became_active":"2024-04-26T21:19:58.727148+0000","last_became_peered":"2024-04-26T21:19:58.727148+0000","last_unstale":"2024-04-26T21:19:59.724969+0000","last_undegraded":"2024-04-26T21:19:59.724969+0000","last_fullsized":"2024-04-26T21:19:59.724969+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:45.362967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733572+0000","last_change":"2024-04-26T21:19:59.735973+0000","last_active":"2024-04-26T21:20:00.733572+0000","last_peered":"2024-04-26T21:20:00.733572+0000","last_clean":"2024-04-26T21:20:00.733572+0000","last_became_active":"2024-04-26T21:19:59.735628+0000","last_became_peered":"2024-04-26T21:19:59.735628+0000","last_unstale":"2024-04-26T21:20:00.733572+0000","last_undegraded":"2024-04-26T21:20:00.733572+0000","last_fullsized":"2024-04-26T21:20:00.733572+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.655606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":22,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733329+0000","last_change":"2024-04-26T21:19:58.724319+0000","last_active":"2024-04-26T21:20:00.733329+0000","last_peered":"2024-04-26T21:20:00.733329+0000","last_clean":"2024-04-26T21:20:00.733329+0000","last_became_active":"2024-04-26T21:19:58.724018+0000","last_became_peered":"2024-04-26T21:19:58.724018+0000","last_unstale":"2024-04-26T21:20:00.733329+0000","last_undegraded":"2024-04-26T21:20:00.733329+0000","last_fullsized":"2024-04-26T21:20:00.733329+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:07.710179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732657+0000","last_change":"2024-04-26T21:19:59.742052+0000","last_active":"2024-04-26T21:20:00.732657+0000","last_peered":"2024-04-26T21:20:00.732657+0000","last_clean":"2024-04-26T21:20:00.732657+0000","last_became_active":"2024-04-26T21:19:59.741625+0000","last_became_peered":"2024-04-26T21:19:59.741625+0000","last_unstale":"2024-04-26T21:20:00.732657+0000","last_undegraded":"2024-04-26T21:20:00.732657+0000","last_fullsized":"2024-04-26T21:20:00.732657+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:53.424094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733443+0000","last_change":"2024-04-26T21:19:58.725519+0000","last_active":"2024-04-26T21:20:00.733443+0000","last_peered":"2024-04-26T21:20:00.733443+0000","last_clean":"2024-04-26T21:20:00.733443+0000","last_became_active":"2024-04-26T21:19:58.725209+0000","last_became_peered":"2024-04-26T21:19:58.725209+0000","last_unstale":"2024-04-26T21:20:00.733443+0000","last_undegraded":"2024-04-26T21:20:00.733443+0000","last_fullsized":"2024-04-26T21:20:00.733443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:51.923983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733677+0000","last_change":"2024-04-26T21:19:59.739608+0000","last_active":"2024-04-26T21:20:00.733677+0000","last_peered":"2024-04-26T21:20:00.733677+0000","last_clean":"2024-04-26T21:20:00.733677+0000","last_became_active":"2024-04-26T21:19:59.739219+0000","last_became_peered":"2024-04-26T21:19:59.739219+0000","last_unstale":"2024-04-26T21:20:00.733677+0000","last_undegraded":"2024-04-26T21:20:00.733677+0000","last_fullsized":"2024-04-26T21:20:00.733677+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:08.715498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331061+0000","last_change":"2024-04-26T21:19:58.725103+0000","last_active":"2024-04-26T21:20:01.331061+0000","last_peered":"2024-04-26T21:20:01.331061+0000","last_clean":"2024-04-26T21:20:01.331061+0000","last_became_active":"2024-04-26T21:19:58.724771+0000","last_became_peered":"2024-04-26T21:19:58.724771+0000","last_unstale":"2024-04-26T21:20:01.331061+0000","last_undegraded":"2024-04-26T21:20:01.331061+0000","last_fullsized":"2024-04-26T21:20:01.331061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:10.208133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332440+0000","last_change":"2024-04-26T21:19:59.737601+0000","last_active":"2024-04-26T21:20:01.332440+0000","last_peered":"2024-04-26T21:20:01.332440+0000","last_clean":"2024-04-26T21:20:01.332440+0000","last_became_active":"2024-04-26T21:19:59.734268+0000","last_became_peered":"2024-04-26T21:19:59.734268+0000","last_unstale":"2024-04-26T21:20:01.332440+0000","last_undegraded":"2024-04-26T21:20:01.332440+0000","last_fullsized":"2024-04-26T21:20:01.332440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.714404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331767+0000","last_change":"2024-04-26T21:19:58.729450+0000","last_active":"2024-04-26T21:20:01.331767+0000","last_peered":"2024-04-26T21:20:01.331767+0000","last_clean":"2024-04-26T21:20:01.331767+0000","last_became_active":"2024-04-26T21:19:58.729140+0000","last_became_peered":"2024-04-26T21:19:58.729140+0000","last_unstale":"2024-04-26T21:20:01.331767+0000","last_undegraded":"2024-04-26T21:20:01.331767+0000","last_fullsized":"2024-04-26T21:20:01.331767+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:13.770285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.748238+0000","last_change":"2024-04-26T21:19:59.743699+0000","last_active":"2024-04-26T21:19:59.748238+0000","last_peered":"2024-04-26T21:19:59.748238+0000","last_clean":"2024-04-26T21:19:59.748238+0000","last_became_active":"2024-04-26T21:19:59.743289+0000","last_became_peered":"2024-04-26T21:19:59.743289+0000","last_unstale":"2024-04-26T21:19:59.748238+0000","last_undegraded":"2024-04-26T21:19:59.748238+0000","last_fullsized":"2024-04-26T21:19:59.748238+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:37.015957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732949+0000","last_change":"2024-04-26T21:19:58.723319+0000","last_active":"2024-04-26T21:20:00.732949+0000","last_peered":"2024-04-26T21:20:00.732949+0000","last_clean":"2024-04-26T21:20:00.732949+0000","last_became_active":"2024-04-26T21:19:58.723040+0000","last_became_peered":"2024-04-26T21:19:58.723040+0000","last_unstale":"2024-04-26T21:20:00.732949+0000","last_undegraded":"2024-04-26T21:20:00.732949+0000","last_fullsized":"2024-04-26T21:20:00.732949+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:35.439137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731494+0000","last_change":"2024-04-26T21:19:59.733110+0000","last_active":"2024-04-26T21:20:00.731494+0000","last_peered":"2024-04-26T21:20:00.731494+0000","last_clean":"2024-04-26T21:20:00.731494+0000","last_became_active":"2024-04-26T21:19:59.732800+0000","last_became_peered":"2024-04-26T21:19:59.732800+0000","last_unstale":"2024-04-26T21:20:00.731494+0000","last_undegraded":"2024-04-26T21:20:00.731494+0000","last_fullsized":"2024-04-26T21:20:00.731494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:05.097868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732778+0000","last_change":"2024-04-26T21:19:58.723046+0000","last_active":"2024-04-26T21:20:00.732778+0000","last_peered":"2024-04-26T21:20:00.732778+0000","last_clean":"2024-04-26T21:20:00.732778+0000","last_became_active":"2024-04-26T21:19:58.722742+0000","last_became_peered":"2024-04-26T21:19:58.722742+0000","last_unstale":"2024-04-26T21:20:00.732778+0000","last_undegraded":"2024-04-26T21:20:00.732778+0000","last_fullsized":"2024-04-26T21:20:00.732778+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:30.918041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331449+0000","last_change":"2024-04-26T21:19:59.736368+0000","last_active":"2024-04-26T21:20:01.331449+0000","last_peered":"2024-04-26T21:20:01.331449+0000","last_clean":"2024-04-26T21:20:01.331449+0000","last_became_active":"2024-04-26T21:19:59.736060+0000","last_became_peered":"2024-04-26T21:19:59.736060+0000","last_unstale":"2024-04-26T21:20:01.331449+0000","last_undegraded":"2024-04-26T21:20:01.331449+0000","last_fullsized":"2024-04-26T21:20:01.331449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:38.088650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332010+0000","last_change":"2024-04-26T21:19:58.728401+0000","last_active":"2024-04-26T21:20:01.332010+0000","last_peered":"2024-04-26T21:20:01.332010+0000","last_clean":"2024-04-26T21:20:01.332010+0000","last_became_active":"2024-04-26T21:19:58.728148+0000","last_became_peered":"2024-04-26T21:19:58.728148+0000","last_unstale":"2024-04-26T21:20:01.332010+0000","last_undegraded":"2024-04-26T21:20:01.332010+0000","last_fullsized":"2024-04-26T21:20:01.332010+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:37.083331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736563+0000","last_change":"2024-04-26T21:19:59.734586+0000","last_active":"2024-04-26T21:19:59.736563+0000","last_peered":"2024-04-26T21:19:59.736563+0000","last_clean":"2024-04-26T21:19:59.736563+0000","last_became_active":"2024-04-26T21:19:59.734306+0000","last_became_peered":"2024-04-26T21:19:59.734306+0000","last_unstale":"2024-04-26T21:19:59.736563+0000","last_undegraded":"2024-04-26T21:19:59.736563+0000","last_fullsized":"2024-04-26T21:19:59.736563+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:40.915182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331424+0000","last_change":"2024-04-26T21:19:58.726518+0000","last_active":"2024-04-26T21:20:01.331424+0000","last_peered":"2024-04-26T21:20:01.331424+0000","last_clean":"2024-04-26T21:20:01.331424+0000","last_became_active":"2024-04-26T21:19:58.726250+0000","last_became_peered":"2024-04-26T21:19:58.726250+0000","last_unstale":"2024-04-26T21:20:01.331424+0000","last_undegraded":"2024-04-26T21:20:01.331424+0000","last_fullsized":"2024-04-26T21:20:01.331424+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:35.650286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331394+0000","last_change":"2024-04-26T21:19:59.731970+0000","last_active":"2024-04-26T21:20:01.331394+0000","last_peered":"2024-04-26T21:20:01.331394+0000","last_clean":"2024-04-26T21:20:01.331394+0000","last_became_active":"2024-04-26T21:19:59.731538+0000","last_became_peered":"2024-04-26T21:19:59.731538+0000","last_unstale":"2024-04-26T21:20:01.331394+0000","last_undegraded":"2024-04-26T21:20:01.331394+0000","last_fullsized":"2024-04-26T21:20:01.331394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:22.117245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331023+0000","last_change":"2024-04-26T21:19:58.725139+0000","last_active":"2024-04-26T21:20:01.331023+0000","last_peered":"2024-04-26T21:20:01.331023+0000","last_clean":"2024-04-26T21:20:01.331023+0000","last_became_active":"2024-04-26T21:19:58.724786+0000","last_became_peered":"2024-04-26T21:19:58.724786+0000","last_unstale":"2024-04-26T21:20:01.331023+0000","last_undegraded":"2024-04-26T21:20:01.331023+0000","last_fullsized":"2024-04-26T21:20:01.331023+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:02.921569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331782+0000","last_change":"2024-04-26T21:19:59.735122+0000","last_active":"2024-04-26T21:20:01.331782+0000","last_peered":"2024-04-26T21:20:01.331782+0000","last_clean":"2024-04-26T21:20:01.331782+0000","last_became_active":"2024-04-26T21:19:59.734855+0000","last_became_peered":"2024-04-26T21:19:59.734855+0000","last_unstale":"2024-04-26T21:20:01.331782+0000","last_undegraded":"2024-04-26T21:20:01.331782+0000","last_fullsized":"2024-04-26T21:20:01.331782+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:05.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731507+0000","last_change":"2024-04-26T21:19:58.717264+0000","last_active":"2024-04-26T21:20:00.731507+0000","last_peered":"2024-04-26T21:20:00.731507+0000","last_clean":"2024-04-26T21:20:00.731507+0000","last_became_active":"2024-04-26T21:19:58.716586+0000","last_became_peered":"2024-04-26T21:19:58.716586+0000","last_unstale":"2024-04-26T21:20:00.731507+0000","last_undegraded":"2024-04-26T21:20:00.731507+0000","last_fullsized":"2024-04-26T21:20:00.731507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:37.548856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731600+0000","last_change":"2024-04-26T21:19:59.742191+0000","last_active":"2024-04-26T21:20:00.731600+0000","last_peered":"2024-04-26T21:20:00.731600+0000","last_clean":"2024-04-26T21:20:00.731600+0000","last_became_active":"2024-04-26T21:19:59.738252+0000","last_became_peered":"2024-04-26T21:19:59.738252+0000","last_unstale":"2024-04-26T21:20:00.731600+0000","last_undegraded":"2024-04-26T21:20:00.731600+0000","last_fullsized":"2024-04-26T21:20:00.731600+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:24.059383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"122'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331131+0000","last_change":"2024-04-26T21:19:58.726369+0000","last_active":"2024-04-26T21:20:01.331131+0000","last_peered":"2024-04-26T21:20:01.331131+0000","last_clean":"2024-04-26T21:20:01.331131+0000","last_became_active":"2024-04-26T21:19:58.726018+0000","last_became_peered":"2024-04-26T21:19:58.726018+0000","last_unstale":"2024-04-26T21:20:01.331131+0000","last_undegraded":"2024-04-26T21:20:01.331131+0000","last_fullsized":"2024-04-26T21:20:01.331131+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:25.079272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736456+0000","last_change":"2024-04-26T21:19:59.734315+0000","last_active":"2024-04-26T21:19:59.736456+0000","last_peered":"2024-04-26T21:19:59.736456+0000","last_clean":"2024-04-26T21:19:59.736456+0000","last_became_active":"2024-04-26T21:19:59.733974+0000","last_became_peered":"2024-04-26T21:19:59.733974+0000","last_unstale":"2024-04-26T21:19:59.736456+0000","last_undegraded":"2024-04-26T21:19:59.736456+0000","last_fullsized":"2024-04-26T21:19:59.736456+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:10.580605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.722980+0000","last_change":"2024-04-26T21:19:58.726792+0000","last_active":"2024-04-26T21:19:59.722980+0000","last_peered":"2024-04-26T21:19:59.722980+0000","last_clean":"2024-04-26T21:19:59.722980+0000","last_became_active":"2024-04-26T21:19:58.726491+0000","last_became_peered":"2024-04-26T21:19:58.726491+0000","last_unstale":"2024-04-26T21:19:59.722980+0000","last_undegraded":"2024-04-26T21:19:59.722980+0000","last_fullsized":"2024-04-26T21:19:59.722980+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:59.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331746+0000","last_change":"2024-04-26T21:19:59.742165+0000","last_active":"2024-04-26T21:20:01.331746+0000","last_peered":"2024-04-26T21:20:01.331746+0000","last_clean":"2024-04-26T21:20:01.331746+0000","last_became_active":"2024-04-26T21:19:59.741364+0000","last_became_peered":"2024-04-26T21:19:59.741364+0000","last_unstale":"2024-04-26T21:20:01.331746+0000","last_undegraded":"2024-04-26T21:20:01.331746+0000","last_fullsized":"2024-04-26T21:20:01.331746+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.987473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331672+0000","last_change":"2024-04-26T21:19:58.725526+0000","last_active":"2024-04-26T21:20:01.331672+0000","last_peered":"2024-04-26T21:20:01.331672+0000","last_clean":"2024-04-26T21:20:01.331672+0000","last_became_active":"2024-04-26T21:19:58.725195+0000","last_became_peered":"2024-04-26T21:19:58.725195+0000","last_unstale":"2024-04-26T21:20:01.331672+0000","last_undegraded":"2024-04-26T21:20:01.331672+0000","last_fullsized":"2024-04-26T21:20:01.331672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:45.475764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331496+0000","last_change":"2024-04-26T21:19:59.733691+0000","last_active":"2024-04-26T21:20:01.331496+0000","last_peered":"2024-04-26T21:20:01.331496+0000","last_clean":"2024-04-26T21:20:01.331496+0000","last_became_active":"2024-04-26T21:19:59.733406+0000","last_became_peered":"2024-04-26T21:19:59.733406+0000","last_unstale":"2024-04-26T21:20:01.331496+0000","last_undegraded":"2024-04-26T21:20:01.331496+0000","last_fullsized":"2024-04-26T21:20:01.331496+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:04.661637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.722652+0000","last_change":"2024-04-26T21:19:58.728134+0000","last_active":"2024-04-26T21:19:59.722652+0000","last_peered":"2024-04-26T21:19:59.722652+0000","last_clean":"2024-04-26T21:19:59.722652+0000","last_became_active":"2024-04-26T21:19:58.727846+0000","last_became_peered":"2024-04-26T21:19:58.727846+0000","last_unstale":"2024-04-26T21:19:59.722652+0000","last_undegraded":"2024-04-26T21:19:59.722652+0000","last_fullsized":"2024-04-26T21:19:59.722652+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:55.620133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732717+0000","last_change":"2024-04-26T21:19:59.743339+0000","last_active":"2024-04-26T21:20:00.732717+0000","last_peered":"2024-04-26T21:20:00.732717+0000","last_clean":"2024-04-26T21:20:00.732717+0000","last_became_active":"2024-04-26T21:19:59.743084+0000","last_became_peered":"2024-04-26T21:19:59.743084+0000","last_unstale":"2024-04-26T21:20:00.732717+0000","last_undegraded":"2024-04-26T21:20:00.732717+0000","last_fullsized":"2024-04-26T21:20:00.732717+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:16.763428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331556+0000","last_change":"2024-04-26T21:19:58.727049+0000","last_active":"2024-04-26T21:20:01.331556+0000","last_peered":"2024-04-26T21:20:01.331556+0000","last_clean":"2024-04-26T21:20:01.331556+0000","last_became_active":"2024-04-26T21:19:58.725578+0000","last_became_peered":"2024-04-26T21:19:58.725578+0000","last_unstale":"2024-04-26T21:20:01.331556+0000","last_undegraded":"2024-04-26T21:20:01.331556+0000","last_fullsized":"2024-04-26T21:20:01.331556+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:04.806615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331534+0000","last_change":"2024-04-26T21:19:59.749499+0000","last_active":"2024-04-26T21:20:01.331534+0000","last_peered":"2024-04-26T21:20:01.331534+0000","last_clean":"2024-04-26T21:20:01.331534+0000","last_became_active":"2024-04-26T21:19:59.748961+0000","last_became_peered":"2024-04-26T21:19:59.748961+0000","last_unstale":"2024-04-26T21:20:01.331534+0000","last_undegraded":"2024-04-26T21:20:01.331534+0000","last_fullsized":"2024-04-26T21:20:01.331534+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:54.154897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331611+0000","last_change":"2024-04-26T21:19:58.727762+0000","last_active":"2024-04-26T21:20:01.331611+0000","last_peered":"2024-04-26T21:20:01.331611+0000","last_clean":"2024-04-26T21:20:01.331611+0000","last_became_active":"2024-04-26T21:19:58.727498+0000","last_became_peered":"2024-04-26T21:19:58.727498+0000","last_unstale":"2024-04-26T21:20:01.331611+0000","last_undegraded":"2024-04-26T21:20:01.331611+0000","last_fullsized":"2024-04-26T21:20:01.331611+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:03.642636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.731444+0000","last_change":"2024-04-26T21:19:59.729048+0000","last_active":"2024-04-26T21:19:59.731444+0000","last_peered":"2024-04-26T21:19:59.731444+0000","last_clean":"2024-04-26T21:19:59.731444+0000","last_became_active":"2024-04-26T21:19:59.728704+0000","last_became_peered":"2024-04-26T21:19:59.728704+0000","last_unstale":"2024-04-26T21:19:59.731444+0000","last_undegraded":"2024-04-26T21:19:59.731444+0000","last_fullsized":"2024-04-26T21:19:59.731444+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.027523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331128+0000","last_change":"2024-04-26T21:19:58.722265+0000","last_active":"2024-04-26T21:20:01.331128+0000","last_peered":"2024-04-26T21:20:01.331128+0000","last_clean":"2024-04-26T21:20:01.331128+0000","last_became_active":"2024-04-26T21:19:58.721957+0000","last_became_peered":"2024-04-26T21:19:58.721957+0000","last_unstale":"2024-04-26T21:20:01.331128+0000","last_undegraded":"2024-04-26T21:20:01.331128+0000","last_fullsized":"2024-04-26T21:20:01.331128+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:45.350609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331336+0000","last_change":"2024-04-26T21:19:59.733181+0000","last_active":"2024-04-26T21:20:01.331336+0000","last_peered":"2024-04-26T21:20:01.331336+0000","last_clean":"2024-04-26T21:20:01.331336+0000","last_became_active":"2024-04-26T21:19:59.732929+0000","last_became_peered":"2024-04-26T21:19:59.732929+0000","last_unstale":"2024-04-26T21:20:01.331336+0000","last_undegraded":"2024-04-26T21:20:01.331336+0000","last_fullsized":"2024-04-26T21:20:01.331336+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:06.706265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331772+0000","last_change":"2024-04-26T21:19:58.728363+0000","last_active":"2024-04-26T21:20:01.331772+0000","last_peered":"2024-04-26T21:20:01.331772+0000","last_clean":"2024-04-26T21:20:01.331772+0000","last_became_active":"2024-04-26T21:19:58.728098+0000","last_became_peered":"2024-04-26T21:19:58.728098+0000","last_unstale":"2024-04-26T21:20:01.331772+0000","last_undegraded":"2024-04-26T21:20:01.331772+0000","last_fullsized":"2024-04-26T21:20:01.331772+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:47.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":351,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330908+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:01.330908+0000","last_peered":"2024-04-26T21:20:01.330908+0000","last_clean":"2024-04-26T21:20:01.330908+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:01.330908+0000","last_undegraded":"2024-04-26T21:20:01.330908+0000","last_fullsized":"2024-04-26T21:20:01.330908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509543,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1244330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":653324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":651824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":651782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509542,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":651872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509542,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1523712,"data_stored":652328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509542,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2109440,"data_stored":1242080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36800,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":1490944,"data_stored":638637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:03.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:03.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.880+0000 7f9980579640 1 Processor -- start 2024-04-26T21:20:03.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.880+0000 7f9980579640 1 -- start start 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99780d11b0 0x7f997806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 0x7f997815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f997806e750 con 0x7f99780d11b0 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f99780d1740 con 0x7f997806e2c0 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f9980579640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99780d1a20 con 0x7f99780cf2e0 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f997db14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99780d11b0 0x7f997806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f997db14640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99780d11b0 0x7f997806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59888/0 (socket says 172.21.15.40:59888) 2024-04-26T21:20:03.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.881+0000 7f997db14640 1 -- 172.21.15.40:0/3412324861 learned_addr learned my addr 172.21.15.40:0/3412324861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997eb16640 1 --2- 172.21.15.40:0/3412324861 >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 0x7f997815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 -- 172.21.15.40:0/3412324861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 msgr2=0x7f997806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997e315640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 -- 172.21.15.40:0/3412324861 >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 msgr2=0x7f997815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 --2- 172.21.15.40:0/3412324861 >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 0x7f997815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 -- 172.21.15.40:0/3412324861 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780d1d00 con 0x7f99780d11b0 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997eb16640 1 --2- 172.21.15.40:0/3412324861 >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 0x7f997815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997e315640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:03.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.882+0000 7f997db14640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997806dd80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9978163ff0 tx=0x7f996c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21ebba3b6e9aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f997d313640 1 -- 172.21.15.40:0/3412324861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c037070 con 0x7f99780d11b0 2024-04-26T21:20:03.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f997d313640 1 -- 172.21.15.40:0/3412324861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c032440 con 0x7f99780d11b0 2024-04-26T21:20:03.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f997d313640 1 -- 172.21.15.40:0/3412324861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c032810 con 0x7f99780d11b0 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 -- 172.21.15.40:0/3412324861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 msgr2=0x7f997806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997806dd80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9978163ff0 tx=0x7f996c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 -- 172.21.15.40:0/3412324861 shutdown_connections 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 --2- 172.21.15.40:0/3412324861 >> v2:172.21.15.184:3300/0 conn(0x7f997806e2c0 0x7f997815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997806dd80 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 --2- 172.21.15.40:0/3412324861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 -- 172.21.15.40:0/3412324861 >> 172.21.15.40:0/3412324861 conn(0x7f99780b4a60 msgr2=0x7f99780b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 -- 172.21.15.40:0/3412324861 shutdown_connections 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.883+0000 7f9980579640 1 -- 172.21.15.40:0/3412324861 wait complete. 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.884+0000 7f9980579640 1 Processor -- start 2024-04-26T21:20:03.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.884+0000 7f9980579640 1 -- start start 2024-04-26T21:20:03.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f997806e2c0 0x7f997809ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997809ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f997806da10 con 0x7f99780d11b0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99780d22c0 con 0x7f997806e2c0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f9980579640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f99780cf6c0 con 0x7f99780cf2e0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997eb16640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997eb16640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59896/0 (socket says 172.21.15.40:59896) 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997eb16640 1 -- 172.21.15.40:0/3786466815 learned_addr learned my addr 172.21.15.40:0/3786466815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997db14640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997809ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997e315640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f997806e2c0 0x7f997809ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997eb16640 1 -- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 msgr2=0x7f997809ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.885+0000 7f997eb16640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997809ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f997eb16640 1 -- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f997806e2c0 msgr2=0x7f997809ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f997eb16640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f997806e2c0 0x7f997809ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f997eb16640 1 -- 172.21.15.40:0/3786466815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780cd1f0 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f997eb16640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f996800c7f0 tx=0x7f996800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968015070 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968007820 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99780c8680 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968007c10 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.886+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99780c8c40 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.887+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997806a290 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.887+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9968018070 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.888+0000 7f9976ffd640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 0x7f996406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.888+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f996808bb40 con 0x7f99780d11b0 2024-04-26T21:20:03.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.888+0000 7f997e315640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 0x7f996406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:03.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.888+0000 7f997e315640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 0x7f996406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f99780c9710 tx=0x7f9970007990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:03.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:03.892+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9968059f60 con 0x7f99780d11b0 2024-04-26T21:20:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.073+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99780c8e10 con 0x7f996406c8f0 2024-04-26T21:20:04.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.080+0000 7f9976ffd640 1 -- 172.21.15.40:0/3786466815 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252471 (secure 0 0 0) 0x7f99780c8e10 con 0x7f996406c8f0 2024-04-26T21:20:04.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:04.085 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 msgr2=0x7f996406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 0x7f996406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f99780c9710 tx=0x7f9970007990 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 msgr2=0x7f997809f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f996800c7f0 tx=0x7f996800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 shutdown_connections 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f996406c8f0 0x7f996406edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99780d11b0 0x7f997809f480 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99780cf2e0 0x7f997809ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 --2- 172.21.15.40:0/3786466815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f997806e2c0 0x7f997809ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 >> 172.21.15.40:0/3786466815 conn(0x7f99780b4a60 msgr2=0x7f9978070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 shutdown_connections 2024-04-26T21:20:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.083+0000 7f9980579640 1 -- 172.21.15.40:0/3786466815 wait complete. 2024-04-26T21:20:04.095 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-26T21:20:03.035382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270832,"kb_used_data":13040,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703888,"statfs":{"total":773094113280,"available":772816781312,"internally_reserved":0,"allocated":13352960,"data_stored":6386177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.329807"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731455+0000","last_change":"2024-04-26T21:19:59.733872+0000","last_active":"2024-04-26T21:20:00.731455+0000","last_peered":"2024-04-26T21:20:00.731455+0000","last_clean":"2024-04-26T21:20:00.731455+0000","last_became_active":"2024-04-26T21:19:59.733535+0000","last_became_peered":"2024-04-26T21:19:59.733535+0000","last_unstale":"2024-04-26T21:20:00.731455+0000","last_undegraded":"2024-04-26T21:20:00.731455+0000","last_fullsized":"2024-04-26T21:20:00.731455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:20.583522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.734709+0000","last_change":"2024-04-26T21:19:59.732087+0000","last_active":"2024-04-26T21:19:59.734709+0000","last_peered":"2024-04-26T21:19:59.734709+0000","last_clean":"2024-04-26T21:19:59.734709+0000","last_became_active":"2024-04-26T21:19:59.731700+0000","last_became_peered":"2024-04-26T21:19:59.731700+0000","last_unstale":"2024-04-26T21:19:59.734709+0000","last_undegraded":"2024-04-26T21:19:59.734709+0000","last_fullsized":"2024-04-26T21:19:59.734709+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:13.608672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733040+0000","last_change":"2024-04-26T21:19:59.738269+0000","last_active":"2024-04-26T21:20:00.733040+0000","last_peered":"2024-04-26T21:20:00.733040+0000","last_clean":"2024-04-26T21:20:00.733040+0000","last_became_active":"2024-04-26T21:19:59.737930+0000","last_became_peered":"2024-04-26T21:19:59.737930+0000","last_unstale":"2024-04-26T21:20:00.733040+0000","last_undegraded":"2024-04-26T21:20:00.733040+0000","last_fullsized":"2024-04-26T21:20:00.733040+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:37.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731284+0000","last_change":"2024-04-26T21:19:59.737753+0000","last_active":"2024-04-26T21:20:00.731284+0000","last_peered":"2024-04-26T21:20:00.731284+0000","last_clean":"2024-04-26T21:20:00.731284+0000","last_became_active":"2024-04-26T21:19:59.736072+0000","last_became_peered":"2024-04-26T21:19:59.736072+0000","last_unstale":"2024-04-26T21:20:00.731284+0000","last_undegraded":"2024-04-26T21:20:00.731284+0000","last_fullsized":"2024-04-26T21:20:00.731284+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:49.851261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331699+0000","last_change":"2024-04-26T21:19:59.739748+0000","last_active":"2024-04-26T21:20:01.331699+0000","last_peered":"2024-04-26T21:20:01.331699+0000","last_clean":"2024-04-26T21:20:01.331699+0000","last_became_active":"2024-04-26T21:19:59.739443+0000","last_became_peered":"2024-04-26T21:19:59.739443+0000","last_unstale":"2024-04-26T21:20:01.331699+0000","last_undegraded":"2024-04-26T21:20:01.331699+0000","last_fullsized":"2024-04-26T21:20:01.331699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:31.202273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331261+0000","last_change":"2024-04-26T21:19:59.742243+0000","last_active":"2024-04-26T21:20:01.331261+0000","last_peered":"2024-04-26T21:20:01.331261+0000","last_clean":"2024-04-26T21:20:01.331261+0000","last_became_active":"2024-04-26T21:19:59.741404+0000","last_became_peered":"2024-04-26T21:19:59.741404+0000","last_unstale":"2024-04-26T21:20:01.331261+0000","last_undegraded":"2024-04-26T21:20:01.331261+0000","last_fullsized":"2024-04-26T21:20:01.331261+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:33.846756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331449+0000","last_change":"2024-04-26T21:19:59.736465+0000","last_active":"2024-04-26T21:20:01.331449+0000","last_peered":"2024-04-26T21:20:01.331449+0000","last_clean":"2024-04-26T21:20:01.331449+0000","last_became_active":"2024-04-26T21:19:59.736076+0000","last_became_peered":"2024-04-26T21:19:59.736076+0000","last_unstale":"2024-04-26T21:20:01.331449+0000","last_undegraded":"2024-04-26T21:20:01.331449+0000","last_fullsized":"2024-04-26T21:20:01.331449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:04.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733267+0000","last_change":"2024-04-26T21:19:59.743604+0000","last_active":"2024-04-26T21:20:00.733267+0000","last_peered":"2024-04-26T21:20:00.733267+0000","last_clean":"2024-04-26T21:20:00.733267+0000","last_became_active":"2024-04-26T21:19:59.743102+0000","last_became_peered":"2024-04-26T21:19:59.743102+0000","last_unstale":"2024-04-26T21:20:00.733267+0000","last_undegraded":"2024-04-26T21:20:00.733267+0000","last_fullsized":"2024-04-26T21:20:00.733267+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:25.244936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733270+0000","last_change":"2024-04-26T21:19:59.736191+0000","last_active":"2024-04-26T21:20:00.733270+0000","last_peered":"2024-04-26T21:20:00.733270+0000","last_clean":"2024-04-26T21:20:00.733270+0000","last_became_active":"2024-04-26T21:19:59.735710+0000","last_became_peered":"2024-04-26T21:19:59.735710+0000","last_unstale":"2024-04-26T21:20:00.733270+0000","last_undegraded":"2024-04-26T21:20:00.733270+0000","last_fullsized":"2024-04-26T21:20:00.733270+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:33:08.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736494+0000","last_change":"2024-04-26T21:19:59.733848+0000","last_active":"2024-04-26T21:19:59.736494+0000","last_peered":"2024-04-26T21:19:59.736494+0000","last_clean":"2024-04-26T21:19:59.736494+0000","last_became_active":"2024-04-26T21:19:59.732101+0000","last_became_peered":"2024-04-26T21:19:59.732101+0000","last_unstale":"2024-04-26T21:19:59.736494+0000","last_undegraded":"2024-04-26T21:19:59.736494+0000","last_fullsized":"2024-04-26T21:19:59.736494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:06.854976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331482+0000","last_change":"2024-04-26T21:19:59.735448+0000","last_active":"2024-04-26T21:20:01.331482+0000","last_peered":"2024-04-26T21:20:01.331482+0000","last_clean":"2024-04-26T21:20:01.331482+0000","last_became_active":"2024-04-26T21:19:59.735133+0000","last_became_peered":"2024-04-26T21:19:59.735133+0000","last_unstale":"2024-04-26T21:20:01.331482+0000","last_undegraded":"2024-04-26T21:20:01.331482+0000","last_fullsized":"2024-04-26T21:20:01.331482+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:02.533988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733534+0000","last_change":"2024-04-26T21:19:59.736254+0000","last_active":"2024-04-26T21:20:00.733534+0000","last_peered":"2024-04-26T21:20:00.733534+0000","last_clean":"2024-04-26T21:20:00.733534+0000","last_became_active":"2024-04-26T21:19:59.735612+0000","last_became_peered":"2024-04-26T21:19:59.735612+0000","last_unstale":"2024-04-26T21:20:00.733534+0000","last_undegraded":"2024-04-26T21:20:00.733534+0000","last_fullsized":"2024-04-26T21:20:00.733534+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:33.846756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731347+0000","last_change":"2024-04-26T21:19:59.737724+0000","last_active":"2024-04-26T21:20:00.731347+0000","last_peered":"2024-04-26T21:20:00.731347+0000","last_clean":"2024-04-26T21:20:00.731347+0000","last_became_active":"2024-04-26T21:19:59.737404+0000","last_became_peered":"2024-04-26T21:19:59.737404+0000","last_unstale":"2024-04-26T21:20:00.731347+0000","last_undegraded":"2024-04-26T21:20:00.731347+0000","last_fullsized":"2024-04-26T21:20:00.731347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:33.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.747633+0000","last_change":"2024-04-26T21:19:59.741481+0000","last_active":"2024-04-26T21:19:59.747633+0000","last_peered":"2024-04-26T21:19:59.747633+0000","last_clean":"2024-04-26T21:19:59.747633+0000","last_became_active":"2024-04-26T21:19:59.741185+0000","last_became_peered":"2024-04-26T21:19:59.741185+0000","last_unstale":"2024-04-26T21:19:59.747633+0000","last_undegraded":"2024-04-26T21:19:59.747633+0000","last_fullsized":"2024-04-26T21:19:59.747633+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:22.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731304+0000","last_change":"2024-04-26T21:19:59.741512+0000","last_active":"2024-04-26T21:20:00.731304+0000","last_peered":"2024-04-26T21:20:00.731304+0000","last_clean":"2024-04-26T21:20:00.731304+0000","last_became_active":"2024-04-26T21:19:59.741227+0000","last_became_peered":"2024-04-26T21:19:59.741227+0000","last_unstale":"2024-04-26T21:20:00.731304+0000","last_undegraded":"2024-04-26T21:20:00.731304+0000","last_fullsized":"2024-04-26T21:20:00.731304+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:59.603706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331328+0000","last_change":"2024-04-26T21:19:59.743606+0000","last_active":"2024-04-26T21:20:01.331328+0000","last_peered":"2024-04-26T21:20:01.331328+0000","last_clean":"2024-04-26T21:20:01.331328+0000","last_became_active":"2024-04-26T21:19:59.743300+0000","last_became_peered":"2024-04-26T21:19:59.743300+0000","last_unstale":"2024-04-26T21:20:01.331328+0000","last_undegraded":"2024-04-26T21:20:01.331328+0000","last_fullsized":"2024-04-26T21:20:01.331328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:01.429730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331425+0000","last_change":"2024-04-26T21:19:59.737320+0000","last_active":"2024-04-26T21:20:01.331425+0000","last_peered":"2024-04-26T21:20:01.331425+0000","last_clean":"2024-04-26T21:20:01.331425+0000","last_became_active":"2024-04-26T21:19:59.736787+0000","last_became_peered":"2024-04-26T21:19:59.736787+0000","last_unstale":"2024-04-26T21:20:01.331425+0000","last_undegraded":"2024-04-26T21:20:01.331425+0000","last_fullsized":"2024-04-26T21:20:01.331425+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:11.586932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332033+0000","last_change":"2024-04-26T21:19:59.751536+0000","last_active":"2024-04-26T21:20:01.332033+0000","last_peered":"2024-04-26T21:20:01.332033+0000","last_clean":"2024-04-26T21:20:01.332033+0000","last_became_active":"2024-04-26T21:19:59.751211+0000","last_became_peered":"2024-04-26T21:19:59.751211+0000","last_unstale":"2024-04-26T21:20:01.332033+0000","last_undegraded":"2024-04-26T21:20:01.332033+0000","last_fullsized":"2024-04-26T21:20:01.332033+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:26.458504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331226+0000","last_change":"2024-04-26T21:19:59.734866+0000","last_active":"2024-04-26T21:20:01.331226+0000","last_peered":"2024-04-26T21:20:01.331226+0000","last_clean":"2024-04-26T21:20:01.331226+0000","last_became_active":"2024-04-26T21:19:59.734566+0000","last_became_peered":"2024-04-26T21:19:59.734566+0000","last_unstale":"2024-04-26T21:20:01.331226+0000","last_undegraded":"2024-04-26T21:20:01.331226+0000","last_fullsized":"2024-04-26T21:20:01.331226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:53.666517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332332+0000","last_change":"2024-04-26T21:19:59.748469+0000","last_active":"2024-04-26T21:20:01.332332+0000","last_peered":"2024-04-26T21:20:01.332332+0000","last_clean":"2024-04-26T21:20:01.332332+0000","last_became_active":"2024-04-26T21:19:59.748096+0000","last_became_peered":"2024-04-26T21:19:59.748096+0000","last_unstale":"2024-04-26T21:20:01.332332+0000","last_undegraded":"2024-04-26T21:20:01.332332+0000","last_fullsized":"2024-04-26T21:20:01.332332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:46.589006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331698+0000","last_change":"2024-04-26T21:19:59.739498+0000","last_active":"2024-04-26T21:20:01.331698+0000","last_peered":"2024-04-26T21:20:01.331698+0000","last_clean":"2024-04-26T21:20:01.331698+0000","last_became_active":"2024-04-26T21:19:59.739220+0000","last_became_peered":"2024-04-26T21:19:59.739220+0000","last_unstale":"2024-04-26T21:20:01.331698+0000","last_undegraded":"2024-04-26T21:20:01.331698+0000","last_fullsized":"2024-04-26T21:20:01.331698+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:01.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331579+0000","last_change":"2024-04-26T21:19:59.733660+0000","last_active":"2024-04-26T21:20:01.331579+0000","last_peered":"2024-04-26T21:20:01.331579+0000","last_clean":"2024-04-26T21:20:01.331579+0000","last_became_active":"2024-04-26T21:19:59.733325+0000","last_became_peered":"2024-04-26T21:19:59.733325+0000","last_unstale":"2024-04-26T21:20:01.331579+0000","last_undegraded":"2024-04-26T21:20:01.331579+0000","last_fullsized":"2024-04-26T21:20:01.331579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:43.789437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733803+0000","last_change":"2024-04-26T21:19:59.737585+0000","last_active":"2024-04-26T21:20:00.733803+0000","last_peered":"2024-04-26T21:20:00.733803+0000","last_clean":"2024-04-26T21:20:00.733803+0000","last_became_active":"2024-04-26T21:19:59.735626+0000","last_became_peered":"2024-04-26T21:19:59.735626+0000","last_unstale":"2024-04-26T21:20:00.733803+0000","last_undegraded":"2024-04-26T21:20:00.733803+0000","last_fullsized":"2024-04-26T21:20:00.733803+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:04.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330961+0000","last_change":"2024-04-26T21:19:59.736557+0000","last_active":"2024-04-26T21:20:01.330961+0000","last_peered":"2024-04-26T21:20:01.330961+0000","last_clean":"2024-04-26T21:20:01.330961+0000","last_became_active":"2024-04-26T21:19:59.735989+0000","last_became_peered":"2024-04-26T21:19:59.735989+0000","last_unstale":"2024-04-26T21:20:01.330961+0000","last_undegraded":"2024-04-26T21:20:01.330961+0000","last_fullsized":"2024-04-26T21:20:01.330961+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:06.346153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.734754+0000","last_change":"2024-04-26T21:19:59.732840+0000","last_active":"2024-04-26T21:19:59.734754+0000","last_peered":"2024-04-26T21:19:59.734754+0000","last_clean":"2024-04-26T21:19:59.734754+0000","last_became_active":"2024-04-26T21:19:59.732549+0000","last_became_peered":"2024-04-26T21:19:59.732549+0000","last_unstale":"2024-04-26T21:19:59.734754+0000","last_undegraded":"2024-04-26T21:19:59.734754+0000","last_fullsized":"2024-04-26T21:19:59.734754+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:47.410089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331969+0000","last_change":"2024-04-26T21:19:59.741217+0000","last_active":"2024-04-26T21:20:01.331969+0000","last_peered":"2024-04-26T21:20:01.331969+0000","last_clean":"2024-04-26T21:20:01.331969+0000","last_became_active":"2024-04-26T21:19:59.739898+0000","last_became_peered":"2024-04-26T21:19:59.739898+0000","last_unstale":"2024-04-26T21:20:01.331969+0000","last_undegraded":"2024-04-26T21:20:01.331969+0000","last_fullsized":"2024-04-26T21:20:01.331969+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:27.371133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731372+0000","last_change":"2024-04-26T21:19:59.734661+0000","last_active":"2024-04-26T21:20:00.731372+0000","last_peered":"2024-04-26T21:20:00.731372+0000","last_clean":"2024-04-26T21:20:00.731372+0000","last_became_active":"2024-04-26T21:19:59.734398+0000","last_became_peered":"2024-04-26T21:19:59.734398+0000","last_unstale":"2024-04-26T21:20:00.731372+0000","last_undegraded":"2024-04-26T21:20:00.731372+0000","last_fullsized":"2024-04-26T21:20:00.731372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:14.286604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732863+0000","last_change":"2024-04-26T21:19:59.746591+0000","last_active":"2024-04-26T21:20:00.732863+0000","last_peered":"2024-04-26T21:20:00.732863+0000","last_clean":"2024-04-26T21:20:00.732863+0000","last_became_active":"2024-04-26T21:19:59.746332+0000","last_became_peered":"2024-04-26T21:19:59.746332+0000","last_unstale":"2024-04-26T21:20:00.732863+0000","last_undegraded":"2024-04-26T21:20:00.732863+0000","last_fullsized":"2024-04-26T21:20:00.732863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:08:56.733811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731588+0000","last_change":"2024-04-26T21:19:59.736974+0000","last_active":"2024-04-26T21:20:00.731588+0000","last_peered":"2024-04-26T21:20:00.731588+0000","last_clean":"2024-04-26T21:20:00.731588+0000","last_became_active":"2024-04-26T21:19:59.736373+0000","last_became_peered":"2024-04-26T21:19:59.736373+0000","last_unstale":"2024-04-26T21:20:00.731588+0000","last_undegraded":"2024-04-26T21:20:00.731588+0000","last_fullsized":"2024-04-26T21:20:00.731588+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:04.888279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731341+0000","last_change":"2024-04-26T21:19:59.736643+0000","last_active":"2024-04-26T21:20:00.731341+0000","last_peered":"2024-04-26T21:20:00.731341+0000","last_clean":"2024-04-26T21:20:00.731341+0000","last_became_active":"2024-04-26T21:19:59.735982+0000","last_became_peered":"2024-04-26T21:19:59.735982+0000","last_unstale":"2024-04-26T21:20:00.731341+0000","last_undegraded":"2024-04-26T21:20:00.731341+0000","last_fullsized":"2024-04-26T21:20:00.731341+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:47.294410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331507+0000","last_change":"2024-04-26T21:19:59.733293+0000","last_active":"2024-04-26T21:20:01.331507+0000","last_peered":"2024-04-26T21:20:01.331507+0000","last_clean":"2024-04-26T21:20:01.331507+0000","last_became_active":"2024-04-26T21:19:59.733025+0000","last_became_peered":"2024-04-26T21:19:59.733025+0000","last_unstale":"2024-04-26T21:20:01.331507+0000","last_undegraded":"2024-04-26T21:20:01.331507+0000","last_fullsized":"2024-04-26T21:20:01.331507+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:01.426059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331363+0000","last_change":"2024-04-26T21:19:59.748579+0000","last_active":"2024-04-26T21:20:01.331363+0000","last_peered":"2024-04-26T21:20:01.331363+0000","last_clean":"2024-04-26T21:20:01.331363+0000","last_became_active":"2024-04-26T21:19:59.748315+0000","last_became_peered":"2024-04-26T21:19:59.748315+0000","last_unstale":"2024-04-26T21:20:01.331363+0000","last_undegraded":"2024-04-26T21:20:01.331363+0000","last_fullsized":"2024-04-26T21:20:01.331363+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:31.428915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332165+0000","last_change":"2024-04-26T21:19:59.743633+0000","last_active":"2024-04-26T21:20:01.332165+0000","last_peered":"2024-04-26T21:20:01.332165+0000","last_clean":"2024-04-26T21:20:01.332165+0000","last_became_active":"2024-04-26T21:19:59.743315+0000","last_became_peered":"2024-04-26T21:19:59.743315+0000","last_unstale":"2024-04-26T21:20:01.332165+0000","last_undegraded":"2024-04-26T21:20:01.332165+0000","last_fullsized":"2024-04-26T21:20:01.332165+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:43.928940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331944+0000","last_change":"2024-04-26T21:19:58.725284+0000","last_active":"2024-04-26T21:20:01.331944+0000","last_peered":"2024-04-26T21:20:01.331944+0000","last_clean":"2024-04-26T21:20:01.331944+0000","last_became_active":"2024-04-26T21:19:58.725042+0000","last_became_peered":"2024-04-26T21:19:58.725042+0000","last_unstale":"2024-04-26T21:20:01.331944+0000","last_undegraded":"2024-04-26T21:20:01.331944+0000","last_fullsized":"2024-04-26T21:20:01.331944+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:35.142245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733427+0000","last_change":"2024-04-26T21:19:59.741239+0000","last_active":"2024-04-26T21:20:00.733427+0000","last_peered":"2024-04-26T21:20:00.733427+0000","last_clean":"2024-04-26T21:20:00.733427+0000","last_became_active":"2024-04-26T21:19:59.740965+0000","last_became_peered":"2024-04-26T21:19:59.740965+0000","last_unstale":"2024-04-26T21:20:00.733427+0000","last_undegraded":"2024-04-26T21:20:00.733427+0000","last_fullsized":"2024-04-26T21:20:00.733427+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:44.082993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732919+0000","last_change":"2024-04-26T21:19:58.723316+0000","last_active":"2024-04-26T21:20:00.732919+0000","last_peered":"2024-04-26T21:20:00.732919+0000","last_clean":"2024-04-26T21:20:00.732919+0000","last_became_active":"2024-04-26T21:19:58.722959+0000","last_became_peered":"2024-04-26T21:19:58.722959+0000","last_unstale":"2024-04-26T21:20:00.732919+0000","last_undegraded":"2024-04-26T21:20:00.732919+0000","last_fullsized":"2024-04-26T21:20:00.732919+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:59.132046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331853+0000","last_change":"2024-04-26T21:19:59.737451+0000","last_active":"2024-04-26T21:20:01.331853+0000","last_peered":"2024-04-26T21:20:01.331853+0000","last_clean":"2024-04-26T21:20:01.331853+0000","last_became_active":"2024-04-26T21:19:59.737141+0000","last_became_peered":"2024-04-26T21:19:59.737141+0000","last_unstale":"2024-04-26T21:20:01.331853+0000","last_undegraded":"2024-04-26T21:20:01.331853+0000","last_fullsized":"2024-04-26T21:20:01.331853+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:11.213452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331302+0000","last_change":"2024-04-26T21:19:58.719431+0000","last_active":"2024-04-26T21:20:01.331302+0000","last_peered":"2024-04-26T21:20:01.331302+0000","last_clean":"2024-04-26T21:20:01.331302+0000","last_became_active":"2024-04-26T21:19:58.719106+0000","last_became_peered":"2024-04-26T21:19:58.719106+0000","last_unstale":"2024-04-26T21:20:01.331302+0000","last_undegraded":"2024-04-26T21:20:01.331302+0000","last_fullsized":"2024-04-26T21:20:01.331302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:23.427070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331151+0000","last_change":"2024-04-26T21:19:59.736357+0000","last_active":"2024-04-26T21:20:01.331151+0000","last_peered":"2024-04-26T21:20:01.331151+0000","last_clean":"2024-04-26T21:20:01.331151+0000","last_became_active":"2024-04-26T21:19:59.736017+0000","last_became_peered":"2024-04-26T21:19:59.736017+0000","last_unstale":"2024-04-26T21:20:01.331151+0000","last_undegraded":"2024-04-26T21:20:01.331151+0000","last_fullsized":"2024-04-26T21:20:01.331151+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.714404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.754755+0000","last_change":"2024-04-26T21:19:58.724703+0000","last_active":"2024-04-26T21:19:59.754755+0000","last_peered":"2024-04-26T21:19:59.754755+0000","last_clean":"2024-04-26T21:19:59.754755+0000","last_became_active":"2024-04-26T21:19:58.724402+0000","last_became_peered":"2024-04-26T21:19:58.724402+0000","last_unstale":"2024-04-26T21:19:59.754755+0000","last_undegraded":"2024-04-26T21:19:59.754755+0000","last_fullsized":"2024-04-26T21:19:59.754755+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.297668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331853+0000","last_change":"2024-04-26T21:19:59.752282+0000","last_active":"2024-04-26T21:20:01.331853+0000","last_peered":"2024-04-26T21:20:01.331853+0000","last_clean":"2024-04-26T21:20:01.331853+0000","last_became_active":"2024-04-26T21:19:59.751977+0000","last_became_peered":"2024-04-26T21:19:59.751977+0000","last_unstale":"2024-04-26T21:20:01.331853+0000","last_undegraded":"2024-04-26T21:20:01.331853+0000","last_fullsized":"2024-04-26T21:20:01.331853+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:00.903302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.756334+0000","last_change":"2024-04-26T21:19:58.729375+0000","last_active":"2024-04-26T21:19:59.756334+0000","last_peered":"2024-04-26T21:19:59.756334+0000","last_clean":"2024-04-26T21:19:59.756334+0000","last_became_active":"2024-04-26T21:19:58.729056+0000","last_became_peered":"2024-04-26T21:19:58.729056+0000","last_unstale":"2024-04-26T21:19:59.756334+0000","last_undegraded":"2024-04-26T21:19:59.756334+0000","last_fullsized":"2024-04-26T21:19:59.756334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:53.987276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331295+0000","last_change":"2024-04-26T21:19:59.738633+0000","last_active":"2024-04-26T21:20:01.331295+0000","last_peered":"2024-04-26T21:20:01.331295+0000","last_clean":"2024-04-26T21:20:01.331295+0000","last_became_active":"2024-04-26T21:19:59.736697+0000","last_became_peered":"2024-04-26T21:19:59.736697+0000","last_unstale":"2024-04-26T21:20:01.331295+0000","last_undegraded":"2024-04-26T21:20:01.331295+0000","last_fullsized":"2024-04-26T21:20:01.331295+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.655606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331929+0000","last_change":"2024-04-26T21:19:58.724843+0000","last_active":"2024-04-26T21:20:01.331929+0000","last_peered":"2024-04-26T21:20:01.331929+0000","last_clean":"2024-04-26T21:20:01.331929+0000","last_became_active":"2024-04-26T21:19:58.724555+0000","last_became_peered":"2024-04-26T21:19:58.724555+0000","last_unstale":"2024-04-26T21:20:01.331929+0000","last_undegraded":"2024-04-26T21:20:01.331929+0000","last_fullsized":"2024-04-26T21:20:01.331929+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:10.581613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733449+0000","last_change":"2024-04-26T21:19:59.743381+0000","last_active":"2024-04-26T21:20:00.733449+0000","last_peered":"2024-04-26T21:20:00.733449+0000","last_clean":"2024-04-26T21:20:00.733449+0000","last_became_active":"2024-04-26T21:19:59.742686+0000","last_became_peered":"2024-04-26T21:19:59.742686+0000","last_unstale":"2024-04-26T21:20:00.733449+0000","last_undegraded":"2024-04-26T21:20:00.733449+0000","last_fullsized":"2024-04-26T21:20:00.733449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:31.428915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330878+0000","last_change":"2024-04-26T21:19:58.727712+0000","last_active":"2024-04-26T21:20:01.330878+0000","last_peered":"2024-04-26T21:20:01.330878+0000","last_clean":"2024-04-26T21:20:01.330878+0000","last_became_active":"2024-04-26T21:19:58.727492+0000","last_became_peered":"2024-04-26T21:19:58.727492+0000","last_unstale":"2024-04-26T21:20:01.330878+0000","last_undegraded":"2024-04-26T21:20:01.330878+0000","last_fullsized":"2024-04-26T21:20:01.330878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:21:29.954861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.747310+0000","last_change":"2024-04-26T21:19:59.742882+0000","last_active":"2024-04-26T21:19:59.747310+0000","last_peered":"2024-04-26T21:19:59.747310+0000","last_clean":"2024-04-26T21:19:59.747310+0000","last_became_active":"2024-04-26T21:19:59.742590+0000","last_became_peered":"2024-04-26T21:19:59.742590+0000","last_unstale":"2024-04-26T21:19:59.747310+0000","last_undegraded":"2024-04-26T21:19:59.747310+0000","last_fullsized":"2024-04-26T21:19:59.747310+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:48.636497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.724352+0000","last_change":"2024-04-26T21:19:58.727584+0000","last_active":"2024-04-26T21:19:59.724352+0000","last_peered":"2024-04-26T21:19:59.724352+0000","last_clean":"2024-04-26T21:19:59.724352+0000","last_became_active":"2024-04-26T21:19:58.727319+0000","last_became_peered":"2024-04-26T21:19:58.727319+0000","last_unstale":"2024-04-26T21:19:59.724352+0000","last_undegraded":"2024-04-26T21:19:59.724352+0000","last_fullsized":"2024-04-26T21:19:59.724352+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:28.723204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731478+0000","last_change":"2024-04-26T21:19:59.738798+0000","last_active":"2024-04-26T21:20:00.731478+0000","last_peered":"2024-04-26T21:20:00.731478+0000","last_clean":"2024-04-26T21:20:00.731478+0000","last_became_active":"2024-04-26T21:19:59.738487+0000","last_became_peered":"2024-04-26T21:19:59.738487+0000","last_unstale":"2024-04-26T21:20:00.731478+0000","last_undegraded":"2024-04-26T21:20:00.731478+0000","last_fullsized":"2024-04-26T21:20:00.731478+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:13.771920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733346+0000","last_change":"2024-04-26T21:19:58.721714+0000","last_active":"2024-04-26T21:20:00.733346+0000","last_peered":"2024-04-26T21:20:00.733346+0000","last_clean":"2024-04-26T21:20:00.733346+0000","last_became_active":"2024-04-26T21:19:58.721465+0000","last_became_peered":"2024-04-26T21:19:58.721465+0000","last_unstale":"2024-04-26T21:20:00.733346+0000","last_undegraded":"2024-04-26T21:20:00.733346+0000","last_fullsized":"2024-04-26T21:20:00.733346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:17.038499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733202+0000","last_change":"2024-04-26T21:19:59.750126+0000","last_active":"2024-04-26T21:20:00.733202+0000","last_peered":"2024-04-26T21:20:00.733202+0000","last_clean":"2024-04-26T21:20:00.733202+0000","last_became_active":"2024-04-26T21:19:59.749826+0000","last_became_peered":"2024-04-26T21:19:59.749826+0000","last_unstale":"2024-04-26T21:20:00.733202+0000","last_undegraded":"2024-04-26T21:20:00.733202+0000","last_fullsized":"2024-04-26T21:20:00.733202+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:21.264648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332095+0000","last_change":"2024-04-26T21:19:58.725110+0000","last_active":"2024-04-26T21:20:01.332095+0000","last_peered":"2024-04-26T21:20:01.332095+0000","last_clean":"2024-04-26T21:20:01.332095+0000","last_became_active":"2024-04-26T21:19:58.724877+0000","last_became_peered":"2024-04-26T21:19:58.724877+0000","last_unstale":"2024-04-26T21:20:01.332095+0000","last_undegraded":"2024-04-26T21:20:01.332095+0000","last_fullsized":"2024-04-26T21:20:01.332095+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:51:32.841437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331905+0000","last_change":"2024-04-26T21:19:59.748929+0000","last_active":"2024-04-26T21:20:01.331905+0000","last_peered":"2024-04-26T21:20:01.331905+0000","last_clean":"2024-04-26T21:20:01.331905+0000","last_became_active":"2024-04-26T21:19:59.748496+0000","last_became_peered":"2024-04-26T21:19:59.748496+0000","last_unstale":"2024-04-26T21:20:01.331905+0000","last_undegraded":"2024-04-26T21:20:01.331905+0000","last_fullsized":"2024-04-26T21:20:01.331905+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:46.481083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733062+0000","last_change":"2024-04-26T21:19:58.719609+0000","last_active":"2024-04-26T21:20:00.733062+0000","last_peered":"2024-04-26T21:20:00.733062+0000","last_clean":"2024-04-26T21:20:00.733062+0000","last_became_active":"2024-04-26T21:19:58.719297+0000","last_became_peered":"2024-04-26T21:19:58.719297+0000","last_unstale":"2024-04-26T21:20:00.733062+0000","last_undegraded":"2024-04-26T21:20:00.733062+0000","last_fullsized":"2024-04-26T21:20:00.733062+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:50.271573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331326+0000","last_change":"2024-04-26T21:19:59.735869+0000","last_active":"2024-04-26T21:20:01.331326+0000","last_peered":"2024-04-26T21:20:01.331326+0000","last_clean":"2024-04-26T21:20:01.331326+0000","last_became_active":"2024-04-26T21:19:59.735547+0000","last_became_peered":"2024-04-26T21:19:59.735547+0000","last_unstale":"2024-04-26T21:20:01.331326+0000","last_undegraded":"2024-04-26T21:20:01.331326+0000","last_fullsized":"2024-04-26T21:20:01.331326+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:00.000224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733586+0000","last_change":"2024-04-26T21:19:58.725217+0000","last_active":"2024-04-26T21:20:00.733586+0000","last_peered":"2024-04-26T21:20:00.733586+0000","last_clean":"2024-04-26T21:20:00.733586+0000","last_became_active":"2024-04-26T21:19:58.724942+0000","last_became_peered":"2024-04-26T21:19:58.724942+0000","last_unstale":"2024-04-26T21:20:00.733586+0000","last_undegraded":"2024-04-26T21:20:00.733586+0000","last_fullsized":"2024-04-26T21:20:00.733586+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:20.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731671+0000","last_change":"2024-04-26T21:19:59.738453+0000","last_active":"2024-04-26T21:20:00.731671+0000","last_peered":"2024-04-26T21:20:00.731671+0000","last_clean":"2024-04-26T21:20:00.731671+0000","last_became_active":"2024-04-26T21:19:59.738094+0000","last_became_peered":"2024-04-26T21:19:59.738094+0000","last_unstale":"2024-04-26T21:20:00.731671+0000","last_undegraded":"2024-04-26T21:20:00.731671+0000","last_fullsized":"2024-04-26T21:20:00.731671+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:51.276892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731546+0000","last_change":"2024-04-26T21:19:58.722720+0000","last_active":"2024-04-26T21:20:00.731546+0000","last_peered":"2024-04-26T21:20:00.731546+0000","last_clean":"2024-04-26T21:20:00.731546+0000","last_became_active":"2024-04-26T21:19:58.722374+0000","last_became_peered":"2024-04-26T21:19:58.722374+0000","last_unstale":"2024-04-26T21:20:00.731546+0000","last_undegraded":"2024-04-26T21:20:00.731546+0000","last_fullsized":"2024-04-26T21:20:00.731546+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:33.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.744172+0000","last_change":"2024-04-26T21:19:59.735713+0000","last_active":"2024-04-26T21:19:59.744172+0000","last_peered":"2024-04-26T21:19:59.744172+0000","last_clean":"2024-04-26T21:19:59.744172+0000","last_became_active":"2024-04-26T21:19:59.735447+0000","last_became_peered":"2024-04-26T21:19:59.735447+0000","last_unstale":"2024-04-26T21:19:59.744172+0000","last_undegraded":"2024-04-26T21:19:59.744172+0000","last_fullsized":"2024-04-26T21:19:59.744172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.161102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.724969+0000","last_change":"2024-04-26T21:19:58.727436+0000","last_active":"2024-04-26T21:19:59.724969+0000","last_peered":"2024-04-26T21:19:59.724969+0000","last_clean":"2024-04-26T21:19:59.724969+0000","last_became_active":"2024-04-26T21:19:58.727148+0000","last_became_peered":"2024-04-26T21:19:58.727148+0000","last_unstale":"2024-04-26T21:19:59.724969+0000","last_undegraded":"2024-04-26T21:19:59.724969+0000","last_fullsized":"2024-04-26T21:19:59.724969+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:45.362967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733572+0000","last_change":"2024-04-26T21:19:59.735973+0000","last_active":"2024-04-26T21:20:00.733572+0000","last_peered":"2024-04-26T21:20:00.733572+0000","last_clean":"2024-04-26T21:20:00.733572+0000","last_became_active":"2024-04-26T21:19:59.735628+0000","last_became_peered":"2024-04-26T21:19:59.735628+0000","last_unstale":"2024-04-26T21:20:00.733572+0000","last_undegraded":"2024-04-26T21:20:00.733572+0000","last_fullsized":"2024-04-26T21:20:00.733572+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:36.655606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":22,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733329+0000","last_change":"2024-04-26T21:19:58.724319+0000","last_active":"2024-04-26T21:20:00.733329+0000","last_peered":"2024-04-26T21:20:00.733329+0000","last_clean":"2024-04-26T21:20:00.733329+0000","last_became_active":"2024-04-26T21:19:58.724018+0000","last_became_peered":"2024-04-26T21:19:58.724018+0000","last_unstale":"2024-04-26T21:20:00.733329+0000","last_undegraded":"2024-04-26T21:20:00.733329+0000","last_fullsized":"2024-04-26T21:20:00.733329+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:07.710179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732657+0000","last_change":"2024-04-26T21:19:59.742052+0000","last_active":"2024-04-26T21:20:00.732657+0000","last_peered":"2024-04-26T21:20:00.732657+0000","last_clean":"2024-04-26T21:20:00.732657+0000","last_became_active":"2024-04-26T21:19:59.741625+0000","last_became_peered":"2024-04-26T21:19:59.741625+0000","last_unstale":"2024-04-26T21:20:00.732657+0000","last_undegraded":"2024-04-26T21:20:00.732657+0000","last_fullsized":"2024-04-26T21:20:00.732657+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:53.424094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733443+0000","last_change":"2024-04-26T21:19:58.725519+0000","last_active":"2024-04-26T21:20:00.733443+0000","last_peered":"2024-04-26T21:20:00.733443+0000","last_clean":"2024-04-26T21:20:00.733443+0000","last_became_active":"2024-04-26T21:19:58.725209+0000","last_became_peered":"2024-04-26T21:19:58.725209+0000","last_unstale":"2024-04-26T21:20:00.733443+0000","last_undegraded":"2024-04-26T21:20:00.733443+0000","last_fullsized":"2024-04-26T21:20:00.733443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:51.923983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.733677+0000","last_change":"2024-04-26T21:19:59.739608+0000","last_active":"2024-04-26T21:20:00.733677+0000","last_peered":"2024-04-26T21:20:00.733677+0000","last_clean":"2024-04-26T21:20:00.733677+0000","last_became_active":"2024-04-26T21:19:59.739219+0000","last_became_peered":"2024-04-26T21:19:59.739219+0000","last_unstale":"2024-04-26T21:20:00.733677+0000","last_undegraded":"2024-04-26T21:20:00.733677+0000","last_fullsized":"2024-04-26T21:20:00.733677+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:08.715498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331061+0000","last_change":"2024-04-26T21:19:58.725103+0000","last_active":"2024-04-26T21:20:01.331061+0000","last_peered":"2024-04-26T21:20:01.331061+0000","last_clean":"2024-04-26T21:20:01.331061+0000","last_became_active":"2024-04-26T21:19:58.724771+0000","last_became_peered":"2024-04-26T21:19:58.724771+0000","last_unstale":"2024-04-26T21:20:01.331061+0000","last_undegraded":"2024-04-26T21:20:01.331061+0000","last_fullsized":"2024-04-26T21:20:01.331061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:10.208133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332440+0000","last_change":"2024-04-26T21:19:59.737601+0000","last_active":"2024-04-26T21:20:01.332440+0000","last_peered":"2024-04-26T21:20:01.332440+0000","last_clean":"2024-04-26T21:20:01.332440+0000","last_became_active":"2024-04-26T21:19:59.734268+0000","last_became_peered":"2024-04-26T21:19:59.734268+0000","last_unstale":"2024-04-26T21:20:01.332440+0000","last_undegraded":"2024-04-26T21:20:01.332440+0000","last_fullsized":"2024-04-26T21:20:01.332440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:58.714404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331767+0000","last_change":"2024-04-26T21:19:58.729450+0000","last_active":"2024-04-26T21:20:01.331767+0000","last_peered":"2024-04-26T21:20:01.331767+0000","last_clean":"2024-04-26T21:20:01.331767+0000","last_became_active":"2024-04-26T21:19:58.729140+0000","last_became_peered":"2024-04-26T21:19:58.729140+0000","last_unstale":"2024-04-26T21:20:01.331767+0000","last_undegraded":"2024-04-26T21:20:01.331767+0000","last_fullsized":"2024-04-26T21:20:01.331767+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:13.770285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.748238+0000","last_change":"2024-04-26T21:19:59.743699+0000","last_active":"2024-04-26T21:19:59.748238+0000","last_peered":"2024-04-26T21:19:59.748238+0000","last_clean":"2024-04-26T21:19:59.748238+0000","last_became_active":"2024-04-26T21:19:59.743289+0000","last_became_peered":"2024-04-26T21:19:59.743289+0000","last_unstale":"2024-04-26T21:19:59.748238+0000","last_undegraded":"2024-04-26T21:19:59.748238+0000","last_fullsized":"2024-04-26T21:19:59.748238+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:37.015957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732949+0000","last_change":"2024-04-26T21:19:58.723319+0000","last_active":"2024-04-26T21:20:00.732949+0000","last_peered":"2024-04-26T21:20:00.732949+0000","last_clean":"2024-04-26T21:20:00.732949+0000","last_became_active":"2024-04-26T21:19:58.723040+0000","last_became_peered":"2024-04-26T21:19:58.723040+0000","last_unstale":"2024-04-26T21:20:00.732949+0000","last_undegraded":"2024-04-26T21:20:00.732949+0000","last_fullsized":"2024-04-26T21:20:00.732949+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:30:35.439137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731494+0000","last_change":"2024-04-26T21:19:59.733110+0000","last_active":"2024-04-26T21:20:00.731494+0000","last_peered":"2024-04-26T21:20:00.731494+0000","last_clean":"2024-04-26T21:20:00.731494+0000","last_became_active":"2024-04-26T21:19:59.732800+0000","last_became_peered":"2024-04-26T21:19:59.732800+0000","last_unstale":"2024-04-26T21:20:00.731494+0000","last_undegraded":"2024-04-26T21:20:00.731494+0000","last_fullsized":"2024-04-26T21:20:00.731494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:05.097868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732778+0000","last_change":"2024-04-26T21:19:58.723046+0000","last_active":"2024-04-26T21:20:00.732778+0000","last_peered":"2024-04-26T21:20:00.732778+0000","last_clean":"2024-04-26T21:20:00.732778+0000","last_became_active":"2024-04-26T21:19:58.722742+0000","last_became_peered":"2024-04-26T21:19:58.722742+0000","last_unstale":"2024-04-26T21:20:00.732778+0000","last_undegraded":"2024-04-26T21:20:00.732778+0000","last_fullsized":"2024-04-26T21:20:00.732778+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:30.918041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331449+0000","last_change":"2024-04-26T21:19:59.736368+0000","last_active":"2024-04-26T21:20:01.331449+0000","last_peered":"2024-04-26T21:20:01.331449+0000","last_clean":"2024-04-26T21:20:01.331449+0000","last_became_active":"2024-04-26T21:19:59.736060+0000","last_became_peered":"2024-04-26T21:19:59.736060+0000","last_unstale":"2024-04-26T21:20:01.331449+0000","last_undegraded":"2024-04-26T21:20:01.331449+0000","last_fullsized":"2024-04-26T21:20:01.331449+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:38.088650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.332010+0000","last_change":"2024-04-26T21:19:58.728401+0000","last_active":"2024-04-26T21:20:01.332010+0000","last_peered":"2024-04-26T21:20:01.332010+0000","last_clean":"2024-04-26T21:20:01.332010+0000","last_became_active":"2024-04-26T21:19:58.728148+0000","last_became_peered":"2024-04-26T21:19:58.728148+0000","last_unstale":"2024-04-26T21:20:01.332010+0000","last_undegraded":"2024-04-26T21:20:01.332010+0000","last_fullsized":"2024-04-26T21:20:01.332010+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:37.083331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736563+0000","last_change":"2024-04-26T21:19:59.734586+0000","last_active":"2024-04-26T21:19:59.736563+0000","last_peered":"2024-04-26T21:19:59.736563+0000","last_clean":"2024-04-26T21:19:59.736563+0000","last_became_active":"2024-04-26T21:19:59.734306+0000","last_became_peered":"2024-04-26T21:19:59.734306+0000","last_unstale":"2024-04-26T21:19:59.736563+0000","last_undegraded":"2024-04-26T21:19:59.736563+0000","last_fullsized":"2024-04-26T21:19:59.736563+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:40.915182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331424+0000","last_change":"2024-04-26T21:19:58.726518+0000","last_active":"2024-04-26T21:20:01.331424+0000","last_peered":"2024-04-26T21:20:01.331424+0000","last_clean":"2024-04-26T21:20:01.331424+0000","last_became_active":"2024-04-26T21:19:58.726250+0000","last_became_peered":"2024-04-26T21:19:58.726250+0000","last_unstale":"2024-04-26T21:20:01.331424+0000","last_undegraded":"2024-04-26T21:20:01.331424+0000","last_fullsized":"2024-04-26T21:20:01.331424+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:35.650286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331394+0000","last_change":"2024-04-26T21:19:59.731970+0000","last_active":"2024-04-26T21:20:01.331394+0000","last_peered":"2024-04-26T21:20:01.331394+0000","last_clean":"2024-04-26T21:20:01.331394+0000","last_became_active":"2024-04-26T21:19:59.731538+0000","last_became_peered":"2024-04-26T21:19:59.731538+0000","last_unstale":"2024-04-26T21:20:01.331394+0000","last_undegraded":"2024-04-26T21:20:01.331394+0000","last_fullsized":"2024-04-26T21:20:01.331394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:22.117245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331023+0000","last_change":"2024-04-26T21:19:58.725139+0000","last_active":"2024-04-26T21:20:01.331023+0000","last_peered":"2024-04-26T21:20:01.331023+0000","last_clean":"2024-04-26T21:20:01.331023+0000","last_became_active":"2024-04-26T21:19:58.724786+0000","last_became_peered":"2024-04-26T21:19:58.724786+0000","last_unstale":"2024-04-26T21:20:01.331023+0000","last_undegraded":"2024-04-26T21:20:01.331023+0000","last_fullsized":"2024-04-26T21:20:01.331023+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:02.921569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331782+0000","last_change":"2024-04-26T21:19:59.735122+0000","last_active":"2024-04-26T21:20:01.331782+0000","last_peered":"2024-04-26T21:20:01.331782+0000","last_clean":"2024-04-26T21:20:01.331782+0000","last_became_active":"2024-04-26T21:19:59.734855+0000","last_became_peered":"2024-04-26T21:19:59.734855+0000","last_unstale":"2024-04-26T21:20:01.331782+0000","last_undegraded":"2024-04-26T21:20:01.331782+0000","last_fullsized":"2024-04-26T21:20:01.331782+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:05.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731507+0000","last_change":"2024-04-26T21:19:58.717264+0000","last_active":"2024-04-26T21:20:00.731507+0000","last_peered":"2024-04-26T21:20:00.731507+0000","last_clean":"2024-04-26T21:20:00.731507+0000","last_became_active":"2024-04-26T21:19:58.716586+0000","last_became_peered":"2024-04-26T21:19:58.716586+0000","last_unstale":"2024-04-26T21:20:00.731507+0000","last_undegraded":"2024-04-26T21:20:00.731507+0000","last_fullsized":"2024-04-26T21:20:00.731507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:37.548856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.731600+0000","last_change":"2024-04-26T21:19:59.742191+0000","last_active":"2024-04-26T21:20:00.731600+0000","last_peered":"2024-04-26T21:20:00.731600+0000","last_clean":"2024-04-26T21:20:00.731600+0000","last_became_active":"2024-04-26T21:19:59.738252+0000","last_became_peered":"2024-04-26T21:19:59.738252+0000","last_unstale":"2024-04-26T21:20:00.731600+0000","last_undegraded":"2024-04-26T21:20:00.731600+0000","last_fullsized":"2024-04-26T21:20:00.731600+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:24.059383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"122'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331131+0000","last_change":"2024-04-26T21:19:58.726369+0000","last_active":"2024-04-26T21:20:01.331131+0000","last_peered":"2024-04-26T21:20:01.331131+0000","last_clean":"2024-04-26T21:20:01.331131+0000","last_became_active":"2024-04-26T21:19:58.726018+0000","last_became_peered":"2024-04-26T21:19:58.726018+0000","last_unstale":"2024-04-26T21:20:01.331131+0000","last_undegraded":"2024-04-26T21:20:01.331131+0000","last_fullsized":"2024-04-26T21:20:01.331131+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:25.079272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.736456+0000","last_change":"2024-04-26T21:19:59.734315+0000","last_active":"2024-04-26T21:19:59.736456+0000","last_peered":"2024-04-26T21:19:59.736456+0000","last_clean":"2024-04-26T21:19:59.736456+0000","last_became_active":"2024-04-26T21:19:59.733974+0000","last_became_peered":"2024-04-26T21:19:59.733974+0000","last_unstale":"2024-04-26T21:19:59.736456+0000","last_undegraded":"2024-04-26T21:19:59.736456+0000","last_fullsized":"2024-04-26T21:19:59.736456+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:10.580605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.722980+0000","last_change":"2024-04-26T21:19:58.726792+0000","last_active":"2024-04-26T21:19:59.722980+0000","last_peered":"2024-04-26T21:19:59.722980+0000","last_clean":"2024-04-26T21:19:59.722980+0000","last_became_active":"2024-04-26T21:19:58.726491+0000","last_became_peered":"2024-04-26T21:19:58.726491+0000","last_unstale":"2024-04-26T21:19:59.722980+0000","last_undegraded":"2024-04-26T21:19:59.722980+0000","last_fullsized":"2024-04-26T21:19:59.722980+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:59.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331746+0000","last_change":"2024-04-26T21:19:59.742165+0000","last_active":"2024-04-26T21:20:01.331746+0000","last_peered":"2024-04-26T21:20:01.331746+0000","last_clean":"2024-04-26T21:20:01.331746+0000","last_became_active":"2024-04-26T21:19:59.741364+0000","last_became_peered":"2024-04-26T21:19:59.741364+0000","last_unstale":"2024-04-26T21:20:01.331746+0000","last_undegraded":"2024-04-26T21:20:01.331746+0000","last_fullsized":"2024-04-26T21:20:01.331746+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:23.987473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331672+0000","last_change":"2024-04-26T21:19:58.725526+0000","last_active":"2024-04-26T21:20:01.331672+0000","last_peered":"2024-04-26T21:20:01.331672+0000","last_clean":"2024-04-26T21:20:01.331672+0000","last_became_active":"2024-04-26T21:19:58.725195+0000","last_became_peered":"2024-04-26T21:19:58.725195+0000","last_unstale":"2024-04-26T21:20:01.331672+0000","last_undegraded":"2024-04-26T21:20:01.331672+0000","last_fullsized":"2024-04-26T21:20:01.331672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:45.475764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331496+0000","last_change":"2024-04-26T21:19:59.733691+0000","last_active":"2024-04-26T21:20:01.331496+0000","last_peered":"2024-04-26T21:20:01.331496+0000","last_clean":"2024-04-26T21:20:01.331496+0000","last_became_active":"2024-04-26T21:19:59.733406+0000","last_became_peered":"2024-04-26T21:19:59.733406+0000","last_unstale":"2024-04-26T21:20:01.331496+0000","last_undegraded":"2024-04-26T21:20:01.331496+0000","last_fullsized":"2024-04-26T21:20:01.331496+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:04.661637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.722652+0000","last_change":"2024-04-26T21:19:58.728134+0000","last_active":"2024-04-26T21:19:59.722652+0000","last_peered":"2024-04-26T21:19:59.722652+0000","last_clean":"2024-04-26T21:19:59.722652+0000","last_became_active":"2024-04-26T21:19:58.727846+0000","last_became_peered":"2024-04-26T21:19:58.727846+0000","last_unstale":"2024-04-26T21:19:59.722652+0000","last_undegraded":"2024-04-26T21:19:59.722652+0000","last_fullsized":"2024-04-26T21:19:59.722652+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:55.620133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:00.732717+0000","last_change":"2024-04-26T21:19:59.743339+0000","last_active":"2024-04-26T21:20:00.732717+0000","last_peered":"2024-04-26T21:20:00.732717+0000","last_clean":"2024-04-26T21:20:00.732717+0000","last_became_active":"2024-04-26T21:19:59.743084+0000","last_became_peered":"2024-04-26T21:19:59.743084+0000","last_unstale":"2024-04-26T21:20:00.732717+0000","last_undegraded":"2024-04-26T21:20:00.732717+0000","last_fullsized":"2024-04-26T21:20:00.732717+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:16.763428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331556+0000","last_change":"2024-04-26T21:19:58.727049+0000","last_active":"2024-04-26T21:20:01.331556+0000","last_peered":"2024-04-26T21:20:01.331556+0000","last_clean":"2024-04-26T21:20:01.331556+0000","last_became_active":"2024-04-26T21:19:58.725578+0000","last_became_peered":"2024-04-26T21:19:58.725578+0000","last_unstale":"2024-04-26T21:20:01.331556+0000","last_undegraded":"2024-04-26T21:20:01.331556+0000","last_fullsized":"2024-04-26T21:20:01.331556+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:04.806615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331534+0000","last_change":"2024-04-26T21:19:59.749499+0000","last_active":"2024-04-26T21:20:01.331534+0000","last_peered":"2024-04-26T21:20:01.331534+0000","last_clean":"2024-04-26T21:20:01.331534+0000","last_became_active":"2024-04-26T21:19:59.748961+0000","last_became_peered":"2024-04-26T21:19:59.748961+0000","last_unstale":"2024-04-26T21:20:01.331534+0000","last_undegraded":"2024-04-26T21:20:01.331534+0000","last_fullsized":"2024-04-26T21:20:01.331534+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:51:54.154897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331611+0000","last_change":"2024-04-26T21:19:58.727762+0000","last_active":"2024-04-26T21:20:01.331611+0000","last_peered":"2024-04-26T21:20:01.331611+0000","last_clean":"2024-04-26T21:20:01.331611+0000","last_became_active":"2024-04-26T21:19:58.727498+0000","last_became_peered":"2024-04-26T21:19:58.727498+0000","last_unstale":"2024-04-26T21:20:01.331611+0000","last_undegraded":"2024-04-26T21:20:01.331611+0000","last_fullsized":"2024-04-26T21:20:01.331611+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:03.642636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-26T21:19:59.731444+0000","last_change":"2024-04-26T21:19:59.729048+0000","last_active":"2024-04-26T21:19:59.731444+0000","last_peered":"2024-04-26T21:19:59.731444+0000","last_clean":"2024-04-26T21:19:59.731444+0000","last_became_active":"2024-04-26T21:19:59.728704+0000","last_became_peered":"2024-04-26T21:19:59.728704+0000","last_unstale":"2024-04-26T21:19:59.731444+0000","last_undegraded":"2024-04-26T21:19:59.731444+0000","last_fullsized":"2024-04-26T21:19:59.731444+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:25.027523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331128+0000","last_change":"2024-04-26T21:19:58.722265+0000","last_active":"2024-04-26T21:20:01.331128+0000","last_peered":"2024-04-26T21:20:01.331128+0000","last_clean":"2024-04-26T21:20:01.331128+0000","last_became_active":"2024-04-26T21:19:58.721957+0000","last_became_peered":"2024-04-26T21:19:58.721957+0000","last_unstale":"2024-04-26T21:20:01.331128+0000","last_undegraded":"2024-04-26T21:20:01.331128+0000","last_fullsized":"2024-04-26T21:20:01.331128+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:45.350609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331336+0000","last_change":"2024-04-26T21:19:59.733181+0000","last_active":"2024-04-26T21:20:01.331336+0000","last_peered":"2024-04-26T21:20:01.331336+0000","last_clean":"2024-04-26T21:20:01.331336+0000","last_became_active":"2024-04-26T21:19:59.732929+0000","last_became_peered":"2024-04-26T21:19:59.732929+0000","last_unstale":"2024-04-26T21:20:01.331336+0000","last_undegraded":"2024-04-26T21:20:01.331336+0000","last_fullsized":"2024-04-26T21:20:01.331336+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:58.704300+0000","last_clean_scrub_stamp":"2024-04-26T21:19:58.704300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:06.706265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.331772+0000","last_change":"2024-04-26T21:19:58.728363+0000","last_active":"2024-04-26T21:20:01.331772+0000","last_peered":"2024-04-26T21:20:01.331772+0000","last_clean":"2024-04-26T21:20:01.331772+0000","last_became_active":"2024-04-26T21:19:58.728098+0000","last_became_peered":"2024-04-26T21:19:58.728098+0000","last_unstale":"2024-04-26T21:20:01.331772+0000","last_undegraded":"2024-04-26T21:20:01.331772+0000","last_fullsized":"2024-04-26T21:20:01.331772+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:19:57.698981+0000","last_clean_scrub_stamp":"2024-04-26T21:19:57.698981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:47.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":351,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-26T21:20:01.330908+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:01.330908+0000","last_peered":"2024-04-26T21:20:01.330908+0000","last_clean":"2024-04-26T21:20:01.330908+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:01.330908+0000","last_undegraded":"2024-04-26T21:20:01.330908+0000","last_fullsized":"2024-04-26T21:20:01.330908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509543,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1244330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":653324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":651824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":651782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509542,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":651872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509542,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1523712,"data_stored":652328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509542,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2109440,"data_stored":1242080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36800,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":1490944,"data_stored":638637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:04.099 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:20:04.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:20:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.243+0000 7f6726a45640 1 Processor -- start 2024-04-26T21:20:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.243+0000 7f6726a45640 1 -- start start 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f6726a45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67200b3ac0 0x7f6720160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f6726a45640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67200b46f0 con 0x7f672009c150 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f6726a45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67200d1f60 con 0x7f67200b3620 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f6726a45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f672006d3b0 con 0x7f67200b3ac0 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f671f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f672009c150 0x7f672006db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53300/0 (socket says 172.21.15.40:53300) 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f671f7fe640 1 -- 172.21.15.40:0/1658364216 learned_addr learned my addr 172.21.15.40:0/1658364216 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> v2:172.21.15.184:3300/0 conn(0x7f67200b3ac0 0x7f6720160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573086256 0 0) 0x7f6710001110 con 0x7f672009c150 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.244+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6708003520 con 0x7f672009c150 2024-04-26T21:20:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872111419 0 0) 0x7f67200d1f60 con 0x7f67200b3620 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6710001110 con 0x7f67200b3620 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4200051201 0 0) 0x7f6708003520 con 0x7f672009c150 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 >> v2:172.21.15.184:3300/0 conn(0x7f67200b3ac0 msgr2=0x7f6720160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 --2- 172.21.15.40:0/1658364216 >> v2:172.21.15.184:3300/0 conn(0x7f67200b3ac0 0x7f6720160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 >> v1:172.21.15.184:6789/0 conn(0x7f67200b3620 legacy=0x7f672009ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200d2240 con 0x7f672009c150 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f6710001840 con 0x7f672009c150 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> v2:172.21.15.184:3300/0 conn(0x7f67200b3ac0 0x7f6720160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 >> v1:172.21.15.40:6789/0 conn(0x7f672009c150 legacy=0x7f672006db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6708003b40 0x7f6708003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6708004800 0x7f6708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f670800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f672006d3b0 con 0x7f67080051c0 2024-04-26T21:20:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f670800c040 con 0x7f6708004800 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f670800c240 con 0x7f6708003b40 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f670800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671f7fe640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6708004800 0x7f6708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671ffff640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6708003b40 0x7f6708003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 -- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6708003b40 msgr2=0x7f6708003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6708003b40 0x7f6708003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 -- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6708004800 msgr2=0x7f6708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6708004800 0x7f6708004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f6724fe2640 1 -- 172.21.15.40:0/1658364216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6708003520 con 0x7f67080051c0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671ffff640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6708003b40 0x7f6708003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.245+0000 7f671f7fe640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6708004800 0x7f6708004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.246+0000 7f6724fe2640 1 -- 172.21.15.40:0/1658364216 reap_dead start 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.246+0000 7f6724fe2640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f670800b9c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f671800e350 tx=0x7f6708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f003ff9b6d0e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.246+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718009de0 con 0x7f67080051c0 2024-04-26T21:20:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.246+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671802e680 con 0x7f67080051c0 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.246+0000 7f671effd640 1 -- 172.21.15.40:0/1658364216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671802ea70 con 0x7f67080051c0 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 -- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 msgr2=0x7f670800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f670800b9c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f671800e350 tx=0x7f6708005830 comp rx=0 tx=0).stop 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 -- 172.21.15.40:0/1658364216 shutdown_connections 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 --2- 172.21.15.40:0/1658364216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f670800b9c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 -- 172.21.15.40:0/1658364216 >> 172.21.15.40:0/1658364216 conn(0x7f67200b4ab0 msgr2=0x7f67200b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 -- 172.21.15.40:0/1658364216 shutdown_connections 2024-04-26T21:20:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.247+0000 7f6726a45640 1 -- 172.21.15.40:0/1658364216 wait complete. 2024-04-26T21:20:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.248+0000 7f6726a45640 1 Processor -- start 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.248+0000 7f6726a45640 1 -- start start 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67200c7600 0x7f672015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c8480 0x7f672015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67200b3ff0 con 0x7f67080051c0 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f672006d600 con 0x7f67200c7600 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f67200b4170 con 0x7f67200c8480 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59924/0 (socket says 172.21.15.40:59924) 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 -- 172.21.15.40:0/887584727 learned_addr learned my addr 172.21.15.40:0/887584727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6724fe2640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c8480 0x7f672015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671f7fe640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67200c7600 0x7f672015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 -- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c8480 msgr2=0x7f672015acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c8480 0x7f672015acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 -- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67200c7600 msgr2=0x7f672015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67200c7600 0x7f672015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 -- 172.21.15.40:0/887584727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200c8db0 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671ffff640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f671400e580 tx=0x7f671400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67140052e0 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6714007020 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67200b0a50 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67140071f0 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.249+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67200b1010 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.251+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67200b0840 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.251+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f671401a020 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.251+0000 7f671cff9640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 0x7f670c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.252+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f671408b2d0 con 0x7f67080051c0 2024-04-26T21:20:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.252+0000 7f671f7fe640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 0x7f670c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.252+0000 7f671f7fe640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 0x7f670c06ede0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f67200c7a40 tx=0x7f6710009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.256+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6714059670 con 0x7f67080051c0 2024-04-26T21:20:04.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.511+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f672006a290 con 0x7f67080051c0 2024-04-26T21:20:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.514+0000 7f671cff9640 1 -- 172.21.15.40:0/887584727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f6714058f70 con 0x7f67080051c0 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:20:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.515+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 msgr2=0x7f670c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 0x7f670c06ede0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f67200c7a40 tx=0x7f6710009040 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 msgr2=0x7f67201f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f671400e580 tx=0x7f671400ea40 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 shutdown_connections 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f670c06c920 0x7f670c06ede0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c8480 0x7f672015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67200c7600 0x7f672015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 --2- 172.21.15.40:0/887584727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67080051c0 0x7f67201f5380 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 >> 172.21.15.40:0/887584727 conn(0x7f67200b4ab0 msgr2=0x7f67200b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 shutdown_connections 2024-04-26T21:20:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.516+0000 7f6726a45640 1 -- 172.21.15.40:0/887584727 wait complete. 2024-04-26T21:20:04.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.627+0000 7f6d7c5c6640 1 Processor -- start 2024-04-26T21:20:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.627+0000 7f6d7c5c6640 1 -- start start 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7c5c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d7415f340 0x7f6d740b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7c5c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d7415f720 0x7f6d740d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7c5c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740b2510 con 0x7f6d740d4f00 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7c5c6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6d740b27f0 con 0x7f6d7415f340 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7c5c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6d740b2ad0 con 0x7f6d7415f720 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d7415f340 0x7f6d740b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d7415f340 0x7f6d740b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59928/0 (socket says 172.21.15.40:59928) 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 -- 172.21.15.40:0/2042847420 learned_addr learned my addr 172.21.15.40:0/2042847420 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d79b61640 1 --2- 172.21.15.40:0/2042847420 >> v2:172.21.15.184:3300/0 conn(0x7f6d7415f720 0x7f6d740d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 -- 172.21.15.40:0/2042847420 >> v2:172.21.15.184:3300/0 conn(0x7f6d7415f720 msgr2=0x7f6d740d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 --2- 172.21.15.40:0/2042847420 >> v2:172.21.15.184:3300/0 conn(0x7f6d7415f720 0x7f6d740d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.628+0000 7f6d7a362640 1 -- 172.21.15.40:0/2042847420 >> v1:172.21.15.184:6789/0 conn(0x7f6d740d4f00 legacy=0x7f6d740d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.629+0000 7f6d7a362640 1 -- 172.21.15.40:0/2042847420 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740b2db0 con 0x7f6d7415f340 2024-04-26T21:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.629+0000 7f6d7a362640 1 --2- 172.21.15.40:0/2042847420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f340 0x7f6d740b1e80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6d740cf8a0 tx=0x7f6d7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbf8274e525940 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.629+0000 7f6d79360640 1 -- 172.21.15.40:0/2042847420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70037070 con 0x7f6d7415f340 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.629+0000 7f6d79360640 1 -- 172.21.15.40:0/2042847420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d70032440 con 0x7f6d7415f340 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.629+0000 7f6d79360640 1 -- 172.21.15.40:0/2042847420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70032810 con 0x7f6d7415f340 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/2042847420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f340 msgr2=0x7f6d740b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/2042847420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f340 0x7f6d740b1e80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6d740cf8a0 tx=0x7f6d7002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/2042847420 shutdown_connections 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/2042847420 >> v2:172.21.15.184:3300/0 conn(0x7f6d7415f720 0x7f6d740d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/2042847420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f340 0x7f6d740b1e80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/2042847420 >> 172.21.15.40:0/2042847420 conn(0x7f6d7408ec70 msgr2=0x7f6d7409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/2042847420 shutdown_connections 2024-04-26T21:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.630+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/2042847420 wait complete. 2024-04-26T21:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.631+0000 7f6d7c5c6640 1 Processor -- start 2024-04-26T21:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.631+0000 7f6d7c5c6640 1 -- start start 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d740d4f00 0x7f6d740b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 0x7f6d740c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740b3940 con 0x7f6d7415f720 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740b37c0 con 0x7f6d740d4f00 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7c5c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740b3ac0 con 0x7f6d7415f340 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59932/0 (socket says 172.21.15.40:59932) 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 -- 172.21.15.40:0/919851311 learned_addr learned my addr 172.21.15.40:0/919851311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d79b61640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 0x7f6d740c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7a362640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d740d4f00 0x7f6d740b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 -- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 msgr2=0x7f6d740c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 0x7f6d740c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 -- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d740d4f00 msgr2=0x7f6d740b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d740d4f00 0x7f6d740b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d7ab63640 1 -- 172.21.15.40:0/919851311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740baae0 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.632+0000 7f6d79b61640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 0x7f6d740c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d7ab63640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c760 tx=0x7f6d6c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c003b70 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6c003d10 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00aa90 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d740ca800 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.633+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d740cadc0 con 0x7f6d7415f720 2024-04-26T21:20:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.634+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7406a290 con 0x7f6d7415f720 2024-04-26T21:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.634+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d6c00ac30 con 0x7f6d7415f720 2024-04-26T21:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.635+0000 7f6d6affd640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 0x7f6d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.635+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f6d6c08b980 con 0x7f6d7415f720 2024-04-26T21:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.635+0000 7f6d7a362640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 0x7f6d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.636+0000 7f6d7a362640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 0x7f6d5806edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6d70034000 tx=0x7f6d7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:04.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.639+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d6c059db0 con 0x7f6d7415f720 2024-04-26T21:20:04.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.862+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d740d1c20 con 0x7f6d7415f720 2024-04-26T21:20:04.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.863+0000 7f6d6affd640 1 -- 172.21.15.40:0/919851311 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f6d6c0596b0 con 0x7f6d7415f720 2024-04-26T21:20:04.865 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:04.865 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:02.108019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:04.865 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 76 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 msgr2=0x7f6d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 0x7f6d5806edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6d70034000 tx=0x7f6d7002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 msgr2=0x7f6d740ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c760 tx=0x7f6d6c00cc20 comp rx=0 tx=0).stop 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 shutdown_connections 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6d5806c8f0 0x7f6d5806edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d7415f720 0x7f6d740ca160 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d7415f340 0x7f6d740c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 --2- 172.21.15.40:0/919851311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d740d4f00 0x7f6d740b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:04.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 >> 172.21.15.40:0/919851311 conn(0x7f6d7408ec70 msgr2=0x7f6d740b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:04.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 shutdown_connections 2024-04-26T21:20:04.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:04.865+0000 7f6d7c5c6640 1 -- 172.21.15.40:0/919851311 wait complete. 2024-04-26T21:20:05.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:19:59.077853+0000', 'modified': '2024-04-26T21:20:02.108019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8475': {'gid': 8475, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/4242446604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 4242446604}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 4242446604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8475, 'qdb_cluster': [8475]} 2024-04-26T21:20:05.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:20:05.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:20:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.382+0000 7f7d064c4640 1 Processor -- start 2024-04-26T21:20:05.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 -- start start 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d0015f370 0x7f7d000cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d000b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d000d03a0 con 0x7f7d0015f750 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7d000b4d10 con 0x7f7d0015f370 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7d064c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d000d0680 con 0x7f7d000d3ee0 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.383+0000 7f7cfffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d0015f370 0x7f7d000cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d0015f370 0x7f7d000cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59934/0 (socket says 172.21.15.40:59934) 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 -- 172.21.15.40:0/592366821 learned_addr learned my addr 172.21.15.40:0/592366821 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7d04a61640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d000b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 -- 172.21.15.40:0/592366821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 msgr2=0x7f7d000b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d000b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 -- 172.21.15.40:0/592366821 >> v1:172.21.15.184:6789/0 conn(0x7f7d0015f750 legacy=0x7f7d000d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 -- 172.21.15.40:0/592366821 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d000d0960 con 0x7f7d0015f370 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.384+0000 7f7cfffff640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d000cfd70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009870 tx=0x7f7cf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc489d3635770cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7cfeffd640 1 -- 172.21.15.40:0/592366821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037070 con 0x7f7d0015f370 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7cfeffd640 1 -- 172.21.15.40:0/592366821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032440 con 0x7f7d0015f370 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7cfeffd640 1 -- 172.21.15.40:0/592366821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0032810 con 0x7f7d0015f370 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 -- 172.21.15.40:0/592366821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 msgr2=0x7f7d000cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d000cfd70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009870 tx=0x7f7cf002a270 comp rx=0 tx=0).stop 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 -- 172.21.15.40:0/592366821 shutdown_connections 2024-04-26T21:20:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d000b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 --2- 172.21.15.40:0/592366821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d000cfd70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 -- 172.21.15.40:0/592366821 >> 172.21.15.40:0/592366821 conn(0x7f7d0008ec10 msgr2=0x7f7d0009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 -- 172.21.15.40:0/592366821 shutdown_connections 2024-04-26T21:20:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.385+0000 7f7d064c4640 1 -- 172.21.15.40:0/592366821 wait complete. 2024-04-26T21:20:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.386+0000 7f7d064c4640 1 Processor -- start 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 -- start start 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d00162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d0015f750 0x7f7d000ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d000b51c0 con 0x7f7d0015f370 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d000d0f20 con 0x7f7d0015f750 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.387+0000 7f7d064c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d000b5340 con 0x7f7d000d3ee0 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59946/0 (socket says 172.21.15.40:59946) 2024-04-26T21:20:05.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 -- 172.21.15.40:0/2791852503 learned_addr learned my addr 172.21.15.40:0/2791852503 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d00162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7d04a61640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d0015f750 0x7f7d000ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 -- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 msgr2=0x7f7d00162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d00162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 -- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d0015f750 msgr2=0x7f7d000ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d0015f750 0x7f7d000ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 -- 172.21.15.40:0/2791852503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d00162a40 con 0x7f7d0015f370 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400c3f0 tx=0x7f7cf400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cfcff9640 1 -- 172.21.15.40:0/2791852503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40052d0 con 0x7f7d0015f370 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cfcff9640 1 -- 172.21.15.40:0/2791852503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4005490 con 0x7f7d0015f370 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d000ccfb0 con 0x7f7d0015f370 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7cfcff9640 1 -- 172.21.15.40:0/2791852503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4007260 con 0x7f7d0015f370 2024-04-26T21:20:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.388+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d000cd570 con 0x7f7d0015f370 2024-04-26T21:20:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.390+0000 7f7cfcff9640 1 -- 172.21.15.40:0/2791852503 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cf4007400 con 0x7f7d0015f370 2024-04-26T21:20:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.390+0000 7f7cfcff9640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.391+0000 7f7cfcff9640 1 -- 172.21.15.40:0/2791852503 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf408b050 con 0x7f7d0015f370 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.391+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.391+0000 7f7d091dc740 1 Processor -- start 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.391+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 0x7f7cec06edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.391+0000 7f7d091dc740 1 -- start start 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f7f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f37120 0x560786f7d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560786e9b740 con 0x560786f61440 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560786e373a0 con 0x560786f7ece0 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560786e97700 con 0x560786f37120 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59948/0 (socket says 172.21.15.40:59948) 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 -- 172.21.15.40:0/18576027 learned_addr learned my addr 172.21.15.40:0/18576027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7d04a61640 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f37120 0x560786f7d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f7f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 -- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f37120 msgr2=0x560786f7d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f37120 0x560786f7d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 -- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 msgr2=0x560786f7f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f7f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.392+0000 7f7cfffff640 1 -- 172.21.15.40:0/18576027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560786e53750 con 0x560786f61440 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7cfffff640 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0035040 tx=0x7f7cf002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882ebe2448d1b9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7ce5ffb640 1 -- 172.21.15.40:0/18576027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf004e070 con 0x560786f61440 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7ce5ffb640 1 -- 172.21.15.40:0/18576027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032e50 con 0x560786f61440 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7ce5ffb640 1 -- 172.21.15.40:0/18576027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037040 con 0x560786f61440 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 -- 172.21.15.40:0/18576027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 msgr2=0x560786f61820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0035040 tx=0x7f7cf002c000 comp rx=0 tx=0).stop 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 -- 172.21.15.40:0/18576027 shutdown_connections 2024-04-26T21:20:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f37120 0x560786f7d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f7f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 --2- 172.21.15.40:0/18576027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f61820 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 -- 172.21.15.40:0/18576027 >> 172.21.15.40:0/18576027 conn(0x560786f33a50 msgr2=0x560786f7c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 -- 172.21.15.40:0/18576027 shutdown_connections 2024-04-26T21:20:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.393+0000 7f7d091dc740 1 -- 172.21.15.40:0/18576027 wait complete. 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.394+0000 7f7d091dc740 1 -- start start 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.394+0000 7f7d091dc740 1 Processor -- start 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f61440 0x560786f5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560786f37ec0 con 0x560786f8cd30 2024-04-26T21:20:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560786e22fe0 con 0x560786f7ece0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560786f31980 con 0x560786f61440 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d091dc740 10 client.0 _ll_register_callbacks cb 0x560786e39150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59960/0 (socket says 172.21.15.40:59960) 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 -- 172.21.15.40:0/2289197690 learned_addr learned my addr 172.21.15.40:0/2289197690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f61440 0x560786f5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 -- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f61440 msgr2=0x560786f5cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f61440 0x560786f5cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 -- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 msgr2=0x560786f8c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f8c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.395+0000 7f7d04a61640 1 -- 172.21.15.40:0/2289197690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560786bad2d0 con 0x560786f8cd30 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d04a61640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80079d0 tx=0x7f7cf8007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcff01779f7bf2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf801b020 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf8011ae0 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf0032e50 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7cf8025050 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 10 client.8622 fetch_fsmap learned FSMap version 76 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x560786c776b0 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032e50 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f7cf8028020 con 0x560786f8cd30 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 10 client.8622 fetch_fsmap finished waiting for FSMap version 76 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 10 client.8622 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 20 client.8622 populate_metadata read hostname 'smithi040' 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f62450 0x560786f62830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 4 client.8622 mds_command: new command op to 8475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 --> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560786ea9350 con 0x560786f62450 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.396+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f62450 0x560786f62830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.397+0000 7f7cdeffd640 10 client.8622 ms_handle_connect on v2:172.21.15.40:6833/4242446604 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.397+0000 7f7cfffff640 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f62450 0x560786f62830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1fc51d770f80c4 server_cookie=8c4a35040696ba9a in_seq=0 out_seq=0 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.397+0000 7f7cdeffd640 1 -- 172.21.15.40:0/2289197690 <== mds.0 v2:172.21.15.40:6833/4242446604 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560786ea9350 con 0x560786f62450 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.397+0000 7f7cdeffd640 10 client.8622 handle_command_reply: tid=0 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.397+0000 7f7d091dc740 1 client.8622 shutdown 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 msgr2=0x560786f5fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80079d0 tx=0x7f7cf8007e90 comp rx=0 tx=0).stop 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 shutdown_connections 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f5fc70 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f62450 0x560786f62830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f8c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 --2- 172.21.15.40:0/2289197690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f61440 0x560786f5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7cdeffd640 0 client.8622 ms_handle_reset on v2:172.21.15.40:6833/4242446604 2024-04-26T21:20:05.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 >> 172.21.15.40:0/2289197690 conn(0x560786f37120 msgr2=0x560786f7f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 shutdown_connections 2024-04-26T21:20:05.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 1 -- 172.21.15.40:0/2289197690 wait complete. 2024-04-26T21:20:05.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.398+0000 7f7d091dc740 20 client.8622 trim_cache size 0 max 16384 2024-04-26T21:20:05.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.416+0000 7f7d091dc740 1 Processor -- start 2024-04-26T21:20:05.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.416+0000 7f7d091dc740 1 -- start start 2024-04-26T21:20:05.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 0x560786f37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d000b9c30 con 0x560786f61440 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d000b8f90 con 0x560786f7ece0 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7d000b95e0 con 0x560786f8cd30 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59964/0 (socket says 172.21.15.40:59964) 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 -- 172.21.15.40:0/3836470872 learned_addr learned my addr 172.21.15.40:0/3836470872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d04a61640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 0x560786f37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 -- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 msgr2=0x560786f37a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 0x560786f37a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 -- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 msgr2=0x560786f37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 -- 172.21.15.40:0/3836470872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560786f5f980 con 0x560786f61440 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7d04a61640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 0x560786f37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009960 tx=0x7f7cf0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9510958c938ae21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7ce57fa640 1 -- 172.21.15.40:0/3836470872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf004e040 con 0x560786f61440 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7ce57fa640 1 -- 172.21.15.40:0/3836470872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf003e760 con 0x560786f61440 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.417+0000 7f7ce57fa640 1 -- 172.21.15.40:0/3836470872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037040 con 0x560786f61440 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 -- 172.21.15.40:0/3836470872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 msgr2=0x560786f96660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009960 tx=0x7f7cf0009240 comp rx=0 tx=0).stop 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 -- 172.21.15.40:0/3836470872 shutdown_connections 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f8cd30 0x560786f37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f7ece0 0x560786f37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3836470872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f61440 0x560786f96660 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 -- 172.21.15.40:0/3836470872 >> 172.21.15.40:0/3836470872 conn(0x560786f37120 msgr2=0x560786f5c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 -- 172.21.15.40:0/3836470872 shutdown_connections 2024-04-26T21:20:05.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.418+0000 7f7d091dc740 1 -- 172.21.15.40:0/3836470872 wait complete. 2024-04-26T21:20:05.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.419+0000 7f7d091dc740 1 -- start start 2024-04-26T21:20:05.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.419+0000 7f7d091dc740 1 Processor -- start 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 0x560786f37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 0x560786f37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560786f74d60 con 0x560786f8cd30 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560786f747f0 con 0x560786f61440 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x560786f64320 con 0x560786f7ece0 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 10 client.0 _ll_register_callbacks cb 0x7f7cf80097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59968/0 (socket says 172.21.15.40:59968) 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 -- 172.21.15.40:0/3209390751 learned_addr learned my addr 172.21.15.40:0/3209390751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 0x560786f37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 0x560786f37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 -- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 msgr2=0x560786f37a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 0x560786f37a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 -- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 msgr2=0x560786f37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 0x560786f37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 -- 172.21.15.40:0/3209390751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560786f96880 con 0x560786f8cd30 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cff7fe640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 0x560786f37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 0x560786f37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d04a61640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x560786f736e0 tx=0x7f7cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff51058a437baf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802d020 con 0x560786f8cd30 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf8016930 con 0x560786f8cd30 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.420+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf003e760 con 0x560786f8cd30 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7cf8037050 con 0x560786f8cd30 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 10 client.8628 fetch_fsmap learned FSMap version 76 2024-04-26T21:20:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x560786ff72f0 con 0x560786f8cd30 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf003e760 con 0x560786f8cd30 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f7cf803a020 con 0x560786f8cd30 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 10 client.8628 fetch_fsmap finished waiting for FSMap version 76 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 10 client.8628 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 20 client.8628 populate_metadata read hostname 'smithi040' 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f59a70 0x560786f30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 4 client.8628 mds_command: new command op to 8475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 --> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] -- command(tid 0: {"prefix": "status"}) -- 0x560786ea31c0 con 0x560786f59a70 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f59a70 0x560786f30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 10 client.8628 ms_handle_connect on v2:172.21.15.40:6833/4242446604 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cfffff640 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f59a70 0x560786f30670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e62b7048c3139bd server_cookie=28e574aa3d076463 in_seq=0 out_seq=0 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 1 -- 172.21.15.40:0/3209390751 <== mds.0 v2:172.21.15.40:6833/4242446604 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x560786ea31c0 con 0x560786f59a70 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7cdeffd640 10 client.8628 handle_command_reply: tid=0 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.421+0000 7f7d091dc740 1 client.8628 shutdown 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 msgr2=0x560786f59270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x560786f736e0 tx=0x7f7cf8004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 shutdown_connections 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560786f8cd30 0x560786f59270 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560786f7ece0 0x560786f37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.40:6833/4242446604,v1:172.21.15.40:6835/4242446604] conn(0x560786f59a70 0x560786f30670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 --2- 172.21.15.40:0/3209390751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560786f61440 0x560786f37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7cdeffd640 0 client.8628 ms_handle_reset on v2:172.21.15.40:6833/4242446604 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 >> 172.21.15.40:0/3209390751 conn(0x560786f37120 msgr2=0x560786f2ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 shutdown_connections 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 1 -- 172.21.15.40:0/3209390751 wait complete. 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d091dc740 20 client.8628 trim_cache size 0 max 16384 2024-04-26T21:20:05.430 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 8475, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 5.6937092820000004, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 76, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 122, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 122, 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.715994025000001 2024-04-26T21:20:05.431 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:20:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 msgr2=0x7f7cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 0x7f7cec06edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 msgr2=0x7f7d00158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400c3f0 tx=0x7f7cf400c8b0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 shutdown_connections 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7cec06c8f0 0x7f7cec06edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d0015f750 0x7f7d000ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015f370 0x7f7d00158a70 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 --2- 172.21.15.40:0/2791852503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d000d3ee0 0x7f7d00162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 >> 172.21.15.40:0/2791852503 conn(0x7f7d0008ec10 msgr2=0x7f7d000d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 shutdown_connections 2024-04-26T21:20:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.422+0000 7f7d064c4640 1 -- 172.21.15.40:0/2791852503 wait complete. 2024-04-26T21:20:05.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:20:05.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 Processor -- start 2024-04-26T21:20:05.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 -- start start 2024-04-26T21:20:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff26013ff70 0x7ff260146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff260140350 0x7ff2601465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2600d05b0 con 0x7ff2600cf810 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600d0890 con 0x7ff26013ff70 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.534+0000 7ff265789640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff26006e450 con 0x7ff260140350 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff26013ff70 0x7ff260146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2600cf810 0x7ff2600cfcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53314/0 (socket says 172.21.15.40:53314) 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25f7fe640 1 -- 172.21.15.40:0/1082314187 learned_addr learned my addr 172.21.15.40:0/1082314187 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25e7fc640 1 --2- 172.21.15.40:0/1082314187 >> v2:172.21.15.184:3300/0 conn(0x7ff260140350 0x7ff2601465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25effd640 1 -- 172.21.15.40:0/1082314187 >> v2:172.21.15.184:3300/0 conn(0x7ff260140350 msgr2=0x7ff2601465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25effd640 1 --2- 172.21.15.40:0/1082314187 >> v2:172.21.15.184:3300/0 conn(0x7ff260140350 0x7ff2601465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25effd640 1 -- 172.21.15.40:0/1082314187 >> v1:172.21.15.40:6789/0 conn(0x7ff2600cf810 legacy=0x7ff2600cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25effd640 1 -- 172.21.15.40:0/1082314187 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26006e730 con 0x7ff26013ff70 2024-04-26T21:20:05.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.535+0000 7ff25dffb640 1 -- 172.21.15.40:0/1082314187 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673278434 0 0) 0x7ff2600d05b0 con 0x7ff2600cf810 2024-04-26T21:20:05.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff25effd640 1 --2- 172.21.15.40:0/1082314187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff26013ff70 0x7ff260146080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff2600cf1c0 tx=0x7ff25002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd788bbb745c263d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff25dffb640 1 -- 172.21.15.40:0/1082314187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250037070 con 0x7ff26013ff70 2024-04-26T21:20:05.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff25dffb640 1 -- 172.21.15.40:0/1082314187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff250032440 con 0x7ff26013ff70 2024-04-26T21:20:05.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff25dffb640 1 -- 172.21.15.40:0/1082314187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25002ae60 con 0x7ff26013ff70 2024-04-26T21:20:05.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 -- 172.21.15.40:0/1082314187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff26013ff70 msgr2=0x7ff260146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 --2- 172.21.15.40:0/1082314187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff26013ff70 0x7ff260146080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff2600cf1c0 tx=0x7ff25002a110 comp rx=0 tx=0).stop 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 -- 172.21.15.40:0/1082314187 shutdown_connections 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 --2- 172.21.15.40:0/1082314187 >> v2:172.21.15.184:3300/0 conn(0x7ff260140350 0x7ff2601465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 --2- 172.21.15.40:0/1082314187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff26013ff70 0x7ff260146080 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 -- 172.21.15.40:0/1082314187 >> 172.21.15.40:0/1082314187 conn(0x7ff2600d0d30 msgr2=0x7ff2600d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 -- 172.21.15.40:0/1082314187 shutdown_connections 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.536+0000 7ff265789640 1 -- 172.21.15.40:0/1082314187 wait complete. 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 Processor -- start 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 -- start start 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2600cf810 0x7ff2600af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26013ff70 0x7ff2601eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff26006efe0 con 0x7ff260140350 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff26006ee60 con 0x7ff26013ff70 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.538+0000 7ff265789640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff26006f160 con 0x7ff2600cf810 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59984/0 (socket says 172.21.15.40:59984) 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 -- 172.21.15.40:0/919847183 learned_addr learned my addr 172.21.15.40:0/919847183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25effd640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2600cf810 0x7ff2600af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25e7fc640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26013ff70 0x7ff2601eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 -- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2600cf810 msgr2=0x7ff2600af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2600cf810 0x7ff2600af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 -- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26013ff70 msgr2=0x7ff2601eedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26013ff70 0x7ff2601eedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 -- 172.21.15.40:0/919847183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600aea30 con 0x7ff260140350 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff25f7fe640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c7a0 tx=0x7ff25800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258018070 con 0x7ff260140350 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25800a640 con 0x7ff260140350 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2601f5c50 con 0x7ff260140350 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25800aab0 con 0x7ff260140350 2024-04-26T21:20:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.539+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2600aefd0 con 0x7ff260140350 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.540+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26006a290 con 0x7ff260140350 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.541+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff25801b070 con 0x7ff260140350 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.541+0000 7ff2477fe640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 0x7ff24c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.541+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7ff25808b160 con 0x7ff260140350 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.542+0000 7ff25effd640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 0x7ff24c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.542+0000 7ff25effd640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 0x7ff24c06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff250034000 tx=0x7ff25002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.545+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff258059590 con 0x7ff260140350 2024-04-26T21:20:05.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.767+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff2600b45e0 con 0x7ff260140350 2024-04-26T21:20:05.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.768+0000 7ff2477fe640 1 -- 172.21.15.40:0/919847183 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7ff258058e90 con 0x7ff260140350 2024-04-26T21:20:05.770 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 msgr2=0x7ff24c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 0x7ff24c06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff250034000 tx=0x7ff25002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 msgr2=0x7ff26015b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c7a0 tx=0x7ff25800cc60 comp rx=0 tx=0).stop 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 shutdown_connections 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff24c06c8f0 0x7ff24c06edb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff260140350 0x7ff26015b6f0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26013ff70 0x7ff2601eedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 --2- 172.21.15.40:0/919847183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2600cf810 0x7ff2600af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:05.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 >> 172.21.15.40:0/919847183 conn(0x7ff2600d0d30 msgr2=0x7ff2600b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:05.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 shutdown_connections 2024-04-26T21:20:05.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:05.770+0000 7ff265789640 1 -- 172.21.15.40:0/919847183 wait complete. 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:20:05.783 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:05.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:05.783 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:05.799 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42264sec preferred_lft 42264sec 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:20:05.800 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:05.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:05.802 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:20:05.863 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:20:05.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:05.865 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:20:05.921 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:20:05.921 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:20:05.921 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:20:05.922 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:20:05.922 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:20:05.922 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:20:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:20:06.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:06.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.048 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:20:06.063 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:20:06.063 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:20:06.063 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:06.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:20:06.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:06.130 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:06.185 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:06.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:06.185 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:06.239 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:20:06.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:06.321 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:06.321 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:06.321 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:06.330 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:20:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:20:06.339 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:20:06.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:06.404 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:20:06.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:20:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:06.506 INFO:teuthology.orchestra.run.smithi040.stdout:client8640 2024-04-26T21:20:06.506 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:06.506 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:06.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:20:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.660+0000 7fc1f9c37640 1 Processor -- start 2024-04-26T21:20:06.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.660+0000 7fc1f9c37640 1 -- start start 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.660+0000 7fc1f9c37640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1f415d360 0x7fc1f40d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f9c37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1f40d1510 0x7fc1f40d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f9c37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f40ceef0 con 0x7fc1f409c450 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f9c37640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f415d740 con 0x7fc1f415d360 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f9c37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f40cda60 con 0x7fc1f40d1510 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1f415d360 0x7fc1f40d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1f409c450 0x7fc1f40ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53322/0 (socket says 172.21.15.40:53322) 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f2ffd640 1 -- 172.21.15.40:0/3640539317 learned_addr learned my addr 172.21.15.40:0/3640539317 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f3fff640 1 --2- 172.21.15.40:0/3640539317 >> v2:172.21.15.184:3300/0 conn(0x7fc1f40d1510 0x7fc1f40d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f37fe640 1 -- 172.21.15.40:0/3640539317 >> v2:172.21.15.184:3300/0 conn(0x7fc1f40d1510 msgr2=0x7fc1f40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f37fe640 1 --2- 172.21.15.40:0/3640539317 >> v2:172.21.15.184:3300/0 conn(0x7fc1f40d1510 0x7fc1f40d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f37fe640 1 -- 172.21.15.40:0/3640539317 >> v1:172.21.15.40:6789/0 conn(0x7fc1f409c450 legacy=0x7fc1f40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f27fc640 1 -- 172.21.15.40:0/3640539317 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3799004240 0 0) 0x7fc1f40ceef0 con 0x7fc1f409c450 2024-04-26T21:20:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.661+0000 7fc1f37fe640 1 -- 172.21.15.40:0/3640539317 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f40d1100 con 0x7fc1f415d360 2024-04-26T21:20:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f37fe640 1 --2- 172.21.15.40:0/3640539317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f415d360 0x7fc1f40d0b20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4074820 tx=0x7fc1e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2c38a55bc76aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f27fc640 1 -- 172.21.15.40:0/3640539317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4037070 con 0x7fc1f415d360 2024-04-26T21:20:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f27fc640 1 -- 172.21.15.40:0/3640539317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e4032440 con 0x7fc1f415d360 2024-04-26T21:20:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f27fc640 1 -- 172.21.15.40:0/3640539317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4032810 con 0x7fc1f415d360 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f9c37640 1 -- 172.21.15.40:0/3640539317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f415d360 msgr2=0x7fc1f40d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.662+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/3640539317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f415d360 0x7fc1f40d0b20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4074820 tx=0x7fc1e402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 -- 172.21.15.40:0/3640539317 shutdown_connections 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/3640539317 >> v2:172.21.15.184:3300/0 conn(0x7fc1f40d1510 0x7fc1f40d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/3640539317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f415d360 0x7fc1f40d0b20 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 -- 172.21.15.40:0/3640539317 >> 172.21.15.40:0/3640539317 conn(0x7fc1f40b4a40 msgr2=0x7fc1f40b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 -- 172.21.15.40:0/3640539317 shutdown_connections 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.663+0000 7fc1f9c37640 1 -- 172.21.15.40:0/3640539317 wait complete. 2024-04-26T21:20:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.664+0000 7fc1f9c37640 1 Processor -- start 2024-04-26T21:20:06.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.664+0000 7fc1f9c37640 1 -- start start 2024-04-26T21:20:06.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f409c450 0x7fc1f40c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f415d360 0x7fc1f40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f409c8b0 con 0x7fc1f40d1510 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f40cdfe0 con 0x7fc1f409c450 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f9c37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f4070310 con 0x7fc1f415d360 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60002/0 (socket says 172.21.15.40:60002) 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 -- 172.21.15.40:0/1025222512 learned_addr learned my addr 172.21.15.40:0/1025222512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f37fe640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f409c450 0x7fc1f40c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f3fff640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f415d360 0x7fc1f40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 -- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f415d360 msgr2=0x7fc1f40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f415d360 0x7fc1f40c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 -- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f409c450 msgr2=0x7fc1f40c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f409c450 0x7fc1f40c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 -- 172.21.15.40:0/1025222512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f40b0420 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.665+0000 7fc1f2ffd640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c500 tx=0x7fc1e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.666+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8018070 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.666+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e8007580 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.666+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f40b0700 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.666+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8004a20 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.666+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f40b0fa0 con 0x7fc1f40d1510 2024-04-26T21:20:06.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.667+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f40b0af0 con 0x7fc1f40d1510 2024-04-26T21:20:06.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.667+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1e801b070 con 0x7fc1f40d1510 2024-04-26T21:20:06.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.667+0000 7fc1dbfff640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:06.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.668+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fc1e808ae20 con 0x7fc1f40d1510 2024-04-26T21:20:06.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.668+0000 7fc1f37fe640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:06.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.668+0000 7fc1f37fe640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 0x7fc1e006edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4034000 tx=0x7fc1e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:06.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.672+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1e80592e0 con 0x7fc1f40d1510 2024-04-26T21:20:06.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.898+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1f40693d0 con 0x7fc1f40d1510 2024-04-26T21:20:06.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.898+0000 7fc1dbfff640 1 -- 172.21.15.40:0/1025222512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fc1e8058be0 con 0x7fc1f40d1510 2024-04-26T21:20:06.900 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:20:06.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 msgr2=0x7fc1e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 0x7fc1e006edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4034000 tx=0x7fc1e402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 msgr2=0x7fc1f415e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c500 tx=0x7fc1e800c9c0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 shutdown_connections 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f415d360 0x7fc1f40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f40d1510 0x7fc1f415e2d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 --2- 172.21.15.40:0/1025222512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f409c450 0x7fc1f40c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 >> 172.21.15.40:0/1025222512 conn(0x7fc1f40b4a40 msgr2=0x7fc1f415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 shutdown_connections 2024-04-26T21:20:06.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:06.900+0000 7fc1f9c37640 1 -- 172.21.15.40:0/1025222512 wait complete. 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:20:06.914 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:06.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.915 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:20:06.933 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:20:06.933 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:20:06.933 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:20:06.933 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:06.934 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42267sec preferred_lft 42267sec 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:06.935 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:20:06.936 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:20:06.936 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:06.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:06.936 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:20:07.001 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:20:07.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:07.003 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:20:07.062 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:20:07.062 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:20:07.062 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:20:07.062 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:20:07.062 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:20:07.062 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:20:07.062 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:20:07.062 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:20:07.062 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:20:07.062 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:20:07.063 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:20:07.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:20:07.185 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:07.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:07.192 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:20:07.207 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:20:07.208 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:20:07.208 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:07.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:07.208 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:20:07.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:07.271 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:07.326 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:07.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:07.327 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:07.380 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:20:07.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:07.381 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:07.461 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:07.461 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:07.461 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:07.469 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:07.469 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:07.469 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:07.469 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:07.470 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:07.484 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:07.484 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:07.484 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:07.484 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:20:07.485 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:20:07.486 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:20:07.487 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:20:07.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:07.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:07.551 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:20:07.639 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:20:07.641 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:07.647 INFO:teuthology.orchestra.run.smithi184.stdout:client7405 2024-04-26T21:20:07.647 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:07.647 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:07.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-26T21:20:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.780+0000 7fe0e0c1d640 1 Processor -- start 2024-04-26T21:20:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.780+0000 7fe0e0c1d640 1 -- start start 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0e0c1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe0dc0d3820 0x7fe0dc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0e0c1d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0e0c1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0dc0d3c90 con 0x7fe0dc15f680 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0e0c1d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc15fa60 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0e0c1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc0d3f50 con 0x7fe0dc0d3820 2024-04-26T21:20:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60008/0 (socket says 172.21.15.40:60008) 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 -- 172.21.15.40:0/3122994493 learned_addr learned my addr 172.21.15.40:0/3122994493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0d9d9b640 1 --2- 172.21.15.40:0/3122994493 >> v2:172.21.15.184:3300/0 conn(0x7fe0dc0d3820 0x7fe0dc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 -- 172.21.15.40:0/3122994493 >> v2:172.21.15.184:3300/0 conn(0x7fe0dc0d3820 msgr2=0x7fe0dc0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 --2- 172.21.15.40:0/3122994493 >> v2:172.21.15.184:3300/0 conn(0x7fe0dc0d3820 0x7fe0dc0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 -- 172.21.15.40:0/3122994493 >> v1:172.21.15.184:6789/0 conn(0x7fe0dc15f680 legacy=0x7fe0dc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.781+0000 7fe0dad9d640 1 -- 172.21.15.40:0/3122994493 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc0b5090 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.782+0000 7fe0dad9d640 1 --2- 172.21.15.40:0/3122994493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc163040 tx=0x7fe0d402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794292bc9e56e643 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:07.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.782+0000 7fe0d959a640 1 -- 172.21.15.40:0/3122994493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4036070 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.782+0000 7fe0d959a640 1 -- 172.21.15.40:0/3122994493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d4031600 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.782+0000 7fe0d959a640 1 -- 172.21.15.40:0/3122994493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40319d0 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/3122994493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0b60b0 msgr2=0x7fe0dc0d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/3122994493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc163040 tx=0x7fe0d402a730 comp rx=0 tx=0).stop 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/3122994493 shutdown_connections 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/3122994493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0b60b0 0x7fe0dc0d4f20 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/3122994493 >> v2:172.21.15.184:3300/0 conn(0x7fe0dc0d3820 0x7fe0dc0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/3122994493 >> 172.21.15.40:0/3122994493 conn(0x7fe0dc08eca0 msgr2=0x7fe0dc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/3122994493 shutdown_connections 2024-04-26T21:20:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.783+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/3122994493 wait complete. 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.784+0000 7fe0e0c1d640 1 Processor -- start 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.784+0000 7fe0e0c1d640 1 -- start start 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 0x7fe0dc0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 0x7fe0dc15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0d55e0 con 0x7fe0dc15f680 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0d5460 con 0x7fe0dc0d3820 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0e0c1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe0dc0d5760 con 0x7fe0dc0b60b0 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60020/0 (socket says 172.21.15.40:60020) 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.785+0000 7fe0dad9d640 1 -- 172.21.15.40:0/1781903494 learned_addr learned my addr 172.21.15.40:0/1781903494 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0d9d9b640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 0x7fe0dc15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0da59c640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 0x7fe0dc0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 -- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 msgr2=0x7fe0dc0ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 0x7fe0dc0ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 -- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 msgr2=0x7fe0dc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 0x7fe0dc15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 -- 172.21.15.40:0/1781903494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc0c9c10 con 0x7fe0dc15f680 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0d9d9b640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 0x7fe0dc15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0da59c640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 0x7fe0dc0ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0dad9d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009f40 tx=0x7fe0d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d403f020 con 0x7fe0dc15f680 2024-04-26T21:20:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d4037040 con 0x7fe0dc15f680 2024-04-26T21:20:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1622b0 con 0x7fe0dc15f680 2024-04-26T21:20:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4036070 con 0x7fe0dc15f680 2024-04-26T21:20:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.786+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc162870 con 0x7fe0dc15f680 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.787+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc06a290 con 0x7fe0dc15f680 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.788+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0d4042020 con 0x7fe0dc15f680 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.788+0000 7fe0caffd640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 0x7fe0bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.788+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fe0d40abe20 con 0x7fe0dc15f680 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.788+0000 7fe0da59c640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 0x7fe0bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.789+0000 7fe0da59c640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 0x7fe0bc06edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0015e0 tx=0x7fe0cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.792+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0d4049030 con 0x7fe0dc15f680 2024-04-26T21:20:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.979+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7fe0dc162a40 con 0x7fe0dc15f680 2024-04-26T21:20:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.982+0000 7fe0caffd640 1 -- 172.21.15.40:0/1781903494 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v47)=0 v47) ==== 121+0+0 (secure 0 0 0) 0x7fe0dc0c9c10 con 0x7fe0dc15f680 2024-04-26T21:20:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 msgr2=0x7fe0bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 0x7fe0bc06edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0015e0 tx=0x7fe0cc009040 comp rx=0 tx=0).stop 2024-04-26T21:20:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 msgr2=0x7fe0dc0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009f40 tx=0x7fe0d4004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 shutdown_connections 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe0bc06c8f0 0x7fe0bc06edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc15f680 0x7fe0dc0cadc0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0dc0d3820 0x7fe0dc15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 --2- 172.21.15.40:0/1781903494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0dc0b60b0 0x7fe0dc0ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 >> 172.21.15.40:0/1781903494 conn(0x7fe0dc08eca0 msgr2=0x7fe0dc0b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 shutdown_connections 2024-04-26T21:20:07.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:07.984+0000 7fe0e0c1d640 1 -- 172.21.15.40:0/1781903494 wait complete. 2024-04-26T21:20:07.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2024-04-26T21:20:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.099+0000 7f586a546640 1 Processor -- start 2024-04-26T21:20:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.099+0000 7f586a546640 1 -- start start 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f586a546640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58641468a0 0x7f58640b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f586a546640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586406d0c0 con 0x7f586415d6f0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f586a546640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5864070740 con 0x7f586406ffd0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f586a546640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f5864070a00 con 0x7f58641468a0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f58637fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f586415d6f0 0x7f586406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53332/0 (socket says 172.21.15.40:53332) 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f58637fe640 1 -- 172.21.15.40:0/3647917382 learned_addr learned my addr 172.21.15.40:0/3647917382 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f5863fff640 1 --2- 172.21.15.40:0/3647917382 >> v2:172.21.15.184:3300/0 conn(0x7f58641468a0 0x7f58640b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817929381 0 0) 0x7f586406d0c0 con 0x7f586415d6f0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.100+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5850003520 con 0x7f586415d6f0 2024-04-26T21:20:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080294691 0 0) 0x7f5864070740 con 0x7f586406ffd0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f586406d0c0 con 0x7f586406ffd0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3900507205 0 0) 0x7f5850003520 con 0x7f586415d6f0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 >> v2:172.21.15.184:3300/0 conn(0x7f58641468a0 msgr2=0x7f58640b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 --2- 172.21.15.40:0/3647917382 >> v2:172.21.15.184:3300/0 conn(0x7f58641468a0 0x7f58640b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 >> v1:172.21.15.184:6789/0 conn(0x7f586406ffd0 legacy=0x7f586406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5864070ce0 con 0x7f586415d6f0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f584c001670 con 0x7f586415d6f0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 >> v1:172.21.15.40:6789/0 conn(0x7f586415d6f0 legacy=0x7f586406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850003b40 0x7f5850003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5850004800 0x7f5850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f585000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5864070a00 con 0x7f58500051c0 2024-04-26T21:20:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f585000c0f0 con 0x7f5850004800 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f585000c2f0 con 0x7f5850003b40 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f585000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5863fff640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850003b40 0x7f5850003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f58637fe640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5850004800 0x7f5850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 -- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850003b40 msgr2=0x7f5850003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5850003b40 0x7f5850003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 -- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5850004800 msgr2=0x7f5850004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5850004800 0x7f5850004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.101+0000 7f5868ae3640 1 -- 172.21.15.40:0/3647917382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5850003520 con 0x7f58500051c0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.102+0000 7f5868ae3640 1 -- 172.21.15.40:0/3647917382 reap_dead start 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.102+0000 7f5868ae3640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f585000b9c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f585c001fd0 tx=0x7f5850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892636be3314baa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.102+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c001bf0 con 0x7f58500051c0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.102+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c003b70 con 0x7f58500051c0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.102+0000 7f5862ffd640 1 -- 172.21.15.40:0/3647917382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0050d0 con 0x7f58500051c0 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 -- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 msgr2=0x7f585000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f585000b9c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f585c001fd0 tx=0x7f5850005830 comp rx=0 tx=0).stop 2024-04-26T21:20:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 -- 172.21.15.40:0/3647917382 shutdown_connections 2024-04-26T21:20:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 --2- 172.21.15.40:0/3647917382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f585000b9c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 -- 172.21.15.40:0/3647917382 >> 172.21.15.40:0/3647917382 conn(0x7f586406f050 msgr2=0x7f586406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 -- 172.21.15.40:0/3647917382 shutdown_connections 2024-04-26T21:20:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.103+0000 7f586a546640 1 -- 172.21.15.40:0/3647917382 wait complete. 2024-04-26T21:20:08.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.104+0000 7f586a546640 1 Processor -- start 2024-04-26T21:20:08.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.104+0000 7f586a546640 1 -- start start 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 0x7f586415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58640b6760 0x7f5864140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5864146c80 con 0x7f58500051c0 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5864071b50 con 0x7f586413d940 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f586a546640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5864146e00 con 0x7f58640b6760 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60042/0 (socket says 172.21.15.40:60042) 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 -- 172.21.15.40:0/404795511 learned_addr learned my addr 172.21.15.40:0/404795511 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f58637fe640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 0x7f586415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5868ae3640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58640b6760 0x7f5864140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 -- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58640b6760 msgr2=0x7f5864140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58640b6760 0x7f5864140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 -- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 msgr2=0x7f586415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 0x7f586415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 -- 172.21.15.40:0/404795511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586413c8a0 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f58637fe640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 0x7f586415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5863fff640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5854011940 tx=0x7f5854011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.105+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585400fa90 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.106+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585400fc30 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.106+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586413cb80 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.106+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854014440 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.106+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586413d140 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.107+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586406a290 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.107+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5854012040 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.108+0000 7f5860ff9640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 0x7f584406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.108+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f585408c1e0 con 0x7f58500051c0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.108+0000 7f58637fe640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 0x7f584406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.108+0000 7f58637fe640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 0x7f584406edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f584c001200 tx=0x7f584c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.112+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5854019030 con 0x7f58500051c0 2024-04-26T21:20:08.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.296+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f58640d1a40 con 0x7f58500051c0 2024-04-26T21:20:08.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.299+0000 7f5860ff9640 1 -- 172.21.15.40:0/404795511 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v48)=0 v48) ==== 119+0+0 (secure 0 0 0) 0x7f585405a610 con 0x7f58500051c0 2024-04-26T21:20:08.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.301+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 msgr2=0x7f584406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.301+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 0x7f584406edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f584c001200 tx=0x7f584c009040 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 msgr2=0x7f586413c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5854011940 tx=0x7f5854011e00 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 shutdown_connections 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f584406c8f0 0x7f584406edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58640b6760 0x7f5864140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f586413d940 0x7f586415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 --2- 172.21.15.40:0/404795511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58500051c0 0x7f586413c210 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 >> 172.21.15.40:0/404795511 conn(0x7f586406f050 msgr2=0x7f58640ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 shutdown_connections 2024-04-26T21:20:08.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.302+0000 7f586a546640 1 -- 172.21.15.40:0/404795511 wait complete. 2024-04-26T21:20:08.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2024-04-26T21:20:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.417+0000 7f71cbd6a640 1 Processor -- start 2024-04-26T21:20:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 -- start start 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71c4162ca0 0x7f71c4070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71c4071330 0x7f71c40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c40d2470 con 0x7f71c40d1c20 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f71c40cdc40 con 0x7f71c4071330 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71cbd6a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f71c40d2750 con 0x7f71c4162ca0 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71ca307640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71c4071330 0x7f71c40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.418+0000 7f71c9305640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f71c40d1c20 0x7f71c4070c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53344/0 (socket says 172.21.15.40:53344) 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71c9305640 1 -- 172.21.15.40:0/820750100 learned_addr learned my addr 172.21.15.40:0/820750100 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71c9b06640 1 --2- 172.21.15.40:0/820750100 >> v2:172.21.15.184:3300/0 conn(0x7f71c4162ca0 0x7f71c4070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71ca307640 1 -- 172.21.15.40:0/820750100 >> v2:172.21.15.184:3300/0 conn(0x7f71c4162ca0 msgr2=0x7f71c4070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71ca307640 1 --2- 172.21.15.40:0/820750100 >> v2:172.21.15.184:3300/0 conn(0x7f71c4162ca0 0x7f71c4070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71ca307640 1 -- 172.21.15.40:0/820750100 >> v1:172.21.15.40:6789/0 conn(0x7f71c40d1c20 legacy=0x7f71c4070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71ca307640 1 -- 172.21.15.40:0/820750100 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c40cd5f0 con 0x7f71c4071330 2024-04-26T21:20:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.419+0000 7f71c8b04640 1 -- 172.21.15.40:0/820750100 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009720993 0 0) 0x7f71c40d2470 con 0x7f71c40d1c20 2024-04-26T21:20:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71ca307640 1 --2- 172.21.15.40:0/820750100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4071330 0x7f71c40ce0d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f71c4072fc0 tx=0x7f71bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae380c430a82389 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71c8b04640 1 -- 172.21.15.40:0/820750100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007740 con 0x7f71c4071330 2024-04-26T21:20:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71c8b04640 1 -- 172.21.15.40:0/820750100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc0078e0 con 0x7f71c4071330 2024-04-26T21:20:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71c8b04640 1 -- 172.21.15.40:0/820750100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007cb0 con 0x7f71c4071330 2024-04-26T21:20:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 -- 172.21.15.40:0/820750100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4071330 msgr2=0x7f71c40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/820750100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4071330 0x7f71c40ce0d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f71c4072fc0 tx=0x7f71bc02a680 comp rx=0 tx=0).stop 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 -- 172.21.15.40:0/820750100 shutdown_connections 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/820750100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4071330 0x7f71c40ce0d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/820750100 >> v2:172.21.15.184:3300/0 conn(0x7f71c4162ca0 0x7f71c4070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 -- 172.21.15.40:0/820750100 >> 172.21.15.40:0/820750100 conn(0x7f71c409ba40 msgr2=0x7f71c409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 -- 172.21.15.40:0/820750100 shutdown_connections 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.420+0000 7f71cbd6a640 1 -- 172.21.15.40:0/820750100 wait complete. 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.421+0000 7f71cbd6a640 1 Processor -- start 2024-04-26T21:20:08.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 -- start start 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 0x7f71c41543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71c4162ca0 0x7f71c406e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71c40d2230 con 0x7f71c40d1c20 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71c40d20b0 con 0x7f71c4071330 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71cbd6a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71c40ce610 con 0x7f71c4162ca0 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71c9305640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71c9305640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60054/0 (socket says 172.21.15.40:60054) 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.422+0000 7f71c9305640 1 -- 172.21.15.40:0/2043669662 learned_addr learned my addr 172.21.15.40:0/2043669662 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71ca307640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71c4162ca0 0x7f71c406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9b06640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 0x7f71c41543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 -- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71c4162ca0 msgr2=0x7f71c406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71c4162ca0 0x7f71c406e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 -- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 msgr2=0x7f71c41543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 0x7f71c41543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 -- 172.21.15.40:0/2043669662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c40a02c0 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9b06640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 0x7f71c41543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71c9305640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c4d0 tx=0x7f71b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4018070 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b4007500 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c40a05a0 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4004a20 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.423+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c40a0b60 con 0x7f71c40d1c20 2024-04-26T21:20:08.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.424+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c40693d0 con 0x7f71c40d1c20 2024-04-26T21:20:08.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.425+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71b401b070 con 0x7f71c40d1c20 2024-04-26T21:20:08.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.425+0000 7f71ba7fc640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.425+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f71b408adb0 con 0x7f71c40d1c20 2024-04-26T21:20:08.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.425+0000 7f71c9b06640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.426+0000 7f71c9b06640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 0x7f71a806ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009a20 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.429+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71b4059270 con 0x7f71c40d1c20 2024-04-26T21:20:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.614+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f71c40d2ab0 con 0x7f71c40d1c20 2024-04-26T21:20:08.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.617+0000 7f71ba7fc640 1 -- 172.21.15.40:0/2043669662 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v49)=0 v49) ==== 128+0+0 (secure 0 0 0) 0x7f71b4058b70 con 0x7f71c40d1c20 2024-04-26T21:20:08.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 msgr2=0x7f71a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 0x7f71a806ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009a20 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 msgr2=0x7f71c409f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c4d0 tx=0x7f71b400c990 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 shutdown_connections 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71a806c920 0x7f71a806ede0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71c4162ca0 0x7f71c406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c40d1c20 0x7f71c409f950 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 --2- 172.21.15.40:0/2043669662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c4071330 0x7f71c41543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 >> 172.21.15.40:0/2043669662 conn(0x7f71c409ba40 msgr2=0x7f71c40d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 shutdown_connections 2024-04-26T21:20:08.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.619+0000 7f71cbd6a640 1 -- 172.21.15.40:0/2043669662 wait complete. 2024-04-26T21:20:08.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-26T21:20:08.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 Processor -- start 2024-04-26T21:20:08.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 -- start start 2024-04-26T21:20:08.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9e406f340 0x7fb9e40b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0300 0x7fb9e406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e406fc60 con 0x7fb9e40b3e50 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e40b46a0 con 0x7fb9e406f340 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.733+0000 7fb9eaa62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e40b5db0 con 0x7fb9e40d0300 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9e406f340 0x7fb9e40b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e8fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb9e40b3e50 0x7fb9e40720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53346/0 (socket says 172.21.15.40:53346) 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e8fff640 1 -- 172.21.15.40:0/4103638394 learned_addr learned my addr 172.21.15.40:0/4103638394 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e37fe640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0300 0x7fb9e406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e2ffd640 1 -- 172.21.15.40:0/4103638394 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299227295 0 0) 0x7fb9e406fc60 con 0x7fb9e40b3e50 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e2ffd640 1 -- 172.21.15.40:0/4103638394 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d0005730 con 0x7fb9e40b3e50 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e3fff640 1 -- 172.21.15.40:0/4103638394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0300 msgr2=0x7fb9e406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e3fff640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0300 0x7fb9e406f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e3fff640 1 -- 172.21.15.40:0/4103638394 >> v1:172.21.15.40:6789/0 conn(0x7fb9e40b3e50 legacy=0x7fb9e40720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.734+0000 7fb9e3fff640 1 -- 172.21.15.40:0/4103638394 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e40b6090 con 0x7fb9e406f340 2024-04-26T21:20:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.735+0000 7fb9e3fff640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e406f340 0x7fb9e40b6b70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40097a0 tx=0x7fb9d402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbc5c8b0fce8508 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.735+0000 7fb9e2ffd640 1 -- 172.21.15.40:0/4103638394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4037070 con 0x7fb9e406f340 2024-04-26T21:20:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.735+0000 7fb9e2ffd640 1 -- 172.21.15.40:0/4103638394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d402adb0 con 0x7fb9e406f340 2024-04-26T21:20:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.735+0000 7fb9e2ffd640 1 -- 172.21.15.40:0/4103638394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4032660 con 0x7fb9e406f340 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 -- 172.21.15.40:0/4103638394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e406f340 msgr2=0x7fb9e40b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e406f340 0x7fb9e40b6b70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40097a0 tx=0x7fb9d402a060 comp rx=0 tx=0).stop 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 -- 172.21.15.40:0/4103638394 shutdown_connections 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0300 0x7fb9e406f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/4103638394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e406f340 0x7fb9e40b6b70 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 -- 172.21.15.40:0/4103638394 >> 172.21.15.40:0/4103638394 conn(0x7fb9e40cd8a0 msgr2=0x7fb9e40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 -- 172.21.15.40:0/4103638394 shutdown_connections 2024-04-26T21:20:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.736+0000 7fb9eaa62640 1 -- 172.21.15.40:0/4103638394 wait complete. 2024-04-26T21:20:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.737+0000 7fb9eaa62640 1 Processor -- start 2024-04-26T21:20:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.737+0000 7fb9eaa62640 1 -- start start 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e406f340 0x7fb9e41f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40b3e50 0x7fb9e4143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e40b42e0 con 0x7fb9e40d0300 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e40b6650 con 0x7fb9e406f340 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e40b4460 con 0x7fb9e40b3e50 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60062/0 (socket says 172.21.15.40:60062) 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 -- 172.21.15.40:0/3443410022 learned_addr learned my addr 172.21.15.40:0/3443410022 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e37fe640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40b3e50 0x7fb9e4143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e3fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e406f340 0x7fb9e41f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 -- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40b3e50 msgr2=0x7fb9e4143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40b3e50 0x7fb9e4143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 -- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e406f340 msgr2=0x7fb9e41f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e406f340 0x7fb9e41f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 -- 172.21.15.40:0/3443410022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e40ccc70 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e8fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00caa0 tx=0x7fb9dc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc017070 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00ac00 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.738+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e40cc990 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.739+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc012400 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.739+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e41344a0 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.740+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e406a290 con 0x7fb9e40d0300 2024-04-26T21:20:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.740+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9dc01a070 con 0x7fb9e40d0300 2024-04-26T21:20:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.740+0000 7fb9e0ff9640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.741+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fb9dc08b8e0 con 0x7fb9e40d0300 2024-04-26T21:20:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.741+0000 7fb9e3fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.741+0000 7fb9e3fff640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.745+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9dc059d10 con 0x7fb9e40d0300 2024-04-26T21:20:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.932+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fb9e406d2b0 con 0x7fb9e40d0300 2024-04-26T21:20:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.932+0000 7fb9e0ff9640 1 -- 172.21.15.40:0/3443410022 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v49) ==== 106+0+4 (secure 0 0 0) 0x7fb9dc059610 con 0x7fb9e40d0300 2024-04-26T21:20:08.934 INFO:teuthology.orchestra.run.smithi040.stdout:100 2024-04-26T21:20:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 msgr2=0x7fb9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 msgr2=0x7fb9e4144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00caa0 tx=0x7fb9dc00cf60 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 shutdown_connections 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40d0300 0x7fb9e4144470 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40b3e50 0x7fb9e4143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 --2- 172.21.15.40:0/3443410022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e406f340 0x7fb9e41f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 >> 172.21.15.40:0/3443410022 conn(0x7fb9e40cd8a0 msgr2=0x7fb9e40cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 shutdown_connections 2024-04-26T21:20:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:08.934+0000 7fb9eaa62640 1 -- 172.21.15.40:0/3443410022 wait complete. 2024-04-26T21:20:08.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-26T21:20:09.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.049+0000 7f9591262640 1 Processor -- start 2024-04-26T21:20:09.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.049+0000 7f9591262640 1 -- start start 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9591262640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f958c0d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9591262640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c0b6aa0 con 0x7f958c15f990 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9591262640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c0d3eb0 con 0x7f958c15f5b0 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9591262640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f958c0d4190 con 0x7f958c0d3340 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f958a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f958c15f990 0x7f958c0d0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53348/0 (socket says 172.21.15.40:53348) 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f958a7fc640 1 -- 172.21.15.40:0/4045780032 learned_addr learned my addr 172.21.15.40:0/4045780032 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f958b7fe640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f958c0d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158480485 0 0) 0x7f958c0b6aa0 con 0x7f958c15f990 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9578003520 con 0x7f958c15f990 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.050+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4180866557 0 0) 0x7f9578003520 con 0x7f958c15f990 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 msgr2=0x7f958c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f958c0d37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 >> v1:172.21.15.184:6789/0 conn(0x7f958c15f5b0 legacy=0x7f958c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0d0200 con 0x7f958c15f990 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9574001670 con 0x7f958c15f990 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 >> v1:172.21.15.40:6789/0 conn(0x7f958c15f990 legacy=0x7f958c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9578004800 0x7f9578004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f9578005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0d4190 con 0x7f95780039e0 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f957800baf0 con 0x7f9578004800 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f957800bcf0 con 0x7f958c0d3340 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958b7fe640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f9578005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958a7fc640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9578004800 0x7f9578004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 -- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 msgr2=0x7f9578005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f9578005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 -- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9578004800 msgr2=0x7f9578004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9578004800 0x7f9578004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958affd640 1 -- 172.21.15.40:0/4045780032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9578003520 con 0x7f95780039e0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958b7fe640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c0d3340 0x7f9578005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.051+0000 7f958a7fc640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9578004800 0x7f9578004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f958affd640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f9578003e00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f957c001e50 tx=0x7f9578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d5676acd9d0599 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c001a70 con 0x7f95780039e0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c003970 con 0x7f95780039e0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9589ffb640 1 -- 172.21.15.40:0/4045780032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c003d60 con 0x7f95780039e0 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9591262640 1 -- 172.21.15.40:0/4045780032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 msgr2=0x7f9578003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9591262640 1 --2- 172.21.15.40:0/4045780032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f9578003e00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f957c001e50 tx=0x7f9578005830 comp rx=0 tx=0).stop 2024-04-26T21:20:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f958b7fe640 1 -- 172.21.15.40:0/4045780032 reap_dead start 2024-04-26T21:20:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9591262640 1 -- 172.21.15.40:0/4045780032 shutdown_connections 2024-04-26T21:20:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.052+0000 7f9591262640 1 -- 172.21.15.40:0/4045780032 >> 172.21.15.40:0/4045780032 conn(0x7f958c08ec40 msgr2=0x7f958c09e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.053+0000 7f9591262640 1 -- 172.21.15.40:0/4045780032 shutdown_connections 2024-04-26T21:20:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.053+0000 7f9591262640 1 -- 172.21.15.40:0/4045780032 wait complete. 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.054+0000 7f9591262640 1 Processor -- start 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.054+0000 7f9591262640 1 -- start start 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c156a00 0x7f958c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 0x7f958c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0b5900 con 0x7f95780039e0 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0b5780 con 0x7f958c0cdc70 2024-04-26T21:20:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f9591262640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f958c0d1110 con 0x7f958c156a00 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60074/0 (socket says 172.21.15.40:60074) 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 -- 172.21.15.40:0/3358277207 learned_addr learned my addr 172.21.15.40:0/3358277207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958a7fc640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c156a00 0x7f958c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958b7fe640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 0x7f958c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 -- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c156a00 msgr2=0x7f958c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c156a00 0x7f958c15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 -- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 msgr2=0x7f958c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 0x7f958c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 -- 172.21.15.40:0/3358277207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c1608c0 con 0x7f95780039e0 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958b7fe640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 0x7f958c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f958affd640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f957c001f70 tx=0x7f957c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.055+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c03f030 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.056+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c0050d0 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.056+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c0cd190 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.056+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c037040 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.056+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c0cd750 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.057+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c06a290 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.057+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f957c042070 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.057+0000 7f95837fe640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 0x7f956c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.058+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f957c0ab780 con 0x7f95780039e0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.058+0000 7f958a7fc640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 0x7f956c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.058+0000 7f958a7fc640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 0x7f956c06edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f958c0babf0 tx=0x7f9574007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.062+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f957c079c90 con 0x7f95780039e0 2024-04-26T21:20:09.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.254+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f958c0d60c0 con 0x7f95780039e0 2024-04-26T21:20:09.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.257+0000 7f95837fe640 1 -- 172.21.15.40:0/3358277207 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v50)=0 v50) ==== 124+0+0 (secure 0 0 0) 0x7f957c079590 con 0x7f95780039e0 2024-04-26T21:20:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 msgr2=0x7f956c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 0x7f956c06edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f958c0babf0 tx=0x7f9574007040 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 msgr2=0x7f958c156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f957c001f70 tx=0x7f957c0014a0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 shutdown_connections 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f956c06c8f0 0x7f956c06edb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f958c0cdc70 0x7f958c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f958c156a00 0x7f958c15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 --2- 172.21.15.40:0/3358277207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780039e0 0x7f958c156290 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 >> 172.21.15.40:0/3358277207 conn(0x7f958c08ec40 msgr2=0x7f958c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 shutdown_connections 2024-04-26T21:20:09.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.259+0000 7f9591262640 1 -- 172.21.15.40:0/3358277207 wait complete. 2024-04-26T21:20:09.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-26T21:20:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.373+0000 7f6fe19e4640 1 Processor -- start 2024-04-26T21:20:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.373+0000 7f6fe19e4640 1 -- start start 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fe19e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fdc06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fe19e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc070b30 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fe19e4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc070e10 con 0x7f6fdc0706a0 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fe19e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0af610 con 0x7f6fdc15f2b0 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fdb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6fdc06c9b0 0x7f6fdc0afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53364/0 (socket says 172.21.15.40:53364) 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fdb7fe640 1 -- 172.21.15.40:0/4036811536 learned_addr learned my addr 172.21.15.40:0/4036811536 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fdc06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563074147 0 0) 0x7f6fdc070b30 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fc8003520 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024933540 0 0) 0x7f6fdc070e10 con 0x7f6fdc0706a0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fdc070b30 con 0x7f6fdc0706a0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4080850192 0 0) 0x7f6fc8003520 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 msgr2=0x7f6fdc06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fdc06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 >> v1:172.21.15.184:6789/0 conn(0x7f6fdc0706a0 legacy=0x7f6fdc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc0710f0 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fdc06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f6fd40019f0 con 0x7f6fdc06c9b0 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 >> v1:172.21.15.40:6789/0 conn(0x7f6fdc06c9b0 legacy=0x7f6fdc0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fc8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.374+0000 7f6fda7fc640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fc8004800 0x7f6fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fda7fc640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800b9c0 con 0x7f6fc8003b70 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800bb70 con 0x7f6fc8004800 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc800bd70 con 0x7f6fdc15f2b0 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fc8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fd3fff640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fc8004800 0x7f6fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdb7fe640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 -- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 msgr2=0x7f6fc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc15f2b0 0x7f6fc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 -- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fc8004800 msgr2=0x7f6fc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fc8004800 0x7f6fc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.375+0000 7f6fdaffd640 1 -- 172.21.15.40:0/4036811536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc8003520 con 0x7f6fc8003b70 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fc8003f70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009c60 tx=0x7f6fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716b3832a6c1d2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc036070 con 0x7f6fc8003b70 2024-04-26T21:20:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc031400 con 0x7f6fc8003b70 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fda7fc640 1 -- 172.21.15.40:0/4036811536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc009860 con 0x7f6fc8003b70 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fe19e4640 1 -- 172.21.15.40:0/4036811536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 msgr2=0x7f6fc8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/4036811536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fc8003f70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009c60 tx=0x7f6fc8005830 comp rx=0 tx=0).stop 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fdb7fe640 1 -- 172.21.15.40:0/4036811536 reap_dead start 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.376+0000 7f6fe19e4640 1 -- 172.21.15.40:0/4036811536 shutdown_connections 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.377+0000 7f6fe19e4640 1 -- 172.21.15.40:0/4036811536 >> 172.21.15.40:0/4036811536 conn(0x7f6fdc06f050 msgr2=0x7f6fdc06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.377+0000 7f6fe19e4640 1 -- 172.21.15.40:0/4036811536 shutdown_connections 2024-04-26T21:20:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.377+0000 7f6fe19e4640 1 -- 172.21.15.40:0/4036811536 wait complete. 2024-04-26T21:20:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.378+0000 7f6fe19e4640 1 Processor -- start 2024-04-26T21:20:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.378+0000 7f6fe19e4640 1 -- start start 2024-04-26T21:20:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.378+0000 7f6fe19e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.378+0000 7f6fe19e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc162340 0x7f6fdc156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.378+0000 7f6fe19e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc0c7ea0 0x7f6fdc162b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fe19e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b1890 con 0x7f6fc8003b70 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fe19e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b4f40 con 0x7f6fdc162340 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fe19e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc15f690 con 0x7f6fdc0c7ea0 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60084/0 (socket says 172.21.15.40:60084) 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 -- 172.21.15.40:0/499597269 learned_addr learned my addr 172.21.15.40:0/499597269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fd3fff640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc162340 0x7f6fdc156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdb7fe640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc0c7ea0 0x7f6fdc162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 -- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc0c7ea0 msgr2=0x7f6fdc162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc0c7ea0 0x7f6fdc162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 -- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc162340 msgr2=0x7f6fdc156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc162340 0x7f6fdc156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 -- 172.21.15.40:0/499597269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc156e70 con 0x7f6fc8003b70 2024-04-26T21:20:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fdaffd640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009d80 tx=0x7f6fcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc040020 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0093e0 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc15b5e0 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc036040 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.379+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc15bba0 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.380+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc06a290 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.381+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fcc043020 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.381+0000 7f6fd8ff9640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.382+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f6fcc0abf30 con 0x7f6fc8003b70 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.382+0000 7f6fd3fff640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.382+0000 7f6fd3fff640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40018d0 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.386+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fcc04a030 con 0x7f6fc8003b70 2024-04-26T21:20:09.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.570+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f6fdc161f50 con 0x7f6fc8003b70 2024-04-26T21:20:09.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.570+0000 7f6fd8ff9640 1 -- 172.21.15.40:0/499597269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v50) ==== 106+0+8 (secure 0 0 0) 0x7f6fdc156e70 con 0x7f6fc8003b70 2024-04-26T21:20:09.572 INFO:teuthology.orchestra.run.smithi040.stdout:1048576 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 msgr2=0x7f6fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40018d0 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 msgr2=0x7f6fdc15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009d80 tx=0x7f6fcc004730 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 shutdown_connections 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6fdc0c7ea0 0x7f6fdc162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc162340 0x7f6fdc156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 --2- 172.21.15.40:0/499597269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc8003b70 0x7f6fdc15af00 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 >> 172.21.15.40:0/499597269 conn(0x7f6fdc06f050 msgr2=0x7f6fdc073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 shutdown_connections 2024-04-26T21:20:09.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.573+0000 7f6fe19e4640 1 -- 172.21.15.40:0/499597269 wait complete. 2024-04-26T21:20:09.586 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-26T21:20:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.686+0000 7f92bce37640 1 Processor -- start 2024-04-26T21:20:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.686+0000 7f92bce37640 1 -- start start 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92bce37640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f92b80d2070 0x7f92b80baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92bce37640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b80d4240 con 0x7f92b815f490 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92bce37640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b80d31a0 con 0x7f92b80d1be0 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92bce37640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f92b80d3480 con 0x7f92b80d2070 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92b659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f92b815f490 0x7f92b80d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53370/0 (socket says 172.21.15.40:53370) 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92b659c640 1 -- 172.21.15.40:0/3720016781 learned_addr learned my addr 172.21.15.40:0/3720016781 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.687+0000 7f92b6d9d640 1 --2- 172.21.15.40:0/3720016781 >> v2:172.21.15.184:3300/0 conn(0x7f92b80d2070 0x7f92b80baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904480490 0 0) 0x7f92b80d4240 con 0x7f92b815f490 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92a4003520 con 0x7f92b815f490 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3777779551 0 0) 0x7f92a4003520 con 0x7f92b815f490 2024-04-26T21:20:09.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 >> v2:172.21.15.184:3300/0 conn(0x7f92b80d2070 msgr2=0x7f92b80baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 --2- 172.21.15.40:0/3720016781 >> v2:172.21.15.184:3300/0 conn(0x7f92b80d2070 0x7f92b80baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 >> v1:172.21.15.184:6789/0 conn(0x7f92b80d1be0 legacy=0x7f92b80ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80d3760 con 0x7f92b815f490 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f92b0001670 con 0x7f92b815f490 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 >> v1:172.21.15.40:6789/0 conn(0x7f92b815f490 legacy=0x7f92b80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 0x7f92a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92a4004800 0x7f92a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40051c0 0x7f92a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d3480 con 0x7f92a40039b0 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92a400c060 con 0x7f92a4004800 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92a400c260 con 0x7f92a40051c0 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 0x7f92a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b5d9b640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92a4004800 0x7f92a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b6d9d640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40051c0 0x7f92a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 -- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40051c0 msgr2=0x7f92a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40051c0 0x7f92a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 -- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92a4004800 msgr2=0x7f92a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92a4004800 0x7f92a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b659c640 1 -- 172.21.15.40:0/3720016781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a4003520 con 0x7f92a40039b0 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b5d9b640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92a4004800 0x7f92a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.688+0000 7f92b6d9d640 1 -- 172.21.15.40:0/3720016781 reap_dead start 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.689+0000 7f92b659c640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 0x7f92a4003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c210 tx=0x7f92a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961986147336027 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.689+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007d30 con 0x7f92a40039b0 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.689+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b0005ad0 con 0x7f92a40039b0 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.689+0000 7f92b559a640 1 -- 172.21.15.40:0/3720016781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007070 con 0x7f92a40039b0 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 -- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 msgr2=0x7f92a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 0x7f92a4003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c210 tx=0x7f92a4005830 comp rx=0 tx=0).stop 2024-04-26T21:20:09.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 -- 172.21.15.40:0/3720016781 shutdown_connections 2024-04-26T21:20:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 --2- 172.21.15.40:0/3720016781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92a40039b0 0x7f92a4003dd0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 -- 172.21.15.40:0/3720016781 >> 172.21.15.40:0/3720016781 conn(0x7f92b809e530 msgr2=0x7f92b80d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 -- 172.21.15.40:0/3720016781 shutdown_connections 2024-04-26T21:20:09.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.690+0000 7f92bce37640 1 -- 172.21.15.40:0/3720016781 wait complete. 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.691+0000 7f92bce37640 1 Processor -- start 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.691+0000 7f92bce37640 1 -- start start 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40039b0 0x7f92b815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 0x7f92b80ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d2480 con 0x7f92b80cc2c0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d3d20 con 0x7f92b80ce8e0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92bce37640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92b815f870 con 0x7f92a40039b0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60094/0 (socket says 172.21.15.40:60094) 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 -- 172.21.15.40:0/2912018045 learned_addr learned my addr 172.21.15.40:0/2912018045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b659c640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40039b0 0x7f92b815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b5d9b640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 0x7f92b80ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 -- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40039b0 msgr2=0x7f92b815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40039b0 0x7f92b815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 -- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 msgr2=0x7f92b80ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 0x7f92b80ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 -- 172.21.15.40:0/2912018045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80b93c0 con 0x7f92b80cc2c0 2024-04-26T21:20:09.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b5d9b640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 0x7f92b80ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92b6d9d640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f92b80d6250 tx=0x7f92a80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.692+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8011e10 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.693+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a801b070 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.693+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b80ccea0 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.693+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8003990 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.693+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b80cd460 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.694+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b806a290 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.694+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92a8005d10 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.694+0000 7f92aeffd640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 0x7f929806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.694+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f92a808c190 con 0x7f92b80cc2c0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.695+0000 7f92b659c640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 0x7f929806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:09.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.695+0000 7f92b659c640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 0x7f929806edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f92b0032000 tx=0x7f92b002d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:09.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.699+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92a8025020 con 0x7f92b80cc2c0 2024-04-26T21:20:09.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.886+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f92b8162790 con 0x7f92b80cc2c0 2024-04-26T21:20:09.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.886+0000 7f92aeffd640 1 -- 172.21.15.40:0/2912018045 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v50) ==== 112+0+10 (secure 0 0 0) 0x7f92b80b93c0 con 0x7f92b80cc2c0 2024-04-26T21:20:09.890 INFO:teuthology.orchestra.run.smithi040.stdout:60.000000 2024-04-26T21:20:09.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 msgr2=0x7f929806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 0x7f929806edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f92b0032000 tx=0x7f92b002d000 comp rx=0 tx=0).stop 2024-04-26T21:20:09.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 msgr2=0x7f92b80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:09.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f92b80d6250 tx=0x7f92a80115a0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 shutdown_connections 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f929806c8f0 0x7f929806edb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92b80cc2c0 0x7f92b80cc770 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92b80ce8e0 0x7f92b80ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 --2- 172.21.15.40:0/2912018045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92a40039b0 0x7f92b815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 >> 172.21.15.40:0/2912018045 conn(0x7f92b809e530 msgr2=0x7f92b80b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 shutdown_connections 2024-04-26T21:20:09.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:09.888+0000 7f92bce37640 1 -- 172.21.15.40:0/2912018045 wait complete. 2024-04-26T21:20:09.900 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:09.900 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:09.916 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:20:10.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:20:10.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:10.015 INFO:teuthology.orchestra.run.smithi040.stdout:client8640 2024-04-26T21:20:10.015 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:10.016 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:10.031 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:20:10.031 DEBUG:teuthology.orchestra.run.smithi040:> import fcntl 2024-04-26T21:20:10.031 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:20:10.031 DEBUG:teuthology.orchestra.run.smithi040:> import time 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> n = 200 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> if not os.path.exists(abs_path): 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(abs_path) 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> handles = [] 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(0, n): 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> fname = "file_"+str(i) 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> path = os.path.join(abs_path, fname) 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> handles.append(open(path, '"'"'w'"'"')) 2024-04-26T21:20:10.032 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> print("waiting with handles open", file=sys.stderr) 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> while True: 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> if os.read(0, 4096) == b"": 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> break 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> except BlockingIOError: 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> pass 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(2) 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-26T21:20:10.033 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:20:10.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:10.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.178+0000 7fc50c893640 1 Processor -- start 2024-04-26T21:20:10.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.178+0000 7fc50c893640 1 -- start start 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50c893640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc508162f30 0x7fc50806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50c893640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc508163830 con 0x7fc50806c2a0 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50c893640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc508163b10 con 0x7fc50815fa20 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50c893640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc508163df0 con 0x7fc508162f30 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc506d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc508162f30 0x7fc50806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc50806c2a0 0x7fc50806c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53376/0 (socket says 172.21.15.40:53376) 2024-04-26T21:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc50659c640 1 -- 172.21.15.40:0/1272566446 learned_addr learned my addr 172.21.15.40:0/1272566446 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc506d9d640 1 -- 172.21.15.40:0/1272566446 >> v1:172.21.15.184:6789/0 conn(0x7fc50815fa20 legacy=0x7fc5080cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc506d9d640 1 -- 172.21.15.40:0/1272566446 >> v1:172.21.15.40:6789/0 conn(0x7fc50806c2a0 legacy=0x7fc50806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc505d9b640 1 -- 172.21.15.40:0/1272566446 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658397602 0 0) 0x7fc508163830 con 0x7fc50806c2a0 2024-04-26T21:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.179+0000 7fc506d9d640 1 -- 172.21.15.40:0/1272566446 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080ce950 con 0x7fc508162f30 2024-04-26T21:20:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.180+0000 7fc506d9d640 1 --2- 172.21.15.40:0/1272566446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50806fb40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc500009d90 tx=0x7fc50002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb971ea0915ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.180+0000 7fc505d9b640 1 -- 172.21.15.40:0/1272566446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500007740 con 0x7fc508162f30 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc505d9b640 1 -- 172.21.15.40:0/1272566446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5000078e0 con 0x7fc508162f30 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc505d9b640 1 -- 172.21.15.40:0/1272566446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500007cb0 con 0x7fc508162f30 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 -- 172.21.15.40:0/1272566446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 msgr2=0x7fc50806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 --2- 172.21.15.40:0/1272566446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50806fb40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc500009d90 tx=0x7fc50002a580 comp rx=0 tx=0).stop 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 -- 172.21.15.40:0/1272566446 shutdown_connections 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 --2- 172.21.15.40:0/1272566446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50806fb40 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 -- 172.21.15.40:0/1272566446 >> 172.21.15.40:0/1272566446 conn(0x7fc5080b3b10 msgr2=0x7fc5080b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 -- 172.21.15.40:0/1272566446 shutdown_connections 2024-04-26T21:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.181+0000 7fc50c893640 1 -- 172.21.15.40:0/1272566446 wait complete. 2024-04-26T21:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.183+0000 7fc50c893640 1 Processor -- start 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.183+0000 7fc50c893640 1 -- start start 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.183+0000 7fc50c893640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc50806c2a0 0x7fc5080b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc50815fa20 0x7fc50815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc508160820 con 0x7fc508162f30 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081606a0 con 0x7fc50815fa20 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc5081633a0 con 0x7fc50806c2a0 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60106/0 (socket says 172.21.15.40:60106) 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 -- 172.21.15.40:0/4055233249 learned_addr learned my addr 172.21.15.40:0/4055233249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50659c640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc50806c2a0 0x7fc5080b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc4fdd9b640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc50815fa20 0x7fc50815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 -- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc50806c2a0 msgr2=0x7fc5080b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc50806c2a0 0x7fc5080b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 -- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc50815fa20 msgr2=0x7fc50815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc50815fa20 0x7fc50815aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 -- 172.21.15.40:0/4055233249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50815afb0 con 0x7fc508162f30 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc506d9d640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc500009eb0 tx=0x7fc500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500040070 con 0x7fc508162f30 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc500031c90 con 0x7fc508162f30 2024-04-26T21:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.184+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5080b2100 con 0x7fc508162f30 2024-04-26T21:20:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.185+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500045680 con 0x7fc508162f30 2024-04-26T21:20:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.185+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5080b2730 con 0x7fc508162f30 2024-04-26T21:20:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.186+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5080b2ef0 con 0x7fc508162f30 2024-04-26T21:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.187+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc500039040 con 0x7fc508162f30 2024-04-26T21:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.187+0000 7fc4ff7fe640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 0x7fc4e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.187+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fc5000ad630 con 0x7fc508162f30 2024-04-26T21:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.187+0000 7fc50659c640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 0x7fc4e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.188+0000 7fc50659c640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 0x7fc4e806ede0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000f60 tx=0x7fc4f8007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.195+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5080b2ef0 con 0x7fc508162f30 2024-04-26T21:20:10.290 INFO:teuthology.orchestra.run.smithi040.stderr:waiting with handles open 2024-04-26T21:20:10.291 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:20:10.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.433+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5080693d0 con 0x7fc508162f30 2024-04-26T21:20:10.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.434+0000 7fc4ff7fe640 1 -- 172.21.15.40:0/4055233249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7fc50007baa0 con 0x7fc508162f30 2024-04-26T21:20:10.436 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:10.436 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:02.108019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 76 2024-04-26T21:20:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.435+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 msgr2=0x7fc4e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.435+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 0x7fc4e806ede0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000f60 tx=0x7fc4f8007c00 comp rx=0 tx=0).stop 2024-04-26T21:20:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 msgr2=0x7fc50815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc500009eb0 tx=0x7fc500004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 shutdown_connections 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc4e806c920 0x7fc4e806ede0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc508162f30 0x7fc50815d960 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc50815fa20 0x7fc50815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 --2- 172.21.15.40:0/4055233249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc50806c2a0 0x7fc5080b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 >> 172.21.15.40:0/4055233249 conn(0x7fc5080b3b10 msgr2=0x7fc50815eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 shutdown_connections 2024-04-26T21:20:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.436+0000 7fc50c893640 1 -- 172.21.15.40:0/4055233249 wait complete. 2024-04-26T21:20:10.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-26T21:20:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":7405,"entity":{"name":{"type":"client","num":7405},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2719632244}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.0869620069999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9909761005627331,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7405 v1:192.168.0.1:0/2719632244","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":8640,"entity":{"name":{"type":"client","num":8640},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1916941838}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":226,"uptime":4.226947977,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":404.67111290760386,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8640 v1:192.168.0.1:0/1916941838","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:20:10.568 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2719632244, "type": "v1" }, "name": { "num": 7405, "type": "client" } }, "id": 7405, "inst": "client.7405 v1:192.168.0.1:0/2719632244", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.990976100562733 }, "state": "open", "uptime": 3.086962007 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1916941838, "type": "v1" }, "name": { "num": 8640, "type": "client" } }, "id": 8640, "inst": "client.8640 v1:192.168.0.1:0/1916941838", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 226, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 404.67111290760386 }, "state": "open", "uptime": 4.226947977 } ] 2024-04-26T21:20:10.568 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-26T21:20:10.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:20:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.667+0000 7f8703566640 1 Processor -- start 2024-04-26T21:20:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.667+0000 7f8703566640 1 -- start start 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8703566640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86fc09ba40 0x7f86fc15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8703566640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86fc0b5530 0x7f86fc0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8703566640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86fc15fce0 con 0x7f86fc070570 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8703566640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86fc0b59c0 con 0x7f86fc09ba40 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8703566640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86fc160610 con 0x7f86fc0b5530 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701302640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86fc09ba40 0x7f86fc15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8700b01640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86fc070570 0x7f86fc09be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54172/0 (socket says 172.21.15.40:54172) 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8700b01640 1 -- 172.21.15.40:0/3513943611 learned_addr learned my addr 172.21.15.40:0/3513943611 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701b03640 1 --2- 172.21.15.40:0/3513943611 >> v2:172.21.15.184:3300/0 conn(0x7f86fc0b5530 0x7f86fc0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701302640 1 -- 172.21.15.40:0/3513943611 >> v2:172.21.15.184:3300/0 conn(0x7f86fc0b5530 msgr2=0x7f86fc0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701302640 1 --2- 172.21.15.40:0/3513943611 >> v2:172.21.15.184:3300/0 conn(0x7f86fc0b5530 0x7f86fc0d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701302640 1 -- 172.21.15.40:0/3513943611 >> v1:172.21.15.40:6789/0 conn(0x7f86fc070570 legacy=0x7f86fc09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f8701302640 1 -- 172.21.15.40:0/3513943611 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc15ffc0 con 0x7f86fc09ba40 2024-04-26T21:20:10.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.668+0000 7f86f3fff640 1 -- 172.21.15.40:0/3513943611 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 575415577 0 0) 0x7f86fc15fce0 con 0x7f86fc070570 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.669+0000 7f8701302640 1 --2- 172.21.15.40:0/3513943611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc09ba40 0x7f86fc15f6a0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f86fc074080 tx=0x7f86f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a135a32e0c0467d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.669+0000 7f86f3fff640 1 -- 172.21.15.40:0/3513943611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8037070 con 0x7f86fc09ba40 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.669+0000 7f86f3fff640 1 -- 172.21.15.40:0/3513943611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f8032440 con 0x7f86fc09ba40 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.669+0000 7f86f3fff640 1 -- 172.21.15.40:0/3513943611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8032810 con 0x7f86fc09ba40 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 -- 172.21.15.40:0/3513943611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc09ba40 msgr2=0x7f86fc15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 --2- 172.21.15.40:0/3513943611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc09ba40 0x7f86fc15f6a0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f86fc074080 tx=0x7f86f802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 -- 172.21.15.40:0/3513943611 shutdown_connections 2024-04-26T21:20:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 --2- 172.21.15.40:0/3513943611 >> v2:172.21.15.184:3300/0 conn(0x7f86fc0b5530 0x7f86fc0d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 --2- 172.21.15.40:0/3513943611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc09ba40 0x7f86fc15f6a0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 -- 172.21.15.40:0/3513943611 >> 172.21.15.40:0/3513943611 conn(0x7f86fc0b4a10 msgr2=0x7f86fc0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 -- 172.21.15.40:0/3513943611 shutdown_connections 2024-04-26T21:20:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.670+0000 7f8703566640 1 -- 172.21.15.40:0/3513943611 wait complete. 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.671+0000 7f8703566640 1 Processor -- start 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.671+0000 7f8703566640 1 -- start start 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 0x7f86fc0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 0x7f86fc1f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc0d1580 con 0x7f86fc0b5530 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc0709b0 con 0x7f86fc09ba40 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86fc0d1700 con 0x7f86fc070570 2024-04-26T21:20:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39336/0 (socket says 172.21.15.40:39336) 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 -- 172.21.15.40:0/3884326093 learned_addr learned my addr 172.21.15.40:0/3884326093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8700b01640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 0x7f86fc1f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701302640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 0x7f86fc0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 -- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 msgr2=0x7f86fc0b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 0x7f86fc0b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 -- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 msgr2=0x7f86fc1f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 0x7f86fc1f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 -- 172.21.15.40:0/3884326093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc0b6430 con 0x7f86fc0b5530 2024-04-26T21:20:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8700b01640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 0x7f86fc1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701302640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 0x7f86fc0b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8701b03640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007e60 tx=0x7f86f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4014420 con 0x7f86fc0b5530 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f4019070 con 0x7f86fc0b5530 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.672+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1f8e50 con 0x7f86fc0b5530 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.673+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4014820 con 0x7f86fc0b5530 2024-04-26T21:20:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.673+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc0b66f0 con 0x7f86fc0b5530 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.674+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc06a290 con 0x7f86fc0b5530 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.674+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86f40149c0 con 0x7f86fc0b5530 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.675+0000 7f86f1ffb640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 0x7f86e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.675+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f86f408c3d0 con 0x7f86fc0b5530 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.675+0000 7f8701302640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 0x7f86e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.675+0000 7f8701302640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 0x7f86e006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f86f8034000 tx=0x7f86f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:10.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.679+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86f405a800 con 0x7f86fc0b5530 2024-04-26T21:20:10.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.905+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f86fc0d0580 con 0x7f86fc0b5530 2024-04-26T21:20:10.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.905+0000 7f86f1ffb640 1 -- 172.21.15.40:0/3884326093 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f86f405a100 con 0x7f86fc0b5530 2024-04-26T21:20:10.907 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:10.907 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:20:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 msgr2=0x7f86e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 0x7f86e006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f86f8034000 tx=0x7f86f802e000 comp rx=0 tx=0).stop 2024-04-26T21:20:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 msgr2=0x7f86fc1f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007e60 tx=0x7f86f400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 shutdown_connections 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86e006c8f0 0x7f86e006edb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86fc0b5530 0x7f86fc1f8390 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86fc09ba40 0x7f86fc1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 --2- 172.21.15.40:0/3884326093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86fc070570 0x7f86fc0b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 >> 172.21.15.40:0/3884326093 conn(0x7f86fc0b4a10 msgr2=0x7f86fc06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 shutdown_connections 2024-04-26T21:20:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:10.907+0000 7f8703566640 1 -- 172.21.15.40:0/3884326093 wait complete. 2024-04-26T21:20:10.920 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-26T21:20:10.920 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:20:12.292 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:20:14.295 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:20:15.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:20:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.032+0000 7fb03f389640 1 Processor -- start 2024-04-26T21:20:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.032+0000 7fb03f389640 1 -- start start 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03f389640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0380d5810 0x7fb0380b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03f389640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0380b2550 con 0x7fb0380d2e20 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03f389640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0380b2830 con 0x7fb03815f4d0 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03f389640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb0380b6180 con 0x7fb0380d5810 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03c924640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0380d5810 0x7fb0380b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03d926640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0380d2e20 0x7fb0380b35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54186/0 (socket says 172.21.15.40:54186) 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.033+0000 7fb03d926640 1 -- 172.21.15.40:0/3292645164 learned_addr learned my addr 172.21.15.40:0/3292645164 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.034+0000 7fb03c924640 1 -- 172.21.15.40:0/3292645164 >> v1:172.21.15.184:6789/0 conn(0x7fb03815f4d0 legacy=0x7fb0380d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.034+0000 7fb03c924640 1 -- 172.21.15.40:0/3292645164 >> v1:172.21.15.40:6789/0 conn(0x7fb0380d2e20 legacy=0x7fb0380b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.034+0000 7fb02bfff640 1 -- 172.21.15.40:0/3292645164 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718288163 0 0) 0x7fb0380b2550 con 0x7fb0380d2e20 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.034+0000 7fb03c924640 1 -- 172.21.15.40:0/3292645164 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0380b2b10 con 0x7fb0380d5810 2024-04-26T21:20:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03c924640 1 --2- 172.21.15.40:0/3292645164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb0380b1ee0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097a0 tx=0x7fb02c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec85965aba918a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb02bfff640 1 -- 172.21.15.40:0/3292645164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c037070 con 0x7fb0380d5810 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb02bfff640 1 -- 172.21.15.40:0/3292645164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02c032440 con 0x7fb0380d5810 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb02bfff640 1 -- 172.21.15.40:0/3292645164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c032810 con 0x7fb0380d5810 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 -- 172.21.15.40:0/3292645164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 msgr2=0x7fb0380b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 --2- 172.21.15.40:0/3292645164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb0380b1ee0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097a0 tx=0x7fb02c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 -- 172.21.15.40:0/3292645164 shutdown_connections 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 --2- 172.21.15.40:0/3292645164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb0380b1ee0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 -- 172.21.15.40:0/3292645164 >> 172.21.15.40:0/3292645164 conn(0x7fb03808ebf0 msgr2=0x7fb03809e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.035+0000 7fb03f389640 1 -- 172.21.15.40:0/3292645164 shutdown_connections 2024-04-26T21:20:16.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.036+0000 7fb03f389640 1 -- 172.21.15.40:0/3292645164 wait complete. 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 Processor -- start 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 -- start start 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0380d2e20 0x7fb03814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03815f4d0 0x7fb038162170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380b3d00 con 0x7fb0380d5810 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380b30d0 con 0x7fb0380d2e20 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.037+0000 7fb03f389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0380b3e80 con 0x7fb03815f4d0 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39356/0 (socket says 172.21.15.40:39356) 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 -- 172.21.15.40:0/944170206 learned_addr learned my addr 172.21.15.40:0/944170206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03d125640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0380d2e20 0x7fb03814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03d926640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03815f4d0 0x7fb038162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 -- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03815f4d0 msgr2=0x7fb038162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03815f4d0 0x7fb038162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 -- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0380d2e20 msgr2=0x7fb03814e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0380d2e20 0x7fb03814e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 -- 172.21.15.40:0/944170206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03815f950 con 0x7fb0380d5810 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03c924640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098c0 tx=0x7fb02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c037040 con 0x7fb0380d5810 2024-04-26T21:20:16.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02c042070 con 0x7fb0380d5810 2024-04-26T21:20:16.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03815fbf0 con 0x7fb0380d5810 2024-04-26T21:20:16.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.038+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c030a70 con 0x7fb0380d5810 2024-04-26T21:20:16.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.039+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0380cdc20 con 0x7fb0380d5810 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.040+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03806a290 con 0x7fb0380d5810 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.040+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb02c030c10 con 0x7fb0380d5810 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.041+0000 7fb029ffb640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.041+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fb02c0ac5c0 con 0x7fb0380d5810 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.041+0000 7fb03d125640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.041+0000 7fb03d125640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 0x7fb01c06edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb0381f8610 tx=0x7fb034009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.045+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb02c07a9e0 con 0x7fb0380d5810 2024-04-26T21:20:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.269+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb038075fb0 con 0x7fb0380d5810 2024-04-26T21:20:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.269+0000 7fb029ffb640 1 -- 172.21.15.40:0/944170206 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fb02c07a2e0 con 0x7fb0380d5810 2024-04-26T21:20:16.270 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:16.270 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:20:16.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 msgr2=0x7fb01c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 0x7fb01c06edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb0381f8610 tx=0x7fb034009040 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 msgr2=0x7fb038161c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098c0 tx=0x7fb02c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 shutdown_connections 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb01c06c8f0 0x7fb01c06edb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb03815f4d0 0x7fb038162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380d5810 0x7fb038161c30 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 --2- 172.21.15.40:0/944170206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0380d2e20 0x7fb03814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 >> 172.21.15.40:0/944170206 conn(0x7fb03808ebf0 msgr2=0x7fb0380d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 shutdown_connections 2024-04-26T21:20:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.271+0000 7fb03f389640 1 -- 172.21.15.40:0/944170206 wait complete. 2024-04-26T21:20:16.283 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:20:16.283 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-26T21:20:16.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:20:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:20:16.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.381+0000 7ff0070f1640 1 Processor -- start 2024-04-26T21:20:16.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 -- start start 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff00015f4d0 0x7ff0000b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7ff0000b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0000b2550 con 0x7ff0000d2e20 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff0000b6180 con 0x7ff00015f4d0 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7ff0070f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0000b2830 con 0x7ff0000d5810 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.382+0000 7feffffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff0000d2e20 0x7ff0000d5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54192/0 (socket says 172.21.15.40:54192) 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7feffffff640 1 -- 172.21.15.40:0/2231213640 learned_addr learned my addr 172.21.15.40:0/2231213640 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7ff00568e640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7ff0000b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2231213640 >> v2:172.21.15.184:3300/0 conn(0x7ff00015f4d0 0x7ff0000b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441571322 0 0) 0x7ff0000b2550 con 0x7ff0000d2e20 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feff0003520 con 0x7ff0000d2e20 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2393991088 0 0) 0x7feff0003520 con 0x7ff0000d2e20 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 msgr2=0x7ff0000b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7ff0000b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 >> v2:172.21.15.184:3300/0 conn(0x7ff00015f4d0 msgr2=0x7ff0000b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2231213640 >> v2:172.21.15.184:3300/0 conn(0x7ff00015f4d0 0x7ff0000b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0000b2b10 con 0x7ff0000d2e20 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fefec001670 con 0x7ff0000d2e20 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 >> v1:172.21.15.40:6789/0 conn(0x7ff0000d2e20 legacy=0x7ff0000d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff00039b0 0x7feff0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff0004800 0x7feff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7feff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000b6180 con 0x7feff00039b0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000b2830 con 0x7feff0004800 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feff000b9c0 con 0x7ff0000d5810 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff00039b0 0x7feff0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7feffffff640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff0004800 0x7feff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.383+0000 7ff00568e640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7feff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 -- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 msgr2=0x7feff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0000d5810 0x7feff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 -- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff0004800 msgr2=0x7feff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff0004800 0x7feff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 -- 172.21.15.40:0/2231213640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff0003520 con 0x7feff00039b0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff00039b0 0x7feff0003dd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feff4009af0 tx=0x7feff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256c99d1e41b54be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4036070 con 0x7feff00039b0 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff4031400 con 0x7feff00039b0 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.384+0000 7fefff7fe640 1 -- 172.21.15.40:0/2231213640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4009690 con 0x7feff00039b0 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 -- 172.21.15.40:0/2231213640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff00039b0 msgr2=0x7feff0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2231213640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff00039b0 0x7feff0003dd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feff4009af0 tx=0x7feff0005830 comp rx=0 tx=0).stop 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff00568e640 1 -- 172.21.15.40:0/2231213640 reap_dead start 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 -- 172.21.15.40:0/2231213640 shutdown_connections 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 -- 172.21.15.40:0/2231213640 >> 172.21.15.40:0/2231213640 conn(0x7ff00008ebf0 msgr2=0x7ff00009e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 -- 172.21.15.40:0/2231213640 shutdown_connections 2024-04-26T21:20:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.385+0000 7ff0070f1640 1 -- 172.21.15.40:0/2231213640 wait complete. 2024-04-26T21:20:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.386+0000 7ff0070f1640 1 Processor -- start 2024-04-26T21:20:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.386+0000 7ff0070f1640 1 -- start start 2024-04-26T21:20:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 0x7ff0001506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 0x7ff0000c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000b35f0 con 0x7ff0000cdec0 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000b30d0 con 0x7feff00039b0 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff0070f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff0000b3770 con 0x7ff00015f960 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39386/0 (socket says 172.21.15.40:39386) 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 -- 172.21.15.40:0/2204379804 learned_addr learned my addr 172.21.15.40:0/2204379804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7feffffff640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 0x7ff0000c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 0x7ff0001506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 -- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 msgr2=0x7ff0000c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 0x7ff0000c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 -- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 msgr2=0x7ff0001506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.387+0000 7ff00568e640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 0x7ff0001506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7ff00568e640 1 -- 172.21.15.40:0/2204379804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff000150be0 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 0x7ff0001506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7feffffff640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 0x7ff0000c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7ff00568e640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff0000d4d70 tx=0x7feff800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff800fde0 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff801d070 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0000c96f0 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8015670 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.388+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0000c9cb0 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.389+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00006a290 con 0x7ff0000cdec0 2024-04-26T21:20:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.389+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feff8015810 con 0x7ff0000cdec0 2024-04-26T21:20:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.390+0000 7feffd7fa640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 0x7fefe406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.390+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7feff808c4c0 con 0x7ff0000cdec0 2024-04-26T21:20:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.390+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 0x7fefe406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.391+0000 7ff004e8d640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 0x7fefe406ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feff4033000 tx=0x7feff4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.394+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff000150be0 con 0x7ff0000cdec0 2024-04-26T21:20:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.618+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0000d28f0 con 0x7ff0000cdec0 2024-04-26T21:20:16.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.619+0000 7feffd7fa640 1 -- 172.21.15.40:0/2204379804 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7feff8010040 con 0x7ff0000cdec0 2024-04-26T21:20:16.620 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:16.620 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 msgr2=0x7fefe406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 0x7fefe406ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feff4033000 tx=0x7feff4008030 comp rx=0 tx=0).stop 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 msgr2=0x7ff0000ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff0000d4d70 tx=0x7feff800f580 comp rx=0 tx=0).stop 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 shutdown_connections 2024-04-26T21:20:16.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fefe406c920 0x7fefe406ede0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0000cdec0 0x7ff0000ce370 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff00015f960 0x7ff0000c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 --2- 172.21.15.40:0/2204379804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff00039b0 0x7ff0001506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 >> 172.21.15.40:0/2204379804 conn(0x7ff00008ebf0 msgr2=0x7ff0000d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 shutdown_connections 2024-04-26T21:20:16.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16.621+0000 7ff0070f1640 1 -- 172.21.15.40:0/2204379804 wait complete. 2024-04-26T21:20:16.632 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:20:16.632 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-26T21:20:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:16 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-26T21:20:18.297 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:20:18.297 INFO:teuthology.orchestra.run.smithi040.stderr:stdin closed, goodbye! 2024-04-26T21:20:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:18.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.464+0000 7f2d066ea640 1 Processor -- start 2024-04-26T21:20:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.464+0000 7f2d066ea640 1 -- start start 2024-04-26T21:20:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d066ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d0015f340 0x7f2d000b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d066ea640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d000d5610 0x7f2d000d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d066ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000b2510 con 0x7f2d0015f720 2024-04-26T21:20:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d066ea640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d000b27f0 con 0x7f2d000d5610 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d066ea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f2d000b2ad0 con 0x7f2d0015f340 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d000d5610 0x7f2d000d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d000d5610 0x7f2d000d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39390/0 (socket says 172.21.15.40:39390) 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 -- 172.21.15.40:0/927614454 learned_addr learned my addr 172.21.15.40:0/927614454 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 -- 172.21.15.40:0/927614454 >> v2:172.21.15.184:3300/0 conn(0x7f2d0015f340 msgr2=0x7f2d000b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 --2- 172.21.15.40:0/927614454 >> v2:172.21.15.184:3300/0 conn(0x7f2d0015f340 0x7f2d000b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 -- 172.21.15.40:0/927614454 >> v1:172.21.15.184:6789/0 conn(0x7f2d0015f720 legacy=0x7f2d000d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.465+0000 7f2d04c87640 1 -- 172.21.15.40:0/927614454 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000b2db0 con 0x7f2d000d5610 2024-04-26T21:20:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.466+0000 7f2d04c87640 1 --2- 172.21.15.40:0/927614454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d000d3d40 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2d000d6000 tx=0x7f2cf802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26f6c83d8bfcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.466+0000 7f2cfeffd640 1 -- 172.21.15.40:0/927614454 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8036070 con 0x7f2d000d5610 2024-04-26T21:20:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.466+0000 7f2cfeffd640 1 -- 172.21.15.40:0/927614454 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8031600 con 0x7f2d000d5610 2024-04-26T21:20:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.466+0000 7f2cfeffd640 1 -- 172.21.15.40:0/927614454 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80319d0 con 0x7f2d000d5610 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 -- 172.21.15.40:0/927614454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 msgr2=0x7f2d000d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 --2- 172.21.15.40:0/927614454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d000d3d40 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2d000d6000 tx=0x7f2cf802a660 comp rx=0 tx=0).stop 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 -- 172.21.15.40:0/927614454 shutdown_connections 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 --2- 172.21.15.40:0/927614454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d000d3d40 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 --2- 172.21.15.40:0/927614454 >> v2:172.21.15.184:3300/0 conn(0x7f2d0015f340 0x7f2d000b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 -- 172.21.15.40:0/927614454 >> 172.21.15.40:0/927614454 conn(0x7f2d0008ec70 msgr2=0x7f2d0009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 -- 172.21.15.40:0/927614454 shutdown_connections 2024-04-26T21:20:18.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.467+0000 7f2d066ea640 1 -- 172.21.15.40:0/927614454 wait complete. 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.468+0000 7f2d066ea640 1 Processor -- start 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.468+0000 7f2d066ea640 1 -- start start 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 0x7f2d001f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0015f720 0x7f2d000cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000b3940 con 0x7f2d000d5610 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000b37c0 con 0x7f2d0015f720 2024-04-26T21:20:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2d066ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2d000b3ac0 con 0x7f2d0015f340 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39394/0 (socket says 172.21.15.40:39394) 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2cfffff640 1 -- 172.21.15.40:0/4029171494 learned_addr learned my addr 172.21.15.40:0/4029171494 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.469+0000 7f2cfffff640 1 -- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 msgr2=0x7f2d001f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2d04c87640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0015f720 0x7f2d000cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 0x7f2d001f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfffff640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 0x7f2d001f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfffff640 1 -- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0015f720 msgr2=0x7f2d000cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfffff640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0015f720 0x7f2d000cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfffff640 1 -- 172.21.15.40:0/4029171494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000cba70 con 0x7f2d000d5610 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 0x7f2d001f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfffff640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b4a0 tx=0x7f2cf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0014070 con 0x7f2d000d5610 2024-04-26T21:20:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf0010610 con 0x7f2d000d5610 2024-04-26T21:20:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d000c8d80 con 0x7f2d000d5610 2024-04-26T21:20:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0010a00 con 0x7f2d000d5610 2024-04-26T21:20:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.470+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d000c9340 con 0x7f2d000d5610 2024-04-26T21:20:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.471+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0006a290 con 0x7f2d000d5610 2024-04-26T21:20:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.472+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf0017070 con 0x7f2d000d5610 2024-04-26T21:20:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.472+0000 7f2cfcff9640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.472+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf008b670 con 0x7f2d000d5610 2024-04-26T21:20:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.472+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.473+0000 7f2cff7fe640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2d000cd850 tx=0x7f2cf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.476+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cf0059a20 con 0x7f2d000d5610 2024-04-26T21:20:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.698+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d001f6ef0 con 0x7f2d000d5610 2024-04-26T21:20:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.699+0000 7f2cfcff9640 1 -- 172.21.15.40:0/4029171494 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5189 (secure 0 0 0) 0x7f2cf0059320 con 0x7f2d000d5610 2024-04-26T21:20:18.700 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:15.759931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 78 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 msgr2=0x7f2cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2d000cd850 tx=0x7f2cf4008040 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 msgr2=0x7f2d001599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b4a0 tx=0x7f2cf000b960 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 shutdown_connections 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d0015f720 0x7f2d000cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d0015f340 0x7f2d001f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 --2- 172.21.15.40:0/4029171494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d000d5610 0x7f2d001599e0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 >> 172.21.15.40:0/4029171494 conn(0x7f2d0008ec70 msgr2=0x7f2d000b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 shutdown_connections 2024-04-26T21:20:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:18.701+0000 7f2d066ea640 1 -- 172.21.15.40:0/4029171494 wait complete. 2024-04-26T21:20:18.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-26T21:20:18.835 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":7405,"entity":{"name":{"type":"client","num":7405},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2719632244}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":11.35386027,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9343888356333618,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7405 v1:192.168.0.1:0/2719632244","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":8640,"entity":{"name":{"type":"client","num":8640},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1916941838}},"state":"open","num_leases":1,"num_caps":102,"request_load_avg":205,"uptime":12.49384624,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":758.96836523981494,"halflife":60},"release_caps":{"value":99.933019039658177,"halflife":60},"recall_caps_throttle":{"value":256.01001625280367,"halflife":1.5},"recall_caps_throttle2o":{"value":118.96266132649195,"halflife":0.5},"session_cache_liveness":{"value":397.01500643752615,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8640 v1:192.168.0.1:0/1916941838","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:20:18.837 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2719632244, "type": "v1" }, "name": { "num": 7405, "type": "client" } }, "id": 7405, "inst": "client.7405 v1:192.168.0.1:0/2719632244", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.934388835633362 }, "state": "open", "uptime": 11.35386027 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1916941838, "type": "v1" }, "name": { "num": 8640, "type": "client" } }, "id": 8640, "inst": "client.8640 v1:192.168.0.1:0/1916941838", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 102, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 758.968365239815 }, "recall_caps_throttle": { "halflife": 1.5, "value": 256.01001625280367 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 118.96266132649195 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 99.93301903965818 }, "request_load_avg": 205, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 397.01500643752615 }, "state": "open", "uptime": 12.49384624 } ] 2024-04-26T21:20:18.837 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:20:18.837 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:20:18.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:20:18.876 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:20:18.920 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:20:18.921 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:20:18.921 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 200\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi040') 2024-04-26T21:20:18.921 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:18.921 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:18.937 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:18.937 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:18.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:19.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.132+0000 7f794fd96640 1 Processor -- start 2024-04-26T21:20:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.133+0000 7f794fd96640 1 -- start start 2024-04-26T21:20:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.133+0000 7f794fd96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79480b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.133+0000 7f794fd96640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79480b2510 con 0x7f794815f340 2024-04-26T21:20:19.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.133+0000 7f794fd96640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79480b27f0 con 0x7f79480d5610 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.133+0000 7f794fd96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79480b2ad0 con 0x7f794815f720 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794db32640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f794815f340 0x7f79480d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54196/0 (socket says 172.21.15.40:54196) 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794db32640 1 -- 172.21.15.40:0/1990688750 learned_addr learned my addr 172.21.15.40:0/1990688750 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794d331640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79480b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242679475 0 0) 0x7f79480b2510 con 0x7f794815f340 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7930003520 con 0x7f794815f340 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3772694695 0 0) 0x7f7930003520 con 0x7f794815f340 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 msgr2=0x7f79480b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79480b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 >> v1:172.21.15.184:6789/0 conn(0x7f79480d5610 legacy=0x7f79480b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79480b2db0 con 0x7f794815f340 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f79380019f0 con 0x7f794815f340 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 >> v1:172.21.15.40:6789/0 conn(0x7f794815f340 legacy=0x7f79480d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7930003f20 0x7f7930004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930004d40 0x7f79300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79480b2ad0 con 0x7f7930004d40 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f793000bad0 con 0x7f7930003f20 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f793000bcd0 con 0x7f794815f720 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794e333640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930004d40 0x7f79300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794d331640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7930003f20 0x7f7930004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.134+0000 7f794db32640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 -- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 msgr2=0x7f79300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815f720 0x7f79300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 -- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7930003f20 msgr2=0x7f7930004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7930003f20 0x7f7930004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 -- 172.21.15.40:0/1990688750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7930003520 con 0x7f7930004d40 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794e333640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930004d40 0x7f79300051c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7940001860 tx=0x7f7930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5cd1c4b02e6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940037070 con 0x7f7930004d40 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794002abf0 con 0x7f7930004d40 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.135+0000 7f794cb30640 1 -- 172.21.15.40:0/1990688750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940001430 con 0x7f7930004d40 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 -- 172.21.15.40:0/1990688750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930004d40 msgr2=0x7f79300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 --2- 172.21.15.40:0/1990688750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930004d40 0x7f79300051c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7940001860 tx=0x7f7930005830 comp rx=0 tx=0).stop 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794e333640 1 -- 172.21.15.40:0/1990688750 reap_dead start 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 -- 172.21.15.40:0/1990688750 shutdown_connections 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 -- 172.21.15.40:0/1990688750 >> 172.21.15.40:0/1990688750 conn(0x7f794808ec70 msgr2=0x7f794809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 -- 172.21.15.40:0/1990688750 shutdown_connections 2024-04-26T21:20:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.136+0000 7f794fd96640 1 -- 172.21.15.40:0/1990688750 wait complete. 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.137+0000 7f794fd96640 1 Processor -- start 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.137+0000 7f794fd96640 1 -- start start 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815b620 0x7f79480cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 0x7f79481f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79480b2210 con 0x7f79481f89f0 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79480b2090 con 0x7f794815ba00 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794fd96640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f79480d3d40 con 0x7f794815b620 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39418/0 (socket says 172.21.15.40:39418) 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 -- 172.21.15.40:0/2243134070 learned_addr learned my addr 172.21.15.40:0/2243134070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794db32640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815b620 0x7f79480cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794d331640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 0x7f79481f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 -- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815b620 msgr2=0x7f79480cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815b620 0x7f79480cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 -- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 msgr2=0x7f79481f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 0x7f79481f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794e333640 1 -- 172.21.15.40:0/2243134070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7948160750 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.138+0000 7f794d331640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 0x7f79481f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f794e333640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f794002a6c0 tx=0x7f7940001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940037040 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7940040070 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7948161630 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940001430 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.139+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7948160230 con 0x7f79481f89f0 2024-04-26T21:20:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.140+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7948160430 con 0x7f79481f89f0 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.140+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794003a070 con 0x7f79481f89f0 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.141+0000 7f79467fc640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 0x7f793406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.141+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f79400ab2c0 con 0x7f79481f89f0 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.141+0000 7f794db32640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 0x7f793406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.141+0000 7f794db32640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 0x7f793406ede0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7938000f80 tx=0x7f7938006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.145+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7948160430 con 0x7f79481f89f0 2024-04-26T21:20:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.330+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f794806a290 con 0x7f79481f89f0 2024-04-26T21:20:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.332+0000 7f79467fc640 1 -- 172.21.15.40:0/2243134070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7f7940079650 con 0x7f79481f89f0 2024-04-26T21:20:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:19.333 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 msgr2=0x7f793406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 0x7f793406ede0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7938000f80 tx=0x7f7938006040 comp rx=0 tx=0).stop 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 msgr2=0x7f79481f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f794002a6c0 tx=0x7f7940001230 comp rx=0 tx=0).stop 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 shutdown_connections 2024-04-26T21:20:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793406c920 0x7f793406ede0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79481f89f0 0x7f79481f8e70 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794815ba00 0x7f79481f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 --2- 172.21.15.40:0/2243134070 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794815b620 0x7f79480cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 >> 172.21.15.40:0/2243134070 conn(0x7f794808ec70 msgr2=0x7f794806e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 shutdown_connections 2024-04-26T21:20:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.334+0000 7f794fd96640 1 -- 172.21.15.40:0/2243134070 wait complete. 2024-04-26T21:20:19.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:57.355243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:58.032686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:20:19.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:20:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.450+0000 7fb8c5425640 1 Processor -- start 2024-04-26T21:20:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.450+0000 7fb8c5425640 1 -- start start 2024-04-26T21:20:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8c5425640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8c00b2420 0x7fb8c009c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8c5425640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8c00d1940 0x7fb8c00702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8c5425640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c00cee60 con 0x7fb8c015d340 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8c5425640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c00ce9d0 con 0x7fb8c00d1940 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8c5425640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c00d0ff0 con 0x7fb8c00b2420 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8c00d1940 0x7fb8c00702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8beffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8c015d340 0x7fb8c00d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54206/0 (socket says 172.21.15.40:54206) 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.451+0000 7fb8beffd640 1 -- 172.21.15.40:0/3484121158 learned_addr learned my addr 172.21.15.40:0/3484121158 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8be7fc640 1 --2- 172.21.15.40:0/3484121158 >> v2:172.21.15.184:3300/0 conn(0x7fb8c00b2420 0x7fb8c009c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bf7fe640 1 -- 172.21.15.40:0/3484121158 >> v2:172.21.15.184:3300/0 conn(0x7fb8c00b2420 msgr2=0x7fb8c009c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bf7fe640 1 --2- 172.21.15.40:0/3484121158 >> v2:172.21.15.184:3300/0 conn(0x7fb8c00b2420 0x7fb8c009c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bf7fe640 1 -- 172.21.15.40:0/3484121158 >> v1:172.21.15.40:6789/0 conn(0x7fb8c015d340 legacy=0x7fb8c00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bf7fe640 1 -- 172.21.15.40:0/3484121158 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c00d0d10 con 0x7fb8c00d1940 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bdffb640 1 -- 172.21.15.40:0/3484121158 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358854909 0 0) 0x7fb8c00cee60 con 0x7fb8c015d340 2024-04-26T21:20:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bf7fe640 1 --2- 172.21.15.40:0/3484121158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00702a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00d0aa0 tx=0x7fb8b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1800a85a465753ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bdffb640 1 -- 172.21.15.40:0/3484121158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007740 con 0x7fb8c00d1940 2024-04-26T21:20:19.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.452+0000 7fb8bdffb640 1 -- 172.21.15.40:0/3484121158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b00078e0 con 0x7fb8c00d1940 2024-04-26T21:20:19.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8bdffb640 1 -- 172.21.15.40:0/3484121158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007cb0 con 0x7fb8c00d1940 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 -- 172.21.15.40:0/3484121158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 msgr2=0x7fb8c00702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 --2- 172.21.15.40:0/3484121158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00702a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00d0aa0 tx=0x7fb8b002a680 comp rx=0 tx=0).stop 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 -- 172.21.15.40:0/3484121158 shutdown_connections 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 --2- 172.21.15.40:0/3484121158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00702a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 --2- 172.21.15.40:0/3484121158 >> v2:172.21.15.184:3300/0 conn(0x7fb8c00b2420 0x7fb8c009c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 -- 172.21.15.40:0/3484121158 >> 172.21.15.40:0/3484121158 conn(0x7fb8c009ba40 msgr2=0x7fb8c009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 -- 172.21.15.40:0/3484121158 shutdown_connections 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.453+0000 7fb8c5425640 1 -- 172.21.15.40:0/3484121158 wait complete. 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.454+0000 7fb8c5425640 1 Processor -- start 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.454+0000 7fb8c5425640 1 -- start start 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 0x7fb8c006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 0x7fb8c0155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c00b2090 con 0x7fb8c00d1940 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c015d720 con 0x7fb8c015d340 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8c5425640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c00b2210 con 0x7fb8c00cb0c0 2024-04-26T21:20:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8beffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39436/0 (socket says 172.21.15.40:39436) 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8beffd640 1 -- 172.21.15.40:0/4059401876 learned_addr learned my addr 172.21.15.40:0/4059401876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8be7fc640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 0x7fb8c006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8bf7fe640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 0x7fb8c0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8beffd640 1 -- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 msgr2=0x7fb8c0155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.455+0000 7fb8beffd640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 0x7fb8c0155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8beffd640 1 -- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 msgr2=0x7fb8c006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8beffd640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 0x7fb8c006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8beffd640 1 -- 172.21.15.40:0/4059401876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c006f3c0 con 0x7fb8c00d1940 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8bf7fe640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 0x7fb8c0155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8be7fc640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 0x7fb8c006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8beffd640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c430 tx=0x7fb8b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80074e0 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b80047e0 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c006e0c0 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8004bd0 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.456+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c006e680 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.457+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c00693d0 con 0x7fb8c00d1940 2024-04-26T21:20:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.457+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8b8011070 con 0x7fb8c00d1940 2024-04-26T21:20:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.458+0000 7fb8b77fe640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.458+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b808b6b0 con 0x7fb8c00d1940 2024-04-26T21:20:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.458+0000 7fb8be7fc640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.458+0000 7fb8be7fc640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00c98f0 tx=0x7fb8a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.462+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8b8059b70 con 0x7fb8c00d1940 2024-04-26T21:20:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.646+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8c00b8e60 con 0x7fb8c00d1940 2024-04-26T21:20:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.646+0000 7fb8b77fe640 1 -- 172.21.15.40:0/4059401876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7fb8b8059470 con 0x7fb8c00d1940 2024-04-26T21:20:19.647 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:20:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 msgr2=0x7fb8a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00c98f0 tx=0x7fb8a8008040 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 msgr2=0x7fb8c00cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c430 tx=0x7fb8b800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 shutdown_connections 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8c00cb0c0 0x7fb8c0155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8c015d340 0x7fb8c006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 --2- 172.21.15.40:0/4059401876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c00d1940 0x7fb8c00cba80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 >> 172.21.15.40:0/4059401876 conn(0x7fb8c009ba40 msgr2=0x7fb8c00c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 shutdown_connections 2024-04-26T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.648+0000 7fb8c5425640 1 -- 172.21.15.40:0/4059401876 wait complete. 2024-04-26T21:20:19.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.764+0000 7f9ff0f11640 1 Processor -- start 2024-04-26T21:20:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.764+0000 7f9ff0f11640 1 -- start start 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9ff0f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fec0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9ff0f11640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec0d4240 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9ff0f11640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec0d31a0 con 0x7f9fec0d22f0 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9ff0f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3480 con 0x7f9fec15f490 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9fec0d0ff0 0x7f9fec0d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54214/0 (socket says 172.21.15.40:54214) 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe3fff640 1 -- 172.21.15.40:0/130235327 learned_addr learned my addr 172.21.15.40:0/130235327 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221481684 0 0) 0x7f9fec0d4240 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fea59c640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fec0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fd8003520 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 277468835 0 0) 0x7f9fd8003520 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 msgr2=0x7f9fec0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fec0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 >> v1:172.21.15.184:6789/0 conn(0x7f9fec0d22f0 legacy=0x7f9fec0ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0d3760 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9fd4001670 con 0x7f9fec0d0ff0 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 >> v1:172.21.15.40:6789/0 conn(0x7f9fec0d0ff0 legacy=0x7f9fec0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.765+0000 7f9fe9d9b640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3480 con 0x7f9fd80039e0 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd800ba70 con 0x7f9fd8004d40 2024-04-26T21:20:19.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd800bc70 con 0x7f9fec15f490 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fe3fff640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fead9d640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 -- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 msgr2=0x7f9fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec15f490 0x7f9fd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 -- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd8004d40 msgr2=0x7f9fd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.766+0000 7f9fea59c640 1 -- 172.21.15.40:0/130235327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd8003520 con 0x7f9fd80039e0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9fea59c640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fd8003e00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009a90 tx=0x7f9fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeb657344f8d2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc036070 con 0x7f9fd80039e0 2024-04-26T21:20:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc031400 con 0x7f9fd80039e0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9fe9d9b640 1 -- 172.21.15.40:0/130235327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc009690 con 0x7f9fd80039e0 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 -- 172.21.15.40:0/130235327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 msgr2=0x7f9fd8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/130235327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fd8003e00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009a90 tx=0x7f9fd8005830 comp rx=0 tx=0).stop 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9fead9d640 1 -- 172.21.15.40:0/130235327 reap_dead start 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 -- 172.21.15.40:0/130235327 shutdown_connections 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 -- 172.21.15.40:0/130235327 >> 172.21.15.40:0/130235327 conn(0x7f9fec09e530 msgr2=0x7f9fec0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 -- 172.21.15.40:0/130235327 shutdown_connections 2024-04-26T21:20:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.767+0000 7f9ff0f11640 1 -- 172.21.15.40:0/130235327 wait complete. 2024-04-26T21:20:19.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 Processor -- start 2024-04-26T21:20:19.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 -- start start 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f8690 0x7f9fec15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 0x7f9fec15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec15f870 con 0x7f9fd80039e0 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3d20 con 0x7f9fec15c360 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.769+0000 7f9ff0f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0d2dc0 con 0x7f9fec1f8690 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39442/0 (socket says 172.21.15.40:39442) 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 -- 172.21.15.40:0/1519988640 learned_addr learned my addr 172.21.15.40:0/1519988640 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fe3fff640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f8690 0x7f9fec15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fead9d640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 0x7f9fec15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 -- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f8690 msgr2=0x7f9fec15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f8690 0x7f9fec15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 -- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 msgr2=0x7f9fec15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 0x7f9fec15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 -- 172.21.15.40:0/1519988640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec1f8e30 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fead9d640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 0x7f9fec15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fea59c640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009a90 tx=0x7f9fdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc040020 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc003bc0 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec1f9110 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc036040 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.770+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec0b9660 con 0x7f9fd80039e0 2024-04-26T21:20:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.771+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec06a290 con 0x7f9fd80039e0 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.772+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fdc043020 con 0x7f9fd80039e0 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.772+0000 7f9fe2ffd640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 0x7f9fcc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.772+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f9fdc0aba30 con 0x7f9fd80039e0 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.772+0000 7f9fe3fff640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 0x7f9fcc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.773+0000 7f9fe3fff640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 0x7f9fcc06ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b9380 tx=0x7f9fd4001170 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.776+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9fdc04a070 con 0x7f9fd80039e0 2024-04-26T21:20:19.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.962+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9fec0d5900 con 0x7f9fd80039e0 2024-04-26T21:20:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.962+0000 7f9fe2ffd640 1 -- 172.21.15.40:0/1519988640 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7f9fdc0044e0 con 0x7f9fd80039e0 2024-04-26T21:20:19.963 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:20:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 msgr2=0x7f9fcc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 0x7f9fcc06ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b9380 tx=0x7f9fd4001170 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 msgr2=0x7f9fec0ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009a90 tx=0x7f9fdc004730 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 shutdown_connections 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9fcc06c920 0x7f9fcc06ede0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec15c360 0x7f9fec15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f8690 0x7f9fec15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 --2- 172.21.15.40:0/1519988640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80039e0 0x7f9fec0ca0f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 >> 172.21.15.40:0/1519988640 conn(0x7f9fec09e530 msgr2=0x7f9fec06e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 shutdown_connections 2024-04-26T21:20:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:19.964+0000 7f9ff0f11640 1 -- 172.21.15.40:0/1519988640 wait complete. 2024-04-26T21:20:19.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:20:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.078+0000 7f3d7be52640 1 Processor -- start 2024-04-26T21:20:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.078+0000 7f3d7be52640 1 -- start start 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7be52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7406f340 0x7f3d740b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7be52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d740b3e50 0x7f3d740720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7be52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d740b46a0 con 0x7f3d740d0300 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7be52640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3d740b5db0 con 0x7f3d740b3e50 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7be52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d740b6090 con 0x7f3d7406f340 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d7a3ef640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d740b3e50 0x7f3d740720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d793ed640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d740d0300 0x7f3d7406f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54216/0 (socket says 172.21.15.40:54216) 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.079+0000 7f3d793ed640 1 -- 172.21.15.40:0/2211198299 learned_addr learned my addr 172.21.15.40:0/2211198299 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d79bee640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7406f340 0x7f3d740b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/2211198299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7406f340 msgr2=0x7f3d740b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d7a3ef640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7406f340 0x7f3d740b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/2211198299 >> v1:172.21.15.40:6789/0 conn(0x7f3d740d0300 legacy=0x7f3d7406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d78bec640 1 -- 172.21.15.40:0/2211198299 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373970454 0 0) 0x7f3d740b46a0 con 0x7f3d740d0300 2024-04-26T21:20:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/2211198299 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d740b6370 con 0x7f3d740b3e50 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.080+0000 7f3d7a3ef640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740b3e50 0x7f3d740720f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009dc0 tx=0x7f3d6402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca4e7912a6683b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d78bec640 1 -- 172.21.15.40:0/2211198299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64007740 con 0x7f3d740b3e50 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d78bec640 1 -- 172.21.15.40:0/2211198299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d640078e0 con 0x7f3d740b3e50 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d78bec640 1 -- 172.21.15.40:0/2211198299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64007cb0 con 0x7f3d740b3e50 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 -- 172.21.15.40:0/2211198299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740b3e50 msgr2=0x7f3d740720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740b3e50 0x7f3d740720f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009dc0 tx=0x7f3d6402a560 comp rx=0 tx=0).stop 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 -- 172.21.15.40:0/2211198299 shutdown_connections 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740b3e50 0x7f3d740720f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 --2- 172.21.15.40:0/2211198299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d7406f340 0x7f3d740b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 -- 172.21.15.40:0/2211198299 >> 172.21.15.40:0/2211198299 conn(0x7f3d740cd8a0 msgr2=0x7f3d740cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 -- 172.21.15.40:0/2211198299 shutdown_connections 2024-04-26T21:20:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.081+0000 7f3d7be52640 1 -- 172.21.15.40:0/2211198299 wait complete. 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.082+0000 7f3d7be52640 1 Processor -- start 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.082+0000 7f3d7be52640 1 -- start start 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7406f340 0x7f3d74158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 0x7f3d740cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740b4460 con 0x7f3d740d0300 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740b42e0 con 0x7f3d7406f340 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7be52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d74142a30 con 0x7f3d740b3e50 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7a3ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7a3ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39474/0 (socket says 172.21.15.40:39474) 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/3381277232 learned_addr learned my addr 172.21.15.40:0/3381277232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:20.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.083+0000 7f3d793ed640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 0x7f3d740cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d79bee640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7406f340 0x7f3d74158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 msgr2=0x7f3d740cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 0x7f3d740cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7406f340 msgr2=0x7f3d74158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7406f340 0x7f3d74158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 -- 172.21.15.40:0/3381277232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7415c880 con 0x7f3d740d0300 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d793ed640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 0x7f3d740cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7a3ef640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009ee0 tx=0x7f3d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6403f070 con 0x7f3d740d0300 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d64031c80 con 0x7f3d740d0300 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64044680 con 0x7f3d740d0300 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7415cb60 con 0x7f3d740d0300 2024-04-26T21:20:20.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.084+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7415d120 con 0x7f3d740d0300 2024-04-26T21:20:20.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.085+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7406a290 con 0x7f3d740d0300 2024-04-26T21:20:20.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.085+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d6403b410 con 0x7f3d740d0300 2024-04-26T21:20:20.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.086+0000 7f3d727fc640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 0x7f3d6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.086+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f3d640ad3c0 con 0x7f3d740d0300 2024-04-26T21:20:20.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.086+0000 7f3d79bee640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 0x7f3d6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.087+0000 7f3d79bee640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 0x7f3d6006edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.090+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d6407b230 con 0x7f3d740d0300 2024-04-26T21:20:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.275+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d740738d0 con 0x7f3d740d0300 2024-04-26T21:20:20.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.275+0000 7f3d727fc640 1 -- 172.21.15.40:0/3381277232 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7f3d6407f530 con 0x7f3d740d0300 2024-04-26T21:20:20.391 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:20:20.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 msgr2=0x7f3d6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 0x7f3d6006edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).stop 2024-04-26T21:20:20.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 msgr2=0x7f3d740cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009ee0 tx=0x7f3d64004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 shutdown_connections 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d6006c8f0 0x7f3d6006edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740d0300 0x7f3d740cb090 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d740b3e50 0x7f3d740cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 --2- 172.21.15.40:0/3381277232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d7406f340 0x7f3d74158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.277+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 >> 172.21.15.40:0/3381277232 conn(0x7f3d740cd8a0 msgr2=0x7f3d7406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.278+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 shutdown_connections 2024-04-26T21:20:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.278+0000 7f3d7be52640 1 -- 172.21.15.40:0/3381277232 wait complete. 2024-04-26T21:20:20.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:20:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.495+0000 7f3d96cfc640 1 Processor -- start 2024-04-26T21:20:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.495+0000 7f3d96cfc640 1 -- start start 2024-04-26T21:20:20.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d96cfc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 0x7f3d900b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d96cfc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d900d5610 0x7f3d900d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d96cfc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d900b2510 con 0x7f3d9015f720 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d96cfc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3d900b27f0 con 0x7f3d900d5610 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d96cfc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d900b2ad0 con 0x7f3d9015f340 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d8ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d9015f720 0x7f3d900d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54222/0 (socket says 172.21.15.40:54222) 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d8ffff640 1 -- 172.21.15.40:0/791199138 learned_addr learned my addr 172.21.15.40:0/791199138 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d94a98640 1 --2- 172.21.15.40:0/791199138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 0x7f3d900b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d95299640 1 --2- 172.21.15.40:0/791199138 >> v2:172.21.15.184:3300/0 conn(0x7f3d900d5610 0x7f3d900d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d8f7fe640 1 -- 172.21.15.40:0/791199138 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166177756 0 0) 0x7f3d900b2510 con 0x7f3d9015f720 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.496+0000 7f3d8f7fe640 1 -- 172.21.15.40:0/791199138 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d78003520 con 0x7f3d9015f720 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.497+0000 7f3d94a98640 1 -- 172.21.15.40:0/791199138 >> v2:172.21.15.184:3300/0 conn(0x7f3d900d5610 msgr2=0x7f3d900d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:20.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.497+0000 7f3d94a98640 1 --2- 172.21.15.40:0/791199138 >> v2:172.21.15.184:3300/0 conn(0x7f3d900d5610 0x7f3d900d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.497+0000 7f3d94a98640 1 -- 172.21.15.40:0/791199138 >> v1:172.21.15.40:6789/0 conn(0x7f3d9015f720 legacy=0x7f3d900d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.497+0000 7f3d94a98640 1 -- 172.21.15.40:0/791199138 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900b2db0 con 0x7f3d9015f340 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.497+0000 7f3d94a98640 1 --2- 172.21.15.40:0/791199138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 0x7f3d900b1e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d800097a0 tx=0x7f3d8002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bd0f743f901d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d8f7fe640 1 -- 172.21.15.40:0/791199138 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80037070 con 0x7f3d9015f340 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d8f7fe640 1 -- 172.21.15.40:0/791199138 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8002aaf0 con 0x7f3d9015f340 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d8f7fe640 1 -- 172.21.15.40:0/791199138 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84001670 con 0x7f3d9015f340 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 -- 172.21.15.40:0/791199138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 msgr2=0x7f3d900b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/791199138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 0x7f3d900b1e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d800097a0 tx=0x7f3d8002a060 comp rx=0 tx=0).stop 2024-04-26T21:20:20.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 -- 172.21.15.40:0/791199138 shutdown_connections 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/791199138 >> v2:172.21.15.184:3300/0 conn(0x7f3d900d5610 0x7f3d900d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/791199138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f340 0x7f3d900b1e80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 -- 172.21.15.40:0/791199138 >> 172.21.15.40:0/791199138 conn(0x7f3d9008ec70 msgr2=0x7f3d9009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 -- 172.21.15.40:0/791199138 shutdown_connections 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.498+0000 7f3d96cfc640 1 -- 172.21.15.40:0/791199138 wait complete. 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.499+0000 7f3d96cfc640 1 Processor -- start 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.499+0000 7f3d96cfc640 1 -- start start 2024-04-26T21:20:20.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d9015f340 0x7f3d900cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f720 0x7f3d900d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d900b3980 con 0x7f3d900d5610 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d900b3800 con 0x7f3d9015f340 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d96cfc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d900b3b00 con 0x7f3d9015f720 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d94a98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d94a98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39484/0 (socket says 172.21.15.40:39484) 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.500+0000 7f3d94a98640 1 -- 172.21.15.40:0/2169920670 learned_addr learned my addr 172.21.15.40:0/2169920670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 -- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f720 msgr2=0x7f3d900d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d95299640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f720 0x7f3d900d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d8ffff640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d9015f340 0x7f3d900cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f720 0x7f3d900d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 -- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d9015f340 msgr2=0x7f3d900cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d9015f340 0x7f3d900cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 -- 172.21.15.40:0/2169920670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900b92c0 con 0x7f3d900d5610 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d94a98640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f3d8002a0a0 tx=0x7f3d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80041070 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8003c460 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d900b95a0 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80037040 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.501+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d900b9b60 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.502+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9006a290 con 0x7f3d900d5610 2024-04-26T21:20:20.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.503+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d80044070 con 0x7f3d900d5610 2024-04-26T21:20:20.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.503+0000 7f3d8d7fa640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 0x7f3d7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.503+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f3d800abf80 con 0x7f3d900d5610 2024-04-26T21:20:20.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.503+0000 7f3d8ffff640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 0x7f3d7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.504+0000 7f3d8ffff640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 0x7f3d7c06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3d840011e0 tx=0x7f3d84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.507+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d8007a3b0 con 0x7f3d900d5610 2024-04-26T21:20:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.703+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3d900baf90 con 0x7f3d900d5610 2024-04-26T21:20:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.703+0000 7f3d8d7fa640 1 -- 172.21.15.40:0/2169920670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7f3d80079cb0 con 0x7f3d900d5610 2024-04-26T21:20:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:20:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 msgr2=0x7f3d7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 0x7f3d7c06edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3d840011e0 tx=0x7f3d84009040 comp rx=0 tx=0).stop 2024-04-26T21:20:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 msgr2=0x7f3d900cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f3d8002a0a0 tx=0x7f3d800046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 shutdown_connections 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d7c06c8f0 0x7f3d7c06edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d9015f720 0x7f3d900d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d9015f340 0x7f3d900cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 --2- 172.21.15.40:0/2169920670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d900d5610 0x7f3d900cd5b0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 >> 172.21.15.40:0/2169920670 conn(0x7f3d9008ec70 msgr2=0x7f3d900d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 shutdown_connections 2024-04-26T21:20:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.705+0000 7f3d96cfc640 1 -- 172.21.15.40:0/2169920670 wait complete. 2024-04-26T21:20:20.717 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:20:20.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:20.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:20.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:20:20.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:20.798 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:20:20.813 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:20:20.813 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:20.813 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:20.830 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:20.830 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:20.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.987+0000 7f050d6cc640 1 Processor -- start 2024-04-26T21:20:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.987+0000 7f050d6cc640 1 -- start start 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 0x7f05080b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f050815f720 0x7f05080d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 0x7f05080d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f05080b2510 con 0x7f050815f720 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f05080b27f0 con 0x7f05080d4f00 2024-04-26T21:20:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f050d6cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05080b2ad0 con 0x7f050815f340 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f050815f720 0x7f05080d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f050815f720 0x7f05080d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45074/0 (socket says 172.21.15.40:45074) 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 -- 172.21.15.40:0/4199990431 learned_addr learned my addr 172.21.15.40:0/4199990431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f0506ffd640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 0x7f05080b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f05077fe640 1 --2- 172.21.15.40:0/4199990431 >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 0x7f05080d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 -- 172.21.15.40:0/4199990431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 msgr2=0x7f05080b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 0x7f05080b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 -- 172.21.15.40:0/4199990431 >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 msgr2=0x7f05080d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 --2- 172.21.15.40:0/4199990431 >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 0x7f05080d5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f04fffff640 1 -- 172.21.15.40:0/4199990431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05080b2db0 con 0x7f050815f720 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f0506ffd640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 0x7f05080b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.988+0000 7f05077fe640 1 --2- 172.21.15.40:0/4199990431 >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 0x7f05080d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.989+0000 7f04fffff640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f720 0x7f05080d3d40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f05080b6300 tx=0x7f04f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4231aff88fa58ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.989+0000 7f05067fc640 1 -- 172.21.15.40:0/4199990431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0037070 con 0x7f050815f720 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.989+0000 7f05067fc640 1 -- 172.21.15.40:0/4199990431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0032440 con 0x7f050815f720 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.989+0000 7f05067fc640 1 -- 172.21.15.40:0/4199990431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0032810 con 0x7f050815f720 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 -- 172.21.15.40:0/4199990431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f720 msgr2=0x7f05080d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f720 0x7f05080d3d40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f05080b6300 tx=0x7f04f002a140 comp rx=0 tx=0).stop 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 -- 172.21.15.40:0/4199990431 shutdown_connections 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 --2- 172.21.15.40:0/4199990431 >> v2:172.21.15.184:3300/0 conn(0x7f05080d4f00 0x7f05080d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f720 0x7f05080d3d40 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 --2- 172.21.15.40:0/4199990431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f340 0x7f05080b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 -- 172.21.15.40:0/4199990431 >> 172.21.15.40:0/4199990431 conn(0x7f050808ec70 msgr2=0x7f050809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 -- 172.21.15.40:0/4199990431 shutdown_connections 2024-04-26T21:20:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.990+0000 7f050d6cc640 1 -- 172.21.15.40:0/4199990431 wait complete. 2024-04-26T21:20:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.991+0000 7f050d6cc640 1 Processor -- start 2024-04-26T21:20:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.991+0000 7f050d6cc640 1 -- start start 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05080d4f00 0x7f0508155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f720 0x7f05080cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05080b5040 con 0x7f050815f340 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05080b4ec0 con 0x7f05080d4f00 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f05080b51c0 con 0x7f050815f720 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45078/0 (socket says 172.21.15.40:45078) 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 -- 172.21.15.40:0/1213318615 learned_addr learned my addr 172.21.15.40:0/1213318615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f05077fe640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f720 0x7f05080cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f0506ffd640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05080d4f00 0x7f0508155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 -- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f720 msgr2=0x7f05080cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f720 0x7f05080cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 -- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05080d4f00 msgr2=0x7f0508155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05080d4f00 0x7f0508155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 -- 172.21.15.40:0/1213318615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0508161d60 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04fffff640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f04f002a180 tx=0x7f04f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0037040 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0042070 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.992+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05080ba9b0 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.993+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0030bb0 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.993+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0508162020 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.993+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050806a290 con 0x7f050815f340 2024-04-26T21:20:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.994+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04f0047460 con 0x7f050815f340 2024-04-26T21:20:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.994+0000 7f04ff7fe640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.995+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f04f00ac640 con 0x7f050815f340 2024-04-26T21:20:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.995+0000 7f0506ffd640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.995+0000 7f0506ffd640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f04f800ab60 tx=0x7f04f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:20.999+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04f007a4b0 con 0x7f050815f340 2024-04-26T21:20:21.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.184+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05080b2210 con 0x7f050815f340 2024-04-26T21:20:21.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.185+0000 7f04ff7fe640 1 -- 172.21.15.40:0/1213318615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7f04f007e7b0 con 0x7f050815f340 2024-04-26T21:20:21.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:21.187 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:21.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 msgr2=0x7f04e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f04f800ab60 tx=0x7f04f8008040 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 msgr2=0x7f0508162820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f04f002a180 tx=0x7f04f00046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 shutdown_connections 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f050815f720 0x7f05080cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f050815f340 0x7f0508162820 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 --2- 172.21.15.40:0/1213318615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05080d4f00 0x7f0508155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 >> 172.21.15.40:0/1213318615 conn(0x7f050808ec70 msgr2=0x7f05080b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 shutdown_connections 2024-04-26T21:20:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.188+0000 7f050d6cc640 1 -- 172.21.15.40:0/1213318615 wait complete. 2024-04-26T21:20:21.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:57.355243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:58.032686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:20:21.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:20:21.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.302+0000 7f72042bd640 1 Processor -- start 2024-04-26T21:20:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 -- start start 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71fc1631f0 0x7f71fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71fc0d1170 0x7f71fc1635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc09c790 con 0x7f71fc0b3300 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f71fc0d15f0 con 0x7f71fc1631f0 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.303+0000 7f72042bd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f71fc163ec0 con 0x7f71fc0d1170 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71fc1631f0 0x7f71fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71fc1631f0 0x7f71fc0b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45082/0 (socket says 172.21.15.40:45082) 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f720285a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f71fc0b3300 0x7f71fc0b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50642/0 (socket says 172.21.15.40:50642) 2024-04-26T21:20:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 -- 172.21.15.40:0/1222810650 learned_addr learned my addr 172.21.15.40:0/1222810650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7201858640 1 --2- 172.21.15.40:0/1222810650 >> v2:172.21.15.184:3300/0 conn(0x7f71fc0d1170 0x7f71fc1635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 -- 172.21.15.40:0/1222810650 >> v2:172.21.15.184:3300/0 conn(0x7f71fc0d1170 msgr2=0x7f71fc1635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 --2- 172.21.15.40:0/1222810650 >> v2:172.21.15.184:3300/0 conn(0x7f71fc0d1170 0x7f71fc1635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 -- 172.21.15.40:0/1222810650 >> v1:172.21.15.40:6789/0 conn(0x7f71fc0b3300 legacy=0x7f71fc0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.304+0000 7f7202059640 1 -- 172.21.15.40:0/1222810650 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc0d1a40 con 0x7f71fc1631f0 2024-04-26T21:20:21.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f7202059640 1 --2- 172.21.15.40:0/1222810650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc1631f0 0x7f71fc0b2000 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0734b0 tx=0x7f71f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bc1462c8ccf2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f7201057640 1 -- 172.21.15.40:0/1222810650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8037070 con 0x7f71fc1631f0 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f7201057640 1 -- 172.21.15.40:0/1222810650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8032440 con 0x7f71fc1631f0 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f7201057640 1 -- 172.21.15.40:0/1222810650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8032810 con 0x7f71fc1631f0 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 -- 172.21.15.40:0/1222810650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc1631f0 msgr2=0x7f71fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 --2- 172.21.15.40:0/1222810650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc1631f0 0x7f71fc0b2000 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0734b0 tx=0x7f71f802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 -- 172.21.15.40:0/1222810650 shutdown_connections 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 --2- 172.21.15.40:0/1222810650 >> v2:172.21.15.184:3300/0 conn(0x7f71fc0d1170 0x7f71fc1635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 --2- 172.21.15.40:0/1222810650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc1631f0 0x7f71fc0b2000 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 -- 172.21.15.40:0/1222810650 >> 172.21.15.40:0/1222810650 conn(0x7f71fc09ba40 msgr2=0x7f71fc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.305+0000 7f72042bd640 1 -- 172.21.15.40:0/1222810650 shutdown_connections 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.306+0000 7f72042bd640 1 -- 172.21.15.40:0/1222810650 wait complete. 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 Processor -- start 2024-04-26T21:20:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 -- start start 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71fc0b3300 0x7f71fc15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc1631f0 0x7f71fc14d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc0d0230 con 0x7f71fc0d1170 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc0d2000 con 0x7f71fc0b3300 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f72042bd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc0cb970 con 0x7f71fc1631f0 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f7201858640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f7201858640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45090/0 (socket says 172.21.15.40:45090) 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.307+0000 7f7201858640 1 -- 172.21.15.40:0/3330761288 learned_addr learned my addr 172.21.15.40:0/3330761288 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f720285a640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc1631f0 0x7f71fc14d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7202059640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71fc0b3300 0x7f71fc15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 -- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc1631f0 msgr2=0x7f71fc14d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc1631f0 0x7f71fc14d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 -- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71fc0b3300 msgr2=0x7f71fc15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71fc0b3300 0x7f71fc15ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 -- 172.21.15.40:0/3330761288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc0cbc10 con 0x7f71fc0d1170 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f7201858640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00d800 tx=0x7f71ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00b6c0 con 0x7f71fc0d1170 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec019070 con 0x7f71fc0d1170 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00ba80 con 0x7f71fc0d1170 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc0cbef0 con 0x7f71fc0d1170 2024-04-26T21:20:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.308+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc1f5900 con 0x7f71fc0d1170 2024-04-26T21:20:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.309+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc06a290 con 0x7f71fc0d1170 2024-04-26T21:20:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.310+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71ec00bc20 con 0x7f71fc0d1170 2024-04-26T21:20:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.310+0000 7f71f2ffd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 0x7f71e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.310+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f71ec08bfb0 con 0x7f71fc0d1170 2024-04-26T21:20:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.310+0000 7f7202059640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 0x7f71e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.311+0000 7f7202059640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 0x7f71e006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f71f8034000 tx=0x7f71f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.314+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71ec01c360 con 0x7f71fc0d1170 2024-04-26T21:20:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.499+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71fc0b8f10 con 0x7f71fc0d1170 2024-04-26T21:20:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.499+0000 7f71f2ffd640 1 -- 172.21.15.40:0/3330761288 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f71ec05a470 con 0x7f71fc0d1170 2024-04-26T21:20:21.500 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:20:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 msgr2=0x7f71e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 0x7f71e006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f71f8034000 tx=0x7f71f802e000 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 msgr2=0x7f71fc1f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00d800 tx=0x7f71ec00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 shutdown_connections 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f71e006c8f0 0x7f71e006edb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71fc1631f0 0x7f71fc14d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71fc0d1170 0x7f71fc1f53c0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 --2- 172.21.15.40:0/3330761288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71fc0b3300 0x7f71fc15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 >> 172.21.15.40:0/3330761288 conn(0x7f71fc09ba40 msgr2=0x7f71fc070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 shutdown_connections 2024-04-26T21:20:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.501+0000 7f72042bd640 1 -- 172.21.15.40:0/3330761288 wait complete. 2024-04-26T21:20:21.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:20:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.615+0000 7f694df67640 1 Processor -- start 2024-04-26T21:20:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 -- start start 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 0x7f69480d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69480d4f20 0x7f69480d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69480d58c0 con 0x7f6948163aa0 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f694815fa60 con 0x7f69480d4f20 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f694df67640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f69480d3f50 con 0x7f694815f680 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f6947fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69480d4f20 0x7f69480d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f6946ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6948163aa0 0x7f69480b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50652/0 (socket says 172.21.15.40:50652) 2024-04-26T21:20:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.616+0000 7f6946ffd640 1 -- 172.21.15.40:0/1117627464 learned_addr learned my addr 172.21.15.40:0/1117627464 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f6947fff640 1 -- 172.21.15.40:0/1117627464 >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 msgr2=0x7f69480d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f69477fe640 1 --2- 172.21.15.40:0/1117627464 >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 0x7f69480d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f69467fc640 1 -- 172.21.15.40:0/1117627464 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271136162 0 0) 0x7f69480d58c0 con 0x7f6948163aa0 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f6947fff640 1 --2- 172.21.15.40:0/1117627464 >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 0x7f69480d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f6947fff640 1 -- 172.21.15.40:0/1117627464 >> v1:172.21.15.40:6789/0 conn(0x7f6948163aa0 legacy=0x7f69480b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f6947fff640 1 -- 172.21.15.40:0/1117627464 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69480b5090 con 0x7f69480d4f20 2024-04-26T21:20:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f69477fe640 1 --2- 172.21.15.40:0/1117627464 >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 0x7f69480d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.617+0000 7f6947fff640 1 --2- 172.21.15.40:0/1117627464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69480d5380 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f69480d6370 tx=0x7f694002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7190b74fe1a4a9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f69467fc640 1 -- 172.21.15.40:0/1117627464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940007740 con 0x7f69480d4f20 2024-04-26T21:20:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f69467fc640 1 -- 172.21.15.40:0/1117627464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69400078e0 con 0x7f69480d4f20 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f69467fc640 1 -- 172.21.15.40:0/1117627464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940007cb0 con 0x7f69480d4f20 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 -- 172.21.15.40:0/1117627464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 msgr2=0x7f69480d5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 --2- 172.21.15.40:0/1117627464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69480d5380 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f69480d6370 tx=0x7f694002a680 comp rx=0 tx=0).stop 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 -- 172.21.15.40:0/1117627464 shutdown_connections 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 --2- 172.21.15.40:0/1117627464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69480d5380 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 --2- 172.21.15.40:0/1117627464 >> v2:172.21.15.184:3300/0 conn(0x7f694815f680 0x7f69480d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 -- 172.21.15.40:0/1117627464 >> 172.21.15.40:0/1117627464 conn(0x7f694808eca0 msgr2=0x7f694809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 -- 172.21.15.40:0/1117627464 shutdown_connections 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.618+0000 7f694df67640 1 -- 172.21.15.40:0/1117627464 wait complete. 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.619+0000 7f694df67640 1 Processor -- start 2024-04-26T21:20:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 -- start start 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 0x7f69481613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6948163aa0 0x7f694814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6948163f30 con 0x7f69480d4f20 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69480b6280 con 0x7f694815f680 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.620+0000 7f694df67640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69480cfd80 con 0x7f6948163aa0 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45102/0 (socket says 172.21.15.40:45102) 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 -- 172.21.15.40:0/1113528802 learned_addr learned my addr 172.21.15.40:0/1113528802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f6946ffd640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 0x7f69481613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f6947fff640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6948163aa0 0x7f694814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 -- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6948163aa0 msgr2=0x7f694814f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6948163aa0 0x7f694814f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 -- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 msgr2=0x7f69481613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 0x7f69481613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 -- 172.21.15.40:0/1113528802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69480cecb0 con 0x7f69480d4f20 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f6946ffd640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 0x7f69481613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f69477fe640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f693800c820 tx=0x7f693800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938015070 con 0x7f69480d4f20 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6938007a30 con 0x7f69480d4f20 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69480cef90 con 0x7f69480d4f20 2024-04-26T21:20:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938007e00 con 0x7f69480d4f20 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.621+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6948159e60 con 0x7f69480d4f20 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.622+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694806a290 con 0x7f69480d4f20 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.623+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6938018070 con 0x7f69480d4f20 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.623+0000 7f692ffff640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 0x7f693406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.623+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f693808bb20 con 0x7f69480d4f20 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.623+0000 7f6946ffd640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 0x7f693406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.624+0000 7f6946ffd640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 0x7f693406edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6948162be0 tx=0x7f693c007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.627+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6938059f50 con 0x7f69480d4f20 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.811+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69480b4cd0 con 0x7f69480d4f20 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.811+0000 7f692ffff640 1 -- 172.21.15.40:0/1113528802 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7f6938059850 con 0x7f69480d4f20 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 msgr2=0x7f693406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 0x7f693406edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6948162be0 tx=0x7f693c007ca0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 msgr2=0x7f69481596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f693800c820 tx=0x7f693800cc20 comp rx=0 tx=0).stop 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 shutdown_connections 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693406c8f0 0x7f693406edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6948163aa0 0x7f694814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f694815f680 0x7f69481613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 --2- 172.21.15.40:0/1113528802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69480d4f20 0x7f69481596f0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 >> 172.21.15.40:0/1113528802 conn(0x7f694808eca0 msgr2=0x7f6948163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 shutdown_connections 2024-04-26T21:20:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.813+0000 7f694df67640 1 -- 172.21.15.40:0/1113528802 wait complete. 2024-04-26T21:20:21.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:20:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.928+0000 7f5fef637640 1 Processor -- start 2024-04-26T21:20:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fef637640 1 -- start start 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fef637640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fe80d4f20 0x7f5fe80d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fef637640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe80d3c90 con 0x7f5fe815f680 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fef637640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe80d58c0 con 0x7f5fe80d3820 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fef637640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe815fa60 con 0x7f5fe80d4f20 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.929+0000 7f5fedbd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fe80d4f20 0x7f5fe80d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fed3d3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5fe815f680 0x7f5fe8163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50660/0 (socket says 172.21.15.40:50660) 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fed3d3640 1 -- 172.21.15.40:0/2537817216 learned_addr learned my addr 172.21.15.40:0/2537817216 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fedbd4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fe80d4f20 0x7f5fe80d5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45110/0 (socket says 172.21.15.40:45110) 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fedbd4640 1 -- 172.21.15.40:0/2537817216 >> v1:172.21.15.184:6789/0 conn(0x7f5fe80d3820 legacy=0x7f5fe80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fedbd4640 1 -- 172.21.15.40:0/2537817216 >> v1:172.21.15.40:6789/0 conn(0x7f5fe815f680 legacy=0x7f5fe8163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fdbfff640 1 -- 172.21.15.40:0/2537817216 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093083428 0 0) 0x7f5fe80d3c90 con 0x7f5fe815f680 2024-04-26T21:20:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.930+0000 7f5fedbd4640 1 -- 172.21.15.40:0/2537817216 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe80d3f50 con 0x7f5fe80d4f20 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fedbd4640 1 --2- 172.21.15.40:0/2537817216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80d5380 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc2b48f32b17563 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fdbfff640 1 -- 172.21.15.40:0/2537817216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007740 con 0x7f5fe80d4f20 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fdbfff640 1 -- 172.21.15.40:0/2537817216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe00078e0 con 0x7f5fe80d4f20 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fdbfff640 1 -- 172.21.15.40:0/2537817216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007cb0 con 0x7f5fe80d4f20 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 -- 172.21.15.40:0/2537817216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe80d4f20 msgr2=0x7f5fe80d5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 --2- 172.21.15.40:0/2537817216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80d5380 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 -- 172.21.15.40:0/2537817216 shutdown_connections 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 --2- 172.21.15.40:0/2537817216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80d5380 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 -- 172.21.15.40:0/2537817216 >> 172.21.15.40:0/2537817216 conn(0x7f5fe808eca0 msgr2=0x7f5fe809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 -- 172.21.15.40:0/2537817216 shutdown_connections 2024-04-26T21:20:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.931+0000 7f5fef637640 1 -- 172.21.15.40:0/2537817216 wait complete. 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 Processor -- start 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 -- start start 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe80d3820 0x7f5fe80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80b6280 con 0x7f5fe815f680 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80b5370 con 0x7f5fe80d4f20 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fef637640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe80cfd80 con 0x7f5fe80d3820 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fedbd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.933+0000 7f5fedbd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45126/0 (socket says 172.21.15.40:45126) 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 -- 172.21.15.40:0/1361827415 learned_addr learned my addr 172.21.15.40:0/1361827415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fed3d3640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe80d3820 0x7f5fe80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fecbd2640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 -- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe80d3820 msgr2=0x7f5fe80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe80d3820 0x7f5fe80cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 -- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe80d4f20 msgr2=0x7f5fe80ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 -- 172.21.15.40:0/1361827415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe815fc80 con 0x7f5fe815f680 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fedbd4640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009eb0 tx=0x7f5fe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0040070 con 0x7f5fe815f680 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0031c40 con 0x7f5fe815f680 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe815ff60 con 0x7f5fe815f680 2024-04-26T21:20:21.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0003990 con 0x7f5fe815f680 2024-04-26T21:20:21.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.934+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe8160520 con 0x7f5fe815f680 2024-04-26T21:20:21.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.936+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe8160720 con 0x7f5fe815f680 2024-04-26T21:20:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.936+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fe0039040 con 0x7f5fe815f680 2024-04-26T21:20:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.937+0000 7f5fd9ffb640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.937+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f5fe00aceb0 con 0x7f5fe815f680 2024-04-26T21:20:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.937+0000 7f5fed3d3640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.937+0000 7f5fed3d3640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000f60 tx=0x7f5fdc007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:21.941+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fe8160720 con 0x7f5fe815f680 2024-04-26T21:20:22.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.123+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5fe806a290 con 0x7f5fe815f680 2024-04-26T21:20:22.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.123+0000 7f5fd9ffb640 1 -- 172.21.15.40:0/1361827415 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7f5fe007ac00 con 0x7f5fe815f680 2024-04-26T21:20:22.125 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 msgr2=0x7f5fcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000f60 tx=0x7f5fdc007e40 comp rx=0 tx=0).stop 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 msgr2=0x7f5fe80d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009eb0 tx=0x7f5fe0004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 shutdown_connections 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fe815f680 0x7f5fe80d0340 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fe80d4f20 0x7f5fe80ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 --2- 172.21.15.40:0/1361827415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fe80d3820 0x7f5fe80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.125+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 >> 172.21.15.40:0/1361827415 conn(0x7f5fe808eca0 msgr2=0x7f5fe815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:22.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.126+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 shutdown_connections 2024-04-26T21:20:22.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.126+0000 7f5fef637640 1 -- 172.21.15.40:0/1361827415 wait complete. 2024-04-26T21:20:22.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:20:22.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.238+0000 7fc835104640 1 Processor -- start 2024-04-26T21:20:22.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.238+0000 7fc835104640 1 -- start start 2024-04-26T21:20:22.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc835104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc83006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc835104640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8300d1740 con 0x7fc8300d11b0 2024-04-26T21:20:22.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc835104640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8300d1a20 con 0x7fc83006e490 2024-04-26T21:20:22.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc835104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8300d1d00 con 0x7fc8300cf2e0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc827fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8300d11b0 0x7fc83006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50666/0 (socket says 172.21.15.40:50666) 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc827fff640 1 -- 172.21.15.40:0/427150407 learned_addr learned my addr 172.21.15.40:0/427150407 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc83006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589063569 0 0) 0x7fc8300d1740 con 0x7fc8300d11b0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc81c003520 con 0x7fc8300d11b0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251704039 0 0) 0x7fc81c003520 con 0x7fc8300d11b0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 msgr2=0x7fc83006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc83006d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 >> v1:172.21.15.184:6789/0 conn(0x7fc83006e490 legacy=0x7fc83015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8300d1fe0 con 0x7fc8300d11b0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fc818001670 con 0x7fc8300d11b0 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.239+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 >> v1:172.21.15.40:6789/0 conn(0x7fc8300d11b0 legacy=0x7fc83006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc81c0039e0 0x7fc81c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc81c004390 0x7fc81c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc81c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300d1d00 con 0x7fc81c0039e0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c00b9c0 con 0x7fc81c004390 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc81c00bbc0 con 0x7fc8300cf2e0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc81c0039e0 0x7fc81c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82f59e640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc81c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc827fff640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc81c004390 0x7fc81c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 -- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 msgr2=0x7fc81c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc8300cf2e0 0x7fc81c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 -- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc81c004390 msgr2=0x7fc81c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc81c004390 0x7fc81c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc82ed9d640 1 -- 172.21.15.40:0/427150407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c003520 con 0x7fc81c0039e0 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.240+0000 7fc827fff640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc81c004390 0x7fc81c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc81c0039e0 0x7fc81c003dc0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ac0 tx=0x7fc81c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6ec66b7d97216 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820036070 con 0x7fc81c0039e0 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820031400 con 0x7fc81c0039e0 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc82e59c640 1 -- 172.21.15.40:0/427150407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820009690 con 0x7fc81c0039e0 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 -- 172.21.15.40:0/427150407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc81c0039e0 msgr2=0x7fc81c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 --2- 172.21.15.40:0/427150407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc81c0039e0 0x7fc81c003dc0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ac0 tx=0x7fc81c005830 comp rx=0 tx=0).stop 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc82f59e640 1 -- 172.21.15.40:0/427150407 reap_dead start 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 -- 172.21.15.40:0/427150407 shutdown_connections 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 -- 172.21.15.40:0/427150407 >> 172.21.15.40:0/427150407 conn(0x7fc8300b4a60 msgr2=0x7fc8300b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 -- 172.21.15.40:0/427150407 shutdown_connections 2024-04-26T21:20:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.241+0000 7fc835104640 1 -- 172.21.15.40:0/427150407 wait complete. 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 Processor -- start 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 -- start start 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 0x7fc8301f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 0x7fc830152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300cf840 con 0x7fc8300b7a40 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8300cf6c0 con 0x7fc8300b75c0 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.243+0000 7fc835104640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc8300ce660 con 0x7fc81c0039e0 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45142/0 (socket says 172.21.15.40:45142) 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 -- 172.21.15.40:0/3225452963 learned_addr learned my addr 172.21.15.40:0/3225452963 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 0x7fc8301f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc827fff640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 0x7fc830152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 -- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 msgr2=0x7fc8301f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 0x7fc8301f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 -- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 msgr2=0x7fc830152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 0x7fc830152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 -- 172.21.15.40:0/3225452963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8301f9a20 con 0x7fc8300b7a40 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 0x7fc8301f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc827fff640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 0x7fc830152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc82f59e640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc83006e8a0 tx=0x7fc82800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828019070 con 0x7fc8300b7a40 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc828006d60 con 0x7fc8300b7a40 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc830153970 con 0x7fc8300b7a40 2024-04-26T21:20:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828003b90 con 0x7fc8300b7a40 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.244+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83009e330 con 0x7fc8300b7a40 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.245+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83006a290 con 0x7fc8300b7a40 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.246+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc82801c070 con 0x7fc8300b7a40 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.246+0000 7fc8277fe640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.246+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fc82808afa0 con 0x7fc8300b7a40 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.247+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.247+0000 7fc82ed9d640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ac0 tx=0x7fc820008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.250+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8301f9a20 con 0x7fc8300b7a40 2024-04-26T21:20:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.449+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8300b8ef0 con 0x7fc8300b7a40 2024-04-26T21:20:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.450+0000 7fc8277fe640 1 -- 172.21.15.40:0/3225452963 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7fc828059330 con 0x7fc8300b7a40 2024-04-26T21:20:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 msgr2=0x7fc81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ac0 tx=0x7fc820008030 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 msgr2=0x7fc830153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc83006e8a0 tx=0x7fc82800ff70 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 shutdown_connections 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8300b7a40 0x7fc830153060 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8300b75c0 0x7fc830152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 --2- 172.21.15.40:0/3225452963 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc81c0039e0 0x7fc8301f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 >> 172.21.15.40:0/3225452963 conn(0x7fc8300b4a60 msgr2=0x7fc83009ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 shutdown_connections 2024-04-26T21:20:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.452+0000 7fc835104640 1 -- 172.21.15.40:0/3225452963 wait complete. 2024-04-26T21:20:22.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:20:22.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:22.463 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:22.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:20:22.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:22.498 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:20:22.556 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:20:22.557 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:22.557 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:22.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:20:22.572 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:20:22.573 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:22.573 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:22.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:20:22.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.713+0000 7f7bb7324640 1 Processor -- start 2024-04-26T21:20:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.713+0000 7f7bb7324640 1 -- start start 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.714+0000 7f7bb7324640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb015d3b0 0x7f7bb00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.714+0000 7f7bb7324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.714+0000 7f7bb7324640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb00b6b80 con 0x7f7bb006c7e0 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.714+0000 7f7bb7324640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb00aedf0 con 0x7f7bb015d3b0 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.714+0000 7f7bb7324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb006cc40 con 0x7f7bb0073b10 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb48bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb50c0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb015d3b0 0x7f7bb00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb48bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35604/0 (socket says 172.21.15.40:35604) 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb48bf640 1 -- 172.21.15.40:0/1659647207 learned_addr learned my addr 172.21.15.40:0/1659647207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb58c1640 1 --1- 172.21.15.40:0/1659647207 >> v1:172.21.15.184:6789/0 conn(0x7f7bb006c7e0 0x7f7bb0160f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:56878/0 (socket says 172.21.15.40:56878) 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb50c0640 1 -- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 msgr2=0x7f7bb006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb50c0640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb50c0640 1 -- 172.21.15.40:0/1659647207 >> v1:172.21.15.184:6789/0 conn(0x7f7bb006c7e0 legacy=0x7f7bb0160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7baffff640 1 -- 172.21.15.40:0/1659647207 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193856926 0 0) 0x7f7bb00b6b80 con 0x7f7bb006c7e0 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb50c0640 1 -- 172.21.15.40:0/1659647207 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb00b6e60 con 0x7f7bb015d3b0 2024-04-26T21:20:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.715+0000 7f7bb48bf640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.716+0000 7f7bb50c0640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb00735d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b809ec9c46fa241 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.716+0000 7f7baffff640 1 -- 172.21.15.40:0/1659647207 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8037070 con 0x7f7bb015d3b0 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.716+0000 7f7baffff640 1 -- 172.21.15.40:0/1659647207 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba802ae50 con 0x7f7bb015d3b0 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.716+0000 7f7baffff640 1 -- 172.21.15.40:0/1659647207 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8032660 con 0x7f7bb015d3b0 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 -- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 msgr2=0x7f7bb00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb00735d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a100 comp rx=0 tx=0).stop 2024-04-26T21:20:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 -- 172.21.15.40:0/1659647207 shutdown_connections 2024-04-26T21:20:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 --2- 172.21.15.40:0/1659647207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb00735d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 -- 172.21.15.40:0/1659647207 >> 172.21.15.40:0/1659647207 conn(0x7f7bb00b3b10 msgr2=0x7f7bb00b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 -- 172.21.15.40:0/1659647207 shutdown_connections 2024-04-26T21:20:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.717+0000 7f7bb7324640 1 -- 172.21.15.40:0/1659647207 wait complete. 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.718+0000 7f7bb7324640 1 Processor -- start 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.718+0000 7f7bb7324640 1 -- start start 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb0139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb0139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb00b6930 con 0x7f7bb006c7e0 2024-04-26T21:20:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0073ef0 con 0x7f7bb015d3b0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb7324640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb00d0170 con 0x7f7bb0073b10 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45152/0 (socket says 172.21.15.40:45152) 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 -- 172.21.15.40:0/3859662482 learned_addr learned my addr 172.21.15.40:0/3859662482 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb48bf640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb0139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb58c1640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb0139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 -- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 msgr2=0x7f7bb0139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb0139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 -- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 msgr2=0x7f7bb0139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb0139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 -- 172.21.15.40:0/3859662482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb00b14b0 con 0x7f7bb006c7e0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7bb50c0640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8035040 tx=0x7f7ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.719+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8037040 con 0x7f7bb006c7e0 2024-04-26T21:20:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.720+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb00b1790 con 0x7f7bb006c7e0 2024-04-26T21:20:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.720+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba8004030 con 0x7f7bb006c7e0 2024-04-26T21:20:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.720+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8006e10 con 0x7f7bb006c7e0 2024-04-26T21:20:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.720+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb00b1d50 con 0x7f7bb006c7e0 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.721+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb00693d0 con 0x7f7bb006c7e0 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.721+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ba8003610 con 0x7f7bb006c7e0 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.722+0000 7f7badffb640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 0x7f7b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.722+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f7ba80ac180 con 0x7f7bb006c7e0 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.722+0000 7f7bb48bf640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 0x7f7b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.723+0000 7f7bb48bf640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 0x7f7b9c06edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4006ca0 tx=0x7f7ba4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.727+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ba8042020 con 0x7f7bb006c7e0 2024-04-26T21:20:22.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.951+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb006f300 con 0x7f7bb006c7e0 2024-04-26T21:20:22.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.951+0000 7f7badffb640 1 -- 172.21.15.40:0/3859662482 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5189 (secure 0 0 0) 0x7f7ba807a090 con 0x7f7bb006c7e0 2024-04-26T21:20:22.952 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:22.952 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:19.759313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:22.953 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 79 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 msgr2=0x7f7b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 0x7f7b9c06edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4006ca0 tx=0x7f7ba4009240 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 msgr2=0x7f7bb00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8035040 tx=0x7f7ba80046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 shutdown_connections 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7b9c06c8f0 0x7f7b9c06edb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb015d3b0 0x7f7bb0139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb0073b10 0x7f7bb0139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 --2- 172.21.15.40:0/3859662482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bb006c7e0 0x7f7bb00afb90 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 >> 172.21.15.40:0/3859662482 conn(0x7f7bb00b3b10 msgr2=0x7f7bb00b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 shutdown_connections 2024-04-26T21:20:22.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:22.953+0000 7f7bb7324640 1 -- 172.21.15.40:0/3859662482 wait complete. 2024-04-26T21:20:22.965 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:23.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.065+0000 7fd768b19640 1 Processor -- start 2024-04-26T21:20:23.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.065+0000 7fd768b19640 1 -- start start 2024-04-26T21:20:23.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd768b19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd76415f720 0x7fd7640b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd768b19640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7640b2510 con 0x7fd76415f340 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd768b19640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7640b27f0 con 0x7fd7640d5610 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd768b19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7640b2ad0 con 0x7fd76415f720 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd76259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd76415f340 0x7fd7640d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50668/0 (socket says 172.21.15.40:50668) 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd76259c640 1 -- 172.21.15.40:0/2451613517 learned_addr learned my addr 172.21.15.40:0/2451613517 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd75bfff640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd76415f720 0x7fd7640b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892129106 0 0) 0x7fd7640b2510 con 0x7fd76415f340 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd750003520 con 0x7fd76415f340 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298020273 0 0) 0x7fd7640b27f0 con 0x7fd7640d5610 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7640b2510 con 0x7fd7640d5610 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2393290279 0 0) 0x7fd750003520 con 0x7fd76415f340 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd76415f720 msgr2=0x7fd7640b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd76415f720 0x7fd7640b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 >> v1:172.21.15.184:6789/0 conn(0x7fd7640d5610 legacy=0x7fd7640b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7640b2db0 con 0x7fd76415f340 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd754001670 con 0x7fd76415f340 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd75bfff640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd76415f720 0x7fd7640b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 >> v1:172.21.15.40:6789/0 conn(0x7fd76415f340 legacy=0x7fd7640d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd750003b70 0x7fd750003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd750004800 0x7fd750004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7500051c0 0x7fd75000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640b2ad0 con 0x7fd7500051c0 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd75000c090 con 0x7fd750004800 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.066+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd75000c290 con 0x7fd750003b70 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7500051c0 0x7fd75000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd75bfff640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd750004800 0x7fd750004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd76259c640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd750003b70 0x7fd750003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 -- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd750003b70 msgr2=0x7fd750003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd750003b70 0x7fd750003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 -- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd750004800 msgr2=0x7fd750004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd750004800 0x7fd750004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.067+0000 7fd762d9d640 1 -- 172.21.15.40:0/2451613517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd750003520 con 0x7fd7500051c0 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd762d9d640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7500051c0 0x7fd75000b9c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd75c001fd0 tx=0x7fd750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9627f8325b91fbaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c001bf0 con 0x7fd7500051c0 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c003b70 con 0x7fd7500051c0 2024-04-26T21:20:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd761d9b640 1 -- 172.21.15.40:0/2451613517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0050d0 con 0x7fd7500051c0 2024-04-26T21:20:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 -- 172.21.15.40:0/2451613517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7500051c0 msgr2=0x7fd75000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 --2- 172.21.15.40:0/2451613517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7500051c0 0x7fd75000b9c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd75c001fd0 tx=0x7fd750005830 comp rx=0 tx=0).stop 2024-04-26T21:20:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd762d9d640 1 -- 172.21.15.40:0/2451613517 reap_dead start 2024-04-26T21:20:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 -- 172.21.15.40:0/2451613517 shutdown_connections 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 -- 172.21.15.40:0/2451613517 >> 172.21.15.40:0/2451613517 conn(0x7fd76408ec70 msgr2=0x7fd76409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 -- 172.21.15.40:0/2451613517 shutdown_connections 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.068+0000 7fd768b19640 1 -- 172.21.15.40:0/2451613517 wait complete. 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 Processor -- start 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 -- start start 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 0x7fd7640b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 0x7fd7641f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640b2210 con 0x7fd764160ac0 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640b2090 con 0x7fd7640d07c0 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd768b19640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd7640d3d40 con 0x7fd7500051c0 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45178/0 (socket says 172.21.15.40:45178) 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.070+0000 7fd75bfff640 1 -- 172.21.15.40:0/3212680167 learned_addr learned my addr 172.21.15.40:0/3212680167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd76259c640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 0x7fd7640b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 -- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 msgr2=0x7fd7640b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd762d9d640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 0x7fd7641f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 0x7fd7640b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 -- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 msgr2=0x7fd7641f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 0x7fd7641f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 -- 172.21.15.40:0/3212680167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7640b97b0 con 0x7fd764160ac0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd76259c640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 0x7fd7640b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd762d9d640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 0x7fd7641f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75bfff640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd74c011c00 tx=0x7fd74c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c011cf0 con 0x7fd764160ac0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74c011e90 con 0x7fd764160ac0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76415fd00 con 0x7fd764160ac0 2024-04-26T21:20:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c014640 con 0x7fd764160ac0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.071+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7641602c0 con 0x7fd764160ac0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.072+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640d0bd0 con 0x7fd764160ac0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.073+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd74c012040 con 0x7fd764160ac0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.073+0000 7fd75affd640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 0x7fd74406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.073+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fd74c08c3e0 con 0x7fd764160ac0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.073+0000 7fd76259c640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 0x7fd74406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.074+0000 7fd76259c640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 0x7fd74406edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd754007b80 tx=0x7fd754009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.077+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7640d0bd0 con 0x7fd764160ac0 2024-04-26T21:20:23.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.262+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd76406a290 con 0x7fd764160ac0 2024-04-26T21:20:23.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.263+0000 7fd75affd640 1 -- 172.21.15.40:0/3212680167 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7fd74c019030 con 0x7fd764160ac0 2024-04-26T21:20:23.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:23.265 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 msgr2=0x7fd74406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 0x7fd74406edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd754007b80 tx=0x7fd754009040 comp rx=0 tx=0).stop 2024-04-26T21:20:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 msgr2=0x7fd7640d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd74c011c00 tx=0x7fd74c00f610 comp rx=0 tx=0).stop 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 shutdown_connections 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd74406c8f0 0x7fd74406edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7640d07c0 0x7fd7641f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd764160ac0 0x7fd7640d0280 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 --2- 172.21.15.40:0/3212680167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7500051c0 0x7fd7640b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 >> 172.21.15.40:0/3212680167 conn(0x7fd76408ec70 msgr2=0x7fd7640b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 shutdown_connections 2024-04-26T21:20:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.265+0000 7fd768b19640 1 -- 172.21.15.40:0/3212680167 wait complete. 2024-04-26T21:20:23.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:19:57.355243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:19:58.032686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:20:23.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:20:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.380+0000 7f275be47640 1 Processor -- start 2024-04-26T21:20:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.380+0000 7f275be47640 1 -- start start 2024-04-26T21:20:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275be47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275be47640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f27540b33e0 0x7f2754163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275be47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f275409c700 con 0x7f27540d0010 2024-04-26T21:20:23.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275be47640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f27540b4480 con 0x7f27540b33e0 2024-04-26T21:20:23.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275be47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f275406e050 con 0x7f27540b3c20 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f27593e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f27593e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35650/0 (socket says 172.21.15.40:35650) 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f27593e2640 1 -- 172.21.15.40:0/23872487 learned_addr learned my addr 172.21.15.40:0/23872487 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f275a3e4640 1 --2- 172.21.15.40:0/23872487 >> v2:172.21.15.184:3300/0 conn(0x7f27540b33e0 0x7f2754163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f27593e2640 1 -- 172.21.15.40:0/23872487 >> v2:172.21.15.184:3300/0 conn(0x7f27540b33e0 msgr2=0x7f2754163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.381+0000 7f27593e2640 1 --2- 172.21.15.40:0/23872487 >> v2:172.21.15.184:3300/0 conn(0x7f27540b33e0 0x7f2754163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.382+0000 7f27593e2640 1 -- 172.21.15.40:0/23872487 >> v1:172.21.15.184:6789/0 conn(0x7f27540d0010 legacy=0x7f27540b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.382+0000 7f2758be1640 1 -- 172.21.15.40:0/23872487 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4033458590 0 0) 0x7f275409c700 con 0x7f27540d0010 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.382+0000 7f27593e2640 1 -- 172.21.15.40:0/23872487 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27540d1c50 con 0x7f27540b3c20 2024-04-26T21:20:23.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.382+0000 7f27593e2640 1 --2- 172.21.15.40:0/23872487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2744009870 tx=0x7f274402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01d95e58c4486ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.382+0000 7f2758be1640 1 -- 172.21.15.40:0/23872487 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744037070 con 0x7f27540b3c20 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f2758be1640 1 -- 172.21.15.40:0/23872487 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744032440 con 0x7f27540b3c20 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f2758be1640 1 -- 172.21.15.40:0/23872487 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744032810 con 0x7f27540b3c20 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 -- 172.21.15.40:0/23872487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 msgr2=0x7f275406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 --2- 172.21.15.40:0/23872487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2744009870 tx=0x7f274402a270 comp rx=0 tx=0).stop 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 -- 172.21.15.40:0/23872487 shutdown_connections 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 --2- 172.21.15.40:0/23872487 >> v2:172.21.15.184:3300/0 conn(0x7f27540b33e0 0x7f2754163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 --2- 172.21.15.40:0/23872487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540b3c20 0x7f275406db10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 -- 172.21.15.40:0/23872487 >> 172.21.15.40:0/23872487 conn(0x7f275409ba40 msgr2=0x7f275409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 -- 172.21.15.40:0/23872487 shutdown_connections 2024-04-26T21:20:23.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.383+0000 7f275be47640 1 -- 172.21.15.40:0/23872487 wait complete. 2024-04-26T21:20:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.384+0000 7f275be47640 1 Processor -- start 2024-04-26T21:20:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.384+0000 7f275be47640 1 -- start start 2024-04-26T21:20:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27540b3c20 0x7f27540b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540d0010 0x7f27540b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27540b39f0 con 0x7f27540b33e0 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27540b3870 con 0x7f27540b3c20 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f275be47640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f27541637c0 con 0x7f27540d0010 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f2759be3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f2759be3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45186/0 (socket says 172.21.15.40:45186) 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.385+0000 7f2759be3640 1 -- 172.21.15.40:0/3168883621 learned_addr learned my addr 172.21.15.40:0/3168883621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f27593e2640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27540b3c20 0x7f27540b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f275a3e4640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540d0010 0x7f27540b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 -- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540d0010 msgr2=0x7f27540b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540d0010 0x7f27540b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 -- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27540b3c20 msgr2=0x7f27540b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27540b3c20 0x7f27540b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 -- 172.21.15.40:0/3168883621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27540b55d0 con 0x7f27540b33e0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f2759be3640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f275000c410 tx=0x7f275000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750017070 con 0x7f27540b33e0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27500035f0 con 0x7f27540b33e0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27540b4d40 con 0x7f27540b33e0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500039e0 con 0x7f27540b33e0 2024-04-26T21:20:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.386+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27540b7610 con 0x7f27540b33e0 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.387+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27540afd90 con 0x7f27540b33e0 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.387+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f275001a070 con 0x7f27540b33e0 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.388+0000 7f274a7fc640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 0x7f273806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.388+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f275008af50 con 0x7f27540b33e0 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.388+0000 7f27593e2640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 0x7f273806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.389+0000 7f27593e2640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 0x7f273806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2744034000 tx=0x7f274402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.392+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27540afd90 con 0x7f27540b33e0 2024-04-26T21:20:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.576+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f275406a290 con 0x7f27540b33e0 2024-04-26T21:20:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.577+0000 7f274a7fc640 1 -- 172.21.15.40:0/3168883621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f2750059410 con 0x7f27540b33e0 2024-04-26T21:20:23.578 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 msgr2=0x7f273806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 0x7f273806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2744034000 tx=0x7f274402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 msgr2=0x7f27540a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f275000c410 tx=0x7f275000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 shutdown_connections 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f273806c8f0 0x7f273806edb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27540d0010 0x7f27540b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27540b3c20 0x7f27540b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 --2- 172.21.15.40:0/3168883621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540b33e0 0x7f27540a0770 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 >> 172.21.15.40:0/3168883621 conn(0x7f275409ba40 msgr2=0x7f275415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 shutdown_connections 2024-04-26T21:20:23.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.579+0000 7f275be47640 1 -- 172.21.15.40:0/3168883621 wait complete. 2024-04-26T21:20:23.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:20:23.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.692+0000 7fcdffb4e640 1 Processor -- start 2024-04-26T21:20:23.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 -- start start 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcdf815f680 0x7fcdf80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf80d3f50 con 0x7fcdf80b60b0 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf815fa60 con 0x7fcdf815f680 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.693+0000 7fcdffb4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80b5090 con 0x7fcdf8163aa0 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfe0eb640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fcdf80b60b0 0x7fcdf80d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:56918/0 (socket says 172.21.15.40:56918) 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd8ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcdf815f680 0x7fcdf80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35666/0 (socket says 172.21.15.40:35666) 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfe0eb640 1 -- 172.21.15.40:0/87975539 learned_addr learned my addr 172.21.15.40:0/87975539 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:23.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/87975539 >> v2:172.21.15.184:3300/0 conn(0x7fcdf815f680 msgr2=0x7fcdf80d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 --2- 172.21.15.40:0/87975539 >> v2:172.21.15.184:3300/0 conn(0x7fcdf815f680 0x7fcdf80d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/87975539 >> v1:172.21.15.184:6789/0 conn(0x7fcdf80b60b0 legacy=0x7fcdf80d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfc8e8640 1 -- 172.21.15.40:0/87975539 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310186832 0 0) 0x7fcdf80d3f50 con 0x7fcdf80b60b0 2024-04-26T21:20:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.694+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/87975539 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf80b5370 con 0x7fcdf8163aa0 2024-04-26T21:20:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.695+0000 7fcdfd0e9640 1 --2- 172.21.15.40:0/87975539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67942e90c4ab19c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.695+0000 7fcdfc8e8640 1 -- 172.21.15.40:0/87975539 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037070 con 0x7fcdf8163aa0 2024-04-26T21:20:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.695+0000 7fcdfc8e8640 1 -- 172.21.15.40:0/87975539 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec032440 con 0x7fcdf8163aa0 2024-04-26T21:20:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.695+0000 7fcdfc8e8640 1 -- 172.21.15.40:0/87975539 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec032810 con 0x7fcdf8163aa0 2024-04-26T21:20:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 -- 172.21.15.40:0/87975539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 msgr2=0x7fcdf80b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/87975539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec02a180 comp rx=0 tx=0).stop 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 -- 172.21.15.40:0/87975539 shutdown_connections 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/87975539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf80b5b70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/87975539 >> v2:172.21.15.184:3300/0 conn(0x7fcdf815f680 0x7fcdf80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 -- 172.21.15.40:0/87975539 >> 172.21.15.40:0/87975539 conn(0x7fcdf808eca0 msgr2=0x7fcdf809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 -- 172.21.15.40:0/87975539 shutdown_connections 2024-04-26T21:20:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.696+0000 7fcdffb4e640 1 -- 172.21.15.40:0/87975539 wait complete. 2024-04-26T21:20:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.697+0000 7fcdffb4e640 1 Processor -- start 2024-04-26T21:20:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.697+0000 7fcdffb4e640 1 -- start start 2024-04-26T21:20:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdf80b60b0 0x7fcdf81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf8162420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80d57b0 con 0x7fcdf815f680 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80d5630 con 0x7fcdf80b60b0 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdffb4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80d5930 con 0x7fcdf8163aa0 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45198/0 (socket says 172.21.15.40:45198) 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/2533425483 learned_addr learned my addr 172.21.15.40:0/2533425483 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfe0eb640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf8162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd8ea640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdf80b60b0 0x7fcdf81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 msgr2=0x7fcdf8162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf8162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdf80b60b0 msgr2=0x7fcdf81537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdf80b60b0 0x7fcdf81537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 -- 172.21.15.40:0/2533425483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf80ccdb0 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.698+0000 7fcdfd0e9640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.699+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037050 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.699+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec038040 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.699+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf80cd090 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.699+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec004490 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.699+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf80cd650 con 0x7fcdf815f680 2024-04-26T21:20:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.700+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf806a290 con 0x7fcdf815f680 2024-04-26T21:20:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.700+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdec03c6d0 con 0x7fcdf815f680 2024-04-26T21:20:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.700+0000 7fcdf67fc640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.700+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fcdec0ac530 con 0x7fcdf815f680 2024-04-26T21:20:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.701+0000 7fcdfd8ea640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.701+0000 7fcdfd8ea640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcde80074a0 tx=0x7fcde800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.705+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdec07a9f0 con 0x7fcdf815f680 2024-04-26T21:20:23.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.892+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcdf80b4cd0 con 0x7fcdf815f680 2024-04-26T21:20:23.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.892+0000 7fcdf67fc640 1 -- 172.21.15.40:0/2533425483 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7fcdec07a2f0 con 0x7fcdf815f680 2024-04-26T21:20:23.893 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:20:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 msgr2=0x7fcde406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcde80074a0 tx=0x7fcde800b040 comp rx=0 tx=0).stop 2024-04-26T21:20:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 msgr2=0x7fcdf8161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 shutdown_connections 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdf8163aa0 0x7fcdf8162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf815f680 0x7fcdf8161ee0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 --2- 172.21.15.40:0/2533425483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdf80b60b0 0x7fcdf81537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 >> 172.21.15.40:0/2533425483 conn(0x7fcdf808eca0 msgr2=0x7fcdf80b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 shutdown_connections 2024-04-26T21:20:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:23.894+0000 7fcdffb4e640 1 -- 172.21.15.40:0/2533425483 wait complete. 2024-04-26T21:20:23.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:20:24.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.008+0000 7f892370a640 1 Processor -- start 2024-04-26T21:20:24.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 -- start start 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891c163aa0 0x7f891c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f891c0d3f50 con 0x7f891c0b60b0 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f891c15fa60 con 0x7f891c163aa0 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.009+0000 7f892370a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f891c0b5090 con 0x7f891c15f680 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891c163aa0 0x7f891c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891c163aa0 0x7f891c0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45200/0 (socket says 172.21.15.40:45200) 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 -- 172.21.15.40:0/3359994341 learned_addr learned my addr 172.21.15.40:0/3359994341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f89214a6640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 -- 172.21.15.40:0/3359994341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 msgr2=0x7f891c0d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 -- 172.21.15.40:0/3359994341 >> v1:172.21.15.184:6789/0 conn(0x7f891c0b60b0 legacy=0x7f891c0d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.010+0000 7f8920ca5640 1 -- 172.21.15.40:0/3359994341 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0b5370 con 0x7f891c163aa0 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f8920ca5640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c0b5b70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8910009870 tx=0x7f891002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf60892f581e5b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f891bfff640 1 -- 172.21.15.40:0/3359994341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910037070 con 0x7f891c163aa0 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f891bfff640 1 -- 172.21.15.40:0/3359994341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8910032440 con 0x7f891c163aa0 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f891bfff640 1 -- 172.21.15.40:0/3359994341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910032810 con 0x7f891c163aa0 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 -- 172.21.15.40:0/3359994341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 msgr2=0x7f891c0b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c0b5b70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8910009870 tx=0x7f891002a270 comp rx=0 tx=0).stop 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 -- 172.21.15.40:0/3359994341 shutdown_connections 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c0b5b70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 --2- 172.21.15.40:0/3359994341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 -- 172.21.15.40:0/3359994341 >> 172.21.15.40:0/3359994341 conn(0x7f891c08eca0 msgr2=0x7f891c09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 -- 172.21.15.40:0/3359994341 shutdown_connections 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.011+0000 7f892370a640 1 -- 172.21.15.40:0/3359994341 wait complete. 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 Processor -- start 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 -- start start 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f891c0b60b0 0x7f891c0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0d57b0 con 0x7f891c163aa0 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0d5630 con 0x7f891c0b60b0 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f892370a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f891c0d5930 con 0x7f891c15f680 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f8921ca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f8921ca7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45208/0 (socket says 172.21.15.40:45208) 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.013+0000 7f8921ca7640 1 -- 172.21.15.40:0/3301819421 learned_addr learned my addr 172.21.15.40:0/3301819421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f89214a6640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f891c0b60b0 0x7f891c0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8920ca5640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 -- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 msgr2=0x7f891c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 -- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f891c0b60b0 msgr2=0x7f891c0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f891c0b60b0 0x7f891c0cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 -- 172.21.15.40:0/3301819421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0cf150 con 0x7f891c163aa0 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8920ca5640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8921ca7640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c7a0 tx=0x7f890c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c003b70 con 0x7f891c163aa0 2024-04-26T21:20:24.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890c003d30 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c00aa90 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c0ce940 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.014+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c160ef0 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.015+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c06a290 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.016+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f890c00ac30 con 0x7f891c163aa0 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.016+0000 7f8919ffb640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.016+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f890c08bab0 con 0x7f891c163aa0 2024-04-26T21:20:24.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.016+0000 7f89214a6640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.017+0000 7f89214a6640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 0x7f890806edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8914009ad0 tx=0x7f8914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.020+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f890c059ee0 con 0x7f891c163aa0 2024-04-26T21:20:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.204+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f891c0b3f80 con 0x7f891c163aa0 2024-04-26T21:20:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.205+0000 7f8919ffb640 1 -- 172.21.15.40:0/3301819421 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7f890c0597e0 con 0x7f891c163aa0 2024-04-26T21:20:24.206 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:20:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 msgr2=0x7f890806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 0x7f890806edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8914009ad0 tx=0x7f8914008040 comp rx=0 tx=0).stop 2024-04-26T21:20:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 msgr2=0x7f891c15bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c7a0 tx=0x7f890c00cc60 comp rx=0 tx=0).stop 2024-04-26T21:20:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 shutdown_connections 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c163aa0 0x7f891c15bf50 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f891c15f680 0x7f891c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 --2- 172.21.15.40:0/3301819421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f891c0b60b0 0x7f891c0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 >> 172.21.15.40:0/3301819421 conn(0x7f891c08eca0 msgr2=0x7f891c0d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 shutdown_connections 2024-04-26T21:20:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.207+0000 7f892370a640 1 -- 172.21.15.40:0/3301819421 wait complete. 2024-04-26T21:20:24.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:24.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.318+0000 7f0c87c93640 1 Processor -- start 2024-04-26T21:20:24.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.318+0000 7f0c87c93640 1 -- start start 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c87c93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0c8015f370 0x7f0c800cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c87c93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0c800d3ee0 0x7f0c800b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c87c93640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800d03a0 con 0x7f0c8015f750 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c87c93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0c800b4d10 con 0x7f0c800d3ee0 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c87c93640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0c800d0680 con 0x7f0c8015f370 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c86230640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0c800d3ee0 0x7f0c800b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c8522e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0c8015f750 0x7f0c800d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50672/0 (socket says 172.21.15.40:50672) 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c86230640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0c800d3ee0 0x7f0c800b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45212/0 (socket says 172.21.15.40:45212) 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.319+0000 7f0c8522e640 1 -- 172.21.15.40:0/3625245594 learned_addr learned my addr 172.21.15.40:0/3625245594 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c85a2f640 1 --2- 172.21.15.40:0/3625245594 >> v2:172.21.15.184:3300/0 conn(0x7f0c8015f370 0x7f0c800cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c86230640 1 -- 172.21.15.40:0/3625245594 >> v2:172.21.15.184:3300/0 conn(0x7f0c8015f370 msgr2=0x7f0c800cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c86230640 1 --2- 172.21.15.40:0/3625245594 >> v2:172.21.15.184:3300/0 conn(0x7f0c8015f370 0x7f0c800cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c86230640 1 -- 172.21.15.40:0/3625245594 >> v1:172.21.15.40:6789/0 conn(0x7f0c8015f750 legacy=0x7f0c800d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c86230640 1 -- 172.21.15.40:0/3625245594 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c800d0960 con 0x7f0c800d3ee0 2024-04-26T21:20:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.320+0000 7f0c84a2d640 1 -- 172.21.15.40:0/3625245594 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270588151 0 0) 0x7f0c800d03a0 con 0x7f0c8015f750 2024-04-26T21:20:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c86230640 1 --2- 172.21.15.40:0/3625245594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c800d3ee0 0x7f0c800b47d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0c800d2d10 tx=0x7f0c7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83540adc77dc4b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c84a2d640 1 -- 172.21.15.40:0/3625245594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78007740 con 0x7f0c800d3ee0 2024-04-26T21:20:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c84a2d640 1 -- 172.21.15.40:0/3625245594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c780078e0 con 0x7f0c800d3ee0 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c84a2d640 1 -- 172.21.15.40:0/3625245594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78007cb0 con 0x7f0c800d3ee0 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 -- 172.21.15.40:0/3625245594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c800d3ee0 msgr2=0x7f0c800b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 --2- 172.21.15.40:0/3625245594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c800d3ee0 0x7f0c800b47d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0c800d2d10 tx=0x7f0c7802a680 comp rx=0 tx=0).stop 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 -- 172.21.15.40:0/3625245594 shutdown_connections 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 --2- 172.21.15.40:0/3625245594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c800d3ee0 0x7f0c800b47d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 --2- 172.21.15.40:0/3625245594 >> v2:172.21.15.184:3300/0 conn(0x7f0c8015f370 0x7f0c800cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 -- 172.21.15.40:0/3625245594 >> 172.21.15.40:0/3625245594 conn(0x7f0c8008ec10 msgr2=0x7f0c8009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 -- 172.21.15.40:0/3625245594 shutdown_connections 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.321+0000 7f0c87c93640 1 -- 172.21.15.40:0/3625245594 wait complete. 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 Processor -- start 2024-04-26T21:20:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 -- start start 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 0x7f0c801620d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 0x7f0c8015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800b51c0 con 0x7f0c8015f370 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800d0f20 con 0x7f0c800d3ee0 2024-04-26T21:20:24.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c87c93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800b5340 con 0x7f0c8015f750 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.323+0000 7f0c8522e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45222/0 (socket says 172.21.15.40:45222) 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 -- 172.21.15.40:0/146186362 learned_addr learned my addr 172.21.15.40:0/146186362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c85a2f640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 0x7f0c801620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c86230640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 0x7f0c8015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 -- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 msgr2=0x7f0c8015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 0x7f0c8015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 -- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 msgr2=0x7f0c801620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 0x7f0c801620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 -- 172.21.15.40:0/146186362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c80160190 con 0x7f0c8015f370 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c85a2f640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 0x7f0c801620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c86230640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 0x7f0c8015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:24.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c8522e640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c430 tx=0x7f0c7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700074e0 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c700047e0 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c80160470 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004bd0 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.324+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c80160a30 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.325+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8006a290 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.326+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c70011070 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.326+0000 7f0c767fc640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 0x7f0c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.326+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f0c7008b8b0 con 0x7f0c8015f370 2024-04-26T21:20:24.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.327+0000 7f0c85a2f640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 0x7f0c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.327+0000 7f0c85a2f640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 0x7f0c6406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006fd0 tx=0x7f0c7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.330+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0c7000e280 con 0x7f0c8015f370 2024-04-26T21:20:24.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.557+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c800b73c0 con 0x7f0c8015f370 2024-04-26T21:20:24.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.557+0000 7f0c767fc640 1 -- 172.21.15.40:0/146186362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5189 (secure 0 0 0) 0x7f0c70059c40 con 0x7f0c8015f370 2024-04-26T21:20:24.559 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:24.559 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:19.759313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:24.559 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 79 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.559+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 msgr2=0x7f0c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.559+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 0x7f0c6406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006fd0 tx=0x7f0c7c008040 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.559+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 msgr2=0x7f0c800cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.559+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c430 tx=0x7f0c7000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 shutdown_connections 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0c6406c8f0 0x7f0c6406edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015f750 0x7f0c8015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c8015f370 0x7f0c800cb300 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 --2- 172.21.15.40:0/146186362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c800d3ee0 0x7f0c801620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 >> 172.21.15.40:0/146186362 conn(0x7f0c8008ec10 msgr2=0x7f0c800b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 shutdown_connections 2024-04-26T21:20:24.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.560+0000 7f0c87c93640 1 -- 172.21.15.40:0/146186362 wait complete. 2024-04-26T21:20:24.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:24.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 Processor -- start 2024-04-26T21:20:24.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 -- start start 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9440afe30 0x7fd94416a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9440aed90 con 0x7fd9440b47d0 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9440af070 con 0x7fd94415f4a0 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.671+0000 7fd94967e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd94415f880 con 0x7fd9440afe30 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9440afe30 0x7fd94416a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9427fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd9440b47d0 0x7fd9440b0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50686/0 (socket says 172.21.15.40:50686) 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9427fc640 1 -- 172.21.15.40:0/2055662043 learned_addr learned my addr 172.21.15.40:0/2055662043 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9437fe640 1 -- 172.21.15.40:0/2055662043 >> v1:172.21.15.184:6789/0 conn(0x7fd94415f4a0 legacy=0x7fd9440b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9437fe640 1 -- 172.21.15.40:0/2055662043 >> v1:172.21.15.40:6789/0 conn(0x7fd9440b47d0 legacy=0x7fd9440b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd9437fe640 1 -- 172.21.15.40:0/2055662043 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9440af350 con 0x7fd9440afe30 2024-04-26T21:20:24.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.672+0000 7fd941ffb640 1 -- 172.21.15.40:0/2055662043 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534474160 0 0) 0x7fd9440aed90 con 0x7fd9440b47d0 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd9437fe640 1 --2- 172.21.15.40:0/2055662043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440afe30 0x7fd94416a0e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009d90 tx=0x7fd93c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade90ab785dbbf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd941ffb640 1 -- 172.21.15.40:0/2055662043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c007740 con 0x7fd9440afe30 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd941ffb640 1 -- 172.21.15.40:0/2055662043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c0078e0 con 0x7fd9440afe30 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd941ffb640 1 -- 172.21.15.40:0/2055662043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c007cb0 con 0x7fd9440afe30 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 -- 172.21.15.40:0/2055662043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440afe30 msgr2=0x7fd94416a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 --2- 172.21.15.40:0/2055662043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440afe30 0x7fd94416a0e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009d90 tx=0x7fd93c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 -- 172.21.15.40:0/2055662043 shutdown_connections 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 --2- 172.21.15.40:0/2055662043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440afe30 0x7fd94416a0e0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 -- 172.21.15.40:0/2055662043 >> 172.21.15.40:0/2055662043 conn(0x7fd9440b6800 msgr2=0x7fd9440738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 -- 172.21.15.40:0/2055662043 shutdown_connections 2024-04-26T21:20:24.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.673+0000 7fd94967e640 1 -- 172.21.15.40:0/2055662043 wait complete. 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 Processor -- start 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 -- start start 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 0x7fd94409bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 0x7fd9441490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9440aea80 con 0x7fd9440b47d0 2024-04-26T21:20:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9440af9a0 con 0x7fd9440afe30 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd94967e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9440b6320 con 0x7fd94415f4a0 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd9427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd9427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45230/0 (socket says 172.21.15.40:45230) 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.675+0000 7fd9427fc640 1 -- 172.21.15.40:0/4236545341 learned_addr learned my addr 172.21.15.40:0/4236545341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd942ffd640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 0x7fd94409bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 -- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 msgr2=0x7fd9441490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9437fe640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 0x7fd9441490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 0x7fd9441490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 -- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 msgr2=0x7fd94409bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 0x7fd94409bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 -- 172.21.15.40:0/4236545341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441f4a60 con 0x7fd9440b47d0 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd942ffd640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 0x7fd94409bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9437fe640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 0x7fd9441490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd9427fc640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c4a0 tx=0x7fd93800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938018070 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9380074e0 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd944149920 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938004a20 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.676+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd944149ee0 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.677+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94406a290 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.678+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd93801b070 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.678+0000 7fd92b7fe640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 0x7fd93006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.678+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fd93808ae30 con 0x7fd9440b47d0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.678+0000 7fd942ffd640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 0x7fd93006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.679+0000 7fd942ffd640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 0x7fd93006edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd934000f60 tx=0x7fd934009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.682+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd938059260 con 0x7fd9440b47d0 2024-04-26T21:20:24.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.868+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9440b29d0 con 0x7fd9440b47d0 2024-04-26T21:20:24.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.869+0000 7fd92b7fe640 1 -- 172.21.15.40:0/4236545341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7fd938058b60 con 0x7fd9440b47d0 2024-04-26T21:20:24.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:24.871 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 msgr2=0x7fd93006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 0x7fd93006edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd934000f60 tx=0x7fd934009040 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 msgr2=0x7fd9441f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c4a0 tx=0x7fd93800c960 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 shutdown_connections 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd93006c8f0 0x7fd93006edb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd94415f4a0 0x7fd9441490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9440b47d0 0x7fd9441f4520 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 --2- 172.21.15.40:0/4236545341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9440afe30 0x7fd94409bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 >> 172.21.15.40:0/4236545341 conn(0x7fd9440b6800 msgr2=0x7fd9440b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 shutdown_connections 2024-04-26T21:20:24.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.872+0000 7fd94967e640 1 -- 172.21.15.40:0/4236545341 wait complete. 2024-04-26T21:20:24.884 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:20:24.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.984+0000 7f190b94e640 1 Processor -- start 2024-04-26T21:20:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.984+0000 7f190b94e640 1 -- start start 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f190b94e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f190415f340 0x7f19040b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f190b94e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19040d5610 0x7f19040d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f190b94e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19040b2510 con 0x7f190415f720 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f190b94e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f19040b27f0 con 0x7f19040d5610 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f190b94e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f19040b2ad0 con 0x7f190415f340 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19040d5610 0x7f19040d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19040d5610 0x7f19040d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45240/0 (socket says 172.21.15.40:45240) 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1908ee9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f190415f720 0x7f19040d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50694/0 (socket says 172.21.15.40:50694) 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 -- 172.21.15.40:0/3588701684 learned_addr learned my addr 172.21.15.40:0/3588701684 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f19096ea640 1 --2- 172.21.15.40:0/3588701684 >> v2:172.21.15.184:3300/0 conn(0x7f190415f340 0x7f19040b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 -- 172.21.15.40:0/3588701684 >> v2:172.21.15.184:3300/0 conn(0x7f190415f340 msgr2=0x7f19040b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 --2- 172.21.15.40:0/3588701684 >> v2:172.21.15.184:3300/0 conn(0x7f190415f340 0x7f19040b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 -- 172.21.15.40:0/3588701684 >> v1:172.21.15.40:6789/0 conn(0x7f190415f720 legacy=0x7f19040d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1903fff640 1 -- 172.21.15.40:0/3588701684 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713047376 0 0) 0x7f19040b2510 con 0x7f190415f720 2024-04-26T21:20:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.985+0000 7f1909eeb640 1 -- 172.21.15.40:0/3588701684 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19040b2db0 con 0x7f19040d5610 2024-04-26T21:20:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.986+0000 7f1909eeb640 1 --2- 172.21.15.40:0/3588701684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040d3d40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f19040d6000 tx=0x7f18f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49c1b99f7427e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.986+0000 7f1903fff640 1 -- 172.21.15.40:0/3588701684 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037070 con 0x7f19040d5610 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.986+0000 7f1903fff640 1 -- 172.21.15.40:0/3588701684 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f4032440 con 0x7f19040d5610 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.986+0000 7f1903fff640 1 -- 172.21.15.40:0/3588701684 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4032810 con 0x7f19040d5610 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 -- 172.21.15.40:0/3588701684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 msgr2=0x7f19040d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 --2- 172.21.15.40:0/3588701684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040d3d40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f19040d6000 tx=0x7f18f402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 -- 172.21.15.40:0/3588701684 shutdown_connections 2024-04-26T21:20:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 --2- 172.21.15.40:0/3588701684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040d3d40 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 --2- 172.21.15.40:0/3588701684 >> v2:172.21.15.184:3300/0 conn(0x7f190415f340 0x7f19040b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 -- 172.21.15.40:0/3588701684 >> 172.21.15.40:0/3588701684 conn(0x7f190408ec70 msgr2=0x7f190409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 -- 172.21.15.40:0/3588701684 shutdown_connections 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.987+0000 7f190b94e640 1 -- 172.21.15.40:0/3588701684 wait complete. 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.988+0000 7f190b94e640 1 Processor -- start 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.988+0000 7f190b94e640 1 -- start start 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 0x7f19040ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f190415f720 0x7f19041572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19040b3940 con 0x7f19040d5610 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19040b37c0 con 0x7f190415f340 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f190b94e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f19040b3ac0 con 0x7f190415f720 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45248/0 (socket says 172.21.15.40:45248) 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 -- 172.21.15.40:0/2482138367 learned_addr learned my addr 172.21.15.40:0/2482138367 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f1908ee9640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 0x7f19040ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f1909eeb640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f190415f720 0x7f19041572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 -- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f190415f720 msgr2=0x7f19041572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f190415f720 0x7f19041572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 -- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 msgr2=0x7f19040ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 0x7f19040ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f19096ea640 1 -- 172.21.15.40:0/2482138367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19040c9c90 con 0x7f19040d5610 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.989+0000 7f1908ee9640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 0x7f19040ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f19096ea640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007f20 tx=0x7f18fc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc014620 con 0x7f19040d5610 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc019070 con 0x7f19040d5610 2024-04-26T21:20:24.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19040c9f70 con 0x7f19040d5610 2024-04-26T21:20:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc014a00 con 0x7f19040d5610 2024-04-26T21:20:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.990+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19040ca530 con 0x7f19040d5610 2024-04-26T21:20:24.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.991+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190409eac0 con 0x7f19040d5610 2024-04-26T21:20:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.991+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18fc014ba0 con 0x7f19040d5610 2024-04-26T21:20:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.992+0000 7f1901ffb640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 0x7f18f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.992+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f18fc08c470 con 0x7f19040d5610 2024-04-26T21:20:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.992+0000 7f1908ee9640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 0x7f18f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.993+0000 7f1908ee9640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 0x7f18f006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f18f8007fd0 tx=0x7f18f8007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:24.996+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f190409eac0 con 0x7f19040d5610 2024-04-26T21:20:25.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.221+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f190406a290 con 0x7f19040d5610 2024-04-26T21:20:25.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.222+0000 7f1901ffb640 1 -- 172.21.15.40:0/2482138367 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5189 (secure 0 0 0) 0x7f18fc05a8a0 con 0x7f19040d5610 2024-04-26T21:20:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:25.223 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:19.759313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:25.223 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 79 2024-04-26T21:20:25.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 msgr2=0x7f18f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 0x7f18f006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f18f8007fd0 tx=0x7f18f8007dc0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 msgr2=0x7f19040cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007f20 tx=0x7f18fc00e570 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 shutdown_connections 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f18f006c8f0 0x7f18f006edb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f190415f720 0x7f19041572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190415f340 0x7f19040ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 --2- 172.21.15.40:0/2482138367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19040d5610 0x7f19040cb520 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 >> 172.21.15.40:0/2482138367 conn(0x7f190408ec70 msgr2=0x7f19040b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 shutdown_connections 2024-04-26T21:20:25.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.224+0000 7f190b94e640 1 -- 172.21.15.40:0/2482138367 wait complete. 2024-04-26T21:20:25.236 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:25.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.335+0000 7f7e3bba4640 1 Processor -- start 2024-04-26T21:20:25.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.336+0000 7f7e3bba4640 1 -- start start 2024-04-26T21:20:25.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.336+0000 7f7e3bba4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7e340b44e0 0x7f7e3406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.336+0000 7f7e3bba4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e340b4a10 con 0x7f7e3406d2b0 2024-04-26T21:20:25.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.336+0000 7f7e3bba4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e340b4cf0 con 0x7f7e3415f1e0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.336+0000 7f7e3bba4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3406eeb0 con 0x7f7e340b44e0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3a141640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e3406d2b0 0x7f7e340b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50698/0 (socket says 172.21.15.40:50698) 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3a141640 1 -- 172.21.15.40:0/1340353804 learned_addr learned my addr 172.21.15.40:0/1340353804 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420643789 0 0) 0x7f7e340b4a10 con 0x7f7e3406d2b0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e20003520 con 0x7f7e3406d2b0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3913f640 1 --2- 172.21.15.40:0/1340353804 >> v2:172.21.15.184:3300/0 conn(0x7f7e340b44e0 0x7f7e3406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 953750073 0 0) 0x7f7e20003520 con 0x7f7e3406d2b0 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 >> v2:172.21.15.184:3300/0 conn(0x7f7e340b44e0 msgr2=0x7f7e3406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 --2- 172.21.15.40:0/1340353804 >> v2:172.21.15.184:3300/0 conn(0x7f7e340b44e0 0x7f7e3406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 >> v1:172.21.15.184:6789/0 conn(0x7f7e3415f1e0 legacy=0x7f7e340ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e340b4fd0 con 0x7f7e3406d2b0 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7e2c0019f0 con 0x7f7e3406d2b0 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 >> v1:172.21.15.40:6789/0 conn(0x7f7e3406d2b0 legacy=0x7f7e340b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e200039b0 0x7f7e20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e20004800 0x7f7e20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3406eeb0 con 0x7f7e200051c0 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2000c060 con 0x7f7e20004800 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2000c260 con 0x7f7e200039b0 2024-04-26T21:20:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3a141640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e39940640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e200039b0 0x7f7e20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.337+0000 7f7e3913f640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e20004800 0x7f7e20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 -- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e200039b0 msgr2=0x7f7e20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e200039b0 0x7f7e20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 -- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e20004800 msgr2=0x7f7e20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e20004800 0x7f7e20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 -- 172.21.15.40:0/1340353804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e20003520 con 0x7f7e200051c0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e39940640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e200039b0 0x7f7e20003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 -- 172.21.15.40:0/1340353804 reap_dead start 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.338+0000 7f7e3a141640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e2000b9c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00a830 tx=0x7f7e20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65da588bac6eab8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30001370 con 0x7f7e200051c0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c02f070 con 0x7f7e200051c0 2024-04-26T21:20:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3893e640 1 -- 172.21.15.40:0/1340353804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00a480 con 0x7f7e200051c0 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 -- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 msgr2=0x7f7e2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e2000b9c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00a830 tx=0x7f7e20005830 comp rx=0 tx=0).stop 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 -- 172.21.15.40:0/1340353804 shutdown_connections 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/1340353804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e2000b9c0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 -- 172.21.15.40:0/1340353804 >> 172.21.15.40:0/1340353804 conn(0x7f7e340728a0 msgr2=0x7f7e34072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 -- 172.21.15.40:0/1340353804 shutdown_connections 2024-04-26T21:20:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.339+0000 7f7e3bba4640 1 -- 172.21.15.40:0/1340353804 wait complete. 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.340+0000 7f7e3bba4640 1 Processor -- start 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.340+0000 7f7e3bba4640 1 -- start start 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e34162520 0x7f7e341629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3415b730 0x7f7e3415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3415f5c0 con 0x7f7e200051c0 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e340b5590 con 0x7f7e3415b730 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e3bba4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3415d090 con 0x7f7e34162520 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45256/0 (socket says 172.21.15.40:45256) 2024-04-26T21:20:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 -- 172.21.15.40:0/3820813417 learned_addr learned my addr 172.21.15.40:0/3820813417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 -- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e34162520 msgr2=0x7f7e341629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e34162520 0x7f7e341629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 -- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3415b730 msgr2=0x7f7e3415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3415b730 0x7f7e3415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.341+0000 7f7e39940640 1 -- 172.21.15.40:0/3820813417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3415c480 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e39940640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000f5f0 tx=0x7f7e3000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e300052e0 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e30018070 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30003810 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3415c760 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.342+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3415cd20 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.343+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e340aff10 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.343+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e300039b0 con 0x7f7e200051c0 2024-04-26T21:20:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.344+0000 7f7e2a7fc640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 0x7f7e1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.344+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f7e3008b850 con 0x7f7e200051c0 2024-04-26T21:20:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.344+0000 7f7e3913f640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 0x7f7e1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.344+0000 7f7e3913f640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 0x7f7e1806ede0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7e3415d4f0 tx=0x7f7e24009320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.348+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e340aff10 con 0x7f7e200051c0 2024-04-26T21:20:25.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.569+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e3406a290 con 0x7f7e200051c0 2024-04-26T21:20:25.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.570+0000 7f7e2a7fc640 1 -- 172.21.15.40:0/3820813417 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5189 (secure 0 0 0) 0x7f7e30059c80 con 0x7f7e200051c0 2024-04-26T21:20:25.571 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:25.571 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:19:59.077853+0000","modified":"2024-04-26T21:20:19.759313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4242446604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4242446604},{"type":"v1","addr":"172.21.15.40:6835","nonce":4242446604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8475,"qdb_cluster":[8475]},"id":9}]} 2024-04-26T21:20:25.571 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 79 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 msgr2=0x7f7e1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 0x7f7e1806ede0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7e3415d4f0 tx=0x7f7e24009320 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 msgr2=0x7f7e341f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000f5f0 tx=0x7f7e3000fab0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 shutdown_connections 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7e1806c920 0x7f7e1806ede0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3415b730 0x7f7e3415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e34162520 0x7f7e341629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 --2- 172.21.15.40:0/3820813417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e200051c0 0x7f7e341f5a00 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 >> 172.21.15.40:0/3820813417 conn(0x7f7e340728a0 msgr2=0x7f7e341555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:25.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 shutdown_connections 2024-04-26T21:20:25.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.572+0000 7f7e3bba4640 1 -- 172.21.15.40:0/3820813417 wait complete. 2024-04-26T21:20:25.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.683+0000 7ff26f0cf640 1 Processor -- start 2024-04-26T21:20:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.683+0000 7ff26f0cf640 1 -- start start 2024-04-26T21:20:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26f0cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 0x7ff2680cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26f0cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2680d03a0 con 0x7ff26815f750 2024-04-26T21:20:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26f0cf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2680d0680 con 0x7ff2680d3ee0 2024-04-26T21:20:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26f0cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680d0960 con 0x7ff26815f370 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff26815f750 0x7ff2680d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50708/0 (socket says 172.21.15.40:50708) 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25ffff640 1 -- 172.21.15.40:0/2953000740 learned_addr learned my addr 172.21.15.40:0/2953000740 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26ce6b640 1 --2- 172.21.15.40:0/2953000740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 0x7ff2680cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639636329 0 0) 0x7ff2680d03a0 con 0x7ff26815f750 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff250003520 con 0x7ff26815f750 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079378160 0 0) 0x7ff2680d0680 con 0x7ff2680d3ee0 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2680d03a0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2953000740 >> v1:172.21.15.184:6789/0 conn(0x7ff2680d3ee0 legacy=0x7ff2680b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2953000740 >> v1:172.21.15.40:6789/0 conn(0x7ff26815f750 legacy=0x7ff2680d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.684+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2953000740 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680d0c40 con 0x7ff26815f370 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.685+0000 7ff26ce6b640 1 --2- 172.21.15.40:0/2953000740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 0x7ff2680cfd70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2640097d0 tx=0x7ff26402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b668b770d026a1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.685+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264037070 con 0x7ff26815f370 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.685+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff264032440 con 0x7ff26815f370 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.685+0000 7ff25f7fe640 1 -- 172.21.15.40:0/2953000740 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264032810 con 0x7ff26815f370 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2953000740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 msgr2=0x7ff2680cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2953000740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 0x7ff2680cfd70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2640097d0 tx=0x7ff26402a180 comp rx=0 tx=0).stop 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2953000740 shutdown_connections 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2953000740 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f370 0x7ff2680cfd70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2953000740 >> 172.21.15.40:0/2953000740 conn(0x7ff26808ec10 msgr2=0x7ff26809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:25.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2953000740 shutdown_connections 2024-04-26T21:20:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.686+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2953000740 wait complete. 2024-04-26T21:20:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.687+0000 7ff26f0cf640 1 Processor -- start 2024-04-26T21:20:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.687+0000 7ff26f0cf640 1 -- start start 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26815f370 0x7ff26815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 0x7ff2680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2680b4950 con 0x7ff2680d3ee0 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2680b47d0 con 0x7ff26815f370 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26f0cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680b4ad0 con 0x7ff26815f750 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45264/0 (socket says 172.21.15.40:45264) 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2425127461 learned_addr learned my addr 172.21.15.40:0/2425127461 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26d66c640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 0x7ff2680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff25ffff640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26815f370 0x7ff26815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 msgr2=0x7ff2680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 0x7ff2680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26815f370 msgr2=0x7ff26815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26815f370 0x7ff26815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 -- 172.21.15.40:0/2425127461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680b8d40 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26d66c640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 0x7ff2680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.688+0000 7ff26ce6b640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff2640098f0 tx=0x7ff2640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.689+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264041020 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.689+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26403d6e0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.689+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2680b8fe0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.689+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264037040 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.689+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2680b95a0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.690+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2680b97e0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.690+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff264044020 con 0x7ff2680d3ee0 2024-04-26T21:20:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.690+0000 7ff25d7fa640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.691+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7ff2640ac1b0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.691+0000 7ff25ffff640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.691+0000 7ff25ffff640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff2680cbd10 tx=0x7ff258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.695+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2680b97e0 con 0x7ff2680d3ee0 2024-04-26T21:20:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.881+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff26806a290 con 0x7ff2680d3ee0 2024-04-26T21:20:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.882+0000 7ff25d7fa640 1 -- 172.21.15.40:0/2425127461 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16679 (secure 0 0 0) 0x7ff26404b030 con 0x7ff2680d3ee0 2024-04-26T21:20:25.883 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:25.883 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":122,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:00.728079+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:19:57.355243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-26T21:19:58.032686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 msgr2=0x7ff25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff2680cbd10 tx=0x7ff258009040 comp rx=0 tx=0).stop 2024-04-26T21:20:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 msgr2=0x7ff2680b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff2640098f0 tx=0x7ff2640046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 shutdown_connections 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26815f750 0x7ff2680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26815f370 0x7ff26815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 --2- 172.21.15.40:0/2425127461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2680d3ee0 0x7ff2680b8800 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 >> 172.21.15.40:0/2425127461 conn(0x7ff26808ec10 msgr2=0x7ff26815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 shutdown_connections 2024-04-26T21:20:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.884+0000 7ff26f0cf640 1 -- 172.21.15.40:0/2425127461 wait complete. 2024-04-26T21:20:25.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:20:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.997+0000 7f21643b4640 1 Processor -- start 2024-04-26T21:20:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.997+0000 7f21643b4640 1 -- start start 2024-04-26T21:20:25.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f21643b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 0x7f215c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f21643b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c0d1740 con 0x7f215c0cf2e0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f21643b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c0d1a20 con 0x7f215c0d11b0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f21643b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f215c0d1d00 con 0x7f215c06e490 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162150640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f215c0cf2e0 0x7f215c06d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50712/0 (socket says 172.21.15.40:50712) 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162150640 1 -- 172.21.15.40:0/1966015573 learned_addr learned my addr 172.21.15.40:0/1966015573 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162951640 1 --2- 172.21.15.40:0/1966015573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 0x7f215c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789220971 0 0) 0x7f215c0d1740 con 0x7f215c0cf2e0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2148003520 con 0x7f215c0cf2e0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343417298 0 0) 0x7f215c0d1a20 con 0x7f215c0d11b0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f215c0d1740 con 0x7f215c0d11b0 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162951640 1 -- 172.21.15.40:0/1966015573 >> v1:172.21.15.184:6789/0 conn(0x7f215c0d11b0 legacy=0x7f215c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162951640 1 -- 172.21.15.40:0/1966015573 >> v1:172.21.15.40:6789/0 conn(0x7f215c0cf2e0 legacy=0x7f215c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.998+0000 7f2162951640 1 -- 172.21.15.40:0/1966015573 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0d1fe0 con 0x7f215c06e490 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.999+0000 7f2162951640 1 --2- 172.21.15.40:0/1966015573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 0x7f215c15ebe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2154009df0 tx=0x7f215402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74aa6c40eeaa4db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.999+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154007740 con 0x7f215c06e490 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.999+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21540078e0 con 0x7f215c06e490 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:25.999+0000 7f216114e640 1 -- 172.21.15.40:0/1966015573 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154007c90 con 0x7f215c06e490 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 -- 172.21.15.40:0/1966015573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 msgr2=0x7f215c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 --2- 172.21.15.40:0/1966015573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 0x7f215c15ebe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2154009df0 tx=0x7f215402a710 comp rx=0 tx=0).stop 2024-04-26T21:20:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 -- 172.21.15.40:0/1966015573 shutdown_connections 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 --2- 172.21.15.40:0/1966015573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c06e490 0x7f215c15ebe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 -- 172.21.15.40:0/1966015573 >> 172.21.15.40:0/1966015573 conn(0x7f215c0b4a60 msgr2=0x7f215c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 -- 172.21.15.40:0/1966015573 shutdown_connections 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.000+0000 7f21643b4640 1 -- 172.21.15.40:0/1966015573 wait complete. 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.001+0000 7f21643b4640 1 Processor -- start 2024-04-26T21:20:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.001+0000 7f21643b4640 1 -- start start 2024-04-26T21:20:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f215c06e490 0x7f215c0b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c0cf2e0 0x7f215c0b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f215c0d11b0 0x7f215c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0cf840 con 0x7f215c0d11b0 2024-04-26T21:20:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0cf6c0 con 0x7f215c06e490 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f21643b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f215c0ce660 con 0x7f215c0cf2e0 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f215c0d11b0 0x7f215c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f215c0d11b0 0x7f215c0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45280/0 (socket says 172.21.15.40:45280) 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 -- 172.21.15.40:0/4173604271 learned_addr learned my addr 172.21.15.40:0/4173604271 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f216194f640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c0cf2e0 0x7f215c0b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162150640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f215c06e490 0x7f215c0b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 -- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c0cf2e0 msgr2=0x7f215c0b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c0cf2e0 0x7f215c0b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 -- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f215c06e490 msgr2=0x7f215c0b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f215c06e490 0x7f215c0b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 -- 172.21.15.40:0/4173604271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0b7060 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f216194f640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215c0cf2e0 0x7f215c0b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.002+0000 7f2162951640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f215c0d11b0 0x7f215c0cd1b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2154009f10 tx=0x7f2154004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.003+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154009450 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.003+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2154004240 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.003+0000 7f21643b4640 1 -- 172.21.15.40:0/4173604271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c0b7340 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.003+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154004610 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.003+0000 7f21643b4640 1 -- 172.21.15.40:0/4173604271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c0b7900 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.004+0000 7f21643b4640 1 -- 172.21.15.40:0/4173604271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c06a290 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.004+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2154039040 con 0x7f215c0d11b0 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.005+0000 7f2152ffd640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f214006c8f0 0x7f214006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.005+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f21540acad0 con 0x7f215c0d11b0 2024-04-26T21:20:26.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.005+0000 7f2162150640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f214006c8f0 0x7f214006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.005+0000 7f2162150640 1 --2- 172.21.15.40:0/4173604271 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f214006c8f0 0x7f214006edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2158000f60 tx=0x7f2158009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.009+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2154040030 con 0x7f215c0d11b0 2024-04-26T21:20:26.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.233+0000 7f21643b4640 1 -- 172.21.15.40:0/4173604271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f215c069500 con 0x7f215c0d11b0 2024-04-26T21:20:26.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.234+0000 7f2152ffd640 1 -- 172.21.15.40:0/4173604271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v79) ==== 316+0+0 (secure 0 0 0) 0x7f215407af00 con 0x7f215c0d11b0 2024-04-26T21:20:26.235 INFO:teuthology.orchestra.run.smithi040.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-04-26T21:20:26.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:20:26.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-04-26T21:20:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.341+0000 7f290ce93640 1 Processor -- start 2024-04-26T21:20:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.341+0000 7f290ce93640 1 -- start start 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290ce93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f290815f260 0x7f290806d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290ce93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29080b4e00 0x7f29080ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290ce93640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f290815f640 con 0x7f2908070560 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290ce93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f290806c2a0 con 0x7f29080b4e00 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290ce93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f290806c580 con 0x7f290815f260 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f2906d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29080b4e00 0x7f29080ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f2906d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29080b4e00 0x7f29080ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45286/0 (socket says 172.21.15.40:45286) 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f2906d9d640 1 -- 172.21.15.40:0/2001195992 learned_addr learned my addr 172.21.15.40:0/2001195992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.342+0000 7f290659c640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f290815f260 0x7f290806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f2906d9d640 1 -- 172.21.15.40:0/2001195992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f290815f260 msgr2=0x7f290806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f2906d9d640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f290815f260 0x7f290806d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f2906d9d640 1 -- 172.21.15.40:0/2001195992 >> v1:172.21.15.184:6789/0 conn(0x7f2908070560 legacy=0x7f29080b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f2906d9d640 1 -- 172.21.15.40:0/2001195992 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290806c860 con 0x7f29080b4e00 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f2906d9d640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29080b4e00 0x7f29080ae8b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009e50 tx=0x7f28f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a7b783b5650707 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f290559a640 1 -- 172.21.15.40:0/2001195992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8036070 con 0x7f29080b4e00 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f290559a640 1 -- 172.21.15.40:0/2001195992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f8031600 con 0x7f29080b4e00 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.343+0000 7f290559a640 1 -- 172.21.15.40:0/2001195992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80319d0 con 0x7f29080b4e00 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 -- 172.21.15.40:0/2001195992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29080b4e00 msgr2=0x7f29080ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29080b4e00 0x7f29080ae8b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009e50 tx=0x7f28f802a710 comp rx=0 tx=0).stop 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 -- 172.21.15.40:0/2001195992 shutdown_connections 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29080b4e00 0x7f29080ae8b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 --2- 172.21.15.40:0/2001195992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f290815f260 0x7f290806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 -- 172.21.15.40:0/2001195992 >> 172.21.15.40:0/2001195992 conn(0x7f29080d1660 msgr2=0x7f29080d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 -- 172.21.15.40:0/2001195992 shutdown_connections 2024-04-26T21:20:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.344+0000 7f290ce93640 1 -- 172.21.15.40:0/2001195992 wait complete. 2024-04-26T21:20:26.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.345+0000 7f290ce93640 1 Processor -- start 2024-04-26T21:20:26.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.345+0000 7f290ce93640 1 -- start start 2024-04-26T21:20:26.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 0x7f290815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29080b4e00 0x7f29080cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f290806d100 con 0x7f290815f260 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f290806ce20 con 0x7f2908070560 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290ce93640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f290806db60 con 0x7f29080b4e00 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f2906d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f2906d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45290/0 (socket says 172.21.15.40:45290) 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f2906d9d640 1 -- 172.21.15.40:0/1826377129 learned_addr learned my addr 172.21.15.40:0/1826377129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f2905d9b640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29080b4e00 0x7f29080cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.346+0000 7f290659c640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 0x7f290815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 -- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29080b4e00 msgr2=0x7f29080cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29080b4e00 0x7f29080cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 -- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 msgr2=0x7f290815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 0x7f290815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 -- 172.21.15.40:0/1826377129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29080d0e20 con 0x7f290815f260 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f290659c640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 0x7f290815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f2906d9d640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f28f8034040 tx=0x7f28f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8036050 con 0x7f290815f260 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2908155560 con 0x7f290815f260 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f8042070 con 0x7f290815f260 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8009510 con 0x7f290815f260 2024-04-26T21:20:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.347+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29080d0140 con 0x7f290815f260 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.348+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29080b5e00 con 0x7f290815f260 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.349+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28f8007bf0 con 0x7f290815f260 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.349+0000 7f28f6ffd640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 0x7f28e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.349+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f28f80ac920 con 0x7f290815f260 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.349+0000 7f290659c640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 0x7f28e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.350+0000 7f290659c640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 0x7f28e806ede0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2900009b90 tx=0x7f2900008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.353+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29080b5e00 con 0x7f290815f260 2024-04-26T21:20:26.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.577+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f290806a290 con 0x7f290815f260 2024-04-26T21:20:26.578 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:20:26.577+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:20:26.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.579+0000 7f28f6ffd640 1 -- 172.21.15.40:0/1826377129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 191+0+0 (secure 0 0 0) 0x7f28f807acd0 con 0x7f290815f260 2024-04-26T21:20:26.580 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:20:26.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 msgr2=0x7f28e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 0x7f28e806ede0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2900009b90 tx=0x7f2900008040 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 msgr2=0x7f29080cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f28f8034040 tx=0x7f28f8004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 shutdown_connections 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f28e806c920 0x7f28e806ede0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290815f260 0x7f29080cf810 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29080b4e00 0x7f29080cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 --2- 172.21.15.40:0/1826377129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2908070560 0x7f290815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 >> 172.21.15.40:0/1826377129 conn(0x7f29080d1660 msgr2=0x7f290815d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 shutdown_connections 2024-04-26T21:20:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.582+0000 7f290ce93640 1 -- 172.21.15.40:0/1826377129 wait complete. 2024-04-26T21:20:26.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:20:26.625 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:20:26.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.701+0000 7f7dfe899640 1 Processor -- start 2024-04-26T21:20:26.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.701+0000 7f7dfe899640 1 -- start start 2024-04-26T21:20:26.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfe899640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7df8161080 0x7f7df8161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfe899640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df81619d0 con 0x7f7df8161cb0 2024-04-26T21:20:26.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfe899640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df80cdd40 con 0x7f7df815e7e0 2024-04-26T21:20:26.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfe899640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7df80d2280 con 0x7f7df8161080 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfce36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7df8161080 0x7f7df8161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7df77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7df8161cb0 0x7f7df80d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50726/0 (socket says 172.21.15.40:50726) 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7df77fe640 1 -- 172.21.15.40:0/88688723 learned_addr learned my addr 172.21.15.40:0/88688723 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfce36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7df8161080 0x7f7df8161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45308/0 (socket says 172.21.15.40:45308) 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfce36640 1 -- 172.21.15.40:0/88688723 >> v1:172.21.15.184:6789/0 conn(0x7f7df815e7e0 legacy=0x7f7df80ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfce36640 1 -- 172.21.15.40:0/88688723 >> v1:172.21.15.40:6789/0 conn(0x7f7df8161cb0 legacy=0x7f7df80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.702+0000 7f7dfce36640 1 -- 172.21.15.40:0/88688723 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df80cdfe0 con 0x7f7df8161080 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.703+0000 7f7dfce36640 1 --2- 172.21.15.40:0/88688723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df8161490 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009d90 tx=0x7f7df002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a17a9e646d969d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.703+0000 7f7df6ffd640 1 -- 172.21.15.40:0/88688723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0031600 con 0x7f7df8161080 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.703+0000 7f7df6ffd640 1 -- 172.21.15.40:0/88688723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df00317a0 con 0x7f7df8161080 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.703+0000 7f7df6ffd640 1 -- 172.21.15.40:0/88688723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0031b70 con 0x7f7df8161080 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 -- 172.21.15.40:0/88688723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 msgr2=0x7f7df8161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 --2- 172.21.15.40:0/88688723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df8161490 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009d90 tx=0x7f7df002a610 comp rx=0 tx=0).stop 2024-04-26T21:20:26.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 -- 172.21.15.40:0/88688723 shutdown_connections 2024-04-26T21:20:26.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 --2- 172.21.15.40:0/88688723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df8161490 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 -- 172.21.15.40:0/88688723 >> 172.21.15.40:0/88688723 conn(0x7f7df809ba40 msgr2=0x7f7df809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:26.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 -- 172.21.15.40:0/88688723 shutdown_connections 2024-04-26T21:20:26.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.704+0000 7f7dfe899640 1 -- 172.21.15.40:0/88688723 wait complete. 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.705+0000 7f7dfe899640 1 Processor -- start 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.705+0000 7f7dfe899640 1 -- start start 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 0x7f7df806ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df8161cb0 0x7f7df815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7df80ce880 con 0x7f7df8161080 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7df80ce5a0 con 0x7f7df815e7e0 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfe899640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7df80d1f60 con 0x7f7df8161cb0 2024-04-26T21:20:26.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45314/0 (socket says 172.21.15.40:45314) 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 -- 172.21.15.40:0/3304991396 learned_addr learned my addr 172.21.15.40:0/3304991396 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7dfce36640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df8161cb0 0x7f7df815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df7fff640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 0x7f7df806ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 -- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df8161cb0 msgr2=0x7f7df815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df8161cb0 0x7f7df815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 -- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 msgr2=0x7f7df806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 0x7f7df806ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.706+0000 7f7df77fe640 1 -- 172.21.15.40:0/3304991396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8154b30 con 0x7f7df8161080 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7df7fff640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 0x7f7df806ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7df77fe640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c470 tx=0x7f7de8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8017070 con 0x7f7df8161080 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8005ce0 con 0x7f7df8161080 2024-04-26T21:20:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df806e610 con 0x7f7df8161080 2024-04-26T21:20:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de80072f0 con 0x7f7df8161080 2024-04-26T21:20:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.707+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df806ebd0 con 0x7f7df8161080 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.708+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df815bbd0 con 0x7f7df8161080 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.709+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7de801a070 con 0x7f7df8161080 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.709+0000 7f7df4ff9640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 0x7f7de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.709+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5795+0+0 (secure 0 0 0) 0x7f7de808aeb0 con 0x7f7df8161080 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.709+0000 7f7df7fff640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 0x7f7de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.710+0000 7f7df7fff640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 0x7f7de406edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7dec006fd0 tx=0x7f7dec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.714+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7df815bbd0 con 0x7f7df8161080 2024-04-26T21:20:26.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:26.938+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7df806a290 con 0x7f7df8161080 2024-04-26T21:20:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.580+0000 7f7df4ff9640 1 -- 172.21.15.40:0/3304991396 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7f7df81619d0 con 0x7f7df8161080 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 msgr2=0x7f7de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 0x7f7de406edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7dec006fd0 tx=0x7f7dec008040 comp rx=0 tx=0).stop 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 msgr2=0x7f7df815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c470 tx=0x7f7de8001e70 comp rx=0 tx=0).stop 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 shutdown_connections 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7de406c8f0 0x7f7de406edb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df8161cb0 0x7f7df815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df8161080 0x7f7df815c4a0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 --2- 172.21.15.40:0/3304991396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df815e7e0 0x7f7df806ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 >> 172.21.15.40:0/3304991396 conn(0x7f7df809ba40 msgr2=0x7f7df80b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 shutdown_connections 2024-04-26T21:20:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.583+0000 7f7dfe899640 1 -- 172.21.15.40:0/3304991396 wait complete. 2024-04-26T21:20:27.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:20:27.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.753+0000 7f220d932640 1 Processor -- start 2024-04-26T21:20:27.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.753+0000 7f220d932640 1 -- start start 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f220d932640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22080d2140 0x7f22080d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f220d932640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22080b6020 0x7f22080cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f220d932640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22080d33a0 con 0x7f2208162a10 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f220d932640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f22080b4970 con 0x7f22080b6020 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f220d932640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f22080d3680 con 0x7f22080d2140 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22080b6020 0x7f22080cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f2206ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2208162a10 0x7f22080d1a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50742/0 (socket says 172.21.15.40:50742) 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f2206ffd640 1 -- 172.21.15.40:0/3710847982 learned_addr learned my addr 172.21.15.40:0/3710847982 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f21fffff640 1 --2- 172.21.15.40:0/3710847982 >> v2:172.21.15.184:3300/0 conn(0x7f22080d2140 0x7f22080d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22077fe640 1 -- 172.21.15.40:0/3710847982 >> v2:172.21.15.184:3300/0 conn(0x7f22080d2140 msgr2=0x7f22080d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22077fe640 1 --2- 172.21.15.40:0/3710847982 >> v2:172.21.15.184:3300/0 conn(0x7f22080d2140 0x7f22080d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22077fe640 1 -- 172.21.15.40:0/3710847982 >> v1:172.21.15.40:6789/0 conn(0x7f2208162a10 legacy=0x7f22080d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22067fc640 1 -- 172.21.15.40:0/3710847982 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099630117 0 0) 0x7f22080d33a0 con 0x7f2208162a10 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.754+0000 7f22077fe640 1 -- 172.21.15.40:0/3710847982 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22080d3960 con 0x7f22080b6020 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.755+0000 7f22077fe640 1 --2- 172.21.15.40:0/3710847982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f22080cffb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f220808f0b0 tx=0x7f220002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8891720f676b3e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.755+0000 7f22067fc640 1 -- 172.21.15.40:0/3710847982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200007740 con 0x7f22080b6020 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.755+0000 7f22067fc640 1 -- 172.21.15.40:0/3710847982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000078e0 con 0x7f22080b6020 2024-04-26T21:20:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.755+0000 7f22067fc640 1 -- 172.21.15.40:0/3710847982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200007cb0 con 0x7f22080b6020 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 -- 172.21.15.40:0/3710847982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 msgr2=0x7f22080cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 --2- 172.21.15.40:0/3710847982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f22080cffb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f220808f0b0 tx=0x7f220002a680 comp rx=0 tx=0).stop 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 -- 172.21.15.40:0/3710847982 shutdown_connections 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 --2- 172.21.15.40:0/3710847982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f22080cffb0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 --2- 172.21.15.40:0/3710847982 >> v2:172.21.15.184:3300/0 conn(0x7f22080d2140 0x7f22080d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 -- 172.21.15.40:0/3710847982 >> 172.21.15.40:0/3710847982 conn(0x7f220809e530 msgr2=0x7f22080d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 -- 172.21.15.40:0/3710847982 shutdown_connections 2024-04-26T21:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.756+0000 7f220d932640 1 -- 172.21.15.40:0/3710847982 wait complete. 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.757+0000 7f220d932640 1 Processor -- start 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.757+0000 7f220d932640 1 -- start start 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22080d2140 0x7f220814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2208162a10 0x7f22080cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2208162df0 con 0x7f22080b6020 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22080d3f20 con 0x7f22080d2140 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f220d932640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22080b6600 con 0x7f2208162a10 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45324/0 (socket says 172.21.15.40:45324) 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 -- 172.21.15.40:0/71892554 learned_addr learned my addr 172.21.15.40:0/71892554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 -- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2208162a10 msgr2=0x7f22080cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2208162a10 0x7f22080cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 -- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22080d2140 msgr2=0x7f220814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f21fffff640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22080d2140 0x7f220814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22080d2140 0x7f220814c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 -- 172.21.15.40:0/71892554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220814c6f0 con 0x7f22080b6020 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.758+0000 7f2206ffd640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c430 tx=0x7f21f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.759+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f80074e0 con 0x7f22080b6020 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.759+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f80047e0 con 0x7f22080b6020 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.759+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22080cbab0 con 0x7f22080b6020 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.759+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8004bd0 con 0x7f22080b6020 2024-04-26T21:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.759+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081609e0 con 0x7f22080b6020 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.760+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220806a290 con 0x7f22080b6020 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.760+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21f8011070 con 0x7f22080b6020 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.761+0000 7f2204ff9640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.761+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5795+0+0 (secure 0 0 0) 0x7f21f808b890 con 0x7f22080b6020 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.761+0000 7f21fffff640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.761+0000 7f21fffff640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 0x7f21e806edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22081f8a40 tx=0x7f21f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.765+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f220814c6f0 con 0x7f22080b6020 2024-04-26T21:20:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:27.953+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22080d27a0 con 0x7f22080b6020 2024-04-26T21:20:28.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.866+0000 7f2204ff9640 1 -- 172.21.15.40:0/71892554 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) ==== 181+0+0 (secure 0 0 0) 0x7f21f8059c60 con 0x7f22080b6020 2024-04-26T21:20:28.867 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:20:28.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.870+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 msgr2=0x7f21e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.870+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 0x7f21e806edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22081f8a40 tx=0x7f21f0008040 comp rx=0 tx=0).stop 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.871+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 msgr2=0x7f2208160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.871+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c430 tx=0x7f21f800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 shutdown_connections 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f21e806c8f0 0x7f21e806edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2208162a10 0x7f22080cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22080d2140 0x7f220814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 --2- 172.21.15.40:0/71892554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22080b6020 0x7f2208160000 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.872+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 >> 172.21.15.40:0/71892554 conn(0x7f220809e530 msgr2=0x7f220806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:28.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.873+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 shutdown_connections 2024-04-26T21:20:28.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.873+0000 7f220d932640 1 -- 172.21.15.40:0/71892554 wait complete. 2024-04-26T21:20:28.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.998+0000 7f9817a09640 1 Processor -- start 2024-04-26T21:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.998+0000 7f9817a09640 1 -- start start 2024-04-26T21:20:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9817a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 0x7f98100d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9817a09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98100d1ac0 0x7f98100d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9817a09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9810146c90 con 0x7f9810145fe0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9817a09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f98101463c0 con 0x7f98100d1ac0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9817a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f98100d0d00 con 0x7f9810161be0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98100d1ac0 0x7f98100d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98100d1ac0 0x7f98100d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45332/0 (socket says 172.21.15.40:45332) 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 -- 172.21.15.40:0/115811443 learned_addr learned my addr 172.21.15.40:0/115811443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f98157a5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9810145fe0 0x7f9810146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50746/0 (socket says 172.21.15.40:50746) 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9814fa4640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 0x7f98100d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 -- 172.21.15.40:0/115811443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 msgr2=0x7f98100d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 0x7f98100d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 -- 172.21.15.40:0/115811443 >> v1:172.21.15.40:6789/0 conn(0x7f9810145fe0 legacy=0x7f9810146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9815fa6640 1 -- 172.21.15.40:0/115811443 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98100d0fe0 con 0x7f98100d1ac0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9807fff640 1 -- 172.21.15.40:0/115811443 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 241332253 0 0) 0x7f9810146c90 con 0x7f9810145fe0 2024-04-26T21:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:28.999+0000 7f9814fa4640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 0x7f98100d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.000+0000 7f9815fa6640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f98100d2730 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f9808009e50 tx=0x7f980802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cacc2c0b1996ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.000+0000 7f9807fff640 1 -- 172.21.15.40:0/115811443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808036070 con 0x7f98100d1ac0 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.000+0000 7f9807fff640 1 -- 172.21.15.40:0/115811443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808031600 con 0x7f98100d1ac0 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.000+0000 7f9807fff640 1 -- 172.21.15.40:0/115811443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080319d0 con 0x7f98100d1ac0 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 -- 172.21.15.40:0/115811443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 msgr2=0x7f98100d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f98100d2730 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f9808009e50 tx=0x7f980802a680 comp rx=0 tx=0).stop 2024-04-26T21:20:29.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 -- 172.21.15.40:0/115811443 shutdown_connections 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f98100d2730 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 --2- 172.21.15.40:0/115811443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810161be0 0x7f98100d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 -- 172.21.15.40:0/115811443 >> 172.21.15.40:0/115811443 conn(0x7f981006d590 msgr2=0x7f981006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 -- 172.21.15.40:0/115811443 shutdown_connections 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.001+0000 7f9817a09640 1 -- 172.21.15.40:0/115811443 wait complete. 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.002+0000 7f9817a09640 1 Processor -- start 2024-04-26T21:20:29.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.002+0000 7f9817a09640 1 -- start start 2024-04-26T21:20:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810145fe0 0x7f98100aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9810161be0 0x7f9810073860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f98100d09b0 con 0x7f98100d1ac0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f98100d15a0 con 0x7f9810161be0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9817a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f981006f440 con 0x7f9810145fe0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45346/0 (socket says 172.21.15.40:45346) 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 -- 172.21.15.40:0/361324429 learned_addr learned my addr 172.21.15.40:0/361324429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9814fa4640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810145fe0 0x7f98100aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9815fa6640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9810161be0 0x7f9810073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 -- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810145fe0 msgr2=0x7f98100aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810145fe0 0x7f98100aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 -- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9810161be0 msgr2=0x7f9810073860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9810161be0 0x7f9810073860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 -- 172.21.15.40:0/361324429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98100b5460 con 0x7f98100d1ac0 2024-04-26T21:20:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f98157a5640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c4d0 tx=0x7f980c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c018070 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.003+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c007500 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.004+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98100b5740 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.004+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c004a20 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.004+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98100b5fe0 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.005+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98100b5910 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.005+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f980c01b070 con 0x7f98100d1ac0 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.006+0000 7f9805ffb640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:29.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.006+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5357+0+0 (secure 0 0 0) 0x7f980c08ad80 con 0x7f98100d1ac0 2024-04-26T21:20:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.006+0000 7f9814fa4640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.007+0000 7f9814fa4640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 0x7f97f406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f981014e670 tx=0x7f9800008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:29.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.010+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f980c059280 con 0x7f98100d1ac0 2024-04-26T21:20:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.195+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9810146580 con 0x7f98100d1ac0 2024-04-26T21:20:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.878+0000 7f9805ffb640 1 -- 172.21.15.40:0/361324429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) ==== 169+0+0 (secure 0 0 0) 0x7f9810146580 con 0x7f98100d1ac0 2024-04-26T21:20:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:20:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.884+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 msgr2=0x7f97f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.884+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 0x7f97f406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f981014e670 tx=0x7f9800008040 comp rx=0 tx=0).stop 2024-04-26T21:20:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.884+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 msgr2=0x7f981013d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.884+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c4d0 tx=0x7f980c00c990 comp rx=0 tx=0).stop 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.885+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 shutdown_connections 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.885+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97f406c8f0 0x7f97f406edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9810161be0 0x7f9810073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9810145fe0 0x7f98100aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 --2- 172.21.15.40:0/361324429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98100d1ac0 0x7f981013d980 secure :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c4d0 tx=0x7f980c00c990 comp rx=0 tx=0).stop 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 >> 172.21.15.40:0/361324429 conn(0x7f981006d590 msgr2=0x7f981006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 shutdown_connections 2024-04-26T21:20:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:29.886+0000 7f9817a09640 1 -- 172.21.15.40:0/361324429 wait complete. 2024-04-26T21:20:29.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-26T21:20:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.011+0000 7fb0b726b640 1 Processor -- start 2024-04-26T21:20:30.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.011+0000 7fb0b726b640 1 -- start start 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b726b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0b00d0010 0x7fb0b006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b726b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b009c700 con 0x7fb0b00b3c20 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b726b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b00d1c50 con 0x7fb0b00b33e0 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b726b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b00b4480 con 0x7fb0b00d0010 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b4806640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0b00b3c20 0x7fb0b00b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50756/0 (socket says 172.21.15.40:50756) 2024-04-26T21:20:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b4806640 1 -- 172.21.15.40:0/850545601 learned_addr learned my addr 172.21.15.40:0/850545601 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284487595 0 0) 0x7fb0b009c700 con 0x7fb0b00b3c20 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb09c003520 con 0x7fb0b00b3c20 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0b5007640 1 --2- 172.21.15.40:0/850545601 >> v2:172.21.15.184:3300/0 conn(0x7fb0b00d0010 0x7fb0b006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3172817597 0 0) 0x7fb09c003520 con 0x7fb0b00b3c20 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 >> v2:172.21.15.184:3300/0 conn(0x7fb0b00d0010 msgr2=0x7fb0b006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 --2- 172.21.15.40:0/850545601 >> v2:172.21.15.184:3300/0 conn(0x7fb0b00d0010 0x7fb0b006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 >> v1:172.21.15.184:6789/0 conn(0x7fb0b00b33e0 legacy=0x7fb0b0163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b00d1f30 con 0x7fb0b00b3c20 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fb0a0001670 con 0x7fb0b00b3c20 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 >> v1:172.21.15.40:6789/0 conn(0x7fb0b00b3c20 legacy=0x7fb0b00b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 0x7fb09c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c004800 0x7fb09c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.012+0000 7fb0a7fff640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c0051c0 0x7fb09c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b00b4480 con 0x7fb09c0039b0 2024-04-26T21:20:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c00c060 con 0x7fb09c004800 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c00c260 con 0x7fb09c0051c0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 0x7fb09c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b4806640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c004800 0x7fb09c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5808640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c0051c0 0x7fb09c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 -- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c0051c0 msgr2=0x7fb09c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c0051c0 0x7fb09c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 -- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c004800 msgr2=0x7fb09c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c004800 0x7fb09c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5007640 1 -- 172.21.15.40:0/850545601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c003520 con 0x7fb09c0039b0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5808640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c0051c0 0x7fb09c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b4806640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c004800 0x7fb09c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.013+0000 7fb0b5808640 1 -- 172.21.15.40:0/850545601 reap_dead start 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b5007640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 0x7fb09c003dd0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ac0 tx=0x7fb09c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb01fe50153cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac036070 con 0x7fb09c0039b0 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac031400 con 0x7fb09c0039b0 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0a7fff640 1 -- 172.21.15.40:0/850545601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac009690 con 0x7fb09c0039b0 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 -- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 msgr2=0x7fb09c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 0x7fb09c003dd0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ac0 tx=0x7fb09c005830 comp rx=0 tx=0).stop 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 -- 172.21.15.40:0/850545601 shutdown_connections 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 --2- 172.21.15.40:0/850545601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb09c0039b0 0x7fb09c003dd0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 -- 172.21.15.40:0/850545601 >> 172.21.15.40:0/850545601 conn(0x7fb0b009ba40 msgr2=0x7fb0b009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 -- 172.21.15.40:0/850545601 shutdown_connections 2024-04-26T21:20:30.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.014+0000 7fb0b726b640 1 -- 172.21.15.40:0/850545601 wait complete. 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.015+0000 7fb0b726b640 1 Processor -- start 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 -- start start 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c0039b0 0x7fb0b006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0b01f6580 0x7fb0b00d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b006db10 con 0x7fb0b01f54d0 2024-04-26T21:20:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b00b3a20 con 0x7fb09c0039b0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b726b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b006dc90 con 0x7fb0b01f6580 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b4806640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b4806640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45354/0 (socket says 172.21.15.40:45354) 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.016+0000 7fb0b4806640 1 -- 172.21.15.40:0/803078423 learned_addr learned my addr 172.21.15.40:0/803078423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b5808640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0b01f6580 0x7fb0b00d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b5007640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c0039b0 0x7fb0b006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 -- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0b01f6580 msgr2=0x7fb0b00d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0b01f6580 0x7fb0b00d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 -- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c0039b0 msgr2=0x7fb0b006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c0039b0 0x7fb0b006f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 -- 172.21.15.40:0/803078423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b006e860 con 0x7fb0b01f54d0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b4806640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e820 tx=0x7fb0a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000c8d0 con 0x7fb0b01f54d0 2024-04-26T21:20:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a0018070 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000cc70 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01f5aa0 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.017+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01f6a00 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.018+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b006a290 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.018+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0a001d460 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.019+0000 7fb0a5ffb640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 0x7fb09406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.019+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7fb0a008b720 con 0x7fb0b01f54d0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.019+0000 7fb0b5007640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 0x7fb09406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.020+0000 7fb0b5007640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 0x7fb09406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac033000 tx=0x7fb0ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.023+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0a0059ee0 con 0x7fb0b01f54d0 2024-04-26T21:20:30.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.209+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7fb0b0068e00 con 0x7fb0b01f54d0 2024-04-26T21:20:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.212+0000 7fb0a5ffb640 1 -- 172.21.15.40:0/803078423 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v51) ==== 111+0+0 (secure 0 0 0) 0x7fb0a00597e0 con 0x7fb0b01f54d0 2024-04-26T21:20:30.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 msgr2=0x7fb09406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 0x7fb09406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac033000 tx=0x7fb0ac008030 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 msgr2=0x7fb0b01577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e820 tx=0x7fb0a000ece0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 shutdown_connections 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb09406c8f0 0x7fb09406edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0b01f6580 0x7fb0b00d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0b01f54d0 0x7fb0b01577b0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 --2- 172.21.15.40:0/803078423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c0039b0 0x7fb0b006f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 >> 172.21.15.40:0/803078423 conn(0x7fb0b009ba40 msgr2=0x7fb0b00d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 shutdown_connections 2024-04-26T21:20:30.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.214+0000 7fb0b726b640 1 -- 172.21.15.40:0/803078423 wait complete. 2024-04-26T21:20:30.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:20:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.328+0000 7f86b9008640 1 Processor -- start 2024-04-26T21:20:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.328+0000 7f86b9008640 1 -- start start 2024-04-26T21:20:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b9008640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86b40d0010 0x7f86b406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b9008640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b9008640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b409c700 con 0x7f86b40b3210 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b9008640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40b4480 con 0x7f86b40d0010 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b9008640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b406e050 con 0x7f86b40b3c20 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86abfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86b40d0010 0x7f86b406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86b40d0010 0x7f86b406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:49258/0 (socket says 172.21.15.40:49258) 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b359e640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f86b40b3210 0x7f86b415fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:57012/0 (socket says 172.21.15.40:57012) 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1269417851 learned_addr learned my addr 172.21.15.40:0/1269417851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86abfff640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b40b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:35886/0 (socket says 172.21.15.40:35886) 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 msgr2=0x7f86b40b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b40b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1269417851 >> v1:172.21.15.184:6789/0 conn(0x7f86b40b3210 legacy=0x7f86b415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.329+0000 7f86b2d9d640 1 -- 172.21.15.40:0/1269417851 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40d1c50 con 0x7f86b40d0010 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.330+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b406db10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009870 tx=0x7f86a402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fba4751fe5c105 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.330+0000 7f86b259c640 1 -- 172.21.15.40:0/1269417851 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86b40d0010 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.330+0000 7f86b259c640 1 -- 172.21.15.40:0/1269417851 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a4032440 con 0x7f86b40d0010 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.330+0000 7f86b259c640 1 -- 172.21.15.40:0/1269417851 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4032810 con 0x7f86b40d0010 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 -- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 msgr2=0x7f86b406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b406db10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009870 tx=0x7f86a402a270 comp rx=0 tx=0).stop 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 -- 172.21.15.40:0/1269417851 shutdown_connections 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b40b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 --2- 172.21.15.40:0/1269417851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d0010 0x7f86b406db10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 -- 172.21.15.40:0/1269417851 >> 172.21.15.40:0/1269417851 conn(0x7f86b409ba40 msgr2=0x7f86b409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 -- 172.21.15.40:0/1269417851 shutdown_connections 2024-04-26T21:20:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.331+0000 7f86b9008640 1 -- 172.21.15.40:0/1269417851 wait complete. 2024-04-26T21:20:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.332+0000 7f86b9008640 1 Processor -- start 2024-04-26T21:20:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.332+0000 7f86b9008640 1 -- start start 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40b3210 0x7f86b415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b41e9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40b3820 con 0x7f86b40d0010 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40b36a0 con 0x7f86b40b3210 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b9008640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40b39a0 con 0x7f86b40b3c20 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45364/0 (socket says 172.21.15.40:45364) 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 -- 172.21.15.40:0/1131627404 learned_addr learned my addr 172.21.15.40:0/1131627404 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40b3210 0x7f86b415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86abfff640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b41e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 -- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 msgr2=0x7f86b41e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b41e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 -- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40b3210 msgr2=0x7f86b415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40b3210 0x7f86b415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.333+0000 7f86b359e640 1 -- 172.21.15.40:0/1131627404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b41e9880 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86b359e640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c730 tx=0x7f86ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac017070 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac00ce40 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00a840 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b41e9b60 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.334+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b41ea120 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.335+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b406a290 con 0x7f86b40d0010 2024-04-26T21:20:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.335+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86ac01a070 con 0x7f86b40d0010 2024-04-26T21:20:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.336+0000 7f86ab7fe640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 0x7f869406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.336+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f86ac08ad30 con 0x7f86b40d0010 2024-04-26T21:20:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.336+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 0x7f869406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.337+0000 7f86b2d9d640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 0x7f869406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034000 tx=0x7f86a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.340+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86ac058e10 con 0x7f86b40d0010 2024-04-26T21:20:30.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.530+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f86b40b2cd0 con 0x7f86b40d0010 2024-04-26T21:20:30.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.533+0000 7f86ab7fe640 1 -- 172.21.15.40:0/1131627404 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f86ac05d110 con 0x7f86b40d0010 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.535+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 msgr2=0x7f869406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.535+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 0x7f869406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034000 tx=0x7f86a402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.535+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 msgr2=0x7f86b409eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.535+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c730 tx=0x7f86ac00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.535+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 shutdown_connections 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f869406c8f0 0x7f869406edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b40d0010 0x7f86b409eaf0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40b3c20 0x7f86b41e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 --2- 172.21.15.40:0/1131627404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40b3210 0x7f86b415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 >> 172.21.15.40:0/1131627404 conn(0x7f86b409ba40 msgr2=0x7f86b4162e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 shutdown_connections 2024-04-26T21:20:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.536+0000 7f86b9008640 1 -- 172.21.15.40:0/1131627404 wait complete. 2024-04-26T21:20:30.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-26T21:20:30.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.649+0000 7f100bb89640 1 Processor -- start 2024-04-26T21:20:30.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.649+0000 7f100bb89640 1 -- start start 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100bb89640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f100415f5b0 0x7f10040b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100bb89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f100415f990 0x7f10040d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100bb89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10040b6aa0 con 0x7f10040b55b0 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100bb89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f10040d3eb0 con 0x7f100415f990 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100bb89640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f10040d4190 con 0x7f100415f5b0 2024-04-26T21:20:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009124640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f100415f990 0x7f10040d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100a126640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10040b55b0 0x7f10040d3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51762/0 (socket says 172.21.15.40:51762) 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f100a126640 1 -- 172.21.15.40:0/110324114 learned_addr learned my addr 172.21.15.40:0/110324114 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009925640 1 --2- 172.21.15.40:0/110324114 >> v2:172.21.15.184:3300/0 conn(0x7f100415f5b0 0x7f10040b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009124640 1 -- 172.21.15.40:0/110324114 >> v2:172.21.15.184:3300/0 conn(0x7f100415f5b0 msgr2=0x7f10040b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009124640 1 --2- 172.21.15.40:0/110324114 >> v2:172.21.15.184:3300/0 conn(0x7f100415f5b0 0x7f10040b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009124640 1 -- 172.21.15.40:0/110324114 >> v1:172.21.15.40:6789/0 conn(0x7f10040b55b0 legacy=0x7f10040d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1008923640 1 -- 172.21.15.40:0/110324114 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951119958 0 0) 0x7f10040b6aa0 con 0x7f10040b55b0 2024-04-26T21:20:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.650+0000 7f1009124640 1 -- 172.21.15.40:0/110324114 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10040d0200 con 0x7f100415f990 2024-04-26T21:20:30.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f1009124640 1 --2- 172.21.15.40:0/110324114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100415f990 0x7f10040d0a00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f100408f0f0 tx=0x7f0ff402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2315f460609c454 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f1008923640 1 -- 172.21.15.40:0/110324114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4037070 con 0x7f100415f990 2024-04-26T21:20:30.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f1008923640 1 -- 172.21.15.40:0/110324114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff4032440 con 0x7f100415f990 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f1008923640 1 -- 172.21.15.40:0/110324114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff402ae60 con 0x7f100415f990 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 -- 172.21.15.40:0/110324114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100415f990 msgr2=0x7f10040d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 --2- 172.21.15.40:0/110324114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100415f990 0x7f10040d0a00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f100408f0f0 tx=0x7f0ff402a110 comp rx=0 tx=0).stop 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 -- 172.21.15.40:0/110324114 shutdown_connections 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 --2- 172.21.15.40:0/110324114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100415f990 0x7f10040d0a00 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 --2- 172.21.15.40:0/110324114 >> v2:172.21.15.184:3300/0 conn(0x7f100415f5b0 0x7f10040b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.651+0000 7f100bb89640 1 -- 172.21.15.40:0/110324114 >> 172.21.15.40:0/110324114 conn(0x7f100408ec40 msgr2=0x7f100409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.652+0000 7f100bb89640 1 -- 172.21.15.40:0/110324114 shutdown_connections 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.652+0000 7f100bb89640 1 -- 172.21.15.40:0/110324114 wait complete. 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 Processor -- start 2024-04-26T21:20:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 -- start start 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f100415f5b0 0x7f10040b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 0x7f1004161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10040d3bd0 con 0x7f10040b55b0 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10040d3a50 con 0x7f100415f990 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f100bb89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10040d0f40 con 0x7f100415f5b0 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.653+0000 7f1009925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56234/0 (socket says 172.21.15.40:56234) 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 -- 172.21.15.40:0/2467025693 learned_addr learned my addr 172.21.15.40:0/2467025693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009124640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f100415f5b0 0x7f10040b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f100a126640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 0x7f1004161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 -- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f100415f5b0 msgr2=0x7f10040b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f100415f5b0 0x7f10040b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 -- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 msgr2=0x7f1004161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 0x7f1004161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 -- 172.21.15.40:0/2467025693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10040b8550 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f100a126640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 0x7f1004161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f1009925640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f100000d800 tx=0x7f100000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100000b6c0 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000019070 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100000ba80 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10040b8830 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.654+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10040b8df0 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.655+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100406a290 con 0x7f10040b55b0 2024-04-26T21:20:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.656+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f100000bc20 con 0x7f10040b55b0 2024-04-26T21:20:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.656+0000 7f0ffa7fc640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.656+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f100008bc10 con 0x7f10040b55b0 2024-04-26T21:20:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.657+0000 7f1009124640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.657+0000 7f1009124640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 0x7f0fe806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4034000 tx=0x7f0ff402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.661+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f100005a3d0 con 0x7f10040b55b0 2024-04-26T21:20:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.847+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f10040d60c0 con 0x7f10040b55b0 2024-04-26T21:20:30.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.850+0000 7f0ffa7fc640 1 -- 172.21.15.40:0/2467025693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v53) ==== 102+0+0 (secure 0 0 0) 0x7f1000059cd0 con 0x7f10040b55b0 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.852+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 msgr2=0x7f0fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.852+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 0x7f0fe806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4034000 tx=0x7f0ff402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 msgr2=0x7f1004153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f100000d800 tx=0x7f100000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 shutdown_connections 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f100415f990 0x7f1004161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f100415f5b0 0x7f10040b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 --2- 172.21.15.40:0/2467025693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10040b55b0 0x7f1004153610 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 >> 172.21.15.40:0/2467025693 conn(0x7f100408ec40 msgr2=0x7f10040b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 shutdown_connections 2024-04-26T21:20:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.853+0000 7f100bb89640 1 -- 172.21.15.40:0/2467025693 wait complete. 2024-04-26T21:20:30.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-26T21:20:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 Processor -- start 2024-04-26T21:20:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 -- start start 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef980b3c20 0x7fef9806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980b33e0 0x7fef98163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9809c700 con 0x7fef980d0010 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef980b4480 con 0x7fef980b3c20 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.967+0000 7fef9d171640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef9806e050 con 0x7fef980b33e0 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef980b3c20 0x7fef9806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef980b3c20 0x7fef9806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56244/0 (socket says 172.21.15.40:56244) 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 -- 172.21.15.40:0/2818179153 learned_addr learned my addr 172.21.15.40:0/2818179153 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9759e640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980b33e0 0x7fef98163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 -- 172.21.15.40:0/2818179153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980b33e0 msgr2=0x7fef98163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980b33e0 0x7fef98163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 -- 172.21.15.40:0/2818179153 >> v1:172.21.15.184:6789/0 conn(0x7fef980d0010 legacy=0x7fef980b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.968+0000 7fef9659c640 1 -- 172.21.15.40:0/2818179153 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980d1c50 con 0x7fef980b3c20 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9659c640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef9806db10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96ea9c5bae030a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef95d9b640 1 -- 172.21.15.40:0/2818179153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c037070 con 0x7fef980b3c20 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef95d9b640 1 -- 172.21.15.40:0/2818179153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c032440 con 0x7fef980b3c20 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef95d9b640 1 -- 172.21.15.40:0/2818179153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c032810 con 0x7fef980b3c20 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 -- 172.21.15.40:0/2818179153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 msgr2=0x7fef9806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef9806db10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c02a270 comp rx=0 tx=0).stop 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 -- 172.21.15.40:0/2818179153 shutdown_connections 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980b33e0 0x7fef98163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 --2- 172.21.15.40:0/2818179153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef9806db10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 -- 172.21.15.40:0/2818179153 >> 172.21.15.40:0/2818179153 conn(0x7fef9809ba40 msgr2=0x7fef9809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 -- 172.21.15.40:0/2818179153 shutdown_connections 2024-04-26T21:20:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.969+0000 7fef9d171640 1 -- 172.21.15.40:0/2818179153 wait complete. 2024-04-26T21:20:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.971+0000 7fef9d171640 1 Processor -- start 2024-04-26T21:20:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.971+0000 7fef9d171640 1 -- start start 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef980b33e0 0x7fef98155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980d0010 0x7fef98161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b39d0 con 0x7fef980b3c20 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b3850 con 0x7fef980b33e0 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fef981637c0 con 0x7fef980d0010 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56258/0 (socket says 172.21.15.40:56258) 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 -- 172.21.15.40:0/1372402844 learned_addr learned my addr 172.21.15.40:0/1372402844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef96d9d640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef980b33e0 0x7fef98155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9759e640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980d0010 0x7fef98161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 -- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980d0010 msgr2=0x7fef98161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980d0010 0x7fef98161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 -- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef980b33e0 msgr2=0x7fef98155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef980b33e0 0x7fef98155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 -- 172.21.15.40:0/1372402844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef981f5540 con 0x7fef980b3c20 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9659c640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fef8c000c00 tx=0x7fef8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c041020 con 0x7fef980b3c20 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c0306c0 con 0x7fef980b3c20 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c037040 con 0x7fef980b3c20 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.972+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9809dec0 con 0x7fef980b3c20 2024-04-26T21:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.973+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9815a660 con 0x7fef980b3c20 2024-04-26T21:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.973+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9806a290 con 0x7fef980b3c20 2024-04-26T21:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.974+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef8c044020 con 0x7fef980b3c20 2024-04-26T21:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.974+0000 7fef7f7fe640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 0x7fef8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.974+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7fef8c0ac060 con 0x7fef980b3c20 2024-04-26T21:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.975+0000 7fef96d9d640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 0x7fef8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.975+0000 7fef96d9d640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 0x7fef8406edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fef90001ca0 tx=0x7fef90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:30.978+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef8c04b030 con 0x7fef980b3c20 2024-04-26T21:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.168+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7fef980b8f10 con 0x7fef980b3c20 2024-04-26T21:20:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.173+0000 7fef7f7fe640 1 -- 172.21.15.40:0/1372402844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v54) ==== 105+0+0 (secure 0 0 0) 0x7fef8c07a780 con 0x7fef980b3c20 2024-04-26T21:20:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 msgr2=0x7fef8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 0x7fef8406edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fef90001ca0 tx=0x7fef90009040 comp rx=0 tx=0).stop 2024-04-26T21:20:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 msgr2=0x7fef980a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fef8c000c00 tx=0x7fef8c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 shutdown_connections 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fef8406c8f0 0x7fef8406edb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef980d0010 0x7fef98161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef980b3c20 0x7fef980a0620 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 --2- 172.21.15.40:0/1372402844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef980b33e0 0x7fef98155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 >> 172.21.15.40:0/1372402844 conn(0x7fef9809ba40 msgr2=0x7fef980b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 shutdown_connections 2024-04-26T21:20:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.175+0000 7fef9d171640 1 -- 172.21.15.40:0/1372402844 wait complete. 2024-04-26T21:20:31.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-26T21:20:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.289+0000 7f3abda59640 1 Processor -- start 2024-04-26T21:20:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.289+0000 7f3abda59640 1 -- start start 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3abda59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3ab8160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3abda59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab8070740 con 0x7f3ab81468a0 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3abda59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab8070a00 con 0x7f3ab815d6f0 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3abda59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab8070ce0 con 0x7f3ab80b7960 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3ab81468a0 0x7f3ab80b7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51772/0 (socket says 172.21.15.40:51772) 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab6ffd640 1 -- 172.21.15.40:0/2585338876 learned_addr learned my addr 172.21.15.40:0/2585338876 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab67fc640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3ab8160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034374348 0 0) 0x7f3ab8070740 con 0x7f3ab81468a0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aa0003520 con 0x7f3ab81468a0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2423125801 0 0) 0x7f3aa0003520 con 0x7f3ab81468a0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 msgr2=0x7f3ab8160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3ab8160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 >> v1:172.21.15.184:6789/0 conn(0x7f3ab815d6f0 legacy=0x7f3ab81611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab8070fc0 con 0x7f3ab81468a0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f3aa8001670 con 0x7f3ab81468a0 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 >> v1:172.21.15.40:6789/0 conn(0x7f3ab81468a0 legacy=0x7f3ab80b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3aa00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0003f20 0x7f3aa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0004d40 0x7f3aa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8070ce0 con 0x7f3aa0004d40 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa000bad0 con 0x7f3aa0003f20 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.290+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa000bcd0 con 0x7f3ab80b7960 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0004d40 0x7f3aa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab6ffd640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3aa00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab67fc640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0003f20 0x7f3aa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 -- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 msgr2=0x7f3aa00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80b7960 0x7f3aa00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 -- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0003f20 msgr2=0x7f3aa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0003f20 0x7f3aa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.291+0000 7f3ab77fe640 1 -- 172.21.15.40:0/2585338876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa0003520 con 0x7f3aa0004d40 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3ab77fe640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0004d40 0x7f3aa00051c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0001f30 tx=0x7f3aa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7aac77b7b75ac61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0001b30 con 0x7f3aa0004d40 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab0031620 con 0x7f3aa0004d40 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3ab5ffb640 1 -- 172.21.15.40:0/2585338876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0031a10 con 0x7f3aa0004d40 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 -- 172.21.15.40:0/2585338876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0004d40 msgr2=0x7f3aa00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 --2- 172.21.15.40:0/2585338876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0004d40 0x7f3aa00051c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0001f30 tx=0x7f3aa0005830 comp rx=0 tx=0).stop 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3ab77fe640 1 -- 172.21.15.40:0/2585338876 reap_dead start 2024-04-26T21:20:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 -- 172.21.15.40:0/2585338876 shutdown_connections 2024-04-26T21:20:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 -- 172.21.15.40:0/2585338876 >> 172.21.15.40:0/2585338876 conn(0x7f3ab806f050 msgr2=0x7f3ab806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 -- 172.21.15.40:0/2585338876 shutdown_connections 2024-04-26T21:20:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.292+0000 7f3abda59640 1 -- 172.21.15.40:0/2585338876 wait complete. 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.293+0000 7f3abda59640 1 Processor -- start 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 -- start start 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0004d40 0x7f3ab8137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80cd120 0x7f3ab80b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8146e00 con 0x7f3ab81f5910 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8146c80 con 0x7f3aa0004d40 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3abda59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab8161900 con 0x7f3ab80cd120 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3ab67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3ab67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56278/0 (socket says 172.21.15.40:56278) 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.294+0000 7f3ab67fc640 1 -- 172.21.15.40:0/105753318 learned_addr learned my addr 172.21.15.40:0/105753318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab6ffd640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0004d40 0x7f3ab8137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab77fe640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80cd120 0x7f3ab80b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 -- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80cd120 msgr2=0x7f3ab80b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80cd120 0x7f3ab80b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 -- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0004d40 msgr2=0x7f3ab8137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0004d40 0x7f3ab8137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 -- 172.21.15.40:0/105753318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab80cb2c0 con 0x7f3ab81f5910 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3ab67fc640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0119a0 tx=0x7f3aac011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00faf0 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aac00fc90 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab80cb5a0 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac014440 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.295+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab80cbb60 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.296+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab8144b40 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.297+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3aac012040 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.297+0000 7f3a9ffff640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.297+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f3aac08bec0 con 0x7f3ab81f5910 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.297+0000 7f3ab6ffd640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.298+0000 7f3ab6ffd640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 0x7f3aa406edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8007c90 tx=0x7f3aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.301+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3aac019030 con 0x7f3ab81f5910 2024-04-26T21:20:31.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.485+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f3ab806a290 con 0x7f3ab81f5910 2024-04-26T21:20:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.490+0000 7f3a9ffff640 1 -- 172.21.15.40:0/105753318 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v55) ==== 107+0+0 (secure 0 0 0) 0x7f3aac05a030 con 0x7f3ab81f5910 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 msgr2=0x7f3aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 0x7f3aa406edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8007c90 tx=0x7f3aa8009040 comp rx=0 tx=0).stop 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 msgr2=0x7f3ab80ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0119a0 tx=0x7f3aac011e60 comp rx=0 tx=0).stop 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 shutdown_connections 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ab80cd120 0x7f3ab80b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab81f5910 0x7f3ab80ccbe0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 --2- 172.21.15.40:0/105753318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3aa0004d40 0x7f3ab8137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.492+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 >> 172.21.15.40:0/105753318 conn(0x7f3ab806f050 msgr2=0x7f3ab8145800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.493+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 shutdown_connections 2024-04-26T21:20:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.493+0000 7f3abda59640 1 -- 172.21.15.40:0/105753318 wait complete. 2024-04-26T21:20:31.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-04-26T21:20:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.604+0000 7fbe8b1b6640 1 Processor -- start 2024-04-26T21:20:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.604+0000 7fbe8b1b6640 1 -- start start 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe8b1b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe84145870 0x7fbe8406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe8b1b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe84073420 0x7fbe8415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe8b1b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe84073a40 con 0x7fbe8406f980 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe8b1b6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8406fde0 con 0x7fbe84145870 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe8b1b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8415d8d0 con 0x7fbe84073420 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe84145870 0x7fbe8406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe84145870 0x7fbe8406f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56282/0 (socket says 172.21.15.40:56282) 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 -- 172.21.15.40:0/1605452859 learned_addr learned my addr 172.21.15.40:0/1605452859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe89753640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe84073420 0x7fbe8415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 -- 172.21.15.40:0/1605452859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe84073420 msgr2=0x7fbe8415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe84073420 0x7fbe8415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 -- 172.21.15.40:0/1605452859 >> v1:172.21.15.184:6789/0 conn(0x7fbe8406f980 legacy=0x7fbe84072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.605+0000 7fbe88f52640 1 -- 172.21.15.40:0/1605452859 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8415dbb0 con 0x7fbe84145870 2024-04-26T21:20:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe88f52640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe8406f440 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009870 tx=0x7fbe7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abf6820cd9e70a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe837fe640 1 -- 172.21.15.40:0/1605452859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78037070 con 0x7fbe84145870 2024-04-26T21:20:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe837fe640 1 -- 172.21.15.40:0/1605452859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78032440 con 0x7fbe84145870 2024-04-26T21:20:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe837fe640 1 -- 172.21.15.40:0/1605452859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78032810 con 0x7fbe84145870 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1605452859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 msgr2=0x7fbe8406f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe8406f440 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009870 tx=0x7fbe7802a270 comp rx=0 tx=0).stop 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1605452859 shutdown_connections 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe84073420 0x7fbe8415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1605452859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe8406f440 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.606+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1605452859 >> 172.21.15.40:0/1605452859 conn(0x7fbe840cd830 msgr2=0x7fbe840cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.607+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1605452859 shutdown_connections 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.607+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1605452859 wait complete. 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.608+0000 7fbe8b1b6640 1 Processor -- start 2024-04-26T21:20:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.608+0000 7fbe8b1b6640 1 -- start start 2024-04-26T21:20:31.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.608+0000 7fbe8b1b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe8406f980 0x7fbe841527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.608+0000 7fbe8b1b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe84073420 0x7fbe840ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.608+0000 7fbe8b1b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe8b1b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84145c50 con 0x7fbe84145870 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe8b1b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8415d5d0 con 0x7fbe84073420 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe8b1b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8415e8f0 con 0x7fbe8406f980 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56284/0 (socket says 172.21.15.40:56284) 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 -- 172.21.15.40:0/1083180010 learned_addr learned my addr 172.21.15.40:0/1083180010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe88f52640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe8406f980 0x7fbe841527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe83fff640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe84073420 0x7fbe840ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 -- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe8406f980 msgr2=0x7fbe841527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe8406f980 0x7fbe841527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 -- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe84073420 msgr2=0x7fbe840ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe84073420 0x7fbe840ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 -- 172.21.15.40:0/1083180010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe840af300 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe89753640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007d90 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014420 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c019070 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe840af5e0 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014820 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.609+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe840afba0 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.610+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe840b0080 con 0x7fbe84145870 2024-04-26T21:20:31.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.611+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe7c0149c0 con 0x7fbe84145870 2024-04-26T21:20:31.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.611+0000 7fbe817fa640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.611+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7fbe7c08c070 con 0x7fbe84145870 2024-04-26T21:20:31.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.611+0000 7fbe88f52640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.612+0000 7fbe88f52640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 0x7fbe6806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbe78034000 tx=0x7fbe7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.616+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe840b0080 con 0x7fbe84145870 2024-04-26T21:20:31.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.878+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7fbe8406a290 con 0x7fbe84145870 2024-04-26T21:20:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.885+0000 7fbe817fa640 1 -- 172.21.15.40:0/1083180010 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v441) ==== 151+0+0 (secure 0 0 0) 0x7fbe7c05a710 con 0x7fbe84145870 2024-04-26T21:20:31.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 msgr2=0x7fbe6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 0x7fbe6806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbe78034000 tx=0x7fbe7802e000 comp rx=0 tx=0).stop 2024-04-26T21:20:31.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 msgr2=0x7fbe84140620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007d90 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 shutdown_connections 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe84145870 0x7fbe84140620 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe84073420 0x7fbe840ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 --2- 172.21.15.40:0/1083180010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbe8406f980 0x7fbe841527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 >> 172.21.15.40:0/1083180010 conn(0x7fbe840cd830 msgr2=0x7fbe840cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 shutdown_connections 2024-04-26T21:20:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.887+0000 7fbe8b1b6640 1 -- 172.21.15.40:0/1083180010 wait complete. 2024-04-26T21:20:31.899 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-26T21:20:31.899 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:20:31.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.999+0000 7fd0d3514640 1 Processor -- start 2024-04-26T21:20:32.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:31.999+0000 7fd0d3514640 1 -- start start 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d3514640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0cc148a30 0x7fd0cc073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d3514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc160640 0x7fd0cc1636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d3514640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0cc0cfa80 con 0x7fd0cc163be0 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d3514640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0cc0cfd60 con 0x7fd0cc148a30 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d3514640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc0d0040 con 0x7fd0cc160640 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d12b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0cc148a30 0x7fd0cc073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d12b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0cc148a30 0x7fd0cc073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56296/0 (socket says 172.21.15.40:56296) 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.000+0000 7fd0d12b0640 1 -- 172.21.15.40:0/1740726756 learned_addr learned my addr 172.21.15.40:0/1740726756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d0aaf640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc160640 0x7fd0cc1636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d12b0640 1 -- 172.21.15.40:0/1740726756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc160640 msgr2=0x7fd0cc1636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d12b0640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc160640 0x7fd0cc1636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d12b0640 1 -- 172.21.15.40:0/1740726756 >> v1:172.21.15.184:6789/0 conn(0x7fd0cc163be0 legacy=0x7fd0cc0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d12b0640 1 -- 172.21.15.40:0/1740726756 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc162ea0 con 0x7fd0cc148a30 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0d12b0640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc148a30 0x7fd0cc073ed0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009870 tx=0x7fd0c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df5940a13beed64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.001+0000 7fd0cbfff640 1 -- 172.21.15.40:0/1740726756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4037070 con 0x7fd0cc148a30 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0cbfff640 1 -- 172.21.15.40:0/1740726756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c4032440 con 0x7fd0cc148a30 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0cbfff640 1 -- 172.21.15.40:0/1740726756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4032810 con 0x7fd0cc148a30 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 -- 172.21.15.40:0/1740726756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc148a30 msgr2=0x7fd0cc073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc148a30 0x7fd0cc073ed0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009870 tx=0x7fd0c402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 -- 172.21.15.40:0/1740726756 shutdown_connections 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc160640 0x7fd0cc1636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1740726756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc148a30 0x7fd0cc073ed0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 -- 172.21.15.40:0/1740726756 >> 172.21.15.40:0/1740726756 conn(0x7fd0cc0ce8b0 msgr2=0x7fd0cc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:32.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 -- 172.21.15.40:0/1740726756 shutdown_connections 2024-04-26T21:20:32.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.002+0000 7fd0d3514640 1 -- 172.21.15.40:0/1740726756 wait complete. 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.003+0000 7fd0d3514640 1 Processor -- start 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.003+0000 7fd0d3514640 1 -- start start 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc148a30 0x7fd0cc0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 0x7fd0cc1394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc074590 con 0x7fd0cc160640 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc074410 con 0x7fd0cc163be0 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d3514640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc148e10 con 0x7fd0cc148a30 2024-04-26T21:20:32.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d0aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d0aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56304/0 (socket says 172.21.15.40:56304) 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d0aaf640 1 -- 172.21.15.40:0/1642485193 learned_addr learned my addr 172.21.15.40:0/1642485193 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d12b0640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc148a30 0x7fd0cc0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.004+0000 7fd0d1ab1640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 0x7fd0cc1394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 -- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc148a30 msgr2=0x7fd0cc0b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc148a30 0x7fd0cc0b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 -- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 msgr2=0x7fd0cc1394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 0x7fd0cc1394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 -- 172.21.15.40:0/1642485193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc0aefb0 con 0x7fd0cc160640 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d1ab1640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 0x7fd0cc1394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d0aaf640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000d7d0 tx=0x7fd0c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000b6c0 con 0x7fd0cc160640 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c0019070 con 0x7fd0cc160640 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc0af290 con 0x7fd0cc160640 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000ba80 con 0x7fd0cc160640 2024-04-26T21:20:32.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.005+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc0af850 con 0x7fd0cc160640 2024-04-26T21:20:32.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.006+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc06a290 con 0x7fd0cc160640 2024-04-26T21:20:32.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.006+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0c000bc20 con 0x7fd0cc160640 2024-04-26T21:20:32.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.007+0000 7fd0c9ffb640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 0x7fd0b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.007+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7fd0c008bc80 con 0x7fd0cc160640 2024-04-26T21:20:32.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.007+0000 7fd0d12b0640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 0x7fd0b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.008+0000 7fd0d12b0640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 0x7fd0b806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4034000 tx=0x7fd0c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.011+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0c005a3b0 con 0x7fd0cc160640 2024-04-26T21:20:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.197+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0cc0739f0 con 0x7fd0cc160640 2024-04-26T21:20:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.197+0000 7fd0c9ffb640 1 -- 172.21.15.40:0/1642485193 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+13030 (secure 0 0 0) 0x7fd0c0059cb0 con 0x7fd0cc160640 2024-04-26T21:20:32.199 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:32.199 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":125,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:29.874210+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/4242446604":"2024-04-27T21:20:26.578374+0000","172.21.15.40:6833/4242446604":"2024-04-27T21:20:26.578374+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 msgr2=0x7fd0b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 0x7fd0b806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4034000 tx=0x7fd0c402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 msgr2=0x7fd0cc0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000d7d0 tx=0x7fd0c000dc90 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 shutdown_connections 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd0b806c8f0 0x7fd0b806edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc163be0 0x7fd0cc1394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc160640 0x7fd0cc0ae8b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 --2- 172.21.15.40:0/1642485193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0cc148a30 0x7fd0cc0b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.199+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 >> 172.21.15.40:0/1642485193 conn(0x7fd0cc0ce8b0 msgr2=0x7fd0cc146db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.200+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 shutdown_connections 2024-04-26T21:20:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.200+0000 7fd0d3514640 1 -- 172.21.15.40:0/1642485193 wait complete. 2024-04-26T21:20:32.211 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:20:32.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:20:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.314+0000 7f29283b6640 1 Processor -- start 2024-04-26T21:20:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.314+0000 7f29283b6640 1 -- start start 2024-04-26T21:20:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f29283b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f29200cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f29283b6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29200b57c0 con 0x7f2920143840 2024-04-26T21:20:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f29283b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292015dca0 con 0x7f29200b4eb0 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f29283b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f2920143c20 con 0x7f29200b5330 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2926152640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2920143840 0x7f29200b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51780/0 (socket says 172.21.15.40:51780) 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2926152640 1 -- 172.21.15.40:0/4120407866 learned_addr learned my addr 172.21.15.40:0/4120407866 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2926953640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f29200cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890848795 0 0) 0x7f29200b57c0 con 0x7f2920143840 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f290c003520 con 0x7f2920143840 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1806568804 0 0) 0x7f290c003520 con 0x7f2920143840 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 msgr2=0x7f29200cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f29200cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 >> v1:172.21.15.184:6789/0 conn(0x7f29200b4eb0 legacy=0x7f29200ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292015df80 con 0x7f2920143840 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f291c001670 con 0x7f2920143840 2024-04-26T21:20:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 >> v1:172.21.15.40:6789/0 conn(0x7f2920143840 legacy=0x7f29200b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.315+0000 7f2925150640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f290c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925150640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f290c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925150640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f290c004d40 0x7f290c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2920143c20 con 0x7f290c003f60 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f290c00baa0 con 0x7f290c004d40 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f290c00bca0 con 0x7f29200b5330 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f290c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2926152640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f290c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2926953640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f290c004d40 0x7f290c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 -- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 msgr2=0x7f290c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200b5330 0x7f290c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 -- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f290c004d40 msgr2=0x7f290c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f290c004d40 0x7f290c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.316+0000 7f2925951640 1 -- 172.21.15.40:0/4120407866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c003520 con 0x7f290c003f60 2024-04-26T21:20:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f2925951640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f290c004800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2910001750 tx=0x7f290c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac909697df1157e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910001370 con 0x7f290c003f60 2024-04-26T21:20:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29100052d0 con 0x7f290c003f60 2024-04-26T21:20:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f2925150640 1 -- 172.21.15.40:0/4120407866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291002abe0 con 0x7f290c003f60 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 -- 172.21.15.40:0/4120407866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 msgr2=0x7f290c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 --2- 172.21.15.40:0/4120407866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f290c004800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2910001750 tx=0x7f290c005830 comp rx=0 tx=0).stop 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f2926953640 1 -- 172.21.15.40:0/4120407866 reap_dead start 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 -- 172.21.15.40:0/4120407866 shutdown_connections 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 -- 172.21.15.40:0/4120407866 >> 172.21.15.40:0/4120407866 conn(0x7f29200cd8a0 msgr2=0x7f29200cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 -- 172.21.15.40:0/4120407866 shutdown_connections 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.317+0000 7f29283b6640 1 -- 172.21.15.40:0/4120407866 wait complete. 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 Processor -- start 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 -- start start 2024-04-26T21:20:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920145740 0x7f292013a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f292013a970 0x7f292013f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29200b46f0 con 0x7f290c003f60 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f292015e540 con 0x7f2920145740 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f29283b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f292015d8c0 con 0x7f292013a970 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f2926152640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f2926152640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56324/0 (socket says 172.21.15.40:56324) 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.319+0000 7f2926152640 1 -- 172.21.15.40:0/2451262418 learned_addr learned my addr 172.21.15.40:0/2451262418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2925951640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920145740 0x7f292013a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926953640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f292013a970 0x7f292013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 -- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f292013a970 msgr2=0x7f292013f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f292013a970 0x7f292013f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 -- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920145740 msgr2=0x7f292013a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920145740 0x7f292013a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 -- 172.21.15.40:0/2451262418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2920145d00 con 0x7f290c003f60 2024-04-26T21:20:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2926152640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f291c007290 tx=0x7f291c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c00f960 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c018070 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c00fd60 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2920145fe0 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.320+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201465a0 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.321+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292006a290 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.322+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f291c01d460 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.322+0000 7f2916ffd640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 0x7f290406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.322+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f291c08b820 con 0x7f290c003f60 2024-04-26T21:20:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.322+0000 7f2925951640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 0x7f290406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.323+0000 7f2925951640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 0x7f290406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2910032000 tx=0x7f291002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.326+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f291c059900 con 0x7f290c003f60 2024-04-26T21:20:32.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.515+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29200ce420 con 0x7f290c003f60 2024-04-26T21:20:32.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.516+0000 7f2916ffd640 1 -- 172.21.15.40:0/2451262418 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7f291c05dc00 con 0x7f290c003f60 2024-04-26T21:20:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 msgr2=0x7f290406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 0x7f290406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2910032000 tx=0x7f291002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 msgr2=0x7f292015a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f291c007290 tx=0x7f291c007750 comp rx=0 tx=0).stop 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 shutdown_connections 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290406c8f0 0x7f290406edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f292013a970 0x7f292013f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920145740 0x7f292013a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 --2- 172.21.15.40:0/2451262418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f290c003f60 0x7f292015a690 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 >> 172.21.15.40:0/2451262418 conn(0x7f29200cd8a0 msgr2=0x7f292015f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 shutdown_connections 2024-04-26T21:20:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.518+0000 7f29283b6640 1 -- 172.21.15.40:0/2451262418 wait complete. 2024-04-26T21:20:32.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-04-26T21:20:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 Processor -- start 2024-04-26T21:20:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 -- start start 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf5c148c10 0x7fcf5c1639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcf5c0cf9c0 0x7fcf5c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf5c0d0130 con 0x7fcf5c0b3b60 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcf5c148ff0 con 0x7fcf5c148c10 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.631+0000 7fcf630b0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcf5c0cfe50 con 0x7fcf5c0cf9c0 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf5c148c10 0x7fcf5c1639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf5c148c10 0x7fcf5c1639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56332/0 (socket says 172.21.15.40:56332) 2024-04-26T21:20:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 -- 172.21.15.40:0/4248015723 learned_addr learned my addr 172.21.15.40:0/4248015723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/4248015723 >> v2:172.21.15.184:3300/0 conn(0x7fcf5c0cf9c0 0x7fcf5c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 -- 172.21.15.40:0/4248015723 >> v2:172.21.15.184:3300/0 conn(0x7fcf5c0cf9c0 msgr2=0x7fcf5c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 --2- 172.21.15.40:0/4248015723 >> v2:172.21.15.184:3300/0 conn(0x7fcf5c0cf9c0 0x7fcf5c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 -- 172.21.15.40:0/4248015723 >> v1:172.21.15.184:6789/0 conn(0x7fcf5c0b3b60 legacy=0x7fcf5c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.632+0000 7fcf60e4c640 1 -- 172.21.15.40:0/4248015723 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c162ee0 con 0x7fcf5c148c10 2024-04-26T21:20:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf60e4c640 1 --2- 172.21.15.40:0/4248015723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c148c10 0x7fcf5c1639c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0d0e50 tx=0x7fcf5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f525e195bf66c4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf5b7fe640 1 -- 172.21.15.40:0/4248015723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50037070 con 0x7fcf5c148c10 2024-04-26T21:20:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf5b7fe640 1 -- 172.21.15.40:0/4248015723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50032440 con 0x7fcf5c148c10 2024-04-26T21:20:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf5b7fe640 1 -- 172.21.15.40:0/4248015723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50032810 con 0x7fcf5c148c10 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 -- 172.21.15.40:0/4248015723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c148c10 msgr2=0x7fcf5c1639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 --2- 172.21.15.40:0/4248015723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c148c10 0x7fcf5c1639c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0d0e50 tx=0x7fcf5002a270 comp rx=0 tx=0).stop 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 -- 172.21.15.40:0/4248015723 shutdown_connections 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 --2- 172.21.15.40:0/4248015723 >> v2:172.21.15.184:3300/0 conn(0x7fcf5c0cf9c0 0x7fcf5c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 --2- 172.21.15.40:0/4248015723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c148c10 0x7fcf5c1639c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 -- 172.21.15.40:0/4248015723 >> 172.21.15.40:0/4248015723 conn(0x7fcf5c0ce920 msgr2=0x7fcf5c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 -- 172.21.15.40:0/4248015723 shutdown_connections 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.633+0000 7fcf630b0640 1 -- 172.21.15.40:0/4248015723 wait complete. 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 Processor -- start 2024-04-26T21:20:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 -- start start 2024-04-26T21:20:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf5c0b3b60 0x7fcf5c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 0x7fcf5c142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c0d19b0 con 0x7fcf5c161a20 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1634a0 con 0x7fcf5c0b3b60 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.635+0000 7fcf630b0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5c0b3fb0 con 0x7fcf5c148c10 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56340/0 (socket says 172.21.15.40:56340) 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 -- 172.21.15.40:0/3817677830 learned_addr learned my addr 172.21.15.40:0/3817677830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 0x7fcf5c142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf60e4c640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf5c0b3b60 0x7fcf5c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 -- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 msgr2=0x7fcf5c142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 0x7fcf5c142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 -- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf5c0b3b60 msgr2=0x7fcf5c0b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf5c0b3b60 0x7fcf5c0b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 -- 172.21.15.40:0/3817677830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c142e40 con 0x7fcf5c161a20 2024-04-26T21:20:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 0x7fcf5c142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf6164d640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400c7a0 tx=0x7fcf5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54003b70 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf54003d30 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c0b5610 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400aa90 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.636+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c0b6fe0 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.637+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5c06a290 con 0x7fcf5c161a20 2024-04-26T21:20:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.638+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf5400ac30 con 0x7fcf5c161a20 2024-04-26T21:20:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.638+0000 7fcf597fa640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 0x7fcf4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.638+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7fcf5408b7b0 con 0x7fcf5c161a20 2024-04-26T21:20:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.639+0000 7fcf60e4c640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 0x7fcf4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.639+0000 7fcf60e4c640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 0x7fcf4006ede0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcf50034000 tx=0x7fcf5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.643+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf54059ee0 con 0x7fcf5c161a20 2024-04-26T21:20:32.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:20:32.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:20:32.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:32.901+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7fcf5c0739d0 con 0x7fcf5c161a20 2024-04-26T21:20:33.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.891+0000 7fcf597fa640 1 -- 172.21.15.40:0/3817677830 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v443) ==== 151+0+0 (secure 0 0 0) 0x7fcf540597e0 con 0x7fcf5c161a20 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 msgr2=0x7fcf4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 0x7fcf4006ede0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcf50034000 tx=0x7fcf5002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 msgr2=0x7fcf5c161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400c7a0 tx=0x7fcf5400cc60 comp rx=0 tx=0).stop 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 shutdown_connections 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcf4006c920 0x7fcf4006ede0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf5c161a20 0x7fcf5c161eb0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:33.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf5c148c10 0x7fcf5c142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:33.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 --2- 172.21.15.40:0/3817677830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf5c0b3b60 0x7fcf5c0b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:33.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 >> 172.21.15.40:0/3817677830 conn(0x7fcf5c0ce920 msgr2=0x7fcf5c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:33.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 shutdown_connections 2024-04-26T21:20:33.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:33.894+0000 7fcf630b0640 1 -- 172.21.15.40:0/3817677830 wait complete. 2024-04-26T21:20:33.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:20:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.017+0000 7efd49bc6640 1 Processor -- start 2024-04-26T21:20:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.017+0000 7efd49bc6640 1 -- start start 2024-04-26T21:20:34.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd49bc6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd440d0010 0x7efd4406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd49bc6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4409c700 con 0x7efd440b3c20 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd49bc6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd440d1c50 con 0x7efd440b33e0 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd49bc6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efd440b4480 con 0x7efd440d0010 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd42ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd440b3c20 0x7efd440b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51786/0 (socket says 172.21.15.40:51786) 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd42ffd640 1 -- 172.21.15.40:0/4110469627 learned_addr learned my addr 172.21.15.40:0/4110469627 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd437fe640 1 --2- 172.21.15.40:0/4110469627 >> v2:172.21.15.184:3300/0 conn(0x7efd440d0010 0x7efd4406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210317779 0 0) 0x7efd4409c700 con 0x7efd440b3c20 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd30003520 con 0x7efd440b3c20 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2901871500 0 0) 0x7efd30003520 con 0x7efd440b3c20 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 >> v2:172.21.15.184:3300/0 conn(0x7efd440d0010 msgr2=0x7efd4406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 --2- 172.21.15.40:0/4110469627 >> v2:172.21.15.184:3300/0 conn(0x7efd440d0010 0x7efd4406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 >> v1:172.21.15.184:6789/0 conn(0x7efd440b33e0 legacy=0x7efd44163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd440d1f30 con 0x7efd440b3c20 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.018+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7efd34001670 con 0x7efd440b3c20 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 >> v1:172.21.15.40:6789/0 conn(0x7efd440b3c20 legacy=0x7efd440b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd300039b0 0x7efd30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd30004800 0x7efd30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 0x7efd3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd440b4480 con 0x7efd300051c0 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd3000c060 con 0x7efd300039b0 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd3000c260 con 0x7efd30004800 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 0x7efd3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd437fe640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd300039b0 0x7efd30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd42ffd640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd30004800 0x7efd30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 -- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd30004800 msgr2=0x7efd30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd30004800 0x7efd30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 -- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd300039b0 msgr2=0x7efd30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd300039b0 0x7efd30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.019+0000 7efd43fff640 1 -- 172.21.15.40:0/4110469627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd30003520 con 0x7efd300051c0 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd43fff640 1 -- 172.21.15.40:0/4110469627 reap_dead start 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd43fff640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 0x7efd3000b9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001e50 tx=0x7efd30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9256e670c54600f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c001a70 con 0x7efd300051c0 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c003970 con 0x7efd300051c0 2024-04-26T21:20:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd427fc640 1 -- 172.21.15.40:0/4110469627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c003d60 con 0x7efd300051c0 2024-04-26T21:20:34.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 -- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 msgr2=0x7efd3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 0x7efd3000b9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001e50 tx=0x7efd30005830 comp rx=0 tx=0).stop 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 -- 172.21.15.40:0/4110469627 shutdown_connections 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 --2- 172.21.15.40:0/4110469627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd300051c0 0x7efd3000b9c0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 -- 172.21.15.40:0/4110469627 >> 172.21.15.40:0/4110469627 conn(0x7efd4409ba40 msgr2=0x7efd4409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 -- 172.21.15.40:0/4110469627 shutdown_connections 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.020+0000 7efd49bc6640 1 -- 172.21.15.40:0/4110469627 wait complete. 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.022+0000 7efd49bc6640 1 Processor -- start 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.022+0000 7efd49bc6640 1 -- start start 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.022+0000 7efd49bc6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd300051c0 0x7efd441f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd4415cdc0 0x7efd4415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd4406db10 con 0x7efd4409d030 2024-04-26T21:20:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd440b3a20 con 0x7efd4415cdc0 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efd4406dc90 con 0x7efd300051c0 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56350/0 (socket says 172.21.15.40:56350) 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 -- 172.21.15.40:0/2244373715 learned_addr learned my addr 172.21.15.40:0/2244373715 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd437fe640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd300051c0 0x7efd441f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd43fff640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd4415cdc0 0x7efd4415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 -- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd300051c0 msgr2=0x7efd441f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd300051c0 0x7efd441f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 -- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd4415cdc0 msgr2=0x7efd4415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd4415cdc0 0x7efd4415d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 -- 172.21.15.40:0/2244373715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4409e7e0 con 0x7efd4409d030 2024-04-26T21:20:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd42ffd640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efd34001050 tx=0x7efd34007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34004e10 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd34018070 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400fbc0 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4415c2e0 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.023+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441f8370 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.024+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4406a290 con 0x7efd4409d030 2024-04-26T21:20:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.025+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd3401d460 con 0x7efd4409d030 2024-04-26T21:20:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.025+0000 7efd3bfff640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 0x7efd2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.025+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7efd3408b780 con 0x7efd4409d030 2024-04-26T21:20:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.026+0000 7efd437fe640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 0x7efd2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.026+0000 7efd437fe640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 0x7efd2406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009b40 tx=0x7efd2c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.030+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd34059f40 con 0x7efd4409d030 2024-04-26T21:20:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.216+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efd440b8f10 con 0x7efd4409d030 2024-04-26T21:20:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.219+0000 7efd3bfff640 1 -- 172.21.15.40:0/2244373715 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v56)=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7efd34059840 con 0x7efd4409d030 2024-04-26T21:20:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 msgr2=0x7efd2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 0x7efd2406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009b40 tx=0x7efd2c009190 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 msgr2=0x7efd4409de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efd34001050 tx=0x7efd34007660 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 shutdown_connections 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efd2406c8f0 0x7efd2406edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd4415cdc0 0x7efd4415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd4409d030 0x7efd4409de40 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 --2- 172.21.15.40:0/2244373715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd300051c0 0x7efd441f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 >> 172.21.15.40:0/2244373715 conn(0x7efd4409ba40 msgr2=0x7efd4406dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 shutdown_connections 2024-04-26T21:20:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.221+0000 7efd49bc6640 1 -- 172.21.15.40:0/2244373715 wait complete. 2024-04-26T21:20:34.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:20:34.233 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:20:34.272 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:20:34.324 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:20:34.324 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:34.324 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:34.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:20:34.340 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:34.340 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:34.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:20:34.358 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:34.358 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:34.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:20:34.395 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:34.395 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:34.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:20:34.415 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:20:34.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.534+0000 7f86ca4da640 1 Processor -- start 2024-04-26T21:20:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.534+0000 7f86ca4da640 1 -- start start 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86ca4da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86c415f1e0 0x7f86c406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86ca4da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86c406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86ca4da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c40b4a10 con 0x7f86c40b44e0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86ca4da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f86c406eeb0 con 0x7f86c415f1e0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86ca4da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86c40b4cf0 con 0x7f86c406d2b0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86c37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86c40b44e0 0x7f86c40ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51802/0 (socket says 172.21.15.40:51802) 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.535+0000 7f86c37fe640 1 -- 172.21.15.40:0/837516266 learned_addr learned my addr 172.21.15.40:0/837516266 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454587269 0 0) 0x7f86c40b4a10 con 0x7f86c40b44e0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86ac003520 con 0x7f86c40b44e0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c3fff640 1 --2- 172.21.15.40:0/837516266 >> v2:172.21.15.184:3300/0 conn(0x7f86c415f1e0 0x7f86c406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86c406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1387588567 0 0) 0x7f86ac003520 con 0x7f86c40b44e0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 msgr2=0x7f86c406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86c406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 >> v2:172.21.15.184:3300/0 conn(0x7f86c415f1e0 msgr2=0x7f86c406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 --2- 172.21.15.40:0/837516266 >> v2:172.21.15.184:3300/0 conn(0x7f86c415f1e0 0x7f86c406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40b4fd0 con 0x7f86c40b44e0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f86b8001670 con 0x7f86c40b44e0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 >> v1:172.21.15.40:6789/0 conn(0x7f86c40b44e0 legacy=0x7f86c40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86ac0039b0 0x7f86ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86ac004d40 0x7f86ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86c406eeb0 con 0x7f86ac004d40 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b4cf0 con 0x7f86ac0039b0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86ac00ba70 con 0x7f86c406d2b0 2024-04-26T21:20:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86ac004d40 0x7f86ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c37fe640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c3fff640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86ac0039b0 0x7f86ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 -- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 msgr2=0x7f86ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c406d2b0 0x7f86ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 -- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86ac0039b0 msgr2=0x7f86ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86ac0039b0 0x7f86ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.536+0000 7f86c8a77640 1 -- 172.21.15.40:0/837516266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac003520 con 0x7f86ac004d40 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.537+0000 7f86c8a77640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86ac004d40 0x7f86ac0051c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f86bc007390 tx=0x7f86ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c23347f660b2c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.537+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc009be0 con 0x7f86ac004d40 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.537+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86bc02e620 con 0x7f86ac004d40 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.537+0000 7f86c2ffd640 1 -- 172.21.15.40:0/837516266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc02e9f0 con 0x7f86ac004d40 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 -- 172.21.15.40:0/837516266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86ac004d40 msgr2=0x7f86ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 --2- 172.21.15.40:0/837516266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86ac004d40 0x7f86ac0051c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f86bc007390 tx=0x7f86ac005830 comp rx=0 tx=0).stop 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86c8a77640 1 -- 172.21.15.40:0/837516266 reap_dead start 2024-04-26T21:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 -- 172.21.15.40:0/837516266 shutdown_connections 2024-04-26T21:20:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 -- 172.21.15.40:0/837516266 >> 172.21.15.40:0/837516266 conn(0x7f86c40728a0 msgr2=0x7f86c4072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 -- 172.21.15.40:0/837516266 shutdown_connections 2024-04-26T21:20:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.538+0000 7f86ca4da640 1 -- 172.21.15.40:0/837516266 wait complete. 2024-04-26T21:20:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.539+0000 7f86ca4da640 1 Processor -- start 2024-04-26T21:20:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.539+0000 7f86ca4da640 1 -- start start 2024-04-26T21:20:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 0x7f86c4160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 0x7f86c4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86c415f5c0 con 0x7f86c4150ac0 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b5590 con 0x7f86c40d0a20 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86ca4da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86c4152e50 con 0x7f86c4161fe0 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56376/0 (socket says 172.21.15.40:56376) 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 -- 172.21.15.40:0/3847878891 learned_addr learned my addr 172.21.15.40:0/3847878891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c3fff640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 0x7f86c4160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c37fe640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 0x7f86c4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 -- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 msgr2=0x7f86c4153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 0x7f86c4153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 -- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 msgr2=0x7f86c4160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 0x7f86c4160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c8a77640 1 -- 172.21.15.40:0/3847878891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c4160f40 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c3fff640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 0x7f86c4160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.540+0000 7f86c37fe640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 0x7f86c4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86c8a77640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0074b0 tx=0x7f86bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc009be0 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86bc02e420 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c4161220 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc038280 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.541+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41617e0 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.542+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c406a290 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.542+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86bc038890 con 0x7f86c4150ac0 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.543+0000 7f86c0ff9640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 0x7f86b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.543+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f86bc0abef0 con 0x7f86c4150ac0 2024-04-26T21:20:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.543+0000 7f86c3fff640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 0x7f86b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.544+0000 7f86c3fff640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 0x7f86b006edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.547+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86bc032030 con 0x7f86c4150ac0 2024-04-26T21:20:34.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.770+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c41635b0 con 0x7f86c4150ac0 2024-04-26T21:20:34.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.771+0000 7f86c0ff9640 1 -- 172.21.15.40:0/3847878891 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+3875 (secure 0 0 0) 0x7f86bc07a590 con 0x7f86c4150ac0 2024-04-26T21:20:34.772 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:34.772 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3231482304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3231482304},{"type":"v1","addr":"172.21.15.184:6836","nonce":3231482304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2794720429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2794720429},{"type":"v1","addr":"172.21.15.184:6837","nonce":2794720429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/3327457893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3327457893},{"type":"v1","addr":"172.21.15.40:6834","nonce":3327457893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2627755319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2627755319},{"type":"v1","addr":"172.21.15.40:6835","nonce":2627755319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-04-26T21:20:34.772 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 81 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 msgr2=0x7f86b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 0x7f86b006edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4008040 comp rx=0 tx=0).stop 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 msgr2=0x7f86c4150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0074b0 tx=0x7f86bc004ab0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 shutdown_connections 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86b006c8f0 0x7f86b006edb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c4150ac0 0x7f86c4150f50 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c4161fe0 0x7f86c4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 --2- 172.21.15.40:0/3847878891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c40d0a20 0x7f86c4160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 >> 172.21.15.40:0/3847878891 conn(0x7f86c40728a0 msgr2=0x7f86c40702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 shutdown_connections 2024-04-26T21:20:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:34.773+0000 7f86ca4da640 1 -- 172.21.15.40:0/3847878891 wait complete. 2024-04-26T21:20:34.785 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:20:34.785 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:20:34.785 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:20:34.785 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:20:34.786 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:20:34.786 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:20:34.786 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:20:34.786 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:20:34.786 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:20:34.786 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:20:34.787 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:20:34.787 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:20:34.787 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:20:34.787 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:20:34.787 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:20:34.787 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:20:34.796 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:20:34.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:20:34.801 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:20:34.801 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:20:34.802 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:20:34.802 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:20:34.838 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:20:34.839 DEBUG:teuthology.parallel:result is None 2024-04-26T21:20:34.844 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:20:34.844 DEBUG:teuthology.parallel:result is None 2024-04-26T21:20:34.845 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:20:34.845 DEBUG:teuthology.parallel:result is None 2024-04-26T21:20:34.913 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:20:34.925 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:20:34.925 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:20:34.925 INFO:tasks.ceph.mds.d.smithi184.stderr:2024-04-26T21:20:34.924+0000 7f3b69047640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T21:20:40.796 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:20:40.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:20:40.799 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:20:40.799 DEBUG:teuthology.parallel:result is None 2024-04-26T21:20:40.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:20:40.885 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:20:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.914+0000 7f978e493640 1 Processor -- start 2024-04-26T21:20:40.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.914+0000 7f978e493640 1 -- start start 2024-04-26T21:20:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97880b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f97880d3320 0x7f97880d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97880d51a0 0x7f97880d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f97880d0030 con 0x7f97880d51a0 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f97880d4060 con 0x7f97880d3320 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978e493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97880d2440 con 0x7f978815f750 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978ca30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97880d51a0 0x7f97880d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978ca30640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97880d51a0 0x7f97880d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34940/0 (socket says 172.21.15.40:34940) 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f978ca30640 1 -- 172.21.15.40:0/451531250 learned_addr learned my addr 172.21.15.40:0/451531250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f9787fff640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97880b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.915+0000 7f977ffff640 1 --2- 172.21.15.40:0/451531250 >> v2:172.21.15.184:3300/0 conn(0x7f97880d3320 0x7f97880d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 -- 172.21.15.40:0/451531250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 msgr2=0x7f97880b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97880b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 -- 172.21.15.40:0/451531250 >> v2:172.21.15.184:3300/0 conn(0x7f97880d3320 msgr2=0x7f97880d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 --2- 172.21.15.40:0/451531250 >> v2:172.21.15.184:3300/0 conn(0x7f97880d3320 0x7f97880d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 -- 172.21.15.40:0/451531250 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97880d2720 con 0x7f97880d51a0 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.916+0000 7f978ca30640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f97880d5630 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f978808fa60 tx=0x7f978002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d057e5bf36e439 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.917+0000 7f97877fe640 1 -- 172.21.15.40:0/451531250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780036070 con 0x7f97880d51a0 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.917+0000 7f97877fe640 1 -- 172.21.15.40:0/451531250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9780031600 con 0x7f97880d51a0 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.917+0000 7f97877fe640 1 -- 172.21.15.40:0/451531250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97800319d0 con 0x7f97880d51a0 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 -- 172.21.15.40:0/451531250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 msgr2=0x7f97880d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:40.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f97880d5630 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f978808fa60 tx=0x7f978002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 -- 172.21.15.40:0/451531250 shutdown_connections 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f97880d5630 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 --2- 172.21.15.40:0/451531250 >> v2:172.21.15.184:3300/0 conn(0x7f97880d3320 0x7f97880d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 --2- 172.21.15.40:0/451531250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97880b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 -- 172.21.15.40:0/451531250 >> 172.21.15.40:0/451531250 conn(0x7f978808ec20 msgr2=0x7f978809e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 -- 172.21.15.40:0/451531250 shutdown_connections 2024-04-26T21:20:40.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.918+0000 7f978e493640 1 -- 172.21.15.40:0/451531250 wait complete. 2024-04-26T21:20:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.919+0000 7f978e493640 1 Processor -- start 2024-04-26T21:20:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.919+0000 7f978e493640 1 -- start start 2024-04-26T21:20:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 0x7f9788153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97881633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97880d0640 con 0x7f97880d51a0 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97880d04c0 con 0x7f97880d3320 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978e493640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97880d07c0 con 0x7f978815f750 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34942/0 (socket says 172.21.15.40:34942) 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 -- 172.21.15.40:0/4041071973 learned_addr learned my addr 172.21.15.40:0/4041071973 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f9787fff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 0x7f9788153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f978ca30640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97881633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 -- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 msgr2=0x7f97881633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97881633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 -- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 msgr2=0x7f9788153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 0x7f9788153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 -- 172.21.15.40:0/4041071973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97880cc380 con 0x7f97880d51a0 2024-04-26T21:20:40.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f9787fff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 0x7f9788153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.920+0000 7f977ffff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f977000e8c0 tx=0x7f977000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.922+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770017070 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.922+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977000c940 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.922+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97880cc620 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.922+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977000cd10 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.922+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9788162900 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.923+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97880693d0 con 0x7f97880d51a0 2024-04-26T21:20:40.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.923+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f977001a070 con 0x7f97880d51a0 2024-04-26T21:20:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.924+0000 7f97857fa640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 0x7f976806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.924+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4943+0+0 (secure 0 0 0) 0x7f977008bbc0 con 0x7f97880d51a0 2024-04-26T21:20:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.924+0000 7f9787fff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 0x7f976806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:40.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.925+0000 7f9787fff640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 0x7f976806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9778009af0 tx=0x7f9778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:40.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:40.928+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9770059c10 con 0x7f97880d51a0 2024-04-26T21:20:41.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-26T21:20:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.127+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f97880d60c0 con 0x7f97880d51a0 2024-04-26T21:20:41.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.923+0000 7f97857fa640 1 -- 172.21.15.40:0/4041071973 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v130) ==== 97+0+0 (secure 0 0 0) 0x7f977005df10 con 0x7f97880d51a0 2024-04-26T21:20:41.924 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 msgr2=0x7f976806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 0x7f976806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9778009af0 tx=0x7f9778008040 comp rx=0 tx=0).stop 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 msgr2=0x7f9788161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f977000e8c0 tx=0x7f977000ed80 comp rx=0 tx=0).stop 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 shutdown_connections 2024-04-26T21:20:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f976806c8f0 0x7f976806edb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f978815f750 0x7f97881633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97880d51a0 0x7f9788161f20 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 --2- 172.21.15.40:0/4041071973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97880d3320 0x7f9788153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 >> 172.21.15.40:0/4041071973 conn(0x7f978808ec20 msgr2=0x7f97880d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 shutdown_connections 2024-04-26T21:20:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:41.926+0000 7f978e493640 1 -- 172.21.15.40:0/4041071973 wait complete. 2024-04-26T21:20:41.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:20:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.047+0000 7fe2719af640 1 Processor -- start 2024-04-26T21:20:42.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.047+0000 7fe2719af640 1 -- start start 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe2719af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 0x7fe26c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe2719af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 0x7fe26c0d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe2719af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c0d2440 con 0x7fe26c0d3320 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe2719af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c0d0030 con 0x7fe26c15f750 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe2719af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c0d2720 con 0x7fe26c0d51a0 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe26a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe26c0d3320 0x7fe26c0d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57684/0 (socket says 172.21.15.40:57684) 2024-04-26T21:20:42.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.048+0000 7fe26a7fc640 1 -- 172.21.15.40:0/4070150020 learned_addr learned my addr 172.21.15.40:0/4070150020 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26b7fe640 1 --2- 172.21.15.40:0/4070150020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 0x7fe26c0d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26affd640 1 --2- 172.21.15.40:0/4070150020 >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 0x7fe26c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe269ffb640 1 -- 172.21.15.40:0/4070150020 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112189235 0 0) 0x7fe26c0d2440 con 0x7fe26c0d3320 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe269ffb640 1 -- 172.21.15.40:0/4070150020 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe254003520 con 0x7fe26c0d3320 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26b7fe640 1 -- 172.21.15.40:0/4070150020 >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 msgr2=0x7fe26c0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26b7fe640 1 --2- 172.21.15.40:0/4070150020 >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 0x7fe26c0b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26b7fe640 1 -- 172.21.15.40:0/4070150020 >> v1:172.21.15.40:6789/0 conn(0x7fe26c0d3320 legacy=0x7fe26c0d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26b7fe640 1 -- 172.21.15.40:0/4070150020 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c0d2a00 con 0x7fe26c0d51a0 2024-04-26T21:20:42.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.049+0000 7fe26affd640 1 --2- 172.21.15.40:0/4070150020 >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 0x7fe26c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe26b7fe640 1 --2- 172.21.15.40:0/4070150020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 0x7fe26c0d5650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009d90 tx=0x7fe25c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=250a93499e6dd90a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe269ffb640 1 -- 172.21.15.40:0/4070150020 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c007740 con 0x7fe26c0d51a0 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe269ffb640 1 -- 172.21.15.40:0/4070150020 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c0078e0 con 0x7fe26c0d51a0 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe269ffb640 1 -- 172.21.15.40:0/4070150020 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c007c90 con 0x7fe26c0d51a0 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 -- 172.21.15.40:0/4070150020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 msgr2=0x7fe26c0d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 --2- 172.21.15.40:0/4070150020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 0x7fe26c0d5650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009d90 tx=0x7fe25c02a660 comp rx=0 tx=0).stop 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 -- 172.21.15.40:0/4070150020 shutdown_connections 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 --2- 172.21.15.40:0/4070150020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c0d51a0 0x7fe26c0d5650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 --2- 172.21.15.40:0/4070150020 >> v2:172.21.15.184:3300/0 conn(0x7fe26c15f750 0x7fe26c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.050+0000 7fe2719af640 1 -- 172.21.15.40:0/4070150020 >> 172.21.15.40:0/4070150020 conn(0x7fe26c08ec20 msgr2=0x7fe26c09e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.051+0000 7fe2719af640 1 -- 172.21.15.40:0/4070150020 shutdown_connections 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.051+0000 7fe2719af640 1 -- 172.21.15.40:0/4070150020 wait complete. 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.052+0000 7fe2719af640 1 Processor -- start 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.052+0000 7fe2719af640 1 -- start start 2024-04-26T21:20:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.052+0000 7fe2719af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.052+0000 7fe2719af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 0x7fe26c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c15f750 0x7fe26c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0d0890 con 0x7fe26c0d3320 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0d0710 con 0x7fe26c0d51a0 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c0d0a10 con 0x7fe26c15f750 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34952/0 (socket says 172.21.15.40:34952) 2024-04-26T21:20:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 -- 172.21.15.40:0/4246444842 learned_addr learned my addr 172.21.15.40:0/4246444842 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26b7fe640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c15f750 0x7fe26c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26a7fc640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 0x7fe26c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 -- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c15f750 msgr2=0x7fe26c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c15f750 0x7fe26c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 -- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 msgr2=0x7fe26c0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 0x7fe26c0b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 -- 172.21.15.40:0/4246444842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c0b2a60 con 0x7fe26c0d3320 2024-04-26T21:20:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26a7fc640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 0x7fe26c0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe26affd640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fe26400e960 tx=0x7fe26400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264017070 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26400ca00 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c0b2d40 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26400cdd0 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.053+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c0b3300 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.054+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0b3740 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.055+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe26401a070 con 0x7fe26c0d3320 2024-04-26T21:20:42.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.055+0000 7fe253fff640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 0x7fe25806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.055+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4857+0+0 (secure 0 0 0) 0x7fe26408b6a0 con 0x7fe26c0d3320 2024-04-26T21:20:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.055+0000 7fe26a7fc640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 0x7fe25806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.056+0000 7fe26a7fc640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 0x7fe25806edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0cf400 tx=0x7fe260009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.060+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe264059eb0 con 0x7fe26c0d3320 2024-04-26T21:20:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.314+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe26c0693d0 con 0x7fe26c0d3320 2024-04-26T21:20:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.314+0000 7fe253fff640 1 -- 172.21.15.40:0/4246444842 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fe2640597b0 con 0x7fe26c0d3320 2024-04-26T21:20:42.316 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:42.316 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:20:42.316 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:20:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:20:42.318 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:20:42.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:20:42.323 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.325 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:20:42.326 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:20:42.329 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.331 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:20:42.331 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:20:42.331 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.332 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.334 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.335 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.337 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:20:42.337 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:20:42.337 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.338 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.338 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:20:42.338 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:20:42.338 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:20:42.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:20:42.340 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.341 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:42.343 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:20:42.344 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 msgr2=0x7fe25806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 0x7fe25806edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0cf400 tx=0x7fe260009040 comp rx=0 tx=0).stop 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 msgr2=0x7fe26c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fe26400e960 tx=0x7fe26400ee20 comp rx=0 tx=0).stop 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 shutdown_connections 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe25806c8f0 0x7fe25806edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe26c15f750 0x7fe26c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe26c0d51a0 0x7fe26c0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.316+0000 7fe2719af640 1 --2- 172.21.15.40:0/4246444842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe26c0d3320 0x7fe26c1501b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.317+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 >> 172.21.15.40:0/4246444842 conn(0x7fe26c08ec20 msgr2=0x7fe26c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.317+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 shutdown_connections 2024-04-26T21:20:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.317+0000 7fe2719af640 1 -- 172.21.15.40:0/4246444842 wait complete. 2024-04-26T21:20:42.348 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:42.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 Processor -- start 2024-04-26T21:20:42.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 -- start start 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a00d1740 con 0x7fb9a006e2c0 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a006e750 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.449+0000 7fb9a55da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00d1a20 con 0x7fb9a00d11b0 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb9a006e2c0 0x7fb9a015ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57692/0 (socket says 172.21.15.40:57692) 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34966/0 (socket says 172.21.15.40:34966) 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99f7fe640 1 -- 172.21.15.40:0/924089983 learned_addr learned my addr 172.21.15.40:0/924089983 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb997fff640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 -- 172.21.15.40:0/924089983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 msgr2=0x7fb9a006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 -- 172.21.15.40:0/924089983 >> v1:172.21.15.40:6789/0 conn(0x7fb9a006e2c0 legacy=0x7fb9a015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99effd640 1 -- 172.21.15.40:0/924089983 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00d1d00 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb99e7fc640 1 -- 172.21.15.40:0/924089983 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512882871 0 0) 0x7fb9a00d1740 con 0x7fb9a006e2c0 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.450+0000 7fb997fff640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb99effd640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097a0 tx=0x7fb99002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c179bf893a493fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb99e7fc640 1 -- 172.21.15.40:0/924089983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990037070 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb99e7fc640 1 -- 172.21.15.40:0/924089983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990032440 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb99e7fc640 1 -- 172.21.15.40:0/924089983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990032810 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 -- 172.21.15.40:0/924089983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a00cf2e0 msgr2=0x7fb9a006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097a0 tx=0x7fb99002a180 comp rx=0 tx=0).stop 2024-04-26T21:20:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 -- 172.21.15.40:0/924089983 shutdown_connections 2024-04-26T21:20:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 --2- 172.21.15.40:0/924089983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a006d4d0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 -- 172.21.15.40:0/924089983 >> 172.21.15.40:0/924089983 conn(0x7fb9a00b4a60 msgr2=0x7fb9a00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.451+0000 7fb9a55da640 1 -- 172.21.15.40:0/924089983 shutdown_connections 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.452+0000 7fb9a55da640 1 -- 172.21.15.40:0/924089983 wait complete. 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 Processor -- start 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 -- start start 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a00b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006da10 con 0x7fb9a006e2c0 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00d22c0 con 0x7fb9a00cf2e0 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb9a55da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00cf6c0 con 0x7fb9a00d11b0 2024-04-26T21:20:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb99effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb99effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34976/0 (socket says 172.21.15.40:34976) 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.453+0000 7fb99effd640 1 -- 172.21.15.40:0/1687631284 learned_addr learned my addr 172.21.15.40:0/1687631284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99f7fe640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a00b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb997fff640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 -- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 msgr2=0x7fb9a00b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a00b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 -- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9a00cf2e0 msgr2=0x7fb9a00b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a00b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 -- 172.21.15.40:0/1687631284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00b08b0 con 0x7fb9a006e2c0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb99effd640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097a0 tx=0x7fb9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990037050 con 0x7fb9a006e2c0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990038040 con 0x7fb9a006e2c0 2024-04-26T21:20:42.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990004490 con 0x7fb9a006e2c0 2024-04-26T21:20:42.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a00b0b90 con 0x7fb9a006e2c0 2024-04-26T21:20:42.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.454+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a00b1150 con 0x7fb9a006e2c0 2024-04-26T21:20:42.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.455+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a006a290 con 0x7fb9a006e2c0 2024-04-26T21:20:42.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.456+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb990030980 con 0x7fb9a006e2c0 2024-04-26T21:20:42.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.456+0000 7fb9977fe640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 0x7fb98006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.456+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4857+0+0 (secure 0 0 0) 0x7fb9900ac340 con 0x7fb9a006e2c0 2024-04-26T21:20:42.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.456+0000 7fb997fff640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 0x7fb98006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.457+0000 7fb997fff640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 0x7fb98006edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b3d30 tx=0x7fb988008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.460+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb99007ab50 con 0x7fb9a006e2c0 2024-04-26T21:20:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.645+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9a0076090 con 0x7fb9a006e2c0 2024-04-26T21:20:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.646+0000 7fb9977fe640 1 -- 172.21.15.40:0/1687631284 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+12901 (secure 0 0 0) 0x7fb99007a450 con 0x7fb9a006e2c0 2024-04-26T21:20:42.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:42.647 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":130,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:41.920947+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 msgr2=0x7fb98006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 0x7fb98006edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b3d30 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 msgr2=0x7fb9a0160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097a0 tx=0x7fb9900046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 shutdown_connections 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb98006c8f0 0x7fb98006edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9a00d11b0 0x7fb9a00b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9a00cf2e0 0x7fb9a00b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 --2- 172.21.15.40:0/1687631284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9a006e2c0 0x7fb9a0160090 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 >> 172.21.15.40:0/1687631284 conn(0x7fb9a00b4a60 msgr2=0x7fb9a0070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 shutdown_connections 2024-04-26T21:20:42.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.648+0000 7fb9a55da640 1 -- 172.21.15.40:0/1687631284 wait complete. 2024-04-26T21:20:42.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:20:42.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:20:42.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.764+0000 7f969f1a4640 1 Processor -- start 2024-04-26T21:20:42.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.764+0000 7f969f1a4640 1 -- start start 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969f1a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96981631f0 0x7f96980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969f1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f96980b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969f1a4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969809c790 con 0x7f96980d1170 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969f1a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f969806eec0 con 0x7f96981631f0 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969f1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96980d15f0 con 0x7f969806ea30 2024-04-26T21:20:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969cf40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96981631f0 0x7f96980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969cf40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96981631f0 0x7f96980b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34978/0 (socket says 172.21.15.40:34978) 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969cf40640 1 -- 172.21.15.40:0/776794759 learned_addr learned my addr 172.21.15.40:0/776794759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.765+0000 7f969d741640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f96980b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f969cf40640 1 -- 172.21.15.40:0/776794759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 msgr2=0x7f96980b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f969cf40640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f96980b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f969cf40640 1 -- 172.21.15.40:0/776794759 >> v1:172.21.15.184:6789/0 conn(0x7f96980d1170 legacy=0x7f96981635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f969cf40640 1 -- 172.21.15.40:0/776794759 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9698163ec0 con 0x7f96981631f0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f969cf40640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96981631f0 0x7f96980b2000 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9690009e80 tx=0x7f969002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0988eca9d2da85e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.766+0000 7f96977fe640 1 -- 172.21.15.40:0/776794759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690036070 con 0x7f96981631f0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f96977fe640 1 -- 172.21.15.40:0/776794759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9690031600 con 0x7f96981631f0 2024-04-26T21:20:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f96977fe640 1 -- 172.21.15.40:0/776794759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96900319d0 con 0x7f96981631f0 2024-04-26T21:20:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 -- 172.21.15.40:0/776794759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96981631f0 msgr2=0x7f96980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96981631f0 0x7f96980b2000 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9690009e80 tx=0x7f969002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 -- 172.21.15.40:0/776794759 shutdown_connections 2024-04-26T21:20:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f96980b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 --2- 172.21.15.40:0/776794759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96981631f0 0x7f96980b2000 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 -- 172.21.15.40:0/776794759 >> 172.21.15.40:0/776794759 conn(0x7f969809ba40 msgr2=0x7f969809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 -- 172.21.15.40:0/776794759 shutdown_connections 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.767+0000 7f969f1a4640 1 -- 172.21.15.40:0/776794759 wait complete. 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.768+0000 7f969f1a4640 1 Processor -- start 2024-04-26T21:20:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.768+0000 7f969f1a4640 1 -- start start 2024-04-26T21:20:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f9698151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96981631f0 0x7f9698162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96980d2000 con 0x7f96980d1170 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96980d1d20 con 0x7f96981631f0 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969f1a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96980d0230 con 0x7f969806ea30 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f9697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f9697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34980/0 (socket says 172.21.15.40:34980) 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f9697fff640 1 -- 172.21.15.40:0/2905889591 learned_addr learned my addr 172.21.15.40:0/2905889591 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.769+0000 7f969cf40640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f9698151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 -- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 msgr2=0x7f9698151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f9698151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f969d741640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96981631f0 0x7f9698162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 -- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96981631f0 msgr2=0x7f9698162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96981631f0 0x7f9698162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 -- 172.21.15.40:0/2905889591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96980ccbe0 con 0x7f96980d1170 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f9697fff640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c400 tx=0x7f968c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0052d0 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c005490 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981f93f0 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c007260 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.770+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981f99b0 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.771+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96981f9bb0 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.771+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f968c007400 con 0x7f96980d1170 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.772+0000 7f96957fa640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 0x7f968406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.772+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4857+0+0 (secure 0 0 0) 0x7f968c08ad00 con 0x7f96980d1170 2024-04-26T21:20:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.772+0000 7f969cf40640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 0x7f968406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.772+0000 7f969cf40640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 0x7f968406edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9690033000 tx=0x7f9690009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.776+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96981f9bb0 con 0x7f96980d1170 2024-04-26T21:20:42.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.963+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f969806a290 con 0x7f96980d1170 2024-04-26T21:20:42.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.963+0000 7f96957fa640 1 -- 172.21.15.40:0/2905889591 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7f968c059510 con 0x7f96980d1170 2024-04-26T21:20:42.966 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 msgr2=0x7f968406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 0x7f968406edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9690033000 tx=0x7f9690009000 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 msgr2=0x7f9698161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c400 tx=0x7f968c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 shutdown_connections 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968406c8f0 0x7f968406edb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96981631f0 0x7f9698162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96980d1170 0x7f9698161de0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 --2- 172.21.15.40:0/2905889591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f969806ea30 0x7f9698151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 >> 172.21.15.40:0/2905889591 conn(0x7f969809ba40 msgr2=0x7f9698070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 shutdown_connections 2024-04-26T21:20:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:42.965+0000 7f969f1a4640 1 -- 172.21.15.40:0/2905889591 wait complete. 2024-04-26T21:20:42.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:20:43.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 Processor -- start 2024-04-26T21:20:43.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 -- start start 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a680cf2e0 0x7f3a6806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a6806e2c0 0x7f3a6815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6806e750 con 0x7f3a680cf2e0 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3a680d1740 con 0x7f3a6806e2c0 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.077+0000 7f3a6f1f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680d1a20 con 0x7f3a680d11b0 2024-04-26T21:20:43.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a680cf2e0 0x7f3a6806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a680cf2e0 0x7f3a6806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34986/0 (socket says 172.21.15.40:34986) 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 -- 172.21.15.40:0/3943927528 learned_addr learned my addr 172.21.15.40:0/3943927528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6d794640 1 --2- 172.21.15.40:0/3943927528 >> v2:172.21.15.184:3300/0 conn(0x7f3a6806e2c0 0x7f3a6815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a5ffff640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 -- 172.21.15.40:0/3943927528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 msgr2=0x7f3a6806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 -- 172.21.15.40:0/3943927528 >> v2:172.21.15.184:3300/0 conn(0x7f3a6806e2c0 msgr2=0x7f3a6815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/3943927528 >> v2:172.21.15.184:3300/0 conn(0x7f3a6806e2c0 0x7f3a6815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a6cf93640 1 -- 172.21.15.40:0/3943927528 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a680d1d00 con 0x7f3a680cf2e0 2024-04-26T21:20:43.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.078+0000 7f3a5ffff640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:43.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a680cf2e0 0x7f3a6806d4d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3a68163ff0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eb8fbf56a952a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a5f7fe640 1 -- 172.21.15.40:0/3943927528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64037070 con 0x7f3a680cf2e0 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a5f7fe640 1 -- 172.21.15.40:0/3943927528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64032440 con 0x7f3a680cf2e0 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a5f7fe640 1 -- 172.21.15.40:0/3943927528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64032810 con 0x7f3a680cf2e0 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/3943927528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a680cf2e0 msgr2=0x7f3a6806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a680cf2e0 0x7f3a6806d4d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3a68163ff0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/3943927528 shutdown_connections 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/3943927528 >> v2:172.21.15.184:3300/0 conn(0x7f3a6806e2c0 0x7f3a6815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/3943927528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a680cf2e0 0x7f3a6806d4d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/3943927528 >> 172.21.15.40:0/3943927528 conn(0x7f3a680b4a60 msgr2=0x7f3a680b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.079+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/3943927528 shutdown_connections 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.080+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/3943927528 wait complete. 2024-04-26T21:20:43.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 Processor -- start 2024-04-26T21:20:43.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 -- start start 2024-04-26T21:20:43.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680cf2e0 0x7f3a6815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6806da10 con 0x7f3a6806e2c0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a680d22c0 con 0x7f3a680cf2e0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.081+0000 7f3a6f1f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680cf6c0 con 0x7f3a680d11b0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35002/0 (socket says 172.21.15.40:35002) 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 -- 172.21.15.40:0/2666068658 learned_addr learned my addr 172.21.15.40:0/2666068658 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a5ffff640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680cf2e0 0x7f3a6815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6d794640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 -- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 msgr2=0x7f3a6815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 -- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680cf2e0 msgr2=0x7f3a6815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680cf2e0 0x7f3a6815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 -- 172.21.15.40:0/2666068658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a680cc8b0 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6cf93640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64041070 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6403c870 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a680ccb90 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64037040 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.082+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a680cd150 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.083+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680cd390 con 0x7f3a6806e2c0 2024-04-26T21:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.084+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a64044070 con 0x7f3a6806e2c0 2024-04-26T21:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.084+0000 7f3a5d7fa640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 0x7f3a5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.084+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4857+0+0 (secure 0 0 0) 0x7f3a640ac450 con 0x7f3a6806e2c0 2024-04-26T21:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.085+0000 7f3a5ffff640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 0x7f3a5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.085+0000 7f3a5ffff640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 0x7f3a5406edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3a68070e80 tx=0x7f3a58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.089+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a680cd390 con 0x7f3a6806e2c0 2024-04-26T21:20:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.274+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3a6806a290 con 0x7f3a6806e2c0 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.275+0000 7f3a5d7fa640 1 -- 172.21.15.40:0/2666068658 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v130) ==== 84+0+65 (secure 0 0 0) 0x7f3a6407ad80 con 0x7f3a6806e2c0 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:20:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:20:43.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 msgr2=0x7f3a5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 0x7f3a5406edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3a68070e80 tx=0x7f3a58008040 comp rx=0 tx=0).stop 2024-04-26T21:20:43.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 msgr2=0x7f3a680ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a640046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 shutdown_connections 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3a5406c8f0 0x7f3a5406edb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a680d11b0 0x7f3a6815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680cf2e0 0x7f3a6815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 --2- 172.21.15.40:0/2666068658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a6806e2c0 0x7f3a680ca830 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 >> 172.21.15.40:0/2666068658 conn(0x7f3a680b4a60 msgr2=0x7f3a68070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 shutdown_connections 2024-04-26T21:20:43.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.277+0000 7f3a6f1f7640 1 -- 172.21.15.40:0/2666068658 wait complete. 2024-04-26T21:20:43.289 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:20:43.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:20:43.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.393+0000 7fea1a95f640 1 Processor -- start 2024-04-26T21:20:43.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.393+0000 7fea1a95f640 1 -- start start 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea1a95f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea1415d2b0 0x7fea140d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea1a95f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 0x7fea1409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea1a95f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea140d12c0 con 0x7fea140d0ae0 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea1a95f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fea140cd9b0 con 0x7fea1415d2b0 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea1a95f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fea140cedd0 con 0x7fea1409ba40 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea1415d2b0 0x7fea140d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea1415d2b0 0x7fea140d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35016/0 (socket says 172.21.15.40:35016) 2024-04-26T21:20:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 -- 172.21.15.40:0/4260903759 learned_addr learned my addr 172.21.15.40:0/4260903759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea18efc640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 0x7fea1409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 -- 172.21.15.40:0/4260903759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 msgr2=0x7fea1409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 0x7fea1409bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 -- 172.21.15.40:0/4260903759 >> v1:172.21.15.184:6789/0 conn(0x7fea140d0ae0 legacy=0x7fea140ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea13fff640 1 -- 172.21.15.40:0/4260903759 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140d0fe0 con 0x7fea1415d2b0 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.394+0000 7fea18efc640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 0x7fea1409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.395+0000 7fea13fff640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1415d2b0 0x7fea140d3a30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fea04009870 tx=0x7fea0402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783482919dfff61a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.395+0000 7fea12ffd640 1 -- 172.21.15.40:0/4260903759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04037070 con 0x7fea1415d2b0 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.395+0000 7fea12ffd640 1 -- 172.21.15.40:0/4260903759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea04032440 con 0x7fea1415d2b0 2024-04-26T21:20:43.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.395+0000 7fea12ffd640 1 -- 172.21.15.40:0/4260903759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04032810 con 0x7fea1415d2b0 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 -- 172.21.15.40:0/4260903759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1415d2b0 msgr2=0x7fea140d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1415d2b0 0x7fea140d3a30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fea04009870 tx=0x7fea0402a210 comp rx=0 tx=0).stop 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 -- 172.21.15.40:0/4260903759 shutdown_connections 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1409ba40 0x7fea1409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 --2- 172.21.15.40:0/4260903759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1415d2b0 0x7fea140d3a30 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 -- 172.21.15.40:0/4260903759 >> 172.21.15.40:0/4260903759 conn(0x7fea140d2f10 msgr2=0x7fea140d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 -- 172.21.15.40:0/4260903759 shutdown_connections 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.396+0000 7fea1a95f640 1 -- 172.21.15.40:0/4260903759 wait complete. 2024-04-26T21:20:43.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.397+0000 7fea1a95f640 1 Processor -- start 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.397+0000 7fea1a95f640 1 -- start start 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea140d0ae0 0x7fea141596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 0x7fea14159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea140ce1a0 con 0x7fea1409ba40 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea140cdec0 con 0x7fea140d0ae0 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea1a95f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fea1409c810 con 0x7fea1415d2b0 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35020/0 (socket says 172.21.15.40:35020) 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 -- 172.21.15.40:0/3519300328 learned_addr learned my addr 172.21.15.40:0/3519300328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea18efc640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 0x7fea14159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea137fe640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea140d0ae0 0x7fea141596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 -- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 msgr2=0x7fea14159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 0x7fea14159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 -- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea140d0ae0 msgr2=0x7fea141596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea140d0ae0 0x7fea141596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 -- 172.21.15.40:0/3519300328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1415a280 con 0x7fea1409ba40 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea18efc640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 0x7fea14159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.398+0000 7fea13fff640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fea04009870 tx=0x7fea040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.399+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04041020 con 0x7fea1409ba40 2024-04-26T21:20:43.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.399+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea04030970 con 0x7fea1409ba40 2024-04-26T21:20:43.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.399+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1415a560 con 0x7fea1409ba40 2024-04-26T21:20:43.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.399+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04037040 con 0x7fea1409ba40 2024-04-26T21:20:43.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.399+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1415ab20 con 0x7fea1409ba40 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.400+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1406a290 con 0x7fea1409ba40 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.400+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea04044020 con 0x7fea1409ba40 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.401+0000 7fea10ff9640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 0x7fea0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.401+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4857+0+0 (secure 0 0 0) 0x7fea040ac160 con 0x7fea1409ba40 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.401+0000 7fea137fe640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 0x7fea0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.402+0000 7fea137fe640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 0x7fea0006edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fea08006fd0 tx=0x7fea08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:43.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.405+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea0404b030 con 0x7fea1409ba40 2024-04-26T21:20:43.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.593+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fea140b7cf0 con 0x7fea1409ba40 2024-04-26T21:20:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.929+0000 7fea10ff9640 1 -- 172.21.15.40:0/3519300328 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v131) ==== 138+0+0 (secure 0 0 0) 0x7fea0407a970 con 0x7fea1409ba40 2024-04-26T21:20:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 msgr2=0x7fea0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 0x7fea0006edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fea08006fd0 tx=0x7fea08008040 comp rx=0 tx=0).stop 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 msgr2=0x7fea140cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fea04009870 tx=0x7fea040046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 shutdown_connections 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fea0006c8f0 0x7fea0006edb0 secure :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fea08006fd0 tx=0x7fea08008040 comp rx=0 tx=0).stop 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea1415d2b0 0x7fea14159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea140d0ae0 0x7fea141596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 --2- 172.21.15.40:0/3519300328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea1409ba40 0x7fea140cb580 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:43.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.932+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 >> 172.21.15.40:0/3519300328 conn(0x7fea140d2f10 msgr2=0x7fea140c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:43.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.933+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 shutdown_connections 2024-04-26T21:20:43.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:43.933+0000 7fea1a95f640 1 -- 172.21.15.40:0/3519300328 wait complete. 2024-04-26T21:20:43.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:20:44.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.056+0000 7f01d53f7640 1 Processor -- start 2024-04-26T21:20:44.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.056+0000 7f01d53f7640 1 -- start start 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.057+0000 7f01d53f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.057+0000 7f01d53f7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01d00d1b80 0x7f01d00d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.057+0000 7f01d53f7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d00b6ab0 con 0x7f01d0163020 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.057+0000 7f01d53f7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00d1260 con 0x7f01d00d1b80 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.057+0000 7f01d53f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00d2520 con 0x7f01d015f3c0 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:39832/0 (socket says 172.21.15.40:39832) 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ce7fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01d00d1b80 0x7f01d00d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 -- 172.21.15.40:0/474411618 learned_addr learned my addr 172.21.15.40:0/474411618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 -- 172.21.15.40:0/474411618 >> v2:172.21.15.184:3300/0 conn(0x7f01d00d1b80 msgr2=0x7f01d00d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:44.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 --2- 172.21.15.40:0/474411618 >> v2:172.21.15.184:3300/0 conn(0x7f01d00d1b80 0x7f01d00d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 -- 172.21.15.40:0/474411618 >> v1:172.21.15.184:6789/0 conn(0x7f01d0163020 legacy=0x7f01d01634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.058+0000 7f01ceffd640 1 -- 172.21.15.40:0/474411618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00d4200 con 0x7f01d015f3c0 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01ceffd640 1 --2- 172.21.15.40:0/474411618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009870 tx=0x7f01c802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59a8d82151a3ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01cdffb640 1 -- 172.21.15.40:0/474411618 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8037070 con 0x7f01d015f3c0 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01cdffb640 1 -- 172.21.15.40:0/474411618 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8032440 con 0x7f01d015f3c0 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01cdffb640 1 -- 172.21.15.40:0/474411618 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8032810 con 0x7f01d015f3c0 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01d53f7640 1 -- 172.21.15.40:0/474411618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 msgr2=0x7f01d00d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.059+0000 7f01d53f7640 1 --2- 172.21.15.40:0/474411618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009870 tx=0x7f01c802a270 comp rx=0 tx=0).stop 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 -- 172.21.15.40:0/474411618 shutdown_connections 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 --2- 172.21.15.40:0/474411618 >> v2:172.21.15.184:3300/0 conn(0x7f01d00d1b80 0x7f01d00d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 --2- 172.21.15.40:0/474411618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d015f3c0 0x7f01d00d0d20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 -- 172.21.15.40:0/474411618 >> 172.21.15.40:0/474411618 conn(0x7f01d009e4f0 msgr2=0x7f01d00d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 -- 172.21.15.40:0/474411618 shutdown_connections 2024-04-26T21:20:44.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.060+0000 7f01d53f7640 1 -- 172.21.15.40:0/474411618 wait complete. 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.061+0000 7f01d53f7640 1 Processor -- start 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.061+0000 7f01d53f7640 1 -- start start 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d00d1b80 0x7f01d00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01d015f3c0 0x7f01d015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00d3720 con 0x7f01d0163020 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00d3440 con 0x7f01d015f3c0 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01d53f7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00d3a00 con 0x7f01d00d1b80 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35028/0 (socket says 172.21.15.40:35028) 2024-04-26T21:20:44.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 -- 172.21.15.40:0/3589538523 learned_addr learned my addr 172.21.15.40:0/3589538523 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01ceffd640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d00d1b80 0x7f01d00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01ce7fc640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01d015f3c0 0x7f01d015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 -- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d00d1b80 msgr2=0x7f01d00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d00d1b80 0x7f01d00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 -- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01d015f3c0 msgr2=0x7f01d015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01d015f3c0 0x7f01d015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 -- 172.21.15.40:0/3589538523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d015dfa0 con 0x7f01d0163020 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01cf7fe640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c6e0 tx=0x7f01c000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.062+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0003b70 con 0x7f01d0163020 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.063+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c0003d30 con 0x7f01d0163020 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.063+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d00b7ee0 con 0x7f01d0163020 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.063+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0005310 con 0x7f01d0163020 2024-04-26T21:20:44.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.063+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00b84a0 con 0x7f01d0163020 2024-04-26T21:20:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.064+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d00693d0 con 0x7f01d0163020 2024-04-26T21:20:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.064+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01c00054b0 con 0x7f01d0163020 2024-04-26T21:20:44.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.065+0000 7f01c77fe640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 0x7f01b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.065+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5227+0+0 (secure 0 0 0) 0x7f01c008b6f0 con 0x7f01d0163020 2024-04-26T21:20:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.065+0000 7f01ceffd640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 0x7f01b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:44.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.066+0000 7f01ceffd640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 0x7f01b006ede0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f01c8034000 tx=0x7f01c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.069+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01c0059620 con 0x7f01d0163020 2024-04-26T21:20:44.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.255+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f01d00d58c0 con 0x7f01d0163020 2024-04-26T21:20:44.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.932+0000 7f01c77fe640 1 -- 172.21.15.40:0/3589538523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v132) ==== 170+0+0 (secure 0 0 0) 0x7f01c005d920 con 0x7f01d0163020 2024-04-26T21:20:44.933 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 msgr2=0x7f01b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 0x7f01b006ede0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f01c8034000 tx=0x7f01c802e000 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 msgr2=0x7f01d00cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c6e0 tx=0x7f01c000cba0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 shutdown_connections 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b006c920 0x7f01b006ede0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d0163020 0x7f01d00cf000 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01d015f3c0 0x7f01d015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 --2- 172.21.15.40:0/3589538523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01d00d1b80 0x7f01d00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 >> 172.21.15.40:0/3589538523 conn(0x7f01d009e4f0 msgr2=0x7f01d00d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 shutdown_connections 2024-04-26T21:20:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:44.935+0000 7f01d53f7640 1 -- 172.21.15.40:0/3589538523 wait complete. 2024-04-26T21:20:44.965 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:20:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.074+0000 7fb9e8a2a640 1 Processor -- start 2024-04-26T21:20:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.074+0000 7fb9e8a2a640 1 -- start start 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e8a2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9e415f990 0x7fb9e40d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e8a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e40d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e8a2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e40b6aa0 con 0x7fb9e415f5b0 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e8a2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e40d3eb0 con 0x7fb9e415f990 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e8a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e40d4190 con 0x7fb9e40d0f40 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9e415f990 0x7fb9e40d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9e415f990 0x7fb9e40d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35036/0 (socket says 172.21.15.40:35036) 2024-04-26T21:20:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 -- 172.21.15.40:0/2629846654 learned_addr learned my addr 172.21.15.40:0/2629846654 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9e2d9d640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e40d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 -- 172.21.15.40:0/2629846654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 msgr2=0x7fb9e40d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e40d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 -- 172.21.15.40:0/2629846654 >> v1:172.21.15.184:6789/0 conn(0x7fb9e415f5b0 legacy=0x7fb9e40b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.075+0000 7fb9dbfff640 1 -- 172.21.15.40:0/2629846654 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e40d0200 con 0x7fb9e415f990 2024-04-26T21:20:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9dbfff640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f990 0x7fb9e40d0a00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce32c29f144702f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9e1d9b640 1 -- 172.21.15.40:0/2629846654 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037070 con 0x7fb9e415f990 2024-04-26T21:20:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9e1d9b640 1 -- 172.21.15.40:0/2629846654 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc032440 con 0x7fb9e415f990 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9e1d9b640 1 -- 172.21.15.40:0/2629846654 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc032810 con 0x7fb9e415f990 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/2629846654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f990 msgr2=0x7fb9e40d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.076+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f990 0x7fb9e40d0a00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/2629846654 shutdown_connections 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e40d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/2629846654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f990 0x7fb9e40d0a00 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/2629846654 >> 172.21.15.40:0/2629846654 conn(0x7fb9e408ec40 msgr2=0x7fb9e409e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/2629846654 shutdown_connections 2024-04-26T21:20:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.077+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/2629846654 wait complete. 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.078+0000 7fb9e8a2a640 1 Processor -- start 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.078+0000 7fb9e8a2a640 1 -- start start 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e41ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 0x7fb9e40ba720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e40b5900 con 0x7fb9e415f5b0 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e40b5780 con 0x7fb9e415f990 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e40d3880 con 0x7fb9e40d0f40 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35052/0 (socket says 172.21.15.40:35052) 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 -- 172.21.15.40:0/1735144080 learned_addr learned my addr 172.21.15.40:0/1735144080 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e259c640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e41ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e2d9d640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 0x7fb9e40ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 -- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 msgr2=0x7fb9e41ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e41ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 -- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 msgr2=0x7fb9e40ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 0x7fb9e40ba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 -- 172.21.15.40:0/1735144080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e41f6370 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e2d9d640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 0x7fb9e40ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e259c640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e41ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9dbfff640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037040 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc042070 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e40bad30 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc030880 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.079+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e41f5890 con 0x7fb9e415f5b0 2024-04-26T21:20:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.081+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e406a290 con 0x7fb9e415f5b0 2024-04-26T21:20:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.081+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9cc02ad10 con 0x7fb9e415f5b0 2024-04-26T21:20:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.081+0000 7fb9daffd640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.081+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5609+0+0 (secure 0 0 0) 0x7fb9cc0ac0c0 con 0x7fb9e415f5b0 2024-04-26T21:20:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.082+0000 7fb9e259c640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.082+0000 7fb9e259c640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9e406e6c0 tx=0x7fb9d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:45.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.086+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9cc07a550 con 0x7fb9e415f5b0 2024-04-26T21:20:45.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.314+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb9e40d1e40 con 0x7fb9e415f5b0 2024-04-26T21:20:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.959+0000 7fb9daffd640 1 -- 172.21.15.40:0/1735144080 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v85) ==== 174+0+0 (secure 0 0 0) 0x7fb9cc079e50 con 0x7fb9e415f5b0 2024-04-26T21:20:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 20 and data pool 21 2024-04-26T21:20:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.968+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 msgr2=0x7fb9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.968+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9e406e6c0 tx=0x7fb9d4009040 comp rx=0 tx=0).stop 2024-04-26T21:20:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.968+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 msgr2=0x7fb9e40ba1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.968+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 shutdown_connections 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb9c406c8f0 0x7fb9c406edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e415f990 0x7fb9e40ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e415f5b0 0x7fb9e40ba1e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 --2- 172.21.15.40:0/1735144080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9e40d0f40 0x7fb9e41ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.969+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 >> 172.21.15.40:0/1735144080 conn(0x7fb9e408ec40 msgr2=0x7fb9e406d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.970+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 shutdown_connections 2024-04-26T21:20:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:45.970+0000 7fb9e8a2a640 1 -- 172.21.15.40:0/1735144080 wait complete. 2024-04-26T21:20:45.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:46.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.097+0000 7f1d9e112640 1 Processor -- start 2024-04-26T21:20:46.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.097+0000 7f1d9e112640 1 -- start start 2024-04-26T21:20:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d9e112640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d9e112640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 0x7f1d980d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d9e112640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d980d0be0 con 0x7f1d98160d50 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d9e112640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980d0ec0 con 0x7f1d980b56d0 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d9e112640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980d11a0 con 0x7f1d9815d380 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d96ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 0x7f1d980d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d97fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f1d98160d50 0x7f1d980b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:37400/0 (socket says 172.21.15.40:37400) 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d97fff640 1 -- 172.21.15.40:0/1958195307 learned_addr learned my addr 172.21.15.40:0/1958195307 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:39870/0 (socket says 172.21.15.40:39870) 2024-04-26T21:20:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d967fc640 1 -- 172.21.15.40:0/1958195307 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592944148 0 0) 0x7f1d980d0be0 con 0x7f1d98160d50 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d967fc640 1 -- 172.21.15.40:0/1958195307 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d80003520 con 0x7f1d98160d50 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 -- 172.21.15.40:0/1958195307 >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 msgr2=0x7f1d980d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 --2- 172.21.15.40:0/1958195307 >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 0x7f1d980d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 -- 172.21.15.40:0/1958195307 >> v1:172.21.15.184:6789/0 conn(0x7f1d98160d50 legacy=0x7f1d980b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d977fe640 1 -- 172.21.15.40:0/1958195307 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980d1480 con 0x7f1d9815d380 2024-04-26T21:20:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.098+0000 7f1d96ffd640 1 --2- 172.21.15.40:0/1958195307 >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 0x7f1d980d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.099+0000 7f1d977fe640 1 --2- 172.21.15.40:0/1958195307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0097a0 tx=0x7f1d8c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ffbeed58a27d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.099+0000 7f1d967fc640 1 -- 172.21.15.40:0/1958195307 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c037070 con 0x7f1d9815d380 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.099+0000 7f1d967fc640 1 -- 172.21.15.40:0/1958195307 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c032440 con 0x7f1d9815d380 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.099+0000 7f1d967fc640 1 -- 172.21.15.40:0/1958195307 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c032810 con 0x7f1d9815d380 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 -- 172.21.15.40:0/1958195307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 msgr2=0x7f1d980ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 --2- 172.21.15.40:0/1958195307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0097a0 tx=0x7f1d8c02a180 comp rx=0 tx=0).stop 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 -- 172.21.15.40:0/1958195307 shutdown_connections 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 --2- 172.21.15.40:0/1958195307 >> v2:172.21.15.184:3300/0 conn(0x7f1d980b56d0 0x7f1d980d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 --2- 172.21.15.40:0/1958195307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d9815d380 0x7f1d980ce590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 -- 172.21.15.40:0/1958195307 >> 172.21.15.40:0/1958195307 conn(0x7f1d980b4a60 msgr2=0x7f1d980b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 -- 172.21.15.40:0/1958195307 shutdown_connections 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.100+0000 7f1d9e112640 1 -- 172.21.15.40:0/1958195307 wait complete. 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.101+0000 7f1d9e112640 1 Processor -- start 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.101+0000 7f1d9e112640 1 -- start start 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d980b56d0 0x7f1d9814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d98160d50 0x7f1d980c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980cedc0 con 0x7f1d9815d380 2024-04-26T21:20:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980d1ee0 con 0x7f1d980b56d0 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d9e112640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d981611e0 con 0x7f1d98160d50 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35056/0 (socket says 172.21.15.40:35056) 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d96ffd640 1 -- 172.21.15.40:0/666758856 learned_addr learned my addr 172.21.15.40:0/666758856 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d977fe640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d980b56d0 0x7f1d9814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.102+0000 7f1d97fff640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d98160d50 0x7f1d980c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 -- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d98160d50 msgr2=0x7f1d980c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d98160d50 0x7f1d980c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 -- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d980b56d0 msgr2=0x7f1d9814c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d980b56d0 0x7f1d9814c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 -- 172.21.15.40:0/666758856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9815db40 con 0x7f1d9815d380 2024-04-26T21:20:46.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d96ffd640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800ebe0 tx=0x7f1d8800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88017070 con 0x7f1d9815d380 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8800cc30 con 0x7f1d9815d380 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9815de20 con 0x7f1d9815d380 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88013470 con 0x7f1d9815d380 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.103+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9815e3e0 con 0x7f1d9815d380 2024-04-26T21:20:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.104+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9806a290 con 0x7f1d9815d380 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.105+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8801a070 con 0x7f1d9815d380 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.105+0000 7f1d7ffff640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 0x7f1d8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.105+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5709+0+0 (secure 0 0 0) 0x7f1d8808b970 con 0x7f1d9815d380 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.105+0000 7f1d977fe640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 0x7f1d8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.106+0000 7f1d977fe640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 0x7f1d8406ede0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c034000 tx=0x7f1d8c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.110+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d880597e0 con 0x7f1d9815d380 2024-04-26T21:20:46.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.298+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d9815e7e0 con 0x7f1d9815d380 2024-04-26T21:20:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.299+0000 7f1d7ffff640 1 -- 172.21.15.40:0/666758856 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16692 (secure 0 0 0) 0x7f1d8805dae0 con 0x7f1d9815d380 2024-04-26T21:20:46.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:46.300 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:45.315688+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.301+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 msgr2=0x7f1d8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.301+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 0x7f1d8406ede0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c034000 tx=0x7f1d8c02e000 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 msgr2=0x7f1d980c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800ebe0 tx=0x7f1d8800c520 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 shutdown_connections 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d8406c920 0x7f1d8406ede0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d98160d50 0x7f1d980c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9815d380 0x7f1d980c8f40 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 --2- 172.21.15.40:0/666758856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d980b56d0 0x7f1d9814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 >> 172.21.15.40:0/666758856 conn(0x7f1d980b4a60 msgr2=0x7f1d9815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 shutdown_connections 2024-04-26T21:20:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.302+0000 7f1d9e112640 1 -- 172.21.15.40:0/666758856 wait complete. 2024-04-26T21:20:46.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:46.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.414+0000 7f61f1823640 1 Processor -- start 2024-04-26T21:20:46.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.414+0000 7f61f1823640 1 -- start start 2024-04-26T21:20:46.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61f1823640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61ec0d5610 0x7f61ec0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61f1823640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0b2510 con 0x7f61ec15f720 2024-04-26T21:20:46.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61f1823640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0b27f0 con 0x7f61ec15f340 2024-04-26T21:20:46.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61f1823640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f61ec0b2ad0 con 0x7f61ec0d5610 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61eb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61ec0d5610 0x7f61ec0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61ea7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61ec15f720 0x7f61ec0b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57700/0 (socket says 172.21.15.40:57700) 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61ea7fc640 1 -- 172.21.15.40:0/1217695455 learned_addr learned my addr 172.21.15.40:0/1217695455 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61eb7fe640 1 -- 172.21.15.40:0/1217695455 >> v1:172.21.15.184:6789/0 conn(0x7f61ec15f340 legacy=0x7f61ec0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61e9ffb640 1 -- 172.21.15.40:0/1217695455 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036268664 0 0) 0x7f61ec0b2510 con 0x7f61ec15f720 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61eb7fe640 1 -- 172.21.15.40:0/1217695455 >> v1:172.21.15.40:6789/0 conn(0x7f61ec15f720 legacy=0x7f61ec0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.415+0000 7f61eb7fe640 1 -- 172.21.15.40:0/1217695455 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec0b2db0 con 0x7f61ec0d5610 2024-04-26T21:20:46.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.416+0000 7f61eb7fe640 1 --2- 172.21.15.40:0/1217695455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec0d5610 0x7f61ec0b1e80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009d90 tx=0x7f61dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef6f4a432950198 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.416+0000 7f61e9ffb640 1 -- 172.21.15.40:0/1217695455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007740 con 0x7f61ec0d5610 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.416+0000 7f61e9ffb640 1 -- 172.21.15.40:0/1217695455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc0078e0 con 0x7f61ec0d5610 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.416+0000 7f61e9ffb640 1 -- 172.21.15.40:0/1217695455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007cb0 con 0x7f61ec0d5610 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 -- 172.21.15.40:0/1217695455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec0d5610 msgr2=0x7f61ec0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 --2- 172.21.15.40:0/1217695455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec0d5610 0x7f61ec0b1e80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009d90 tx=0x7f61dc02a660 comp rx=0 tx=0).stop 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 -- 172.21.15.40:0/1217695455 shutdown_connections 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 --2- 172.21.15.40:0/1217695455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec0d5610 0x7f61ec0b1e80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 -- 172.21.15.40:0/1217695455 >> 172.21.15.40:0/1217695455 conn(0x7f61ec08ec70 msgr2=0x7f61ec09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 -- 172.21.15.40:0/1217695455 shutdown_connections 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.417+0000 7f61f1823640 1 -- 172.21.15.40:0/1217695455 wait complete. 2024-04-26T21:20:46.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.418+0000 7f61f1823640 1 Processor -- start 2024-04-26T21:20:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.418+0000 7f61f1823640 1 -- start start 2024-04-26T21:20:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61ec0d5610 0x7f61ec152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61ec15f340 0x7f61ec1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0d40d0 con 0x7f61ec15f720 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0d3f50 con 0x7f61ec0d5610 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61f1823640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61ec0b3cc0 con 0x7f61ec15f340 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35076/0 (socket says 172.21.15.40:35076) 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 -- 172.21.15.40:0/690341550 learned_addr learned my addr 172.21.15.40:0/690341550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eaffd640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61ec0d5610 0x7f61ec152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61ea7fc640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61ec15f340 0x7f61ec1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 -- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61ec15f340 msgr2=0x7f61ec1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61ec15f340 0x7f61ec1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 -- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61ec0d5610 msgr2=0x7f61ec152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61ec0d5610 0x7f61ec152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.419+0000 7f61eb7fe640 1 -- 172.21.15.40:0/690341550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec1535e0 con 0x7f61ec15f720 2024-04-26T21:20:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61eb7fe640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009eb0 tx=0x7f61dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc040070 con 0x7f61ec15f720 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc031c90 con 0x7f61ec15f720 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec1538c0 con 0x7f61ec15f720 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc03c640 con 0x7f61ec15f720 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.420+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec153e80 con 0x7f61ec15f720 2024-04-26T21:20:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.421+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec06a290 con 0x7f61ec15f720 2024-04-26T21:20:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.421+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61dc039040 con 0x7f61ec15f720 2024-04-26T21:20:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.422+0000 7f61e3fff640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 0x7f61cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.422+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5709+0+0 (secure 0 0 0) 0x7f61dc0acd50 con 0x7f61ec15f720 2024-04-26T21:20:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.422+0000 7f61eaffd640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 0x7f61cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.422+0000 7f61eaffd640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 0x7f61cc06ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001780 tx=0x7f61e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.426+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61dc07aad0 con 0x7f61ec15f720 2024-04-26T21:20:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.613+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61ec0baea0 con 0x7f61ec15f720 2024-04-26T21:20:46.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.615+0000 7f61e3fff640 1 -- 172.21.15.40:0/690341550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16692 (secure 0 0 0) 0x7f61dc07edd0 con 0x7f61ec15f720 2024-04-26T21:20:46.616 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:46.616 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:45.315688+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 msgr2=0x7f61cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 0x7f61cc06ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001780 tx=0x7f61e4009040 comp rx=0 tx=0).stop 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 msgr2=0x7f61ec154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009eb0 tx=0x7f61dc004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 shutdown_connections 2024-04-26T21:20:46.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f61cc06c920 0x7f61cc06ede0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61ec15f720 0x7f61ec154ef0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61ec15f340 0x7f61ec1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 --2- 172.21.15.40:0/690341550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61ec0d5610 0x7f61ec152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 >> 172.21.15.40:0/690341550 conn(0x7f61ec08ec70 msgr2=0x7f61ec06d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 shutdown_connections 2024-04-26T21:20:46.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.617+0000 7f61f1823640 1 -- 172.21.15.40:0/690341550 wait complete. 2024-04-26T21:20:46.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:20:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.732+0000 7f51eabaf640 1 Processor -- start 2024-04-26T21:20:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.732+0000 7f51eabaf640 1 -- start start 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51eabaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51e40d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51eabaf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e40d58f0 con 0x7f51e415f680 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51eabaf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e40d3c90 con 0x7f51e40d3820 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51eabaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51e415fa60 con 0x7f51e40d4f20 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51e894b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51e415f680 0x7f51e4163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57708/0 (socket says 172.21.15.40:57708) 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.733+0000 7f51e894b640 1 -- 172.21.15.40:0/3328123726 learned_addr learned my addr 172.21.15.40:0/3328123726 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e914c640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51e40d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686665288 0 0) 0x7f51e40d58f0 con 0x7f51e415f680 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51cc003520 con 0x7f51e415f680 2024-04-26T21:20:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965208423 0 0) 0x7f51e40d3c90 con 0x7f51e40d3820 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e40d58f0 con 0x7f51e40d3820 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1043324074 0 0) 0x7f51cc003520 con 0x7f51e415f680 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 msgr2=0x7f51e40d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51e40d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 >> v1:172.21.15.184:6789/0 conn(0x7f51e40d3820 legacy=0x7f51e40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e40d3f50 con 0x7f51e415f680 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f51dc0019f0 con 0x7f51e415f680 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 >> v1:172.21.15.40:6789/0 conn(0x7f51e415f680 legacy=0x7f51e4163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51cc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51cc004d40 0x7f51cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51e415fa60 con 0x7f51cc004120 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc00bb00 con 0x7f51cc004d40 2024-04-26T21:20:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc00bd00 con 0x7f51e40d4f20 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e894b640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51cc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e914c640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51cc004d40 0x7f51cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 -- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 msgr2=0x7f51cc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40d4f20 0x7f51cc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 -- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51cc004d40 msgr2=0x7f51cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51cc004d40 0x7f51cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e3fff640 1 -- 172.21.15.40:0/3328123726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc003520 con 0x7f51cc004120 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.734+0000 7f51e914c640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51cc004d40 0x7f51cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.735+0000 7f51e3fff640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51cc004800 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f51d80018d0 tx=0x7f51cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d35ce2a5a165f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.735+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80014f0 con 0x7f51cc004120 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.735+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d80052d0 con 0x7f51cc004120 2024-04-26T21:20:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.735+0000 7f51e37fe640 1 -- 172.21.15.40:0/3328123726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d802abe0 con 0x7f51cc004120 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 -- 172.21.15.40:0/3328123726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 msgr2=0x7f51cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 --2- 172.21.15.40:0/3328123726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51cc004800 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f51d80018d0 tx=0x7f51cc005830 comp rx=0 tx=0).stop 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51e914c640 1 -- 172.21.15.40:0/3328123726 reap_dead start 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 -- 172.21.15.40:0/3328123726 shutdown_connections 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 -- 172.21.15.40:0/3328123726 >> 172.21.15.40:0/3328123726 conn(0x7f51e408eca0 msgr2=0x7f51e409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 -- 172.21.15.40:0/3328123726 shutdown_connections 2024-04-26T21:20:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.736+0000 7f51eabaf640 1 -- 172.21.15.40:0/3328123726 wait complete. 2024-04-26T21:20:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.737+0000 7f51eabaf640 1 Processor -- start 2024-04-26T21:20:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.737+0000 7f51eabaf640 1 -- start start 2024-04-26T21:20:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 0x7f51e41f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 0x7f51e40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40b6280 con 0x7f51cc004120 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40b5370 con 0x7f51e40cb840 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51eabaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51e40d53b0 con 0x7f51e40b31a0 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35100/0 (socket says 172.21.15.40:35100) 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 -- 172.21.15.40:0/1526195099 learned_addr learned my addr 172.21.15.40:0/1526195099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e914c640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 0x7f51e40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e3fff640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 0x7f51e41f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 -- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 msgr2=0x7f51e40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 0x7f51e40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 -- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 msgr2=0x7f51e41f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:46.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 0x7f51e41f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 -- 172.21.15.40:0/1526195099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41f58f0 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e3fff640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 0x7f51e41f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e914c640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 0x7f51e40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.738+0000 7f51e894b640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0079a0 tx=0x7f51dc00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.739+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc017020 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.739+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc007a90 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.739+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e41f5bd0 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.739+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc013610 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.739+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e4160500 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.740+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e406a290 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.740+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51dc01a020 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.741+0000 7f51e17fa640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 0x7f51d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.741+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5709+0+0 (secure 0 0 0) 0x7f51dc08c640 con 0x7f51cc004120 2024-04-26T21:20:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.741+0000 7f51e3fff640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 0x7f51d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:46.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.741+0000 7f51e3fff640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 0x7f51d006ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f51d8032000 tx=0x7f51d8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:46.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.745+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51dc021020 con 0x7f51cc004120 2024-04-26T21:20:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:46.971+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51e40b4cd0 con 0x7f51cc004120 2024-04-26T21:20:47.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-26T21:20:47.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.970+0000 7f51e17fa640 1 -- 172.21.15.40:0/1526195099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) ==== 118+0+0 (secure 0 0 0) 0x7f51dc05a9d0 con 0x7f51cc004120 2024-04-26T21:20:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 msgr2=0x7f51d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 0x7f51d006ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f51d8032000 tx=0x7f51d8009030 comp rx=0 tx=0).stop 2024-04-26T21:20:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 msgr2=0x7f51e41f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0079a0 tx=0x7f51dc00a600 comp rx=0 tx=0).stop 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 shutdown_connections 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f51d006c920 0x7f51d006ede0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40b31a0 0x7f51e40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e40cb840 0x7f51e41f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 --2- 172.21.15.40:0/1526195099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51cc004120 0x7f51e41f4dc0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 >> 172.21.15.40:0/1526195099 conn(0x7f51e408eca0 msgr2=0x7f51e40b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 shutdown_connections 2024-04-26T21:20:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:47.973+0000 7f51eabaf640 1 -- 172.21.15.40:0/1526195099 wait complete. 2024-04-26T21:20:47.988 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:20:47.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:20:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.097+0000 7efecef11640 1 Processor -- start 2024-04-26T21:20:48.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.097+0000 7efecef11640 1 -- start start 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efec8143840 0x7efec80b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efec80b4520 0x7efec80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec8160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efec8143c20 con 0x7efec8143840 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efec815dca0 con 0x7efec80b4520 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecef11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efec815df80 con 0x7efec80b53f0 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efec8143840 0x7efec80b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efec8143840 0x7efec80b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35116/0 (socket says 172.21.15.40:35116) 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 -- 172.21.15.40:0/2222510205 learned_addr learned my addr 172.21.15.40:0/2222510205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 -- 172.21.15.40:0/2222510205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 msgr2=0x7efec8160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 --2- 172.21.15.40:0/2222510205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec8160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 -- 172.21.15.40:0/2222510205 >> v2:172.21.15.184:3300/0 conn(0x7efec80b4520 msgr2=0x7efec80b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 --2- 172.21.15.40:0/2222510205 >> v2:172.21.15.184:3300/0 conn(0x7efec80b4520 0x7efec80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.098+0000 7efecccad640 1 -- 172.21.15.40:0/2222510205 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec815e260 con 0x7efec8143840 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.099+0000 7efecccad640 1 --2- 172.21.15.40:0/2222510205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec8143840 0x7efec80b3fe0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7efec80b0a90 tx=0x7efebc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a3d68bc4276755 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.099+0000 7efec77fe640 1 -- 172.21.15.40:0/2222510205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc037070 con 0x7efec8143840 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.099+0000 7efec77fe640 1 -- 172.21.15.40:0/2222510205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc032440 con 0x7efec8143840 2024-04-26T21:20:48.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.099+0000 7efec77fe640 1 -- 172.21.15.40:0/2222510205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc032810 con 0x7efec8143840 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 -- 172.21.15.40:0/2222510205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec8143840 msgr2=0x7efec80b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 --2- 172.21.15.40:0/2222510205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec8143840 0x7efec80b3fe0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7efec80b0a90 tx=0x7efebc02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 -- 172.21.15.40:0/2222510205 shutdown_connections 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 --2- 172.21.15.40:0/2222510205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec8160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 --2- 172.21.15.40:0/2222510205 >> v2:172.21.15.184:3300/0 conn(0x7efec80b4520 0x7efec80b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 --2- 172.21.15.40:0/2222510205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec8143840 0x7efec80b3fe0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 -- 172.21.15.40:0/2222510205 >> 172.21.15.40:0/2222510205 conn(0x7efec80cd8a0 msgr2=0x7efec80cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 -- 172.21.15.40:0/2222510205 shutdown_connections 2024-04-26T21:20:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.100+0000 7efecef11640 1 -- 172.21.15.40:0/2222510205 wait complete. 2024-04-26T21:20:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.101+0000 7efecef11640 1 Processor -- start 2024-04-26T21:20:48.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.101+0000 7efecef11640 1 -- start start 2024-04-26T21:20:48.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec80726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efec8143840 0x7efec8135dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efec815da40 con 0x7efec80b4520 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efec815d8c0 con 0x7efec8143840 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efec8161380 con 0x7efec80b53f0 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35132/0 (socket says 172.21.15.40:35132) 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 -- 172.21.15.40:0/3391870030 learned_addr learned my addr 172.21.15.40:0/3391870030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecd4ae640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efec8143840 0x7efec8135dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efec7fff640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec80726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 -- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 msgr2=0x7efec80726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec80726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 -- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efec8143840 msgr2=0x7efec8135dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efec8143840 0x7efec8135dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 -- 172.21.15.40:0/3391870030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec8072ed0 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecccad640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc041070 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc03c870 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.102+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec80731b0 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.103+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc037040 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.103+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec8073770 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.104+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec806a290 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.104+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efebc044070 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.104+0000 7efec57fa640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.104+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7efebc0ac8c0 con 0x7efec80b4520 2024-04-26T21:20:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.105+0000 7efec7fff640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.105+0000 7efec7fff640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 0x7efeac06edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efec813a520 tx=0x7efeb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.109+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efebc07aea0 con 0x7efec80b4520 2024-04-26T21:20:48.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.337+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efec8133ea0 con 0x7efec80b4520 2024-04-26T21:20:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.974+0000 7efec57fa640 1 -- 172.21.15.40:0/3391870030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7efec8133ea0 con 0x7efec80b4520 2024-04-26T21:20:48.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.976+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 msgr2=0x7efeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.976+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 0x7efeac06edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efec813a520 tx=0x7efeb4008040 comp rx=0 tx=0).stop 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.976+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 msgr2=0x7efec8072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.976+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 shutdown_connections 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efec8143840 0x7efec8135dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efec80b53f0 0x7efec80726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 --2- 172.21.15.40:0/3391870030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efec80b4520 0x7efec8072160 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 >> 172.21.15.40:0/3391870030 conn(0x7efec80cd8a0 msgr2=0x7efec80d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:48.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 shutdown_connections 2024-04-26T21:20:48.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:48.977+0000 7efecef11640 1 -- 172.21.15.40:0/3391870030 wait complete. 2024-04-26T21:20:48.991 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:20:48.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.097+0000 7f694b46d640 1 Processor -- start 2024-04-26T21:20:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.097+0000 7f694b46d640 1 -- start start 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f694b46d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f694415f720 0x7f69440b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f694b46d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 0x7f69440d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f694b46d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69440b2510 con 0x7f694415f340 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f694b46d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f69440b27f0 con 0x7f694415f720 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f694b46d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f69440b2ad0 con 0x7f69440d5610 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f6948a08640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f694415f720 0x7f69440b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f6948a08640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f694415f720 0x7f69440b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35136/0 (socket says 172.21.15.40:35136) 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.098+0000 7f6948a08640 1 -- 172.21.15.40:0/3377305634 learned_addr learned my addr 172.21.15.40:0/3377305634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6948a08640 1 -- 172.21.15.40:0/3377305634 >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 msgr2=0x7f69440d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6949a0a640 1 --2- 172.21.15.40:0/3377305634 >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 0x7f69440d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6948a08640 1 --2- 172.21.15.40:0/3377305634 >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 0x7f69440d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6948a08640 1 -- 172.21.15.40:0/3377305634 >> v1:172.21.15.184:6789/0 conn(0x7f694415f340 legacy=0x7f69440d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6948a08640 1 -- 172.21.15.40:0/3377305634 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440b2db0 con 0x7f694415f720 2024-04-26T21:20:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6949a0a640 1 --2- 172.21.15.40:0/3377305634 >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 0x7f69440d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:49.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.099+0000 7f6948a08640 1 --2- 172.21.15.40:0/3377305634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f720 0x7f69440b1e80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f69440d4c20 tx=0x7f693802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aadf5b9fb2c5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f6943fff640 1 -- 172.21.15.40:0/3377305634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938037070 con 0x7f694415f720 2024-04-26T21:20:49.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f6943fff640 1 -- 172.21.15.40:0/3377305634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6938032440 con 0x7f694415f720 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f6943fff640 1 -- 172.21.15.40:0/3377305634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938032810 con 0x7f694415f720 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 -- 172.21.15.40:0/3377305634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f720 msgr2=0x7f69440b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 --2- 172.21.15.40:0/3377305634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f720 0x7f69440b1e80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f69440d4c20 tx=0x7f693802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 -- 172.21.15.40:0/3377305634 shutdown_connections 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 --2- 172.21.15.40:0/3377305634 >> v2:172.21.15.184:3300/0 conn(0x7f69440d5610 0x7f69440d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 --2- 172.21.15.40:0/3377305634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f720 0x7f69440b1e80 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 -- 172.21.15.40:0/3377305634 >> 172.21.15.40:0/3377305634 conn(0x7f694408ec70 msgr2=0x7f694409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 -- 172.21.15.40:0/3377305634 shutdown_connections 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.100+0000 7f694b46d640 1 -- 172.21.15.40:0/3377305634 wait complete. 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.101+0000 7f694b46d640 1 Processor -- start 2024-04-26T21:20:49.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.101+0000 7f694b46d640 1 -- start start 2024-04-26T21:20:49.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 0x7f694415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f694415f720 0x7f69440b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69440b3940 con 0x7f694415f340 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69440b37c0 con 0x7f69440d5610 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f694b46d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69440b3ac0 con 0x7f694415f720 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35140/0 (socket says 172.21.15.40:35140) 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 -- 172.21.15.40:0/1773856607 learned_addr learned my addr 172.21.15.40:0/1773856607 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 -- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f694415f720 msgr2=0x7f69440b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f694415f720 0x7f69440b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6948a08640 1 -- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 msgr2=0x7f694415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.102+0000 7f6949209640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 0x7f694415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6948a08640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 0x7f694415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6948a08640 1 -- 172.21.15.40:0/1773856607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440b8690 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6949209640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 0x7f694415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6948a08640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f6938000c00 tx=0x7f69380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938037040 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6938042070 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69440b8970 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938030bb0 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.103+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69440b8f30 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.104+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694406a290 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.104+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6938047460 con 0x7f694415f340 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.105+0000 7f6941ffb640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 0x7f693006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.105+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f69380ac750 con 0x7f694415f340 2024-04-26T21:20:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.105+0000 7f6949209640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 0x7f693006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.106+0000 7f6949209640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 0x7f693006edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f693c003eb0 tx=0x7f693c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.109+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f693807ab80 con 0x7f694415f340 2024-04-26T21:20:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.331+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69440b2020 con 0x7f694415f340 2024-04-26T21:20:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.332+0000 7f6941ffb640 1 -- 172.21.15.40:0/1773856607 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5191 (secure 0 0 0) 0x7f693807a480 con 0x7f694415f340 2024-04-26T21:20:49.333 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:49.333 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:20:48.339156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:20:49.333 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 88 2024-04-26T21:20:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 msgr2=0x7f693006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 0x7f693006edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f693c003eb0 tx=0x7f693c009040 comp rx=0 tx=0).stop 2024-04-26T21:20:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 msgr2=0x7f69440b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f6938000c00 tx=0x7f69380046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 shutdown_connections 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f693006c8f0 0x7f693006edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f694415f720 0x7f69440b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f694415f340 0x7f69440b7e40 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 --2- 172.21.15.40:0/1773856607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69440d5610 0x7f694415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 >> 172.21.15.40:0/1773856607 conn(0x7f694408ec70 msgr2=0x7f69440b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 shutdown_connections 2024-04-26T21:20:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.334+0000 7f694b46d640 1 -- 172.21.15.40:0/1773856607 wait complete. 2024-04-26T21:20:49.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:20:49.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.446+0000 7efe592fc640 1 Processor -- start 2024-04-26T21:20:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.446+0000 7efe592fc640 1 -- start start 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe592fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe540b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe592fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efe540d3320 0x7efe540d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe592fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe540d4060 con 0x7efe540d51a0 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe592fc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7efe540d0030 con 0x7efe540d3320 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe592fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe540d2440 con 0x7efe5415f750 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe537fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe540d51a0 0x7efe540d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57722/0 (socket says 172.21.15.40:57722) 2024-04-26T21:20:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.447+0000 7efe537fe640 1 -- 172.21.15.40:0/3064264022 learned_addr learned my addr 172.21.15.40:0/3064264022 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe527fc640 1 --2- 172.21.15.40:0/3064264022 >> v2:172.21.15.184:3300/0 conn(0x7efe540d3320 0x7efe540d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe52ffd640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe540b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303279191 0 0) 0x7efe540d4060 con 0x7efe540d51a0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe40003520 con 0x7efe540d51a0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450694738 0 0) 0x7efe40003520 con 0x7efe540d51a0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 msgr2=0x7efe540b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe540b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 >> v2:172.21.15.184:3300/0 conn(0x7efe540d3320 msgr2=0x7efe540d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 --2- 172.21.15.40:0/3064264022 >> v2:172.21.15.184:3300/0 conn(0x7efe540d3320 0x7efe540d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe540d2720 con 0x7efe540d51a0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7efe440019f0 con 0x7efe540d51a0 2024-04-26T21:20:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 >> v1:172.21.15.40:6789/0 conn(0x7efe540d51a0 legacy=0x7efe540d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe40003f30 0x7efe40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe40004d40 0x7efe400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe540d0030 con 0x7efe40004d40 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe540d2440 con 0x7efe40003f30 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe4000b9c0 con 0x7efe5415f750 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe537fe640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe40004d40 0x7efe400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe52ffd640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.448+0000 7efe527fc640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe40003f30 0x7efe40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 -- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 msgr2=0x7efe400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5415f750 0x7efe400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 -- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe40003f30 msgr2=0x7efe40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe40003f30 0x7efe40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 -- 172.21.15.40:0/3064264022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe40003520 con 0x7efe40004d40 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe537fe640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe40004d40 0x7efe400051c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efe440013d0 tx=0x7efe40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f97f186e290559 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4400a4e0 con 0x7efe40004d40 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4402f070 con 0x7efe40004d40 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.449+0000 7efe51ffb640 1 -- 172.21.15.40:0/3064264022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4402bcd0 con 0x7efe40004d40 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 -- 172.21.15.40:0/3064264022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe40004d40 msgr2=0x7efe400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 --2- 172.21.15.40:0/3064264022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe40004d40 0x7efe400051c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efe440013d0 tx=0x7efe40005830 comp rx=0 tx=0).stop 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe537fe640 1 -- 172.21.15.40:0/3064264022 reap_dead start 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 -- 172.21.15.40:0/3064264022 shutdown_connections 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 -- 172.21.15.40:0/3064264022 >> 172.21.15.40:0/3064264022 conn(0x7efe5408ec20 msgr2=0x7efe5409e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 -- 172.21.15.40:0/3064264022 shutdown_connections 2024-04-26T21:20:49.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.450+0000 7efe592fc640 1 -- 172.21.15.40:0/3064264022 wait complete. 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.451+0000 7efe592fc640 1 Processor -- start 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.451+0000 7efe592fc640 1 -- start start 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe540cc620 0x7efe540ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540ceca0 0x7efe540b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe540b5260 con 0x7efe54154020 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe540b50e0 con 0x7efe540cc620 2024-04-26T21:20:49.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe592fc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efe540b53e0 con 0x7efe540ceca0 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35148/0 (socket says 172.21.15.40:35148) 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 -- 172.21.15.40:0/2411082416 learned_addr learned my addr 172.21.15.40:0/2411082416 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 -- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540ceca0 msgr2=0x7efe540b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540ceca0 0x7efe540b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 -- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe540cc620 msgr2=0x7efe540ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe540cc620 0x7efe540ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.452+0000 7efe52ffd640 1 -- 172.21.15.40:0/2411082416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe540ccbc0 con 0x7efe54154020 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe52ffd640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0119a0 tx=0x7efe4c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00fa10 con 0x7efe54154020 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c00fbb0 con 0x7efe54154020 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe540ccea0 con 0x7efe54154020 2024-04-26T21:20:49.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c014440 con 0x7efe54154020 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.453+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe540cd460 con 0x7efe54154020 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.454+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe540cf0c0 con 0x7efe54154020 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.454+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe4c012040 con 0x7efe54154020 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.455+0000 7efe4b7fe640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 0x7efe3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.455+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7efe4c08c160 con 0x7efe54154020 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.455+0000 7efe527fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 0x7efe3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.455+0000 7efe527fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 0x7efe3406ede0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efe3c006b10 tx=0x7efe3c006aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.459+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe4c019030 con 0x7efe54154020 2024-04-26T21:20:49.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.645+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe540693d0 con 0x7efe54154020 2024-04-26T21:20:49.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.646+0000 7efe4b7fe640 1 -- 172.21.15.40:0/2411082416 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7efe4c05a590 con 0x7efe54154020 2024-04-26T21:20:49.648 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:49.648 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:20:49.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 msgr2=0x7efe3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 0x7efe3406ede0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efe3c006b10 tx=0x7efe3c006aa0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 msgr2=0x7efe540b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0119a0 tx=0x7efe4c011e60 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 shutdown_connections 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efe3406c920 0x7efe3406ede0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe540ceca0 0x7efe540b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe540cc620 0x7efe540ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 --2- 172.21.15.40:0/2411082416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54154020 0x7efe540b7ec0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 >> 172.21.15.40:0/2411082416 conn(0x7efe5408ec20 msgr2=0x7efe540b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 shutdown_connections 2024-04-26T21:20:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.648+0000 7efe592fc640 1 -- 172.21.15.40:0/2411082416 wait complete. 2024-04-26T21:20:49.661 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:20:49.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 Processor -- start 2024-04-26T21:20:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 -- start start 2024-04-26T21:20:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4b015f4a0 0x7ff4b00b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 0x7ff4b00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b00aed90 con 0x7ff4b00afe30 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b015f880 con 0x7ff4b015f4a0 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.761+0000 7ff4b5d9e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b00af9a0 con 0x7ff4b00b6320 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4af7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4b015f4a0 0x7ff4b00b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4aeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff4b00afe30 0x7ff4b0162f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57732/0 (socket says 172.21.15.40:57732) 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4aeffd640 1 -- 172.21.15.40:0/3128713585 learned_addr learned my addr 172.21.15.40:0/3128713585 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4af7fe640 1 -- 172.21.15.40:0/3128713585 >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 msgr2=0x7ff4b00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:49.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4affff640 1 --2- 172.21.15.40:0/3128713585 >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 0x7ff4b00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4ae7fc640 1 -- 172.21.15.40:0/3128713585 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903067045 0 0) 0x7ff4b00aed90 con 0x7ff4b00afe30 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4af7fe640 1 --2- 172.21.15.40:0/3128713585 >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 0x7ff4b00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4af7fe640 1 -- 172.21.15.40:0/3128713585 >> v1:172.21.15.40:6789/0 conn(0x7ff4b00afe30 legacy=0x7ff4b0162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4af7fe640 1 -- 172.21.15.40:0/3128713585 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b00af070 con 0x7ff4b015f4a0 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.762+0000 7ff4affff640 1 --2- 172.21.15.40:0/3128713585 >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 0x7ff4b00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4af7fe640 1 --2- 172.21.15.40:0/3128713585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b015f4a0 0x7ff4b00b40c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0071cd0 tx=0x7ff4a002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e910146e992041e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4ae7fc640 1 -- 172.21.15.40:0/3128713585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007740 con 0x7ff4b015f4a0 2024-04-26T21:20:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4ae7fc640 1 -- 172.21.15.40:0/3128713585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a00078e0 con 0x7ff4b015f4a0 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4ae7fc640 1 -- 172.21.15.40:0/3128713585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007cb0 con 0x7ff4b015f4a0 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/3128713585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b015f4a0 msgr2=0x7ff4b00b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/3128713585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b015f4a0 0x7ff4b00b40c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0071cd0 tx=0x7ff4a002a680 comp rx=0 tx=0).stop 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/3128713585 shutdown_connections 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/3128713585 >> v2:172.21.15.184:3300/0 conn(0x7ff4b00b6320 0x7ff4b00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/3128713585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b015f4a0 0x7ff4b00b40c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/3128713585 >> 172.21.15.40:0/3128713585 conn(0x7ff4b00b6800 msgr2=0x7ff4b00738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/3128713585 shutdown_connections 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.763+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/3128713585 wait complete. 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.764+0000 7ff4b5d9e640 1 Processor -- start 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 -- start start 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4b00b6320 0x7ff4b0143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 0x7ff4b009a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b00aea80 con 0x7ff4b00afe30 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b00af630 con 0x7ff4b015f4a0 2024-04-26T21:20:49.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4b5d9e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b00b1480 con 0x7ff4b00b6320 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.765+0000 7ff4af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35176/0 (socket says 172.21.15.40:35176) 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 -- 172.21.15.40:0/4221340472 learned_addr learned my addr 172.21.15.40:0/4221340472 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4affff640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 0x7ff4b009a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4aeffd640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4b00b6320 0x7ff4b0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 -- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4b00b6320 msgr2=0x7ff4b0143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4b00b6320 0x7ff4b0143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 -- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 msgr2=0x7ff4b009a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 0x7ff4b009a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 -- 172.21.15.40:0/4221340472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b009aed0 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4affff640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 0x7ff4b009a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4af7fe640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac70 tx=0x7ff4a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a003f070 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0031d00 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0044680 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b01438a0 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.766+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b0143e60 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.767+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b006a290 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.768+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4a003b410 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.768+0000 7ff497fff640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.768+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a00ad4a0 con 0x7ff4b00afe30 2024-04-26T21:20:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.769+0000 7ff4aeffd640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.769+0000 7ff4aeffd640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 0x7ff49c06edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0153d90 tx=0x7ff4a4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.773+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4b009aed0 con 0x7ff4b00afe30 2024-04-26T21:20:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.953+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4b006f950 con 0x7ff49c06c8f0 2024-04-26T21:20:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.960+0000 7ff497fff640 1 -- 172.21.15.40:0/4221340472 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7ff4b006f950 con 0x7ff49c06c8f0 2024-04-26T21:20:49.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:49.965 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 msgr2=0x7ff49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 0x7ff49c06edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0153d90 tx=0x7ff4a4007b50 comp rx=0 tx=0).stop 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 msgr2=0x7ff4b0145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac70 tx=0x7ff4a0004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 shutdown_connections 2024-04-26T21:20:49.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4b015f4a0 0x7ff4b009a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4b00b6320 0x7ff4b0143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 --2- 172.21.15.40:0/4221340472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4b00afe30 0x7ff4b0145d10 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 >> 172.21.15.40:0/4221340472 conn(0x7ff4b00b6800 msgr2=0x7ff4b006fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 shutdown_connections 2024-04-26T21:20:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:49.963+0000 7ff4b5d9e640 1 -- 172.21.15.40:0/4221340472 wait complete. 2024-04-26T21:20:49.975 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T21:20:49.045926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272044,"kb_used_data":14228,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754702676,"statfs":{"total":773094113280,"available":772815540224,"internally_reserved":0,"allocated":14569472,"data_stored":6921140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114211"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945782+0000","last_change":"2024-04-26T21:20:44.945782+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945782+0000","last_undegraded":"2024-04-26T21:20:44.945782+0000","last_fullsized":"2024-04-26T21:20:44.945782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336059+0000","last_change":"2024-04-26T21:20:45.965877+0000","last_active":"2024-04-26T21:20:47.336059+0000","last_peered":"2024-04-26T21:20:47.336059+0000","last_clean":"2024-04-26T21:20:47.336059+0000","last_became_active":"2024-04-26T21:20:45.965531+0000","last_became_peered":"2024-04-26T21:20:45.965531+0000","last_unstale":"2024-04-26T21:20:47.336059+0000","last_undegraded":"2024-04-26T21:20:47.336059+0000","last_fullsized":"2024-04-26T21:20:47.336059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:02.567341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336527+0000","last_change":"2024-04-26T21:20:45.972531+0000","last_active":"2024-04-26T21:20:47.336527+0000","last_peered":"2024-04-26T21:20:47.336527+0000","last_clean":"2024-04-26T21:20:47.336527+0000","last_became_active":"2024-04-26T21:20:45.972241+0000","last_became_peered":"2024-04-26T21:20:45.972241+0000","last_unstale":"2024-04-26T21:20:47.336527+0000","last_undegraded":"2024-04-26T21:20:47.336527+0000","last_fullsized":"2024-04-26T21:20:47.336527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336969+0000","last_change":"2024-04-26T21:20:45.973682+0000","last_active":"2024-04-26T21:20:47.336969+0000","last_peered":"2024-04-26T21:20:47.336969+0000","last_clean":"2024-04-26T21:20:47.336969+0000","last_became_active":"2024-04-26T21:20:45.973381+0000","last_became_peered":"2024-04-26T21:20:45.973381+0000","last_unstale":"2024-04-26T21:20:47.336969+0000","last_undegraded":"2024-04-26T21:20:47.336969+0000","last_fullsized":"2024-04-26T21:20:47.336969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:34.199283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962690+0000","last_change":"2024-04-26T21:20:45.967509+0000","last_active":"2024-04-26T21:20:46.962690+0000","last_peered":"2024-04-26T21:20:46.962690+0000","last_clean":"2024-04-26T21:20:46.962690+0000","last_became_active":"2024-04-26T21:20:45.967269+0000","last_became_peered":"2024-04-26T21:20:45.967269+0000","last_unstale":"2024-04-26T21:20:46.962690+0000","last_undegraded":"2024-04-26T21:20:46.962690+0000","last_fullsized":"2024-04-26T21:20:46.962690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.129198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943159+0000","last_change":"2024-04-26T21:20:44.943159+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943159+0000","last_undegraded":"2024-04-26T21:20:44.943159+0000","last_fullsized":"2024-04-26T21:20:44.943159+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335603+0000","last_change":"2024-04-26T21:20:45.963229+0000","last_active":"2024-04-26T21:20:47.335603+0000","last_peered":"2024-04-26T21:20:47.335603+0000","last_clean":"2024-04-26T21:20:47.335603+0000","last_became_active":"2024-04-26T21:20:45.962872+0000","last_became_peered":"2024-04-26T21:20:45.962872+0000","last_unstale":"2024-04-26T21:20:47.335603+0000","last_undegraded":"2024-04-26T21:20:47.335603+0000","last_fullsized":"2024-04-26T21:20:47.335603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942563+0000","last_change":"2024-04-26T21:20:44.942563+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942563+0000","last_undegraded":"2024-04-26T21:20:44.942563+0000","last_fullsized":"2024-04-26T21:20:44.942563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336113+0000","last_change":"2024-04-26T21:20:45.968152+0000","last_active":"2024-04-26T21:20:47.336113+0000","last_peered":"2024-04-26T21:20:47.336113+0000","last_clean":"2024-04-26T21:20:47.336113+0000","last_became_active":"2024-04-26T21:20:45.967845+0000","last_became_peered":"2024-04-26T21:20:45.967845+0000","last_unstale":"2024-04-26T21:20:47.336113+0000","last_undegraded":"2024-04-26T21:20:47.336113+0000","last_fullsized":"2024-04-26T21:20:47.336113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.001965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961661+0000","last_change":"2024-04-26T21:20:45.977906+0000","last_active":"2024-04-26T21:20:46.961661+0000","last_peered":"2024-04-26T21:20:46.961661+0000","last_clean":"2024-04-26T21:20:46.961661+0000","last_became_active":"2024-04-26T21:20:45.977567+0000","last_became_peered":"2024-04-26T21:20:45.977567+0000","last_unstale":"2024-04-26T21:20:46.961661+0000","last_undegraded":"2024-04-26T21:20:46.961661+0000","last_fullsized":"2024-04-26T21:20:46.961661+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946943+0000","last_change":"2024-04-26T21:20:44.946943+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946943+0000","last_undegraded":"2024-04-26T21:20:44.946943+0000","last_fullsized":"2024-04-26T21:20:44.946943+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961702+0000","last_change":"2024-04-26T21:20:45.972783+0000","last_active":"2024-04-26T21:20:46.961702+0000","last_peered":"2024-04-26T21:20:46.961702+0000","last_clean":"2024-04-26T21:20:46.961702+0000","last_became_active":"2024-04-26T21:20:45.972522+0000","last_became_peered":"2024-04-26T21:20:45.972522+0000","last_unstale":"2024-04-26T21:20:46.961702+0000","last_undegraded":"2024-04-26T21:20:46.961702+0000","last_fullsized":"2024-04-26T21:20:46.961702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962682+0000","last_change":"2024-04-26T21:20:45.967507+0000","last_active":"2024-04-26T21:20:46.962682+0000","last_peered":"2024-04-26T21:20:46.962682+0000","last_clean":"2024-04-26T21:20:46.962682+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.962682+0000","last_undegraded":"2024-04-26T21:20:46.962682+0000","last_fullsized":"2024-04-26T21:20:46.962682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:40.380793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:45.967357+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:45.966578+0000","last_became_peered":"2024-04-26T21:20:45.966578+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961095+0000","last_change":"2024-04-26T21:20:45.974674+0000","last_active":"2024-04-26T21:20:46.961095+0000","last_peered":"2024-04-26T21:20:46.961095+0000","last_clean":"2024-04-26T21:20:46.961095+0000","last_became_active":"2024-04-26T21:20:45.974359+0000","last_became_peered":"2024-04-26T21:20:45.974359+0000","last_unstale":"2024-04-26T21:20:46.961095+0000","last_undegraded":"2024-04-26T21:20:46.961095+0000","last_fullsized":"2024-04-26T21:20:46.961095+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:49.051775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959220+0000","last_change":"2024-04-26T21:20:45.968322+0000","last_active":"2024-04-26T21:20:46.959220+0000","last_peered":"2024-04-26T21:20:46.959220+0000","last_clean":"2024-04-26T21:20:46.959220+0000","last_became_active":"2024-04-26T21:20:45.967746+0000","last_became_peered":"2024-04-26T21:20:45.967746+0000","last_unstale":"2024-04-26T21:20:46.959220+0000","last_undegraded":"2024-04-26T21:20:46.959220+0000","last_fullsized":"2024-04-26T21:20:46.959220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.881502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962071+0000","last_change":"2024-04-26T21:20:45.979392+0000","last_active":"2024-04-26T21:20:46.962071+0000","last_peered":"2024-04-26T21:20:46.962071+0000","last_clean":"2024-04-26T21:20:46.962071+0000","last_became_active":"2024-04-26T21:20:45.978894+0000","last_became_peered":"2024-04-26T21:20:45.978894+0000","last_unstale":"2024-04-26T21:20:46.962071+0000","last_undegraded":"2024-04-26T21:20:46.962071+0000","last_fullsized":"2024-04-26T21:20:46.962071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:04.038090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336798+0000","last_change":"2024-04-26T21:20:45.971458+0000","last_active":"2024-04-26T21:20:47.336798+0000","last_peered":"2024-04-26T21:20:47.336798+0000","last_clean":"2024-04-26T21:20:47.336798+0000","last_became_active":"2024-04-26T21:20:45.971186+0000","last_became_peered":"2024-04-26T21:20:45.971186+0000","last_unstale":"2024-04-26T21:20:47.336798+0000","last_undegraded":"2024-04-26T21:20:47.336798+0000","last_fullsized":"2024-04-26T21:20:47.336798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945523+0000","last_change":"2024-04-26T21:20:44.945523+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945523+0000","last_undegraded":"2024-04-26T21:20:44.945523+0000","last_fullsized":"2024-04-26T21:20:44.945523+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959492+0000","last_change":"2024-04-26T21:20:45.968075+0000","last_active":"2024-04-26T21:20:46.959492+0000","last_peered":"2024-04-26T21:20:46.959492+0000","last_clean":"2024-04-26T21:20:46.959492+0000","last_became_active":"2024-04-26T21:20:45.967734+0000","last_became_peered":"2024-04-26T21:20:45.967734+0000","last_unstale":"2024-04-26T21:20:46.959492+0000","last_undegraded":"2024-04-26T21:20:46.959492+0000","last_fullsized":"2024-04-26T21:20:46.959492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.314546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961395+0000","last_change":"2024-04-26T21:20:45.977923+0000","last_active":"2024-04-26T21:20:46.961395+0000","last_peered":"2024-04-26T21:20:46.961395+0000","last_clean":"2024-04-26T21:20:46.961395+0000","last_became_active":"2024-04-26T21:20:45.977624+0000","last_became_peered":"2024-04-26T21:20:45.977624+0000","last_unstale":"2024-04-26T21:20:46.961395+0000","last_undegraded":"2024-04-26T21:20:46.961395+0000","last_fullsized":"2024-04-26T21:20:46.961395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336657+0000","last_change":"2024-04-26T21:20:45.956281+0000","last_active":"2024-04-26T21:20:47.336657+0000","last_peered":"2024-04-26T21:20:47.336657+0000","last_clean":"2024-04-26T21:20:47.336657+0000","last_became_active":"2024-04-26T21:20:45.955972+0000","last_became_peered":"2024-04-26T21:20:45.955972+0000","last_unstale":"2024-04-26T21:20:47.336657+0000","last_undegraded":"2024-04-26T21:20:47.336657+0000","last_fullsized":"2024-04-26T21:20:47.336657+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336022+0000","last_change":"2024-04-26T21:20:45.967466+0000","last_active":"2024-04-26T21:20:47.336022+0000","last_peered":"2024-04-26T21:20:47.336022+0000","last_clean":"2024-04-26T21:20:47.336022+0000","last_became_active":"2024-04-26T21:20:45.967180+0000","last_became_peered":"2024-04-26T21:20:45.967180+0000","last_unstale":"2024-04-26T21:20:47.336022+0000","last_undegraded":"2024-04-26T21:20:47.336022+0000","last_fullsized":"2024-04-26T21:20:47.336022+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961176+0000","last_change":"2024-04-26T21:20:45.966830+0000","last_active":"2024-04-26T21:20:46.961176+0000","last_peered":"2024-04-26T21:20:46.961176+0000","last_clean":"2024-04-26T21:20:46.961176+0000","last_became_active":"2024-04-26T21:20:45.966121+0000","last_became_peered":"2024-04-26T21:20:45.966121+0000","last_unstale":"2024-04-26T21:20:46.961176+0000","last_undegraded":"2024-04-26T21:20:46.961176+0000","last_fullsized":"2024-04-26T21:20:46.961176+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:50.753188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959755+0000","last_change":"2024-04-26T21:20:45.968233+0000","last_active":"2024-04-26T21:20:46.959755+0000","last_peered":"2024-04-26T21:20:46.959755+0000","last_clean":"2024-04-26T21:20:46.959755+0000","last_became_active":"2024-04-26T21:20:45.967940+0000","last_became_peered":"2024-04-26T21:20:45.967940+0000","last_unstale":"2024-04-26T21:20:46.959755+0000","last_undegraded":"2024-04-26T21:20:46.959755+0000","last_fullsized":"2024-04-26T21:20:46.959755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:34.779995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.944578+0000","last_change":"2024-04-26T21:20:44.944578+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.944578+0000","last_undegraded":"2024-04-26T21:20:44.944578+0000","last_fullsized":"2024-04-26T21:20:44.944578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959794+0000","last_change":"2024-04-26T21:20:45.973518+0000","last_active":"2024-04-26T21:20:46.959794+0000","last_peered":"2024-04-26T21:20:46.959794+0000","last_clean":"2024-04-26T21:20:46.959794+0000","last_became_active":"2024-04-26T21:20:45.973227+0000","last_became_peered":"2024-04-26T21:20:45.973227+0000","last_unstale":"2024-04-26T21:20:46.959794+0000","last_undegraded":"2024-04-26T21:20:46.959794+0000","last_fullsized":"2024-04-26T21:20:46.959794+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:12.310488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336254+0000","last_change":"2024-04-26T21:20:45.973958+0000","last_active":"2024-04-26T21:20:47.336254+0000","last_peered":"2024-04-26T21:20:47.336254+0000","last_clean":"2024-04-26T21:20:47.336254+0000","last_became_active":"2024-04-26T21:20:45.973675+0000","last_became_peered":"2024-04-26T21:20:45.973675+0000","last_unstale":"2024-04-26T21:20:47.336254+0000","last_undegraded":"2024-04-26T21:20:47.336254+0000","last_fullsized":"2024-04-26T21:20:47.336254+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943772+0000","last_change":"2024-04-26T21:20:44.943772+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943772+0000","last_undegraded":"2024-04-26T21:20:44.943772+0000","last_fullsized":"2024-04-26T21:20:44.943772+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336855+0000","last_change":"2024-04-26T21:20:45.970795+0000","last_active":"2024-04-26T21:20:47.336855+0000","last_peered":"2024-04-26T21:20:47.336855+0000","last_clean":"2024-04-26T21:20:47.336855+0000","last_became_active":"2024-04-26T21:20:45.970509+0000","last_became_peered":"2024-04-26T21:20:45.970509+0000","last_unstale":"2024-04-26T21:20:47.336855+0000","last_undegraded":"2024-04-26T21:20:47.336855+0000","last_fullsized":"2024-04-26T21:20:47.336855+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:30.015334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335701+0000","last_change":"2024-04-26T21:20:45.972356+0000","last_active":"2024-04-26T21:20:47.335701+0000","last_peered":"2024-04-26T21:20:47.335701+0000","last_clean":"2024-04-26T21:20:47.335701+0000","last_became_active":"2024-04-26T21:20:45.972064+0000","last_became_peered":"2024-04-26T21:20:45.972064+0000","last_unstale":"2024-04-26T21:20:47.335701+0000","last_undegraded":"2024-04-26T21:20:47.335701+0000","last_fullsized":"2024-04-26T21:20:47.335701+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:50.058209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959285+0000","last_change":"2024-04-26T21:20:45.968435+0000","last_active":"2024-04-26T21:20:46.959285+0000","last_peered":"2024-04-26T21:20:46.959285+0000","last_clean":"2024-04-26T21:20:46.959285+0000","last_became_active":"2024-04-26T21:20:45.966848+0000","last_became_peered":"2024-04-26T21:20:45.966848+0000","last_unstale":"2024-04-26T21:20:46.959285+0000","last_undegraded":"2024-04-26T21:20:46.959285+0000","last_fullsized":"2024-04-26T21:20:46.959285+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:50.873852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961713+0000","last_change":"2024-04-26T21:20:45.973734+0000","last_active":"2024-04-26T21:20:46.961713+0000","last_peered":"2024-04-26T21:20:46.961713+0000","last_clean":"2024-04-26T21:20:46.961713+0000","last_became_active":"2024-04-26T21:20:45.973472+0000","last_became_peered":"2024-04-26T21:20:45.973472+0000","last_unstale":"2024-04-26T21:20:46.961713+0000","last_undegraded":"2024-04-26T21:20:46.961713+0000","last_fullsized":"2024-04-26T21:20:46.961713+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961961+0000","last_change":"2024-04-26T21:20:44.958400+0000","last_active":"2024-04-26T21:20:46.961961+0000","last_peered":"2024-04-26T21:20:46.961961+0000","last_clean":"2024-04-26T21:20:46.961961+0000","last_became_active":"2024-04-26T21:20:44.958081+0000","last_became_peered":"2024-04-26T21:20:44.958081+0000","last_unstale":"2024-04-26T21:20:46.961961+0000","last_undegraded":"2024-04-26T21:20:46.961961+0000","last_fullsized":"2024-04-26T21:20:46.961961+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:38.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336119+0000","last_change":"2024-04-26T21:20:45.964932+0000","last_active":"2024-04-26T21:20:47.336119+0000","last_peered":"2024-04-26T21:20:47.336119+0000","last_clean":"2024-04-26T21:20:47.336119+0000","last_became_active":"2024-04-26T21:20:45.961496+0000","last_became_peered":"2024-04-26T21:20:45.961496+0000","last_unstale":"2024-04-26T21:20:47.336119+0000","last_undegraded":"2024-04-26T21:20:47.336119+0000","last_fullsized":"2024-04-26T21:20:47.336119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.885551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335890+0000","last_change":"2024-04-26T21:20:44.956115+0000","last_active":"2024-04-26T21:20:47.335890+0000","last_peered":"2024-04-26T21:20:47.335890+0000","last_clean":"2024-04-26T21:20:47.335890+0000","last_became_active":"2024-04-26T21:20:44.955813+0000","last_became_peered":"2024-04-26T21:20:44.955813+0000","last_unstale":"2024-04-26T21:20:47.335890+0000","last_undegraded":"2024-04-26T21:20:47.335890+0000","last_fullsized":"2024-04-26T21:20:47.335890+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:21.778534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942776+0000","last_change":"2024-04-26T21:20:44.942776+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942776+0000","last_undegraded":"2024-04-26T21:20:44.942776+0000","last_fullsized":"2024-04-26T21:20:44.942776+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.949711+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.949393+0000","last_became_peered":"2024-04-26T21:20:44.949393+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961372+0000","last_change":"2024-04-26T21:20:45.964911+0000","last_active":"2024-04-26T21:20:46.961372+0000","last_peered":"2024-04-26T21:20:46.961372+0000","last_clean":"2024-04-26T21:20:46.961372+0000","last_became_active":"2024-04-26T21:20:45.963945+0000","last_became_peered":"2024-04-26T21:20:45.963945+0000","last_unstale":"2024-04-26T21:20:46.961372+0000","last_undegraded":"2024-04-26T21:20:46.961372+0000","last_fullsized":"2024-04-26T21:20:46.961372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:41.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.337000+0000","last_change":"2024-04-26T21:20:44.949483+0000","last_active":"2024-04-26T21:20:47.337000+0000","last_peered":"2024-04-26T21:20:47.337000+0000","last_clean":"2024-04-26T21:20:47.337000+0000","last_became_active":"2024-04-26T21:20:44.948183+0000","last_became_peered":"2024-04-26T21:20:44.948183+0000","last_unstale":"2024-04-26T21:20:47.337000+0000","last_undegraded":"2024-04-26T21:20:47.337000+0000","last_fullsized":"2024-04-26T21:20:47.337000+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335767+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:47.335767+0000","last_peered":"2024-04-26T21:20:47.335767+0000","last_clean":"2024-04-26T21:20:47.335767+0000","last_became_active":"2024-04-26T21:20:45.973350+0000","last_became_peered":"2024-04-26T21:20:45.973350+0000","last_unstale":"2024-04-26T21:20:47.335767+0000","last_undegraded":"2024-04-26T21:20:47.335767+0000","last_fullsized":"2024-04-26T21:20:47.335767+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.225182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959303+0000","last_change":"2024-04-26T21:20:44.954463+0000","last_active":"2024-04-26T21:20:46.959303+0000","last_peered":"2024-04-26T21:20:46.959303+0000","last_clean":"2024-04-26T21:20:46.959303+0000","last_became_active":"2024-04-26T21:20:44.954133+0000","last_became_peered":"2024-04-26T21:20:44.954133+0000","last_unstale":"2024-04-26T21:20:46.959303+0000","last_undegraded":"2024-04-26T21:20:46.959303+0000","last_fullsized":"2024-04-26T21:20:46.959303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:16.181685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336085+0000","last_change":"2024-04-26T21:20:45.978602+0000","last_active":"2024-04-26T21:20:47.336085+0000","last_peered":"2024-04-26T21:20:47.336085+0000","last_clean":"2024-04-26T21:20:47.336085+0000","last_became_active":"2024-04-26T21:20:45.978300+0000","last_became_peered":"2024-04-26T21:20:45.978300+0000","last_unstale":"2024-04-26T21:20:47.336085+0000","last_undegraded":"2024-04-26T21:20:47.336085+0000","last_fullsized":"2024-04-26T21:20:47.336085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962713+0000","last_change":"2024-04-26T21:20:44.953754+0000","last_active":"2024-04-26T21:20:46.962713+0000","last_peered":"2024-04-26T21:20:46.962713+0000","last_clean":"2024-04-26T21:20:46.962713+0000","last_became_active":"2024-04-26T21:20:44.953471+0000","last_became_peered":"2024-04-26T21:20:44.953471+0000","last_unstale":"2024-04-26T21:20:46.962713+0000","last_undegraded":"2024-04-26T21:20:46.962713+0000","last_fullsized":"2024-04-26T21:20:46.962713+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:21.877111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335605+0000","last_change":"2024-04-26T21:20:45.957101+0000","last_active":"2024-04-26T21:20:47.335605+0000","last_peered":"2024-04-26T21:20:47.335605+0000","last_clean":"2024-04-26T21:20:47.335605+0000","last_became_active":"2024-04-26T21:20:45.956746+0000","last_became_peered":"2024-04-26T21:20:45.956746+0000","last_unstale":"2024-04-26T21:20:47.335605+0000","last_undegraded":"2024-04-26T21:20:47.335605+0000","last_fullsized":"2024-04-26T21:20:47.335605+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:20.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959438+0000","last_change":"2024-04-26T21:20:44.945604+0000","last_active":"2024-04-26T21:20:46.959438+0000","last_peered":"2024-04-26T21:20:46.959438+0000","last_clean":"2024-04-26T21:20:46.959438+0000","last_became_active":"2024-04-26T21:20:44.945311+0000","last_became_peered":"2024-04-26T21:20:44.945311+0000","last_unstale":"2024-04-26T21:20:46.959438+0000","last_undegraded":"2024-04-26T21:20:46.959438+0000","last_fullsized":"2024-04-26T21:20:46.959438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:59.997109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962422+0000","last_change":"2024-04-26T21:20:45.968415+0000","last_active":"2024-04-26T21:20:46.962422+0000","last_peered":"2024-04-26T21:20:46.962422+0000","last_clean":"2024-04-26T21:20:46.962422+0000","last_became_active":"2024-04-26T21:20:45.968148+0000","last_became_peered":"2024-04-26T21:20:45.968148+0000","last_unstale":"2024-04-26T21:20:46.962422+0000","last_undegraded":"2024-04-26T21:20:46.962422+0000","last_fullsized":"2024-04-26T21:20:46.962422+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:30.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335772+0000","last_change":"2024-04-26T21:20:44.953539+0000","last_active":"2024-04-26T21:20:47.335772+0000","last_peered":"2024-04-26T21:20:47.335772+0000","last_clean":"2024-04-26T21:20:47.335772+0000","last_became_active":"2024-04-26T21:20:44.953286+0000","last_became_peered":"2024-04-26T21:20:44.953286+0000","last_unstale":"2024-04-26T21:20:47.335772+0000","last_undegraded":"2024-04-26T21:20:47.335772+0000","last_fullsized":"2024-04-26T21:20:47.335772+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962742+0000","last_change":"2024-04-26T21:20:45.957232+0000","last_active":"2024-04-26T21:20:46.962742+0000","last_peered":"2024-04-26T21:20:46.962742+0000","last_clean":"2024-04-26T21:20:46.962742+0000","last_became_active":"2024-04-26T21:20:45.956611+0000","last_became_peered":"2024-04-26T21:20:45.956611+0000","last_unstale":"2024-04-26T21:20:46.962742+0000","last_undegraded":"2024-04-26T21:20:46.962742+0000","last_fullsized":"2024-04-26T21:20:46.962742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:13.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336218+0000","last_change":"2024-04-26T21:20:44.952646+0000","last_active":"2024-04-26T21:20:47.336218+0000","last_peered":"2024-04-26T21:20:47.336218+0000","last_clean":"2024-04-26T21:20:47.336218+0000","last_became_active":"2024-04-26T21:20:44.952351+0000","last_became_peered":"2024-04-26T21:20:44.952351+0000","last_unstale":"2024-04-26T21:20:47.336218+0000","last_undegraded":"2024-04-26T21:20:47.336218+0000","last_fullsized":"2024-04-26T21:20:47.336218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:33.194892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961915+0000","last_change":"2024-04-26T21:20:45.979158+0000","last_active":"2024-04-26T21:20:46.961915+0000","last_peered":"2024-04-26T21:20:46.961915+0000","last_clean":"2024-04-26T21:20:46.961915+0000","last_became_active":"2024-04-26T21:20:45.978509+0000","last_became_peered":"2024-04-26T21:20:45.978509+0000","last_unstale":"2024-04-26T21:20:46.961915+0000","last_undegraded":"2024-04-26T21:20:46.961915+0000","last_fullsized":"2024-04-26T21:20:46.961915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:18.599945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961941+0000","last_change":"2024-04-26T21:20:44.958683+0000","last_active":"2024-04-26T21:20:46.961941+0000","last_peered":"2024-04-26T21:20:46.961941+0000","last_clean":"2024-04-26T21:20:46.961941+0000","last_became_active":"2024-04-26T21:20:44.958250+0000","last_became_peered":"2024-04-26T21:20:44.958250+0000","last_unstale":"2024-04-26T21:20:46.961941+0000","last_undegraded":"2024-04-26T21:20:46.961941+0000","last_fullsized":"2024-04-26T21:20:46.961941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:19.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959625+0000","last_change":"2024-04-26T21:20:45.968143+0000","last_active":"2024-04-26T21:20:46.959625+0000","last_peered":"2024-04-26T21:20:46.959625+0000","last_clean":"2024-04-26T21:20:46.959625+0000","last_became_active":"2024-04-26T21:20:45.966158+0000","last_became_peered":"2024-04-26T21:20:45.966158+0000","last_unstale":"2024-04-26T21:20:46.959625+0000","last_undegraded":"2024-04-26T21:20:46.959625+0000","last_fullsized":"2024-04-26T21:20:46.959625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:10.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336377+0000","last_change":"2024-04-26T21:20:44.951683+0000","last_active":"2024-04-26T21:20:47.336377+0000","last_peered":"2024-04-26T21:20:47.336377+0000","last_clean":"2024-04-26T21:20:47.336377+0000","last_became_active":"2024-04-26T21:20:44.951213+0000","last_became_peered":"2024-04-26T21:20:44.951213+0000","last_unstale":"2024-04-26T21:20:47.336377+0000","last_undegraded":"2024-04-26T21:20:47.336377+0000","last_fullsized":"2024-04-26T21:20:47.336377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:36.881160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336537+0000","last_change":"2024-04-26T21:20:45.968505+0000","last_active":"2024-04-26T21:20:47.336537+0000","last_peered":"2024-04-26T21:20:47.336537+0000","last_clean":"2024-04-26T21:20:47.336537+0000","last_became_active":"2024-04-26T21:20:45.968185+0000","last_became_peered":"2024-04-26T21:20:45.968185+0000","last_unstale":"2024-04-26T21:20:47.336537+0000","last_undegraded":"2024-04-26T21:20:47.336537+0000","last_fullsized":"2024-04-26T21:20:47.336537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336545+0000","last_change":"2024-04-26T21:20:44.951050+0000","last_active":"2024-04-26T21:20:47.336545+0000","last_peered":"2024-04-26T21:20:47.336545+0000","last_clean":"2024-04-26T21:20:47.336545+0000","last_became_active":"2024-04-26T21:20:44.950758+0000","last_became_peered":"2024-04-26T21:20:44.950758+0000","last_unstale":"2024-04-26T21:20:47.336545+0000","last_undegraded":"2024-04-26T21:20:47.336545+0000","last_fullsized":"2024-04-26T21:20:47.336545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:07.338750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335606+0000","last_change":"2024-04-26T21:20:45.965874+0000","last_active":"2024-04-26T21:20:47.335606+0000","last_peered":"2024-04-26T21:20:47.335606+0000","last_clean":"2024-04-26T21:20:47.335606+0000","last_became_active":"2024-04-26T21:20:45.965561+0000","last_became_peered":"2024-04-26T21:20:45.965561+0000","last_unstale":"2024-04-26T21:20:47.335606+0000","last_undegraded":"2024-04-26T21:20:47.335606+0000","last_fullsized":"2024-04-26T21:20:47.335606+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:00.164990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:44.949774+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:44.949464+0000","last_became_peered":"2024-04-26T21:20:44.949464+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962548+0000","last_change":"2024-04-26T21:20:45.965418+0000","last_active":"2024-04-26T21:20:46.962548+0000","last_peered":"2024-04-26T21:20:46.962548+0000","last_clean":"2024-04-26T21:20:46.962548+0000","last_became_active":"2024-04-26T21:20:45.965125+0000","last_became_peered":"2024-04-26T21:20:45.965125+0000","last_unstale":"2024-04-26T21:20:46.962548+0000","last_undegraded":"2024-04-26T21:20:46.962548+0000","last_fullsized":"2024-04-26T21:20:46.962548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:47.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953649+0000","last_change":"2024-04-26T21:20:44.952556+0000","last_active":"2024-04-26T21:20:44.953649+0000","last_peered":"2024-04-26T21:20:44.953649+0000","last_clean":"2024-04-26T21:20:44.953649+0000","last_became_active":"2024-04-26T21:20:44.952257+0000","last_became_peered":"2024-04-26T21:20:44.952257+0000","last_unstale":"2024-04-26T21:20:44.953649+0000","last_undegraded":"2024-04-26T21:20:44.953649+0000","last_fullsized":"2024-04-26T21:20:44.953649+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.028292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336273+0000","last_change":"2024-04-26T21:20:45.971847+0000","last_active":"2024-04-26T21:20:47.336273+0000","last_peered":"2024-04-26T21:20:47.336273+0000","last_clean":"2024-04-26T21:20:47.336273+0000","last_became_active":"2024-04-26T21:20:45.971310+0000","last_became_peered":"2024-04-26T21:20:45.971310+0000","last_unstale":"2024-04-26T21:20:47.336273+0000","last_undegraded":"2024-04-26T21:20:47.336273+0000","last_fullsized":"2024-04-26T21:20:47.336273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.464436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959325+0000","last_change":"2024-04-26T21:20:44.950519+0000","last_active":"2024-04-26T21:20:46.959325+0000","last_peered":"2024-04-26T21:20:46.959325+0000","last_clean":"2024-04-26T21:20:46.959325+0000","last_became_active":"2024-04-26T21:20:44.950288+0000","last_became_peered":"2024-04-26T21:20:44.950288+0000","last_unstale":"2024-04-26T21:20:46.959325+0000","last_undegraded":"2024-04-26T21:20:46.959325+0000","last_fullsized":"2024-04-26T21:20:46.959325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:07.328646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961844+0000","last_change":"2024-04-26T21:20:45.968413+0000","last_active":"2024-04-26T21:20:46.961844+0000","last_peered":"2024-04-26T21:20:46.961844+0000","last_clean":"2024-04-26T21:20:46.961844+0000","last_became_active":"2024-04-26T21:20:45.968149+0000","last_became_peered":"2024-04-26T21:20:45.968149+0000","last_unstale":"2024-04-26T21:20:46.961844+0000","last_undegraded":"2024-04-26T21:20:46.961844+0000","last_fullsized":"2024-04-26T21:20:46.961844+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:11.931969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962569+0000","last_change":"2024-04-26T21:20:44.952076+0000","last_active":"2024-04-26T21:20:46.962569+0000","last_peered":"2024-04-26T21:20:46.962569+0000","last_clean":"2024-04-26T21:20:46.962569+0000","last_became_active":"2024-04-26T21:20:44.950995+0000","last_became_peered":"2024-04-26T21:20:44.950995+0000","last_unstale":"2024-04-26T21:20:46.962569+0000","last_undegraded":"2024-04-26T21:20:46.962569+0000","last_fullsized":"2024-04-26T21:20:46.962569+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:49.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959745+0000","last_change":"2024-04-26T21:20:45.967568+0000","last_active":"2024-04-26T21:20:46.959745+0000","last_peered":"2024-04-26T21:20:46.959745+0000","last_clean":"2024-04-26T21:20:46.959745+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.959745+0000","last_undegraded":"2024-04-26T21:20:46.959745+0000","last_fullsized":"2024-04-26T21:20:46.959745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:52.037830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336020+0000","last_change":"2024-04-26T21:20:44.949326+0000","last_active":"2024-04-26T21:20:47.336020+0000","last_peered":"2024-04-26T21:20:47.336020+0000","last_clean":"2024-04-26T21:20:47.336020+0000","last_became_active":"2024-04-26T21:20:44.948970+0000","last_became_peered":"2024-04-26T21:20:44.948970+0000","last_unstale":"2024-04-26T21:20:47.336020+0000","last_undegraded":"2024-04-26T21:20:47.336020+0000","last_fullsized":"2024-04-26T21:20:47.336020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.645489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336403+0000","last_change":"2024-04-26T21:20:45.967295+0000","last_active":"2024-04-26T21:20:47.336403+0000","last_peered":"2024-04-26T21:20:47.336403+0000","last_clean":"2024-04-26T21:20:47.336403+0000","last_became_active":"2024-04-26T21:20:45.966533+0000","last_became_peered":"2024-04-26T21:20:45.966533+0000","last_unstale":"2024-04-26T21:20:47.336403+0000","last_undegraded":"2024-04-26T21:20:47.336403+0000","last_fullsized":"2024-04-26T21:20:47.336403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959724+0000","last_change":"2024-04-26T21:20:44.953637+0000","last_active":"2024-04-26T21:20:46.959724+0000","last_peered":"2024-04-26T21:20:46.959724+0000","last_clean":"2024-04-26T21:20:46.959724+0000","last_became_active":"2024-04-26T21:20:44.953364+0000","last_became_peered":"2024-04-26T21:20:44.953364+0000","last_unstale":"2024-04-26T21:20:46.959724+0000","last_undegraded":"2024-04-26T21:20:46.959724+0000","last_fullsized":"2024-04-26T21:20:46.959724+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:04.043064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946421+0000","last_change":"2024-04-26T21:20:44.946421+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946421+0000","last_undegraded":"2024-04-26T21:20:44.946421+0000","last_fullsized":"2024-04-26T21:20:44.946421+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336034+0000","last_change":"2024-04-26T21:20:44.950539+0000","last_active":"2024-04-26T21:20:47.336034+0000","last_peered":"2024-04-26T21:20:47.336034+0000","last_clean":"2024-04-26T21:20:47.336034+0000","last_became_active":"2024-04-26T21:20:44.950262+0000","last_became_peered":"2024-04-26T21:20:44.950262+0000","last_unstale":"2024-04-26T21:20:47.336034+0000","last_undegraded":"2024-04-26T21:20:47.336034+0000","last_fullsized":"2024-04-26T21:20:47.336034+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:19.053104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336857+0000","last_change":"2024-04-26T21:20:45.970609+0000","last_active":"2024-04-26T21:20:47.336857+0000","last_peered":"2024-04-26T21:20:47.336857+0000","last_clean":"2024-04-26T21:20:47.336857+0000","last_became_active":"2024-04-26T21:20:45.970312+0000","last_became_peered":"2024-04-26T21:20:45.970312+0000","last_unstale":"2024-04-26T21:20:47.336857+0000","last_undegraded":"2024-04-26T21:20:47.336857+0000","last_fullsized":"2024-04-26T21:20:47.336857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336100+0000","last_change":"2024-04-26T21:20:44.949930+0000","last_active":"2024-04-26T21:20:47.336100+0000","last_peered":"2024-04-26T21:20:47.336100+0000","last_clean":"2024-04-26T21:20:47.336100+0000","last_became_active":"2024-04-26T21:20:44.949603+0000","last_became_peered":"2024-04-26T21:20:44.949603+0000","last_unstale":"2024-04-26T21:20:47.336100+0000","last_undegraded":"2024-04-26T21:20:47.336100+0000","last_fullsized":"2024-04-26T21:20:47.336100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:01.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335637+0000","last_change":"2024-04-26T21:20:45.964537+0000","last_active":"2024-04-26T21:20:47.335637+0000","last_peered":"2024-04-26T21:20:47.335637+0000","last_clean":"2024-04-26T21:20:47.335637+0000","last_became_active":"2024-04-26T21:20:45.964253+0000","last_became_peered":"2024-04-26T21:20:45.964253+0000","last_unstale":"2024-04-26T21:20:47.335637+0000","last_undegraded":"2024-04-26T21:20:47.335637+0000","last_fullsized":"2024-04-26T21:20:47.335637+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961394+0000","last_change":"2024-04-26T21:20:44.958433+0000","last_active":"2024-04-26T21:20:46.961394+0000","last_peered":"2024-04-26T21:20:46.961394+0000","last_clean":"2024-04-26T21:20:46.961394+0000","last_became_active":"2024-04-26T21:20:44.958109+0000","last_became_peered":"2024-04-26T21:20:44.958109+0000","last_unstale":"2024-04-26T21:20:46.961394+0000","last_undegraded":"2024-04-26T21:20:46.961394+0000","last_fullsized":"2024-04-26T21:20:46.961394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:56.399692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335976+0000","last_change":"2024-04-26T21:20:45.963605+0000","last_active":"2024-04-26T21:20:47.335976+0000","last_peered":"2024-04-26T21:20:47.335976+0000","last_clean":"2024-04-26T21:20:47.335976+0000","last_became_active":"2024-04-26T21:20:45.963047+0000","last_became_peered":"2024-04-26T21:20:45.963047+0000","last_unstale":"2024-04-26T21:20:47.335976+0000","last_undegraded":"2024-04-26T21:20:47.335976+0000","last_fullsized":"2024-04-26T21:20:47.335976+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961119+0000","last_change":"2024-04-26T21:20:44.957474+0000","last_active":"2024-04-26T21:20:46.961119+0000","last_peered":"2024-04-26T21:20:46.961119+0000","last_clean":"2024-04-26T21:20:46.961119+0000","last_became_active":"2024-04-26T21:20:44.957134+0000","last_became_peered":"2024-04-26T21:20:44.957134+0000","last_unstale":"2024-04-26T21:20:46.961119+0000","last_undegraded":"2024-04-26T21:20:46.961119+0000","last_fullsized":"2024-04-26T21:20:46.961119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:50.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335784+0000","last_change":"2024-04-26T21:20:45.961150+0000","last_active":"2024-04-26T21:20:47.335784+0000","last_peered":"2024-04-26T21:20:47.335784+0000","last_clean":"2024-04-26T21:20:47.335784+0000","last_became_active":"2024-04-26T21:20:45.960842+0000","last_became_peered":"2024-04-26T21:20:45.960842+0000","last_unstale":"2024-04-26T21:20:47.335784+0000","last_undegraded":"2024-04-26T21:20:47.335784+0000","last_fullsized":"2024-04-26T21:20:47.335784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.951589+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.950834+0000","last_became_peered":"2024-04-26T21:20:44.950834+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:51.927771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962749+0000","last_change":"2024-04-26T21:20:45.967744+0000","last_active":"2024-04-26T21:20:46.962749+0000","last_peered":"2024-04-26T21:20:46.962749+0000","last_clean":"2024-04-26T21:20:46.962749+0000","last_became_active":"2024-04-26T21:20:45.967409+0000","last_became_peered":"2024-04-26T21:20:45.967409+0000","last_unstale":"2024-04-26T21:20:46.962749+0000","last_undegraded":"2024-04-26T21:20:46.962749+0000","last_fullsized":"2024-04-26T21:20:46.962749+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:12.684400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962057+0000","last_change":"2024-04-26T21:20:44.958845+0000","last_active":"2024-04-26T21:20:46.962057+0000","last_peered":"2024-04-26T21:20:46.962057+0000","last_clean":"2024-04-26T21:20:46.962057+0000","last_became_active":"2024-04-26T21:20:44.958476+0000","last_became_peered":"2024-04-26T21:20:44.958476+0000","last_unstale":"2024-04-26T21:20:46.962057+0000","last_undegraded":"2024-04-26T21:20:46.962057+0000","last_fullsized":"2024-04-26T21:20:46.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:53.203282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336659+0000","last_change":"2024-04-26T21:20:45.967883+0000","last_active":"2024-04-26T21:20:47.336659+0000","last_peered":"2024-04-26T21:20:47.336659+0000","last_clean":"2024-04-26T21:20:47.336659+0000","last_became_active":"2024-04-26T21:20:45.967589+0000","last_became_peered":"2024-04-26T21:20:45.967589+0000","last_unstale":"2024-04-26T21:20:47.336659+0000","last_undegraded":"2024-04-26T21:20:47.336659+0000","last_fullsized":"2024-04-26T21:20:47.336659+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.957180+0000","last_change":"2024-04-26T21:20:44.955921+0000","last_active":"2024-04-26T21:20:44.957180+0000","last_peered":"2024-04-26T21:20:44.957180+0000","last_clean":"2024-04-26T21:20:44.957180+0000","last_became_active":"2024-04-26T21:20:44.955598+0000","last_became_peered":"2024-04-26T21:20:44.955598+0000","last_unstale":"2024-04-26T21:20:44.957180+0000","last_undegraded":"2024-04-26T21:20:44.957180+0000","last_fullsized":"2024-04-26T21:20:44.957180+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:53.995022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335972+0000","last_change":"2024-04-26T21:20:45.958489+0000","last_active":"2024-04-26T21:20:47.335972+0000","last_peered":"2024-04-26T21:20:47.335972+0000","last_clean":"2024-04-26T21:20:47.335972+0000","last_became_active":"2024-04-26T21:20:45.958208+0000","last_became_peered":"2024-04-26T21:20:45.958208+0000","last_unstale":"2024-04-26T21:20:47.335972+0000","last_undegraded":"2024-04-26T21:20:47.335972+0000","last_fullsized":"2024-04-26T21:20:47.335972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:32.662963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953383+0000","last_change":"2024-04-26T21:20:44.951269+0000","last_active":"2024-04-26T21:20:44.953383+0000","last_peered":"2024-04-26T21:20:44.953383+0000","last_clean":"2024-04-26T21:20:44.953383+0000","last_became_active":"2024-04-26T21:20:44.950960+0000","last_became_peered":"2024-04-26T21:20:44.950960+0000","last_unstale":"2024-04-26T21:20:44.953383+0000","last_undegraded":"2024-04-26T21:20:44.953383+0000","last_fullsized":"2024-04-26T21:20:44.953383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:29.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945946+0000","last_change":"2024-04-26T21:20:44.945946+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945946+0000","last_undegraded":"2024-04-26T21:20:44.945946+0000","last_fullsized":"2024-04-26T21:20:44.945946+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953430+0000","last_change":"2024-04-26T21:20:44.952291+0000","last_active":"2024-04-26T21:20:44.953430+0000","last_peered":"2024-04-26T21:20:44.953430+0000","last_clean":"2024-04-26T21:20:44.953430+0000","last_became_active":"2024-04-26T21:20:44.951911+0000","last_became_peered":"2024-04-26T21:20:44.951911+0000","last_unstale":"2024-04-26T21:20:44.953430+0000","last_undegraded":"2024-04-26T21:20:44.953430+0000","last_fullsized":"2024-04-26T21:20:44.953430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:31.605201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959372+0000","last_change":"2024-04-26T21:20:45.962616+0000","last_active":"2024-04-26T21:20:46.959372+0000","last_peered":"2024-04-26T21:20:46.959372+0000","last_clean":"2024-04-26T21:20:46.959372+0000","last_became_active":"2024-04-26T21:20:45.962318+0000","last_became_peered":"2024-04-26T21:20:45.962318+0000","last_unstale":"2024-04-26T21:20:46.959372+0000","last_undegraded":"2024-04-26T21:20:46.959372+0000","last_fullsized":"2024-04-26T21:20:46.959372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:57.812828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336229+0000","last_change":"2024-04-26T21:20:44.944630+0000","last_active":"2024-04-26T21:20:47.336229+0000","last_peered":"2024-04-26T21:20:47.336229+0000","last_clean":"2024-04-26T21:20:47.336229+0000","last_became_active":"2024-04-26T21:20:44.944271+0000","last_became_peered":"2024-04-26T21:20:44.944271+0000","last_unstale":"2024-04-26T21:20:47.336229+0000","last_undegraded":"2024-04-26T21:20:47.336229+0000","last_fullsized":"2024-04-26T21:20:47.336229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:09.997574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962808+0000","last_change":"2024-04-26T21:20:45.976347+0000","last_active":"2024-04-26T21:20:46.962808+0000","last_peered":"2024-04-26T21:20:46.962808+0000","last_clean":"2024-04-26T21:20:46.962808+0000","last_became_active":"2024-04-26T21:20:45.976080+0000","last_became_peered":"2024-04-26T21:20:45.976080+0000","last_unstale":"2024-04-26T21:20:46.962808+0000","last_undegraded":"2024-04-26T21:20:46.962808+0000","last_fullsized":"2024-04-26T21:20:46.962808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:30.154836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961487+0000","last_change":"2024-04-26T21:20:44.958470+0000","last_active":"2024-04-26T21:20:46.961487+0000","last_peered":"2024-04-26T21:20:46.961487+0000","last_clean":"2024-04-26T21:20:46.961487+0000","last_became_active":"2024-04-26T21:20:44.958191+0000","last_became_peered":"2024-04-26T21:20:44.958191+0000","last_unstale":"2024-04-26T21:20:46.961487+0000","last_undegraded":"2024-04-26T21:20:46.961487+0000","last_fullsized":"2024-04-26T21:20:46.961487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:51.907338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961496+0000","last_change":"2024-04-26T21:20:45.964088+0000","last_active":"2024-04-26T21:20:46.961496+0000","last_peered":"2024-04-26T21:20:46.961496+0000","last_clean":"2024-04-26T21:20:46.961496+0000","last_became_active":"2024-04-26T21:20:45.963798+0000","last_became_peered":"2024-04-26T21:20:45.963798+0000","last_unstale":"2024-04-26T21:20:46.961496+0000","last_undegraded":"2024-04-26T21:20:46.961496+0000","last_fullsized":"2024-04-26T21:20:46.961496+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336390+0000","last_change":"2024-04-26T21:20:44.952271+0000","last_active":"2024-04-26T21:20:47.336390+0000","last_peered":"2024-04-26T21:20:47.336390+0000","last_clean":"2024-04-26T21:20:47.336390+0000","last_became_active":"2024-04-26T21:20:44.951860+0000","last_became_peered":"2024-04-26T21:20:44.951860+0000","last_unstale":"2024-04-26T21:20:47.336390+0000","last_undegraded":"2024-04-26T21:20:47.336390+0000","last_fullsized":"2024-04-26T21:20:47.336390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962494+0000","last_change":"2024-04-26T21:20:45.969084+0000","last_active":"2024-04-26T21:20:46.962494+0000","last_peered":"2024-04-26T21:20:46.962494+0000","last_clean":"2024-04-26T21:20:46.962494+0000","last_became_active":"2024-04-26T21:20:45.968806+0000","last_became_peered":"2024-04-26T21:20:45.968806+0000","last_unstale":"2024-04-26T21:20:46.962494+0000","last_undegraded":"2024-04-26T21:20:46.962494+0000","last_fullsized":"2024-04-26T21:20:46.962494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:20.402971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335712+0000","last_change":"2024-04-26T21:20:44.955072+0000","last_active":"2024-04-26T21:20:47.335712+0000","last_peered":"2024-04-26T21:20:47.335712+0000","last_clean":"2024-04-26T21:20:47.335712+0000","last_became_active":"2024-04-26T21:20:44.954230+0000","last_became_peered":"2024-04-26T21:20:44.954230+0000","last_unstale":"2024-04-26T21:20:47.335712+0000","last_undegraded":"2024-04-26T21:20:47.335712+0000","last_fullsized":"2024-04-26T21:20:47.335712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:04.483717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336287+0000","last_change":"2024-04-26T21:20:45.964348+0000","last_active":"2024-04-26T21:20:47.336287+0000","last_peered":"2024-04-26T21:20:47.336287+0000","last_clean":"2024-04-26T21:20:47.336287+0000","last_became_active":"2024-04-26T21:20:45.964043+0000","last_became_peered":"2024-04-26T21:20:45.964043+0000","last_unstale":"2024-04-26T21:20:47.336287+0000","last_undegraded":"2024-04-26T21:20:47.336287+0000","last_fullsized":"2024-04-26T21:20:47.336287+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336648+0000","last_change":"2024-04-26T21:20:44.955598+0000","last_active":"2024-04-26T21:20:47.336648+0000","last_peered":"2024-04-26T21:20:47.336648+0000","last_clean":"2024-04-26T21:20:47.336648+0000","last_became_active":"2024-04-26T21:20:44.955310+0000","last_became_peered":"2024-04-26T21:20:44.955310+0000","last_unstale":"2024-04-26T21:20:47.336648+0000","last_undegraded":"2024-04-26T21:20:47.336648+0000","last_fullsized":"2024-04-26T21:20:47.336648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961583+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:46.961583+0000","last_peered":"2024-04-26T21:20:46.961583+0000","last_clean":"2024-04-26T21:20:46.961583+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:46.961583+0000","last_undegraded":"2024-04-26T21:20:46.961583+0000","last_fullsized":"2024-04-26T21:20:46.961583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2273280,"data_stored":1311247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1683456,"data_stored":720373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1671168,"data_stored":720227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":714017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33416,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338424,"statfs":{"total":96636764160,"available":96602546176,"internally_reserved":0,"allocated":2281472,"data_stored":1312241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36940,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334900,"statfs":{"total":96636764160,"available":96598937600,"internally_reserved":0,"allocated":1634304,"data_stored":700877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:49.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:50.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.115+0000 7f6ddb074640 1 Processor -- start 2024-04-26T21:20:50.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.115+0000 7f6ddb074640 1 -- start start 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6ddb074640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40d11b0 0x7f6dd406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6ddb074640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6dd406e2c0 0x7f6dd415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6ddb074640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd40d1740 con 0x7f6dd40cf2e0 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6ddb074640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd406e710 con 0x7f6dd40d11b0 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6ddb074640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd40d1a20 con 0x7f6dd406e2c0 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40d11b0 0x7f6dd406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40d11b0 0x7f6dd406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35188/0 (socket says 172.21.15.40:35188) 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 -- 172.21.15.40:0/4179886931 learned_addr learned my addr 172.21.15.40:0/4179886931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:50.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 -- 172.21.15.40:0/4179886931 >> v2:172.21.15.184:3300/0 conn(0x7f6dd406e2c0 msgr2=0x7f6dd415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 --2- 172.21.15.40:0/4179886931 >> v2:172.21.15.184:3300/0 conn(0x7f6dd406e2c0 0x7f6dd415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 -- 172.21.15.40:0/4179886931 >> v1:172.21.15.184:6789/0 conn(0x7f6dd40cf2e0 legacy=0x7f6dd406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.116+0000 7f6dcbfff640 1 -- 172.21.15.40:0/4179886931 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd40d1d00 con 0x7f6dd40d11b0 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6dcbfff640 1 --2- 172.21.15.40:0/4179886931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d11b0 0x7f6dd406dd80 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31d687492f73837 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6dcb7fe640 1 -- 172.21.15.40:0/4179886931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037070 con 0x7f6dd40d11b0 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6dcb7fe640 1 -- 172.21.15.40:0/4179886931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4032440 con 0x7f6dd40d11b0 2024-04-26T21:20:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6dcb7fe640 1 -- 172.21.15.40:0/4179886931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4032810 con 0x7f6dd40d11b0 2024-04-26T21:20:50.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6ddb074640 1 -- 172.21.15.40:0/4179886931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d11b0 msgr2=0x7f6dd406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.117+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4179886931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d11b0 0x7f6dd406dd80 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a270 comp rx=0 tx=0).stop 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 -- 172.21.15.40:0/4179886931 shutdown_connections 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4179886931 >> v2:172.21.15.184:3300/0 conn(0x7f6dd406e2c0 0x7f6dd415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4179886931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d11b0 0x7f6dd406dd80 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 -- 172.21.15.40:0/4179886931 >> 172.21.15.40:0/4179886931 conn(0x7f6dd40b4a60 msgr2=0x7f6dd40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 -- 172.21.15.40:0/4179886931 shutdown_connections 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.118+0000 7f6ddb074640 1 -- 172.21.15.40:0/4179886931 wait complete. 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.119+0000 7f6ddb074640 1 Processor -- start 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.119+0000 7f6ddb074640 1 -- start start 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40cf2e0 0x7f6dd40b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd40d11b0 0x7f6dd40b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40cf6c0 con 0x7f6dd406e2c0 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40d22c0 con 0x7f6dd40cf2e0 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd40cf840 con 0x7f6dd40d11b0 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35198/0 (socket says 172.21.15.40:35198) 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 -- 172.21.15.40:0/4182925268 learned_addr learned my addr 172.21.15.40:0/4182925268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dcbfff640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40cf2e0 0x7f6dd40b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 -- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd40d11b0 msgr2=0x7f6dd40b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd40d11b0 0x7f6dd40b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 -- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40cf2e0 msgr2=0x7f6dd40b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40cf2e0 0x7f6dd40b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 -- 172.21.15.40:0/4182925268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd41ea780 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dd8e10640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b3e0 tx=0x7f6dd000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0015020 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd0011850 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41eaa60 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011c40 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.120+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd41eb300 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.121+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd406a290 con 0x7f6dd406e2c0 2024-04-26T21:20:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.122+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dd0018020 con 0x7f6dd406e2c0 2024-04-26T21:20:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.122+0000 7f6dc97fa640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 0x7f6dc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.122+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f6dd008ba60 con 0x7f6dd406e2c0 2024-04-26T21:20:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.123+0000 7f6dcbfff640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 0x7f6dc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.123+0000 7f6dcbfff640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 0x7f6dc006ede0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4034000 tx=0x7f6dc402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.127+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dd001f020 con 0x7f6dd406e2c0 2024-04-26T21:20:50.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.308+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6dd40b8ef0 con 0x7f6dc006c920 2024-04-26T21:20:50.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.315+0000 7f6dc97fa640 1 -- 172.21.15.40:0/4182925268 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7f6dd40b8ef0 con 0x7f6dc006c920 2024-04-26T21:20:50.318 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:50.319 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.317+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 msgr2=0x7f6dc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.317+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 0x7f6dc006ede0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4034000 tx=0x7f6dc402e000 comp rx=0 tx=0).stop 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.317+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 msgr2=0x7f6dd41e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.317+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b3e0 tx=0x7f6dd000b8a0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 shutdown_connections 2024-04-26T21:20:50.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c920 0x7f6dc006ede0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd40d11b0 0x7f6dd40b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40cf2e0 0x7f6dd40b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 --2- 172.21.15.40:0/4182925268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd406e2c0 0x7f6dd41e9da0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 >> 172.21.15.40:0/4182925268 conn(0x7f6dd40b4a60 msgr2=0x7f6dd40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 shutdown_connections 2024-04-26T21:20:50.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.318+0000 7f6ddb074640 1 -- 172.21.15.40:0/4182925268 wait complete. 2024-04-26T21:20:50.329 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T21:20:49.045926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272044,"kb_used_data":14228,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754702676,"statfs":{"total":773094113280,"available":772815540224,"internally_reserved":0,"allocated":14569472,"data_stored":6921140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114211"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945782+0000","last_change":"2024-04-26T21:20:44.945782+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945782+0000","last_undegraded":"2024-04-26T21:20:44.945782+0000","last_fullsized":"2024-04-26T21:20:44.945782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336059+0000","last_change":"2024-04-26T21:20:45.965877+0000","last_active":"2024-04-26T21:20:47.336059+0000","last_peered":"2024-04-26T21:20:47.336059+0000","last_clean":"2024-04-26T21:20:47.336059+0000","last_became_active":"2024-04-26T21:20:45.965531+0000","last_became_peered":"2024-04-26T21:20:45.965531+0000","last_unstale":"2024-04-26T21:20:47.336059+0000","last_undegraded":"2024-04-26T21:20:47.336059+0000","last_fullsized":"2024-04-26T21:20:47.336059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:02.567341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336527+0000","last_change":"2024-04-26T21:20:45.972531+0000","last_active":"2024-04-26T21:20:47.336527+0000","last_peered":"2024-04-26T21:20:47.336527+0000","last_clean":"2024-04-26T21:20:47.336527+0000","last_became_active":"2024-04-26T21:20:45.972241+0000","last_became_peered":"2024-04-26T21:20:45.972241+0000","last_unstale":"2024-04-26T21:20:47.336527+0000","last_undegraded":"2024-04-26T21:20:47.336527+0000","last_fullsized":"2024-04-26T21:20:47.336527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336969+0000","last_change":"2024-04-26T21:20:45.973682+0000","last_active":"2024-04-26T21:20:47.336969+0000","last_peered":"2024-04-26T21:20:47.336969+0000","last_clean":"2024-04-26T21:20:47.336969+0000","last_became_active":"2024-04-26T21:20:45.973381+0000","last_became_peered":"2024-04-26T21:20:45.973381+0000","last_unstale":"2024-04-26T21:20:47.336969+0000","last_undegraded":"2024-04-26T21:20:47.336969+0000","last_fullsized":"2024-04-26T21:20:47.336969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:34.199283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962690+0000","last_change":"2024-04-26T21:20:45.967509+0000","last_active":"2024-04-26T21:20:46.962690+0000","last_peered":"2024-04-26T21:20:46.962690+0000","last_clean":"2024-04-26T21:20:46.962690+0000","last_became_active":"2024-04-26T21:20:45.967269+0000","last_became_peered":"2024-04-26T21:20:45.967269+0000","last_unstale":"2024-04-26T21:20:46.962690+0000","last_undegraded":"2024-04-26T21:20:46.962690+0000","last_fullsized":"2024-04-26T21:20:46.962690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.129198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943159+0000","last_change":"2024-04-26T21:20:44.943159+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943159+0000","last_undegraded":"2024-04-26T21:20:44.943159+0000","last_fullsized":"2024-04-26T21:20:44.943159+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335603+0000","last_change":"2024-04-26T21:20:45.963229+0000","last_active":"2024-04-26T21:20:47.335603+0000","last_peered":"2024-04-26T21:20:47.335603+0000","last_clean":"2024-04-26T21:20:47.335603+0000","last_became_active":"2024-04-26T21:20:45.962872+0000","last_became_peered":"2024-04-26T21:20:45.962872+0000","last_unstale":"2024-04-26T21:20:47.335603+0000","last_undegraded":"2024-04-26T21:20:47.335603+0000","last_fullsized":"2024-04-26T21:20:47.335603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942563+0000","last_change":"2024-04-26T21:20:44.942563+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942563+0000","last_undegraded":"2024-04-26T21:20:44.942563+0000","last_fullsized":"2024-04-26T21:20:44.942563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336113+0000","last_change":"2024-04-26T21:20:45.968152+0000","last_active":"2024-04-26T21:20:47.336113+0000","last_peered":"2024-04-26T21:20:47.336113+0000","last_clean":"2024-04-26T21:20:47.336113+0000","last_became_active":"2024-04-26T21:20:45.967845+0000","last_became_peered":"2024-04-26T21:20:45.967845+0000","last_unstale":"2024-04-26T21:20:47.336113+0000","last_undegraded":"2024-04-26T21:20:47.336113+0000","last_fullsized":"2024-04-26T21:20:47.336113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.001965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961661+0000","last_change":"2024-04-26T21:20:45.977906+0000","last_active":"2024-04-26T21:20:46.961661+0000","last_peered":"2024-04-26T21:20:46.961661+0000","last_clean":"2024-04-26T21:20:46.961661+0000","last_became_active":"2024-04-26T21:20:45.977567+0000","last_became_peered":"2024-04-26T21:20:45.977567+0000","last_unstale":"2024-04-26T21:20:46.961661+0000","last_undegraded":"2024-04-26T21:20:46.961661+0000","last_fullsized":"2024-04-26T21:20:46.961661+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946943+0000","last_change":"2024-04-26T21:20:44.946943+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946943+0000","last_undegraded":"2024-04-26T21:20:44.946943+0000","last_fullsized":"2024-04-26T21:20:44.946943+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961702+0000","last_change":"2024-04-26T21:20:45.972783+0000","last_active":"2024-04-26T21:20:46.961702+0000","last_peered":"2024-04-26T21:20:46.961702+0000","last_clean":"2024-04-26T21:20:46.961702+0000","last_became_active":"2024-04-26T21:20:45.972522+0000","last_became_peered":"2024-04-26T21:20:45.972522+0000","last_unstale":"2024-04-26T21:20:46.961702+0000","last_undegraded":"2024-04-26T21:20:46.961702+0000","last_fullsized":"2024-04-26T21:20:46.961702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962682+0000","last_change":"2024-04-26T21:20:45.967507+0000","last_active":"2024-04-26T21:20:46.962682+0000","last_peered":"2024-04-26T21:20:46.962682+0000","last_clean":"2024-04-26T21:20:46.962682+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.962682+0000","last_undegraded":"2024-04-26T21:20:46.962682+0000","last_fullsized":"2024-04-26T21:20:46.962682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:40.380793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:45.967357+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:45.966578+0000","last_became_peered":"2024-04-26T21:20:45.966578+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961095+0000","last_change":"2024-04-26T21:20:45.974674+0000","last_active":"2024-04-26T21:20:46.961095+0000","last_peered":"2024-04-26T21:20:46.961095+0000","last_clean":"2024-04-26T21:20:46.961095+0000","last_became_active":"2024-04-26T21:20:45.974359+0000","last_became_peered":"2024-04-26T21:20:45.974359+0000","last_unstale":"2024-04-26T21:20:46.961095+0000","last_undegraded":"2024-04-26T21:20:46.961095+0000","last_fullsized":"2024-04-26T21:20:46.961095+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:49.051775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959220+0000","last_change":"2024-04-26T21:20:45.968322+0000","last_active":"2024-04-26T21:20:46.959220+0000","last_peered":"2024-04-26T21:20:46.959220+0000","last_clean":"2024-04-26T21:20:46.959220+0000","last_became_active":"2024-04-26T21:20:45.967746+0000","last_became_peered":"2024-04-26T21:20:45.967746+0000","last_unstale":"2024-04-26T21:20:46.959220+0000","last_undegraded":"2024-04-26T21:20:46.959220+0000","last_fullsized":"2024-04-26T21:20:46.959220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.881502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962071+0000","last_change":"2024-04-26T21:20:45.979392+0000","last_active":"2024-04-26T21:20:46.962071+0000","last_peered":"2024-04-26T21:20:46.962071+0000","last_clean":"2024-04-26T21:20:46.962071+0000","last_became_active":"2024-04-26T21:20:45.978894+0000","last_became_peered":"2024-04-26T21:20:45.978894+0000","last_unstale":"2024-04-26T21:20:46.962071+0000","last_undegraded":"2024-04-26T21:20:46.962071+0000","last_fullsized":"2024-04-26T21:20:46.962071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:04.038090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336798+0000","last_change":"2024-04-26T21:20:45.971458+0000","last_active":"2024-04-26T21:20:47.336798+0000","last_peered":"2024-04-26T21:20:47.336798+0000","last_clean":"2024-04-26T21:20:47.336798+0000","last_became_active":"2024-04-26T21:20:45.971186+0000","last_became_peered":"2024-04-26T21:20:45.971186+0000","last_unstale":"2024-04-26T21:20:47.336798+0000","last_undegraded":"2024-04-26T21:20:47.336798+0000","last_fullsized":"2024-04-26T21:20:47.336798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945523+0000","last_change":"2024-04-26T21:20:44.945523+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945523+0000","last_undegraded":"2024-04-26T21:20:44.945523+0000","last_fullsized":"2024-04-26T21:20:44.945523+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959492+0000","last_change":"2024-04-26T21:20:45.968075+0000","last_active":"2024-04-26T21:20:46.959492+0000","last_peered":"2024-04-26T21:20:46.959492+0000","last_clean":"2024-04-26T21:20:46.959492+0000","last_became_active":"2024-04-26T21:20:45.967734+0000","last_became_peered":"2024-04-26T21:20:45.967734+0000","last_unstale":"2024-04-26T21:20:46.959492+0000","last_undegraded":"2024-04-26T21:20:46.959492+0000","last_fullsized":"2024-04-26T21:20:46.959492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.314546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961395+0000","last_change":"2024-04-26T21:20:45.977923+0000","last_active":"2024-04-26T21:20:46.961395+0000","last_peered":"2024-04-26T21:20:46.961395+0000","last_clean":"2024-04-26T21:20:46.961395+0000","last_became_active":"2024-04-26T21:20:45.977624+0000","last_became_peered":"2024-04-26T21:20:45.977624+0000","last_unstale":"2024-04-26T21:20:46.961395+0000","last_undegraded":"2024-04-26T21:20:46.961395+0000","last_fullsized":"2024-04-26T21:20:46.961395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336657+0000","last_change":"2024-04-26T21:20:45.956281+0000","last_active":"2024-04-26T21:20:47.336657+0000","last_peered":"2024-04-26T21:20:47.336657+0000","last_clean":"2024-04-26T21:20:47.336657+0000","last_became_active":"2024-04-26T21:20:45.955972+0000","last_became_peered":"2024-04-26T21:20:45.955972+0000","last_unstale":"2024-04-26T21:20:47.336657+0000","last_undegraded":"2024-04-26T21:20:47.336657+0000","last_fullsized":"2024-04-26T21:20:47.336657+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336022+0000","last_change":"2024-04-26T21:20:45.967466+0000","last_active":"2024-04-26T21:20:47.336022+0000","last_peered":"2024-04-26T21:20:47.336022+0000","last_clean":"2024-04-26T21:20:47.336022+0000","last_became_active":"2024-04-26T21:20:45.967180+0000","last_became_peered":"2024-04-26T21:20:45.967180+0000","last_unstale":"2024-04-26T21:20:47.336022+0000","last_undegraded":"2024-04-26T21:20:47.336022+0000","last_fullsized":"2024-04-26T21:20:47.336022+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961176+0000","last_change":"2024-04-26T21:20:45.966830+0000","last_active":"2024-04-26T21:20:46.961176+0000","last_peered":"2024-04-26T21:20:46.961176+0000","last_clean":"2024-04-26T21:20:46.961176+0000","last_became_active":"2024-04-26T21:20:45.966121+0000","last_became_peered":"2024-04-26T21:20:45.966121+0000","last_unstale":"2024-04-26T21:20:46.961176+0000","last_undegraded":"2024-04-26T21:20:46.961176+0000","last_fullsized":"2024-04-26T21:20:46.961176+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:50.753188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959755+0000","last_change":"2024-04-26T21:20:45.968233+0000","last_active":"2024-04-26T21:20:46.959755+0000","last_peered":"2024-04-26T21:20:46.959755+0000","last_clean":"2024-04-26T21:20:46.959755+0000","last_became_active":"2024-04-26T21:20:45.967940+0000","last_became_peered":"2024-04-26T21:20:45.967940+0000","last_unstale":"2024-04-26T21:20:46.959755+0000","last_undegraded":"2024-04-26T21:20:46.959755+0000","last_fullsized":"2024-04-26T21:20:46.959755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:34.779995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.944578+0000","last_change":"2024-04-26T21:20:44.944578+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.944578+0000","last_undegraded":"2024-04-26T21:20:44.944578+0000","last_fullsized":"2024-04-26T21:20:44.944578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959794+0000","last_change":"2024-04-26T21:20:45.973518+0000","last_active":"2024-04-26T21:20:46.959794+0000","last_peered":"2024-04-26T21:20:46.959794+0000","last_clean":"2024-04-26T21:20:46.959794+0000","last_became_active":"2024-04-26T21:20:45.973227+0000","last_became_peered":"2024-04-26T21:20:45.973227+0000","last_unstale":"2024-04-26T21:20:46.959794+0000","last_undegraded":"2024-04-26T21:20:46.959794+0000","last_fullsized":"2024-04-26T21:20:46.959794+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:12.310488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336254+0000","last_change":"2024-04-26T21:20:45.973958+0000","last_active":"2024-04-26T21:20:47.336254+0000","last_peered":"2024-04-26T21:20:47.336254+0000","last_clean":"2024-04-26T21:20:47.336254+0000","last_became_active":"2024-04-26T21:20:45.973675+0000","last_became_peered":"2024-04-26T21:20:45.973675+0000","last_unstale":"2024-04-26T21:20:47.336254+0000","last_undegraded":"2024-04-26T21:20:47.336254+0000","last_fullsized":"2024-04-26T21:20:47.336254+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943772+0000","last_change":"2024-04-26T21:20:44.943772+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943772+0000","last_undegraded":"2024-04-26T21:20:44.943772+0000","last_fullsized":"2024-04-26T21:20:44.943772+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336855+0000","last_change":"2024-04-26T21:20:45.970795+0000","last_active":"2024-04-26T21:20:47.336855+0000","last_peered":"2024-04-26T21:20:47.336855+0000","last_clean":"2024-04-26T21:20:47.336855+0000","last_became_active":"2024-04-26T21:20:45.970509+0000","last_became_peered":"2024-04-26T21:20:45.970509+0000","last_unstale":"2024-04-26T21:20:47.336855+0000","last_undegraded":"2024-04-26T21:20:47.336855+0000","last_fullsized":"2024-04-26T21:20:47.336855+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:30.015334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335701+0000","last_change":"2024-04-26T21:20:45.972356+0000","last_active":"2024-04-26T21:20:47.335701+0000","last_peered":"2024-04-26T21:20:47.335701+0000","last_clean":"2024-04-26T21:20:47.335701+0000","last_became_active":"2024-04-26T21:20:45.972064+0000","last_became_peered":"2024-04-26T21:20:45.972064+0000","last_unstale":"2024-04-26T21:20:47.335701+0000","last_undegraded":"2024-04-26T21:20:47.335701+0000","last_fullsized":"2024-04-26T21:20:47.335701+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:50.058209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959285+0000","last_change":"2024-04-26T21:20:45.968435+0000","last_active":"2024-04-26T21:20:46.959285+0000","last_peered":"2024-04-26T21:20:46.959285+0000","last_clean":"2024-04-26T21:20:46.959285+0000","last_became_active":"2024-04-26T21:20:45.966848+0000","last_became_peered":"2024-04-26T21:20:45.966848+0000","last_unstale":"2024-04-26T21:20:46.959285+0000","last_undegraded":"2024-04-26T21:20:46.959285+0000","last_fullsized":"2024-04-26T21:20:46.959285+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:50.873852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961713+0000","last_change":"2024-04-26T21:20:45.973734+0000","last_active":"2024-04-26T21:20:46.961713+0000","last_peered":"2024-04-26T21:20:46.961713+0000","last_clean":"2024-04-26T21:20:46.961713+0000","last_became_active":"2024-04-26T21:20:45.973472+0000","last_became_peered":"2024-04-26T21:20:45.973472+0000","last_unstale":"2024-04-26T21:20:46.961713+0000","last_undegraded":"2024-04-26T21:20:46.961713+0000","last_fullsized":"2024-04-26T21:20:46.961713+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961961+0000","last_change":"2024-04-26T21:20:44.958400+0000","last_active":"2024-04-26T21:20:46.961961+0000","last_peered":"2024-04-26T21:20:46.961961+0000","last_clean":"2024-04-26T21:20:46.961961+0000","last_became_active":"2024-04-26T21:20:44.958081+0000","last_became_peered":"2024-04-26T21:20:44.958081+0000","last_unstale":"2024-04-26T21:20:46.961961+0000","last_undegraded":"2024-04-26T21:20:46.961961+0000","last_fullsized":"2024-04-26T21:20:46.961961+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:38.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336119+0000","last_change":"2024-04-26T21:20:45.964932+0000","last_active":"2024-04-26T21:20:47.336119+0000","last_peered":"2024-04-26T21:20:47.336119+0000","last_clean":"2024-04-26T21:20:47.336119+0000","last_became_active":"2024-04-26T21:20:45.961496+0000","last_became_peered":"2024-04-26T21:20:45.961496+0000","last_unstale":"2024-04-26T21:20:47.336119+0000","last_undegraded":"2024-04-26T21:20:47.336119+0000","last_fullsized":"2024-04-26T21:20:47.336119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.885551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335890+0000","last_change":"2024-04-26T21:20:44.956115+0000","last_active":"2024-04-26T21:20:47.335890+0000","last_peered":"2024-04-26T21:20:47.335890+0000","last_clean":"2024-04-26T21:20:47.335890+0000","last_became_active":"2024-04-26T21:20:44.955813+0000","last_became_peered":"2024-04-26T21:20:44.955813+0000","last_unstale":"2024-04-26T21:20:47.335890+0000","last_undegraded":"2024-04-26T21:20:47.335890+0000","last_fullsized":"2024-04-26T21:20:47.335890+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:21.778534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942776+0000","last_change":"2024-04-26T21:20:44.942776+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942776+0000","last_undegraded":"2024-04-26T21:20:44.942776+0000","last_fullsized":"2024-04-26T21:20:44.942776+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.949711+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.949393+0000","last_became_peered":"2024-04-26T21:20:44.949393+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961372+0000","last_change":"2024-04-26T21:20:45.964911+0000","last_active":"2024-04-26T21:20:46.961372+0000","last_peered":"2024-04-26T21:20:46.961372+0000","last_clean":"2024-04-26T21:20:46.961372+0000","last_became_active":"2024-04-26T21:20:45.963945+0000","last_became_peered":"2024-04-26T21:20:45.963945+0000","last_unstale":"2024-04-26T21:20:46.961372+0000","last_undegraded":"2024-04-26T21:20:46.961372+0000","last_fullsized":"2024-04-26T21:20:46.961372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:41.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.337000+0000","last_change":"2024-04-26T21:20:44.949483+0000","last_active":"2024-04-26T21:20:47.337000+0000","last_peered":"2024-04-26T21:20:47.337000+0000","last_clean":"2024-04-26T21:20:47.337000+0000","last_became_active":"2024-04-26T21:20:44.948183+0000","last_became_peered":"2024-04-26T21:20:44.948183+0000","last_unstale":"2024-04-26T21:20:47.337000+0000","last_undegraded":"2024-04-26T21:20:47.337000+0000","last_fullsized":"2024-04-26T21:20:47.337000+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335767+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:47.335767+0000","last_peered":"2024-04-26T21:20:47.335767+0000","last_clean":"2024-04-26T21:20:47.335767+0000","last_became_active":"2024-04-26T21:20:45.973350+0000","last_became_peered":"2024-04-26T21:20:45.973350+0000","last_unstale":"2024-04-26T21:20:47.335767+0000","last_undegraded":"2024-04-26T21:20:47.335767+0000","last_fullsized":"2024-04-26T21:20:47.335767+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.225182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959303+0000","last_change":"2024-04-26T21:20:44.954463+0000","last_active":"2024-04-26T21:20:46.959303+0000","last_peered":"2024-04-26T21:20:46.959303+0000","last_clean":"2024-04-26T21:20:46.959303+0000","last_became_active":"2024-04-26T21:20:44.954133+0000","last_became_peered":"2024-04-26T21:20:44.954133+0000","last_unstale":"2024-04-26T21:20:46.959303+0000","last_undegraded":"2024-04-26T21:20:46.959303+0000","last_fullsized":"2024-04-26T21:20:46.959303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:16.181685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336085+0000","last_change":"2024-04-26T21:20:45.978602+0000","last_active":"2024-04-26T21:20:47.336085+0000","last_peered":"2024-04-26T21:20:47.336085+0000","last_clean":"2024-04-26T21:20:47.336085+0000","last_became_active":"2024-04-26T21:20:45.978300+0000","last_became_peered":"2024-04-26T21:20:45.978300+0000","last_unstale":"2024-04-26T21:20:47.336085+0000","last_undegraded":"2024-04-26T21:20:47.336085+0000","last_fullsized":"2024-04-26T21:20:47.336085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962713+0000","last_change":"2024-04-26T21:20:44.953754+0000","last_active":"2024-04-26T21:20:46.962713+0000","last_peered":"2024-04-26T21:20:46.962713+0000","last_clean":"2024-04-26T21:20:46.962713+0000","last_became_active":"2024-04-26T21:20:44.953471+0000","last_became_peered":"2024-04-26T21:20:44.953471+0000","last_unstale":"2024-04-26T21:20:46.962713+0000","last_undegraded":"2024-04-26T21:20:46.962713+0000","last_fullsized":"2024-04-26T21:20:46.962713+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:21.877111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335605+0000","last_change":"2024-04-26T21:20:45.957101+0000","last_active":"2024-04-26T21:20:47.335605+0000","last_peered":"2024-04-26T21:20:47.335605+0000","last_clean":"2024-04-26T21:20:47.335605+0000","last_became_active":"2024-04-26T21:20:45.956746+0000","last_became_peered":"2024-04-26T21:20:45.956746+0000","last_unstale":"2024-04-26T21:20:47.335605+0000","last_undegraded":"2024-04-26T21:20:47.335605+0000","last_fullsized":"2024-04-26T21:20:47.335605+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:20.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959438+0000","last_change":"2024-04-26T21:20:44.945604+0000","last_active":"2024-04-26T21:20:46.959438+0000","last_peered":"2024-04-26T21:20:46.959438+0000","last_clean":"2024-04-26T21:20:46.959438+0000","last_became_active":"2024-04-26T21:20:44.945311+0000","last_became_peered":"2024-04-26T21:20:44.945311+0000","last_unstale":"2024-04-26T21:20:46.959438+0000","last_undegraded":"2024-04-26T21:20:46.959438+0000","last_fullsized":"2024-04-26T21:20:46.959438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:59.997109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962422+0000","last_change":"2024-04-26T21:20:45.968415+0000","last_active":"2024-04-26T21:20:46.962422+0000","last_peered":"2024-04-26T21:20:46.962422+0000","last_clean":"2024-04-26T21:20:46.962422+0000","last_became_active":"2024-04-26T21:20:45.968148+0000","last_became_peered":"2024-04-26T21:20:45.968148+0000","last_unstale":"2024-04-26T21:20:46.962422+0000","last_undegraded":"2024-04-26T21:20:46.962422+0000","last_fullsized":"2024-04-26T21:20:46.962422+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:30.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335772+0000","last_change":"2024-04-26T21:20:44.953539+0000","last_active":"2024-04-26T21:20:47.335772+0000","last_peered":"2024-04-26T21:20:47.335772+0000","last_clean":"2024-04-26T21:20:47.335772+0000","last_became_active":"2024-04-26T21:20:44.953286+0000","last_became_peered":"2024-04-26T21:20:44.953286+0000","last_unstale":"2024-04-26T21:20:47.335772+0000","last_undegraded":"2024-04-26T21:20:47.335772+0000","last_fullsized":"2024-04-26T21:20:47.335772+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962742+0000","last_change":"2024-04-26T21:20:45.957232+0000","last_active":"2024-04-26T21:20:46.962742+0000","last_peered":"2024-04-26T21:20:46.962742+0000","last_clean":"2024-04-26T21:20:46.962742+0000","last_became_active":"2024-04-26T21:20:45.956611+0000","last_became_peered":"2024-04-26T21:20:45.956611+0000","last_unstale":"2024-04-26T21:20:46.962742+0000","last_undegraded":"2024-04-26T21:20:46.962742+0000","last_fullsized":"2024-04-26T21:20:46.962742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:13.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336218+0000","last_change":"2024-04-26T21:20:44.952646+0000","last_active":"2024-04-26T21:20:47.336218+0000","last_peered":"2024-04-26T21:20:47.336218+0000","last_clean":"2024-04-26T21:20:47.336218+0000","last_became_active":"2024-04-26T21:20:44.952351+0000","last_became_peered":"2024-04-26T21:20:44.952351+0000","last_unstale":"2024-04-26T21:20:47.336218+0000","last_undegraded":"2024-04-26T21:20:47.336218+0000","last_fullsized":"2024-04-26T21:20:47.336218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:33.194892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961915+0000","last_change":"2024-04-26T21:20:45.979158+0000","last_active":"2024-04-26T21:20:46.961915+0000","last_peered":"2024-04-26T21:20:46.961915+0000","last_clean":"2024-04-26T21:20:46.961915+0000","last_became_active":"2024-04-26T21:20:45.978509+0000","last_became_peered":"2024-04-26T21:20:45.978509+0000","last_unstale":"2024-04-26T21:20:46.961915+0000","last_undegraded":"2024-04-26T21:20:46.961915+0000","last_fullsized":"2024-04-26T21:20:46.961915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:18.599945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961941+0000","last_change":"2024-04-26T21:20:44.958683+0000","last_active":"2024-04-26T21:20:46.961941+0000","last_peered":"2024-04-26T21:20:46.961941+0000","last_clean":"2024-04-26T21:20:46.961941+0000","last_became_active":"2024-04-26T21:20:44.958250+0000","last_became_peered":"2024-04-26T21:20:44.958250+0000","last_unstale":"2024-04-26T21:20:46.961941+0000","last_undegraded":"2024-04-26T21:20:46.961941+0000","last_fullsized":"2024-04-26T21:20:46.961941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:19.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959625+0000","last_change":"2024-04-26T21:20:45.968143+0000","last_active":"2024-04-26T21:20:46.959625+0000","last_peered":"2024-04-26T21:20:46.959625+0000","last_clean":"2024-04-26T21:20:46.959625+0000","last_became_active":"2024-04-26T21:20:45.966158+0000","last_became_peered":"2024-04-26T21:20:45.966158+0000","last_unstale":"2024-04-26T21:20:46.959625+0000","last_undegraded":"2024-04-26T21:20:46.959625+0000","last_fullsized":"2024-04-26T21:20:46.959625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:10.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336377+0000","last_change":"2024-04-26T21:20:44.951683+0000","last_active":"2024-04-26T21:20:47.336377+0000","last_peered":"2024-04-26T21:20:47.336377+0000","last_clean":"2024-04-26T21:20:47.336377+0000","last_became_active":"2024-04-26T21:20:44.951213+0000","last_became_peered":"2024-04-26T21:20:44.951213+0000","last_unstale":"2024-04-26T21:20:47.336377+0000","last_undegraded":"2024-04-26T21:20:47.336377+0000","last_fullsized":"2024-04-26T21:20:47.336377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:36.881160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336537+0000","last_change":"2024-04-26T21:20:45.968505+0000","last_active":"2024-04-26T21:20:47.336537+0000","last_peered":"2024-04-26T21:20:47.336537+0000","last_clean":"2024-04-26T21:20:47.336537+0000","last_became_active":"2024-04-26T21:20:45.968185+0000","last_became_peered":"2024-04-26T21:20:45.968185+0000","last_unstale":"2024-04-26T21:20:47.336537+0000","last_undegraded":"2024-04-26T21:20:47.336537+0000","last_fullsized":"2024-04-26T21:20:47.336537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336545+0000","last_change":"2024-04-26T21:20:44.951050+0000","last_active":"2024-04-26T21:20:47.336545+0000","last_peered":"2024-04-26T21:20:47.336545+0000","last_clean":"2024-04-26T21:20:47.336545+0000","last_became_active":"2024-04-26T21:20:44.950758+0000","last_became_peered":"2024-04-26T21:20:44.950758+0000","last_unstale":"2024-04-26T21:20:47.336545+0000","last_undegraded":"2024-04-26T21:20:47.336545+0000","last_fullsized":"2024-04-26T21:20:47.336545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:07.338750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335606+0000","last_change":"2024-04-26T21:20:45.965874+0000","last_active":"2024-04-26T21:20:47.335606+0000","last_peered":"2024-04-26T21:20:47.335606+0000","last_clean":"2024-04-26T21:20:47.335606+0000","last_became_active":"2024-04-26T21:20:45.965561+0000","last_became_peered":"2024-04-26T21:20:45.965561+0000","last_unstale":"2024-04-26T21:20:47.335606+0000","last_undegraded":"2024-04-26T21:20:47.335606+0000","last_fullsized":"2024-04-26T21:20:47.335606+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:00.164990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:44.949774+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:44.949464+0000","last_became_peered":"2024-04-26T21:20:44.949464+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962548+0000","last_change":"2024-04-26T21:20:45.965418+0000","last_active":"2024-04-26T21:20:46.962548+0000","last_peered":"2024-04-26T21:20:46.962548+0000","last_clean":"2024-04-26T21:20:46.962548+0000","last_became_active":"2024-04-26T21:20:45.965125+0000","last_became_peered":"2024-04-26T21:20:45.965125+0000","last_unstale":"2024-04-26T21:20:46.962548+0000","last_undegraded":"2024-04-26T21:20:46.962548+0000","last_fullsized":"2024-04-26T21:20:46.962548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:47.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953649+0000","last_change":"2024-04-26T21:20:44.952556+0000","last_active":"2024-04-26T21:20:44.953649+0000","last_peered":"2024-04-26T21:20:44.953649+0000","last_clean":"2024-04-26T21:20:44.953649+0000","last_became_active":"2024-04-26T21:20:44.952257+0000","last_became_peered":"2024-04-26T21:20:44.952257+0000","last_unstale":"2024-04-26T21:20:44.953649+0000","last_undegraded":"2024-04-26T21:20:44.953649+0000","last_fullsized":"2024-04-26T21:20:44.953649+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.028292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336273+0000","last_change":"2024-04-26T21:20:45.971847+0000","last_active":"2024-04-26T21:20:47.336273+0000","last_peered":"2024-04-26T21:20:47.336273+0000","last_clean":"2024-04-26T21:20:47.336273+0000","last_became_active":"2024-04-26T21:20:45.971310+0000","last_became_peered":"2024-04-26T21:20:45.971310+0000","last_unstale":"2024-04-26T21:20:47.336273+0000","last_undegraded":"2024-04-26T21:20:47.336273+0000","last_fullsized":"2024-04-26T21:20:47.336273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.464436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959325+0000","last_change":"2024-04-26T21:20:44.950519+0000","last_active":"2024-04-26T21:20:46.959325+0000","last_peered":"2024-04-26T21:20:46.959325+0000","last_clean":"2024-04-26T21:20:46.959325+0000","last_became_active":"2024-04-26T21:20:44.950288+0000","last_became_peered":"2024-04-26T21:20:44.950288+0000","last_unstale":"2024-04-26T21:20:46.959325+0000","last_undegraded":"2024-04-26T21:20:46.959325+0000","last_fullsized":"2024-04-26T21:20:46.959325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:07.328646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961844+0000","last_change":"2024-04-26T21:20:45.968413+0000","last_active":"2024-04-26T21:20:46.961844+0000","last_peered":"2024-04-26T21:20:46.961844+0000","last_clean":"2024-04-26T21:20:46.961844+0000","last_became_active":"2024-04-26T21:20:45.968149+0000","last_became_peered":"2024-04-26T21:20:45.968149+0000","last_unstale":"2024-04-26T21:20:46.961844+0000","last_undegraded":"2024-04-26T21:20:46.961844+0000","last_fullsized":"2024-04-26T21:20:46.961844+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:11.931969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962569+0000","last_change":"2024-04-26T21:20:44.952076+0000","last_active":"2024-04-26T21:20:46.962569+0000","last_peered":"2024-04-26T21:20:46.962569+0000","last_clean":"2024-04-26T21:20:46.962569+0000","last_became_active":"2024-04-26T21:20:44.950995+0000","last_became_peered":"2024-04-26T21:20:44.950995+0000","last_unstale":"2024-04-26T21:20:46.962569+0000","last_undegraded":"2024-04-26T21:20:46.962569+0000","last_fullsized":"2024-04-26T21:20:46.962569+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:49.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959745+0000","last_change":"2024-04-26T21:20:45.967568+0000","last_active":"2024-04-26T21:20:46.959745+0000","last_peered":"2024-04-26T21:20:46.959745+0000","last_clean":"2024-04-26T21:20:46.959745+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.959745+0000","last_undegraded":"2024-04-26T21:20:46.959745+0000","last_fullsized":"2024-04-26T21:20:46.959745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:52.037830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336020+0000","last_change":"2024-04-26T21:20:44.949326+0000","last_active":"2024-04-26T21:20:47.336020+0000","last_peered":"2024-04-26T21:20:47.336020+0000","last_clean":"2024-04-26T21:20:47.336020+0000","last_became_active":"2024-04-26T21:20:44.948970+0000","last_became_peered":"2024-04-26T21:20:44.948970+0000","last_unstale":"2024-04-26T21:20:47.336020+0000","last_undegraded":"2024-04-26T21:20:47.336020+0000","last_fullsized":"2024-04-26T21:20:47.336020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.645489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336403+0000","last_change":"2024-04-26T21:20:45.967295+0000","last_active":"2024-04-26T21:20:47.336403+0000","last_peered":"2024-04-26T21:20:47.336403+0000","last_clean":"2024-04-26T21:20:47.336403+0000","last_became_active":"2024-04-26T21:20:45.966533+0000","last_became_peered":"2024-04-26T21:20:45.966533+0000","last_unstale":"2024-04-26T21:20:47.336403+0000","last_undegraded":"2024-04-26T21:20:47.336403+0000","last_fullsized":"2024-04-26T21:20:47.336403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959724+0000","last_change":"2024-04-26T21:20:44.953637+0000","last_active":"2024-04-26T21:20:46.959724+0000","last_peered":"2024-04-26T21:20:46.959724+0000","last_clean":"2024-04-26T21:20:46.959724+0000","last_became_active":"2024-04-26T21:20:44.953364+0000","last_became_peered":"2024-04-26T21:20:44.953364+0000","last_unstale":"2024-04-26T21:20:46.959724+0000","last_undegraded":"2024-04-26T21:20:46.959724+0000","last_fullsized":"2024-04-26T21:20:46.959724+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:04.043064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946421+0000","last_change":"2024-04-26T21:20:44.946421+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946421+0000","last_undegraded":"2024-04-26T21:20:44.946421+0000","last_fullsized":"2024-04-26T21:20:44.946421+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336034+0000","last_change":"2024-04-26T21:20:44.950539+0000","last_active":"2024-04-26T21:20:47.336034+0000","last_peered":"2024-04-26T21:20:47.336034+0000","last_clean":"2024-04-26T21:20:47.336034+0000","last_became_active":"2024-04-26T21:20:44.950262+0000","last_became_peered":"2024-04-26T21:20:44.950262+0000","last_unstale":"2024-04-26T21:20:47.336034+0000","last_undegraded":"2024-04-26T21:20:47.336034+0000","last_fullsized":"2024-04-26T21:20:47.336034+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:19.053104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336857+0000","last_change":"2024-04-26T21:20:45.970609+0000","last_active":"2024-04-26T21:20:47.336857+0000","last_peered":"2024-04-26T21:20:47.336857+0000","last_clean":"2024-04-26T21:20:47.336857+0000","last_became_active":"2024-04-26T21:20:45.970312+0000","last_became_peered":"2024-04-26T21:20:45.970312+0000","last_unstale":"2024-04-26T21:20:47.336857+0000","last_undegraded":"2024-04-26T21:20:47.336857+0000","last_fullsized":"2024-04-26T21:20:47.336857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336100+0000","last_change":"2024-04-26T21:20:44.949930+0000","last_active":"2024-04-26T21:20:47.336100+0000","last_peered":"2024-04-26T21:20:47.336100+0000","last_clean":"2024-04-26T21:20:47.336100+0000","last_became_active":"2024-04-26T21:20:44.949603+0000","last_became_peered":"2024-04-26T21:20:44.949603+0000","last_unstale":"2024-04-26T21:20:47.336100+0000","last_undegraded":"2024-04-26T21:20:47.336100+0000","last_fullsized":"2024-04-26T21:20:47.336100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:01.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335637+0000","last_change":"2024-04-26T21:20:45.964537+0000","last_active":"2024-04-26T21:20:47.335637+0000","last_peered":"2024-04-26T21:20:47.335637+0000","last_clean":"2024-04-26T21:20:47.335637+0000","last_became_active":"2024-04-26T21:20:45.964253+0000","last_became_peered":"2024-04-26T21:20:45.964253+0000","last_unstale":"2024-04-26T21:20:47.335637+0000","last_undegraded":"2024-04-26T21:20:47.335637+0000","last_fullsized":"2024-04-26T21:20:47.335637+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961394+0000","last_change":"2024-04-26T21:20:44.958433+0000","last_active":"2024-04-26T21:20:46.961394+0000","last_peered":"2024-04-26T21:20:46.961394+0000","last_clean":"2024-04-26T21:20:46.961394+0000","last_became_active":"2024-04-26T21:20:44.958109+0000","last_became_peered":"2024-04-26T21:20:44.958109+0000","last_unstale":"2024-04-26T21:20:46.961394+0000","last_undegraded":"2024-04-26T21:20:46.961394+0000","last_fullsized":"2024-04-26T21:20:46.961394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:56.399692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335976+0000","last_change":"2024-04-26T21:20:45.963605+0000","last_active":"2024-04-26T21:20:47.335976+0000","last_peered":"2024-04-26T21:20:47.335976+0000","last_clean":"2024-04-26T21:20:47.335976+0000","last_became_active":"2024-04-26T21:20:45.963047+0000","last_became_peered":"2024-04-26T21:20:45.963047+0000","last_unstale":"2024-04-26T21:20:47.335976+0000","last_undegraded":"2024-04-26T21:20:47.335976+0000","last_fullsized":"2024-04-26T21:20:47.335976+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961119+0000","last_change":"2024-04-26T21:20:44.957474+0000","last_active":"2024-04-26T21:20:46.961119+0000","last_peered":"2024-04-26T21:20:46.961119+0000","last_clean":"2024-04-26T21:20:46.961119+0000","last_became_active":"2024-04-26T21:20:44.957134+0000","last_became_peered":"2024-04-26T21:20:44.957134+0000","last_unstale":"2024-04-26T21:20:46.961119+0000","last_undegraded":"2024-04-26T21:20:46.961119+0000","last_fullsized":"2024-04-26T21:20:46.961119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:50.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335784+0000","last_change":"2024-04-26T21:20:45.961150+0000","last_active":"2024-04-26T21:20:47.335784+0000","last_peered":"2024-04-26T21:20:47.335784+0000","last_clean":"2024-04-26T21:20:47.335784+0000","last_became_active":"2024-04-26T21:20:45.960842+0000","last_became_peered":"2024-04-26T21:20:45.960842+0000","last_unstale":"2024-04-26T21:20:47.335784+0000","last_undegraded":"2024-04-26T21:20:47.335784+0000","last_fullsized":"2024-04-26T21:20:47.335784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.951589+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.950834+0000","last_became_peered":"2024-04-26T21:20:44.950834+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:51.927771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962749+0000","last_change":"2024-04-26T21:20:45.967744+0000","last_active":"2024-04-26T21:20:46.962749+0000","last_peered":"2024-04-26T21:20:46.962749+0000","last_clean":"2024-04-26T21:20:46.962749+0000","last_became_active":"2024-04-26T21:20:45.967409+0000","last_became_peered":"2024-04-26T21:20:45.967409+0000","last_unstale":"2024-04-26T21:20:46.962749+0000","last_undegraded":"2024-04-26T21:20:46.962749+0000","last_fullsized":"2024-04-26T21:20:46.962749+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:12.684400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962057+0000","last_change":"2024-04-26T21:20:44.958845+0000","last_active":"2024-04-26T21:20:46.962057+0000","last_peered":"2024-04-26T21:20:46.962057+0000","last_clean":"2024-04-26T21:20:46.962057+0000","last_became_active":"2024-04-26T21:20:44.958476+0000","last_became_peered":"2024-04-26T21:20:44.958476+0000","last_unstale":"2024-04-26T21:20:46.962057+0000","last_undegraded":"2024-04-26T21:20:46.962057+0000","last_fullsized":"2024-04-26T21:20:46.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:53.203282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336659+0000","last_change":"2024-04-26T21:20:45.967883+0000","last_active":"2024-04-26T21:20:47.336659+0000","last_peered":"2024-04-26T21:20:47.336659+0000","last_clean":"2024-04-26T21:20:47.336659+0000","last_became_active":"2024-04-26T21:20:45.967589+0000","last_became_peered":"2024-04-26T21:20:45.967589+0000","last_unstale":"2024-04-26T21:20:47.336659+0000","last_undegraded":"2024-04-26T21:20:47.336659+0000","last_fullsized":"2024-04-26T21:20:47.336659+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.957180+0000","last_change":"2024-04-26T21:20:44.955921+0000","last_active":"2024-04-26T21:20:44.957180+0000","last_peered":"2024-04-26T21:20:44.957180+0000","last_clean":"2024-04-26T21:20:44.957180+0000","last_became_active":"2024-04-26T21:20:44.955598+0000","last_became_peered":"2024-04-26T21:20:44.955598+0000","last_unstale":"2024-04-26T21:20:44.957180+0000","last_undegraded":"2024-04-26T21:20:44.957180+0000","last_fullsized":"2024-04-26T21:20:44.957180+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:53.995022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335972+0000","last_change":"2024-04-26T21:20:45.958489+0000","last_active":"2024-04-26T21:20:47.335972+0000","last_peered":"2024-04-26T21:20:47.335972+0000","last_clean":"2024-04-26T21:20:47.335972+0000","last_became_active":"2024-04-26T21:20:45.958208+0000","last_became_peered":"2024-04-26T21:20:45.958208+0000","last_unstale":"2024-04-26T21:20:47.335972+0000","last_undegraded":"2024-04-26T21:20:47.335972+0000","last_fullsized":"2024-04-26T21:20:47.335972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:32.662963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953383+0000","last_change":"2024-04-26T21:20:44.951269+0000","last_active":"2024-04-26T21:20:44.953383+0000","last_peered":"2024-04-26T21:20:44.953383+0000","last_clean":"2024-04-26T21:20:44.953383+0000","last_became_active":"2024-04-26T21:20:44.950960+0000","last_became_peered":"2024-04-26T21:20:44.950960+0000","last_unstale":"2024-04-26T21:20:44.953383+0000","last_undegraded":"2024-04-26T21:20:44.953383+0000","last_fullsized":"2024-04-26T21:20:44.953383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:29.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945946+0000","last_change":"2024-04-26T21:20:44.945946+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945946+0000","last_undegraded":"2024-04-26T21:20:44.945946+0000","last_fullsized":"2024-04-26T21:20:44.945946+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953430+0000","last_change":"2024-04-26T21:20:44.952291+0000","last_active":"2024-04-26T21:20:44.953430+0000","last_peered":"2024-04-26T21:20:44.953430+0000","last_clean":"2024-04-26T21:20:44.953430+0000","last_became_active":"2024-04-26T21:20:44.951911+0000","last_became_peered":"2024-04-26T21:20:44.951911+0000","last_unstale":"2024-04-26T21:20:44.953430+0000","last_undegraded":"2024-04-26T21:20:44.953430+0000","last_fullsized":"2024-04-26T21:20:44.953430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:31.605201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959372+0000","last_change":"2024-04-26T21:20:45.962616+0000","last_active":"2024-04-26T21:20:46.959372+0000","last_peered":"2024-04-26T21:20:46.959372+0000","last_clean":"2024-04-26T21:20:46.959372+0000","last_became_active":"2024-04-26T21:20:45.962318+0000","last_became_peered":"2024-04-26T21:20:45.962318+0000","last_unstale":"2024-04-26T21:20:46.959372+0000","last_undegraded":"2024-04-26T21:20:46.959372+0000","last_fullsized":"2024-04-26T21:20:46.959372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:57.812828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336229+0000","last_change":"2024-04-26T21:20:44.944630+0000","last_active":"2024-04-26T21:20:47.336229+0000","last_peered":"2024-04-26T21:20:47.336229+0000","last_clean":"2024-04-26T21:20:47.336229+0000","last_became_active":"2024-04-26T21:20:44.944271+0000","last_became_peered":"2024-04-26T21:20:44.944271+0000","last_unstale":"2024-04-26T21:20:47.336229+0000","last_undegraded":"2024-04-26T21:20:47.336229+0000","last_fullsized":"2024-04-26T21:20:47.336229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:09.997574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962808+0000","last_change":"2024-04-26T21:20:45.976347+0000","last_active":"2024-04-26T21:20:46.962808+0000","last_peered":"2024-04-26T21:20:46.962808+0000","last_clean":"2024-04-26T21:20:46.962808+0000","last_became_active":"2024-04-26T21:20:45.976080+0000","last_became_peered":"2024-04-26T21:20:45.976080+0000","last_unstale":"2024-04-26T21:20:46.962808+0000","last_undegraded":"2024-04-26T21:20:46.962808+0000","last_fullsized":"2024-04-26T21:20:46.962808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:30.154836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961487+0000","last_change":"2024-04-26T21:20:44.958470+0000","last_active":"2024-04-26T21:20:46.961487+0000","last_peered":"2024-04-26T21:20:46.961487+0000","last_clean":"2024-04-26T21:20:46.961487+0000","last_became_active":"2024-04-26T21:20:44.958191+0000","last_became_peered":"2024-04-26T21:20:44.958191+0000","last_unstale":"2024-04-26T21:20:46.961487+0000","last_undegraded":"2024-04-26T21:20:46.961487+0000","last_fullsized":"2024-04-26T21:20:46.961487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:51.907338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961496+0000","last_change":"2024-04-26T21:20:45.964088+0000","last_active":"2024-04-26T21:20:46.961496+0000","last_peered":"2024-04-26T21:20:46.961496+0000","last_clean":"2024-04-26T21:20:46.961496+0000","last_became_active":"2024-04-26T21:20:45.963798+0000","last_became_peered":"2024-04-26T21:20:45.963798+0000","last_unstale":"2024-04-26T21:20:46.961496+0000","last_undegraded":"2024-04-26T21:20:46.961496+0000","last_fullsized":"2024-04-26T21:20:46.961496+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336390+0000","last_change":"2024-04-26T21:20:44.952271+0000","last_active":"2024-04-26T21:20:47.336390+0000","last_peered":"2024-04-26T21:20:47.336390+0000","last_clean":"2024-04-26T21:20:47.336390+0000","last_became_active":"2024-04-26T21:20:44.951860+0000","last_became_peered":"2024-04-26T21:20:44.951860+0000","last_unstale":"2024-04-26T21:20:47.336390+0000","last_undegraded":"2024-04-26T21:20:47.336390+0000","last_fullsized":"2024-04-26T21:20:47.336390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962494+0000","last_change":"2024-04-26T21:20:45.969084+0000","last_active":"2024-04-26T21:20:46.962494+0000","last_peered":"2024-04-26T21:20:46.962494+0000","last_clean":"2024-04-26T21:20:46.962494+0000","last_became_active":"2024-04-26T21:20:45.968806+0000","last_became_peered":"2024-04-26T21:20:45.968806+0000","last_unstale":"2024-04-26T21:20:46.962494+0000","last_undegraded":"2024-04-26T21:20:46.962494+0000","last_fullsized":"2024-04-26T21:20:46.962494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:20.402971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335712+0000","last_change":"2024-04-26T21:20:44.955072+0000","last_active":"2024-04-26T21:20:47.335712+0000","last_peered":"2024-04-26T21:20:47.335712+0000","last_clean":"2024-04-26T21:20:47.335712+0000","last_became_active":"2024-04-26T21:20:44.954230+0000","last_became_peered":"2024-04-26T21:20:44.954230+0000","last_unstale":"2024-04-26T21:20:47.335712+0000","last_undegraded":"2024-04-26T21:20:47.335712+0000","last_fullsized":"2024-04-26T21:20:47.335712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:04.483717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336287+0000","last_change":"2024-04-26T21:20:45.964348+0000","last_active":"2024-04-26T21:20:47.336287+0000","last_peered":"2024-04-26T21:20:47.336287+0000","last_clean":"2024-04-26T21:20:47.336287+0000","last_became_active":"2024-04-26T21:20:45.964043+0000","last_became_peered":"2024-04-26T21:20:45.964043+0000","last_unstale":"2024-04-26T21:20:47.336287+0000","last_undegraded":"2024-04-26T21:20:47.336287+0000","last_fullsized":"2024-04-26T21:20:47.336287+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336648+0000","last_change":"2024-04-26T21:20:44.955598+0000","last_active":"2024-04-26T21:20:47.336648+0000","last_peered":"2024-04-26T21:20:47.336648+0000","last_clean":"2024-04-26T21:20:47.336648+0000","last_became_active":"2024-04-26T21:20:44.955310+0000","last_became_peered":"2024-04-26T21:20:44.955310+0000","last_unstale":"2024-04-26T21:20:47.336648+0000","last_undegraded":"2024-04-26T21:20:47.336648+0000","last_fullsized":"2024-04-26T21:20:47.336648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961583+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:46.961583+0000","last_peered":"2024-04-26T21:20:46.961583+0000","last_clean":"2024-04-26T21:20:46.961583+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:46.961583+0000","last_undegraded":"2024-04-26T21:20:46.961583+0000","last_fullsized":"2024-04-26T21:20:46.961583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2273280,"data_stored":1311247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1683456,"data_stored":720373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1671168,"data_stored":720227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":714017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33416,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338424,"statfs":{"total":96636764160,"available":96602546176,"internally_reserved":0,"allocated":2281472,"data_stored":1312241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36940,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334900,"statfs":{"total":96636764160,"available":96598937600,"internally_reserved":0,"allocated":1634304,"data_stored":700877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:50.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:50.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.469+0000 7f4ad9eb0640 1 Processor -- start 2024-04-26T21:20:50.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.470+0000 7f4ad9eb0640 1 -- start start 2024-04-26T21:20:50.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.470+0000 7f4ad9eb0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ad415f750 0x7f4ad40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.470+0000 7f4ad9eb0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad40d03a0 con 0x7f4ad415f370 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.470+0000 7f4ad9eb0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad40d0680 con 0x7f4ad40d3ee0 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.470+0000 7f4ad9eb0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad40d0960 con 0x7f4ad415f750 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ad415f750 0x7f4ad40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4ad415f370 0x7f4ad40d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57744/0 (socket says 172.21.15.40:57744) 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad37fe640 1 -- 172.21.15.40:0/2186970711 learned_addr learned my addr 172.21.15.40:0/2186970711 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad3fff640 1 -- 172.21.15.40:0/2186970711 >> v1:172.21.15.184:6789/0 conn(0x7f4ad40d3ee0 legacy=0x7f4ad40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad3fff640 1 -- 172.21.15.40:0/2186970711 >> v1:172.21.15.40:6789/0 conn(0x7f4ad415f370 legacy=0x7f4ad40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad3fff640 1 -- 172.21.15.40:0/2186970711 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad40d0c40 con 0x7f4ad415f750 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.471+0000 7f4ad27fc640 1 -- 172.21.15.40:0/2186970711 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908234920 0 0) 0x7f4ad40d03a0 con 0x7f4ad415f370 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad3fff640 1 --2- 172.21.15.40:0/2186970711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f750 0x7f4ad40cfd70 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009d90 tx=0x7f4acc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4679ad16a2f5f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad27fc640 1 -- 172.21.15.40:0/2186970711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc007740 con 0x7f4ad415f750 2024-04-26T21:20:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad27fc640 1 -- 172.21.15.40:0/2186970711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc0078e0 con 0x7f4ad415f750 2024-04-26T21:20:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad27fc640 1 -- 172.21.15.40:0/2186970711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc007cb0 con 0x7f4ad415f750 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/2186970711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f750 msgr2=0x7f4ad40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/2186970711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f750 0x7f4ad40cfd70 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009d90 tx=0x7f4acc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/2186970711 shutdown_connections 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/2186970711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f750 0x7f4ad40cfd70 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/2186970711 >> 172.21.15.40:0/2186970711 conn(0x7f4ad408ec10 msgr2=0x7f4ad409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/2186970711 shutdown_connections 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.472+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/2186970711 wait complete. 2024-04-26T21:20:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.473+0000 7f4ad9eb0640 1 Processor -- start 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 -- start start 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 0x7f4ad40b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad415f750 0x7f4ad40ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad40b4950 con 0x7f4ad415f370 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad40b47d0 con 0x7f4ad415f750 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.474+0000 7f4ad9eb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad40b4ad0 con 0x7f4ad40d3ee0 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35230/0 (socket says 172.21.15.40:35230) 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 -- 172.21.15.40:0/4155945751 learned_addr learned my addr 172.21.15.40:0/4155945751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad3fff640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad415f750 0x7f4ad40ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad37fe640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 0x7f4ad40b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 -- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 msgr2=0x7f4ad40b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 0x7f4ad40b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 -- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad415f750 msgr2=0x7f4ad40ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad415f750 0x7f4ad40ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 -- 172.21.15.40:0/4155945751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4158d00 con 0x7f4ad415f370 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad37fe640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 0x7f4ad40b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad2ffd640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c3f0 tx=0x7f4ac800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8007020 con 0x7f4ad415f370 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac80071c0 con 0x7f4ad415f370 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad40ba900 con 0x7f4ad415f370 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8007590 con 0x7f4ad415f370 2024-04-26T21:20:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.475+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad40c8f80 con 0x7f4ad415f370 2024-04-26T21:20:50.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.476+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad406a290 con 0x7f4ad415f370 2024-04-26T21:20:50.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.477+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ac8019020 con 0x7f4ad415f370 2024-04-26T21:20:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.477+0000 7f4abbfff640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 0x7f4ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.478+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f4ac808b250 con 0x7f4ad415f370 2024-04-26T21:20:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.478+0000 7f4ad37fe640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 0x7f4ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.478+0000 7f4ad37fe640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 0x7f4ac006edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000f60 tx=0x7f4ac4007c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.482+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ac8059680 con 0x7f4ad415f370 2024-04-26T21:20:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.663+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ad40d4fe0 con 0x7f4ac006c8f0 2024-04-26T21:20:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.669+0000 7f4abbfff640 1 -- 172.21.15.40:0/4155945751 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7f4ad40d4fe0 con 0x7f4ac006c8f0 2024-04-26T21:20:50.673 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 msgr2=0x7f4ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 0x7f4ac006edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000f60 tx=0x7f4ac4007c30 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 msgr2=0x7f4ad40b9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c3f0 tx=0x7f4ac800c8b0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 shutdown_connections 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4ac006c8f0 0x7f4ac006edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ad415f750 0x7f4ad40ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ad415f370 0x7f4ad40b9d60 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 --2- 172.21.15.40:0/4155945751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ad40d3ee0 0x7f4ad40b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 >> 172.21.15.40:0/4155945751 conn(0x7f4ad408ec10 msgr2=0x7f4ad409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 shutdown_connections 2024-04-26T21:20:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.672+0000 7f4ad9eb0640 1 -- 172.21.15.40:0/4155945751 wait complete. 2024-04-26T21:20:50.684 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T21:20:49.045926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272044,"kb_used_data":14228,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754702676,"statfs":{"total":773094113280,"available":772815540224,"internally_reserved":0,"allocated":14569472,"data_stored":6921140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114211"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945782+0000","last_change":"2024-04-26T21:20:44.945782+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945782+0000","last_undegraded":"2024-04-26T21:20:44.945782+0000","last_fullsized":"2024-04-26T21:20:44.945782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336059+0000","last_change":"2024-04-26T21:20:45.965877+0000","last_active":"2024-04-26T21:20:47.336059+0000","last_peered":"2024-04-26T21:20:47.336059+0000","last_clean":"2024-04-26T21:20:47.336059+0000","last_became_active":"2024-04-26T21:20:45.965531+0000","last_became_peered":"2024-04-26T21:20:45.965531+0000","last_unstale":"2024-04-26T21:20:47.336059+0000","last_undegraded":"2024-04-26T21:20:47.336059+0000","last_fullsized":"2024-04-26T21:20:47.336059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:02.567341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336527+0000","last_change":"2024-04-26T21:20:45.972531+0000","last_active":"2024-04-26T21:20:47.336527+0000","last_peered":"2024-04-26T21:20:47.336527+0000","last_clean":"2024-04-26T21:20:47.336527+0000","last_became_active":"2024-04-26T21:20:45.972241+0000","last_became_peered":"2024-04-26T21:20:45.972241+0000","last_unstale":"2024-04-26T21:20:47.336527+0000","last_undegraded":"2024-04-26T21:20:47.336527+0000","last_fullsized":"2024-04-26T21:20:47.336527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336969+0000","last_change":"2024-04-26T21:20:45.973682+0000","last_active":"2024-04-26T21:20:47.336969+0000","last_peered":"2024-04-26T21:20:47.336969+0000","last_clean":"2024-04-26T21:20:47.336969+0000","last_became_active":"2024-04-26T21:20:45.973381+0000","last_became_peered":"2024-04-26T21:20:45.973381+0000","last_unstale":"2024-04-26T21:20:47.336969+0000","last_undegraded":"2024-04-26T21:20:47.336969+0000","last_fullsized":"2024-04-26T21:20:47.336969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:34.199283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962690+0000","last_change":"2024-04-26T21:20:45.967509+0000","last_active":"2024-04-26T21:20:46.962690+0000","last_peered":"2024-04-26T21:20:46.962690+0000","last_clean":"2024-04-26T21:20:46.962690+0000","last_became_active":"2024-04-26T21:20:45.967269+0000","last_became_peered":"2024-04-26T21:20:45.967269+0000","last_unstale":"2024-04-26T21:20:46.962690+0000","last_undegraded":"2024-04-26T21:20:46.962690+0000","last_fullsized":"2024-04-26T21:20:46.962690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.129198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943159+0000","last_change":"2024-04-26T21:20:44.943159+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943159+0000","last_undegraded":"2024-04-26T21:20:44.943159+0000","last_fullsized":"2024-04-26T21:20:44.943159+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335603+0000","last_change":"2024-04-26T21:20:45.963229+0000","last_active":"2024-04-26T21:20:47.335603+0000","last_peered":"2024-04-26T21:20:47.335603+0000","last_clean":"2024-04-26T21:20:47.335603+0000","last_became_active":"2024-04-26T21:20:45.962872+0000","last_became_peered":"2024-04-26T21:20:45.962872+0000","last_unstale":"2024-04-26T21:20:47.335603+0000","last_undegraded":"2024-04-26T21:20:47.335603+0000","last_fullsized":"2024-04-26T21:20:47.335603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942563+0000","last_change":"2024-04-26T21:20:44.942563+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942563+0000","last_undegraded":"2024-04-26T21:20:44.942563+0000","last_fullsized":"2024-04-26T21:20:44.942563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336113+0000","last_change":"2024-04-26T21:20:45.968152+0000","last_active":"2024-04-26T21:20:47.336113+0000","last_peered":"2024-04-26T21:20:47.336113+0000","last_clean":"2024-04-26T21:20:47.336113+0000","last_became_active":"2024-04-26T21:20:45.967845+0000","last_became_peered":"2024-04-26T21:20:45.967845+0000","last_unstale":"2024-04-26T21:20:47.336113+0000","last_undegraded":"2024-04-26T21:20:47.336113+0000","last_fullsized":"2024-04-26T21:20:47.336113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.001965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961661+0000","last_change":"2024-04-26T21:20:45.977906+0000","last_active":"2024-04-26T21:20:46.961661+0000","last_peered":"2024-04-26T21:20:46.961661+0000","last_clean":"2024-04-26T21:20:46.961661+0000","last_became_active":"2024-04-26T21:20:45.977567+0000","last_became_peered":"2024-04-26T21:20:45.977567+0000","last_unstale":"2024-04-26T21:20:46.961661+0000","last_undegraded":"2024-04-26T21:20:46.961661+0000","last_fullsized":"2024-04-26T21:20:46.961661+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946943+0000","last_change":"2024-04-26T21:20:44.946943+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946943+0000","last_undegraded":"2024-04-26T21:20:44.946943+0000","last_fullsized":"2024-04-26T21:20:44.946943+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961702+0000","last_change":"2024-04-26T21:20:45.972783+0000","last_active":"2024-04-26T21:20:46.961702+0000","last_peered":"2024-04-26T21:20:46.961702+0000","last_clean":"2024-04-26T21:20:46.961702+0000","last_became_active":"2024-04-26T21:20:45.972522+0000","last_became_peered":"2024-04-26T21:20:45.972522+0000","last_unstale":"2024-04-26T21:20:46.961702+0000","last_undegraded":"2024-04-26T21:20:46.961702+0000","last_fullsized":"2024-04-26T21:20:46.961702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962682+0000","last_change":"2024-04-26T21:20:45.967507+0000","last_active":"2024-04-26T21:20:46.962682+0000","last_peered":"2024-04-26T21:20:46.962682+0000","last_clean":"2024-04-26T21:20:46.962682+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.962682+0000","last_undegraded":"2024-04-26T21:20:46.962682+0000","last_fullsized":"2024-04-26T21:20:46.962682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:40.380793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:45.967357+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:45.966578+0000","last_became_peered":"2024-04-26T21:20:45.966578+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961095+0000","last_change":"2024-04-26T21:20:45.974674+0000","last_active":"2024-04-26T21:20:46.961095+0000","last_peered":"2024-04-26T21:20:46.961095+0000","last_clean":"2024-04-26T21:20:46.961095+0000","last_became_active":"2024-04-26T21:20:45.974359+0000","last_became_peered":"2024-04-26T21:20:45.974359+0000","last_unstale":"2024-04-26T21:20:46.961095+0000","last_undegraded":"2024-04-26T21:20:46.961095+0000","last_fullsized":"2024-04-26T21:20:46.961095+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:49.051775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959220+0000","last_change":"2024-04-26T21:20:45.968322+0000","last_active":"2024-04-26T21:20:46.959220+0000","last_peered":"2024-04-26T21:20:46.959220+0000","last_clean":"2024-04-26T21:20:46.959220+0000","last_became_active":"2024-04-26T21:20:45.967746+0000","last_became_peered":"2024-04-26T21:20:45.967746+0000","last_unstale":"2024-04-26T21:20:46.959220+0000","last_undegraded":"2024-04-26T21:20:46.959220+0000","last_fullsized":"2024-04-26T21:20:46.959220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.881502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962071+0000","last_change":"2024-04-26T21:20:45.979392+0000","last_active":"2024-04-26T21:20:46.962071+0000","last_peered":"2024-04-26T21:20:46.962071+0000","last_clean":"2024-04-26T21:20:46.962071+0000","last_became_active":"2024-04-26T21:20:45.978894+0000","last_became_peered":"2024-04-26T21:20:45.978894+0000","last_unstale":"2024-04-26T21:20:46.962071+0000","last_undegraded":"2024-04-26T21:20:46.962071+0000","last_fullsized":"2024-04-26T21:20:46.962071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:04.038090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336798+0000","last_change":"2024-04-26T21:20:45.971458+0000","last_active":"2024-04-26T21:20:47.336798+0000","last_peered":"2024-04-26T21:20:47.336798+0000","last_clean":"2024-04-26T21:20:47.336798+0000","last_became_active":"2024-04-26T21:20:45.971186+0000","last_became_peered":"2024-04-26T21:20:45.971186+0000","last_unstale":"2024-04-26T21:20:47.336798+0000","last_undegraded":"2024-04-26T21:20:47.336798+0000","last_fullsized":"2024-04-26T21:20:47.336798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945523+0000","last_change":"2024-04-26T21:20:44.945523+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945523+0000","last_undegraded":"2024-04-26T21:20:44.945523+0000","last_fullsized":"2024-04-26T21:20:44.945523+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959492+0000","last_change":"2024-04-26T21:20:45.968075+0000","last_active":"2024-04-26T21:20:46.959492+0000","last_peered":"2024-04-26T21:20:46.959492+0000","last_clean":"2024-04-26T21:20:46.959492+0000","last_became_active":"2024-04-26T21:20:45.967734+0000","last_became_peered":"2024-04-26T21:20:45.967734+0000","last_unstale":"2024-04-26T21:20:46.959492+0000","last_undegraded":"2024-04-26T21:20:46.959492+0000","last_fullsized":"2024-04-26T21:20:46.959492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.314546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961395+0000","last_change":"2024-04-26T21:20:45.977923+0000","last_active":"2024-04-26T21:20:46.961395+0000","last_peered":"2024-04-26T21:20:46.961395+0000","last_clean":"2024-04-26T21:20:46.961395+0000","last_became_active":"2024-04-26T21:20:45.977624+0000","last_became_peered":"2024-04-26T21:20:45.977624+0000","last_unstale":"2024-04-26T21:20:46.961395+0000","last_undegraded":"2024-04-26T21:20:46.961395+0000","last_fullsized":"2024-04-26T21:20:46.961395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336657+0000","last_change":"2024-04-26T21:20:45.956281+0000","last_active":"2024-04-26T21:20:47.336657+0000","last_peered":"2024-04-26T21:20:47.336657+0000","last_clean":"2024-04-26T21:20:47.336657+0000","last_became_active":"2024-04-26T21:20:45.955972+0000","last_became_peered":"2024-04-26T21:20:45.955972+0000","last_unstale":"2024-04-26T21:20:47.336657+0000","last_undegraded":"2024-04-26T21:20:47.336657+0000","last_fullsized":"2024-04-26T21:20:47.336657+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336022+0000","last_change":"2024-04-26T21:20:45.967466+0000","last_active":"2024-04-26T21:20:47.336022+0000","last_peered":"2024-04-26T21:20:47.336022+0000","last_clean":"2024-04-26T21:20:47.336022+0000","last_became_active":"2024-04-26T21:20:45.967180+0000","last_became_peered":"2024-04-26T21:20:45.967180+0000","last_unstale":"2024-04-26T21:20:47.336022+0000","last_undegraded":"2024-04-26T21:20:47.336022+0000","last_fullsized":"2024-04-26T21:20:47.336022+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961176+0000","last_change":"2024-04-26T21:20:45.966830+0000","last_active":"2024-04-26T21:20:46.961176+0000","last_peered":"2024-04-26T21:20:46.961176+0000","last_clean":"2024-04-26T21:20:46.961176+0000","last_became_active":"2024-04-26T21:20:45.966121+0000","last_became_peered":"2024-04-26T21:20:45.966121+0000","last_unstale":"2024-04-26T21:20:46.961176+0000","last_undegraded":"2024-04-26T21:20:46.961176+0000","last_fullsized":"2024-04-26T21:20:46.961176+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:50.753188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959755+0000","last_change":"2024-04-26T21:20:45.968233+0000","last_active":"2024-04-26T21:20:46.959755+0000","last_peered":"2024-04-26T21:20:46.959755+0000","last_clean":"2024-04-26T21:20:46.959755+0000","last_became_active":"2024-04-26T21:20:45.967940+0000","last_became_peered":"2024-04-26T21:20:45.967940+0000","last_unstale":"2024-04-26T21:20:46.959755+0000","last_undegraded":"2024-04-26T21:20:46.959755+0000","last_fullsized":"2024-04-26T21:20:46.959755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:34.779995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.944578+0000","last_change":"2024-04-26T21:20:44.944578+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.944578+0000","last_undegraded":"2024-04-26T21:20:44.944578+0000","last_fullsized":"2024-04-26T21:20:44.944578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959794+0000","last_change":"2024-04-26T21:20:45.973518+0000","last_active":"2024-04-26T21:20:46.959794+0000","last_peered":"2024-04-26T21:20:46.959794+0000","last_clean":"2024-04-26T21:20:46.959794+0000","last_became_active":"2024-04-26T21:20:45.973227+0000","last_became_peered":"2024-04-26T21:20:45.973227+0000","last_unstale":"2024-04-26T21:20:46.959794+0000","last_undegraded":"2024-04-26T21:20:46.959794+0000","last_fullsized":"2024-04-26T21:20:46.959794+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:12.310488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336254+0000","last_change":"2024-04-26T21:20:45.973958+0000","last_active":"2024-04-26T21:20:47.336254+0000","last_peered":"2024-04-26T21:20:47.336254+0000","last_clean":"2024-04-26T21:20:47.336254+0000","last_became_active":"2024-04-26T21:20:45.973675+0000","last_became_peered":"2024-04-26T21:20:45.973675+0000","last_unstale":"2024-04-26T21:20:47.336254+0000","last_undegraded":"2024-04-26T21:20:47.336254+0000","last_fullsized":"2024-04-26T21:20:47.336254+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.943772+0000","last_change":"2024-04-26T21:20:44.943772+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.943772+0000","last_undegraded":"2024-04-26T21:20:44.943772+0000","last_fullsized":"2024-04-26T21:20:44.943772+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336855+0000","last_change":"2024-04-26T21:20:45.970795+0000","last_active":"2024-04-26T21:20:47.336855+0000","last_peered":"2024-04-26T21:20:47.336855+0000","last_clean":"2024-04-26T21:20:47.336855+0000","last_became_active":"2024-04-26T21:20:45.970509+0000","last_became_peered":"2024-04-26T21:20:45.970509+0000","last_unstale":"2024-04-26T21:20:47.336855+0000","last_undegraded":"2024-04-26T21:20:47.336855+0000","last_fullsized":"2024-04-26T21:20:47.336855+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:30.015334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335701+0000","last_change":"2024-04-26T21:20:45.972356+0000","last_active":"2024-04-26T21:20:47.335701+0000","last_peered":"2024-04-26T21:20:47.335701+0000","last_clean":"2024-04-26T21:20:47.335701+0000","last_became_active":"2024-04-26T21:20:45.972064+0000","last_became_peered":"2024-04-26T21:20:45.972064+0000","last_unstale":"2024-04-26T21:20:47.335701+0000","last_undegraded":"2024-04-26T21:20:47.335701+0000","last_fullsized":"2024-04-26T21:20:47.335701+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:50.058209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959285+0000","last_change":"2024-04-26T21:20:45.968435+0000","last_active":"2024-04-26T21:20:46.959285+0000","last_peered":"2024-04-26T21:20:46.959285+0000","last_clean":"2024-04-26T21:20:46.959285+0000","last_became_active":"2024-04-26T21:20:45.966848+0000","last_became_peered":"2024-04-26T21:20:45.966848+0000","last_unstale":"2024-04-26T21:20:46.959285+0000","last_undegraded":"2024-04-26T21:20:46.959285+0000","last_fullsized":"2024-04-26T21:20:46.959285+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:50.873852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961713+0000","last_change":"2024-04-26T21:20:45.973734+0000","last_active":"2024-04-26T21:20:46.961713+0000","last_peered":"2024-04-26T21:20:46.961713+0000","last_clean":"2024-04-26T21:20:46.961713+0000","last_became_active":"2024-04-26T21:20:45.973472+0000","last_became_peered":"2024-04-26T21:20:45.973472+0000","last_unstale":"2024-04-26T21:20:46.961713+0000","last_undegraded":"2024-04-26T21:20:46.961713+0000","last_fullsized":"2024-04-26T21:20:46.961713+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961961+0000","last_change":"2024-04-26T21:20:44.958400+0000","last_active":"2024-04-26T21:20:46.961961+0000","last_peered":"2024-04-26T21:20:46.961961+0000","last_clean":"2024-04-26T21:20:46.961961+0000","last_became_active":"2024-04-26T21:20:44.958081+0000","last_became_peered":"2024-04-26T21:20:44.958081+0000","last_unstale":"2024-04-26T21:20:46.961961+0000","last_undegraded":"2024-04-26T21:20:46.961961+0000","last_fullsized":"2024-04-26T21:20:46.961961+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:38.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336119+0000","last_change":"2024-04-26T21:20:45.964932+0000","last_active":"2024-04-26T21:20:47.336119+0000","last_peered":"2024-04-26T21:20:47.336119+0000","last_clean":"2024-04-26T21:20:47.336119+0000","last_became_active":"2024-04-26T21:20:45.961496+0000","last_became_peered":"2024-04-26T21:20:45.961496+0000","last_unstale":"2024-04-26T21:20:47.336119+0000","last_undegraded":"2024-04-26T21:20:47.336119+0000","last_fullsized":"2024-04-26T21:20:47.336119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.885551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335890+0000","last_change":"2024-04-26T21:20:44.956115+0000","last_active":"2024-04-26T21:20:47.335890+0000","last_peered":"2024-04-26T21:20:47.335890+0000","last_clean":"2024-04-26T21:20:47.335890+0000","last_became_active":"2024-04-26T21:20:44.955813+0000","last_became_peered":"2024-04-26T21:20:44.955813+0000","last_unstale":"2024-04-26T21:20:47.335890+0000","last_undegraded":"2024-04-26T21:20:47.335890+0000","last_fullsized":"2024-04-26T21:20:47.335890+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:21.778534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.942776+0000","last_change":"2024-04-26T21:20:44.942776+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.942776+0000","last_undegraded":"2024-04-26T21:20:44.942776+0000","last_fullsized":"2024-04-26T21:20:44.942776+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.949711+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.949393+0000","last_became_peered":"2024-04-26T21:20:44.949393+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961372+0000","last_change":"2024-04-26T21:20:45.964911+0000","last_active":"2024-04-26T21:20:46.961372+0000","last_peered":"2024-04-26T21:20:46.961372+0000","last_clean":"2024-04-26T21:20:46.961372+0000","last_became_active":"2024-04-26T21:20:45.963945+0000","last_became_peered":"2024-04-26T21:20:45.963945+0000","last_unstale":"2024-04-26T21:20:46.961372+0000","last_undegraded":"2024-04-26T21:20:46.961372+0000","last_fullsized":"2024-04-26T21:20:46.961372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:41.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.337000+0000","last_change":"2024-04-26T21:20:44.949483+0000","last_active":"2024-04-26T21:20:47.337000+0000","last_peered":"2024-04-26T21:20:47.337000+0000","last_clean":"2024-04-26T21:20:47.337000+0000","last_became_active":"2024-04-26T21:20:44.948183+0000","last_became_peered":"2024-04-26T21:20:44.948183+0000","last_unstale":"2024-04-26T21:20:47.337000+0000","last_undegraded":"2024-04-26T21:20:47.337000+0000","last_fullsized":"2024-04-26T21:20:47.337000+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335767+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:47.335767+0000","last_peered":"2024-04-26T21:20:47.335767+0000","last_clean":"2024-04-26T21:20:47.335767+0000","last_became_active":"2024-04-26T21:20:45.973350+0000","last_became_peered":"2024-04-26T21:20:45.973350+0000","last_unstale":"2024-04-26T21:20:47.335767+0000","last_undegraded":"2024-04-26T21:20:47.335767+0000","last_fullsized":"2024-04-26T21:20:47.335767+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.225182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959303+0000","last_change":"2024-04-26T21:20:44.954463+0000","last_active":"2024-04-26T21:20:46.959303+0000","last_peered":"2024-04-26T21:20:46.959303+0000","last_clean":"2024-04-26T21:20:46.959303+0000","last_became_active":"2024-04-26T21:20:44.954133+0000","last_became_peered":"2024-04-26T21:20:44.954133+0000","last_unstale":"2024-04-26T21:20:46.959303+0000","last_undegraded":"2024-04-26T21:20:46.959303+0000","last_fullsized":"2024-04-26T21:20:46.959303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:16.181685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336085+0000","last_change":"2024-04-26T21:20:45.978602+0000","last_active":"2024-04-26T21:20:47.336085+0000","last_peered":"2024-04-26T21:20:47.336085+0000","last_clean":"2024-04-26T21:20:47.336085+0000","last_became_active":"2024-04-26T21:20:45.978300+0000","last_became_peered":"2024-04-26T21:20:45.978300+0000","last_unstale":"2024-04-26T21:20:47.336085+0000","last_undegraded":"2024-04-26T21:20:47.336085+0000","last_fullsized":"2024-04-26T21:20:47.336085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962713+0000","last_change":"2024-04-26T21:20:44.953754+0000","last_active":"2024-04-26T21:20:46.962713+0000","last_peered":"2024-04-26T21:20:46.962713+0000","last_clean":"2024-04-26T21:20:46.962713+0000","last_became_active":"2024-04-26T21:20:44.953471+0000","last_became_peered":"2024-04-26T21:20:44.953471+0000","last_unstale":"2024-04-26T21:20:46.962713+0000","last_undegraded":"2024-04-26T21:20:46.962713+0000","last_fullsized":"2024-04-26T21:20:46.962713+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:21.877111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335605+0000","last_change":"2024-04-26T21:20:45.957101+0000","last_active":"2024-04-26T21:20:47.335605+0000","last_peered":"2024-04-26T21:20:47.335605+0000","last_clean":"2024-04-26T21:20:47.335605+0000","last_became_active":"2024-04-26T21:20:45.956746+0000","last_became_peered":"2024-04-26T21:20:45.956746+0000","last_unstale":"2024-04-26T21:20:47.335605+0000","last_undegraded":"2024-04-26T21:20:47.335605+0000","last_fullsized":"2024-04-26T21:20:47.335605+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:20.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959438+0000","last_change":"2024-04-26T21:20:44.945604+0000","last_active":"2024-04-26T21:20:46.959438+0000","last_peered":"2024-04-26T21:20:46.959438+0000","last_clean":"2024-04-26T21:20:46.959438+0000","last_became_active":"2024-04-26T21:20:44.945311+0000","last_became_peered":"2024-04-26T21:20:44.945311+0000","last_unstale":"2024-04-26T21:20:46.959438+0000","last_undegraded":"2024-04-26T21:20:46.959438+0000","last_fullsized":"2024-04-26T21:20:46.959438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:59.997109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962422+0000","last_change":"2024-04-26T21:20:45.968415+0000","last_active":"2024-04-26T21:20:46.962422+0000","last_peered":"2024-04-26T21:20:46.962422+0000","last_clean":"2024-04-26T21:20:46.962422+0000","last_became_active":"2024-04-26T21:20:45.968148+0000","last_became_peered":"2024-04-26T21:20:45.968148+0000","last_unstale":"2024-04-26T21:20:46.962422+0000","last_undegraded":"2024-04-26T21:20:46.962422+0000","last_fullsized":"2024-04-26T21:20:46.962422+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:30.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335772+0000","last_change":"2024-04-26T21:20:44.953539+0000","last_active":"2024-04-26T21:20:47.335772+0000","last_peered":"2024-04-26T21:20:47.335772+0000","last_clean":"2024-04-26T21:20:47.335772+0000","last_became_active":"2024-04-26T21:20:44.953286+0000","last_became_peered":"2024-04-26T21:20:44.953286+0000","last_unstale":"2024-04-26T21:20:47.335772+0000","last_undegraded":"2024-04-26T21:20:47.335772+0000","last_fullsized":"2024-04-26T21:20:47.335772+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962742+0000","last_change":"2024-04-26T21:20:45.957232+0000","last_active":"2024-04-26T21:20:46.962742+0000","last_peered":"2024-04-26T21:20:46.962742+0000","last_clean":"2024-04-26T21:20:46.962742+0000","last_became_active":"2024-04-26T21:20:45.956611+0000","last_became_peered":"2024-04-26T21:20:45.956611+0000","last_unstale":"2024-04-26T21:20:46.962742+0000","last_undegraded":"2024-04-26T21:20:46.962742+0000","last_fullsized":"2024-04-26T21:20:46.962742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:13.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336218+0000","last_change":"2024-04-26T21:20:44.952646+0000","last_active":"2024-04-26T21:20:47.336218+0000","last_peered":"2024-04-26T21:20:47.336218+0000","last_clean":"2024-04-26T21:20:47.336218+0000","last_became_active":"2024-04-26T21:20:44.952351+0000","last_became_peered":"2024-04-26T21:20:44.952351+0000","last_unstale":"2024-04-26T21:20:47.336218+0000","last_undegraded":"2024-04-26T21:20:47.336218+0000","last_fullsized":"2024-04-26T21:20:47.336218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:33.194892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961915+0000","last_change":"2024-04-26T21:20:45.979158+0000","last_active":"2024-04-26T21:20:46.961915+0000","last_peered":"2024-04-26T21:20:46.961915+0000","last_clean":"2024-04-26T21:20:46.961915+0000","last_became_active":"2024-04-26T21:20:45.978509+0000","last_became_peered":"2024-04-26T21:20:45.978509+0000","last_unstale":"2024-04-26T21:20:46.961915+0000","last_undegraded":"2024-04-26T21:20:46.961915+0000","last_fullsized":"2024-04-26T21:20:46.961915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:18.599945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961941+0000","last_change":"2024-04-26T21:20:44.958683+0000","last_active":"2024-04-26T21:20:46.961941+0000","last_peered":"2024-04-26T21:20:46.961941+0000","last_clean":"2024-04-26T21:20:46.961941+0000","last_became_active":"2024-04-26T21:20:44.958250+0000","last_became_peered":"2024-04-26T21:20:44.958250+0000","last_unstale":"2024-04-26T21:20:46.961941+0000","last_undegraded":"2024-04-26T21:20:46.961941+0000","last_fullsized":"2024-04-26T21:20:46.961941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:19.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959625+0000","last_change":"2024-04-26T21:20:45.968143+0000","last_active":"2024-04-26T21:20:46.959625+0000","last_peered":"2024-04-26T21:20:46.959625+0000","last_clean":"2024-04-26T21:20:46.959625+0000","last_became_active":"2024-04-26T21:20:45.966158+0000","last_became_peered":"2024-04-26T21:20:45.966158+0000","last_unstale":"2024-04-26T21:20:46.959625+0000","last_undegraded":"2024-04-26T21:20:46.959625+0000","last_fullsized":"2024-04-26T21:20:46.959625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:10.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336377+0000","last_change":"2024-04-26T21:20:44.951683+0000","last_active":"2024-04-26T21:20:47.336377+0000","last_peered":"2024-04-26T21:20:47.336377+0000","last_clean":"2024-04-26T21:20:47.336377+0000","last_became_active":"2024-04-26T21:20:44.951213+0000","last_became_peered":"2024-04-26T21:20:44.951213+0000","last_unstale":"2024-04-26T21:20:47.336377+0000","last_undegraded":"2024-04-26T21:20:47.336377+0000","last_fullsized":"2024-04-26T21:20:47.336377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:36.881160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336537+0000","last_change":"2024-04-26T21:20:45.968505+0000","last_active":"2024-04-26T21:20:47.336537+0000","last_peered":"2024-04-26T21:20:47.336537+0000","last_clean":"2024-04-26T21:20:47.336537+0000","last_became_active":"2024-04-26T21:20:45.968185+0000","last_became_peered":"2024-04-26T21:20:45.968185+0000","last_unstale":"2024-04-26T21:20:47.336537+0000","last_undegraded":"2024-04-26T21:20:47.336537+0000","last_fullsized":"2024-04-26T21:20:47.336537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336545+0000","last_change":"2024-04-26T21:20:44.951050+0000","last_active":"2024-04-26T21:20:47.336545+0000","last_peered":"2024-04-26T21:20:47.336545+0000","last_clean":"2024-04-26T21:20:47.336545+0000","last_became_active":"2024-04-26T21:20:44.950758+0000","last_became_peered":"2024-04-26T21:20:44.950758+0000","last_unstale":"2024-04-26T21:20:47.336545+0000","last_undegraded":"2024-04-26T21:20:47.336545+0000","last_fullsized":"2024-04-26T21:20:47.336545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:07.338750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335606+0000","last_change":"2024-04-26T21:20:45.965874+0000","last_active":"2024-04-26T21:20:47.335606+0000","last_peered":"2024-04-26T21:20:47.335606+0000","last_clean":"2024-04-26T21:20:47.335606+0000","last_became_active":"2024-04-26T21:20:45.965561+0000","last_became_peered":"2024-04-26T21:20:45.965561+0000","last_unstale":"2024-04-26T21:20:47.335606+0000","last_undegraded":"2024-04-26T21:20:47.335606+0000","last_fullsized":"2024-04-26T21:20:47.335606+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:00.164990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:44.949774+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:44.949464+0000","last_became_peered":"2024-04-26T21:20:44.949464+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962548+0000","last_change":"2024-04-26T21:20:45.965418+0000","last_active":"2024-04-26T21:20:46.962548+0000","last_peered":"2024-04-26T21:20:46.962548+0000","last_clean":"2024-04-26T21:20:46.962548+0000","last_became_active":"2024-04-26T21:20:45.965125+0000","last_became_peered":"2024-04-26T21:20:45.965125+0000","last_unstale":"2024-04-26T21:20:46.962548+0000","last_undegraded":"2024-04-26T21:20:46.962548+0000","last_fullsized":"2024-04-26T21:20:46.962548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:47.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953649+0000","last_change":"2024-04-26T21:20:44.952556+0000","last_active":"2024-04-26T21:20:44.953649+0000","last_peered":"2024-04-26T21:20:44.953649+0000","last_clean":"2024-04-26T21:20:44.953649+0000","last_became_active":"2024-04-26T21:20:44.952257+0000","last_became_peered":"2024-04-26T21:20:44.952257+0000","last_unstale":"2024-04-26T21:20:44.953649+0000","last_undegraded":"2024-04-26T21:20:44.953649+0000","last_fullsized":"2024-04-26T21:20:44.953649+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.028292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336273+0000","last_change":"2024-04-26T21:20:45.971847+0000","last_active":"2024-04-26T21:20:47.336273+0000","last_peered":"2024-04-26T21:20:47.336273+0000","last_clean":"2024-04-26T21:20:47.336273+0000","last_became_active":"2024-04-26T21:20:45.971310+0000","last_became_peered":"2024-04-26T21:20:45.971310+0000","last_unstale":"2024-04-26T21:20:47.336273+0000","last_undegraded":"2024-04-26T21:20:47.336273+0000","last_fullsized":"2024-04-26T21:20:47.336273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.464436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959325+0000","last_change":"2024-04-26T21:20:44.950519+0000","last_active":"2024-04-26T21:20:46.959325+0000","last_peered":"2024-04-26T21:20:46.959325+0000","last_clean":"2024-04-26T21:20:46.959325+0000","last_became_active":"2024-04-26T21:20:44.950288+0000","last_became_peered":"2024-04-26T21:20:44.950288+0000","last_unstale":"2024-04-26T21:20:46.959325+0000","last_undegraded":"2024-04-26T21:20:46.959325+0000","last_fullsized":"2024-04-26T21:20:46.959325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:07.328646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961844+0000","last_change":"2024-04-26T21:20:45.968413+0000","last_active":"2024-04-26T21:20:46.961844+0000","last_peered":"2024-04-26T21:20:46.961844+0000","last_clean":"2024-04-26T21:20:46.961844+0000","last_became_active":"2024-04-26T21:20:45.968149+0000","last_became_peered":"2024-04-26T21:20:45.968149+0000","last_unstale":"2024-04-26T21:20:46.961844+0000","last_undegraded":"2024-04-26T21:20:46.961844+0000","last_fullsized":"2024-04-26T21:20:46.961844+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:11.931969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962569+0000","last_change":"2024-04-26T21:20:44.952076+0000","last_active":"2024-04-26T21:20:46.962569+0000","last_peered":"2024-04-26T21:20:46.962569+0000","last_clean":"2024-04-26T21:20:46.962569+0000","last_became_active":"2024-04-26T21:20:44.950995+0000","last_became_peered":"2024-04-26T21:20:44.950995+0000","last_unstale":"2024-04-26T21:20:46.962569+0000","last_undegraded":"2024-04-26T21:20:46.962569+0000","last_fullsized":"2024-04-26T21:20:46.962569+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:49.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959745+0000","last_change":"2024-04-26T21:20:45.967568+0000","last_active":"2024-04-26T21:20:46.959745+0000","last_peered":"2024-04-26T21:20:46.959745+0000","last_clean":"2024-04-26T21:20:46.959745+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.959745+0000","last_undegraded":"2024-04-26T21:20:46.959745+0000","last_fullsized":"2024-04-26T21:20:46.959745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:52.037830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336020+0000","last_change":"2024-04-26T21:20:44.949326+0000","last_active":"2024-04-26T21:20:47.336020+0000","last_peered":"2024-04-26T21:20:47.336020+0000","last_clean":"2024-04-26T21:20:47.336020+0000","last_became_active":"2024-04-26T21:20:44.948970+0000","last_became_peered":"2024-04-26T21:20:44.948970+0000","last_unstale":"2024-04-26T21:20:47.336020+0000","last_undegraded":"2024-04-26T21:20:47.336020+0000","last_fullsized":"2024-04-26T21:20:47.336020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.645489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336403+0000","last_change":"2024-04-26T21:20:45.967295+0000","last_active":"2024-04-26T21:20:47.336403+0000","last_peered":"2024-04-26T21:20:47.336403+0000","last_clean":"2024-04-26T21:20:47.336403+0000","last_became_active":"2024-04-26T21:20:45.966533+0000","last_became_peered":"2024-04-26T21:20:45.966533+0000","last_unstale":"2024-04-26T21:20:47.336403+0000","last_undegraded":"2024-04-26T21:20:47.336403+0000","last_fullsized":"2024-04-26T21:20:47.336403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959724+0000","last_change":"2024-04-26T21:20:44.953637+0000","last_active":"2024-04-26T21:20:46.959724+0000","last_peered":"2024-04-26T21:20:46.959724+0000","last_clean":"2024-04-26T21:20:46.959724+0000","last_became_active":"2024-04-26T21:20:44.953364+0000","last_became_peered":"2024-04-26T21:20:44.953364+0000","last_unstale":"2024-04-26T21:20:46.959724+0000","last_undegraded":"2024-04-26T21:20:46.959724+0000","last_fullsized":"2024-04-26T21:20:46.959724+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:04.043064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.946421+0000","last_change":"2024-04-26T21:20:44.946421+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.946421+0000","last_undegraded":"2024-04-26T21:20:44.946421+0000","last_fullsized":"2024-04-26T21:20:44.946421+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336034+0000","last_change":"2024-04-26T21:20:44.950539+0000","last_active":"2024-04-26T21:20:47.336034+0000","last_peered":"2024-04-26T21:20:47.336034+0000","last_clean":"2024-04-26T21:20:47.336034+0000","last_became_active":"2024-04-26T21:20:44.950262+0000","last_became_peered":"2024-04-26T21:20:44.950262+0000","last_unstale":"2024-04-26T21:20:47.336034+0000","last_undegraded":"2024-04-26T21:20:47.336034+0000","last_fullsized":"2024-04-26T21:20:47.336034+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:19.053104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336857+0000","last_change":"2024-04-26T21:20:45.970609+0000","last_active":"2024-04-26T21:20:47.336857+0000","last_peered":"2024-04-26T21:20:47.336857+0000","last_clean":"2024-04-26T21:20:47.336857+0000","last_became_active":"2024-04-26T21:20:45.970312+0000","last_became_peered":"2024-04-26T21:20:45.970312+0000","last_unstale":"2024-04-26T21:20:47.336857+0000","last_undegraded":"2024-04-26T21:20:47.336857+0000","last_fullsized":"2024-04-26T21:20:47.336857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336100+0000","last_change":"2024-04-26T21:20:44.949930+0000","last_active":"2024-04-26T21:20:47.336100+0000","last_peered":"2024-04-26T21:20:47.336100+0000","last_clean":"2024-04-26T21:20:47.336100+0000","last_became_active":"2024-04-26T21:20:44.949603+0000","last_became_peered":"2024-04-26T21:20:44.949603+0000","last_unstale":"2024-04-26T21:20:47.336100+0000","last_undegraded":"2024-04-26T21:20:47.336100+0000","last_fullsized":"2024-04-26T21:20:47.336100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:01.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335637+0000","last_change":"2024-04-26T21:20:45.964537+0000","last_active":"2024-04-26T21:20:47.335637+0000","last_peered":"2024-04-26T21:20:47.335637+0000","last_clean":"2024-04-26T21:20:47.335637+0000","last_became_active":"2024-04-26T21:20:45.964253+0000","last_became_peered":"2024-04-26T21:20:45.964253+0000","last_unstale":"2024-04-26T21:20:47.335637+0000","last_undegraded":"2024-04-26T21:20:47.335637+0000","last_fullsized":"2024-04-26T21:20:47.335637+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961394+0000","last_change":"2024-04-26T21:20:44.958433+0000","last_active":"2024-04-26T21:20:46.961394+0000","last_peered":"2024-04-26T21:20:46.961394+0000","last_clean":"2024-04-26T21:20:46.961394+0000","last_became_active":"2024-04-26T21:20:44.958109+0000","last_became_peered":"2024-04-26T21:20:44.958109+0000","last_unstale":"2024-04-26T21:20:46.961394+0000","last_undegraded":"2024-04-26T21:20:46.961394+0000","last_fullsized":"2024-04-26T21:20:46.961394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:56.399692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335976+0000","last_change":"2024-04-26T21:20:45.963605+0000","last_active":"2024-04-26T21:20:47.335976+0000","last_peered":"2024-04-26T21:20:47.335976+0000","last_clean":"2024-04-26T21:20:47.335976+0000","last_became_active":"2024-04-26T21:20:45.963047+0000","last_became_peered":"2024-04-26T21:20:45.963047+0000","last_unstale":"2024-04-26T21:20:47.335976+0000","last_undegraded":"2024-04-26T21:20:47.335976+0000","last_fullsized":"2024-04-26T21:20:47.335976+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961119+0000","last_change":"2024-04-26T21:20:44.957474+0000","last_active":"2024-04-26T21:20:46.961119+0000","last_peered":"2024-04-26T21:20:46.961119+0000","last_clean":"2024-04-26T21:20:46.961119+0000","last_became_active":"2024-04-26T21:20:44.957134+0000","last_became_peered":"2024-04-26T21:20:44.957134+0000","last_unstale":"2024-04-26T21:20:46.961119+0000","last_undegraded":"2024-04-26T21:20:46.961119+0000","last_fullsized":"2024-04-26T21:20:46.961119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:50.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335784+0000","last_change":"2024-04-26T21:20:45.961150+0000","last_active":"2024-04-26T21:20:47.335784+0000","last_peered":"2024-04-26T21:20:47.335784+0000","last_clean":"2024-04-26T21:20:47.335784+0000","last_became_active":"2024-04-26T21:20:45.960842+0000","last_became_peered":"2024-04-26T21:20:45.960842+0000","last_unstale":"2024-04-26T21:20:47.335784+0000","last_undegraded":"2024-04-26T21:20:47.335784+0000","last_fullsized":"2024-04-26T21:20:47.335784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.951589+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.950834+0000","last_became_peered":"2024-04-26T21:20:44.950834+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:51.927771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962749+0000","last_change":"2024-04-26T21:20:45.967744+0000","last_active":"2024-04-26T21:20:46.962749+0000","last_peered":"2024-04-26T21:20:46.962749+0000","last_clean":"2024-04-26T21:20:46.962749+0000","last_became_active":"2024-04-26T21:20:45.967409+0000","last_became_peered":"2024-04-26T21:20:45.967409+0000","last_unstale":"2024-04-26T21:20:46.962749+0000","last_undegraded":"2024-04-26T21:20:46.962749+0000","last_fullsized":"2024-04-26T21:20:46.962749+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:12.684400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962057+0000","last_change":"2024-04-26T21:20:44.958845+0000","last_active":"2024-04-26T21:20:46.962057+0000","last_peered":"2024-04-26T21:20:46.962057+0000","last_clean":"2024-04-26T21:20:46.962057+0000","last_became_active":"2024-04-26T21:20:44.958476+0000","last_became_peered":"2024-04-26T21:20:44.958476+0000","last_unstale":"2024-04-26T21:20:46.962057+0000","last_undegraded":"2024-04-26T21:20:46.962057+0000","last_fullsized":"2024-04-26T21:20:46.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:53.203282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336659+0000","last_change":"2024-04-26T21:20:45.967883+0000","last_active":"2024-04-26T21:20:47.336659+0000","last_peered":"2024-04-26T21:20:47.336659+0000","last_clean":"2024-04-26T21:20:47.336659+0000","last_became_active":"2024-04-26T21:20:45.967589+0000","last_became_peered":"2024-04-26T21:20:45.967589+0000","last_unstale":"2024-04-26T21:20:47.336659+0000","last_undegraded":"2024-04-26T21:20:47.336659+0000","last_fullsized":"2024-04-26T21:20:47.336659+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.957180+0000","last_change":"2024-04-26T21:20:44.955921+0000","last_active":"2024-04-26T21:20:44.957180+0000","last_peered":"2024-04-26T21:20:44.957180+0000","last_clean":"2024-04-26T21:20:44.957180+0000","last_became_active":"2024-04-26T21:20:44.955598+0000","last_became_peered":"2024-04-26T21:20:44.955598+0000","last_unstale":"2024-04-26T21:20:44.957180+0000","last_undegraded":"2024-04-26T21:20:44.957180+0000","last_fullsized":"2024-04-26T21:20:44.957180+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:53.995022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335972+0000","last_change":"2024-04-26T21:20:45.958489+0000","last_active":"2024-04-26T21:20:47.335972+0000","last_peered":"2024-04-26T21:20:47.335972+0000","last_clean":"2024-04-26T21:20:47.335972+0000","last_became_active":"2024-04-26T21:20:45.958208+0000","last_became_peered":"2024-04-26T21:20:45.958208+0000","last_unstale":"2024-04-26T21:20:47.335972+0000","last_undegraded":"2024-04-26T21:20:47.335972+0000","last_fullsized":"2024-04-26T21:20:47.335972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:32.662963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953383+0000","last_change":"2024-04-26T21:20:44.951269+0000","last_active":"2024-04-26T21:20:44.953383+0000","last_peered":"2024-04-26T21:20:44.953383+0000","last_clean":"2024-04-26T21:20:44.953383+0000","last_became_active":"2024-04-26T21:20:44.950960+0000","last_became_peered":"2024-04-26T21:20:44.950960+0000","last_unstale":"2024-04-26T21:20:44.953383+0000","last_undegraded":"2024-04-26T21:20:44.953383+0000","last_fullsized":"2024-04-26T21:20:44.953383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:29.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2024-04-26T21:20:44.945946+0000","last_change":"2024-04-26T21:20:44.945946+0000","last_active":"2024-04-26T21:20:44.930196+0000","last_peered":"2024-04-26T21:20:44.930196+0000","last_clean":"2024-04-26T21:20:44.930196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:20:44.945946+0000","last_undegraded":"2024-04-26T21:20:44.945946+0000","last_fullsized":"2024-04-26T21:20:44.945946+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T21:20:44.953430+0000","last_change":"2024-04-26T21:20:44.952291+0000","last_active":"2024-04-26T21:20:44.953430+0000","last_peered":"2024-04-26T21:20:44.953430+0000","last_clean":"2024-04-26T21:20:44.953430+0000","last_became_active":"2024-04-26T21:20:44.951911+0000","last_became_peered":"2024-04-26T21:20:44.951911+0000","last_unstale":"2024-04-26T21:20:44.953430+0000","last_undegraded":"2024-04-26T21:20:44.953430+0000","last_fullsized":"2024-04-26T21:20:44.953430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:31.605201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959372+0000","last_change":"2024-04-26T21:20:45.962616+0000","last_active":"2024-04-26T21:20:46.959372+0000","last_peered":"2024-04-26T21:20:46.959372+0000","last_clean":"2024-04-26T21:20:46.959372+0000","last_became_active":"2024-04-26T21:20:45.962318+0000","last_became_peered":"2024-04-26T21:20:45.962318+0000","last_unstale":"2024-04-26T21:20:46.959372+0000","last_undegraded":"2024-04-26T21:20:46.959372+0000","last_fullsized":"2024-04-26T21:20:46.959372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:57.812828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336229+0000","last_change":"2024-04-26T21:20:44.944630+0000","last_active":"2024-04-26T21:20:47.336229+0000","last_peered":"2024-04-26T21:20:47.336229+0000","last_clean":"2024-04-26T21:20:47.336229+0000","last_became_active":"2024-04-26T21:20:44.944271+0000","last_became_peered":"2024-04-26T21:20:44.944271+0000","last_unstale":"2024-04-26T21:20:47.336229+0000","last_undegraded":"2024-04-26T21:20:47.336229+0000","last_fullsized":"2024-04-26T21:20:47.336229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:09.997574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962808+0000","last_change":"2024-04-26T21:20:45.976347+0000","last_active":"2024-04-26T21:20:46.962808+0000","last_peered":"2024-04-26T21:20:46.962808+0000","last_clean":"2024-04-26T21:20:46.962808+0000","last_became_active":"2024-04-26T21:20:45.976080+0000","last_became_peered":"2024-04-26T21:20:45.976080+0000","last_unstale":"2024-04-26T21:20:46.962808+0000","last_undegraded":"2024-04-26T21:20:46.962808+0000","last_fullsized":"2024-04-26T21:20:46.962808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:30.154836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961487+0000","last_change":"2024-04-26T21:20:44.958470+0000","last_active":"2024-04-26T21:20:46.961487+0000","last_peered":"2024-04-26T21:20:46.961487+0000","last_clean":"2024-04-26T21:20:46.961487+0000","last_became_active":"2024-04-26T21:20:44.958191+0000","last_became_peered":"2024-04-26T21:20:44.958191+0000","last_unstale":"2024-04-26T21:20:46.961487+0000","last_undegraded":"2024-04-26T21:20:46.961487+0000","last_fullsized":"2024-04-26T21:20:46.961487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:51.907338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961496+0000","last_change":"2024-04-26T21:20:45.964088+0000","last_active":"2024-04-26T21:20:46.961496+0000","last_peered":"2024-04-26T21:20:46.961496+0000","last_clean":"2024-04-26T21:20:46.961496+0000","last_became_active":"2024-04-26T21:20:45.963798+0000","last_became_peered":"2024-04-26T21:20:45.963798+0000","last_unstale":"2024-04-26T21:20:46.961496+0000","last_undegraded":"2024-04-26T21:20:46.961496+0000","last_fullsized":"2024-04-26T21:20:46.961496+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336390+0000","last_change":"2024-04-26T21:20:44.952271+0000","last_active":"2024-04-26T21:20:47.336390+0000","last_peered":"2024-04-26T21:20:47.336390+0000","last_clean":"2024-04-26T21:20:47.336390+0000","last_became_active":"2024-04-26T21:20:44.951860+0000","last_became_peered":"2024-04-26T21:20:44.951860+0000","last_unstale":"2024-04-26T21:20:47.336390+0000","last_undegraded":"2024-04-26T21:20:47.336390+0000","last_fullsized":"2024-04-26T21:20:47.336390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962494+0000","last_change":"2024-04-26T21:20:45.969084+0000","last_active":"2024-04-26T21:20:46.962494+0000","last_peered":"2024-04-26T21:20:46.962494+0000","last_clean":"2024-04-26T21:20:46.962494+0000","last_became_active":"2024-04-26T21:20:45.968806+0000","last_became_peered":"2024-04-26T21:20:45.968806+0000","last_unstale":"2024-04-26T21:20:46.962494+0000","last_undegraded":"2024-04-26T21:20:46.962494+0000","last_fullsized":"2024-04-26T21:20:46.962494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:20.402971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335712+0000","last_change":"2024-04-26T21:20:44.955072+0000","last_active":"2024-04-26T21:20:47.335712+0000","last_peered":"2024-04-26T21:20:47.335712+0000","last_clean":"2024-04-26T21:20:47.335712+0000","last_became_active":"2024-04-26T21:20:44.954230+0000","last_became_peered":"2024-04-26T21:20:44.954230+0000","last_unstale":"2024-04-26T21:20:47.335712+0000","last_undegraded":"2024-04-26T21:20:47.335712+0000","last_fullsized":"2024-04-26T21:20:47.335712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:04.483717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336287+0000","last_change":"2024-04-26T21:20:45.964348+0000","last_active":"2024-04-26T21:20:47.336287+0000","last_peered":"2024-04-26T21:20:47.336287+0000","last_clean":"2024-04-26T21:20:47.336287+0000","last_became_active":"2024-04-26T21:20:45.964043+0000","last_became_peered":"2024-04-26T21:20:45.964043+0000","last_unstale":"2024-04-26T21:20:47.336287+0000","last_undegraded":"2024-04-26T21:20:47.336287+0000","last_fullsized":"2024-04-26T21:20:47.336287+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336648+0000","last_change":"2024-04-26T21:20:44.955598+0000","last_active":"2024-04-26T21:20:47.336648+0000","last_peered":"2024-04-26T21:20:47.336648+0000","last_clean":"2024-04-26T21:20:47.336648+0000","last_became_active":"2024-04-26T21:20:44.955310+0000","last_became_peered":"2024-04-26T21:20:44.955310+0000","last_unstale":"2024-04-26T21:20:47.336648+0000","last_undegraded":"2024-04-26T21:20:47.336648+0000","last_fullsized":"2024-04-26T21:20:47.336648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961583+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:46.961583+0000","last_peered":"2024-04-26T21:20:46.961583+0000","last_clean":"2024-04-26T21:20:46.961583+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:46.961583+0000","last_undegraded":"2024-04-26T21:20:46.961583+0000","last_fullsized":"2024-04-26T21:20:46.961583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2273280,"data_stored":1311247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1683456,"data_stored":720373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1671168,"data_stored":720227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":714017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33416,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338424,"statfs":{"total":96636764160,"available":96602546176,"internally_reserved":0,"allocated":2281472,"data_stored":1312241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36940,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334900,"statfs":{"total":96636764160,"available":96598937600,"internally_reserved":0,"allocated":1634304,"data_stored":700877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:50.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.945782+0000', 'last_change': '2024-04-26T21:20:44.945782+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.945782+0000', 'last_undegraded': '2024-04-26T21:20:44.945782+0000', 'last_fullsized': '2024-04-26T21:20:44.945782+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.943159+0000', 'last_change': '2024-04-26T21:20:44.943159+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.943159+0000', 'last_undegraded': '2024-04-26T21:20:44.943159+0000', 'last_fullsized': '2024-04-26T21:20:44.943159+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.942563+0000', 'last_change': '2024-04-26T21:20:44.942563+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.942563+0000', 'last_undegraded': '2024-04-26T21:20:44.942563+0000', 'last_fullsized': '2024-04-26T21:20:44.942563+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.946943+0000', 'last_change': '2024-04-26T21:20:44.946943+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.946943+0000', 'last_undegraded': '2024-04-26T21:20:44.946943+0000', 'last_fullsized': '2024-04-26T21:20:44.946943+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.945523+0000', 'last_change': '2024-04-26T21:20:44.945523+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.945523+0000', 'last_undegraded': '2024-04-26T21:20:44.945523+0000', 'last_fullsized': '2024-04-26T21:20:44.945523+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2024-04-26T21:20:50.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.944578+0000', 'last_change': '2024-04-26T21:20:44.944578+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.944578+0000', 'last_undegraded': '2024-04-26T21:20:44.944578+0000', 'last_fullsized': '2024-04-26T21:20:44.944578+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.943772+0000', 'last_change': '2024-04-26T21:20:44.943772+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.943772+0000', 'last_undegraded': '2024-04-26T21:20:44.943772+0000', 'last_fullsized': '2024-04-26T21:20:44.943772+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.942776+0000', 'last_change': '2024-04-26T21:20:44.942776+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.942776+0000', 'last_undegraded': '2024-04-26T21:20:44.942776+0000', 'last_fullsized': '2024-04-26T21:20:44.942776+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.946421+0000', 'last_change': '2024-04-26T21:20:44.946421+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.946421+0000', 'last_undegraded': '2024-04-26T21:20:44.946421+0000', 'last_fullsized': '2024-04-26T21:20:44.946421+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2024-04-26T21:20:50.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 132, 'state': 'creating+peering', 'last_fresh': '2024-04-26T21:20:44.945946+0000', 'last_change': '2024-04-26T21:20:44.945946+0000', 'last_active': '2024-04-26T21:20:44.930196+0000', 'last_peered': '2024-04-26T21:20:44.930196+0000', 'last_clean': '2024-04-26T21:20:44.930196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T21:20:44.945946+0000', 'last_undegraded': '2024-04-26T21:20:44.945946+0000', 'last_fullsized': '2024-04-26T21:20:44.945946+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'last_clean_scrub_stamp': '2024-04-26T21:20:44.930196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T21:20:50.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T21:20:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.823+0000 7f35a2cc8640 1 Processor -- start 2024-04-26T21:20:50.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.823+0000 7f35a2cc8640 1 -- start start 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a2cc8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f359c0d1b80 0x7f359c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a2cc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c0b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a2cc8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359c0d0220 con 0x7f359c0cfda0 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a2cc8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f359c0d0500 con 0x7f359c0d1b80 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a2cc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f359c0d07e0 con 0x7f359c0b60a0 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a0a64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f359c0d1b80 0x7f359c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a0a64640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f359c0d1b80 0x7f359c0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:59490/0 (socket says 172.21.15.40:59490) 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a1265640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c0b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f35a0a64640 1 -- 172.21.15.40:0/611647697 learned_addr learned my addr 172.21.15.40:0/611647697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.824+0000 7f359bfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f359c0cfda0 0x7f359c0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39920/0 (socket says 172.21.15.40:39920) 2024-04-26T21:20:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.825+0000 7f35a0a64640 1 -- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 msgr2=0x7f359c0b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.825+0000 7f35a0a64640 1 --2- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c0b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.825+0000 7f35a0a64640 1 -- 172.21.15.40:0/611647697 >> v1:172.21.15.184:6789/0 conn(0x7f359c0cfda0 legacy=0x7f359c0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.825+0000 7f359b7fe640 1 -- 172.21.15.40:0/611647697 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103592764 0 0) 0x7f359c0d0220 con 0x7f359c0cfda0 2024-04-26T21:20:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.825+0000 7f35a0a64640 1 -- 172.21.15.40:0/611647697 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c0d0ac0 con 0x7f359c0d1b80 2024-04-26T21:20:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a0a64640 1 --2- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0d1b80 0x7f359c0d1f60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f358c009870 tx=0x7f358c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2f40a193bc55dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f359b7fe640 1 -- 172.21.15.40:0/611647697 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c037070 con 0x7f359c0d1b80 2024-04-26T21:20:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f359b7fe640 1 -- 172.21.15.40:0/611647697 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c032440 con 0x7f359c0d1b80 2024-04-26T21:20:50.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f359b7fe640 1 -- 172.21.15.40:0/611647697 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c032810 con 0x7f359c0d1b80 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 -- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0d1b80 msgr2=0x7f359c0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0d1b80 0x7f359c0d1f60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f358c009870 tx=0x7f358c02a270 comp rx=0 tx=0).stop 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 -- 172.21.15.40:0/611647697 shutdown_connections 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c0b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/611647697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0d1b80 0x7f359c0d1f60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 -- 172.21.15.40:0/611647697 >> 172.21.15.40:0/611647697 conn(0x7f359c09e540 msgr2=0x7f359c0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 -- 172.21.15.40:0/611647697 shutdown_connections 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.826+0000 7f35a2cc8640 1 -- 172.21.15.40:0/611647697 wait complete. 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.827+0000 7f35a2cc8640 1 Processor -- start 2024-04-26T21:20:50.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.827+0000 7f35a2cc8640 1 -- start start 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 0x7f359c1f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0d4170 con 0x7f359c0d1b80 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0b6a70 con 0x7f359c0cfda0 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a2cc8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f359c15ee50 con 0x7f359c0b60a0 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.828+0000 7f35a1265640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48482/0 (socket says 172.21.15.40:48482) 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 -- 172.21.15.40:0/3443709335 learned_addr learned my addr 172.21.15.40:0/3443709335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:50.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f359bfff640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 0x7f359c1f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a0a64640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 -- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 msgr2=0x7f359c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 -- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 msgr2=0x7f359c1f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 0x7f359c1f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 -- 172.21.15.40:0/3443709335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c1f5020 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f359bfff640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 0x7f359c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a1265640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f3594009dd0 tx=0x7f359400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594018020 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3594014480 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1617c0 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594014870 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.829+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c0cba90 con 0x7f359c0d1b80 2024-04-26T21:20:50.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.830+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c06a290 con 0x7f359c0d1b80 2024-04-26T21:20:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.831+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3594005d10 con 0x7f359c0d1b80 2024-04-26T21:20:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.831+0000 7f35997fa640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 0x7f358806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.831+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f359408c520 con 0x7f359c0d1b80 2024-04-26T21:20:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.831+0000 7f35a0a64640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 0x7f358806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.832+0000 7f35a0a64640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 0x7f358806edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:50.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:50.836+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3594022070 con 0x7f359c0d1b80 2024-04-26T21:20:50.926 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-26T21:20:50.925+0000 7f48780db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:20:51.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.095+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f359c0d27b0 con 0x7f359c0d1b80 2024-04-26T21:20:51.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.095+0000 7f35997fa640 1 -- 172.21.15.40:0/3443709335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1522 (secure 0 0 0) 0x7f3594010040 con 0x7f359c0d1b80 2024-04-26T21:20:51.129 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:51.129 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165907,"num_in_osds":8,"osd_in_since":1714165898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":87},{"state_name":"creating+peering","count":10}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":278573056,"bytes_avail":772815540224,"bytes_total":773094113280,"inactive_pgs_ratio":0.10309278219938278,"write_bytes_sec":2803,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":88,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9009}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T21:20:27.041470+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T21:20:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 msgr2=0x7f358806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 0x7f358806edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02e000 comp rx=0 tx=0).stop 2024-04-26T21:20:51.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 msgr2=0x7f359c1f57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f3594009dd0 tx=0x7f359400c490 comp rx=0 tx=0).stop 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 shutdown_connections 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358806c8f0 0x7f358806edb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f359c0d1b80 0x7f359c1f57e0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f359c0cfda0 0x7f359c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 --2- 172.21.15.40:0/3443709335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f359c0b60a0 0x7f359c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 >> 172.21.15.40:0/3443709335 conn(0x7f359c09e540 msgr2=0x7f359c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 shutdown_connections 2024-04-26T21:20:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.097+0000 7f35a2cc8640 1 -- 172.21.15.40:0/3443709335 wait complete. 2024-04-26T21:20:51.131 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T21:20:51.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.231+0000 7f5577a1d640 1 Processor -- start 2024-04-26T21:20:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.231+0000 7f5577a1d640 1 -- start start 2024-04-26T21:20:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5577a1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f557015f720 0x7f55700b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5577a1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55700b2510 con 0x7f557015f340 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5577a1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55700b27f0 con 0x7f55700d5610 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5577a1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f55700b2ad0 con 0x7f557015f720 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f55757b9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f557015f340 0x7f55700d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49704/0 (socket says 172.21.15.40:49704) 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f55757b9640 1 -- 172.21.15.40:0/3016633308 learned_addr learned my addr 172.21.15.40:0/3016633308 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510751231 0 0) 0x7f55700b2510 con 0x7f557015f340 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5574fb8640 1 --2- 172.21.15.40:0/3016633308 >> v2:172.21.15.184:3300/0 conn(0x7f557015f720 0x7f55700b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.232+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f555c003520 con 0x7f557015f340 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3489514117 0 0) 0x7f555c003520 con 0x7f557015f340 2024-04-26T21:20:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 >> v2:172.21.15.184:3300/0 conn(0x7f557015f720 msgr2=0x7f55700b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 --2- 172.21.15.40:0/3016633308 >> v2:172.21.15.184:3300/0 conn(0x7f557015f720 0x7f55700b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 >> v1:172.21.15.184:6789/0 conn(0x7f55700d5610 legacy=0x7f55700b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55700b2db0 con 0x7f557015f340 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f556c001670 con 0x7f557015f340 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 >> v1:172.21.15.40:6789/0 conn(0x7f557015f340 legacy=0x7f55700d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c004800 0x7f555c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f555c0051c0 0x7f555c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55700b2ad0 con 0x7f555c0039b0 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f555c00c160 con 0x7f555c004800 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f555c00c360 con 0x7f555c0051c0 2024-04-26T21:20:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5574fb8640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c004800 0x7f555c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5575fba640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f555c0051c0 0x7f555c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 -- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f555c0051c0 msgr2=0x7f555c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f555c0051c0 0x7f555c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 -- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c004800 msgr2=0x7f555c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c004800 0x7f555c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f55757b9640 1 -- 172.21.15.40:0/3016633308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c003520 con 0x7f555c0039b0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.233+0000 7f5575fba640 1 -- 172.21.15.40:0/3016633308 reap_dead start 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.234+0000 7f55757b9640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c240 tx=0x7f555c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c76ec5c1592995 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.234+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c007d10 con 0x7f555c0039b0 2024-04-26T21:20:51.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.234+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c0058d0 con 0x7f555c0039b0 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.234+0000 7f5567fff640 1 -- 172.21.15.40:0/3016633308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c005d40 con 0x7f555c0039b0 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 -- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 msgr2=0x7f555c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c240 tx=0x7f555c005830 comp rx=0 tx=0).stop 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 -- 172.21.15.40:0/3016633308 shutdown_connections 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 --2- 172.21.15.40:0/3016633308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555c0039b0 0x7f555c003dd0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 -- 172.21.15.40:0/3016633308 >> 172.21.15.40:0/3016633308 conn(0x7f557008ec70 msgr2=0x7f557009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 -- 172.21.15.40:0/3016633308 shutdown_connections 2024-04-26T21:20:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.235+0000 7f5577a1d640 1 -- 172.21.15.40:0/3016633308 wait complete. 2024-04-26T21:20:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.236+0000 7f5577a1d640 1 Processor -- start 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.236+0000 7f5577a1d640 1 -- start start 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 0x7f557015b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 0x7f55700b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55700b2210 con 0x7f557015b970 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55700b2090 con 0x7f555c0039b0 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f55700d3d40 con 0x7f55700cd8b0 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48502/0 (socket says 172.21.15.40:48502) 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 -- 172.21.15.40:0/267143236 learned_addr learned my addr 172.21.15.40:0/267143236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f55757b9640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 0x7f557015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5575fba640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 0x7f55700b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 -- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 msgr2=0x7f55700b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 0x7f55700b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 -- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 msgr2=0x7f557015b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 0x7f557015b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 -- 172.21.15.40:0/267143236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557015bdf0 con 0x7f557015b970 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5575fba640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 0x7f55700b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f55757b9640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 0x7f557015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5574fb8640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5560011990 tx=0x7f5560011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560019070 con 0x7f557015b970 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5560004240 con 0x7f557015b970 2024-04-26T21:20:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55700cdee0 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.237+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556000f860 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.238+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557015fbc0 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.238+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557006a290 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.239+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f556001c070 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.239+0000 7f5565ffb640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 0x7f555406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.240+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f556008bf60 con 0x7f557015b970 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.240+0000 7f55757b9640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 0x7f555406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.240+0000 7f55757b9640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 0x7f555406ede0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f556c033000 tx=0x7f556c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.244+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f557015bdf0 con 0x7f557015b970 2024-04-26T21:20:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.425+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55700cdc90 con 0x7f555406c920 2024-04-26T21:20:51.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.432+0000 7f5565ffb640 1 -- 172.21.15.40:0/267143236 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f55700cdc90 con 0x7f555406c920 2024-04-26T21:20:51.436 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 msgr2=0x7f555406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 0x7f555406ede0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f556c033000 tx=0x7f556c02e000 comp rx=0 tx=0).stop 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 msgr2=0x7f55700cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5560011990 tx=0x7f5560011e50 comp rx=0 tx=0).stop 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 shutdown_connections 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f555406c920 0x7f555406ede0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55700cd8b0 0x7f55700b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f557015b970 0x7f55700cd370 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 --2- 172.21.15.40:0/267143236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f555c0039b0 0x7f557015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 >> 172.21.15.40:0/267143236 conn(0x7f557008ec70 msgr2=0x7f55700ba620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 shutdown_connections 2024-04-26T21:20:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:51.435+0000 7f5577a1d640 1 -- 172.21.15.40:0/267143236 wait complete. 2024-04-26T21:20:51.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-04-26T21:20:51.046588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272088,"kb_used_data":14268,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754702632,"statfs":{"total":773094113280,"available":772815495168,"internally_reserved":0,"allocated":14610432,"data_stored":6941072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.110247"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965835+0000","last_change":"2024-04-26T21:20:45.980429+0000","last_active":"2024-04-26T21:20:46.965835+0000","last_peered":"2024-04-26T21:20:46.965835+0000","last_clean":"2024-04-26T21:20:46.965835+0000","last_became_active":"2024-04-26T21:20:45.980082+0000","last_became_peered":"2024-04-26T21:20:45.980082+0000","last_unstale":"2024-04-26T21:20:46.965835+0000","last_undegraded":"2024-04-26T21:20:46.965835+0000","last_fullsized":"2024-04-26T21:20:46.965835+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:05.055182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336059+0000","last_change":"2024-04-26T21:20:45.965877+0000","last_active":"2024-04-26T21:20:47.336059+0000","last_peered":"2024-04-26T21:20:47.336059+0000","last_clean":"2024-04-26T21:20:47.336059+0000","last_became_active":"2024-04-26T21:20:45.965531+0000","last_became_peered":"2024-04-26T21:20:45.965531+0000","last_unstale":"2024-04-26T21:20:47.336059+0000","last_undegraded":"2024-04-26T21:20:47.336059+0000","last_fullsized":"2024-04-26T21:20:47.336059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:02.567341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336527+0000","last_change":"2024-04-26T21:20:45.972531+0000","last_active":"2024-04-26T21:20:47.336527+0000","last_peered":"2024-04-26T21:20:47.336527+0000","last_clean":"2024-04-26T21:20:47.336527+0000","last_became_active":"2024-04-26T21:20:45.972241+0000","last_became_peered":"2024-04-26T21:20:45.972241+0000","last_unstale":"2024-04-26T21:20:47.336527+0000","last_undegraded":"2024-04-26T21:20:47.336527+0000","last_fullsized":"2024-04-26T21:20:47.336527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336969+0000","last_change":"2024-04-26T21:20:45.973682+0000","last_active":"2024-04-26T21:20:47.336969+0000","last_peered":"2024-04-26T21:20:47.336969+0000","last_clean":"2024-04-26T21:20:47.336969+0000","last_became_active":"2024-04-26T21:20:45.973381+0000","last_became_peered":"2024-04-26T21:20:45.973381+0000","last_unstale":"2024-04-26T21:20:47.336969+0000","last_undegraded":"2024-04-26T21:20:47.336969+0000","last_fullsized":"2024-04-26T21:20:47.336969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:34.199283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962690+0000","last_change":"2024-04-26T21:20:45.967509+0000","last_active":"2024-04-26T21:20:46.962690+0000","last_peered":"2024-04-26T21:20:46.962690+0000","last_clean":"2024-04-26T21:20:46.962690+0000","last_became_active":"2024-04-26T21:20:45.967269+0000","last_became_peered":"2024-04-26T21:20:45.967269+0000","last_unstale":"2024-04-26T21:20:46.962690+0000","last_undegraded":"2024-04-26T21:20:46.962690+0000","last_fullsized":"2024-04-26T21:20:46.962690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.129198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965424+0000","last_change":"2024-04-26T21:20:45.972739+0000","last_active":"2024-04-26T21:20:46.965424+0000","last_peered":"2024-04-26T21:20:46.965424+0000","last_clean":"2024-04-26T21:20:46.965424+0000","last_became_active":"2024-04-26T21:20:45.972431+0000","last_became_peered":"2024-04-26T21:20:45.972431+0000","last_unstale":"2024-04-26T21:20:46.965424+0000","last_undegraded":"2024-04-26T21:20:46.965424+0000","last_fullsized":"2024-04-26T21:20:46.965424+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:11.950116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335603+0000","last_change":"2024-04-26T21:20:45.963229+0000","last_active":"2024-04-26T21:20:47.335603+0000","last_peered":"2024-04-26T21:20:47.335603+0000","last_clean":"2024-04-26T21:20:47.335603+0000","last_became_active":"2024-04-26T21:20:45.962872+0000","last_became_peered":"2024-04-26T21:20:45.962872+0000","last_unstale":"2024-04-26T21:20:47.335603+0000","last_undegraded":"2024-04-26T21:20:47.335603+0000","last_fullsized":"2024-04-26T21:20:47.335603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965205+0000","last_change":"2024-04-26T21:20:45.970806+0000","last_active":"2024-04-26T21:20:46.965205+0000","last_peered":"2024-04-26T21:20:46.965205+0000","last_clean":"2024-04-26T21:20:46.965205+0000","last_became_active":"2024-04-26T21:20:45.970485+0000","last_became_peered":"2024-04-26T21:20:45.970485+0000","last_unstale":"2024-04-26T21:20:46.965205+0000","last_undegraded":"2024-04-26T21:20:46.965205+0000","last_fullsized":"2024-04-26T21:20:46.965205+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.873993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336113+0000","last_change":"2024-04-26T21:20:45.968152+0000","last_active":"2024-04-26T21:20:47.336113+0000","last_peered":"2024-04-26T21:20:47.336113+0000","last_clean":"2024-04-26T21:20:47.336113+0000","last_became_active":"2024-04-26T21:20:45.967845+0000","last_became_peered":"2024-04-26T21:20:45.967845+0000","last_unstale":"2024-04-26T21:20:47.336113+0000","last_undegraded":"2024-04-26T21:20:47.336113+0000","last_fullsized":"2024-04-26T21:20:47.336113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.001965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961661+0000","last_change":"2024-04-26T21:20:45.977906+0000","last_active":"2024-04-26T21:20:46.961661+0000","last_peered":"2024-04-26T21:20:46.961661+0000","last_clean":"2024-04-26T21:20:46.961661+0000","last_became_active":"2024-04-26T21:20:45.977567+0000","last_became_peered":"2024-04-26T21:20:45.977567+0000","last_unstale":"2024-04-26T21:20:46.961661+0000","last_undegraded":"2024-04-26T21:20:46.961661+0000","last_fullsized":"2024-04-26T21:20:46.961661+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965364+0000","last_change":"2024-04-26T21:20:45.970225+0000","last_active":"2024-04-26T21:20:46.965364+0000","last_peered":"2024-04-26T21:20:46.965364+0000","last_clean":"2024-04-26T21:20:46.965364+0000","last_became_active":"2024-04-26T21:20:45.969734+0000","last_became_peered":"2024-04-26T21:20:45.969734+0000","last_unstale":"2024-04-26T21:20:46.965364+0000","last_undegraded":"2024-04-26T21:20:46.965364+0000","last_fullsized":"2024-04-26T21:20:46.965364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:45.806197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961702+0000","last_change":"2024-04-26T21:20:45.972783+0000","last_active":"2024-04-26T21:20:46.961702+0000","last_peered":"2024-04-26T21:20:46.961702+0000","last_clean":"2024-04-26T21:20:46.961702+0000","last_became_active":"2024-04-26T21:20:45.972522+0000","last_became_peered":"2024-04-26T21:20:45.972522+0000","last_unstale":"2024-04-26T21:20:46.961702+0000","last_undegraded":"2024-04-26T21:20:46.961702+0000","last_fullsized":"2024-04-26T21:20:46.961702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962682+0000","last_change":"2024-04-26T21:20:45.967507+0000","last_active":"2024-04-26T21:20:46.962682+0000","last_peered":"2024-04-26T21:20:46.962682+0000","last_clean":"2024-04-26T21:20:46.962682+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.962682+0000","last_undegraded":"2024-04-26T21:20:46.962682+0000","last_fullsized":"2024-04-26T21:20:46.962682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:40.380793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:45.967357+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:45.966578+0000","last_became_peered":"2024-04-26T21:20:45.966578+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961095+0000","last_change":"2024-04-26T21:20:45.974674+0000","last_active":"2024-04-26T21:20:46.961095+0000","last_peered":"2024-04-26T21:20:46.961095+0000","last_clean":"2024-04-26T21:20:46.961095+0000","last_became_active":"2024-04-26T21:20:45.974359+0000","last_became_peered":"2024-04-26T21:20:45.974359+0000","last_unstale":"2024-04-26T21:20:46.961095+0000","last_undegraded":"2024-04-26T21:20:46.961095+0000","last_fullsized":"2024-04-26T21:20:46.961095+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:49.051775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959220+0000","last_change":"2024-04-26T21:20:45.968322+0000","last_active":"2024-04-26T21:20:46.959220+0000","last_peered":"2024-04-26T21:20:46.959220+0000","last_clean":"2024-04-26T21:20:46.959220+0000","last_became_active":"2024-04-26T21:20:45.967746+0000","last_became_peered":"2024-04-26T21:20:45.967746+0000","last_unstale":"2024-04-26T21:20:46.959220+0000","last_undegraded":"2024-04-26T21:20:46.959220+0000","last_fullsized":"2024-04-26T21:20:46.959220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.881502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962071+0000","last_change":"2024-04-26T21:20:45.979392+0000","last_active":"2024-04-26T21:20:46.962071+0000","last_peered":"2024-04-26T21:20:46.962071+0000","last_clean":"2024-04-26T21:20:46.962071+0000","last_became_active":"2024-04-26T21:20:45.978894+0000","last_became_peered":"2024-04-26T21:20:45.978894+0000","last_unstale":"2024-04-26T21:20:46.962071+0000","last_undegraded":"2024-04-26T21:20:46.962071+0000","last_fullsized":"2024-04-26T21:20:46.962071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:04.038090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336798+0000","last_change":"2024-04-26T21:20:45.971458+0000","last_active":"2024-04-26T21:20:47.336798+0000","last_peered":"2024-04-26T21:20:47.336798+0000","last_clean":"2024-04-26T21:20:47.336798+0000","last_became_active":"2024-04-26T21:20:45.971186+0000","last_became_peered":"2024-04-26T21:20:45.971186+0000","last_unstale":"2024-04-26T21:20:47.336798+0000","last_undegraded":"2024-04-26T21:20:47.336798+0000","last_fullsized":"2024-04-26T21:20:47.336798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965696+0000","last_change":"2024-04-26T21:20:45.981233+0000","last_active":"2024-04-26T21:20:46.965696+0000","last_peered":"2024-04-26T21:20:46.965696+0000","last_clean":"2024-04-26T21:20:46.965696+0000","last_became_active":"2024-04-26T21:20:45.979321+0000","last_became_peered":"2024-04-26T21:20:45.979321+0000","last_unstale":"2024-04-26T21:20:46.965696+0000","last_undegraded":"2024-04-26T21:20:46.965696+0000","last_fullsized":"2024-04-26T21:20:46.965696+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:54.473013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959492+0000","last_change":"2024-04-26T21:20:45.968075+0000","last_active":"2024-04-26T21:20:46.959492+0000","last_peered":"2024-04-26T21:20:46.959492+0000","last_clean":"2024-04-26T21:20:46.959492+0000","last_became_active":"2024-04-26T21:20:45.967734+0000","last_became_peered":"2024-04-26T21:20:45.967734+0000","last_unstale":"2024-04-26T21:20:46.959492+0000","last_undegraded":"2024-04-26T21:20:46.959492+0000","last_fullsized":"2024-04-26T21:20:46.959492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.314546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961395+0000","last_change":"2024-04-26T21:20:45.977923+0000","last_active":"2024-04-26T21:20:46.961395+0000","last_peered":"2024-04-26T21:20:46.961395+0000","last_clean":"2024-04-26T21:20:46.961395+0000","last_became_active":"2024-04-26T21:20:45.977624+0000","last_became_peered":"2024-04-26T21:20:45.977624+0000","last_unstale":"2024-04-26T21:20:46.961395+0000","last_undegraded":"2024-04-26T21:20:46.961395+0000","last_fullsized":"2024-04-26T21:20:46.961395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336657+0000","last_change":"2024-04-26T21:20:45.956281+0000","last_active":"2024-04-26T21:20:47.336657+0000","last_peered":"2024-04-26T21:20:47.336657+0000","last_clean":"2024-04-26T21:20:47.336657+0000","last_became_active":"2024-04-26T21:20:45.955972+0000","last_became_peered":"2024-04-26T21:20:45.955972+0000","last_unstale":"2024-04-26T21:20:47.336657+0000","last_undegraded":"2024-04-26T21:20:47.336657+0000","last_fullsized":"2024-04-26T21:20:47.336657+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336022+0000","last_change":"2024-04-26T21:20:45.967466+0000","last_active":"2024-04-26T21:20:47.336022+0000","last_peered":"2024-04-26T21:20:47.336022+0000","last_clean":"2024-04-26T21:20:47.336022+0000","last_became_active":"2024-04-26T21:20:45.967180+0000","last_became_peered":"2024-04-26T21:20:45.967180+0000","last_unstale":"2024-04-26T21:20:47.336022+0000","last_undegraded":"2024-04-26T21:20:47.336022+0000","last_fullsized":"2024-04-26T21:20:47.336022+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961176+0000","last_change":"2024-04-26T21:20:45.966830+0000","last_active":"2024-04-26T21:20:46.961176+0000","last_peered":"2024-04-26T21:20:46.961176+0000","last_clean":"2024-04-26T21:20:46.961176+0000","last_became_active":"2024-04-26T21:20:45.966121+0000","last_became_peered":"2024-04-26T21:20:45.966121+0000","last_unstale":"2024-04-26T21:20:46.961176+0000","last_undegraded":"2024-04-26T21:20:46.961176+0000","last_fullsized":"2024-04-26T21:20:46.961176+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:50.753188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959755+0000","last_change":"2024-04-26T21:20:45.968233+0000","last_active":"2024-04-26T21:20:46.959755+0000","last_peered":"2024-04-26T21:20:46.959755+0000","last_clean":"2024-04-26T21:20:46.959755+0000","last_became_active":"2024-04-26T21:20:45.967940+0000","last_became_peered":"2024-04-26T21:20:45.967940+0000","last_unstale":"2024-04-26T21:20:46.959755+0000","last_undegraded":"2024-04-26T21:20:46.959755+0000","last_fullsized":"2024-04-26T21:20:46.959755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:34.779995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965643+0000","last_change":"2024-04-26T21:20:45.977678+0000","last_active":"2024-04-26T21:20:46.965643+0000","last_peered":"2024-04-26T21:20:46.965643+0000","last_clean":"2024-04-26T21:20:46.965643+0000","last_became_active":"2024-04-26T21:20:45.977380+0000","last_became_peered":"2024-04-26T21:20:45.977380+0000","last_unstale":"2024-04-26T21:20:46.965643+0000","last_undegraded":"2024-04-26T21:20:46.965643+0000","last_fullsized":"2024-04-26T21:20:46.965643+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:19.539549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959794+0000","last_change":"2024-04-26T21:20:45.973518+0000","last_active":"2024-04-26T21:20:46.959794+0000","last_peered":"2024-04-26T21:20:46.959794+0000","last_clean":"2024-04-26T21:20:46.959794+0000","last_became_active":"2024-04-26T21:20:45.973227+0000","last_became_peered":"2024-04-26T21:20:45.973227+0000","last_unstale":"2024-04-26T21:20:46.959794+0000","last_undegraded":"2024-04-26T21:20:46.959794+0000","last_fullsized":"2024-04-26T21:20:46.959794+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:12.310488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336254+0000","last_change":"2024-04-26T21:20:45.973958+0000","last_active":"2024-04-26T21:20:47.336254+0000","last_peered":"2024-04-26T21:20:47.336254+0000","last_clean":"2024-04-26T21:20:47.336254+0000","last_became_active":"2024-04-26T21:20:45.973675+0000","last_became_peered":"2024-04-26T21:20:45.973675+0000","last_unstale":"2024-04-26T21:20:47.336254+0000","last_undegraded":"2024-04-26T21:20:47.336254+0000","last_fullsized":"2024-04-26T21:20:47.336254+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965335+0000","last_change":"2024-04-26T21:20:45.978358+0000","last_active":"2024-04-26T21:20:46.965335+0000","last_peered":"2024-04-26T21:20:46.965335+0000","last_clean":"2024-04-26T21:20:46.965335+0000","last_became_active":"2024-04-26T21:20:45.978047+0000","last_became_peered":"2024-04-26T21:20:45.978047+0000","last_unstale":"2024-04-26T21:20:46.965335+0000","last_undegraded":"2024-04-26T21:20:46.965335+0000","last_fullsized":"2024-04-26T21:20:46.965335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:33.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336855+0000","last_change":"2024-04-26T21:20:45.970795+0000","last_active":"2024-04-26T21:20:47.336855+0000","last_peered":"2024-04-26T21:20:47.336855+0000","last_clean":"2024-04-26T21:20:47.336855+0000","last_became_active":"2024-04-26T21:20:45.970509+0000","last_became_peered":"2024-04-26T21:20:45.970509+0000","last_unstale":"2024-04-26T21:20:47.336855+0000","last_undegraded":"2024-04-26T21:20:47.336855+0000","last_fullsized":"2024-04-26T21:20:47.336855+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:30.015334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335701+0000","last_change":"2024-04-26T21:20:45.972356+0000","last_active":"2024-04-26T21:20:47.335701+0000","last_peered":"2024-04-26T21:20:47.335701+0000","last_clean":"2024-04-26T21:20:47.335701+0000","last_became_active":"2024-04-26T21:20:45.972064+0000","last_became_peered":"2024-04-26T21:20:45.972064+0000","last_unstale":"2024-04-26T21:20:47.335701+0000","last_undegraded":"2024-04-26T21:20:47.335701+0000","last_fullsized":"2024-04-26T21:20:47.335701+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:50.058209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959285+0000","last_change":"2024-04-26T21:20:45.968435+0000","last_active":"2024-04-26T21:20:46.959285+0000","last_peered":"2024-04-26T21:20:46.959285+0000","last_clean":"2024-04-26T21:20:46.959285+0000","last_became_active":"2024-04-26T21:20:45.966848+0000","last_became_peered":"2024-04-26T21:20:45.966848+0000","last_unstale":"2024-04-26T21:20:46.959285+0000","last_undegraded":"2024-04-26T21:20:46.959285+0000","last_fullsized":"2024-04-26T21:20:46.959285+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:50.873852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961713+0000","last_change":"2024-04-26T21:20:45.973734+0000","last_active":"2024-04-26T21:20:46.961713+0000","last_peered":"2024-04-26T21:20:46.961713+0000","last_clean":"2024-04-26T21:20:46.961713+0000","last_became_active":"2024-04-26T21:20:45.973472+0000","last_became_peered":"2024-04-26T21:20:45.973472+0000","last_unstale":"2024-04-26T21:20:46.961713+0000","last_undegraded":"2024-04-26T21:20:46.961713+0000","last_fullsized":"2024-04-26T21:20:46.961713+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961961+0000","last_change":"2024-04-26T21:20:44.958400+0000","last_active":"2024-04-26T21:20:46.961961+0000","last_peered":"2024-04-26T21:20:46.961961+0000","last_clean":"2024-04-26T21:20:46.961961+0000","last_became_active":"2024-04-26T21:20:44.958081+0000","last_became_peered":"2024-04-26T21:20:44.958081+0000","last_unstale":"2024-04-26T21:20:46.961961+0000","last_undegraded":"2024-04-26T21:20:46.961961+0000","last_fullsized":"2024-04-26T21:20:46.961961+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:38.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336119+0000","last_change":"2024-04-26T21:20:45.964932+0000","last_active":"2024-04-26T21:20:47.336119+0000","last_peered":"2024-04-26T21:20:47.336119+0000","last_clean":"2024-04-26T21:20:47.336119+0000","last_became_active":"2024-04-26T21:20:45.961496+0000","last_became_peered":"2024-04-26T21:20:45.961496+0000","last_unstale":"2024-04-26T21:20:47.336119+0000","last_undegraded":"2024-04-26T21:20:47.336119+0000","last_fullsized":"2024-04-26T21:20:47.336119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.885551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335890+0000","last_change":"2024-04-26T21:20:44.956115+0000","last_active":"2024-04-26T21:20:47.335890+0000","last_peered":"2024-04-26T21:20:47.335890+0000","last_clean":"2024-04-26T21:20:47.335890+0000","last_became_active":"2024-04-26T21:20:44.955813+0000","last_became_peered":"2024-04-26T21:20:44.955813+0000","last_unstale":"2024-04-26T21:20:47.335890+0000","last_undegraded":"2024-04-26T21:20:47.335890+0000","last_fullsized":"2024-04-26T21:20:47.335890+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:21.778534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965448+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:46.965448+0000","last_peered":"2024-04-26T21:20:46.965448+0000","last_clean":"2024-04-26T21:20:46.965448+0000","last_became_active":"2024-04-26T21:20:45.973396+0000","last_became_peered":"2024-04-26T21:20:45.973396+0000","last_unstale":"2024-04-26T21:20:46.965448+0000","last_undegraded":"2024-04-26T21:20:46.965448+0000","last_fullsized":"2024-04-26T21:20:46.965448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:02.776803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.949711+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.949393+0000","last_became_peered":"2024-04-26T21:20:44.949393+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961372+0000","last_change":"2024-04-26T21:20:45.964911+0000","last_active":"2024-04-26T21:20:46.961372+0000","last_peered":"2024-04-26T21:20:46.961372+0000","last_clean":"2024-04-26T21:20:46.961372+0000","last_became_active":"2024-04-26T21:20:45.963945+0000","last_became_peered":"2024-04-26T21:20:45.963945+0000","last_unstale":"2024-04-26T21:20:46.961372+0000","last_undegraded":"2024-04-26T21:20:46.961372+0000","last_fullsized":"2024-04-26T21:20:46.961372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:41.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.337000+0000","last_change":"2024-04-26T21:20:44.949483+0000","last_active":"2024-04-26T21:20:47.337000+0000","last_peered":"2024-04-26T21:20:47.337000+0000","last_clean":"2024-04-26T21:20:47.337000+0000","last_became_active":"2024-04-26T21:20:44.948183+0000","last_became_peered":"2024-04-26T21:20:44.948183+0000","last_unstale":"2024-04-26T21:20:47.337000+0000","last_undegraded":"2024-04-26T21:20:47.337000+0000","last_fullsized":"2024-04-26T21:20:47.337000+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335767+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:47.335767+0000","last_peered":"2024-04-26T21:20:47.335767+0000","last_clean":"2024-04-26T21:20:47.335767+0000","last_became_active":"2024-04-26T21:20:45.973350+0000","last_became_peered":"2024-04-26T21:20:45.973350+0000","last_unstale":"2024-04-26T21:20:47.335767+0000","last_undegraded":"2024-04-26T21:20:47.335767+0000","last_fullsized":"2024-04-26T21:20:47.335767+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.225182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959303+0000","last_change":"2024-04-26T21:20:44.954463+0000","last_active":"2024-04-26T21:20:46.959303+0000","last_peered":"2024-04-26T21:20:46.959303+0000","last_clean":"2024-04-26T21:20:46.959303+0000","last_became_active":"2024-04-26T21:20:44.954133+0000","last_became_peered":"2024-04-26T21:20:44.954133+0000","last_unstale":"2024-04-26T21:20:46.959303+0000","last_undegraded":"2024-04-26T21:20:46.959303+0000","last_fullsized":"2024-04-26T21:20:46.959303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:16.181685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336085+0000","last_change":"2024-04-26T21:20:45.978602+0000","last_active":"2024-04-26T21:20:47.336085+0000","last_peered":"2024-04-26T21:20:47.336085+0000","last_clean":"2024-04-26T21:20:47.336085+0000","last_became_active":"2024-04-26T21:20:45.978300+0000","last_became_peered":"2024-04-26T21:20:45.978300+0000","last_unstale":"2024-04-26T21:20:47.336085+0000","last_undegraded":"2024-04-26T21:20:47.336085+0000","last_fullsized":"2024-04-26T21:20:47.336085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962713+0000","last_change":"2024-04-26T21:20:44.953754+0000","last_active":"2024-04-26T21:20:46.962713+0000","last_peered":"2024-04-26T21:20:46.962713+0000","last_clean":"2024-04-26T21:20:46.962713+0000","last_became_active":"2024-04-26T21:20:44.953471+0000","last_became_peered":"2024-04-26T21:20:44.953471+0000","last_unstale":"2024-04-26T21:20:46.962713+0000","last_undegraded":"2024-04-26T21:20:46.962713+0000","last_fullsized":"2024-04-26T21:20:46.962713+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:21.877111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335605+0000","last_change":"2024-04-26T21:20:45.957101+0000","last_active":"2024-04-26T21:20:47.335605+0000","last_peered":"2024-04-26T21:20:47.335605+0000","last_clean":"2024-04-26T21:20:47.335605+0000","last_became_active":"2024-04-26T21:20:45.956746+0000","last_became_peered":"2024-04-26T21:20:45.956746+0000","last_unstale":"2024-04-26T21:20:47.335605+0000","last_undegraded":"2024-04-26T21:20:47.335605+0000","last_fullsized":"2024-04-26T21:20:47.335605+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:20.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959438+0000","last_change":"2024-04-26T21:20:44.945604+0000","last_active":"2024-04-26T21:20:46.959438+0000","last_peered":"2024-04-26T21:20:46.959438+0000","last_clean":"2024-04-26T21:20:46.959438+0000","last_became_active":"2024-04-26T21:20:44.945311+0000","last_became_peered":"2024-04-26T21:20:44.945311+0000","last_unstale":"2024-04-26T21:20:46.959438+0000","last_undegraded":"2024-04-26T21:20:46.959438+0000","last_fullsized":"2024-04-26T21:20:46.959438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:59.997109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962422+0000","last_change":"2024-04-26T21:20:45.968415+0000","last_active":"2024-04-26T21:20:46.962422+0000","last_peered":"2024-04-26T21:20:46.962422+0000","last_clean":"2024-04-26T21:20:46.962422+0000","last_became_active":"2024-04-26T21:20:45.968148+0000","last_became_peered":"2024-04-26T21:20:45.968148+0000","last_unstale":"2024-04-26T21:20:46.962422+0000","last_undegraded":"2024-04-26T21:20:46.962422+0000","last_fullsized":"2024-04-26T21:20:46.962422+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:30.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335772+0000","last_change":"2024-04-26T21:20:44.953539+0000","last_active":"2024-04-26T21:20:47.335772+0000","last_peered":"2024-04-26T21:20:47.335772+0000","last_clean":"2024-04-26T21:20:47.335772+0000","last_became_active":"2024-04-26T21:20:44.953286+0000","last_became_peered":"2024-04-26T21:20:44.953286+0000","last_unstale":"2024-04-26T21:20:47.335772+0000","last_undegraded":"2024-04-26T21:20:47.335772+0000","last_fullsized":"2024-04-26T21:20:47.335772+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962742+0000","last_change":"2024-04-26T21:20:45.957232+0000","last_active":"2024-04-26T21:20:46.962742+0000","last_peered":"2024-04-26T21:20:46.962742+0000","last_clean":"2024-04-26T21:20:46.962742+0000","last_became_active":"2024-04-26T21:20:45.956611+0000","last_became_peered":"2024-04-26T21:20:45.956611+0000","last_unstale":"2024-04-26T21:20:46.962742+0000","last_undegraded":"2024-04-26T21:20:46.962742+0000","last_fullsized":"2024-04-26T21:20:46.962742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:13.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336218+0000","last_change":"2024-04-26T21:20:44.952646+0000","last_active":"2024-04-26T21:20:47.336218+0000","last_peered":"2024-04-26T21:20:47.336218+0000","last_clean":"2024-04-26T21:20:47.336218+0000","last_became_active":"2024-04-26T21:20:44.952351+0000","last_became_peered":"2024-04-26T21:20:44.952351+0000","last_unstale":"2024-04-26T21:20:47.336218+0000","last_undegraded":"2024-04-26T21:20:47.336218+0000","last_fullsized":"2024-04-26T21:20:47.336218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:33.194892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961915+0000","last_change":"2024-04-26T21:20:45.979158+0000","last_active":"2024-04-26T21:20:46.961915+0000","last_peered":"2024-04-26T21:20:46.961915+0000","last_clean":"2024-04-26T21:20:46.961915+0000","last_became_active":"2024-04-26T21:20:45.978509+0000","last_became_peered":"2024-04-26T21:20:45.978509+0000","last_unstale":"2024-04-26T21:20:46.961915+0000","last_undegraded":"2024-04-26T21:20:46.961915+0000","last_fullsized":"2024-04-26T21:20:46.961915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:18.599945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961941+0000","last_change":"2024-04-26T21:20:44.958683+0000","last_active":"2024-04-26T21:20:46.961941+0000","last_peered":"2024-04-26T21:20:46.961941+0000","last_clean":"2024-04-26T21:20:46.961941+0000","last_became_active":"2024-04-26T21:20:44.958250+0000","last_became_peered":"2024-04-26T21:20:44.958250+0000","last_unstale":"2024-04-26T21:20:46.961941+0000","last_undegraded":"2024-04-26T21:20:46.961941+0000","last_fullsized":"2024-04-26T21:20:46.961941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:19.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959625+0000","last_change":"2024-04-26T21:20:45.968143+0000","last_active":"2024-04-26T21:20:46.959625+0000","last_peered":"2024-04-26T21:20:46.959625+0000","last_clean":"2024-04-26T21:20:46.959625+0000","last_became_active":"2024-04-26T21:20:45.966158+0000","last_became_peered":"2024-04-26T21:20:45.966158+0000","last_unstale":"2024-04-26T21:20:46.959625+0000","last_undegraded":"2024-04-26T21:20:46.959625+0000","last_fullsized":"2024-04-26T21:20:46.959625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:10.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336377+0000","last_change":"2024-04-26T21:20:44.951683+0000","last_active":"2024-04-26T21:20:47.336377+0000","last_peered":"2024-04-26T21:20:47.336377+0000","last_clean":"2024-04-26T21:20:47.336377+0000","last_became_active":"2024-04-26T21:20:44.951213+0000","last_became_peered":"2024-04-26T21:20:44.951213+0000","last_unstale":"2024-04-26T21:20:47.336377+0000","last_undegraded":"2024-04-26T21:20:47.336377+0000","last_fullsized":"2024-04-26T21:20:47.336377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:36.881160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336537+0000","last_change":"2024-04-26T21:20:45.968505+0000","last_active":"2024-04-26T21:20:47.336537+0000","last_peered":"2024-04-26T21:20:47.336537+0000","last_clean":"2024-04-26T21:20:47.336537+0000","last_became_active":"2024-04-26T21:20:45.968185+0000","last_became_peered":"2024-04-26T21:20:45.968185+0000","last_unstale":"2024-04-26T21:20:47.336537+0000","last_undegraded":"2024-04-26T21:20:47.336537+0000","last_fullsized":"2024-04-26T21:20:47.336537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336545+0000","last_change":"2024-04-26T21:20:44.951050+0000","last_active":"2024-04-26T21:20:47.336545+0000","last_peered":"2024-04-26T21:20:47.336545+0000","last_clean":"2024-04-26T21:20:47.336545+0000","last_became_active":"2024-04-26T21:20:44.950758+0000","last_became_peered":"2024-04-26T21:20:44.950758+0000","last_unstale":"2024-04-26T21:20:47.336545+0000","last_undegraded":"2024-04-26T21:20:47.336545+0000","last_fullsized":"2024-04-26T21:20:47.336545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:07.338750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335606+0000","last_change":"2024-04-26T21:20:45.965874+0000","last_active":"2024-04-26T21:20:47.335606+0000","last_peered":"2024-04-26T21:20:47.335606+0000","last_clean":"2024-04-26T21:20:47.335606+0000","last_became_active":"2024-04-26T21:20:45.965561+0000","last_became_peered":"2024-04-26T21:20:45.965561+0000","last_unstale":"2024-04-26T21:20:47.335606+0000","last_undegraded":"2024-04-26T21:20:47.335606+0000","last_fullsized":"2024-04-26T21:20:47.335606+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:00.164990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:44.949774+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:44.949464+0000","last_became_peered":"2024-04-26T21:20:44.949464+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962548+0000","last_change":"2024-04-26T21:20:45.965418+0000","last_active":"2024-04-26T21:20:46.962548+0000","last_peered":"2024-04-26T21:20:46.962548+0000","last_clean":"2024-04-26T21:20:46.962548+0000","last_became_active":"2024-04-26T21:20:45.965125+0000","last_became_peered":"2024-04-26T21:20:45.965125+0000","last_unstale":"2024-04-26T21:20:46.962548+0000","last_undegraded":"2024-04-26T21:20:46.962548+0000","last_fullsized":"2024-04-26T21:20:46.962548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:47.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965231+0000","last_change":"2024-04-26T21:20:44.952556+0000","last_active":"2024-04-26T21:20:46.965231+0000","last_peered":"2024-04-26T21:20:46.965231+0000","last_clean":"2024-04-26T21:20:46.965231+0000","last_became_active":"2024-04-26T21:20:44.952257+0000","last_became_peered":"2024-04-26T21:20:44.952257+0000","last_unstale":"2024-04-26T21:20:46.965231+0000","last_undegraded":"2024-04-26T21:20:46.965231+0000","last_fullsized":"2024-04-26T21:20:46.965231+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:49.597526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336273+0000","last_change":"2024-04-26T21:20:45.971847+0000","last_active":"2024-04-26T21:20:47.336273+0000","last_peered":"2024-04-26T21:20:47.336273+0000","last_clean":"2024-04-26T21:20:47.336273+0000","last_became_active":"2024-04-26T21:20:45.971310+0000","last_became_peered":"2024-04-26T21:20:45.971310+0000","last_unstale":"2024-04-26T21:20:47.336273+0000","last_undegraded":"2024-04-26T21:20:47.336273+0000","last_fullsized":"2024-04-26T21:20:47.336273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.464436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959325+0000","last_change":"2024-04-26T21:20:44.950519+0000","last_active":"2024-04-26T21:20:46.959325+0000","last_peered":"2024-04-26T21:20:46.959325+0000","last_clean":"2024-04-26T21:20:46.959325+0000","last_became_active":"2024-04-26T21:20:44.950288+0000","last_became_peered":"2024-04-26T21:20:44.950288+0000","last_unstale":"2024-04-26T21:20:46.959325+0000","last_undegraded":"2024-04-26T21:20:46.959325+0000","last_fullsized":"2024-04-26T21:20:46.959325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:07.328646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961844+0000","last_change":"2024-04-26T21:20:45.968413+0000","last_active":"2024-04-26T21:20:46.961844+0000","last_peered":"2024-04-26T21:20:46.961844+0000","last_clean":"2024-04-26T21:20:46.961844+0000","last_became_active":"2024-04-26T21:20:45.968149+0000","last_became_peered":"2024-04-26T21:20:45.968149+0000","last_unstale":"2024-04-26T21:20:46.961844+0000","last_undegraded":"2024-04-26T21:20:46.961844+0000","last_fullsized":"2024-04-26T21:20:46.961844+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:11.931969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962569+0000","last_change":"2024-04-26T21:20:44.952076+0000","last_active":"2024-04-26T21:20:46.962569+0000","last_peered":"2024-04-26T21:20:46.962569+0000","last_clean":"2024-04-26T21:20:46.962569+0000","last_became_active":"2024-04-26T21:20:44.950995+0000","last_became_peered":"2024-04-26T21:20:44.950995+0000","last_unstale":"2024-04-26T21:20:46.962569+0000","last_undegraded":"2024-04-26T21:20:46.962569+0000","last_fullsized":"2024-04-26T21:20:46.962569+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:49.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959745+0000","last_change":"2024-04-26T21:20:45.967568+0000","last_active":"2024-04-26T21:20:46.959745+0000","last_peered":"2024-04-26T21:20:46.959745+0000","last_clean":"2024-04-26T21:20:46.959745+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.959745+0000","last_undegraded":"2024-04-26T21:20:46.959745+0000","last_fullsized":"2024-04-26T21:20:46.959745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:52.037830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336020+0000","last_change":"2024-04-26T21:20:44.949326+0000","last_active":"2024-04-26T21:20:47.336020+0000","last_peered":"2024-04-26T21:20:47.336020+0000","last_clean":"2024-04-26T21:20:47.336020+0000","last_became_active":"2024-04-26T21:20:44.948970+0000","last_became_peered":"2024-04-26T21:20:44.948970+0000","last_unstale":"2024-04-26T21:20:47.336020+0000","last_undegraded":"2024-04-26T21:20:47.336020+0000","last_fullsized":"2024-04-26T21:20:47.336020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.645489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336403+0000","last_change":"2024-04-26T21:20:45.967295+0000","last_active":"2024-04-26T21:20:47.336403+0000","last_peered":"2024-04-26T21:20:47.336403+0000","last_clean":"2024-04-26T21:20:47.336403+0000","last_became_active":"2024-04-26T21:20:45.966533+0000","last_became_peered":"2024-04-26T21:20:45.966533+0000","last_unstale":"2024-04-26T21:20:47.336403+0000","last_undegraded":"2024-04-26T21:20:47.336403+0000","last_fullsized":"2024-04-26T21:20:47.336403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959724+0000","last_change":"2024-04-26T21:20:44.953637+0000","last_active":"2024-04-26T21:20:46.959724+0000","last_peered":"2024-04-26T21:20:46.959724+0000","last_clean":"2024-04-26T21:20:46.959724+0000","last_became_active":"2024-04-26T21:20:44.953364+0000","last_became_peered":"2024-04-26T21:20:44.953364+0000","last_unstale":"2024-04-26T21:20:46.959724+0000","last_undegraded":"2024-04-26T21:20:46.959724+0000","last_fullsized":"2024-04-26T21:20:46.959724+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:04.043064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965518+0000","last_change":"2024-04-26T21:20:45.977550+0000","last_active":"2024-04-26T21:20:46.965518+0000","last_peered":"2024-04-26T21:20:46.965518+0000","last_clean":"2024-04-26T21:20:46.965518+0000","last_became_active":"2024-04-26T21:20:45.977244+0000","last_became_peered":"2024-04-26T21:20:45.977244+0000","last_unstale":"2024-04-26T21:20:46.965518+0000","last_undegraded":"2024-04-26T21:20:46.965518+0000","last_fullsized":"2024-04-26T21:20:46.965518+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:36.978003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336034+0000","last_change":"2024-04-26T21:20:44.950539+0000","last_active":"2024-04-26T21:20:47.336034+0000","last_peered":"2024-04-26T21:20:47.336034+0000","last_clean":"2024-04-26T21:20:47.336034+0000","last_became_active":"2024-04-26T21:20:44.950262+0000","last_became_peered":"2024-04-26T21:20:44.950262+0000","last_unstale":"2024-04-26T21:20:47.336034+0000","last_undegraded":"2024-04-26T21:20:47.336034+0000","last_fullsized":"2024-04-26T21:20:47.336034+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:19.053104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336857+0000","last_change":"2024-04-26T21:20:45.970609+0000","last_active":"2024-04-26T21:20:47.336857+0000","last_peered":"2024-04-26T21:20:47.336857+0000","last_clean":"2024-04-26T21:20:47.336857+0000","last_became_active":"2024-04-26T21:20:45.970312+0000","last_became_peered":"2024-04-26T21:20:45.970312+0000","last_unstale":"2024-04-26T21:20:47.336857+0000","last_undegraded":"2024-04-26T21:20:47.336857+0000","last_fullsized":"2024-04-26T21:20:47.336857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336100+0000","last_change":"2024-04-26T21:20:44.949930+0000","last_active":"2024-04-26T21:20:47.336100+0000","last_peered":"2024-04-26T21:20:47.336100+0000","last_clean":"2024-04-26T21:20:47.336100+0000","last_became_active":"2024-04-26T21:20:44.949603+0000","last_became_peered":"2024-04-26T21:20:44.949603+0000","last_unstale":"2024-04-26T21:20:47.336100+0000","last_undegraded":"2024-04-26T21:20:47.336100+0000","last_fullsized":"2024-04-26T21:20:47.336100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:01.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335637+0000","last_change":"2024-04-26T21:20:45.964537+0000","last_active":"2024-04-26T21:20:47.335637+0000","last_peered":"2024-04-26T21:20:47.335637+0000","last_clean":"2024-04-26T21:20:47.335637+0000","last_became_active":"2024-04-26T21:20:45.964253+0000","last_became_peered":"2024-04-26T21:20:45.964253+0000","last_unstale":"2024-04-26T21:20:47.335637+0000","last_undegraded":"2024-04-26T21:20:47.335637+0000","last_fullsized":"2024-04-26T21:20:47.335637+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961394+0000","last_change":"2024-04-26T21:20:44.958433+0000","last_active":"2024-04-26T21:20:46.961394+0000","last_peered":"2024-04-26T21:20:46.961394+0000","last_clean":"2024-04-26T21:20:46.961394+0000","last_became_active":"2024-04-26T21:20:44.958109+0000","last_became_peered":"2024-04-26T21:20:44.958109+0000","last_unstale":"2024-04-26T21:20:46.961394+0000","last_undegraded":"2024-04-26T21:20:46.961394+0000","last_fullsized":"2024-04-26T21:20:46.961394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:56.399692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335976+0000","last_change":"2024-04-26T21:20:45.963605+0000","last_active":"2024-04-26T21:20:47.335976+0000","last_peered":"2024-04-26T21:20:47.335976+0000","last_clean":"2024-04-26T21:20:47.335976+0000","last_became_active":"2024-04-26T21:20:45.963047+0000","last_became_peered":"2024-04-26T21:20:45.963047+0000","last_unstale":"2024-04-26T21:20:47.335976+0000","last_undegraded":"2024-04-26T21:20:47.335976+0000","last_fullsized":"2024-04-26T21:20:47.335976+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961119+0000","last_change":"2024-04-26T21:20:44.957474+0000","last_active":"2024-04-26T21:20:46.961119+0000","last_peered":"2024-04-26T21:20:46.961119+0000","last_clean":"2024-04-26T21:20:46.961119+0000","last_became_active":"2024-04-26T21:20:44.957134+0000","last_became_peered":"2024-04-26T21:20:44.957134+0000","last_unstale":"2024-04-26T21:20:46.961119+0000","last_undegraded":"2024-04-26T21:20:46.961119+0000","last_fullsized":"2024-04-26T21:20:46.961119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:50.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335784+0000","last_change":"2024-04-26T21:20:45.961150+0000","last_active":"2024-04-26T21:20:47.335784+0000","last_peered":"2024-04-26T21:20:47.335784+0000","last_clean":"2024-04-26T21:20:47.335784+0000","last_became_active":"2024-04-26T21:20:45.960842+0000","last_became_peered":"2024-04-26T21:20:45.960842+0000","last_unstale":"2024-04-26T21:20:47.335784+0000","last_undegraded":"2024-04-26T21:20:47.335784+0000","last_fullsized":"2024-04-26T21:20:47.335784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.951589+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.950834+0000","last_became_peered":"2024-04-26T21:20:44.950834+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:51.927771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962749+0000","last_change":"2024-04-26T21:20:45.967744+0000","last_active":"2024-04-26T21:20:46.962749+0000","last_peered":"2024-04-26T21:20:46.962749+0000","last_clean":"2024-04-26T21:20:46.962749+0000","last_became_active":"2024-04-26T21:20:45.967409+0000","last_became_peered":"2024-04-26T21:20:45.967409+0000","last_unstale":"2024-04-26T21:20:46.962749+0000","last_undegraded":"2024-04-26T21:20:46.962749+0000","last_fullsized":"2024-04-26T21:20:46.962749+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:12.684400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962057+0000","last_change":"2024-04-26T21:20:44.958845+0000","last_active":"2024-04-26T21:20:46.962057+0000","last_peered":"2024-04-26T21:20:46.962057+0000","last_clean":"2024-04-26T21:20:46.962057+0000","last_became_active":"2024-04-26T21:20:44.958476+0000","last_became_peered":"2024-04-26T21:20:44.958476+0000","last_unstale":"2024-04-26T21:20:46.962057+0000","last_undegraded":"2024-04-26T21:20:46.962057+0000","last_fullsized":"2024-04-26T21:20:46.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:53.203282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336659+0000","last_change":"2024-04-26T21:20:45.967883+0000","last_active":"2024-04-26T21:20:47.336659+0000","last_peered":"2024-04-26T21:20:47.336659+0000","last_clean":"2024-04-26T21:20:47.336659+0000","last_became_active":"2024-04-26T21:20:45.967589+0000","last_became_peered":"2024-04-26T21:20:45.967589+0000","last_unstale":"2024-04-26T21:20:47.336659+0000","last_undegraded":"2024-04-26T21:20:47.336659+0000","last_fullsized":"2024-04-26T21:20:47.336659+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"134'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965739+0000","last_change":"2024-04-26T21:20:44.955921+0000","last_active":"2024-04-26T21:20:46.965739+0000","last_peered":"2024-04-26T21:20:46.965739+0000","last_clean":"2024-04-26T21:20:46.965739+0000","last_became_active":"2024-04-26T21:20:44.955598+0000","last_became_peered":"2024-04-26T21:20:44.955598+0000","last_unstale":"2024-04-26T21:20:46.965739+0000","last_undegraded":"2024-04-26T21:20:46.965739+0000","last_fullsized":"2024-04-26T21:20:46.965739+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:53.281999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335972+0000","last_change":"2024-04-26T21:20:45.958489+0000","last_active":"2024-04-26T21:20:47.335972+0000","last_peered":"2024-04-26T21:20:47.335972+0000","last_clean":"2024-04-26T21:20:47.335972+0000","last_became_active":"2024-04-26T21:20:45.958208+0000","last_became_peered":"2024-04-26T21:20:45.958208+0000","last_unstale":"2024-04-26T21:20:47.335972+0000","last_undegraded":"2024-04-26T21:20:47.335972+0000","last_fullsized":"2024-04-26T21:20:47.335972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:32.662963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965626+0000","last_change":"2024-04-26T21:20:44.951269+0000","last_active":"2024-04-26T21:20:46.965626+0000","last_peered":"2024-04-26T21:20:46.965626+0000","last_clean":"2024-04-26T21:20:46.965626+0000","last_became_active":"2024-04-26T21:20:44.950960+0000","last_became_peered":"2024-04-26T21:20:44.950960+0000","last_unstale":"2024-04-26T21:20:46.965626+0000","last_undegraded":"2024-04-26T21:20:46.965626+0000","last_fullsized":"2024-04-26T21:20:46.965626+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:50.430297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965534+0000","last_change":"2024-04-26T21:20:45.975184+0000","last_active":"2024-04-26T21:20:46.965534+0000","last_peered":"2024-04-26T21:20:46.965534+0000","last_clean":"2024-04-26T21:20:46.965534+0000","last_became_active":"2024-04-26T21:20:45.974458+0000","last_became_peered":"2024-04-26T21:20:45.974458+0000","last_unstale":"2024-04-26T21:20:46.965534+0000","last_undegraded":"2024-04-26T21:20:46.965534+0000","last_fullsized":"2024-04-26T21:20:46.965534+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:31.050443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965377+0000","last_change":"2024-04-26T21:20:44.952291+0000","last_active":"2024-04-26T21:20:46.965377+0000","last_peered":"2024-04-26T21:20:46.965377+0000","last_clean":"2024-04-26T21:20:46.965377+0000","last_became_active":"2024-04-26T21:20:44.951911+0000","last_became_peered":"2024-04-26T21:20:44.951911+0000","last_unstale":"2024-04-26T21:20:46.965377+0000","last_undegraded":"2024-04-26T21:20:46.965377+0000","last_fullsized":"2024-04-26T21:20:46.965377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:10.339949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959372+0000","last_change":"2024-04-26T21:20:45.962616+0000","last_active":"2024-04-26T21:20:46.959372+0000","last_peered":"2024-04-26T21:20:46.959372+0000","last_clean":"2024-04-26T21:20:46.959372+0000","last_became_active":"2024-04-26T21:20:45.962318+0000","last_became_peered":"2024-04-26T21:20:45.962318+0000","last_unstale":"2024-04-26T21:20:46.959372+0000","last_undegraded":"2024-04-26T21:20:46.959372+0000","last_fullsized":"2024-04-26T21:20:46.959372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:57.812828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336229+0000","last_change":"2024-04-26T21:20:44.944630+0000","last_active":"2024-04-26T21:20:47.336229+0000","last_peered":"2024-04-26T21:20:47.336229+0000","last_clean":"2024-04-26T21:20:47.336229+0000","last_became_active":"2024-04-26T21:20:44.944271+0000","last_became_peered":"2024-04-26T21:20:44.944271+0000","last_unstale":"2024-04-26T21:20:47.336229+0000","last_undegraded":"2024-04-26T21:20:47.336229+0000","last_fullsized":"2024-04-26T21:20:47.336229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:09.997574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962808+0000","last_change":"2024-04-26T21:20:45.976347+0000","last_active":"2024-04-26T21:20:46.962808+0000","last_peered":"2024-04-26T21:20:46.962808+0000","last_clean":"2024-04-26T21:20:46.962808+0000","last_became_active":"2024-04-26T21:20:45.976080+0000","last_became_peered":"2024-04-26T21:20:45.976080+0000","last_unstale":"2024-04-26T21:20:46.962808+0000","last_undegraded":"2024-04-26T21:20:46.962808+0000","last_fullsized":"2024-04-26T21:20:46.962808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:30.154836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961487+0000","last_change":"2024-04-26T21:20:44.958470+0000","last_active":"2024-04-26T21:20:46.961487+0000","last_peered":"2024-04-26T21:20:46.961487+0000","last_clean":"2024-04-26T21:20:46.961487+0000","last_became_active":"2024-04-26T21:20:44.958191+0000","last_became_peered":"2024-04-26T21:20:44.958191+0000","last_unstale":"2024-04-26T21:20:46.961487+0000","last_undegraded":"2024-04-26T21:20:46.961487+0000","last_fullsized":"2024-04-26T21:20:46.961487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:51.907338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961496+0000","last_change":"2024-04-26T21:20:45.964088+0000","last_active":"2024-04-26T21:20:46.961496+0000","last_peered":"2024-04-26T21:20:46.961496+0000","last_clean":"2024-04-26T21:20:46.961496+0000","last_became_active":"2024-04-26T21:20:45.963798+0000","last_became_peered":"2024-04-26T21:20:45.963798+0000","last_unstale":"2024-04-26T21:20:46.961496+0000","last_undegraded":"2024-04-26T21:20:46.961496+0000","last_fullsized":"2024-04-26T21:20:46.961496+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336390+0000","last_change":"2024-04-26T21:20:44.952271+0000","last_active":"2024-04-26T21:20:47.336390+0000","last_peered":"2024-04-26T21:20:47.336390+0000","last_clean":"2024-04-26T21:20:47.336390+0000","last_became_active":"2024-04-26T21:20:44.951860+0000","last_became_peered":"2024-04-26T21:20:44.951860+0000","last_unstale":"2024-04-26T21:20:47.336390+0000","last_undegraded":"2024-04-26T21:20:47.336390+0000","last_fullsized":"2024-04-26T21:20:47.336390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962494+0000","last_change":"2024-04-26T21:20:45.969084+0000","last_active":"2024-04-26T21:20:46.962494+0000","last_peered":"2024-04-26T21:20:46.962494+0000","last_clean":"2024-04-26T21:20:46.962494+0000","last_became_active":"2024-04-26T21:20:45.968806+0000","last_became_peered":"2024-04-26T21:20:45.968806+0000","last_unstale":"2024-04-26T21:20:46.962494+0000","last_undegraded":"2024-04-26T21:20:46.962494+0000","last_fullsized":"2024-04-26T21:20:46.962494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:20.402971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335712+0000","last_change":"2024-04-26T21:20:44.955072+0000","last_active":"2024-04-26T21:20:47.335712+0000","last_peered":"2024-04-26T21:20:47.335712+0000","last_clean":"2024-04-26T21:20:47.335712+0000","last_became_active":"2024-04-26T21:20:44.954230+0000","last_became_peered":"2024-04-26T21:20:44.954230+0000","last_unstale":"2024-04-26T21:20:47.335712+0000","last_undegraded":"2024-04-26T21:20:47.335712+0000","last_fullsized":"2024-04-26T21:20:47.335712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:04.483717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336287+0000","last_change":"2024-04-26T21:20:45.964348+0000","last_active":"2024-04-26T21:20:47.336287+0000","last_peered":"2024-04-26T21:20:47.336287+0000","last_clean":"2024-04-26T21:20:47.336287+0000","last_became_active":"2024-04-26T21:20:45.964043+0000","last_became_peered":"2024-04-26T21:20:45.964043+0000","last_unstale":"2024-04-26T21:20:47.336287+0000","last_undegraded":"2024-04-26T21:20:47.336287+0000","last_fullsized":"2024-04-26T21:20:47.336287+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336648+0000","last_change":"2024-04-26T21:20:44.955598+0000","last_active":"2024-04-26T21:20:47.336648+0000","last_peered":"2024-04-26T21:20:47.336648+0000","last_clean":"2024-04-26T21:20:47.336648+0000","last_became_active":"2024-04-26T21:20:44.955310+0000","last_became_peered":"2024-04-26T21:20:44.955310+0000","last_unstale":"2024-04-26T21:20:47.336648+0000","last_undegraded":"2024-04-26T21:20:47.336648+0000","last_fullsized":"2024-04-26T21:20:47.336648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961583+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:46.961583+0000","last_peered":"2024-04-26T21:20:46.961583+0000","last_clean":"2024-04-26T21:20:46.961583+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:46.961583+0000","last_undegraded":"2024-04-26T21:20:46.961583+0000","last_fullsized":"2024-04-26T21:20:46.961583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2273280,"data_stored":1311247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1683456,"data_stored":720373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1671168,"data_stored":720227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":714017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33416,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338424,"statfs":{"total":96636764160,"available":96602546176,"internally_reserved":0,"allocated":2281472,"data_stored":1312241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1675264,"data_stored":720809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:54.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:20:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.564+0000 7f06d955f640 1 Processor -- start 2024-04-26T21:20:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.564+0000 7f06d955f640 1 -- start start 2024-04-26T21:20:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d955f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06d415d380 0x7f06d40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d955f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06d40b56d0 0x7f06d40d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d955f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d40d0be0 con 0x7f06d4160d50 2024-04-26T21:20:54.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d955f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40d0ec0 con 0x7f06d40b56d0 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d955f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40d11a0 con 0x7f06d415d380 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06d40b56d0 0x7f06d40d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06d40b56d0 0x7f06d40d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48504/0 (socket says 172.21.15.40:48504) 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 -- 172.21.15.40:0/2830076567 learned_addr learned my addr 172.21.15.40:0/2830076567 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d2ffd640 1 --2- 172.21.15.40:0/2830076567 >> v2:172.21.15.184:3300/0 conn(0x7f06d415d380 0x7f06d40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 -- 172.21.15.40:0/2830076567 >> v2:172.21.15.184:3300/0 conn(0x7f06d415d380 msgr2=0x7f06d40ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 --2- 172.21.15.40:0/2830076567 >> v2:172.21.15.184:3300/0 conn(0x7f06d415d380 0x7f06d40ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 -- 172.21.15.40:0/2830076567 >> v1:172.21.15.184:6789/0 conn(0x7f06d4160d50 legacy=0x7f06d40b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.565+0000 7f06d27fc640 1 -- 172.21.15.40:0/2830076567 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d40d1480 con 0x7f06d40b56d0 2024-04-26T21:20:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.566+0000 7f06d27fc640 1 --2- 172.21.15.40:0/2830076567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d40b56d0 0x7f06d40d19a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f06d4073bf0 tx=0x7f06c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f07c20e508cdb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.566+0000 7f06d1ffb640 1 -- 172.21.15.40:0/2830076567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8037070 con 0x7f06d40b56d0 2024-04-26T21:20:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.566+0000 7f06d1ffb640 1 -- 172.21.15.40:0/2830076567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c8032440 con 0x7f06d40b56d0 2024-04-26T21:20:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.566+0000 7f06d1ffb640 1 -- 172.21.15.40:0/2830076567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8032810 con 0x7f06d40b56d0 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 -- 172.21.15.40:0/2830076567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d40b56d0 msgr2=0x7f06d40d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 --2- 172.21.15.40:0/2830076567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d40b56d0 0x7f06d40d19a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f06d4073bf0 tx=0x7f06c802a230 comp rx=0 tx=0).stop 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 -- 172.21.15.40:0/2830076567 shutdown_connections 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 --2- 172.21.15.40:0/2830076567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d40b56d0 0x7f06d40d19a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 --2- 172.21.15.40:0/2830076567 >> v2:172.21.15.184:3300/0 conn(0x7f06d415d380 0x7f06d40ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 -- 172.21.15.40:0/2830076567 >> 172.21.15.40:0/2830076567 conn(0x7f06d40b4a60 msgr2=0x7f06d40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 -- 172.21.15.40:0/2830076567 shutdown_connections 2024-04-26T21:20:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.567+0000 7f06d955f640 1 -- 172.21.15.40:0/2830076567 wait complete. 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.568+0000 7f06d955f640 1 Processor -- start 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.568+0000 7f06d955f640 1 -- start start 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 0x7f06d414c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 0x7f06d409f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40cedc0 con 0x7f06d4160d50 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40d1ee0 con 0x7f06d40b56d0 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d955f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06d40b64d0 con 0x7f06d415d380 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48506/0 (socket says 172.21.15.40:48506) 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d37fe640 1 -- 172.21.15.40:0/4102915057 learned_addr learned my addr 172.21.15.40:0/4102915057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d2ffd640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 0x7f06d414c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d27fc640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 0x7f06d409f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d37fe640 1 -- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 msgr2=0x7f06d409f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.569+0000 7f06d37fe640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 0x7f06d409f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d37fe640 1 -- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 msgr2=0x7f06d414c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d37fe640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 0x7f06d414c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d37fe640 1 -- 172.21.15.40:0/4102915057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d414c970 con 0x7f06d4160d50 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d2ffd640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 0x7f06d414c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d27fc640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 0x7f06d409f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d37fe640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b970 tx=0x7f06cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc019070 con 0x7f06d4160d50 2024-04-26T21:20:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc00cb40 con 0x7f06d4160d50 2024-04-26T21:20:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d40a02e0 con 0x7f06d4160d50 2024-04-26T21:20:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc014440 con 0x7f06d4160d50 2024-04-26T21:20:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.570+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d40a08a0 con 0x7f06d4160d50 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.571+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d406a290 con 0x7f06d4160d50 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.572+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06cc01c070 con 0x7f06d4160d50 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.572+0000 7f06bb7fe640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 0x7f06c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.572+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f06cc08b6b0 con 0x7f06d4160d50 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.572+0000 7f06d2ffd640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 0x7f06c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.573+0000 7f06d2ffd640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 0x7f06c006edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006fd0 tx=0x7f06c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.576+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06d414c970 con 0x7f06d4160d50 2024-04-26T21:20:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.759+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06d40b54b0 con 0x7f06c006c8f0 2024-04-26T21:20:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.766+0000 7f06bb7fe640 1 -- 172.21.15.40:0/4102915057 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252400 (secure 0 0 0) 0x7f06d40b54b0 con 0x7f06c006c8f0 2024-04-26T21:20:54.769 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:20:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 msgr2=0x7f06c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 0x7f06c006edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006fd0 tx=0x7f06c4008040 comp rx=0 tx=0).stop 2024-04-26T21:20:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 msgr2=0x7f06d409fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b970 tx=0x7f06cc00be30 comp rx=0 tx=0).stop 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 shutdown_connections 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f06c006c8f0 0x7f06c006edb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06d4160d50 0x7f06d409fab0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d415d380 0x7f06d409f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 --2- 172.21.15.40:0/4102915057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d40b56d0 0x7f06d414c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 >> 172.21.15.40:0/4102915057 conn(0x7f06d40b4a60 msgr2=0x7f06d415a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 shutdown_connections 2024-04-26T21:20:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.769+0000 7f06d955f640 1 -- 172.21.15.40:0/4102915057 wait complete. 2024-04-26T21:20:54.780 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-04-26T21:20:53.047053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272104,"kb_used_data":14280,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754702616,"statfs":{"total":773094113280,"available":772815478784,"internally_reserved":0,"allocated":14622720,"data_stored":6947372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001959"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965835+0000","last_change":"2024-04-26T21:20:45.980429+0000","last_active":"2024-04-26T21:20:46.965835+0000","last_peered":"2024-04-26T21:20:46.965835+0000","last_clean":"2024-04-26T21:20:46.965835+0000","last_became_active":"2024-04-26T21:20:45.980082+0000","last_became_peered":"2024-04-26T21:20:45.980082+0000","last_unstale":"2024-04-26T21:20:46.965835+0000","last_undegraded":"2024-04-26T21:20:46.965835+0000","last_fullsized":"2024-04-26T21:20:46.965835+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:05.055182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336059+0000","last_change":"2024-04-26T21:20:45.965877+0000","last_active":"2024-04-26T21:20:47.336059+0000","last_peered":"2024-04-26T21:20:47.336059+0000","last_clean":"2024-04-26T21:20:47.336059+0000","last_became_active":"2024-04-26T21:20:45.965531+0000","last_became_peered":"2024-04-26T21:20:45.965531+0000","last_unstale":"2024-04-26T21:20:47.336059+0000","last_undegraded":"2024-04-26T21:20:47.336059+0000","last_fullsized":"2024-04-26T21:20:47.336059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:02.567341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336527+0000","last_change":"2024-04-26T21:20:45.972531+0000","last_active":"2024-04-26T21:20:47.336527+0000","last_peered":"2024-04-26T21:20:47.336527+0000","last_clean":"2024-04-26T21:20:47.336527+0000","last_became_active":"2024-04-26T21:20:45.972241+0000","last_became_peered":"2024-04-26T21:20:45.972241+0000","last_unstale":"2024-04-26T21:20:47.336527+0000","last_undegraded":"2024-04-26T21:20:47.336527+0000","last_fullsized":"2024-04-26T21:20:47.336527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336969+0000","last_change":"2024-04-26T21:20:45.973682+0000","last_active":"2024-04-26T21:20:47.336969+0000","last_peered":"2024-04-26T21:20:47.336969+0000","last_clean":"2024-04-26T21:20:47.336969+0000","last_became_active":"2024-04-26T21:20:45.973381+0000","last_became_peered":"2024-04-26T21:20:45.973381+0000","last_unstale":"2024-04-26T21:20:47.336969+0000","last_undegraded":"2024-04-26T21:20:47.336969+0000","last_fullsized":"2024-04-26T21:20:47.336969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:34.199283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962690+0000","last_change":"2024-04-26T21:20:45.967509+0000","last_active":"2024-04-26T21:20:46.962690+0000","last_peered":"2024-04-26T21:20:46.962690+0000","last_clean":"2024-04-26T21:20:46.962690+0000","last_became_active":"2024-04-26T21:20:45.967269+0000","last_became_peered":"2024-04-26T21:20:45.967269+0000","last_unstale":"2024-04-26T21:20:46.962690+0000","last_undegraded":"2024-04-26T21:20:46.962690+0000","last_fullsized":"2024-04-26T21:20:46.962690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.129198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965424+0000","last_change":"2024-04-26T21:20:45.972739+0000","last_active":"2024-04-26T21:20:46.965424+0000","last_peered":"2024-04-26T21:20:46.965424+0000","last_clean":"2024-04-26T21:20:46.965424+0000","last_became_active":"2024-04-26T21:20:45.972431+0000","last_became_peered":"2024-04-26T21:20:45.972431+0000","last_unstale":"2024-04-26T21:20:46.965424+0000","last_undegraded":"2024-04-26T21:20:46.965424+0000","last_fullsized":"2024-04-26T21:20:46.965424+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:11.950116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335603+0000","last_change":"2024-04-26T21:20:45.963229+0000","last_active":"2024-04-26T21:20:47.335603+0000","last_peered":"2024-04-26T21:20:47.335603+0000","last_clean":"2024-04-26T21:20:47.335603+0000","last_became_active":"2024-04-26T21:20:45.962872+0000","last_became_peered":"2024-04-26T21:20:45.962872+0000","last_unstale":"2024-04-26T21:20:47.335603+0000","last_undegraded":"2024-04-26T21:20:47.335603+0000","last_fullsized":"2024-04-26T21:20:47.335603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965205+0000","last_change":"2024-04-26T21:20:45.970806+0000","last_active":"2024-04-26T21:20:46.965205+0000","last_peered":"2024-04-26T21:20:46.965205+0000","last_clean":"2024-04-26T21:20:46.965205+0000","last_became_active":"2024-04-26T21:20:45.970485+0000","last_became_peered":"2024-04-26T21:20:45.970485+0000","last_unstale":"2024-04-26T21:20:46.965205+0000","last_undegraded":"2024-04-26T21:20:46.965205+0000","last_fullsized":"2024-04-26T21:20:46.965205+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:23.873993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336113+0000","last_change":"2024-04-26T21:20:45.968152+0000","last_active":"2024-04-26T21:20:47.336113+0000","last_peered":"2024-04-26T21:20:47.336113+0000","last_clean":"2024-04-26T21:20:47.336113+0000","last_became_active":"2024-04-26T21:20:45.967845+0000","last_became_peered":"2024-04-26T21:20:45.967845+0000","last_unstale":"2024-04-26T21:20:47.336113+0000","last_undegraded":"2024-04-26T21:20:47.336113+0000","last_fullsized":"2024-04-26T21:20:47.336113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:11.001965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961661+0000","last_change":"2024-04-26T21:20:45.977906+0000","last_active":"2024-04-26T21:20:46.961661+0000","last_peered":"2024-04-26T21:20:46.961661+0000","last_clean":"2024-04-26T21:20:46.961661+0000","last_became_active":"2024-04-26T21:20:45.977567+0000","last_became_peered":"2024-04-26T21:20:45.977567+0000","last_unstale":"2024-04-26T21:20:46.961661+0000","last_undegraded":"2024-04-26T21:20:46.961661+0000","last_fullsized":"2024-04-26T21:20:46.961661+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965364+0000","last_change":"2024-04-26T21:20:45.970225+0000","last_active":"2024-04-26T21:20:46.965364+0000","last_peered":"2024-04-26T21:20:46.965364+0000","last_clean":"2024-04-26T21:20:46.965364+0000","last_became_active":"2024-04-26T21:20:45.969734+0000","last_became_peered":"2024-04-26T21:20:45.969734+0000","last_unstale":"2024-04-26T21:20:46.965364+0000","last_undegraded":"2024-04-26T21:20:46.965364+0000","last_fullsized":"2024-04-26T21:20:46.965364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:45.806197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961702+0000","last_change":"2024-04-26T21:20:45.972783+0000","last_active":"2024-04-26T21:20:46.961702+0000","last_peered":"2024-04-26T21:20:46.961702+0000","last_clean":"2024-04-26T21:20:46.961702+0000","last_became_active":"2024-04-26T21:20:45.972522+0000","last_became_peered":"2024-04-26T21:20:45.972522+0000","last_unstale":"2024-04-26T21:20:46.961702+0000","last_undegraded":"2024-04-26T21:20:46.961702+0000","last_fullsized":"2024-04-26T21:20:46.961702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962682+0000","last_change":"2024-04-26T21:20:45.967507+0000","last_active":"2024-04-26T21:20:46.962682+0000","last_peered":"2024-04-26T21:20:46.962682+0000","last_clean":"2024-04-26T21:20:46.962682+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.962682+0000","last_undegraded":"2024-04-26T21:20:46.962682+0000","last_fullsized":"2024-04-26T21:20:46.962682+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:40.380793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:45.967357+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:45.966578+0000","last_became_peered":"2024-04-26T21:20:45.966578+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961095+0000","last_change":"2024-04-26T21:20:45.974674+0000","last_active":"2024-04-26T21:20:46.961095+0000","last_peered":"2024-04-26T21:20:46.961095+0000","last_clean":"2024-04-26T21:20:46.961095+0000","last_became_active":"2024-04-26T21:20:45.974359+0000","last_became_peered":"2024-04-26T21:20:45.974359+0000","last_unstale":"2024-04-26T21:20:46.961095+0000","last_undegraded":"2024-04-26T21:20:46.961095+0000","last_fullsized":"2024-04-26T21:20:46.961095+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:49.051775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959220+0000","last_change":"2024-04-26T21:20:45.968322+0000","last_active":"2024-04-26T21:20:46.959220+0000","last_peered":"2024-04-26T21:20:46.959220+0000","last_clean":"2024-04-26T21:20:46.959220+0000","last_became_active":"2024-04-26T21:20:45.967746+0000","last_became_peered":"2024-04-26T21:20:45.967746+0000","last_unstale":"2024-04-26T21:20:46.959220+0000","last_undegraded":"2024-04-26T21:20:46.959220+0000","last_fullsized":"2024-04-26T21:20:46.959220+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:22.881502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962071+0000","last_change":"2024-04-26T21:20:45.979392+0000","last_active":"2024-04-26T21:20:46.962071+0000","last_peered":"2024-04-26T21:20:46.962071+0000","last_clean":"2024-04-26T21:20:46.962071+0000","last_became_active":"2024-04-26T21:20:45.978894+0000","last_became_peered":"2024-04-26T21:20:45.978894+0000","last_unstale":"2024-04-26T21:20:46.962071+0000","last_undegraded":"2024-04-26T21:20:46.962071+0000","last_fullsized":"2024-04-26T21:20:46.962071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:04.038090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336798+0000","last_change":"2024-04-26T21:20:45.971458+0000","last_active":"2024-04-26T21:20:47.336798+0000","last_peered":"2024-04-26T21:20:47.336798+0000","last_clean":"2024-04-26T21:20:47.336798+0000","last_became_active":"2024-04-26T21:20:45.971186+0000","last_became_peered":"2024-04-26T21:20:45.971186+0000","last_unstale":"2024-04-26T21:20:47.336798+0000","last_undegraded":"2024-04-26T21:20:47.336798+0000","last_fullsized":"2024-04-26T21:20:47.336798+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:48.759884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965696+0000","last_change":"2024-04-26T21:20:45.981233+0000","last_active":"2024-04-26T21:20:46.965696+0000","last_peered":"2024-04-26T21:20:46.965696+0000","last_clean":"2024-04-26T21:20:46.965696+0000","last_became_active":"2024-04-26T21:20:45.979321+0000","last_became_peered":"2024-04-26T21:20:45.979321+0000","last_unstale":"2024-04-26T21:20:46.965696+0000","last_undegraded":"2024-04-26T21:20:46.965696+0000","last_fullsized":"2024-04-26T21:20:46.965696+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:54.473013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959492+0000","last_change":"2024-04-26T21:20:45.968075+0000","last_active":"2024-04-26T21:20:46.959492+0000","last_peered":"2024-04-26T21:20:46.959492+0000","last_clean":"2024-04-26T21:20:46.959492+0000","last_became_active":"2024-04-26T21:20:45.967734+0000","last_became_peered":"2024-04-26T21:20:45.967734+0000","last_unstale":"2024-04-26T21:20:46.959492+0000","last_undegraded":"2024-04-26T21:20:46.959492+0000","last_fullsized":"2024-04-26T21:20:46.959492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.314546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961395+0000","last_change":"2024-04-26T21:20:45.977923+0000","last_active":"2024-04-26T21:20:46.961395+0000","last_peered":"2024-04-26T21:20:46.961395+0000","last_clean":"2024-04-26T21:20:46.961395+0000","last_became_active":"2024-04-26T21:20:45.977624+0000","last_became_peered":"2024-04-26T21:20:45.977624+0000","last_unstale":"2024-04-26T21:20:46.961395+0000","last_undegraded":"2024-04-26T21:20:46.961395+0000","last_fullsized":"2024-04-26T21:20:46.961395+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:25.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336657+0000","last_change":"2024-04-26T21:20:45.956281+0000","last_active":"2024-04-26T21:20:47.336657+0000","last_peered":"2024-04-26T21:20:47.336657+0000","last_clean":"2024-04-26T21:20:47.336657+0000","last_became_active":"2024-04-26T21:20:45.955972+0000","last_became_peered":"2024-04-26T21:20:45.955972+0000","last_unstale":"2024-04-26T21:20:47.336657+0000","last_undegraded":"2024-04-26T21:20:47.336657+0000","last_fullsized":"2024-04-26T21:20:47.336657+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336022+0000","last_change":"2024-04-26T21:20:45.967466+0000","last_active":"2024-04-26T21:20:47.336022+0000","last_peered":"2024-04-26T21:20:47.336022+0000","last_clean":"2024-04-26T21:20:47.336022+0000","last_became_active":"2024-04-26T21:20:45.967180+0000","last_became_peered":"2024-04-26T21:20:45.967180+0000","last_unstale":"2024-04-26T21:20:47.336022+0000","last_undegraded":"2024-04-26T21:20:47.336022+0000","last_fullsized":"2024-04-26T21:20:47.336022+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:21.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961176+0000","last_change":"2024-04-26T21:20:45.966830+0000","last_active":"2024-04-26T21:20:46.961176+0000","last_peered":"2024-04-26T21:20:46.961176+0000","last_clean":"2024-04-26T21:20:46.961176+0000","last_became_active":"2024-04-26T21:20:45.966121+0000","last_became_peered":"2024-04-26T21:20:45.966121+0000","last_unstale":"2024-04-26T21:20:46.961176+0000","last_undegraded":"2024-04-26T21:20:46.961176+0000","last_fullsized":"2024-04-26T21:20:46.961176+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:50.753188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959755+0000","last_change":"2024-04-26T21:20:45.968233+0000","last_active":"2024-04-26T21:20:46.959755+0000","last_peered":"2024-04-26T21:20:46.959755+0000","last_clean":"2024-04-26T21:20:46.959755+0000","last_became_active":"2024-04-26T21:20:45.967940+0000","last_became_peered":"2024-04-26T21:20:45.967940+0000","last_unstale":"2024-04-26T21:20:46.959755+0000","last_undegraded":"2024-04-26T21:20:46.959755+0000","last_fullsized":"2024-04-26T21:20:46.959755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:34.779995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965643+0000","last_change":"2024-04-26T21:20:45.977678+0000","last_active":"2024-04-26T21:20:46.965643+0000","last_peered":"2024-04-26T21:20:46.965643+0000","last_clean":"2024-04-26T21:20:46.965643+0000","last_became_active":"2024-04-26T21:20:45.977380+0000","last_became_peered":"2024-04-26T21:20:45.977380+0000","last_unstale":"2024-04-26T21:20:46.965643+0000","last_undegraded":"2024-04-26T21:20:46.965643+0000","last_fullsized":"2024-04-26T21:20:46.965643+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:19.539549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959794+0000","last_change":"2024-04-26T21:20:45.973518+0000","last_active":"2024-04-26T21:20:46.959794+0000","last_peered":"2024-04-26T21:20:46.959794+0000","last_clean":"2024-04-26T21:20:46.959794+0000","last_became_active":"2024-04-26T21:20:45.973227+0000","last_became_peered":"2024-04-26T21:20:45.973227+0000","last_unstale":"2024-04-26T21:20:46.959794+0000","last_undegraded":"2024-04-26T21:20:46.959794+0000","last_fullsized":"2024-04-26T21:20:46.959794+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:12.310488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336254+0000","last_change":"2024-04-26T21:20:45.973958+0000","last_active":"2024-04-26T21:20:47.336254+0000","last_peered":"2024-04-26T21:20:47.336254+0000","last_clean":"2024-04-26T21:20:47.336254+0000","last_became_active":"2024-04-26T21:20:45.973675+0000","last_became_peered":"2024-04-26T21:20:45.973675+0000","last_unstale":"2024-04-26T21:20:47.336254+0000","last_undegraded":"2024-04-26T21:20:47.336254+0000","last_fullsized":"2024-04-26T21:20:47.336254+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:29.330859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965335+0000","last_change":"2024-04-26T21:20:45.978358+0000","last_active":"2024-04-26T21:20:46.965335+0000","last_peered":"2024-04-26T21:20:46.965335+0000","last_clean":"2024-04-26T21:20:46.965335+0000","last_became_active":"2024-04-26T21:20:45.978047+0000","last_became_peered":"2024-04-26T21:20:45.978047+0000","last_unstale":"2024-04-26T21:20:46.965335+0000","last_undegraded":"2024-04-26T21:20:46.965335+0000","last_fullsized":"2024-04-26T21:20:46.965335+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:33.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336855+0000","last_change":"2024-04-26T21:20:45.970795+0000","last_active":"2024-04-26T21:20:47.336855+0000","last_peered":"2024-04-26T21:20:47.336855+0000","last_clean":"2024-04-26T21:20:47.336855+0000","last_became_active":"2024-04-26T21:20:45.970509+0000","last_became_peered":"2024-04-26T21:20:45.970509+0000","last_unstale":"2024-04-26T21:20:47.336855+0000","last_undegraded":"2024-04-26T21:20:47.336855+0000","last_fullsized":"2024-04-26T21:20:47.336855+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:30.015334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335701+0000","last_change":"2024-04-26T21:20:45.972356+0000","last_active":"2024-04-26T21:20:47.335701+0000","last_peered":"2024-04-26T21:20:47.335701+0000","last_clean":"2024-04-26T21:20:47.335701+0000","last_became_active":"2024-04-26T21:20:45.972064+0000","last_became_peered":"2024-04-26T21:20:45.972064+0000","last_unstale":"2024-04-26T21:20:47.335701+0000","last_undegraded":"2024-04-26T21:20:47.335701+0000","last_fullsized":"2024-04-26T21:20:47.335701+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:50.058209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959285+0000","last_change":"2024-04-26T21:20:45.968435+0000","last_active":"2024-04-26T21:20:46.959285+0000","last_peered":"2024-04-26T21:20:46.959285+0000","last_clean":"2024-04-26T21:20:46.959285+0000","last_became_active":"2024-04-26T21:20:45.966848+0000","last_became_peered":"2024-04-26T21:20:45.966848+0000","last_unstale":"2024-04-26T21:20:46.959285+0000","last_undegraded":"2024-04-26T21:20:46.959285+0000","last_fullsized":"2024-04-26T21:20:46.959285+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:50.873852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961713+0000","last_change":"2024-04-26T21:20:45.973734+0000","last_active":"2024-04-26T21:20:46.961713+0000","last_peered":"2024-04-26T21:20:46.961713+0000","last_clean":"2024-04-26T21:20:46.961713+0000","last_became_active":"2024-04-26T21:20:45.973472+0000","last_became_peered":"2024-04-26T21:20:45.973472+0000","last_unstale":"2024-04-26T21:20:46.961713+0000","last_undegraded":"2024-04-26T21:20:46.961713+0000","last_fullsized":"2024-04-26T21:20:46.961713+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961961+0000","last_change":"2024-04-26T21:20:44.958400+0000","last_active":"2024-04-26T21:20:46.961961+0000","last_peered":"2024-04-26T21:20:46.961961+0000","last_clean":"2024-04-26T21:20:46.961961+0000","last_became_active":"2024-04-26T21:20:44.958081+0000","last_became_peered":"2024-04-26T21:20:44.958081+0000","last_unstale":"2024-04-26T21:20:46.961961+0000","last_undegraded":"2024-04-26T21:20:46.961961+0000","last_fullsized":"2024-04-26T21:20:46.961961+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:38.330074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336119+0000","last_change":"2024-04-26T21:20:45.964932+0000","last_active":"2024-04-26T21:20:47.336119+0000","last_peered":"2024-04-26T21:20:47.336119+0000","last_clean":"2024-04-26T21:20:47.336119+0000","last_became_active":"2024-04-26T21:20:45.961496+0000","last_became_peered":"2024-04-26T21:20:45.961496+0000","last_unstale":"2024-04-26T21:20:47.336119+0000","last_undegraded":"2024-04-26T21:20:47.336119+0000","last_fullsized":"2024-04-26T21:20:47.336119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:37.885551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335890+0000","last_change":"2024-04-26T21:20:44.956115+0000","last_active":"2024-04-26T21:20:47.335890+0000","last_peered":"2024-04-26T21:20:47.335890+0000","last_clean":"2024-04-26T21:20:47.335890+0000","last_became_active":"2024-04-26T21:20:44.955813+0000","last_became_peered":"2024-04-26T21:20:44.955813+0000","last_unstale":"2024-04-26T21:20:47.335890+0000","last_undegraded":"2024-04-26T21:20:47.335890+0000","last_fullsized":"2024-04-26T21:20:47.335890+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:21.778534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965448+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:46.965448+0000","last_peered":"2024-04-26T21:20:46.965448+0000","last_clean":"2024-04-26T21:20:46.965448+0000","last_became_active":"2024-04-26T21:20:45.973396+0000","last_became_peered":"2024-04-26T21:20:45.973396+0000","last_unstale":"2024-04-26T21:20:46.965448+0000","last_undegraded":"2024-04-26T21:20:46.965448+0000","last_fullsized":"2024-04-26T21:20:46.965448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:02.776803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.949711+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.949393+0000","last_became_peered":"2024-04-26T21:20:44.949393+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961372+0000","last_change":"2024-04-26T21:20:45.964911+0000","last_active":"2024-04-26T21:20:46.961372+0000","last_peered":"2024-04-26T21:20:46.961372+0000","last_clean":"2024-04-26T21:20:46.961372+0000","last_became_active":"2024-04-26T21:20:45.963945+0000","last_became_peered":"2024-04-26T21:20:45.963945+0000","last_unstale":"2024-04-26T21:20:46.961372+0000","last_undegraded":"2024-04-26T21:20:46.961372+0000","last_fullsized":"2024-04-26T21:20:46.961372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:41.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.337000+0000","last_change":"2024-04-26T21:20:44.949483+0000","last_active":"2024-04-26T21:20:47.337000+0000","last_peered":"2024-04-26T21:20:47.337000+0000","last_clean":"2024-04-26T21:20:47.337000+0000","last_became_active":"2024-04-26T21:20:44.948183+0000","last_became_peered":"2024-04-26T21:20:44.948183+0000","last_unstale":"2024-04-26T21:20:47.337000+0000","last_undegraded":"2024-04-26T21:20:47.337000+0000","last_fullsized":"2024-04-26T21:20:47.337000+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335767+0000","last_change":"2024-04-26T21:20:45.973650+0000","last_active":"2024-04-26T21:20:47.335767+0000","last_peered":"2024-04-26T21:20:47.335767+0000","last_clean":"2024-04-26T21:20:47.335767+0000","last_became_active":"2024-04-26T21:20:45.973350+0000","last_became_peered":"2024-04-26T21:20:45.973350+0000","last_unstale":"2024-04-26T21:20:47.335767+0000","last_undegraded":"2024-04-26T21:20:47.335767+0000","last_fullsized":"2024-04-26T21:20:47.335767+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.225182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959303+0000","last_change":"2024-04-26T21:20:44.954463+0000","last_active":"2024-04-26T21:20:46.959303+0000","last_peered":"2024-04-26T21:20:46.959303+0000","last_clean":"2024-04-26T21:20:46.959303+0000","last_became_active":"2024-04-26T21:20:44.954133+0000","last_became_peered":"2024-04-26T21:20:44.954133+0000","last_unstale":"2024-04-26T21:20:46.959303+0000","last_undegraded":"2024-04-26T21:20:46.959303+0000","last_fullsized":"2024-04-26T21:20:46.959303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:16.181685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336085+0000","last_change":"2024-04-26T21:20:45.978602+0000","last_active":"2024-04-26T21:20:47.336085+0000","last_peered":"2024-04-26T21:20:47.336085+0000","last_clean":"2024-04-26T21:20:47.336085+0000","last_became_active":"2024-04-26T21:20:45.978300+0000","last_became_peered":"2024-04-26T21:20:45.978300+0000","last_unstale":"2024-04-26T21:20:47.336085+0000","last_undegraded":"2024-04-26T21:20:47.336085+0000","last_fullsized":"2024-04-26T21:20:47.336085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962713+0000","last_change":"2024-04-26T21:20:44.953754+0000","last_active":"2024-04-26T21:20:46.962713+0000","last_peered":"2024-04-26T21:20:46.962713+0000","last_clean":"2024-04-26T21:20:46.962713+0000","last_became_active":"2024-04-26T21:20:44.953471+0000","last_became_peered":"2024-04-26T21:20:44.953471+0000","last_unstale":"2024-04-26T21:20:46.962713+0000","last_undegraded":"2024-04-26T21:20:46.962713+0000","last_fullsized":"2024-04-26T21:20:46.962713+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:21.877111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335605+0000","last_change":"2024-04-26T21:20:45.957101+0000","last_active":"2024-04-26T21:20:47.335605+0000","last_peered":"2024-04-26T21:20:47.335605+0000","last_clean":"2024-04-26T21:20:47.335605+0000","last_became_active":"2024-04-26T21:20:45.956746+0000","last_became_peered":"2024-04-26T21:20:45.956746+0000","last_unstale":"2024-04-26T21:20:47.335605+0000","last_undegraded":"2024-04-26T21:20:47.335605+0000","last_fullsized":"2024-04-26T21:20:47.335605+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:20.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959438+0000","last_change":"2024-04-26T21:20:44.945604+0000","last_active":"2024-04-26T21:20:46.959438+0000","last_peered":"2024-04-26T21:20:46.959438+0000","last_clean":"2024-04-26T21:20:46.959438+0000","last_became_active":"2024-04-26T21:20:44.945311+0000","last_became_peered":"2024-04-26T21:20:44.945311+0000","last_unstale":"2024-04-26T21:20:46.959438+0000","last_undegraded":"2024-04-26T21:20:46.959438+0000","last_fullsized":"2024-04-26T21:20:46.959438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:59.997109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962422+0000","last_change":"2024-04-26T21:20:45.968415+0000","last_active":"2024-04-26T21:20:46.962422+0000","last_peered":"2024-04-26T21:20:46.962422+0000","last_clean":"2024-04-26T21:20:46.962422+0000","last_became_active":"2024-04-26T21:20:45.968148+0000","last_became_peered":"2024-04-26T21:20:45.968148+0000","last_unstale":"2024-04-26T21:20:46.962422+0000","last_undegraded":"2024-04-26T21:20:46.962422+0000","last_fullsized":"2024-04-26T21:20:46.962422+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:30.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335772+0000","last_change":"2024-04-26T21:20:44.953539+0000","last_active":"2024-04-26T21:20:47.335772+0000","last_peered":"2024-04-26T21:20:47.335772+0000","last_clean":"2024-04-26T21:20:47.335772+0000","last_became_active":"2024-04-26T21:20:44.953286+0000","last_became_peered":"2024-04-26T21:20:44.953286+0000","last_unstale":"2024-04-26T21:20:47.335772+0000","last_undegraded":"2024-04-26T21:20:47.335772+0000","last_fullsized":"2024-04-26T21:20:47.335772+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962742+0000","last_change":"2024-04-26T21:20:45.957232+0000","last_active":"2024-04-26T21:20:46.962742+0000","last_peered":"2024-04-26T21:20:46.962742+0000","last_clean":"2024-04-26T21:20:46.962742+0000","last_became_active":"2024-04-26T21:20:45.956611+0000","last_became_peered":"2024-04-26T21:20:45.956611+0000","last_unstale":"2024-04-26T21:20:46.962742+0000","last_undegraded":"2024-04-26T21:20:46.962742+0000","last_fullsized":"2024-04-26T21:20:46.962742+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:13.597030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336218+0000","last_change":"2024-04-26T21:20:44.952646+0000","last_active":"2024-04-26T21:20:47.336218+0000","last_peered":"2024-04-26T21:20:47.336218+0000","last_clean":"2024-04-26T21:20:47.336218+0000","last_became_active":"2024-04-26T21:20:44.952351+0000","last_became_peered":"2024-04-26T21:20:44.952351+0000","last_unstale":"2024-04-26T21:20:47.336218+0000","last_undegraded":"2024-04-26T21:20:47.336218+0000","last_fullsized":"2024-04-26T21:20:47.336218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:33.194892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961915+0000","last_change":"2024-04-26T21:20:45.979158+0000","last_active":"2024-04-26T21:20:46.961915+0000","last_peered":"2024-04-26T21:20:46.961915+0000","last_clean":"2024-04-26T21:20:46.961915+0000","last_became_active":"2024-04-26T21:20:45.978509+0000","last_became_peered":"2024-04-26T21:20:45.978509+0000","last_unstale":"2024-04-26T21:20:46.961915+0000","last_undegraded":"2024-04-26T21:20:46.961915+0000","last_fullsized":"2024-04-26T21:20:46.961915+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:18.599945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961941+0000","last_change":"2024-04-26T21:20:44.958683+0000","last_active":"2024-04-26T21:20:46.961941+0000","last_peered":"2024-04-26T21:20:46.961941+0000","last_clean":"2024-04-26T21:20:46.961941+0000","last_became_active":"2024-04-26T21:20:44.958250+0000","last_became_peered":"2024-04-26T21:20:44.958250+0000","last_unstale":"2024-04-26T21:20:46.961941+0000","last_undegraded":"2024-04-26T21:20:46.961941+0000","last_fullsized":"2024-04-26T21:20:46.961941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:19.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959625+0000","last_change":"2024-04-26T21:20:45.968143+0000","last_active":"2024-04-26T21:20:46.959625+0000","last_peered":"2024-04-26T21:20:46.959625+0000","last_clean":"2024-04-26T21:20:46.959625+0000","last_became_active":"2024-04-26T21:20:45.966158+0000","last_became_peered":"2024-04-26T21:20:45.966158+0000","last_unstale":"2024-04-26T21:20:46.959625+0000","last_undegraded":"2024-04-26T21:20:46.959625+0000","last_fullsized":"2024-04-26T21:20:46.959625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:10.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336377+0000","last_change":"2024-04-26T21:20:44.951683+0000","last_active":"2024-04-26T21:20:47.336377+0000","last_peered":"2024-04-26T21:20:47.336377+0000","last_clean":"2024-04-26T21:20:47.336377+0000","last_became_active":"2024-04-26T21:20:44.951213+0000","last_became_peered":"2024-04-26T21:20:44.951213+0000","last_unstale":"2024-04-26T21:20:47.336377+0000","last_undegraded":"2024-04-26T21:20:47.336377+0000","last_fullsized":"2024-04-26T21:20:47.336377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:36.881160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336537+0000","last_change":"2024-04-26T21:20:45.968505+0000","last_active":"2024-04-26T21:20:47.336537+0000","last_peered":"2024-04-26T21:20:47.336537+0000","last_clean":"2024-04-26T21:20:47.336537+0000","last_became_active":"2024-04-26T21:20:45.968185+0000","last_became_peered":"2024-04-26T21:20:45.968185+0000","last_unstale":"2024-04-26T21:20:47.336537+0000","last_undegraded":"2024-04-26T21:20:47.336537+0000","last_fullsized":"2024-04-26T21:20:47.336537+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:52.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336545+0000","last_change":"2024-04-26T21:20:44.951050+0000","last_active":"2024-04-26T21:20:47.336545+0000","last_peered":"2024-04-26T21:20:47.336545+0000","last_clean":"2024-04-26T21:20:47.336545+0000","last_became_active":"2024-04-26T21:20:44.950758+0000","last_became_peered":"2024-04-26T21:20:44.950758+0000","last_unstale":"2024-04-26T21:20:47.336545+0000","last_undegraded":"2024-04-26T21:20:47.336545+0000","last_fullsized":"2024-04-26T21:20:47.336545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:07.338750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335606+0000","last_change":"2024-04-26T21:20:45.965874+0000","last_active":"2024-04-26T21:20:47.335606+0000","last_peered":"2024-04-26T21:20:47.335606+0000","last_clean":"2024-04-26T21:20:47.335606+0000","last_became_active":"2024-04-26T21:20:45.965561+0000","last_became_peered":"2024-04-26T21:20:45.965561+0000","last_unstale":"2024-04-26T21:20:47.335606+0000","last_undegraded":"2024-04-26T21:20:47.335606+0000","last_fullsized":"2024-04-26T21:20:47.335606+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:00.164990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336663+0000","last_change":"2024-04-26T21:20:44.949774+0000","last_active":"2024-04-26T21:20:47.336663+0000","last_peered":"2024-04-26T21:20:47.336663+0000","last_clean":"2024-04-26T21:20:47.336663+0000","last_became_active":"2024-04-26T21:20:44.949464+0000","last_became_peered":"2024-04-26T21:20:44.949464+0000","last_unstale":"2024-04-26T21:20:47.336663+0000","last_undegraded":"2024-04-26T21:20:47.336663+0000","last_fullsized":"2024-04-26T21:20:47.336663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:49.883143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962548+0000","last_change":"2024-04-26T21:20:45.965418+0000","last_active":"2024-04-26T21:20:46.962548+0000","last_peered":"2024-04-26T21:20:46.962548+0000","last_clean":"2024-04-26T21:20:46.962548+0000","last_became_active":"2024-04-26T21:20:45.965125+0000","last_became_peered":"2024-04-26T21:20:45.965125+0000","last_unstale":"2024-04-26T21:20:46.962548+0000","last_undegraded":"2024-04-26T21:20:46.962548+0000","last_fullsized":"2024-04-26T21:20:46.962548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:47.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965231+0000","last_change":"2024-04-26T21:20:44.952556+0000","last_active":"2024-04-26T21:20:46.965231+0000","last_peered":"2024-04-26T21:20:46.965231+0000","last_clean":"2024-04-26T21:20:46.965231+0000","last_became_active":"2024-04-26T21:20:44.952257+0000","last_became_peered":"2024-04-26T21:20:44.952257+0000","last_unstale":"2024-04-26T21:20:46.965231+0000","last_undegraded":"2024-04-26T21:20:46.965231+0000","last_fullsized":"2024-04-26T21:20:46.965231+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:49.597526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336273+0000","last_change":"2024-04-26T21:20:45.971847+0000","last_active":"2024-04-26T21:20:47.336273+0000","last_peered":"2024-04-26T21:20:47.336273+0000","last_clean":"2024-04-26T21:20:47.336273+0000","last_became_active":"2024-04-26T21:20:45.971310+0000","last_became_peered":"2024-04-26T21:20:45.971310+0000","last_unstale":"2024-04-26T21:20:47.336273+0000","last_undegraded":"2024-04-26T21:20:47.336273+0000","last_fullsized":"2024-04-26T21:20:47.336273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:45.464436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959325+0000","last_change":"2024-04-26T21:20:44.950519+0000","last_active":"2024-04-26T21:20:46.959325+0000","last_peered":"2024-04-26T21:20:46.959325+0000","last_clean":"2024-04-26T21:20:46.959325+0000","last_became_active":"2024-04-26T21:20:44.950288+0000","last_became_peered":"2024-04-26T21:20:44.950288+0000","last_unstale":"2024-04-26T21:20:46.959325+0000","last_undegraded":"2024-04-26T21:20:46.959325+0000","last_fullsized":"2024-04-26T21:20:46.959325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:07.328646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961844+0000","last_change":"2024-04-26T21:20:45.968413+0000","last_active":"2024-04-26T21:20:46.961844+0000","last_peered":"2024-04-26T21:20:46.961844+0000","last_clean":"2024-04-26T21:20:46.961844+0000","last_became_active":"2024-04-26T21:20:45.968149+0000","last_became_peered":"2024-04-26T21:20:45.968149+0000","last_unstale":"2024-04-26T21:20:46.961844+0000","last_undegraded":"2024-04-26T21:20:46.961844+0000","last_fullsized":"2024-04-26T21:20:46.961844+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:11.931969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962569+0000","last_change":"2024-04-26T21:20:44.952076+0000","last_active":"2024-04-26T21:20:46.962569+0000","last_peered":"2024-04-26T21:20:46.962569+0000","last_clean":"2024-04-26T21:20:46.962569+0000","last_became_active":"2024-04-26T21:20:44.950995+0000","last_became_peered":"2024-04-26T21:20:44.950995+0000","last_unstale":"2024-04-26T21:20:46.962569+0000","last_undegraded":"2024-04-26T21:20:46.962569+0000","last_fullsized":"2024-04-26T21:20:46.962569+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:49.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959745+0000","last_change":"2024-04-26T21:20:45.967568+0000","last_active":"2024-04-26T21:20:46.959745+0000","last_peered":"2024-04-26T21:20:46.959745+0000","last_clean":"2024-04-26T21:20:46.959745+0000","last_became_active":"2024-04-26T21:20:45.967218+0000","last_became_peered":"2024-04-26T21:20:45.967218+0000","last_unstale":"2024-04-26T21:20:46.959745+0000","last_undegraded":"2024-04-26T21:20:46.959745+0000","last_fullsized":"2024-04-26T21:20:46.959745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:52.037830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336020+0000","last_change":"2024-04-26T21:20:44.949326+0000","last_active":"2024-04-26T21:20:47.336020+0000","last_peered":"2024-04-26T21:20:47.336020+0000","last_clean":"2024-04-26T21:20:47.336020+0000","last_became_active":"2024-04-26T21:20:44.948970+0000","last_became_peered":"2024-04-26T21:20:44.948970+0000","last_unstale":"2024-04-26T21:20:47.336020+0000","last_undegraded":"2024-04-26T21:20:47.336020+0000","last_fullsized":"2024-04-26T21:20:47.336020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:28.645489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336403+0000","last_change":"2024-04-26T21:20:45.967295+0000","last_active":"2024-04-26T21:20:47.336403+0000","last_peered":"2024-04-26T21:20:47.336403+0000","last_clean":"2024-04-26T21:20:47.336403+0000","last_became_active":"2024-04-26T21:20:45.966533+0000","last_became_peered":"2024-04-26T21:20:45.966533+0000","last_unstale":"2024-04-26T21:20:47.336403+0000","last_undegraded":"2024-04-26T21:20:47.336403+0000","last_fullsized":"2024-04-26T21:20:47.336403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959724+0000","last_change":"2024-04-26T21:20:44.953637+0000","last_active":"2024-04-26T21:20:46.959724+0000","last_peered":"2024-04-26T21:20:46.959724+0000","last_clean":"2024-04-26T21:20:46.959724+0000","last_became_active":"2024-04-26T21:20:44.953364+0000","last_became_peered":"2024-04-26T21:20:44.953364+0000","last_unstale":"2024-04-26T21:20:46.959724+0000","last_undegraded":"2024-04-26T21:20:46.959724+0000","last_fullsized":"2024-04-26T21:20:46.959724+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:04.043064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965518+0000","last_change":"2024-04-26T21:20:45.977550+0000","last_active":"2024-04-26T21:20:46.965518+0000","last_peered":"2024-04-26T21:20:46.965518+0000","last_clean":"2024-04-26T21:20:46.965518+0000","last_became_active":"2024-04-26T21:20:45.977244+0000","last_became_peered":"2024-04-26T21:20:45.977244+0000","last_unstale":"2024-04-26T21:20:46.965518+0000","last_undegraded":"2024-04-26T21:20:46.965518+0000","last_fullsized":"2024-04-26T21:20:46.965518+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:36.978003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336034+0000","last_change":"2024-04-26T21:20:44.950539+0000","last_active":"2024-04-26T21:20:47.336034+0000","last_peered":"2024-04-26T21:20:47.336034+0000","last_clean":"2024-04-26T21:20:47.336034+0000","last_became_active":"2024-04-26T21:20:44.950262+0000","last_became_peered":"2024-04-26T21:20:44.950262+0000","last_unstale":"2024-04-26T21:20:47.336034+0000","last_undegraded":"2024-04-26T21:20:47.336034+0000","last_fullsized":"2024-04-26T21:20:47.336034+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:19.053104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336857+0000","last_change":"2024-04-26T21:20:45.970609+0000","last_active":"2024-04-26T21:20:47.336857+0000","last_peered":"2024-04-26T21:20:47.336857+0000","last_clean":"2024-04-26T21:20:47.336857+0000","last_became_active":"2024-04-26T21:20:45.970312+0000","last_became_peered":"2024-04-26T21:20:45.970312+0000","last_unstale":"2024-04-26T21:20:47.336857+0000","last_undegraded":"2024-04-26T21:20:47.336857+0000","last_fullsized":"2024-04-26T21:20:47.336857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:20.057495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336100+0000","last_change":"2024-04-26T21:20:44.949930+0000","last_active":"2024-04-26T21:20:47.336100+0000","last_peered":"2024-04-26T21:20:47.336100+0000","last_clean":"2024-04-26T21:20:47.336100+0000","last_became_active":"2024-04-26T21:20:44.949603+0000","last_became_peered":"2024-04-26T21:20:44.949603+0000","last_unstale":"2024-04-26T21:20:47.336100+0000","last_undegraded":"2024-04-26T21:20:47.336100+0000","last_fullsized":"2024-04-26T21:20:47.336100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:01.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335637+0000","last_change":"2024-04-26T21:20:45.964537+0000","last_active":"2024-04-26T21:20:47.335637+0000","last_peered":"2024-04-26T21:20:47.335637+0000","last_clean":"2024-04-26T21:20:47.335637+0000","last_became_active":"2024-04-26T21:20:45.964253+0000","last_became_peered":"2024-04-26T21:20:45.964253+0000","last_unstale":"2024-04-26T21:20:47.335637+0000","last_undegraded":"2024-04-26T21:20:47.335637+0000","last_fullsized":"2024-04-26T21:20:47.335637+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961394+0000","last_change":"2024-04-26T21:20:44.958433+0000","last_active":"2024-04-26T21:20:46.961394+0000","last_peered":"2024-04-26T21:20:46.961394+0000","last_clean":"2024-04-26T21:20:46.961394+0000","last_became_active":"2024-04-26T21:20:44.958109+0000","last_became_peered":"2024-04-26T21:20:44.958109+0000","last_unstale":"2024-04-26T21:20:46.961394+0000","last_undegraded":"2024-04-26T21:20:46.961394+0000","last_fullsized":"2024-04-26T21:20:46.961394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:56.399692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335976+0000","last_change":"2024-04-26T21:20:45.963605+0000","last_active":"2024-04-26T21:20:47.335976+0000","last_peered":"2024-04-26T21:20:47.335976+0000","last_clean":"2024-04-26T21:20:47.335976+0000","last_became_active":"2024-04-26T21:20:45.963047+0000","last_became_peered":"2024-04-26T21:20:45.963047+0000","last_unstale":"2024-04-26T21:20:47.335976+0000","last_undegraded":"2024-04-26T21:20:47.335976+0000","last_fullsized":"2024-04-26T21:20:47.335976+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:50.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961119+0000","last_change":"2024-04-26T21:20:44.957474+0000","last_active":"2024-04-26T21:20:46.961119+0000","last_peered":"2024-04-26T21:20:46.961119+0000","last_clean":"2024-04-26T21:20:46.961119+0000","last_became_active":"2024-04-26T21:20:44.957134+0000","last_became_peered":"2024-04-26T21:20:44.957134+0000","last_unstale":"2024-04-26T21:20:46.961119+0000","last_undegraded":"2024-04-26T21:20:46.961119+0000","last_fullsized":"2024-04-26T21:20:46.961119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:50.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335784+0000","last_change":"2024-04-26T21:20:45.961150+0000","last_active":"2024-04-26T21:20:47.335784+0000","last_peered":"2024-04-26T21:20:47.335784+0000","last_clean":"2024-04-26T21:20:47.335784+0000","last_became_active":"2024-04-26T21:20:45.960842+0000","last_became_peered":"2024-04-26T21:20:45.960842+0000","last_unstale":"2024-04-26T21:20:47.335784+0000","last_undegraded":"2024-04-26T21:20:47.335784+0000","last_fullsized":"2024-04-26T21:20:47.335784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:36.804670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335959+0000","last_change":"2024-04-26T21:20:44.951589+0000","last_active":"2024-04-26T21:20:47.335959+0000","last_peered":"2024-04-26T21:20:47.335959+0000","last_clean":"2024-04-26T21:20:47.335959+0000","last_became_active":"2024-04-26T21:20:44.950834+0000","last_became_peered":"2024-04-26T21:20:44.950834+0000","last_unstale":"2024-04-26T21:20:47.335959+0000","last_undegraded":"2024-04-26T21:20:47.335959+0000","last_fullsized":"2024-04-26T21:20:47.335959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:51.927771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962749+0000","last_change":"2024-04-26T21:20:45.967744+0000","last_active":"2024-04-26T21:20:46.962749+0000","last_peered":"2024-04-26T21:20:46.962749+0000","last_clean":"2024-04-26T21:20:46.962749+0000","last_became_active":"2024-04-26T21:20:45.967409+0000","last_became_peered":"2024-04-26T21:20:45.967409+0000","last_unstale":"2024-04-26T21:20:46.962749+0000","last_undegraded":"2024-04-26T21:20:46.962749+0000","last_fullsized":"2024-04-26T21:20:46.962749+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:12.684400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962057+0000","last_change":"2024-04-26T21:20:44.958845+0000","last_active":"2024-04-26T21:20:46.962057+0000","last_peered":"2024-04-26T21:20:46.962057+0000","last_clean":"2024-04-26T21:20:46.962057+0000","last_became_active":"2024-04-26T21:20:44.958476+0000","last_became_peered":"2024-04-26T21:20:44.958476+0000","last_unstale":"2024-04-26T21:20:46.962057+0000","last_undegraded":"2024-04-26T21:20:46.962057+0000","last_fullsized":"2024-04-26T21:20:46.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:53.203282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336659+0000","last_change":"2024-04-26T21:20:45.967883+0000","last_active":"2024-04-26T21:20:47.336659+0000","last_peered":"2024-04-26T21:20:47.336659+0000","last_clean":"2024-04-26T21:20:47.336659+0000","last_became_active":"2024-04-26T21:20:45.967589+0000","last_became_peered":"2024-04-26T21:20:45.967589+0000","last_unstale":"2024-04-26T21:20:47.336659+0000","last_undegraded":"2024-04-26T21:20:47.336659+0000","last_fullsized":"2024-04-26T21:20:47.336659+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:39.892414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"134'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965739+0000","last_change":"2024-04-26T21:20:44.955921+0000","last_active":"2024-04-26T21:20:46.965739+0000","last_peered":"2024-04-26T21:20:46.965739+0000","last_clean":"2024-04-26T21:20:46.965739+0000","last_became_active":"2024-04-26T21:20:44.955598+0000","last_became_peered":"2024-04-26T21:20:44.955598+0000","last_unstale":"2024-04-26T21:20:46.965739+0000","last_undegraded":"2024-04-26T21:20:46.965739+0000","last_fullsized":"2024-04-26T21:20:46.965739+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:53.281999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335972+0000","last_change":"2024-04-26T21:20:45.958489+0000","last_active":"2024-04-26T21:20:47.335972+0000","last_peered":"2024-04-26T21:20:47.335972+0000","last_clean":"2024-04-26T21:20:47.335972+0000","last_became_active":"2024-04-26T21:20:45.958208+0000","last_became_peered":"2024-04-26T21:20:45.958208+0000","last_unstale":"2024-04-26T21:20:47.335972+0000","last_undegraded":"2024-04-26T21:20:47.335972+0000","last_fullsized":"2024-04-26T21:20:47.335972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:32.662963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965626+0000","last_change":"2024-04-26T21:20:44.951269+0000","last_active":"2024-04-26T21:20:46.965626+0000","last_peered":"2024-04-26T21:20:46.965626+0000","last_clean":"2024-04-26T21:20:46.965626+0000","last_became_active":"2024-04-26T21:20:44.950960+0000","last_became_peered":"2024-04-26T21:20:44.950960+0000","last_unstale":"2024-04-26T21:20:46.965626+0000","last_undegraded":"2024-04-26T21:20:46.965626+0000","last_fullsized":"2024-04-26T21:20:46.965626+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:50.430297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965534+0000","last_change":"2024-04-26T21:20:45.975184+0000","last_active":"2024-04-26T21:20:46.965534+0000","last_peered":"2024-04-26T21:20:46.965534+0000","last_clean":"2024-04-26T21:20:46.965534+0000","last_became_active":"2024-04-26T21:20:45.974458+0000","last_became_peered":"2024-04-26T21:20:45.974458+0000","last_unstale":"2024-04-26T21:20:46.965534+0000","last_undegraded":"2024-04-26T21:20:46.965534+0000","last_fullsized":"2024-04-26T21:20:46.965534+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:31.050443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.965377+0000","last_change":"2024-04-26T21:20:44.952291+0000","last_active":"2024-04-26T21:20:46.965377+0000","last_peered":"2024-04-26T21:20:46.965377+0000","last_clean":"2024-04-26T21:20:46.965377+0000","last_became_active":"2024-04-26T21:20:44.951911+0000","last_became_peered":"2024-04-26T21:20:44.951911+0000","last_unstale":"2024-04-26T21:20:46.965377+0000","last_undegraded":"2024-04-26T21:20:46.965377+0000","last_fullsized":"2024-04-26T21:20:46.965377+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:10.339949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.959372+0000","last_change":"2024-04-26T21:20:45.962616+0000","last_active":"2024-04-26T21:20:46.959372+0000","last_peered":"2024-04-26T21:20:46.959372+0000","last_clean":"2024-04-26T21:20:46.959372+0000","last_became_active":"2024-04-26T21:20:45.962318+0000","last_became_peered":"2024-04-26T21:20:45.962318+0000","last_unstale":"2024-04-26T21:20:46.959372+0000","last_undegraded":"2024-04-26T21:20:46.959372+0000","last_fullsized":"2024-04-26T21:20:46.959372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:57.812828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336229+0000","last_change":"2024-04-26T21:20:44.944630+0000","last_active":"2024-04-26T21:20:47.336229+0000","last_peered":"2024-04-26T21:20:47.336229+0000","last_clean":"2024-04-26T21:20:47.336229+0000","last_became_active":"2024-04-26T21:20:44.944271+0000","last_became_peered":"2024-04-26T21:20:44.944271+0000","last_unstale":"2024-04-26T21:20:47.336229+0000","last_undegraded":"2024-04-26T21:20:47.336229+0000","last_fullsized":"2024-04-26T21:20:47.336229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:09.997574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962808+0000","last_change":"2024-04-26T21:20:45.976347+0000","last_active":"2024-04-26T21:20:46.962808+0000","last_peered":"2024-04-26T21:20:46.962808+0000","last_clean":"2024-04-26T21:20:46.962808+0000","last_became_active":"2024-04-26T21:20:45.976080+0000","last_became_peered":"2024-04-26T21:20:45.976080+0000","last_unstale":"2024-04-26T21:20:46.962808+0000","last_undegraded":"2024-04-26T21:20:46.962808+0000","last_fullsized":"2024-04-26T21:20:46.962808+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:30.154836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961487+0000","last_change":"2024-04-26T21:20:44.958470+0000","last_active":"2024-04-26T21:20:46.961487+0000","last_peered":"2024-04-26T21:20:46.961487+0000","last_clean":"2024-04-26T21:20:46.961487+0000","last_became_active":"2024-04-26T21:20:44.958191+0000","last_became_peered":"2024-04-26T21:20:44.958191+0000","last_unstale":"2024-04-26T21:20:46.961487+0000","last_undegraded":"2024-04-26T21:20:46.961487+0000","last_fullsized":"2024-04-26T21:20:46.961487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:51.907338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961496+0000","last_change":"2024-04-26T21:20:45.964088+0000","last_active":"2024-04-26T21:20:46.961496+0000","last_peered":"2024-04-26T21:20:46.961496+0000","last_clean":"2024-04-26T21:20:46.961496+0000","last_became_active":"2024-04-26T21:20:45.963798+0000","last_became_peered":"2024-04-26T21:20:45.963798+0000","last_unstale":"2024-04-26T21:20:46.961496+0000","last_undegraded":"2024-04-26T21:20:46.961496+0000","last_fullsized":"2024-04-26T21:20:46.961496+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:58.395061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336390+0000","last_change":"2024-04-26T21:20:44.952271+0000","last_active":"2024-04-26T21:20:47.336390+0000","last_peered":"2024-04-26T21:20:47.336390+0000","last_clean":"2024-04-26T21:20:47.336390+0000","last_became_active":"2024-04-26T21:20:44.951860+0000","last_became_peered":"2024-04-26T21:20:44.951860+0000","last_unstale":"2024-04-26T21:20:47.336390+0000","last_undegraded":"2024-04-26T21:20:47.336390+0000","last_fullsized":"2024-04-26T21:20:47.336390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:46.647564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.962494+0000","last_change":"2024-04-26T21:20:45.969084+0000","last_active":"2024-04-26T21:20:46.962494+0000","last_peered":"2024-04-26T21:20:46.962494+0000","last_clean":"2024-04-26T21:20:46.962494+0000","last_became_active":"2024-04-26T21:20:45.968806+0000","last_became_peered":"2024-04-26T21:20:45.968806+0000","last_unstale":"2024-04-26T21:20:46.962494+0000","last_undegraded":"2024-04-26T21:20:46.962494+0000","last_fullsized":"2024-04-26T21:20:46.962494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:20.402971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.335712+0000","last_change":"2024-04-26T21:20:44.955072+0000","last_active":"2024-04-26T21:20:47.335712+0000","last_peered":"2024-04-26T21:20:47.335712+0000","last_clean":"2024-04-26T21:20:47.335712+0000","last_became_active":"2024-04-26T21:20:44.954230+0000","last_became_peered":"2024-04-26T21:20:44.954230+0000","last_unstale":"2024-04-26T21:20:47.335712+0000","last_undegraded":"2024-04-26T21:20:47.335712+0000","last_fullsized":"2024-04-26T21:20:47.335712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:04.483717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336287+0000","last_change":"2024-04-26T21:20:45.964348+0000","last_active":"2024-04-26T21:20:47.336287+0000","last_peered":"2024-04-26T21:20:47.336287+0000","last_clean":"2024-04-26T21:20:47.336287+0000","last_became_active":"2024-04-26T21:20:45.964043+0000","last_became_peered":"2024-04-26T21:20:45.964043+0000","last_unstale":"2024-04-26T21:20:47.336287+0000","last_undegraded":"2024-04-26T21:20:47.336287+0000","last_fullsized":"2024-04-26T21:20:47.336287+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:44.930196+0000","last_clean_scrub_stamp":"2024-04-26T21:20:44.930196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:43.480199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:47.336648+0000","last_change":"2024-04-26T21:20:44.955598+0000","last_active":"2024-04-26T21:20:47.336648+0000","last_peered":"2024-04-26T21:20:47.336648+0000","last_clean":"2024-04-26T21:20:47.336648+0000","last_became_active":"2024-04-26T21:20:44.955310+0000","last_became_peered":"2024-04-26T21:20:44.955310+0000","last_unstale":"2024-04-26T21:20:47.336648+0000","last_undegraded":"2024-04-26T21:20:47.336648+0000","last_fullsized":"2024-04-26T21:20:47.336648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:20:43.925805+0000","last_clean_scrub_stamp":"2024-04-26T21:20:43.925805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:46.323895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-26T21:20:46.961583+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:20:46.961583+0000","last_peered":"2024-04-26T21:20:46.961583+0000","last_clean":"2024-04-26T21:20:46.961583+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:20:46.961583+0000","last_undegraded":"2024-04-26T21:20:46.961583+0000","last_fullsized":"2024-04-26T21:20:46.961583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2273280,"data_stored":1311247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509553,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1683456,"data_stored":720373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509552,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509552,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1671168,"data_stored":720227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509552,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":720317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33416,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338424,"statfs":{"total":96636764160,"available":96602546176,"internally_reserved":0,"allocated":2281472,"data_stored":1312241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1675264,"data_stored":720809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:20:54.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:20:54.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:20:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.928+0000 7fb42bfa4640 1 Processor -- start 2024-04-26T21:20:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.928+0000 7fb42bfa4640 1 -- start start 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42bfa4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4240b3620 0x7fb424160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42bfa4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb42409c150 0x7fb42406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42bfa4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4240b46f0 con 0x7fb4240b3ac0 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42bfa4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb42406d3b0 con 0x7fb4240b3620 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42bfa4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4240d1f60 con 0x7fb42409c150 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4240b3620 0x7fb424160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4240b3620 0x7fb424160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48522/0 (socket says 172.21.15.40:48522) 2024-04-26T21:20:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 -- 172.21.15.40:0/2534649306 learned_addr learned my addr 172.21.15.40:0/2534649306 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb42a541640 1 --2- 172.21.15.40:0/2534649306 >> v2:172.21.15.184:3300/0 conn(0x7fb42409c150 0x7fb42406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 -- 172.21.15.40:0/2534649306 >> v2:172.21.15.184:3300/0 conn(0x7fb42409c150 msgr2=0x7fb42406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 --2- 172.21.15.40:0/2534649306 >> v2:172.21.15.184:3300/0 conn(0x7fb42409c150 0x7fb42406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 -- 172.21.15.40:0/2534649306 >> v1:172.21.15.184:6789/0 conn(0x7fb4240b3ac0 legacy=0x7fb42409ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.929+0000 7fb429d40640 1 -- 172.21.15.40:0/2534649306 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4240d2240 con 0x7fb4240b3620 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.930+0000 7fb429d40640 1 --2- 172.21.15.40:0/2534649306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb424160f00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb42406f570 tx=0x7fb42002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908b477f0ddb0785 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.930+0000 7fb428d3e640 1 -- 172.21.15.40:0/2534649306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420037070 con 0x7fb4240b3620 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.930+0000 7fb428d3e640 1 -- 172.21.15.40:0/2534649306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb420032440 con 0x7fb4240b3620 2024-04-26T21:20:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.930+0000 7fb428d3e640 1 -- 172.21.15.40:0/2534649306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420032810 con 0x7fb4240b3620 2024-04-26T21:20:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 -- 172.21.15.40:0/2534649306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 msgr2=0x7fb424160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/2534649306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb424160f00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb42406f570 tx=0x7fb42002a230 comp rx=0 tx=0).stop 2024-04-26T21:20:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 -- 172.21.15.40:0/2534649306 shutdown_connections 2024-04-26T21:20:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/2534649306 >> v2:172.21.15.184:3300/0 conn(0x7fb42409c150 0x7fb42406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/2534649306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb424160f00 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 -- 172.21.15.40:0/2534649306 >> 172.21.15.40:0/2534649306 conn(0x7fb4240b4ab0 msgr2=0x7fb4240b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 -- 172.21.15.40:0/2534649306 shutdown_connections 2024-04-26T21:20:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.931+0000 7fb42bfa4640 1 -- 172.21.15.40:0/2534649306 wait complete. 2024-04-26T21:20:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.932+0000 7fb42bfa4640 1 Processor -- start 2024-04-26T21:20:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.932+0000 7fb42bfa4640 1 -- start start 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 0x7fb42415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 0x7fb4241f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4240b3ff0 con 0x7fb4240b3620 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb42406d600 con 0x7fb42409c150 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42bfa4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4240b4170 con 0x7fb4240b3ac0 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48534/0 (socket says 172.21.15.40:48534) 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 -- 172.21.15.40:0/1234337943 learned_addr learned my addr 172.21.15.40:0/1234337943 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb429d40640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 0x7fb42415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42a541640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 0x7fb4241f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 -- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 msgr2=0x7fb4241f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 0x7fb4241f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 -- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 msgr2=0x7fb42415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 0x7fb42415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42953f640 1 -- 172.21.15.40:0/1234337943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4241ea500 con 0x7fb4240b3620 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb429d40640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 0x7fb42415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.933+0000 7fb42a541640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 0x7fb4241f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb42953f640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c430 tx=0x7fb41400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414011790 con 0x7fb4240b3620 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414017070 con 0x7fb4240b3620 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4240c7da0 con 0x7fb4240b3620 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414011b70 con 0x7fb4240b3620 2024-04-26T21:20:54.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.934+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42415d7e0 con 0x7fb4240b3620 2024-04-26T21:20:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.935+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42406a290 con 0x7fb4240b3620 2024-04-26T21:20:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.935+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb414011d10 con 0x7fb4240b3620 2024-04-26T21:20:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.936+0000 7fb41a7fc640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 0x7fb40806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.936+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fb41408ba20 con 0x7fb4240b3620 2024-04-26T21:20:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.936+0000 7fb429d40640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 0x7fb40806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.937+0000 7fb429d40640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 0x7fb40806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb420034000 tx=0x7fb42002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:54.940+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb414059e50 con 0x7fb4240b3620 2024-04-26T21:20:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.199+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4240745c0 con 0x7fb4240b3620 2024-04-26T21:20:55.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.201+0000 7fb41a7fc640 1 -- 172.21.15.40:0/1234337943 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7fb414059750 con 0x7fb4240b3620 2024-04-26T21:20:55.202 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:20:55.202 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:20:55.202 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:20:55.202 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:20:55.203 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:20:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:20:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 msgr2=0x7fb40806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 0x7fb40806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb420034000 tx=0x7fb42002e000 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 msgr2=0x7fb4241f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c430 tx=0x7fb41400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 shutdown_connections 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb40806c8f0 0x7fb40806edb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4240b3ac0 0x7fb4241f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.203+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4240b3620 0x7fb4241f8d10 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.204+0000 7fb42bfa4640 1 --2- 172.21.15.40:0/1234337943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb42409c150 0x7fb42415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.204+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 >> 172.21.15.40:0/1234337943 conn(0x7fb4240b4ab0 msgr2=0x7fb4240afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.204+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 shutdown_connections 2024-04-26T21:20:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.204+0000 7fb42bfa4640 1 -- 172.21.15.40:0/1234337943 wait complete. 2024-04-26T21:20:55.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:20:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.315+0000 7fe73188c640 1 Processor -- start 2024-04-26T21:20:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.315+0000 7fe73188c640 1 -- start start 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe72c15f340 0x7fe72c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe72c15f720 0x7fe72c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c0d4f00 0x7fe72c0d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe72c0d58d0 con 0x7fe72c15f720 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe72c0b2510 con 0x7fe72c15f340 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe73188c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c0b27f0 con 0x7fe72c0d4f00 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe72c15f720 0x7fe72c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe72c15f720 0x7fe72c0d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48542/0 (socket says 172.21.15.40:48542) 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 -- 172.21.15.40:0/4284613962 learned_addr learned my addr 172.21.15.40:0/4284613962 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72affd640 1 --2- 172.21.15.40:0/4284613962 >> v2:172.21.15.184:3300/0 conn(0x7fe72c15f340 0x7fe72c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c0d4f00 0x7fe72c0d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 -- 172.21.15.40:0/4284613962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c0d4f00 msgr2=0x7fe72c0d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c0d4f00 0x7fe72c0d5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 -- 172.21.15.40:0/4284613962 >> v2:172.21.15.184:3300/0 conn(0x7fe72c15f340 msgr2=0x7fe72c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 --2- 172.21.15.40:0/4284613962 >> v2:172.21.15.184:3300/0 conn(0x7fe72c15f340 0x7fe72c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.316+0000 7fe72a7fc640 1 -- 172.21.15.40:0/4284613962 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0b2ad0 con 0x7fe72c15f720 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.317+0000 7fe72a7fc640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c0d3d40 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0d6000 tx=0x7fe72002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5cc223482b1a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.317+0000 7fe729ffb640 1 -- 172.21.15.40:0/4284613962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720037070 con 0x7fe72c15f720 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.317+0000 7fe729ffb640 1 -- 172.21.15.40:0/4284613962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720032440 con 0x7fe72c15f720 2024-04-26T21:20:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.317+0000 7fe729ffb640 1 -- 172.21.15.40:0/4284613962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720032810 con 0x7fe72c15f720 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 -- 172.21.15.40:0/4284613962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 msgr2=0x7fe72c0d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c0d3d40 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0d6000 tx=0x7fe72002a140 comp rx=0 tx=0).stop 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 -- 172.21.15.40:0/4284613962 shutdown_connections 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c0d4f00 0x7fe72c0d5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 --2- 172.21.15.40:0/4284613962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c0d3d40 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 --2- 172.21.15.40:0/4284613962 >> v2:172.21.15.184:3300/0 conn(0x7fe72c15f340 0x7fe72c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 -- 172.21.15.40:0/4284613962 >> 172.21.15.40:0/4284613962 conn(0x7fe72c08ec70 msgr2=0x7fe72c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 -- 172.21.15.40:0/4284613962 shutdown_connections 2024-04-26T21:20:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.318+0000 7fe73188c640 1 -- 172.21.15.40:0/4284613962 wait complete. 2024-04-26T21:20:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.319+0000 7fe73188c640 1 Processor -- start 2024-04-26T21:20:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.319+0000 7fe73188c640 1 -- start start 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72c0d4f00 0x7fe72c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c15f340 0x7fe72c1f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0b5040 con 0x7fe72c15f720 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0b4ec0 con 0x7fe72c0d4f00 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe73188c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c0b51c0 con 0x7fe72c15f340 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48558/0 (socket says 172.21.15.40:48558) 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 -- 172.21.15.40:0/3262332911 learned_addr learned my addr 172.21.15.40:0/3262332911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72affd640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72c0d4f00 0x7fe72c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72a7fc640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c15f340 0x7fe72c1f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 -- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c15f340 msgr2=0x7fe72c1f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c15f340 0x7fe72c1f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 -- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72c0d4f00 msgr2=0x7fe72c15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72c0d4f00 0x7fe72c15d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 -- 172.21.15.40:0/3262332911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c1f2ba0 con 0x7fe72c15f720 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe72b7fe640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe724007e60 tx=0x7fe72400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.320+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724014620 con 0x7fe72c15f720 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.321+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724019070 con 0x7fe72c15f720 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.321+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c1f5a90 con 0x7fe72c15f720 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.321+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724014a20 con 0x7fe72c15f720 2024-04-26T21:20:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.321+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c1f3140 con 0x7fe72c15f720 2024-04-26T21:20:55.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.322+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c1f5e90 con 0x7fe72c15f720 2024-04-26T21:20:55.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.322+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe724014bc0 con 0x7fe72c15f720 2024-04-26T21:20:55.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.323+0000 7fe713fff640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 0x7fe71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:55.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.323+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fe72408c290 con 0x7fe72c15f720 2024-04-26T21:20:55.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.323+0000 7fe72affd640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 0x7fe71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.324+0000 7fe72affd640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 0x7fe71806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe71c006fd0 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:55.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.327+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe72405a750 con 0x7fe72c15f720 2024-04-26T21:20:55.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.548+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe72c06a290 con 0x7fe72c15f720 2024-04-26T21:20:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.549+0000 7fe713fff640 1 -- 172.21.15.40:0/3262332911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5191 (secure 0 0 0) 0x7fe72405a050 con 0x7fe72c15f720 2024-04-26T21:20:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:20:55.551 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:20:48.339156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:20:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 88 2024-04-26T21:20:55.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 msgr2=0x7fe71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 0x7fe71806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe71c006fd0 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2024-04-26T21:20:55.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 msgr2=0x7fe72c1f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:55.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe724007e60 tx=0x7fe72400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 shutdown_connections 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe71806c8f0 0x7fe71806edb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe72c15f720 0x7fe72c1f54a0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe72c15f340 0x7fe72c1f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 --2- 172.21.15.40:0/3262332911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72c0d4f00 0x7fe72c15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 >> 172.21.15.40:0/3262332911 conn(0x7fe72c08ec70 msgr2=0x7fe72c0b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 shutdown_connections 2024-04-26T21:20:55.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:55.551+0000 7fe73188c640 1 -- 172.21.15.40:0/3262332911 wait complete. 2024-04-26T21:20:55.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:20:45.315641+0000', 'modified': '2024-04-26T21:20:48.339156+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9009}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9009': {'gid': 9009, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3716452175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3716452175}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3716452175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9009, 'qdb_cluster': [9009]} 2024-04-26T21:20:55.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:20:55.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T21:20:56.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.072+0000 7f92ec8da640 1 Processor -- start 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 -- start start 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92e815f370 0x7f92e80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e80d03a0 con 0x7f92e80d37d0 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f92e80b4d10 con 0x7f92e815f370 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92ec8da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92e80d0680 con 0x7f92e815f750 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92e659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92e815f370 0x7f92e80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.073+0000 7f92e6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f92e80d37d0 0x7f92e80d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49718/0 (socket says 172.21.15.40:49718) 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e6d9d640 1 -- 172.21.15.40:0/4272376446 learned_addr learned my addr 172.21.15.40:0/4272376446 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92dffff640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e659c640 1 -- 172.21.15.40:0/4272376446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 msgr2=0x7f92e80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e659c640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e659c640 1 -- 172.21.15.40:0/4272376446 >> v1:172.21.15.40:6789/0 conn(0x7f92e80d37d0 legacy=0x7f92e80d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e659c640 1 -- 172.21.15.40:0/4272376446 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e80d0960 con 0x7f92e815f370 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.074+0000 7f92e5d9b640 1 -- 172.21.15.40:0/4272376446 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1816169477 0 0) 0x7f92e80d03a0 con 0x7f92e80d37d0 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92e659c640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80cfd70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d5005d10c258e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92e5d9b640 1 -- 172.21.15.40:0/4272376446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8037070 con 0x7f92e815f370 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92e5d9b640 1 -- 172.21.15.40:0/4272376446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d802ae50 con 0x7f92e815f370 2024-04-26T21:20:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92e5d9b640 1 -- 172.21.15.40:0/4272376446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8032660 con 0x7f92e815f370 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 -- 172.21.15.40:0/4272376446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 msgr2=0x7f92e80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80cfd70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d802a100 comp rx=0 tx=0).stop 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 -- 172.21.15.40:0/4272376446 shutdown_connections 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 --2- 172.21.15.40:0/4272376446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80cfd70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 -- 172.21.15.40:0/4272376446 >> 172.21.15.40:0/4272376446 conn(0x7f92e808ec10 msgr2=0x7f92e809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 -- 172.21.15.40:0/4272376446 shutdown_connections 2024-04-26T21:20:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.075+0000 7f92ec8da640 1 -- 172.21.15.40:0/4272376446 wait complete. 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 Processor -- start 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 -- start start 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e80d37d0 0x7f92e815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92e80b51c0 con 0x7f92e815f370 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92e80d0f20 con 0x7f92e80d37d0 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.077+0000 7f92ec8da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92e80b5340 con 0x7f92e815f750 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48576/0 (socket says 172.21.15.40:48576) 2024-04-26T21:20:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 -- 172.21.15.40:0/1469353734 learned_addr learned my addr 172.21.15.40:0/1469353734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92e659c640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e80d37d0 0x7f92e815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 -- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 msgr2=0x7f92e80d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 -- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e80d37d0 msgr2=0x7f92e815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e80d37d0 0x7f92e815c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 -- 172.21.15.40:0/1469353734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e80cd2d0 con 0x7f92e815f370 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92dffff640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f92d000c7f0 tx=0x7f92d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92deffd640 1 -- 172.21.15.40:0/1469353734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0015070 con 0x7f92e815f370 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92deffd640 1 -- 172.21.15.40:0/1469353734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0004220 con 0x7f92e815f370 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e80cd5b0 con 0x7f92e815f370 2024-04-26T21:20:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92deffd640 1 -- 172.21.15.40:0/1469353734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0007620 con 0x7f92e815f370 2024-04-26T21:20:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.078+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e80cdb70 con 0x7f92e815f370 2024-04-26T21:20:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.080+0000 7f92deffd640 1 -- 172.21.15.40:0/1469353734 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92d00079d0 con 0x7f92e815f370 2024-04-26T21:20:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.081+0000 7f92deffd640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.081+0000 7f92deffd640 1 -- 172.21.15.40:0/1469353734 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f92d008bef0 con 0x7f92e815f370 2024-04-26T21:20:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.081+0000 7f92e659c640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.081+0000 7f92ef5fb740 1 Processor -- start 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 -- start start 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92e659c640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 0x7f92c806edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f92d8034000 tx=0x7f92d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545386f440 0x56545386f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545388d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5654537a9740 con 0x565453845120 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654537453a0 con 0x56545388cce0 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x5654537a5700 con 0x56545386f440 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92e6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48580/0 (socket says 172.21.15.40:48580) 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.082+0000 7f92e6d9d640 1 -- 172.21.15.40:0/2767866154 learned_addr learned my addr 172.21.15.40:0/2767866154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92dffff640 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545388d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e659c640 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545386f440 0x56545386f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 -- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545386f440 msgr2=0x56545386f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545386f440 0x56545386f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 -- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 msgr2=0x56545388d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545388d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 -- 172.21.15.40:0/2767866154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565453761750 con 0x565453845120 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f92e00072b0 tx=0x7f92e0007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fd9b80f7e3a620 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92c7fff640 1 -- 172.21.15.40:0/2767866154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0019070 con 0x565453845120 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92c7fff640 1 -- 172.21.15.40:0/2767866154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e00079c0 con 0x565453845120 2024-04-26T21:20:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.083+0000 7f92c7fff640 1 -- 172.21.15.40:0/2767866154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0001c10 con 0x565453845120 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- 172.21.15.40:0/2767866154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 msgr2=0x56545388b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f92e00072b0 tx=0x7f92e0007770 comp rx=0 tx=0).stop 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- 172.21.15.40:0/2767866154 shutdown_connections 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565453845120 0x56545388b3a0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545388d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/2767866154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545386f440 0x56545386f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- 172.21.15.40:0/2767866154 >> 172.21.15.40:0/2767866154 conn(0x565453841a50 msgr2=0x56545388a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- 172.21.15.40:0/2767866154 shutdown_connections 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- 172.21.15.40:0/2767866154 wait complete. 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 -- start start 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.084+0000 7f92ef5fb740 1 Processor -- start 2024-04-26T21:20:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 0x56545385cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x56545388b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x565453845ec0 con 0x56545388cce0 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565453730fe0 con 0x56545386f440 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56545383f980 con 0x56545388b530 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.085+0000 7f92ef5fb740 10 client.0 _ll_register_callbacks cb 0x565453747150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48594/0 (socket says 172.21.15.40:48594) 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 -- 172.21.15.40:0/1089682480 learned_addr learned my addr 172.21.15.40:0/1089682480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x56545388b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92e659c640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 0x56545385cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 -- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 msgr2=0x56545388b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x56545388b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 -- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 msgr2=0x56545385cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 0x56545385cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 -- 172.21.15.40:0/1089682480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654534bb2d0 con 0x56545388cce0 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92e659c640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 0x56545385cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92dffff640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f92d0059940 tx=0x7f92d005e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec793fa866b9f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0088060 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f92d0062570 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e00079c0 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f92d0077050 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92ef5fb740 10 client.9159 fetch_fsmap learned FSMap version 88 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5654535856b0 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.086+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e00079c0 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f92d0059090 con 0x56545388cce0 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 10 client.9159 fetch_fsmap finished waiting for FSMap version 88 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 10 client.9159 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 20 client.9159 populate_metadata read hostname 'smithi040' 2024-04-26T21:20:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x565453881e00 0x5654538821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 4 client.9159 mds_command: new command op to 9009 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 --> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654537b7350 con 0x565453881e00 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92e659c640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x565453881e00 0x5654538821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92c57fa640 10 client.9159 ms_handle_connect on v2:172.21.15.40:6834/3716452175 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.087+0000 7f92e659c640 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x565453881e00 0x5654538821e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c151b4dcb12daf2 server_cookie=abe553ee0a3fca4f in_seq=0 out_seq=0 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92c57fa640 1 -- 172.21.15.40:0/1089682480 <== mds.0 v2:172.21.15.40:6834/3716452175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654537b7350 con 0x565453881e00 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92c57fa640 10 client.9159 handle_command_reply: tid=0 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 client.9159 shutdown 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 msgr2=0x565453871ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f92d0059940 tx=0x7f92d005e080 comp rx=0 tx=0).stop 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 shutdown_connections 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x56545388b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x565453881e00 0x5654538821e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545388cce0 0x565453871ed0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1089682480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545386f440 0x56545385cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 >> 172.21.15.40:0/1089682480 conn(0x565453845120 msgr2=0x56545388e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92c57fa640 0 client.9159 ms_handle_reset on v2:172.21.15.40:6834/3716452175 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 shutdown_connections 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1089682480 wait complete. 2024-04-26T21:20:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.088+0000 7f92ef5fb740 20 client.9159 trim_cache size 0 max 16384 2024-04-26T21:20:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.107+0000 7f92ef5fb740 1 Processor -- start 2024-04-26T21:20:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.107+0000 7f92ef5fb740 1 -- start start 2024-04-26T21:20:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x5654538456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545386a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8201820 con 0x56545386f440 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8200830 con 0x56545388cce0 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f92e81e7d10 con 0x56545388b530 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48602/0 (socket says 172.21.15.40:48602) 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 -- 172.21.15.40:0/1888652192 learned_addr learned my addr 172.21.15.40:0/1888652192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545386a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92dffff640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x5654538456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 -- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 msgr2=0x5654538456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x5654538456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 -- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 msgr2=0x56545386a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545386a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 -- 172.21.15.40:0/1888652192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565453884300 con 0x56545386f440 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92dffff640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x5654538456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545386a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.108+0000 7f92e659c640 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f92d803e040 tx=0x7f92d802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8370486273b90a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.109+0000 7f92c77fe640 1 -- 172.21.15.40:0/1888652192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8037040 con 0x56545386f440 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.109+0000 7f92c77fe640 1 -- 172.21.15.40:0/1888652192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8056070 con 0x56545386f440 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.109+0000 7f92c77fe640 1 -- 172.21.15.40:0/1888652192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80505c0 con 0x56545386f440 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1888652192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 msgr2=0x56545388b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f92d803e040 tx=0x7f92d802c000 comp rx=0 tx=0).stop 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1888652192 shutdown_connections 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388cce0 0x56545386a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388b530 0x5654538456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/1888652192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x56545388b9b0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1888652192 >> 172.21.15.40:0/1888652192 conn(0x565453845120 msgr2=0x56545385ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1888652192 shutdown_connections 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.110+0000 7f92ef5fb740 1 -- 172.21.15.40:0/1888652192 wait complete. 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.111+0000 7f92ef5fb740 1 -- start start 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.111+0000 7f92ef5fb740 1 Processor -- start 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388b530 0x565453888d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388cce0 0x565453889290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5654538705d0 con 0x56545386f440 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92e81e8360 con 0x56545388b530 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56545389dec0 con 0x56545388cce0 2024-04-26T21:20:56.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 10 client.0 _ll_register_callbacks cb 0x7f92d0055ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48604/0 (socket says 172.21.15.40:48604) 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 -- 172.21.15.40:0/3734167433 learned_addr learned my addr 172.21.15.40:0/3734167433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e6d9d640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388cce0 0x565453889290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92dffff640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388b530 0x565453888d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 -- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388cce0 msgr2=0x565453889290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388cce0 0x565453889290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 -- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388b530 msgr2=0x565453888d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388b530 0x565453888d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 -- 172.21.15.40:0/3734167433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545386e240 con 0x56545386f440 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92e659c640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1176f51debd1883d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8040030 con 0x56545386f440 2024-04-26T21:20:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f92d8050600 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.112+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d8056070 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f92d8069050 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 10 client.9165 fetch_fsmap learned FSMap version 88 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x565453882460 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8056070 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f92d806c020 con 0x56545386f440 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 10 client.9165 fetch_fsmap finished waiting for FSMap version 88 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 10 client.9165 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 20 client.9165 populate_metadata read hostname 'smithi040' 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x5654538ad560 0x56545385b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 4 client.9165 mds_command: new command op to 9009 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 --> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] -- command(tid 0: {"prefix": "status"}) -- 0x5654537b11c0 con 0x5654538ad560 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92dffff640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x5654538ad560 0x56545385b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92c57fa640 10 client.9165 ms_handle_connect on v2:172.21.15.40:6834/3716452175 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92dffff640 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x5654538ad560 0x56545385b0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389fcb60a3442a6c server_cookie=cdab690faa52b06f in_seq=0 out_seq=0 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.113+0000 7f92c57fa640 1 -- 172.21.15.40:0/3734167433 <== mds.0 v2:172.21.15.40:6834/3716452175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654537b11c0 con 0x5654538ad560 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92c57fa640 10 client.9165 handle_command_reply: tid=0 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 client.9165 shutdown 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 msgr2=0x565453882920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d8004730 comp rx=0 tx=0).stop 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 shutdown_connections 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56545388cce0 0x565453889290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56545388b530 0x565453888d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:6834/3716452175,v1:172.21.15.40:6835/3716452175] conn(0x5654538ad560 0x56545385b0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 --2- 172.21.15.40:0/3734167433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56545386f440 0x565453882920 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92c57fa640 0 client.9165 ms_handle_reset on v2:172.21.15.40:6834/3716452175 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 >> 172.21.15.40:0/3734167433 conn(0x565453845120 msgr2=0x565453880580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 shutdown_connections 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 1 -- 172.21.15.40:0/3734167433 wait complete. 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ef5fb740 20 client.9165 trim_cache size 0 max 16384 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:20:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 9009, 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.157551409, 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 88, 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 134, 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 134, 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.228015168000001 2024-04-26T21:20:56.123 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 msgr2=0x7f92c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 0x7f92c806edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f92d8034000 tx=0x7f92d802e000 comp rx=0 tx=0).stop 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 msgr2=0x7f92e80d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f92d000c7f0 tx=0x7f92d000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 shutdown_connections 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e815f750 0x7f92e80d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92e815f370 0x7f92e80d4fa0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 --2- 172.21.15.40:0/1469353734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e80d37d0 0x7f92e815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.114+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 >> 172.21.15.40:0/1469353734 conn(0x7f92e808ec10 msgr2=0x7f92e80b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.115+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 shutdown_connections 2024-04-26T21:20:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.115+0000 7f92ec8da640 1 -- 172.21.15.40:0/1469353734 wait complete. 2024-04-26T21:20:56.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:20:56.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.226+0000 7f4d9c862640 1 Processor -- start 2024-04-26T21:20:56.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d9c862640 1 -- start start 2024-04-26T21:20:56.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d9c862640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4d9815f370 0x7f4d980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d9c862640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d980d03a0 con 0x7f4d9815f750 2024-04-26T21:20:56.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d9c862640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d980d0680 con 0x7f4d980d3ee0 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d9c862640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4d980d0960 con 0x7f4d9815f370 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d8ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4d9815f750 0x7f4d980d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49722/0 (socket says 172.21.15.40:49722) 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.227+0000 7f4d8ffff640 1 -- 172.21.15.40:0/2298988071 learned_addr learned my addr 172.21.15.40:0/2298988071 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d9659c640 1 --2- 172.21.15.40:0/2298988071 >> v2:172.21.15.184:3300/0 conn(0x7f4d9815f370 0x7f4d980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190332539 0 0) 0x7f4d980d03a0 con 0x7f4d9815f750 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d84003520 con 0x7f4d9815f750 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2108692073 0 0) 0x7f4d84003520 con 0x7f4d9815f750 2024-04-26T21:20:56.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 >> v2:172.21.15.184:3300/0 conn(0x7f4d9815f370 msgr2=0x7f4d980cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 --2- 172.21.15.40:0/2298988071 >> v2:172.21.15.184:3300/0 conn(0x7f4d9815f370 0x7f4d980cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 >> v1:172.21.15.184:6789/0 conn(0x7f4d980d3ee0 legacy=0x7f4d980b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d980d0c40 con 0x7f4d9815f750 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f4d80001670 con 0x7f4d9815f750 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 >> v1:172.21.15.40:6789/0 conn(0x7f4d9815f750 legacy=0x7f4d980d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d840039b0 0x7f4d84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d840051c0 0x7f4d8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980d0960 con 0x7f4d84004800 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8400c060 con 0x7f4d840051c0 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8400c260 con 0x7f4d840039b0 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d9659c640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d840039b0 0x7f4d84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d96d9d640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d840051c0 0x7f4d8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d8ffff640 1 -- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d840039b0 msgr2=0x7f4d84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d840039b0 0x7f4d84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.228+0000 7f4d8ffff640 1 -- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d840051c0 msgr2=0x7f4d8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d840051c0 0x7f4d8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d8ffff640 1 -- 172.21.15.40:0/2298988071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d84003520 con 0x7f4d84004800 2024-04-26T21:20:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d96d9d640 1 -- 172.21.15.40:0/2298988071 reap_dead start 2024-04-26T21:20:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d84004c80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000c240 tx=0x7f4d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca849d94e6fc2c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80007cf0 con 0x7f4d84004800 2024-04-26T21:20:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d800058d0 con 0x7f4d84004800 2024-04-26T21:20:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.229+0000 7f4d95d9b640 1 -- 172.21.15.40:0/2298988071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80005d20 con 0x7f4d84004800 2024-04-26T21:20:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 -- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 msgr2=0x7f4d84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d84004c80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000c240 tx=0x7f4d84005830 comp rx=0 tx=0).stop 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 -- 172.21.15.40:0/2298988071 shutdown_connections 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 --2- 172.21.15.40:0/2298988071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d84004c80 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 -- 172.21.15.40:0/2298988071 >> 172.21.15.40:0/2298988071 conn(0x7f4d9808ec10 msgr2=0x7f4d9809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 -- 172.21.15.40:0/2298988071 shutdown_connections 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.230+0000 7f4d9c862640 1 -- 172.21.15.40:0/2298988071 wait complete. 2024-04-26T21:20:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.231+0000 7f4d9c862640 1 Processor -- start 2024-04-26T21:20:56.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.231+0000 7f4d9c862640 1 -- start start 2024-04-26T21:20:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 0x7f4d981505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 0x7f4d980b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980cfef0 con 0x7f4d84004800 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980cfd70 con 0x7f4d980cb500 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9c862640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d980d0070 con 0x7f4d980c9ef0 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48620/0 (socket says 172.21.15.40:48620) 2024-04-26T21:20:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 -- 172.21.15.40:0/1811782478 learned_addr learned my addr 172.21.15.40:0/1811782478 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 0x7f4d981505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d96d9d640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 0x7f4d980b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 -- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 msgr2=0x7f4d981505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 0x7f4d981505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 -- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 msgr2=0x7f4d980b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 0x7f4d980b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d9659c640 1 -- 172.21.15.40:0/1811782478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d980ca460 con 0x7f4d84004800 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d96d9d640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 0x7f4d980b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.232+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 0x7f4d981505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d9659c640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4d880114e0 tx=0x7f4d880119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88015870 con 0x7f4d84004800 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88015a10 con 0x7f4d84004800 2024-04-26T21:20:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d980ca740 con 0x7f4d84004800 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88015dc0 con 0x7f4d84004800 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.233+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d980cad00 con 0x7f4d84004800 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.234+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9806a290 con 0x7f4d84004800 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.234+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d88013400 con 0x7f4d84004800 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.235+0000 7f4d8effd640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 0x7f4d7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.235+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f4d8808c440 con 0x7f4d84004800 2024-04-26T21:20:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.235+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 0x7f4d7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.235+0000 7f4d8ffff640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 0x7f4d7806ede0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4d80033000 tx=0x7f4d80009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.239+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d88019030 con 0x7f4d84004800 2024-04-26T21:20:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.464+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4d980d1690 con 0x7f4d84004800 2024-04-26T21:20:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.464+0000 7f4d8effd640 1 -- 172.21.15.40:0/1811782478 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f4d8805a870 con 0x7f4d84004800 2024-04-26T21:20:56.465 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:20:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 msgr2=0x7f4d7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 0x7f4d7806ede0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4d80033000 tx=0x7f4d80009030 comp rx=0 tx=0).stop 2024-04-26T21:20:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 msgr2=0x7f4d980b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4d880114e0 tx=0x7f4d880119a0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 shutdown_connections 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d7806c920 0x7f4d7806ede0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d980cb500 0x7f4d980b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d980c9ef0 0x7f4d981505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 --2- 172.21.15.40:0/1811782478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d84004800 0x7f4d980b7df0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 >> 172.21.15.40:0/1811782478 conn(0x7f4d9808ec10 msgr2=0x7f4d9806e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 shutdown_connections 2024-04-26T21:20:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56.466+0000 7f4d9c862640 1 -- 172.21.15.40:0/1811782478 wait complete. 2024-04-26T21:20:56.478 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:20:56.478 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:20:56.478 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:20:56.478 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:20:56.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:56.479 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:20:56.480 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:20:56.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:20:56.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:56.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.480 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:56.496 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42213sec preferred_lft 42213sec 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:20:56.497 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:56.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.498 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:20:56.558 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:20:56.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.560 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:20:56.616 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:20:56.616 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:20:56.616 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:20:56.616 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:20:56.617 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:20:56.617 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:20:56.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:20:56.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:56.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.743 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:20:56.758 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:20:56.758 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:20:56.758 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:56.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:20:56.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:56.822 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:56.878 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:20:56.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:56.879 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:56.934 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:20:56.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:56.934 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:57.017 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:57.017 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:57.017 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:57.025 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:57.026 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:57.026 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:57.026 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:57.026 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:57.030 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:20:57.030 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:20:57.030 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:20:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:20:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:20:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:20:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:20:57.033 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:20:57.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:20:57.097 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:20:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:20:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:57.199 INFO:teuthology.orchestra.run.smithi040.stdout:client6836 2024-04-26T21:20:57.199 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:20:57.199 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:57.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:20:57.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.355+0000 7f8169797640 1 Processor -- start 2024-04-26T21:20:57.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.355+0000 7f8169797640 1 -- start start 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f8169797640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81640cf2e0 0x7f816406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f8169797640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f816415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f8169797640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640d1740 con 0x7f81640d11b0 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f8169797640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f81640d1a20 con 0x7f81640cf2e0 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f8169797640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81640d1d00 con 0x7f816406e490 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f81627fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81640d11b0 0x7f816406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49732/0 (socket says 172.21.15.40:49732) 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.356+0000 7f81627fc640 1 -- 172.21.15.40:0/659483752 learned_addr learned my addr 172.21.15.40:0/659483752 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494029167 0 0) 0x7f81640d1740 con 0x7f81640d11b0 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8162ffd640 1 --2- 172.21.15.40:0/659483752 >> v2:172.21.15.184:3300/0 conn(0x7f81640cf2e0 0x7f816406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f81637fe640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f816415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8150003520 con 0x7f81640d11b0 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2271918353 0 0) 0x7f8150003520 con 0x7f81640d11b0 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 msgr2=0x7f816415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f816415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 >> v2:172.21.15.184:3300/0 conn(0x7f81640cf2e0 msgr2=0x7f816406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 --2- 172.21.15.40:0/659483752 >> v2:172.21.15.184:3300/0 conn(0x7f81640cf2e0 0x7f816406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640d1fe0 con 0x7f81640d11b0 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f814c001670 con 0x7f81640d11b0 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 >> v1:172.21.15.40:6789/0 conn(0x7f81640d11b0 legacy=0x7f816406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f8150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f81500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d1a20 con 0x7f8150004d40 2024-04-26T21:20:57.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d1d00 con 0x7f81500039b0 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f815000b9c0 con 0x7f816406e490 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f81637fe640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f81500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f8162ffd640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f81627fc640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f8150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f81637fe640 1 -- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 msgr2=0x7f8150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.357+0000 7f81637fe640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816406e490 0x7f8150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f81637fe640 1 -- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81500039b0 msgr2=0x7f8150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f81637fe640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f81637fe640 1 -- 172.21.15.40:0/659483752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8150003520 con 0x7f8150004d40 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f81637fe640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f81500051c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f816406e8a0 tx=0x7f8150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e017163c9a4ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037070 con 0x7f8150004d40 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8154032600 con 0x7f8150004d40 2024-04-26T21:20:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.358+0000 7f8161ffb640 1 -- 172.21.15.40:0/659483752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154009c10 con 0x7f8150004d40 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 -- 172.21.15.40:0/659483752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 msgr2=0x7f81500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 --2- 172.21.15.40:0/659483752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f81500051c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f816406e8a0 tx=0x7f8150005830 comp rx=0 tx=0).stop 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f81637fe640 1 -- 172.21.15.40:0/659483752 reap_dead start 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 -- 172.21.15.40:0/659483752 shutdown_connections 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 -- 172.21.15.40:0/659483752 >> 172.21.15.40:0/659483752 conn(0x7f81640b4a60 msgr2=0x7f81640b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 -- 172.21.15.40:0/659483752 shutdown_connections 2024-04-26T21:20:57.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.359+0000 7f8169797640 1 -- 172.21.15.40:0/659483752 wait complete. 2024-04-26T21:20:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.360+0000 7f8169797640 1 Processor -- start 2024-04-26T21:20:57.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.360+0000 7f8169797640 1 -- start start 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816414c0a0 0x7f816414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816415f530 0x7f816415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81640cf6c0 con 0x7f8150004d40 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f816406da10 con 0x7f816415f530 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8169797640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81640cf840 con 0x7f816414c0a0 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48632/0 (socket says 172.21.15.40:48632) 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 -- 172.21.15.40:0/716990846 learned_addr learned my addr 172.21.15.40:0/716990846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f81627fc640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816414c0a0 0x7f816414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f81637fe640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816415f530 0x7f816415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 -- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816414c0a0 msgr2=0x7f816414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816414c0a0 0x7f816414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 -- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816415f530 msgr2=0x7f816415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816415f530 0x7f816415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 -- 172.21.15.40:0/716990846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816414bb60 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.361+0000 7f8162ffd640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f815c011aa0 tx=0x7f815c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.362+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c00fb80 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.362+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c00fd20 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.362+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816414cb40 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.362+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c014660 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.362+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816414d100 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.363+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816406a290 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.363+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f815c012040 con 0x7f8150004d40 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.363+0000 7f815b7fe640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 0x7f814406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.364+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f815c08c240 con 0x7f8150004d40 2024-04-26T21:20:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.364+0000 7f81627fc640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 0x7f814406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.364+0000 7f81627fc640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 0x7f814406edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f81640b8750 tx=0x7f814c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.368+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f815c019030 con 0x7f8150004d40 2024-04-26T21:20:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.595+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f81640b8ef0 con 0x7f8150004d40 2024-04-26T21:20:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.595+0000 7f815b7fe640 1 -- 172.21.15.40:0/716990846 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f815c05a670 con 0x7f8150004d40 2024-04-26T21:20:57.596 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 msgr2=0x7f814406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 0x7f814406edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f81640b8750 tx=0x7f814c009040 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 msgr2=0x7f816414b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f815c011aa0 tx=0x7f815c011f60 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 shutdown_connections 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f814406c8f0 0x7f814406edb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816415f530 0x7f816415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f816414c0a0 0x7f816414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 --2- 172.21.15.40:0/716990846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150004d40 0x7f816414b620 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 >> 172.21.15.40:0/716990846 conn(0x7f81640b4a60 msgr2=0x7f8164162ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 shutdown_connections 2024-04-26T21:20:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:57.597+0000 7f8169797640 1 -- 172.21.15.40:0/716990846 wait complete. 2024-04-26T21:20:57.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:20:57.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:20:57.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:57.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.611 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:20:57.631 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:20:57.631 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:20:57.631 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:20:57.631 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42216sec preferred_lft 42216sec 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:20:57.633 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:20:57.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.634 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:20:57.697 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:20:57.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.699 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:20:57.756 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:20:57.757 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:20:57.757 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:20:57.757 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:20:57.757 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:20:57.757 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:20:57.757 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:20:57.757 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:20:57.757 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:20:57.758 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:20:57.758 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:20:57.849 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:20:57.880 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:57.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.887 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:20:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:20:57.902 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:20:57.902 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:57.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:57.902 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:20:57.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:57.966 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:58.021 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:20:58.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:20:58.022 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:58.076 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:20:58.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:58.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:58.158 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:58.158 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:58.158 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:58.166 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:58.166 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:58.166 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:58.167 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:58.167 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:20:58.172 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:20:58.175 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:20:58.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:20:58.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:20:58.238 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:20:58.327 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:20:58.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:20:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:20:58.335 INFO:teuthology.orchestra.run.smithi184.stdout:client9180 2024-04-26T21:20:58.335 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:20:58.335 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:20:58.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-26T21:20:58.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.465+0000 7fa74cfbf640 1 Processor -- start 2024-04-26T21:20:58.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.465+0000 7fa74cfbf640 1 -- start start 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.466+0000 7fa74cfbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 0x7fa7480d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.466+0000 7fa74cfbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa748163aa0 0x7fa7480b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.466+0000 7fa74cfbf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7480d3f50 con 0x7fa7480b60b0 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.466+0000 7fa74cfbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa74815fa60 con 0x7fa748163aa0 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.466+0000 7fa74cfbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7480b5090 con 0x7fa74815f680 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa745d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa748163aa0 0x7fa7480b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa74659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 0x7fa7480d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa746d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fa7480b60b0 0x7fa7480d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:39958/0 (socket says 172.21.15.40:39958) 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa746d9d640 1 -- 172.21.15.40:0/2291380845 learned_addr learned my addr 172.21.15.40:0/2291380845 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa74559a640 1 -- 172.21.15.40:0/2291380845 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017472572 0 0) 0x7fa7480d3f50 con 0x7fa7480b60b0 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa745d9b640 1 -- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 msgr2=0x7fa7480d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa745d9b640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 0x7fa7480d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa745d9b640 1 -- 172.21.15.40:0/2291380845 >> v1:172.21.15.184:6789/0 conn(0x7fa7480b60b0 legacy=0x7fa7480d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa745d9b640 1 -- 172.21.15.40:0/2291380845 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7480b5370 con 0x7fa748163aa0 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.467+0000 7fa74659c640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 0x7fa7480d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa745d9b640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa748163aa0 0x7fa7480b5b70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0097a0 tx=0x7fa73c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3865752c384ba9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa74559a640 1 -- 172.21.15.40:0/2291380845 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c037070 con 0x7fa748163aa0 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa74559a640 1 -- 172.21.15.40:0/2291380845 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c032440 con 0x7fa748163aa0 2024-04-26T21:20:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa74559a640 1 -- 172.21.15.40:0/2291380845 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c032810 con 0x7fa748163aa0 2024-04-26T21:20:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa748163aa0 msgr2=0x7fa7480b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.468+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa748163aa0 0x7fa7480b5b70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0097a0 tx=0x7fa73c02a180 comp rx=0 tx=0).stop 2024-04-26T21:20:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2291380845 shutdown_connections 2024-04-26T21:20:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa748163aa0 0x7fa7480b5b70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2291380845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa74815f680 0x7fa7480d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2291380845 >> 172.21.15.40:0/2291380845 conn(0x7fa74808eca0 msgr2=0x7fa74809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2291380845 shutdown_connections 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.469+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2291380845 wait complete. 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.470+0000 7fa74cfbf640 1 Processor -- start 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.470+0000 7fa74cfbf640 1 -- start start 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7480b60b0 0x7fa7480cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 0x7fa7480cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7480d57b0 con 0x7fa748163aa0 2024-04-26T21:20:58.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7480d5630 con 0x7fa74815f680 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74cfbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa7480d5930 con 0x7fa7480b60b0 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa74659c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7480b60b0 0x7fa7480cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48642/0 (socket says 172.21.15.40:48642) 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 -- 172.21.15.40:0/2987013443 learned_addr learned my addr 172.21.15.40:0/2987013443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa745d9b640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 0x7fa7480cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 -- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7480b60b0 msgr2=0x7fa7480cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7480b60b0 0x7fa7480cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 -- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 msgr2=0x7fa7480cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.471+0000 7fa746d9d640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 0x7fa7480cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa746d9d640 1 -- 172.21.15.40:0/2987013443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7480cca40 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa745d9b640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 0x7fa7480cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa746d9d640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa73800c7a0 tx=0x7fa73800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738018070 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73800a640 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73800aab0 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7480ccd20 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.472+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7480cd2e0 con 0x7fa748163aa0 2024-04-26T21:20:58.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.473+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74806a290 con 0x7fa748163aa0 2024-04-26T21:20:58.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.473+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa73801b070 con 0x7fa748163aa0 2024-04-26T21:20:58.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.474+0000 7fa736ffd640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 0x7fa72806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.474+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fa73808b040 con 0x7fa748163aa0 2024-04-26T21:20:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.474+0000 7fa74659c640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 0x7fa72806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.475+0000 7fa74659c640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 0x7fa72806edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa740009560 tx=0x7fa740009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.479+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa738058eb0 con 0x7fa748163aa0 2024-04-26T21:20:58.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.668+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7fa7480b4cd0 con 0x7fa748163aa0 2024-04-26T21:20:58.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.672+0000 7fa736ffd640 1 -- 172.21.15.40:0/2987013443 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v57)=0 v57) ==== 121+0+0 (secure 0 0 0) 0x7fa73805d1b0 con 0x7fa748163aa0 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 msgr2=0x7fa72806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 0x7fa72806edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa740009560 tx=0x7fa740009190 comp rx=0 tx=0).stop 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 msgr2=0x7fa748154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa73800c7a0 tx=0x7fa73800cc60 comp rx=0 tx=0).stop 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 shutdown_connections 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa72806c8f0 0x7fa72806edb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa748163aa0 0x7fa748154800 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa74815f680 0x7fa7480cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 --2- 172.21.15.40:0/2987013443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7480b60b0 0x7fa7480cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 >> 172.21.15.40:0/2987013443 conn(0x7fa74808eca0 msgr2=0x7fa7480d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 shutdown_connections 2024-04-26T21:20:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.674+0000 7fa74cfbf640 1 -- 172.21.15.40:0/2987013443 wait complete. 2024-04-26T21:20:58.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-04-26T21:20:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.788+0000 7fcd84ac8640 1 Processor -- start 2024-04-26T21:20:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 -- start start 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd800d3820 0x7fcd800b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd800b60b0 0x7fcd800d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd800d3c90 con 0x7fcd8015f680 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8015fa60 con 0x7fcd800d3820 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.789+0000 7fcd84ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd800d3f50 con 0x7fcd800b60b0 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd800d3820 0x7fcd800b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd800d3820 0x7fcd800b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:59678/0 (socket says 172.21.15.40:59678) 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/444158283 learned_addr learned my addr 172.21.15.40:0/444158283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7ed9d640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd800b60b0 0x7fcd800d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd800b60b0 msgr2=0x7fcd800d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd800b60b0 0x7fcd800d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/444158283 >> v1:172.21.15.184:6789/0 conn(0x7fcd8015f680 legacy=0x7fcd80163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7d59a640 1 -- 172.21.15.40:0/444158283 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595279261 0 0) 0x7fcd800d3c90 con 0x7fcd8015f680 2024-04-26T21:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.790+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/444158283 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd800b5090 con 0x7fcd800d3820 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.791+0000 7fcd7dd9b640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800d3820 0x7fcd800b5b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009870 tx=0x7fcd6802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e696db15dc6097 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.791+0000 7fcd7d59a640 1 -- 172.21.15.40:0/444158283 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037070 con 0x7fcd800d3820 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.791+0000 7fcd7d59a640 1 -- 172.21.15.40:0/444158283 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68032440 con 0x7fcd800d3820 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.791+0000 7fcd7d59a640 1 -- 172.21.15.40:0/444158283 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68032810 con 0x7fcd800d3820 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 -- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800d3820 msgr2=0x7fcd800b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800d3820 0x7fcd800b5b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009870 tx=0x7fcd6802a270 comp rx=0 tx=0).stop 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 -- 172.21.15.40:0/444158283 shutdown_connections 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd800b60b0 0x7fcd800d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/444158283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800d3820 0x7fcd800b5b70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 -- 172.21.15.40:0/444158283 >> 172.21.15.40:0/444158283 conn(0x7fcd8008eca0 msgr2=0x7fcd8009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 -- 172.21.15.40:0/444158283 shutdown_connections 2024-04-26T21:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.792+0000 7fcd84ac8640 1 -- 172.21.15.40:0/444158283 wait complete. 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.793+0000 7fcd84ac8640 1 Processor -- start 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.793+0000 7fcd84ac8640 1 -- start start 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800b60b0 0x7fcd800cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 0x7fcd800ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcd800d55e0 con 0x7fcd800d3820 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd800d5460 con 0x7fcd800b60b0 2024-04-26T21:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd84ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fcd800d5760 con 0x7fcd8015f680 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48656/0 (socket says 172.21.15.40:48656) 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/3669917624 learned_addr learned my addr 172.21.15.40:0/3669917624 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7ed9d640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 0x7fcd800ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7e59c640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800b60b0 0x7fcd800cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 msgr2=0x7fcd800ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 0x7fcd800ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800b60b0 msgr2=0x7fcd800cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800b60b0 0x7fcd800cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 -- 172.21.15.40:0/3669917624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd80162680 con 0x7fcd800d3820 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7ed9d640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 0x7fcd800ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.794+0000 7fcd7dd9b640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcd68035040 tx=0x7fcd680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.795+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037040 con 0x7fcd800d3820 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.795+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68004030 con 0x7fcd800d3820 2024-04-26T21:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.795+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004420 con 0x7fcd800d3820 2024-04-26T21:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.795+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd801616a0 con 0x7fcd800d3820 2024-04-26T21:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.795+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd80154cd0 con 0x7fcd800d3820 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.796+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8006a290 con 0x7fcd800d3820 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.796+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd6803a070 con 0x7fcd800d3820 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.797+0000 7fcd76ffd640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 0x7fcd6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.797+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fcd680ac0a0 con 0x7fcd800d3820 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.797+0000 7fcd7e59c640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 0x7fcd6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.797+0000 7fcd7e59c640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 0x7fcd6006edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000f60 tx=0x7fcd78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.801+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd68041020 con 0x7fcd800d3820 2024-04-26T21:20:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.985+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7fcd800b4cd0 con 0x7fcd800d3820 2024-04-26T21:20:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.988+0000 7fcd76ffd640 1 -- 172.21.15.40:0/3669917624 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v58)=0 v58) ==== 119+0+0 (secure 0 0 0) 0x7fcd6807a560 con 0x7fcd800d3820 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.990+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 msgr2=0x7fcd6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.990+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 0x7fcd6006edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000f60 tx=0x7fcd78009040 comp rx=0 tx=0).stop 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.990+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 msgr2=0x7fcd80154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.990+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcd68035040 tx=0x7fcd680046c0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 shutdown_connections 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fcd6006c8f0 0x7fcd6006edb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8015f680 0x7fcd800ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd800d3820 0x7fcd80154790 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 --2- 172.21.15.40:0/3669917624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd800b60b0 0x7fcd800cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 >> 172.21.15.40:0/3669917624 conn(0x7fcd8008eca0 msgr2=0x7fcd800b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 shutdown_connections 2024-04-26T21:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:58.991+0000 7fcd84ac8640 1 -- 172.21.15.40:0/3669917624 wait complete. 2024-04-26T21:20:59.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-04-26T21:20:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.105+0000 7fd5192c8640 1 Processor -- start 2024-04-26T21:20:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.105+0000 7fd5192c8640 1 -- start start 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5192c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5140d4f20 0x7fd5140d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5192c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5140d3c90 con 0x7fd5140d3820 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5192c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5140d58c0 con 0x7fd51415f680 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5192c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd51415fa60 con 0x7fd5140d4f20 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5127fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5140d3820 0x7fd5140b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49738/0 (socket says 172.21.15.40:49738) 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5127fc640 1 -- 172.21.15.40:0/1558689342 learned_addr learned my addr 172.21.15.40:0/1558689342 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd5137fe640 1 --2- 172.21.15.40:0/1558689342 >> v2:172.21.15.184:3300/0 conn(0x7fd5140d4f20 0x7fd5140d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974591585 0 0) 0x7fd5140d3c90 con 0x7fd5140d3820 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.106+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4fc003520 con 0x7fd5140d3820 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884423662 0 0) 0x7fd5140d58c0 con 0x7fd51415f680 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5140d3c90 con 0x7fd51415f680 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3896007656 0 0) 0x7fd4fc003520 con 0x7fd5140d3820 2024-04-26T21:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd5137fe640 1 -- 172.21.15.40:0/1558689342 >> v1:172.21.15.184:6789/0 conn(0x7fd51415f680 legacy=0x7fd514163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd5137fe640 1 -- 172.21.15.40:0/1558689342 >> v1:172.21.15.40:6789/0 conn(0x7fd5140d3820 legacy=0x7fd5140b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd5137fe640 1 -- 172.21.15.40:0/1558689342 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5140d3f50 con 0x7fd5140d4f20 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.107+0000 7fd5137fe640 1 --2- 172.21.15.40:0/1558689342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d4f20 0x7fd5140d5380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd504009d90 tx=0x7fd50402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5701e6581f6cfd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508001670 con 0x7fd5140d4f20 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd504007540 con 0x7fd5140d4f20 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd511ffb640 1 -- 172.21.15.40:0/1558689342 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504007910 con 0x7fd5140d4f20 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 -- 172.21.15.40:0/1558689342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d4f20 msgr2=0x7fd5140d5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1558689342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d4f20 0x7fd5140d5380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd504009d90 tx=0x7fd50402a680 comp rx=0 tx=0).stop 2024-04-26T21:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 -- 172.21.15.40:0/1558689342 shutdown_connections 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1558689342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d4f20 0x7fd5140d5380 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 -- 172.21.15.40:0/1558689342 >> 172.21.15.40:0/1558689342 conn(0x7fd51408eca0 msgr2=0x7fd51409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 -- 172.21.15.40:0/1558689342 shutdown_connections 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.108+0000 7fd5192c8640 1 -- 172.21.15.40:0/1558689342 wait complete. 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.109+0000 7fd5192c8640 1 Processor -- start 2024-04-26T21:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 -- start start 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d3820 0x7fd5140ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 0x7fd5140ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd5140b6280 con 0x7fd5140d4f20 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5140b5370 con 0x7fd5140d3820 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5192c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd5140cfd80 con 0x7fd51415f680 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5127fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5127fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48672/0 (socket says 172.21.15.40:48672) 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.110+0000 7fd5127fc640 1 -- 172.21.15.40:0/1949531012 learned_addr learned my addr 172.21.15.40:0/1949531012 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:59.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5137fe640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 0x7fd5140ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd512ffd640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d3820 0x7fd5140ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 -- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 msgr2=0x7fd5140ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 0x7fd5140ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 -- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d3820 msgr2=0x7fd5140ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d3820 0x7fd5140ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 -- 172.21.15.40:0/1949531012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd514161550 con 0x7fd5140d4f20 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5137fe640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 0x7fd5140ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5127fc640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c5c0 tx=0x7fd50800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508017070 con 0x7fd5140d4f20 2024-04-26T21:20:59.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd508005ce0 con 0x7fd5140d4f20 2024-04-26T21:20:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd514160790 con 0x7fd5140d4f20 2024-04-26T21:20:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080072b0 con 0x7fd5140d4f20 2024-04-26T21:20:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.111+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd514160d50 con 0x7fd5140d4f20 2024-04-26T21:20:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.112+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51406a290 con 0x7fd5140d4f20 2024-04-26T21:20:59.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.113+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd50801a070 con 0x7fd5140d4f20 2024-04-26T21:20:59.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.113+0000 7fd4fb7fe640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 0x7fd50006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.113+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fd50808b020 con 0x7fd5140d4f20 2024-04-26T21:20:59.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.113+0000 7fd512ffd640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 0x7fd50006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.114+0000 7fd512ffd640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 0x7fd50006edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0010e0 tx=0x7fd50c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.118+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5080594e0 con 0x7fd5140d4f20 2024-04-26T21:20:59.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.307+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fd5140b5c00 con 0x7fd5140d4f20 2024-04-26T21:20:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.310+0000 7fd4fb7fe640 1 -- 172.21.15.40:0/1949531012 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v59)=0 v59) ==== 128+0+0 (secure 0 0 0) 0x7fd508058de0 con 0x7fd5140d4f20 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 msgr2=0x7fd50006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 0x7fd50006edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0010e0 tx=0x7fd50c009040 comp rx=0 tx=0).stop 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 msgr2=0x7fd5140ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c5c0 tx=0x7fd50800ca80 comp rx=0 tx=0).stop 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 shutdown_connections 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd50006c8f0 0x7fd50006edb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd51415f680 0x7fd5140ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5140d4f20 0x7fd5140ce840 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 --2- 172.21.15.40:0/1949531012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5140d3820 0x7fd5140ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 >> 172.21.15.40:0/1949531012 conn(0x7fd51408eca0 msgr2=0x7fd51406e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 shutdown_connections 2024-04-26T21:20:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.313+0000 7fd5192c8640 1 -- 172.21.15.40:0/1949531012 wait complete. 2024-04-26T21:20:59.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-26T21:20:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.427+0000 7fa9238c8640 1 Processor -- start 2024-04-26T21:20:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.427+0000 7fa9238c8640 1 -- start start 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa9238c8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa91c0d2070 0x7fa91c0baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa9238c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91c0d4240 con 0x7fa91c0d1be0 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa9238c8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91c0d31a0 con 0x7fa91c15f490 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa9238c8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa91c0d3480 con 0x7fa91c0d2070 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa920e63640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa91c0d1be0 0x7fa91c0ba3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49752/0 (socket says 172.21.15.40:49752) 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.428+0000 7fa920e63640 1 -- 172.21.15.40:0/695934724 learned_addr learned my addr 172.21.15.40:0/695934724 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798753948 0 0) 0x7fa91c0d4240 con 0x7fa91c0d1be0 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa921e65640 1 --2- 172.21.15.40:0/695934724 >> v2:172.21.15.184:3300/0 conn(0x7fa91c0d2070 0x7fa91c0baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa904003520 con 0x7fa91c0d1be0 2024-04-26T21:20:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 405672282 0 0) 0x7fa904003520 con 0x7fa91c0d1be0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 >> v2:172.21.15.184:3300/0 conn(0x7fa91c0d2070 msgr2=0x7fa91c0baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 --2- 172.21.15.40:0/695934724 >> v2:172.21.15.184:3300/0 conn(0x7fa91c0d2070 0x7fa91c0baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 >> v1:172.21.15.184:6789/0 conn(0x7fa91c15f490 legacy=0x7fa91c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c0d3760 con 0x7fa91c0d1be0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa910001670 con 0x7fa91c0d1be0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 >> v1:172.21.15.40:6789/0 conn(0x7fa91c0d1be0 legacy=0x7fa91c0ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9040039b0 0x7fa904003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 0x7fa904004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9040051c0 0x7fa90400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0d3480 con 0x7fa904004800 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa90400c060 con 0x7fa9040039b0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa90400c260 con 0x7fa9040051c0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 0x7fa904004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa921664640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9040039b0 0x7fa904003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa921e65640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9040051c0 0x7fa90400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 -- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9040051c0 msgr2=0x7fa90400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9040051c0 0x7fa90400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 -- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9040039b0 msgr2=0x7fa904003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9040039b0 0x7fa904003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa920e63640 1 -- 172.21.15.40:0/695934724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa904003520 con 0x7fa904004800 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.429+0000 7fa921e65640 1 -- 172.21.15.40:0/695934724 reap_dead start 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.430+0000 7fa920e63640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 0x7fa904004c80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c210 tx=0x7fa904005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a582e162d12d44ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.430+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910007d30 con 0x7fa904004800 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.430+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910005ad0 con 0x7fa904004800 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.430+0000 7fa91bfff640 1 -- 172.21.15.40:0/695934724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910007070 con 0x7fa904004800 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 -- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 msgr2=0x7fa904004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 0x7fa904004c80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c210 tx=0x7fa904005830 comp rx=0 tx=0).stop 2024-04-26T21:20:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 -- 172.21.15.40:0/695934724 shutdown_connections 2024-04-26T21:20:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 --2- 172.21.15.40:0/695934724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa904004800 0x7fa904004c80 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 -- 172.21.15.40:0/695934724 >> 172.21.15.40:0/695934724 conn(0x7fa91c09e530 msgr2=0x7fa91c0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 -- 172.21.15.40:0/695934724 shutdown_connections 2024-04-26T21:20:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.431+0000 7fa9238c8640 1 -- 172.21.15.40:0/695934724 wait complete. 2024-04-26T21:20:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.432+0000 7fa9238c8640 1 Processor -- start 2024-04-26T21:20:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.432+0000 7fa9238c8640 1 -- start start 2024-04-26T21:20:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa904004800 0x7fa91c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 0x7fa91c161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0d2480 con 0x7fa91c152760 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0d3d20 con 0x7fa91c0ced60 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa9238c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c15f870 con 0x7fa904004800 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48692/0 (socket says 172.21.15.40:48692) 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 -- 172.21.15.40:0/3958930951 learned_addr learned my addr 172.21.15.40:0/3958930951 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa921664640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa904004800 0x7fa91c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa921e65640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 0x7fa91c161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 -- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa904004800 msgr2=0x7fa91c15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa904004800 0x7fa91c15c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 -- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 msgr2=0x7fa91c161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 0x7fa91c161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa920e63640 1 -- 172.21.15.40:0/3958930951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c152be0 con 0x7fa91c152760 2024-04-26T21:20:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.433+0000 7fa921e65640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 0x7fa91c161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:20:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa920e63640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa910001970 tx=0x7fa910001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910040030 con 0x7fa91c152760 2024-04-26T21:20:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910007d10 con 0x7fa91c152760 2024-04-26T21:20:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910037040 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c0cdcc0 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.434+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c0ce280 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.435+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c06a290 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.435+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa910043070 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.436+0000 7fa919ffb640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 0x7fa90806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.436+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fa9100ab180 con 0x7fa91c152760 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.436+0000 7fa921664640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 0x7fa90806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.437+0000 7fa921664640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 0x7fa90806ede0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.440+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9100443c0 con 0x7fa91c152760 2024-04-26T21:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.626+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fa91c0d4240 con 0x7fa91c152760 2024-04-26T21:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.626+0000 7fa919ffb640 1 -- 172.21.15.40:0/3958930951 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v59) ==== 106+0+4 (secure 0 0 0) 0x7fa910079640 con 0x7fa91c152760 2024-04-26T21:20:59.628 INFO:teuthology.orchestra.run.smithi040.stdout:100 2024-04-26T21:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 msgr2=0x7fa90806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 0x7fa90806ede0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914009040 comp rx=0 tx=0).stop 2024-04-26T21:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 msgr2=0x7fa91c0b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa910001970 tx=0x7fa910001a40 comp rx=0 tx=0).stop 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 shutdown_connections 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa90806c920 0x7fa90806ede0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0ced60 0x7fa91c161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c152760 0x7fa91c0b7e10 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 --2- 172.21.15.40:0/3958930951 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa904004800 0x7fa91c15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.628+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 >> 172.21.15.40:0/3958930951 conn(0x7fa91c09e530 msgr2=0x7fa91c0d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.629+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 shutdown_connections 2024-04-26T21:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.629+0000 7fa9238c8640 1 -- 172.21.15.40:0/3958930951 wait complete. 2024-04-26T21:20:59.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-26T21:20:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.744+0000 7f875e36a640 1 Processor -- start 2024-04-26T21:20:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.744+0000 7f875e36a640 1 -- start start 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f875e36a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f875816a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f875e36a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875809c700 con 0x7f87580b3c20 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f875e36a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87580d1c50 con 0x7f87580d0010 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f875e36a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f87580b4480 con 0x7f875806db10 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f87577fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f87580b3c20 0x7f87580b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49756/0 (socket says 172.21.15.40:49756) 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f87577fe640 1 -- 172.21.15.40:0/616283527 learned_addr learned my addr 172.21.15.40:0/616283527 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f875c907640 1 --2- 172.21.15.40:0/616283527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f875816a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748258171 0 0) 0x7f875809c700 con 0x7f87580b3c20 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.745+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8744003520 con 0x7f87580b3c20 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.746+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3839780980 0 0) 0x7f8744003520 con 0x7f87580b3c20 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.746+0000 7f875c907640 1 -- 172.21.15.40:0/616283527 >> v1:172.21.15.184:6789/0 conn(0x7f87580d0010 legacy=0x7f87580b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.746+0000 7f875c907640 1 -- 172.21.15.40:0/616283527 >> v1:172.21.15.40:6789/0 conn(0x7f87580b3c20 legacy=0x7f87580b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.746+0000 7f875c907640 1 -- 172.21.15.40:0/616283527 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87580d1f30 con 0x7f875806db10 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.746+0000 7f875c907640 1 --2- 172.21.15.40:0/616283527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f875816a5b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8750009dc0 tx=0x7f875002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2a97308dae8b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748001670 con 0x7f875806db10 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750007540 con 0x7f875806db10 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f8756ffd640 1 -- 172.21.15.40:0/616283527 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500078f0 con 0x7f875806db10 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 -- 172.21.15.40:0/616283527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 msgr2=0x7f875816a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 --2- 172.21.15.40:0/616283527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f875816a5b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8750009dc0 tx=0x7f875002a660 comp rx=0 tx=0).stop 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 -- 172.21.15.40:0/616283527 shutdown_connections 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 --2- 172.21.15.40:0/616283527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f875816a5b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 -- 172.21.15.40:0/616283527 >> 172.21.15.40:0/616283527 conn(0x7f875809ba40 msgr2=0x7f875809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 -- 172.21.15.40:0/616283527 shutdown_connections 2024-04-26T21:20:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.747+0000 7f875e36a640 1 -- 172.21.15.40:0/616283527 wait complete. 2024-04-26T21:20:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.748+0000 7f875e36a640 1 Processor -- start 2024-04-26T21:20:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 -- start start 2024-04-26T21:20:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f8758155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87580b3c20 0x7f8758162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f875816ac70 con 0x7f87580d0010 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f875816aaf0 con 0x7f87580b3c20 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875e36a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f875816adf0 con 0x7f875806db10 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.749+0000 7f875c907640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48706/0 (socket says 172.21.15.40:48706) 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 -- 172.21.15.40:0/1011713509 learned_addr learned my addr 172.21.15.40:0/1011713509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f8757fff640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f8758155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f87577fe640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87580b3c20 0x7f8758162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 -- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 msgr2=0x7f8758155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f8758155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 -- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87580b3c20 msgr2=0x7f8758162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87580b3c20 0x7f8758162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 -- 172.21.15.40:0/1011713509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8758162d60 con 0x7f87580d0010 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875c907640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8750009ee0 tx=0x7f8750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750040070 con 0x7f87580d0010 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750031c40 con 0x7f87580d0010 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8758163040 con 0x7f87580d0010 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750009450 con 0x7f87580d0010 2024-04-26T21:20:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.750+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8758163600 con 0x7f87580d0010 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.751+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875806a290 con 0x7f87580d0010 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.752+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8750039040 con 0x7f87580d0010 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.752+0000 7f8754ff9640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 0x7f873806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.752+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f87500acea0 con 0x7f87580d0010 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.752+0000 7f8757fff640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 0x7f873806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.753+0000 7f8757fff640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 0x7f873806edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8740000f60 tx=0x7f8740009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.756+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f875007ac00 con 0x7f87580d0010 2024-04-26T21:20:59.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.945+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f8758068e00 con 0x7f87580d0010 2024-04-26T21:20:59.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.948+0000 7f8754ff9640 1 -- 172.21.15.40:0/1011713509 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v60)=0 v60) ==== 124+0+0 (secure 0 0 0) 0x7f875007ef00 con 0x7f87580d0010 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 msgr2=0x7f873806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 0x7f873806edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8740000f60 tx=0x7f8740009040 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 msgr2=0x7f87580b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8750009ee0 tx=0x7f8750004a40 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 shutdown_connections 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f873806c8f0 0x7f873806edb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87580d0010 0x7f87580b1100 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87580b3c20 0x7f8758162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 --2- 172.21.15.40:0/1011713509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f875806db10 0x7f8758155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.950+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 >> 172.21.15.40:0/1011713509 conn(0x7f875809ba40 msgr2=0x7f87580b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:59.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.951+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 shutdown_connections 2024-04-26T21:20:59.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:20:59.951+0000 7f875e36a640 1 -- 172.21.15.40:0/1011713509 wait complete. 2024-04-26T21:20:59.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-26T21:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.065+0000 7fbde8c6d640 1 Processor -- start 2024-04-26T21:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.065+0000 7fbde8c6d640 1 -- start start 2024-04-26T21:21:00.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde8c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde40d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde8c6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbde4163aa0 0x7fbde40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde8c6d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde40d3f50 con 0x7fbde40b60b0 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde8c6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbde415fa60 con 0x7fbde4163aa0 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde8c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40b5090 con 0x7fbde415f680 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde1d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbde4163aa0 0x7fbde40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde1d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbde4163aa0 0x7fbde40b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48716/0 (socket says 172.21.15.40:48716) 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbde40b60b0 0x7fbde40d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49762/0 (socket says 172.21.15.40:49762) 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.066+0000 7fbde1d9b640 1 -- 172.21.15.40:0/1038646654 learned_addr learned my addr 172.21.15.40:0/1038646654 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde259c640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde40d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde1d9b640 1 -- 172.21.15.40:0/1038646654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 msgr2=0x7fbde40d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde1d9b640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde40d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde1d9b640 1 -- 172.21.15.40:0/1038646654 >> v1:172.21.15.40:6789/0 conn(0x7fbde40b60b0 legacy=0x7fbde40d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde1d9b640 1 -- 172.21.15.40:0/1038646654 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40b5370 con 0x7fbde4163aa0 2024-04-26T21:21:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde159a640 1 -- 172.21.15.40:0/1038646654 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298325905 0 0) 0x7fbde40d3f50 con 0x7fbde40b60b0 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.067+0000 7fbde1d9b640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde4163aa0 0x7fbde40b5b70 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009870 tx=0x7fbdcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bf8e23391ebdf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde159a640 1 -- 172.21.15.40:0/1038646654 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc037070 con 0x7fbde4163aa0 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde159a640 1 -- 172.21.15.40:0/1038646654 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc032440 con 0x7fbde4163aa0 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde159a640 1 -- 172.21.15.40:0/1038646654 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc032810 con 0x7fbde4163aa0 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 -- 172.21.15.40:0/1038646654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde4163aa0 msgr2=0x7fbde40b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde4163aa0 0x7fbde40b5b70 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009870 tx=0x7fbdcc02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 -- 172.21.15.40:0/1038646654 shutdown_connections 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde4163aa0 0x7fbde40b5b70 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/1038646654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde40d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 -- 172.21.15.40:0/1038646654 >> 172.21.15.40:0/1038646654 conn(0x7fbde408eca0 msgr2=0x7fbde409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 -- 172.21.15.40:0/1038646654 shutdown_connections 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.068+0000 7fbde8c6d640 1 -- 172.21.15.40:0/1038646654 wait complete. 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.069+0000 7fbde8c6d640 1 Processor -- start 2024-04-26T21:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 -- start start 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde4163aa0 0x7fbde415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40d57b0 con 0x7fbde40b60b0 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40d5630 con 0x7fbde4163aa0 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde8c6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40d5930 con 0x7fbde415f680 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48724/0 (socket says 172.21.15.40:48724) 2024-04-26T21:21:00.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.070+0000 7fbde259c640 1 -- 172.21.15.40:0/3907548380 learned_addr learned my addr 172.21.15.40:0/3907548380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde1d9b640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde2d9d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde4163aa0 0x7fbde415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 -- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 msgr2=0x7fbde415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 -- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde4163aa0 msgr2=0x7fbde415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde4163aa0 0x7fbde415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 -- 172.21.15.40:0/3907548380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40d1da0 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde259c640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00d800 tx=0x7fbddc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00b6c0 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc019070 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde40d1ac0 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00ba80 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.071+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde40cdb70 con 0x7fbde40b60b0 2024-04-26T21:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.072+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde406a290 con 0x7fbde40b60b0 2024-04-26T21:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.073+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbddc00bc20 con 0x7fbde40b60b0 2024-04-26T21:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.073+0000 7fbddaffd640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.073+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fbddc08bf80 con 0x7fbde40b60b0 2024-04-26T21:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.073+0000 7fbde1d9b640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.074+0000 7fbde1d9b640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 0x7fbdc406edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034000 tx=0x7fbdcc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.078+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbddc05a330 con 0x7fbde40b60b0 2024-04-26T21:21:00.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.263+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7fbde40b4cd0 con 0x7fbde40b60b0 2024-04-26T21:21:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.264+0000 7fbddaffd640 1 -- 172.21.15.40:0/3907548380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v60) ==== 106+0+8 (secure 0 0 0) 0x7fbddc059c30 con 0x7fbde40b60b0 2024-04-26T21:21:00.265 INFO:teuthology.orchestra.run.smithi040.stdout:1048576 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 msgr2=0x7fbdc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 0x7fbdc406edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034000 tx=0x7fbdcc02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 msgr2=0x7fbde40c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00d800 tx=0x7fbddc00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 shutdown_connections 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbde4163aa0 0x7fbde415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbde415f680 0x7fbde415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.265+0000 7fbde8c6d640 1 --2- 172.21.15.40:0/3907548380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde40b60b0 0x7fbde40c7ee0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.266+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 >> 172.21.15.40:0/3907548380 conn(0x7fbde408eca0 msgr2=0x7fbde4163280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.266+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 shutdown_connections 2024-04-26T21:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.266+0000 7fbde8c6d640 1 -- 172.21.15.40:0/3907548380 wait complete. 2024-04-26T21:21:00.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-26T21:21:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.378+0000 7fd182a72640 1 Processor -- start 2024-04-26T21:21:00.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd182a72640 1 -- start start 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd182a72640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd182a72640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17c0d3f50 con 0x7fd17c0d4f20 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd182a72640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17c0b5090 con 0x7fd17c163aa0 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd182a72640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c15fa60 con 0x7fd17c15f680 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.379+0000 7fd18100f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd17c0d4f20 0x7fd17c0d53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49778/0 (socket says 172.21.15.40:49778) 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd18100f640 1 -- 172.21.15.40:0/2075519689 learned_addr learned my addr 172.21.15.40:0/2075519689 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd18080e640 1 --2- 172.21.15.40:0/2075519689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487519728 0 0) 0x7fd17c0d3f50 con 0x7fd17c0d4f20 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd16c003520 con 0x7fd17c0d4f20 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975357934 0 0) 0x7fd17c0b5090 con 0x7fd17c163aa0 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd18080e640 1 -- 172.21.15.40:0/2075519689 >> v1:172.21.15.184:6789/0 conn(0x7fd17c163aa0 legacy=0x7fd17c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd18080e640 1 -- 172.21.15.40:0/2075519689 >> v1:172.21.15.40:6789/0 conn(0x7fd17c0d4f20 legacy=0x7fd17c0d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.380+0000 7fd18080e640 1 -- 172.21.15.40:0/2075519689 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c0b5370 con 0x7fd17c15f680 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd18080e640 1 --2- 172.21.15.40:0/2075519689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c0d3820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097d0 tx=0x7fd17002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc61768e53ee416 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170037070 con 0x7fd17c15f680 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17002ac10 con 0x7fd17c15f680 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd17b7fe640 1 -- 172.21.15.40:0/2075519689 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740019f0 con 0x7fd17c15f680 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd182a72640 1 -- 172.21.15.40:0/2075519689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 msgr2=0x7fd17c0d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.381+0000 7fd182a72640 1 --2- 172.21.15.40:0/2075519689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c0d3820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097d0 tx=0x7fd17002a180 comp rx=0 tx=0).stop 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.382+0000 7fd182a72640 1 -- 172.21.15.40:0/2075519689 shutdown_connections 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.382+0000 7fd182a72640 1 --2- 172.21.15.40:0/2075519689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c0d3820 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.382+0000 7fd182a72640 1 -- 172.21.15.40:0/2075519689 >> 172.21.15.40:0/2075519689 conn(0x7fd17c08eca0 msgr2=0x7fd17c09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.382+0000 7fd182a72640 1 -- 172.21.15.40:0/2075519689 shutdown_connections 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.382+0000 7fd182a72640 1 -- 172.21.15.40:0/2075519689 wait complete. 2024-04-26T21:21:00.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.383+0000 7fd182a72640 1 Processor -- start 2024-04-26T21:21:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.383+0000 7fd182a72640 1 -- start start 2024-04-26T21:21:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd17c163aa0 0x7fd17c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c0cfd80 con 0x7fd17c0d4f20 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c163f00 con 0x7fd17c163aa0 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd182a72640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c0cff00 con 0x7fd17c15f680 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48738/0 (socket says 172.21.15.40:48738) 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 -- 172.21.15.40:0/2136532195 learned_addr learned my addr 172.21.15.40:0/2136532195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18100f640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd17c163aa0 0x7fd17c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd17bfff640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 -- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 msgr2=0x7fd17c152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 -- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd17c163aa0 msgr2=0x7fd17c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd17c163aa0 0x7fd17c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 -- 172.21.15.40:0/2136532195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c0b90d0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd18080e640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097d0 tx=0x7fd1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170037040 con 0x7fd17c0d4f20 2024-04-26T21:21:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.384+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd170004030 con 0x7fd17c0d4f20 2024-04-26T21:21:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.385+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170004420 con 0x7fd17c0d4f20 2024-04-26T21:21:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.385+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c0cd3c0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.385+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c0cd980 con 0x7fd17c0d4f20 2024-04-26T21:21:00.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.386+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c0cdbc0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.386+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd17003a040 con 0x7fd17c0d4f20 2024-04-26T21:21:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.386+0000 7fd1797fa640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.386+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fd1700ac2f0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.387+0000 7fd17bfff640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.387+0000 7fd17bfff640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 0x7fd16006edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d3710 tx=0x7fd168009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.391+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd170042020 con 0x7fd17c0d4f20 2024-04-26T21:21:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.577+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7fd17c0b4cd0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.577+0000 7fd1797fa640 1 -- 172.21.15.40:0/2136532195 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v60) ==== 112+0+10 (secure 0 0 0) 0x7fd17007a1f0 con 0x7fd17c0d4f20 2024-04-26T21:21:00.578 INFO:teuthology.orchestra.run.smithi040.stdout:60.000000 2024-04-26T21:21:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 msgr2=0x7fd16006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 0x7fd16006edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0d3710 tx=0x7fd168009040 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 msgr2=0x7fd17c0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd1700097d0 tx=0x7fd1700046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 shutdown_connections 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd17c163aa0 0x7fd17c1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd17c15f680 0x7fd17c152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 --2- 172.21.15.40:0/2136532195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd17c0d4f20 0x7fd17c0d0a60 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 >> 172.21.15.40:0/2136532195 conn(0x7fd17c08eca0 msgr2=0x7fd17c0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 shutdown_connections 2024-04-26T21:21:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.580+0000 7fd182a72640 1 -- 172.21.15.40:0/2136532195 wait complete. 2024-04-26T21:21:00.592 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:00.592 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:00.607 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:21:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:00.706 INFO:teuthology.orchestra.run.smithi040.stdout:client6836 2024-04-26T21:21:00.706 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:00.706 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> import fcntl 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> import sys 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> import time 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-26T21:21:00.722 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> n = 400 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> if not os.path.exists(abs_path): 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(abs_path) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> handles = [] 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(0, n): 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> fname = "file_"+str(i) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> path = os.path.join(abs_path, fname) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> handles.append(open(path, '"'"'w'"'"')) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> print("waiting with handles open", file=sys.stderr) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> while True: 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-04-26T21:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> if os.read(0, 4096) == b"": 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> break 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> except BlockingIOError: 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> pass 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(2) 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-26T21:21:00.724 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-26T21:21:00.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:00.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.864+0000 7f1cbfc62640 1 Processor -- start 2024-04-26T21:21:00.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.864+0000 7f1cbfc62640 1 -- start start 2024-04-26T21:21:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbfc62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1cb8160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbfc62640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb80b4530 con 0x7f1cb8145980 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbfc62640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb80b4810 con 0x7f1cb80b3de0 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbfc62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb80b5140 con 0x7f1cb80b61a0 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbd9fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1cb8145980 0x7f1cb80b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44784/0 (socket says 172.21.15.40:44784) 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.865+0000 7f1cbd9fe640 1 -- 172.21.15.40:0/1287966903 learned_addr learned my addr 172.21.15.40:0/1287966903 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1cb8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927779444 0 0) 0x7f1cb80b4530 con 0x7f1cb8145980 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca4003520 con 0x7f1cb8145980 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546198735 0 0) 0x7f1cb80b4810 con 0x7f1cb80b3de0 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cb80b4530 con 0x7f1cb80b3de0 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1451330007 0 0) 0x7f1ca4003520 con 0x7f1cb8145980 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 msgr2=0x7f1cb8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1cb8160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 >> v1:172.21.15.184:6789/0 conn(0x7f1cb80b3de0 legacy=0x7f1cb80720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb80b4af0 con 0x7f1cb8145980 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f1cb4001670 con 0x7f1cb8145980 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1cb8160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 >> v1:172.21.15.40:6789/0 conn(0x7f1cb8145980 legacy=0x7f1cb80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1ca4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca4004100 0x7f1ca4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004d40 0x7f1ca40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb80b5140 con 0x7f1ca4004100 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca400dd90 con 0x7f1ca4004d40 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca400df90 con 0x7f1cb80b61a0 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca4004100 0x7f1ca4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.866+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1ca4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbe1ff640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004d40 0x7f1ca40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 msgr2=0x7f1ca4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1ca4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004d40 msgr2=0x7f1ca40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004d40 0x7f1ca40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/1287966903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca4003520 con 0x7f1ca4004100 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.867+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb80b61a0 0x7f1ca4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca4004100 0x7f1ca4004800 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009c70 tx=0x7f1ca4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5013188c6b1656d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8036070 con 0x7f1ca4004100 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca8031400 con 0x7f1ca4004100 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbc9fc640 1 -- 172.21.15.40:0/1287966903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8009840 con 0x7f1ca4004100 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbfc62640 1 -- 172.21.15.40:0/1287966903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca4004100 msgr2=0x7f1ca4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/1287966903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ca4004100 0x7f1ca4004800 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009c70 tx=0x7f1ca4005830 comp rx=0 tx=0).stop 2024-04-26T21:21:00.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbe1ff640 1 -- 172.21.15.40:0/1287966903 reap_dead start 2024-04-26T21:21:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbfc62640 1 -- 172.21.15.40:0/1287966903 shutdown_connections 2024-04-26T21:21:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.868+0000 7f1cbfc62640 1 -- 172.21.15.40:0/1287966903 >> 172.21.15.40:0/1287966903 conn(0x7f1cb80d0d30 msgr2=0x7f1cb80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.869+0000 7f1cbfc62640 1 -- 172.21.15.40:0/1287966903 shutdown_connections 2024-04-26T21:21:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.869+0000 7f1cbfc62640 1 -- 172.21.15.40:0/1287966903 wait complete. 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.870+0000 7f1cbfc62640 1 Processor -- start 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.870+0000 7f1cbfc62640 1 -- start start 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.870+0000 7f1cbfc62640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 0x7f1cb815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb815f610 0x7f1cb80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8160630 con 0x7f1cb813a2b0 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb80b4240 con 0x7f1ca4004100 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb81607b0 con 0x7f1cb815f610 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46284/0 (socket says 172.21.15.40:46284) 2024-04-26T21:21:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/2971632642 learned_addr learned my addr 172.21.15.40:0/2971632642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 0x7f1cb815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbe1ff640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb815f610 0x7f1cb80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb815f610 msgr2=0x7f1cb80b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb815f610 0x7f1cb80b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 msgr2=0x7f1cb815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 0x7f1cb815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 -- 172.21.15.40:0/2971632642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb80aea50 con 0x7f1cb813a2b0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 0x7f1cb815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbd1fd640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009d90 tx=0x7f1ca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8040020 con 0x7f1cb813a2b0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca80093e0 con 0x7f1cb813a2b0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb80aed30 con 0x7f1cb813a2b0 2024-04-26T21:21:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8036040 con 0x7f1cb813a2b0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.871+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb815bab0 con 0x7f1cb813a2b0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.873+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb815fa20 con 0x7f1cb813a2b0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.873+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ca8043020 con 0x7f1cb813a2b0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.873+0000 7f1cae7fc640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 0x7f1c9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.873+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f1ca80abd00 con 0x7f1cb813a2b0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.874+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 0x7f1c9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.874+0000 7f1cbd9fe640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 0x7f1c9c06edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4001790 tx=0x7f1cb4000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:00.878+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cb815fa20 con 0x7f1cb813a2b0 2024-04-26T21:21:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.141+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cb806a290 con 0x7f1cb813a2b0 2024-04-26T21:21:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.141+0000 7f1cae7fc640 1 -- 172.21.15.40:0/2971632642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5191 (secure 0 0 0) 0x7f1ca804a070 con 0x7f1cb813a2b0 2024-04-26T21:21:01.142 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:01.142 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:20:48.339156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:01.143 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 88 2024-04-26T21:21:01.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 msgr2=0x7f1c9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 0x7f1c9c06edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4001790 tx=0x7f1cb4000ed0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 msgr2=0x7f1cb815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009d90 tx=0x7f1ca8004730 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 shutdown_connections 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c9c06c8f0 0x7f1c9c06edb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cb815f610 0x7f1cb80b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb813a2b0 0x7f1cb815f0d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 --2- 172.21.15.40:0/2971632642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ca4004100 0x7f1cb815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 >> 172.21.15.40:0/2971632642 conn(0x7f1cb80d0d30 msgr2=0x7f1cb80ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 shutdown_connections 2024-04-26T21:21:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.143+0000 7f1cbfc62640 1 -- 172.21.15.40:0/2971632642 wait complete. 2024-04-26T21:21:01.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-26T21:21:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:waiting with handles open 2024-04-26T21:21:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:21:01.275 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":9180,"entity":{"name":{"type":"client","num":9180},"addr":{"type":"v1","addr":"172.21.15.184:0","nonce":3143194780}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.0969618859999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9908563014713949,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9180 v1:172.21.15.184:0/3143194780","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":6836,"entity":{"name":{"type":"client","num":6836},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":3099832244}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":424,"uptime":4.2389478299999999,"requests_in_flight":316,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":57.800005429230794,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":50.515356192431376,"halflife":1.5},"recall_caps_throttle2o":{"value":38.319009250440367,"halflife":0.5},"session_cache_liveness":{"value":1201.2517707798741,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":112,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6836 v1:172.21.15.40:0/3099832244","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000248","length":919}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:21:01.276 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 3143194780, "type": "v1" }, "name": { "num": 9180, "type": "client" } }, "id": 9180, "inst": "client.9180 v1:172.21.15.184:0/3143194780", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.990856301471395 }, "state": "open", "uptime": 3.096961886 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 3099832244, "type": "v1" }, "name": { "num": 6836, "type": "client" } }, "id": 6836, "inst": "client.6836 v1:172.21.15.40:0/3099832244", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 112, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 919, "start": "0x10000000248" } ], "recall_caps": { "halflife": 60, "value": 57.800005429230794 }, "recall_caps_throttle": { "halflife": 1.5, "value": 50.515356192431376 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 38.31900925044037 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 424, "requests_in_flight": 316, "session_cache_liveness": { "halflife": 300, "value": 1201.251770779874 }, "state": "open", "uptime": 4.23894783 } ] 2024-04-26T21:21:01.276 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-26T21:21:01.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:21:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.414+0000 7f22c0ac8640 1 Processor -- start 2024-04-26T21:21:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.414+0000 7f22c0ac8640 1 -- start start 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22c0ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22bc0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22c0ac8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc15fb50 con 0x7f22bc0b5890 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22c0ac8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc0b5da0 con 0x7f22bc0b43c0 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22c0ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc0b6080 con 0x7f22bc1598a0 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22b9d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f22bc0b5890 0x7f22bc0b3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44796/0 (socket says 172.21.15.40:44796) 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.415+0000 7f22b9d9b640 1 -- 172.21.15.40:0/2249235534 learned_addr learned my addr 172.21.15.40:0/2249235534 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22ba59c640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22bc0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139681838 0 0) 0x7f22bc15fb50 con 0x7f22bc0b5890 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22a8003520 con 0x7f22bc0b5890 2024-04-26T21:21:01.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309003699 0 0) 0x7f22bc0b5da0 con 0x7f22bc0b43c0 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22bc15fb50 con 0x7f22bc0b43c0 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2209850571 0 0) 0x7f22a8003520 con 0x7f22bc0b5890 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 msgr2=0x7f22bc0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22bc0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 >> v1:172.21.15.184:6789/0 conn(0x7f22bc0b43c0 legacy=0x7f22bc162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc15fe10 con 0x7f22bc0b5890 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f22a4001670 con 0x7f22bc0b5890 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 >> v1:172.21.15.40:6789/0 conn(0x7f22bc0b5890 legacy=0x7f22bc0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22a8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a80040d0 0x7f22a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a8004d40 0x7f22a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc0b6080 con 0x7f22a80040d0 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22a800bb50 con 0x7f22a8004d40 2024-04-26T21:21:01.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22a800bd20 con 0x7f22bc1598a0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a80040d0 0x7f22a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22ba59c640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22a8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22bad9d640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a8004d40 0x7f22a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 -- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 msgr2=0x7f22a8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22bc1598a0 0x7f22a8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 -- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a8004d40 msgr2=0x7f22a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a8004d40 0x7f22a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.416+0000 7f22b9d9b640 1 -- 172.21.15.40:0/2249235534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a8003520 con 0x7f22a80040d0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.417+0000 7f22bad9d640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22a8004d40 0x7f22a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.417+0000 7f22b9d9b640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a80040d0 0x7f22a8004800 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b5cd0 tx=0x7f22a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d23c541ba7c395 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.417+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4007ce0 con 0x7f22a80040d0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.417+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a4005ad0 con 0x7f22a80040d0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.417+0000 7f22b959a640 1 -- 172.21.15.40:0/2249235534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4007070 con 0x7f22a80040d0 2024-04-26T21:21:01.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2249235534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a80040d0 msgr2=0x7f22a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2249235534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a80040d0 0x7f22a8004800 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b5cd0 tx=0x7f22a8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22bad9d640 1 -- 172.21.15.40:0/2249235534 reap_dead start 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2249235534 shutdown_connections 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2249235534 >> 172.21.15.40:0/2249235534 conn(0x7f22bc06d3b0 msgr2=0x7f22bc06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2249235534 shutdown_connections 2024-04-26T21:21:01.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.418+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2249235534 wait complete. 2024-04-26T21:21:01.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.419+0000 7f22c0ac8640 1 Processor -- start 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.419+0000 7f22c0ac8640 1 -- start start 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a80040d0 0x7f22bc14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22bc15b9f0 0x7f22bc1633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc159c80 con 0x7f22bc15be70 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc06f750 con 0x7f22bc15b9f0 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22c0ac8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc14b5b0 con 0x7f22a80040d0 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46296/0 (socket says 172.21.15.40:46296) 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 -- 172.21.15.40:0/2300707383 learned_addr learned my addr 172.21.15.40:0/2300707383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22ba59c640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a80040d0 0x7f22bc14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22b9d9b640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22bc15b9f0 0x7f22bc1633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 -- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a80040d0 msgr2=0x7f22bc14a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a80040d0 0x7f22bc14a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 -- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22bc15b9f0 msgr2=0x7f22bc1633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22bc15b9f0 0x7f22bc1633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.420+0000 7f22bad9d640 1 -- 172.21.15.40:0/2300707383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc15c440 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22bad9d640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f22ac001330 tx=0x7f22ac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac019070 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac00ee50 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc15c720 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003dc0 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.421+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc14e0e0 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.422+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc0693d0 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.422+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22ac01c070 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.423+0000 7f22b2ffd640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 0x7f229c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.423+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f22ac08af30 con 0x7f22bc15be70 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.423+0000 7f22ba59c640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 0x7f229c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.424+0000 7f22ba59c640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 0x7f229c06edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009a20 tx=0x7f22b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:01.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.427+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22ac0592c0 con 0x7f22bc15be70 2024-04-26T21:21:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.652+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f22bc06f380 con 0x7f22bc15be70 2024-04-26T21:21:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.653+0000 7f22b2ffd640 1 -- 172.21.15.40:0/2300707383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f22ac058bc0 con 0x7f22bc15be70 2024-04-26T21:21:01.654 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:01.654 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 msgr2=0x7f229c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 0x7f229c06edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009a20 tx=0x7f22b4008040 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 msgr2=0x7f22bc163910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f22ac001330 tx=0x7f22ac00ec00 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 shutdown_connections 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f229c06c8f0 0x7f229c06edb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22bc15be70 0x7f22bc163910 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22bc15b9f0 0x7f22bc1633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 --2- 172.21.15.40:0/2300707383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22a80040d0 0x7f22bc14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 >> 172.21.15.40:0/2300707383 conn(0x7f22bc06d3b0 msgr2=0x7f22bc06fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 shutdown_connections 2024-04-26T21:21:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:01.655+0000 7f22c0ac8640 1 -- 172.21.15.40:0/2300707383 wait complete. 2024-04-26T21:21:01.666 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-26T21:21:01.666 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-26T21:21:02.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:21:02.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:21:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:21:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:21:06.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.779+0000 7f0b6fc23640 1 Processor -- start 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.779+0000 7f0b6fc23640 1 -- start start 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6fc23640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b68146070 0x7f0b680b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6fc23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 0x7f0b680b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6fc23640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b68160e50 con 0x7f0b68145c90 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6fc23640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0b68074630 con 0x7f0b68146070 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6fc23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b68161740 con 0x7f0b680b4ed0 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6d1be640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b68146070 0x7f0b680b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6d1be640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b68146070 0x7f0b680b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46302/0 (socket says 172.21.15.40:46302) 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6d1be640 1 -- 172.21.15.40:0/3298155449 learned_addr learned my addr 172.21.15.40:0/3298155449 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6e1c0640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 0x7f0b680b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.780+0000 7f0b6d1be640 1 -- 172.21.15.40:0/3298155449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 msgr2=0x7f0b680b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6d1be640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 0x7f0b680b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6d1be640 1 -- 172.21.15.40:0/3298155449 >> v1:172.21.15.184:6789/0 conn(0x7f0b68145c90 legacy=0x7f0b68073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6d1be640 1 -- 172.21.15.40:0/3298155449 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b68161bd0 con 0x7f0b68146070 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6e1c0640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 0x7f0b680b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6d1be640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b68146070 0x7f0b680b3b60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0b68161020 tx=0x7f0b5c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fdbdbbf372d436 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6c9bd640 1 -- 172.21.15.40:0/3298155449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c037070 con 0x7f0b68146070 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6c9bd640 1 -- 172.21.15.40:0/3298155449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c032440 con 0x7f0b68146070 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.781+0000 7f0b6c9bd640 1 -- 172.21.15.40:0/3298155449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c032810 con 0x7f0b68146070 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 -- 172.21.15.40:0/3298155449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b68146070 msgr2=0x7f0b680b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b68146070 0x7f0b680b3b60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0b68161020 tx=0x7f0b5c02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 -- 172.21.15.40:0/3298155449 shutdown_connections 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b680b4ed0 0x7f0b680b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/3298155449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b68146070 0x7f0b680b3b60 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 -- 172.21.15.40:0/3298155449 >> 172.21.15.40:0/3298155449 conn(0x7f0b680ce920 msgr2=0x7f0b680ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 -- 172.21.15.40:0/3298155449 shutdown_connections 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.782+0000 7f0b6fc23640 1 -- 172.21.15.40:0/3298155449 wait complete. 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.783+0000 7f0b6fc23640 1 Processor -- start 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.783+0000 7f0b6fc23640 1 -- start start 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b68145c90 0x7f0b681f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b68146070 0x7f0b681f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680b6ac0 con 0x7f0b680b4ed0 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681613d0 con 0x7f0b68146070 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6fc23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0b680b6c40 con 0x7f0b68145c90 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6d9bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6d9bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46314/0 (socket says 172.21.15.40:46314) 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6d9bf640 1 -- 172.21.15.40:0/910819787 learned_addr learned my addr 172.21.15.40:0/910819787 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6d1be640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b68145c90 0x7f0b681f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.784+0000 7f0b6e1c0640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b68146070 0x7f0b681f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 -- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b68145c90 msgr2=0x7f0b681f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b68145c90 0x7f0b681f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 -- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b68146070 msgr2=0x7f0b681f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b68146070 0x7f0b681f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 -- 172.21.15.40:0/910819787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b68144bf0 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6d9bf640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c430 tx=0x7f0b6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600052d0 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60005490 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60007260 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6815f550 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.785+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6815fb10 con 0x7f0b680b4ed0 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.786+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6806a290 con 0x7f0b680b4ed0 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.787+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b60007400 con 0x7f0b680b4ed0 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.787+0000 7f0b667fc640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.787+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6008af70 con 0x7f0b680b4ed0 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.787+0000 7f0b6d1be640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.788+0000 7f0b6d1be640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c034000 tx=0x7f0b5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:06.791+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b60058de0 con 0x7f0b680b4ed0 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.016+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0b680736f0 con 0x7f0b680b4ed0 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.017+0000 7f0b667fc640 1 -- 172.21.15.40:0/910819787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f0b6005d0e0 con 0x7f0b680b4ed0 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 msgr2=0x7f0b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c034000 tx=0x7f0b5c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 msgr2=0x7f0b68137640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c430 tx=0x7f0b6000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 shutdown_connections 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b68146070 0x7f0b681f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b68145c90 0x7f0b681f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 --2- 172.21.15.40:0/910819787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b680b4ed0 0x7f0b68137640 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 >> 172.21.15.40:0/910819787 conn(0x7f0b680ce920 msgr2=0x7f0b680b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 shutdown_connections 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.019+0000 7f0b6fc23640 1 -- 172.21.15.40:0/910819787 wait complete. 2024-04-26T21:21:07.031 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-26T21:21:07.031 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-26T21:21:07.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 Processor -- start 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 -- start start 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 0x7f1d0006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d00072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d00073250 0x7f1d0015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0006fde0 con 0x7f1d00073250 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1d00073a40 con 0x7f1d00145870 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.130+0000 7f1d060f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0015d8d0 con 0x7f1d0006f980 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d00073250 0x7f1d0015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d00073250 0x7f1d0015fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46318/0 (socket says 172.21.15.40:46318) 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 -- 172.21.15.40:0/3339067290 learned_addr learned my addr 172.21.15.40:0/3339067290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/3339067290 >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 0x7f1d0006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfeffd640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d00072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 -- 172.21.15.40:0/3339067290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 msgr2=0x7f1d00072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d00072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 -- 172.21.15.40:0/3339067290 >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 msgr2=0x7f1d0006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 --2- 172.21.15.40:0/3339067290 >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 0x7f1d0006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfffff640 1 -- 172.21.15.40:0/3339067290 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0015dbb0 con 0x7f1d00073250 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/3339067290 >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 0x7f1d0006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.131+0000 7f1cfeffd640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d00072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1cfffff640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00073250 0x7f1d0015fab0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f1d000cf1a0 tx=0x7f1cf802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5554517dd014668c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1cfe7fc640 1 -- 172.21.15.40:0/3339067290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8036070 con 0x7f1d00073250 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1cfe7fc640 1 -- 172.21.15.40:0/3339067290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8031600 con 0x7f1d00073250 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1cfe7fc640 1 -- 172.21.15.40:0/3339067290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80319d0 con 0x7f1d00073250 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 -- 172.21.15.40:0/3339067290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00073250 msgr2=0x7f1d0015fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00073250 0x7f1d0015fab0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f1d000cf1a0 tx=0x7f1cf802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 -- 172.21.15.40:0/3339067290 shutdown_connections 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00073250 0x7f1d0015fab0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 --2- 172.21.15.40:0/3339067290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d00072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 --2- 172.21.15.40:0/3339067290 >> v2:172.21.15.184:3300/0 conn(0x7f1d00145870 0x7f1d0006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 -- 172.21.15.40:0/3339067290 >> 172.21.15.40:0/3339067290 conn(0x7f1d000cd830 msgr2=0x7f1d000cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 -- 172.21.15.40:0/3339067290 shutdown_connections 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.132+0000 7f1d060f1640 1 -- 172.21.15.40:0/3339067290 wait complete. 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 Processor -- start 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 -- start start 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d0013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d00073250 0x7f1d0013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00145c50 con 0x7f1d00145870 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0015d5d0 con 0x7f1d00073250 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.134+0000 7f1d060f1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d000736e0 con 0x7f1d0006f980 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46328/0 (socket says 172.21.15.40:46328) 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 -- 172.21.15.40:0/482175545 learned_addr learned my addr 172.21.15.40:0/482175545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d0013c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfeffd640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d00073250 0x7f1d0013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 -- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 msgr2=0x7f1d0013c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d0013c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 -- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d00073250 msgr2=0x7f1d0013ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d00073250 0x7f1d0013ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 -- 172.21.15.40:0/482175545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d000b5c10 con 0x7f1d00145870 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1cfffff640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009f40 tx=0x7f1cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8036050 con 0x7f1d00145870 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8042070 con 0x7f1d00145870 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8009510 con 0x7f1d00145870 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d000b5ef0 con 0x7f1d00145870 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.135+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d00142660 con 0x7f1d00145870 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.136+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d000cea50 con 0x7f1d00145870 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.137+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cf8007c70 con 0x7f1d00145870 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.137+0000 7f1ce7fff640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 0x7f1cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.137+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f1cf80aca30 con 0x7f1d00145870 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.137+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 0x7f1cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.138+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 0x7f1cec06edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000aa40 tx=0x7f1cf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.141+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d000cea50 con 0x7f1d00145870 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.367+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d0006a290 con 0x7f1d00145870 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.368+0000 7f1ce7fff640 1 -- 172.21.15.40:0/482175545 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f1cf807a810 con 0x7f1d00145870 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 msgr2=0x7f1cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 0x7f1cec06edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000aa40 tx=0x7f1cf0008040 comp rx=0 tx=0).stop 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 msgr2=0x7f1d00158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009f40 tx=0x7f1cf8004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 shutdown_connections 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cec06c8f0 0x7f1cec06edb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00145870 0x7f1d00158370 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d00073250 0x7f1d0013ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 --2- 172.21.15.40:0/482175545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d0006f980 0x7f1d0013c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 >> 172.21.15.40:0/482175545 conn(0x7f1d000cd830 msgr2=0x7f1d000cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 shutdown_connections 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07.370+0000 7f1d060f1640 1 -- 172.21.15.40:0/482175545 wait complete. 2024-04-26T21:21:07.381 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:21:07.381 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:07 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-26T21:21:09.184 INFO:teuthology.orchestra.run.smithi040.stderr:open_n_background: keeping files open 2024-04-26T21:21:09.184 INFO:teuthology.orchestra.run.smithi040.stderr:stdin closed, goodbye! 2024-04-26T21:21:09.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:09.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.354+0000 7f7714cf1640 1 Processor -- start 2024-04-26T21:21:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.354+0000 7f7714cf1640 1 -- start start 2024-04-26T21:21:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.355+0000 7f7714cf1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f771015f370 0x7f77100cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.355+0000 7f7714cf1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77100d3ee0 0x7f77100b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.355+0000 7f7714cf1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77100d03a0 con 0x7f771015f750 2024-04-26T21:21:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.355+0000 7f7714cf1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f77100b4d10 con 0x7f771015f370 2024-04-26T21:21:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.355+0000 7f7714cf1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77100d0680 con 0x7f77100d3ee0 2024-04-26T21:21:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770e59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f771015f370 0x7f77100cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770ed9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77100d3ee0 0x7f77100b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770dd9b640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f771015f750 0x7f77100d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:36286/0 (socket says 172.21.15.40:36286) 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770dd9b640 1 -- 172.21.15.40:0/2718546275 learned_addr learned my addr 172.21.15.40:0/2718546275 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770e59c640 1 -- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77100d3ee0 msgr2=0x7f77100b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770e59c640 1 --2- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77100d3ee0 0x7f77100b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770e59c640 1 -- 172.21.15.40:0/2718546275 >> v1:172.21.15.184:6789/0 conn(0x7f771015f750 legacy=0x7f77100d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770d59a640 1 -- 172.21.15.40:0/2718546275 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691507370 0 0) 0x7f77100d03a0 con 0x7f771015f750 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.356+0000 7f770e59c640 1 -- 172.21.15.40:0/2718546275 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77100d0960 con 0x7f771015f370 2024-04-26T21:21:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.357+0000 7f770e59c640 1 --2- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f370 0x7f77100cfd70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43c4c367471b7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.357+0000 7f770d59a640 1 -- 172.21.15.40:0/2718546275 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708037070 con 0x7f771015f370 2024-04-26T21:21:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.357+0000 7f770d59a640 1 -- 172.21.15.40:0/2718546275 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708032440 con 0x7f771015f370 2024-04-26T21:21:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.357+0000 7f770d59a640 1 -- 172.21.15.40:0/2718546275 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708032810 con 0x7f771015f370 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 -- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f370 msgr2=0x7f77100cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 --2- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f370 0x7f77100cfd70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802a180 comp rx=0 tx=0).stop 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 -- 172.21.15.40:0/2718546275 shutdown_connections 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 --2- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77100d3ee0 0x7f77100b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 --2- 172.21.15.40:0/2718546275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f370 0x7f77100cfd70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 -- 172.21.15.40:0/2718546275 >> 172.21.15.40:0/2718546275 conn(0x7f771008ec10 msgr2=0x7f771009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 -- 172.21.15.40:0/2718546275 shutdown_connections 2024-04-26T21:21:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.358+0000 7f7714cf1640 1 -- 172.21.15.40:0/2718546275 wait complete. 2024-04-26T21:21:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.359+0000 7f7714cf1640 1 Processor -- start 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.359+0000 7f7714cf1640 1 -- start start 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 0x7f77101f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 0x7f771015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77100b51c0 con 0x7f77100d3ee0 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77100d0f20 con 0x7f771015f750 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f7714cf1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77100b5340 con 0x7f771015f370 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46344/0 (socket says 172.21.15.40:46344) 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 -- 172.21.15.40:0/1381602512 learned_addr learned my addr 172.21.15.40:0/1381602512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770ed9d640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 0x7f771015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770dd9b640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 0x7f77101f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 -- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 msgr2=0x7f77101f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 0x7f77101f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 -- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 msgr2=0x7f771015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 0x7f771015fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770e59c640 1 -- 172.21.15.40:0/1381602512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7710161ab0 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770ed9d640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 0x7f771015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.360+0000 7f770dd9b640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 0x7f77101f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f770e59c640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f77080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708037050 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708038040 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7710161d30 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708004490 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.361+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101622f0 con 0x7f77100d3ee0 2024-04-26T21:21:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.362+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771006a290 con 0x7f77100d3ee0 2024-04-26T21:21:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.362+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f770803c6d0 con 0x7f77100d3ee0 2024-04-26T21:21:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.363+0000 7f7706ffd640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 0x7f76f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.363+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f77080ac5d0 con 0x7f77100d3ee0 2024-04-26T21:21:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.363+0000 7f770dd9b640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 0x7f76f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.363+0000 7f770dd9b640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 0x7f76f006edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f77100d24f0 tx=0x7f76f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.367+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7710161ab0 con 0x7f77100d3ee0 2024-04-26T21:21:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.591+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f771015dd10 con 0x7f77100d3ee0 2024-04-26T21:21:09.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.592+0000 7f7706ffd640 1 -- 172.21.15.40:0/1381602512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5191 (secure 0 0 0) 0x7f770807aa00 con 0x7f77100d3ee0 2024-04-26T21:21:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:09.593 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:21:05.985678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:09.593 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2024-04-26T21:21:09.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 msgr2=0x7f76f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 0x7f76f006edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f77100d24f0 tx=0x7f76f8009040 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 msgr2=0x7f77100cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f77080046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 shutdown_connections 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f76f006c8f0 0x7f76f006edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f771015f750 0x7f771015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f771015f370 0x7f77101f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 --2- 172.21.15.40:0/1381602512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77100d3ee0 0x7f77100cb780 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 >> 172.21.15.40:0/1381602512 conn(0x7f771008ec10 msgr2=0x7f771006e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 shutdown_connections 2024-04-26T21:21:09.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:09.594+0000 7f7714cf1640 1 -- 172.21.15.40:0/1381602512 wait complete. 2024-04-26T21:21:09.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi040.stdout:[{"id":9180,"entity":{"name":{"type":"client","num":9180},"addr":{"type":"v1","addr":"172.21.15.184:0","nonce":3143194780}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":11.548857833,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9330241362401863,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9180 v1:172.21.15.184:0/3143194780","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi184","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}},{"id":6836,"entity":{"name":{"type":"client","num":6836},"addr":{"type":"v1","addr":"172.21.15.40:0","nonce":3099832244}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":385,"uptime":12.689843789999999,"requests_in_flight":0,"num_completed_requests":316,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1576.3370739031959,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":374.29009503450834,"halflife":1.5},"recall_caps_throttle2o":{"value":94.795387224518166,"halflife":0.5},"session_cache_liveness":{"value":1178.0239778371485,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":112,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6836 v1:172.21.15.40:0/3099832244","completed_requests":[{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"},{"tid":229,"created_ino":"0x100000002bd"},{"tid":230,"created_ino":"0x100000002be"},{"tid":231,"created_ino":"0x100000002bf"},{"tid":232,"created_ino":"0x100000002c0"},{"tid":233,"created_ino":"0x100000002c1"},{"tid":234,"created_ino":"0x100000002c2"},{"tid":235,"created_ino":"0x100000002c3"},{"tid":236,"created_ino":"0x100000002c4"},{"tid":237,"created_ino":"0x100000002c5"},{"tid":238,"created_ino":"0x100000002c6"},{"tid":239,"created_ino":"0x100000002c7"},{"tid":240,"created_ino":"0x100000002c8"},{"tid":241,"created_ino":"0x100000002c9"},{"tid":242,"created_ino":"0x100000002ca"},{"tid":243,"created_ino":"0x100000002cb"},{"tid":244,"created_ino":"0x100000002cc"},{"tid":245,"created_ino":"0x100000002cd"},{"tid":246,"created_ino":"0x100000002ce"},{"tid":247,"created_ino":"0x100000002cf"},{"tid":248,"created_ino":"0x100000002d0"},{"tid":249,"created_ino":"0x100000002d1"},{"tid":250,"created_ino":"0x100000002d2"},{"tid":251,"created_ino":"0x100000002d3"},{"tid":252,"created_ino":"0x100000002d4"},{"tid":253,"created_ino":"0x100000002d5"},{"tid":254,"created_ino":"0x100000002d6"},{"tid":255,"created_ino":"0x100000002d7"},{"tid":256,"created_ino":"0x100000002d8"},{"tid":257,"created_ino":"0x100000002d9"},{"tid":258,"created_ino":"0x100000002da"},{"tid":259,"created_ino":"0x100000002db"},{"tid":260,"created_ino":"0x100000002dc"},{"tid":261,"created_ino":"0x100000002dd"},{"tid":262,"created_ino":"0x100000002de"},{"tid":263,"created_ino":"0x100000002df"},{"tid":264,"created_ino":"0x100000002e0"},{"tid":265,"created_ino":"0x100000002e1"},{"tid":266,"created_ino":"0x100000002e2"},{"tid":267,"created_ino":"0x100000002e3"},{"tid":268,"created_ino":"0x100000002e4"},{"tid":269,"created_ino":"0x100000002e5"},{"tid":270,"created_ino":"0x100000002e6"},{"tid":271,"created_ino":"0x100000002e7"},{"tid":272,"created_ino":"0x100000002e8"},{"tid":273,"created_ino":"0x100000002e9"},{"tid":274,"created_ino":"0x100000002ea"},{"tid":275,"created_ino":"0x100000002eb"},{"tid":276,"created_ino":"0x100000002ec"},{"tid":277,"created_ino":"0x100000002ed"},{"tid":278,"created_ino":"0x100000002ee"},{"tid":279,"created_ino":"0x100000002ef"},{"tid":280,"created_ino":"0x100000002f0"},{"tid":281,"created_ino":"0x100000002f1"},{"tid":282,"created_ino":"0x100000002f2"},{"tid":283,"created_ino":"0x100000002f3"},{"tid":284,"created_ino":"0x100000002f4"},{"tid":285,"created_ino":"0x100000002f5"},{"tid":286,"created_ino":"0x100000002f6"},{"tid":287,"created_ino":"0x100000002f7"},{"tid":288,"created_ino":"0x100000002f8"},{"tid":289,"created_ino":"0x100000002f9"},{"tid":290,"created_ino":"0x100000002fa"},{"tid":291,"created_ino":"0x100000002fb"},{"tid":292,"created_ino":"0x100000002fc"},{"tid":293,"created_ino":"0x100000002fd"},{"tid":294,"created_ino":"0x100000002fe"},{"tid":295,"created_ino":"0x100000002ff"},{"tid":296,"created_ino":"0x10000000300"},{"tid":297,"created_ino":"0x10000000301"},{"tid":298,"created_ino":"0x10000000302"},{"tid":299,"created_ino":"0x10000000303"},{"tid":300,"created_ino":"0x10000000304"},{"tid":301,"created_ino":"0x10000000305"},{"tid":302,"created_ino":"0x10000000306"},{"tid":303,"created_ino":"0x10000000307"},{"tid":304,"created_ino":"0x10000000308"},{"tid":305,"created_ino":"0x10000000309"},{"tid":306,"created_ino":"0x1000000030a"},{"tid":307,"created_ino":"0x1000000030b"},{"tid":308,"created_ino":"0x1000000030c"},{"tid":309,"created_ino":"0x1000000030d"},{"tid":310,"created_ino":"0x1000000030e"},{"tid":311,"created_ino":"0x1000000030f"},{"tid":312,"created_ino":"0x10000000310"},{"tid":313,"created_ino":"0x10000000311"},{"tid":314,"created_ino":"0x10000000312"},{"tid":315,"created_ino":"0x10000000313"},{"tid":316,"created_ino":"0x10000000314"},{"tid":317,"created_ino":"0x10000000315"},{"tid":318,"created_ino":"0x10000000316"},{"tid":319,"created_ino":"0x10000000317"},{"tid":320,"created_ino":"0x10000000318"},{"tid":321,"created_ino":"0x10000000319"},{"tid":322,"created_ino":"0x1000000031a"},{"tid":323,"created_ino":"0x1000000031b"},{"tid":324,"created_ino":"0x1000000031c"},{"tid":325,"created_ino":"0x1000000031d"},{"tid":326,"created_ino":"0x1000000031e"},{"tid":327,"created_ino":"0x1000000031f"},{"tid":328,"created_ino":"0x10000000320"},{"tid":329,"created_ino":"0x10000000321"},{"tid":330,"created_ino":"0x10000000322"},{"tid":331,"created_ino":"0x10000000323"},{"tid":332,"created_ino":"0x10000000324"},{"tid":333,"created_ino":"0x10000000325"},{"tid":334,"created_ino":"0x10000000326"},{"tid":335,"created_ino":"0x10000000327"},{"tid":336,"created_ino":"0x10000000328"},{"tid":337,"created_ino":"0x10000000329"},{"tid":338,"created_ino":"0x1000000032a"},{"tid":339,"created_ino":"0x1000000032b"},{"tid":340,"created_ino":"0x1000000032c"},{"tid":341,"created_ino":"0x1000000032d"},{"tid":342,"created_ino":"0x1000000032e"},{"tid":343,"created_ino":"0x1000000032f"},{"tid":344,"created_ino":"0x10000000330"},{"tid":345,"created_ino":"0x10000000331"},{"tid":346,"created_ino":"0x10000000332"},{"tid":347,"created_ino":"0x10000000333"},{"tid":348,"created_ino":"0x10000000334"},{"tid":349,"created_ino":"0x10000000335"},{"tid":350,"created_ino":"0x10000000336"},{"tid":351,"created_ino":"0x10000000337"},{"tid":352,"created_ino":"0x10000000338"},{"tid":353,"created_ino":"0x10000000339"},{"tid":354,"created_ino":"0x1000000033a"},{"tid":355,"created_ino":"0x1000000033b"},{"tid":356,"created_ino":"0x1000000033c"},{"tid":357,"created_ino":"0x1000000033d"},{"tid":358,"created_ino":"0x1000000033e"},{"tid":359,"created_ino":"0x1000000033f"},{"tid":360,"created_ino":"0x10000000340"},{"tid":361,"created_ino":"0x10000000341"},{"tid":362,"created_ino":"0x10000000342"},{"tid":363,"created_ino":"0x10000000343"},{"tid":364,"created_ino":"0x10000000344"},{"tid":365,"created_ino":"0x10000000345"},{"tid":366,"created_ino":"0x10000000346"},{"tid":367,"created_ino":"0x10000000347"},{"tid":368,"created_ino":"0x10000000348"},{"tid":369,"created_ino":"0x10000000349"},{"tid":370,"created_ino":"0x1000000034a"},{"tid":371,"created_ino":"0x1000000034b"},{"tid":372,"created_ino":"0x1000000034c"},{"tid":373,"created_ino":"0x1000000034d"},{"tid":374,"created_ino":"0x1000000034e"},{"tid":375,"created_ino":"0x1000000034f"},{"tid":376,"created_ino":"0x10000000350"},{"tid":377,"created_ino":"0x10000000351"},{"tid":378,"created_ino":"0x10000000352"},{"tid":379,"created_ino":"0x10000000353"},{"tid":380,"created_ino":"0x10000000354"},{"tid":381,"created_ino":"0x10000000355"},{"tid":382,"created_ino":"0x10000000356"},{"tid":383,"created_ino":"0x10000000357"},{"tid":384,"created_ino":"0x10000000358"},{"tid":385,"created_ino":"0x10000000359"},{"tid":386,"created_ino":"0x1000000035a"},{"tid":387,"created_ino":"0x1000000035b"},{"tid":388,"created_ino":"0x1000000035c"},{"tid":389,"created_ino":"0x1000000035d"},{"tid":390,"created_ino":"0x1000000035e"},{"tid":391,"created_ino":"0x1000000035f"},{"tid":392,"created_ino":"0x10000000360"},{"tid":393,"created_ino":"0x10000000361"},{"tid":394,"created_ino":"0x10000000362"},{"tid":395,"created_ino":"0x10000000363"},{"tid":396,"created_ino":"0x10000000364"},{"tid":397,"created_ino":"0x10000000365"},{"tid":398,"created_ino":"0x10000000366"},{"tid":399,"created_ino":"0x10000000367"},{"tid":400,"created_ino":"0x10000000368"},{"tid":401,"created_ino":"0x10000000369"},{"tid":402,"created_ino":"0x1000000036a"},{"tid":403,"created_ino":"0x1000000036b"},{"tid":404,"created_ino":"0x1000000036c"},{"tid":405,"created_ino":"0x1000000036d"},{"tid":406,"created_ino":"0x1000000036e"},{"tid":407,"created_ino":"0x1000000036f"},{"tid":408,"created_ino":"0x10000000370"},{"tid":409,"created_ino":"0x10000000371"},{"tid":410,"created_ino":"0x10000000372"},{"tid":411,"created_ino":"0x10000000373"},{"tid":412,"created_ino":"0x10000000374"},{"tid":413,"created_ino":"0x10000000375"},{"tid":414,"created_ino":"0x10000000376"},{"tid":415,"created_ino":"0x10000000377"},{"tid":416,"created_ino":"0x10000000378"},{"tid":417,"created_ino":"0x10000000379"},{"tid":418,"created_ino":"0x1000000037a"},{"tid":419,"created_ino":"0x1000000037b"},{"tid":420,"created_ino":"0x1000000037c"},{"tid":421,"created_ino":"0x1000000037d"},{"tid":422,"created_ino":"0x1000000037e"},{"tid":423,"created_ino":"0x1000000037f"},{"tid":424,"created_ino":"0x10000000380"},{"tid":425,"created_ino":"0x10000000381"},{"tid":426,"created_ino":"0x10000000382"},{"tid":427,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x0000000000017bff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi040","kernel_version":"5.14.0-437.el9.x86_64","root":"/"}}] 2024-04-26T21:21:09.730 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "1", "hostname": "smithi184", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.184:0", "nonce": 3143194780, "type": "v1" }, "name": { "num": 9180, "type": "client" } }, "id": 9180, "inst": "client.9180 v1:172.21.15.184:0/3143194780", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9330241362401863 }, "state": "open", "uptime": 11.548857833 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x0000000000017bff" }, "entity_id": "0", "hostname": "smithi040", "kernel_version": "5.14.0-437.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 }, { "created_ino": "0x100000002bd", "tid": 229 }, { "created_ino": "0x100000002be", "tid": 230 }, { "created_ino": "0x100000002bf", "tid": 231 }, { "created_ino": "0x100000002c0", "tid": 232 }, { "created_ino": "0x100000002c1", "tid": 233 }, { "created_ino": "0x100000002c2", "tid": 234 }, { "created_ino": "0x100000002c3", "tid": 235 }, { "created_ino": "0x100000002c4", "tid": 236 }, { "created_ino": "0x100000002c5", "tid": 237 }, { "created_ino": "0x100000002c6", "tid": 238 }, { "created_ino": "0x100000002c7", "tid": 239 }, { "created_ino": "0x100000002c8", "tid": 240 }, { "created_ino": "0x100000002c9", "tid": 241 }, { "created_ino": "0x100000002ca", "tid": 242 }, { "created_ino": "0x100000002cb", "tid": 243 }, { "created_ino": "0x100000002cc", "tid": 244 }, { "created_ino": "0x100000002cd", "tid": 245 }, { "created_ino": "0x100000002ce", "tid": 246 }, { "created_ino": "0x100000002cf", "tid": 247 }, { "created_ino": "0x100000002d0", "tid": 248 }, { "created_ino": "0x100000002d1", "tid": 249 }, { "created_ino": "0x100000002d2", "tid": 250 }, { "created_ino": "0x100000002d3", "tid": 251 }, { "created_ino": "0x100000002d4", "tid": 252 }, { "created_ino": "0x100000002d5", "tid": 253 }, { "created_ino": "0x100000002d6", "tid": 254 }, { "created_ino": "0x100000002d7", "tid": 255 }, { "created_ino": "0x100000002d8", "tid": 256 }, { "created_ino": "0x100000002d9", "tid": 257 }, { "created_ino": "0x100000002da", "tid": 258 }, { "created_ino": "0x100000002db", "tid": 259 }, { "created_ino": "0x100000002dc", "tid": 260 }, { "created_ino": "0x100000002dd", "tid": 261 }, { "created_ino": "0x100000002de", "tid": 262 }, { "created_ino": "0x100000002df", "tid": 263 }, { "created_ino": "0x100000002e0", "tid": 264 }, { "created_ino": "0x100000002e1", "tid": 265 }, { "created_ino": "0x100000002e2", "tid": 266 }, { "created_ino": "0x100000002e3", "tid": 267 }, { "created_ino": "0x100000002e4", "tid": 268 }, { "created_ino": "0x100000002e5", "tid": 269 }, { "created_ino": "0x100000002e6", "tid": 270 }, { "created_ino": "0x100000002e7", "tid": 271 }, { "created_ino": "0x100000002e8", "tid": 272 }, { "created_ino": "0x100000002e9", "tid": 273 }, { "created_ino": "0x100000002ea", "tid": 274 }, { "created_ino": "0x100000002eb", "tid": 275 }, { "created_ino": "0x100000002ec", "tid": 276 }, { "created_ino": "0x100000002ed", "tid": 277 }, { "created_ino": "0x100000002ee", "tid": 278 }, { "created_ino": "0x100000002ef", "tid": 279 }, { "created_ino": "0x100000002f0", "tid": 280 }, { "created_ino": "0x100000002f1", "tid": 281 }, { "created_ino": "0x100000002f2", "tid": 282 }, { "created_ino": "0x100000002f3", "tid": 283 }, { "created_ino": "0x100000002f4", "tid": 284 }, { "created_ino": "0x100000002f5", "tid": 285 }, { "created_ino": "0x100000002f6", "tid": 286 }, { "created_ino": "0x100000002f7", "tid": 287 }, { "created_ino": "0x100000002f8", "tid": 288 }, { "created_ino": "0x100000002f9", "tid": 289 }, { "created_ino": "0x100000002fa", "tid": 290 }, { "created_ino": "0x100000002fb", "tid": 291 }, { "created_ino": "0x100000002fc", "tid": 292 }, { "created_ino": "0x100000002fd", "tid": 293 }, { "created_ino": "0x100000002fe", "tid": 294 }, { "created_ino": "0x100000002ff", "tid": 295 }, { "created_ino": "0x10000000300", "tid": 296 }, { "created_ino": "0x10000000301", "tid": 297 }, { "created_ino": "0x10000000302", "tid": 298 }, { "created_ino": "0x10000000303", "tid": 299 }, { "created_ino": "0x10000000304", "tid": 300 }, { "created_ino": "0x10000000305", "tid": 301 }, { "created_ino": "0x10000000306", "tid": 302 }, { "created_ino": "0x10000000307", "tid": 303 }, { "created_ino": "0x10000000308", "tid": 304 }, { "created_ino": "0x10000000309", "tid": 305 }, { "created_ino": "0x1000000030a", "tid": 306 }, { "created_ino": "0x1000000030b", "tid": 307 }, { "created_ino": "0x1000000030c", "tid": 308 }, { "created_ino": "0x1000000030d", "tid": 309 }, { "created_ino": "0x1000000030e", "tid": 310 }, { "created_ino": "0x1000000030f", "tid": 311 }, { "created_ino": "0x10000000310", "tid": 312 }, { "created_ino": "0x10000000311", "tid": 313 }, { "created_ino": "0x10000000312", "tid": 314 }, { "created_ino": "0x10000000313", "tid": 315 }, { "created_ino": "0x10000000314", "tid": 316 }, { "created_ino": "0x10000000315", "tid": 317 }, { "created_ino": "0x10000000316", "tid": 318 }, { "created_ino": "0x10000000317", "tid": 319 }, { "created_ino": "0x10000000318", "tid": 320 }, { "created_ino": "0x10000000319", "tid": 321 }, { "created_ino": "0x1000000031a", "tid": 322 }, { "created_ino": "0x1000000031b", "tid": 323 }, { "created_ino": "0x1000000031c", "tid": 324 }, { "created_ino": "0x1000000031d", "tid": 325 }, { "created_ino": "0x1000000031e", "tid": 326 }, { "created_ino": "0x1000000031f", "tid": 327 }, { "created_ino": "0x10000000320", "tid": 328 }, { "created_ino": "0x10000000321", "tid": 329 }, { "created_ino": "0x10000000322", "tid": 330 }, { "created_ino": "0x10000000323", "tid": 331 }, { "created_ino": "0x10000000324", "tid": 332 }, { "created_ino": "0x10000000325", "tid": 333 }, { "created_ino": "0x10000000326", "tid": 334 }, { "created_ino": "0x10000000327", "tid": 335 }, { "created_ino": "0x10000000328", "tid": 336 }, { "created_ino": "0x10000000329", "tid": 337 }, { "created_ino": "0x1000000032a", "tid": 338 }, { "created_ino": "0x1000000032b", "tid": 339 }, { "created_ino": "0x1000000032c", "tid": 340 }, { "created_ino": "0x1000000032d", "tid": 341 }, { "created_ino": "0x1000000032e", "tid": 342 }, { "created_ino": "0x1000000032f", "tid": 343 }, { "created_ino": "0x10000000330", "tid": 344 }, { "created_ino": "0x10000000331", "tid": 345 }, { "created_ino": "0x10000000332", "tid": 346 }, { "created_ino": "0x10000000333", "tid": 347 }, { "created_ino": "0x10000000334", "tid": 348 }, { "created_ino": "0x10000000335", "tid": 349 }, { "created_ino": "0x10000000336", "tid": 350 }, { "created_ino": "0x10000000337", "tid": 351 }, { "created_ino": "0x10000000338", "tid": 352 }, { "created_ino": "0x10000000339", "tid": 353 }, { "created_ino": "0x1000000033a", "tid": 354 }, { "created_ino": "0x1000000033b", "tid": 355 }, { "created_ino": "0x1000000033c", "tid": 356 }, { "created_ino": "0x1000000033d", "tid": 357 }, { "created_ino": "0x1000000033e", "tid": 358 }, { "created_ino": "0x1000000033f", "tid": 359 }, { "created_ino": "0x10000000340", "tid": 360 }, { "created_ino": "0x10000000341", "tid": 361 }, { "created_ino": "0x10000000342", "tid": 362 }, { "created_ino": "0x10000000343", "tid": 363 }, { "created_ino": "0x10000000344", "tid": 364 }, { "created_ino": "0x10000000345", "tid": 365 }, { "created_ino": "0x10000000346", "tid": 366 }, { "created_ino": "0x10000000347", "tid": 367 }, { "created_ino": "0x10000000348", "tid": 368 }, { "created_ino": "0x10000000349", "tid": 369 }, { "created_ino": "0x1000000034a", "tid": 370 }, { "created_ino": "0x1000000034b", "tid": 371 }, { "created_ino": "0x1000000034c", "tid": 372 }, { "created_ino": "0x1000000034d", "tid": 373 }, { "created_ino": "0x1000000034e", "tid": 374 }, { "created_ino": "0x1000000034f", "tid": 375 }, { "created_ino": "0x10000000350", "tid": 376 }, { "created_ino": "0x10000000351", "tid": 377 }, { "created_ino": "0x10000000352", "tid": 378 }, { "created_ino": "0x10000000353", "tid": 379 }, { "created_ino": "0x10000000354", "tid": 380 }, { "created_ino": "0x10000000355", "tid": 381 }, { "created_ino": "0x10000000356", "tid": 382 }, { "created_ino": "0x10000000357", "tid": 383 }, { "created_ino": "0x10000000358", "tid": 384 }, { "created_ino": "0x10000000359", "tid": 385 }, { "created_ino": "0x1000000035a", "tid": 386 }, { "created_ino": "0x1000000035b", "tid": 387 }, { "created_ino": "0x1000000035c", "tid": 388 }, { "created_ino": "0x1000000035d", "tid": 389 }, { "created_ino": "0x1000000035e", "tid": 390 }, { "created_ino": "0x1000000035f", "tid": 391 }, { "created_ino": "0x10000000360", "tid": 392 }, { "created_ino": "0x10000000361", "tid": 393 }, { "created_ino": "0x10000000362", "tid": 394 }, { "created_ino": "0x10000000363", "tid": 395 }, { "created_ino": "0x10000000364", "tid": 396 }, { "created_ino": "0x10000000365", "tid": 397 }, { "created_ino": "0x10000000366", "tid": 398 }, { "created_ino": "0x10000000367", "tid": 399 }, { "created_ino": "0x10000000368", "tid": 400 }, { "created_ino": "0x10000000369", "tid": 401 }, { "created_ino": "0x1000000036a", "tid": 402 }, { "created_ino": "0x1000000036b", "tid": 403 }, { "created_ino": "0x1000000036c", "tid": 404 }, { "created_ino": "0x1000000036d", "tid": 405 }, { "created_ino": "0x1000000036e", "tid": 406 }, { "created_ino": "0x1000000036f", "tid": 407 }, { "created_ino": "0x10000000370", "tid": 408 }, { "created_ino": "0x10000000371", "tid": 409 }, { "created_ino": "0x10000000372", "tid": 410 }, { "created_ino": "0x10000000373", "tid": 411 }, { "created_ino": "0x10000000374", "tid": 412 }, { "created_ino": "0x10000000375", "tid": 413 }, { "created_ino": "0x10000000376", "tid": 414 }, { "created_ino": "0x10000000377", "tid": 415 }, { "created_ino": "0x10000000378", "tid": 416 }, { "created_ino": "0x10000000379", "tid": 417 }, { "created_ino": "0x1000000037a", "tid": 418 }, { "created_ino": "0x1000000037b", "tid": 419 }, { "created_ino": "0x1000000037c", "tid": 420 }, { "created_ino": "0x1000000037d", "tid": 421 }, { "created_ino": "0x1000000037e", "tid": 422 }, { "created_ino": "0x1000000037f", "tid": 423 }, { "created_ino": "0x10000000380", "tid": 424 }, { "created_ino": "0x10000000381", "tid": 425 }, { "created_ino": "0x10000000382", "tid": 426 }, { "created_ino": "0x10000000383", "tid": 427 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.40:0", "nonce": 3099832244, "type": "v1" }, "name": { "num": 6836, "type": "client" } }, "id": 6836, "inst": "client.6836 v1:172.21.15.40:0/3099832244", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 112, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 316, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1576.337073903196 }, "recall_caps_throttle": { "halflife": 1.5, "value": 374.29009503450834 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 94.79538722451817 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 385, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1178.0239778371485 }, "state": "open", "uptime": 12.68984379 } ] 2024-04-26T21:21:09.730 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T21:21:09.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:21:09.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:09.767 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:09.808 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:21:09.809 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:09.809 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/."\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi040') 2024-04-26T21:21:09.809 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:09.809 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:09.825 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:09.825 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:09.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:10.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.034+0000 7fda0d1e4640 1 Processor -- start 2024-04-26T21:21:10.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.034+0000 7fda0d1e4640 1 -- start start 2024-04-26T21:21:10.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0d1e4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fda0809c400 0x7fda080cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0d1e4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda080d14c0 con 0x7fda08070520 2024-04-26T21:21:10.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0d1e4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda080d1760 con 0x7fda08070d20 2024-04-26T21:21:10.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0d1e4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fda080d1290 con 0x7fda0809c400 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0759e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda08070520 0x7fda080cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44808/0 (socket says 172.21.15.40:44808) 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.035+0000 7fda0759e640 1 -- 172.21.15.40:0/958559186 learned_addr learned my addr 172.21.15.40:0/958559186 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda0659c640 1 --2- 172.21.15.40:0/958559186 >> v2:172.21.15.184:3300/0 conn(0x7fda0809c400 0x7fda080cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240257494 0 0) 0x7fda080d14c0 con 0x7fda08070520 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9f4003520 con 0x7fda08070520 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1705876845 0 0) 0x7fd9f4003520 con 0x7fda08070520 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 >> v2:172.21.15.184:3300/0 conn(0x7fda0809c400 msgr2=0x7fda080cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 --2- 172.21.15.40:0/958559186 >> v2:172.21.15.184:3300/0 conn(0x7fda0809c400 0x7fda080cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 >> v1:172.21.15.184:6789/0 conn(0x7fda08070d20 legacy=0x7fda08071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda080d1a40 con 0x7fda08070520 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd9f80019f0 con 0x7fda08070520 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 >> v1:172.21.15.40:6789/0 conn(0x7fda08070520 legacy=0x7fda080cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f4004800 0x7fd9f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40051c0 0x7fd9f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f400c030 con 0x7fd9f40039b0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f400c1e0 con 0x7fd9f4004800 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f400c3e0 con 0x7fd9f40051c0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda0659c640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f4004800 0x7fd9f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda0759e640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40051c0 0x7fd9f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 -- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40051c0 msgr2=0x7fd9f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40051c0 0x7fd9f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 -- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f4004800 msgr2=0x7fd9f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9f4004800 0x7fd9f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda06d9d640 1 -- 172.21.15.40:0/958559186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f4003520 con 0x7fd9f40039b0 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda0759e640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40051c0 0x7fd9f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.036+0000 7fda0759e640 1 -- 172.21.15.40:0/958559186 reap_dead start 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.037+0000 7fda06d9d640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003dd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fda00001750 tx=0x7fd9f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8381ff878958861b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.037+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00001370 con 0x7fd9f40039b0 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.037+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda000052d0 con 0x7fd9f40039b0 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.037+0000 7fda05d9b640 1 -- 172.21.15.40:0/958559186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0002abc0 con 0x7fd9f40039b0 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 -- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 msgr2=0x7fd9f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003dd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fda00001750 tx=0x7fd9f4005830 comp rx=0 tx=0).stop 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 -- 172.21.15.40:0/958559186 shutdown_connections 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/958559186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003dd0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 -- 172.21.15.40:0/958559186 >> 172.21.15.40:0/958559186 conn(0x7fda080b4a40 msgr2=0x7fda080b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 -- 172.21.15.40:0/958559186 shutdown_connections 2024-04-26T21:21:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.038+0000 7fda0d1e4640 1 -- 172.21.15.40:0/958559186 wait complete. 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.039+0000 7fda0d1e4640 1 Processor -- start 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.039+0000 7fda0d1e4640 1 -- start start 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40039b0 0x7fda081f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 0x7fda080b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda080cf350 con 0x7fda080b6870 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda080d2000 con 0x7fda080b7fb0 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0d1e4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fda080d2520 con 0x7fd9f40039b0 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46358/0 (socket says 172.21.15.40:46358) 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 -- 172.21.15.40:0/2724899575 learned_addr learned my addr 172.21.15.40:0/2724899575 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0759e640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 0x7fda080b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 -- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40039b0 msgr2=0x7fda081f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda06d9d640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40039b0 0x7fda081f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40039b0 0x7fda081f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 -- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 msgr2=0x7fda080b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 0x7fda080b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0659c640 1 -- 172.21.15.40:0/2724899575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda081f5cd0 con 0x7fda080b6870 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.040+0000 7fda0759e640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 0x7fda080b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fda0659c640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0011a70 tx=0x7fd9f0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000fb50 con 0x7fda080b6870 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f000fcf0 con 0x7fda080b6870 2024-04-26T21:21:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda080c76f0 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000fea0 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.041+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda080b0ec0 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.042+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda080693d0 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.042+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9f0012040 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.043+0000 7fd9ff7fe640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 0x7fd9e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.043+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f008c160 con 0x7fda080b6870 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.043+0000 7fda06d9d640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 0x7fd9e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.043+0000 7fda06d9d640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 0x7fd9e806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fda00032000 tx=0x7fda0002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.047+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda081f5cd0 con 0x7fda080b6870 2024-04-26T21:21:10.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.232+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda08074880 con 0x7fda080b6870 2024-04-26T21:21:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.233+0000 7fd9ff7fe640 1 -- 172.21.15.40:0/2724899575 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7fd9f0019030 con 0x7fda080b6870 2024-04-26T21:21:10.234 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:10.234 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 msgr2=0x7fd9e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 0x7fd9e806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fda00032000 tx=0x7fda0002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 msgr2=0x7fda080c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0011a70 tx=0x7fd9f0011f30 comp rx=0 tx=0).stop 2024-04-26T21:21:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 shutdown_connections 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd9e806c8f0 0x7fd9e806edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda080b7fb0 0x7fda080b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda080b6870 0x7fda080c93c0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 --2- 172.21.15.40:0/2724899575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9f40039b0 0x7fda081f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 >> 172.21.15.40:0/2724899575 conn(0x7fda080b4a40 msgr2=0x7fda08161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 shutdown_connections 2024-04-26T21:21:10.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.235+0000 7fda0d1e4640 1 -- 172.21.15.40:0/2724899575 wait complete. 2024-04-26T21:21:10.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:20:43.597608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:20:44.259933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:10.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.350+0000 7f1d668f9640 1 Processor -- start 2024-04-26T21:21:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.350+0000 7f1d668f9640 1 -- start start 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d6015f2b0 0x7f1d6006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d600706a0 0x7f1d6006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d6006c7e0 0x7f1d60162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600af610 con 0x7f1d6015f2b0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600700b0 con 0x7f1d6006c7e0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d668f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6006cc30 con 0x7f1d600706a0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d6015f2b0 0x7f1d6006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d6015f2b0 0x7f1d6006fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46374/0 (socket says 172.21.15.40:46374) 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3020422781 learned_addr learned my addr 172.21.15.40:0/3020422781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d64e96640 1 --2- 172.21.15.40:0/3020422781 >> v2:172.21.15.184:3300/0 conn(0x7f1d6006c7e0 0x7f1d60162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5f7fe640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d600706a0 0x7f1d6006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3020422781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d600706a0 msgr2=0x7f1d6006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d600706a0 0x7f1d6006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3020422781 >> v2:172.21.15.184:3300/0 conn(0x7f1d6006c7e0 msgr2=0x7f1d60162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3020422781 >> v2:172.21.15.184:3300/0 conn(0x7f1d6006c7e0 0x7f1d60162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.351+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3020422781 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60070b30 con 0x7f1d6015f2b0 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6015f2b0 0x7f1d6006fb70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b1580 tx=0x7f1d5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37de852104a8721 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d5effd640 1 -- 172.21.15.40:0/3020422781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037070 con 0x7f1d6015f2b0 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d5effd640 1 -- 172.21.15.40:0/3020422781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50032440 con 0x7f1d6015f2b0 2024-04-26T21:21:10.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d5effd640 1 -- 172.21.15.40:0/3020422781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50032810 con 0x7f1d6015f2b0 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 -- 172.21.15.40:0/3020422781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6015f2b0 msgr2=0x7f1d6006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6015f2b0 0x7f1d6006fb70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b1580 tx=0x7f1d5002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 -- 172.21.15.40:0/3020422781 shutdown_connections 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3020422781 >> v2:172.21.15.184:3300/0 conn(0x7f1d6006c7e0 0x7f1d60162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d600706a0 0x7f1d6006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3020422781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6015f2b0 0x7f1d6006fb70 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.352+0000 7f1d668f9640 1 -- 172.21.15.40:0/3020422781 >> 172.21.15.40:0/3020422781 conn(0x7f1d6006f050 msgr2=0x7f1d6006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.353+0000 7f1d668f9640 1 -- 172.21.15.40:0/3020422781 shutdown_connections 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.353+0000 7f1d668f9640 1 -- 172.21.15.40:0/3020422781 wait complete. 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.354+0000 7f1d668f9640 1 Processor -- start 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.354+0000 7f1d668f9640 1 -- start start 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6015f2b0 0x7f1d6015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d601f5120 0x7f1d6015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600713d0 con 0x7f1d6006c7e0 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600710f0 con 0x7f1d6015f2b0 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600b4f40 con 0x7f1d601f5120 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46384/0 (socket says 172.21.15.40:46384) 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3345523344 learned_addr learned my addr 172.21.15.40:0/3345523344 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d64e96640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d601f5120 0x7f1d6015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5f7fe640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6015f2b0 0x7f1d6015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d601f5120 msgr2=0x7f1d6015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d601f5120 0x7f1d6015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6015f2b0 msgr2=0x7f1d6015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6015f2b0 0x7f1d6015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 -- 172.21.15.40:0/3345523344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d600c7270 con 0x7f1d6006c7e0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5ffff640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002a220 tx=0x7f1d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50041070 con 0x7f1d6006c7e0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5003c870 con 0x7f1d6006c7e0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d600c7550 con 0x7f1d6006c7e0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.355+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037040 con 0x7f1d6006c7e0 2024-04-26T21:21:10.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.356+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d600c7b10 con 0x7f1d6006c7e0 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.357+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6006a290 con 0x7f1d6006c7e0 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.357+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d50044070 con 0x7f1d6006c7e0 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.358+0000 7f1d5cff9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.358+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f1d500ac5d0 con 0x7f1d6006c7e0 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.358+0000 7f1d5f7fe640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.358+0000 7f1d5f7fe640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1d6015d2a0 tx=0x7f1d54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.362+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d5007a320 con 0x7f1d6006c7e0 2024-04-26T21:21:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.549+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d6015a410 con 0x7f1d6006c7e0 2024-04-26T21:21:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.550+0000 7f1d5cff9640 1 -- 172.21.15.40:0/3345523344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f1d5007e620 con 0x7f1d6006c7e0 2024-04-26T21:21:10.551 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 msgr2=0x7f1d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1d6015d2a0 tx=0x7f1d54008040 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 msgr2=0x7f1d6015d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002a220 tx=0x7f1d500046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 shutdown_connections 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d601f5120 0x7f1d6015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6015f2b0 0x7f1d6015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 --2- 172.21.15.40:0/3345523344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d6006c7e0 0x7f1d6015d310 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 >> 172.21.15.40:0/3345523344 conn(0x7f1d6006f050 msgr2=0x7f1d600b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 shutdown_connections 2024-04-26T21:21:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.552+0000 7f1d668f9640 1 -- 172.21.15.40:0/3345523344 wait complete. 2024-04-26T21:21:10.563 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:10.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.665+0000 7f29237af640 1 Processor -- start 2024-04-26T21:21:10.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 -- start start 2024-04-26T21:21:10.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 0x7f291c1635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f291c0b3300 0x7f291c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f291c0d15f0 con 0x7f291c0b3300 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f291c09c790 con 0x7f291c0d1170 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.666+0000 7f29237af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f291c163ec0 con 0x7f291c1631f0 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f291c0b3300 0x7f291c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f291c0b3300 0x7f291c0b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48900/0 (socket says 172.21.15.40:48900) 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 -- 172.21.15.40:0/3238391114 learned_addr learned my addr 172.21.15.40:0/3238391114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f292154b640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2920d4a640 1 --2- 172.21.15.40:0/3238391114 >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 0x7f291c1635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 -- 172.21.15.40:0/3238391114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 msgr2=0x7f291c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 -- 172.21.15.40:0/3238391114 >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 msgr2=0x7f291c1635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 --2- 172.21.15.40:0/3238391114 >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 0x7f291c1635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2921d4c640 1 -- 172.21.15.40:0/3238391114 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0d1a40 con 0x7f291c0b3300 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.667+0000 7f2920d4a640 1 --2- 172.21.15.40:0/3238391114 >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 0x7f291c1635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f2921d4c640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0b3300 0x7f291c0b3790 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f291c070210 tx=0x7f291402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54514451f59e1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f2913fff640 1 -- 172.21.15.40:0/3238391114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914036070 con 0x7f291c0b3300 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f2913fff640 1 -- 172.21.15.40:0/3238391114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914031600 con 0x7f291c0b3300 2024-04-26T21:21:10.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f2913fff640 1 -- 172.21.15.40:0/3238391114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140319d0 con 0x7f291c0b3300 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 -- 172.21.15.40:0/3238391114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0b3300 msgr2=0x7f291c0b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0b3300 0x7f291c0b3790 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f291c070210 tx=0x7f291402a610 comp rx=0 tx=0).stop 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 -- 172.21.15.40:0/3238391114 shutdown_connections 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0b3300 0x7f291c0b3790 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 --2- 172.21.15.40:0/3238391114 >> v2:172.21.15.184:3300/0 conn(0x7f291c0d1170 0x7f291c1635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 --2- 172.21.15.40:0/3238391114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.668+0000 7f29237af640 1 -- 172.21.15.40:0/3238391114 >> 172.21.15.40:0/3238391114 conn(0x7f291c09ba40 msgr2=0x7f291c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.669+0000 7f29237af640 1 -- 172.21.15.40:0/3238391114 shutdown_connections 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.669+0000 7f29237af640 1 -- 172.21.15.40:0/3238391114 wait complete. 2024-04-26T21:21:10.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 Processor -- start 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 -- start start 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 0x7f291c0b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c09ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0d0230 con 0x7f291c0d1170 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0d2000 con 0x7f291c0b3300 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f29237af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f291c153970 con 0x7f291c1631f0 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.670+0000 7f2920d4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48916/0 (socket says 172.21.15.40:48916) 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 -- 172.21.15.40:0/719038115 learned_addr learned my addr 172.21.15.40:0/719038115 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2921d4c640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c09ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f292154b640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 0x7f291c0b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 -- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 msgr2=0x7f291c09ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c09ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 -- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 msgr2=0x7f291c0b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 0x7f291c0b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 -- 172.21.15.40:0/719038115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1624b0 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2921d4c640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c09ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f292154b640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 0x7f291c0b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2920d4a640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e840 tx=0x7f290c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00f040 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c004030 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c162790 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c004400 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.671+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c0b4c20 con 0x7f291c0d1170 2024-04-26T21:21:10.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.672+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c162cb0 con 0x7f291c0d1170 2024-04-26T21:21:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.673+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f290c00c6d0 con 0x7f291c0d1170 2024-04-26T21:21:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.673+0000 7f2911ffb640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.673+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f290c08b8f0 con 0x7f291c0d1170 2024-04-26T21:21:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.673+0000 7f292154b640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.674+0000 7f292154b640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2918009a20 tx=0x7f2918008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.677+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f291c162cb0 con 0x7f291c0d1170 2024-04-26T21:21:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.865+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f291c06a290 con 0x7f291c0d1170 2024-04-26T21:21:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.865+0000 7f2911ffb640 1 -- 172.21.15.40:0/719038115 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f290c017020 con 0x7f291c0d1170 2024-04-26T21:21:10.866 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 msgr2=0x7f290006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2918009a20 tx=0x7f2918008040 comp rx=0 tx=0).stop 2024-04-26T21:21:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 msgr2=0x7f291c153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e840 tx=0x7f290c00ed00 comp rx=0 tx=0).stop 2024-04-26T21:21:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 shutdown_connections 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f291c1631f0 0x7f291c09ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f291c0d1170 0x7f291c153430 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 --2- 172.21.15.40:0/719038115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c0b3300 0x7f291c0b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 >> 172.21.15.40:0/719038115 conn(0x7f291c09ba40 msgr2=0x7f291c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 shutdown_connections 2024-04-26T21:21:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.867+0000 7f29237af640 1 -- 172.21.15.40:0/719038115 wait complete. 2024-04-26T21:21:10.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.982+0000 7f7952c1d640 1 Processor -- start 2024-04-26T21:21:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.982+0000 7f7952c1d640 1 -- start start 2024-04-26T21:21:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f7952c1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 0x7f794c0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f7952c1d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f794c163aa0 0x7f794c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f7952c1d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c0d3f50 con 0x7f794c0b60b0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f7952c1d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f794c15fa60 con 0x7f794c163aa0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f7952c1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0b5090 con 0x7f794c15f680 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f79511ba640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f794c0b60b0 0x7f794c0d4f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38534/0 (socket says 172.21.15.40:38534) 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f79511ba640 1 -- 172.21.15.40:0/1630787704 learned_addr learned my addr 172.21.15.40:0/1630787704 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f794bfff640 1 --2- 172.21.15.40:0/1630787704 >> v2:172.21.15.184:3300/0 conn(0x7f794c163aa0 0x7f794c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f79509b9640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 0x7f794c0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f794b7fe640 1 -- 172.21.15.40:0/1630787704 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617462103 0 0) 0x7f794c0d3f50 con 0x7f794c0b60b0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.984+0000 7f794b7fe640 1 -- 172.21.15.40:0/1630787704 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f793c003520 con 0x7f794c0b60b0 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f794bfff640 1 -- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 msgr2=0x7f794c0d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f794bfff640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 0x7f794c0d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f794bfff640 1 -- 172.21.15.40:0/1630787704 >> v1:172.21.15.40:6789/0 conn(0x7f794c0b60b0 legacy=0x7f794c0d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f794bfff640 1 -- 172.21.15.40:0/1630787704 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0b5370 con 0x7f794c163aa0 2024-04-26T21:21:10.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f79509b9640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 0x7f794c0d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.985+0000 7f794bfff640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c163aa0 0x7f794c0b5b70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f79380097a0 tx=0x7f793802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63526d5fe3d03e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f794b7fe640 1 -- 172.21.15.40:0/1630787704 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938037070 con 0x7f794c163aa0 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f794b7fe640 1 -- 172.21.15.40:0/1630787704 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7938032440 con 0x7f794c163aa0 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f794b7fe640 1 -- 172.21.15.40:0/1630787704 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938032810 con 0x7f794c163aa0 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 -- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c163aa0 msgr2=0x7f794c0b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c163aa0 0x7f794c0b5b70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f79380097a0 tx=0x7f793802a180 comp rx=0 tx=0).stop 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 -- 172.21.15.40:0/1630787704 shutdown_connections 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c163aa0 0x7f794c0b5b70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 --2- 172.21.15.40:0/1630787704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c15f680 0x7f794c0d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 -- 172.21.15.40:0/1630787704 >> 172.21.15.40:0/1630787704 conn(0x7f794c08eca0 msgr2=0x7f794c09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:10.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.986+0000 7f7952c1d640 1 -- 172.21.15.40:0/1630787704 shutdown_connections 2024-04-26T21:21:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.987+0000 7f7952c1d640 1 -- 172.21.15.40:0/1630787704 wait complete. 2024-04-26T21:21:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.988+0000 7f7952c1d640 1 Processor -- start 2024-04-26T21:21:10.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.988+0000 7f7952c1d640 1 -- start start 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 0x7f794c14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c15f680 0x7f794c162420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0d57b0 con 0x7f794c163aa0 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0d5630 con 0x7f794c15f680 2024-04-26T21:21:10.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f7952c1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0d5930 con 0x7f794c0b60b0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48926/0 (socket says 172.21.15.40:48926) 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 -- 172.21.15.40:0/3023311 learned_addr learned my addr 172.21.15.40:0/3023311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79509b9640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 0x7f794c14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f794bfff640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c15f680 0x7f794c162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 -- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 msgr2=0x7f794c14c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 0x7f794c14c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 -- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c15f680 msgr2=0x7f794c162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c15f680 0x7f794c162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79511ba640 1 -- 172.21.15.40:0/3023311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0cec70 con 0x7f794c163aa0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.990+0000 7f79509b9640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 0x7f794c14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f79511ba640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f794400cb40 tx=0x7f794400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944017070 con 0x7f794c163aa0 2024-04-26T21:21:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794400aca0 con 0x7f794c163aa0 2024-04-26T21:21:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c0cef50 con 0x7f794c163aa0 2024-04-26T21:21:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944012620 con 0x7f794c163aa0 2024-04-26T21:21:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.991+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c14cce0 con 0x7f794c163aa0 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.992+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c06a290 con 0x7f794c163aa0 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.992+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794401a070 con 0x7f794c163aa0 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.993+0000 7f79497fa640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 0x7f793006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.993+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f794408ba90 con 0x7f794c163aa0 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.993+0000 7f79509b9640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 0x7f793006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.994+0000 7f79509b9640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 0x7f793006edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ba0 tx=0x7f7940008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:10.998+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7944059d90 con 0x7f794c163aa0 2024-04-26T21:21:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.184+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f794c0b4cd0 con 0x7f794c163aa0 2024-04-26T21:21:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.184+0000 7f79497fa640 1 -- 172.21.15.40:0/3023311 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f7944059690 con 0x7f794c163aa0 2024-04-26T21:21:11.185 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 msgr2=0x7f793006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 0x7f793006edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ba0 tx=0x7f7940008040 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 msgr2=0x7f794c15d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f794400cb40 tx=0x7f794400a060 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 shutdown_connections 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f793006c8f0 0x7f793006edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f794c163aa0 0x7f794c15d750 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f794c15f680 0x7f794c162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 --2- 172.21.15.40:0/3023311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f794c0b60b0 0x7f794c14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 >> 172.21.15.40:0/3023311 conn(0x7f794c08eca0 msgr2=0x7f794c163310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 shutdown_connections 2024-04-26T21:21:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.187+0000 7f7952c1d640 1 -- 172.21.15.40:0/3023311 wait complete. 2024-04-26T21:21:11.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:21:11.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.299+0000 7f0e29a52640 1 Processor -- start 2024-04-26T21:21:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 -- start start 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e2406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0e240d11b0 0x7f0e2406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e240d1740 con 0x7f0e2406e2c0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2406e750 con 0x7f0e240d11b0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e29a52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240d1a20 con 0x7f0e240cf2e0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e237fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e2406e2c0 0x7f0e2415ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38546/0 (socket says 172.21.15.40:38546) 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.300+0000 7f0e237fe640 1 -- 172.21.15.40:0/1440087680 learned_addr learned my addr 172.21.15.40:0/1440087680 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877741182 0 0) 0x7f0e240d1740 con 0x7f0e2406e2c0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e22ffd640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e2406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 --2- 172.21.15.40:0/1440087680 >> v2:172.21.15.184:3300/0 conn(0x7f0e240d11b0 0x7f0e2406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e0c003520 con 0x7f0e2406e2c0 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 632468874 0 0) 0x7f0e0c003520 con 0x7f0e2406e2c0 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 msgr2=0x7f0e2406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e2406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 >> v2:172.21.15.184:3300/0 conn(0x7f0e240d11b0 msgr2=0x7f0e2406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 --2- 172.21.15.40:0/1440087680 >> v2:172.21.15.184:3300/0 conn(0x7f0e240d11b0 0x7f0e2406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e240d1d00 con 0x7f0e2406e2c0 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f0e1c0019f0 con 0x7f0e2406e2c0 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 >> v1:172.21.15.40:6789/0 conn(0x7f0e2406e2c0 legacy=0x7f0e2415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0c003f30 0x7f0e0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e0c004d40 0x7f0e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2406e750 con 0x7f0e0c003f30 2024-04-26T21:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e240d1a20 con 0x7f0e0c004d40 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c00b9c0 con 0x7f0e240cf2e0 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0c003f30 0x7f0e0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e22ffd640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e237fe640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e0c004d40 0x7f0e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 -- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 msgr2=0x7f0e0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e240cf2e0 0x7f0e0c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 -- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e0c004d40 msgr2=0x7f0e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.301+0000 7f0e227fc640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e0c004d40 0x7f0e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e227fc640 1 -- 172.21.15.40:0/1440087680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c003520 con 0x7f0e0c003f30 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e237fe640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e0c004d40 0x7f0e0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e227fc640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0c003f30 0x7f0e0c004800 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009ac0 tx=0x7f0e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fe60d4b315b534 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18036070 con 0x7f0e0c003f30 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18031400 con 0x7f0e0c003f30 2024-04-26T21:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.302+0000 7f0e21ffb640 1 -- 172.21.15.40:0/1440087680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18009690 con 0x7f0e0c003f30 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 -- 172.21.15.40:0/1440087680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0c003f30 msgr2=0x7f0e0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 --2- 172.21.15.40:0/1440087680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0c003f30 0x7f0e0c004800 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009ac0 tx=0x7f0e0c005830 comp rx=0 tx=0).stop 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e237fe640 1 -- 172.21.15.40:0/1440087680 reap_dead start 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 -- 172.21.15.40:0/1440087680 shutdown_connections 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 -- 172.21.15.40:0/1440087680 >> 172.21.15.40:0/1440087680 conn(0x7f0e240b4a60 msgr2=0x7f0e240b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 -- 172.21.15.40:0/1440087680 shutdown_connections 2024-04-26T21:21:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.303+0000 7f0e29a52640 1 -- 172.21.15.40:0/1440087680 wait complete. 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.304+0000 7f0e29a52640 1 Processor -- start 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.304+0000 7f0e29a52640 1 -- start start 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0c003f30 0x7f0e241521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e240c81a0 0x7f0e240c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0e240cf6c0 con 0x7f0e2409fa80 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e240d22c0 con 0x7f0e240c81a0 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e29a52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240cf840 con 0x7f0e0c003f30 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48946/0 (socket says 172.21.15.40:48946) 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 -- 172.21.15.40:0/3607541859 learned_addr learned my addr 172.21.15.40:0/3607541859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e22ffd640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0c003f30 0x7f0e241521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e237fe640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e240c81a0 0x7f0e240c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 -- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0c003f30 msgr2=0x7f0e241521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0c003f30 0x7f0e241521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 -- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e240c81a0 msgr2=0x7f0e240c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e240c81a0 0x7f0e240c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.305+0000 7f0e227fc640 1 -- 172.21.15.40:0/3607541859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e24152a90 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e227fc640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0e18034040 tx=0x7f0e18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18040020 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18003c60 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18036040 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e24152d70 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.306+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e24153330 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.307+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e240c8b60 con 0x7f0e2409fa80 2024-04-26T21:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.307+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e18043020 con 0x7f0e2409fa80 2024-04-26T21:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.308+0000 7f0e0bfff640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 0x7f0e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.308+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f0e180abde0 con 0x7f0e2409fa80 2024-04-26T21:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.308+0000 7f0e22ffd640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 0x7f0e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.309+0000 7f0e22ffd640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 0x7f0e1006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009900 tx=0x7f0e14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.312+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e1804a070 con 0x7f0e2409fa80 2024-04-26T21:21:11.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.507+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e240b8ef0 con 0x7f0e2409fa80 2024-04-26T21:21:11.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.508+0000 7f0e0bfff640 1 -- 172.21.15.40:0/3607541859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f0e180044e0 con 0x7f0e2409fa80 2024-04-26T21:21:11.509 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 msgr2=0x7f0e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 0x7f0e1006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009900 tx=0x7f0e14008040 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 msgr2=0x7f0e24155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0e18034040 tx=0x7f0e18004730 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 shutdown_connections 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0e1006c8f0 0x7f0e1006edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e240c81a0 0x7f0e240c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e2409fa80 0x7f0e24155f80 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 --2- 172.21.15.40:0/3607541859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0c003f30 0x7f0e241521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 >> 172.21.15.40:0/3607541859 conn(0x7f0e240b4a60 msgr2=0x7f0e2409ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:11.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 shutdown_connections 2024-04-26T21:21:11.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.510+0000 7f0e29a52640 1 -- 172.21.15.40:0/3607541859 wait complete. 2024-04-26T21:21:11.521 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:21:11.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:11.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:11.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:21:11.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:11.606 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:21:11.621 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:11.621 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:11.621 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:11.638 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:11.638 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:11.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.795+0000 7f86a1d0a640 1 Processor -- start 2024-04-26T21:21:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.795+0000 7f86a1d0a640 1 -- start start 2024-04-26T21:21:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f86a1d0a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f869c0cf2e0 0x7f869c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f86a1d0a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f869c0d1740 con 0x7f869c0d11b0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f86a1d0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f869c0d1a20 con 0x7f869c06e490 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f86a1d0a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f869c0d1d00 con 0x7f869c0cf2e0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f869c0cf2e0 0x7f869c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f869c0cf2e0 0x7f869c06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48956/0 (socket says 172.21.15.40:48956) 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 -- 172.21.15.40:0/1705093650 learned_addr learned my addr 172.21.15.40:0/1705093650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869affd640 1 --1- 172.21.15.40:0/1705093650 >> v1:172.21.15.40:6789/0 conn(0x7f869c0d11b0 0x7f869c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38562/0 (socket says 172.21.15.40:38562) 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 -- 172.21.15.40:0/1705093650 >> v1:172.21.15.184:6789/0 conn(0x7f869c06e490 legacy=0x7f869c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 -- 172.21.15.40:0/1705093650 >> v1:172.21.15.40:6789/0 conn(0x7f869c0d11b0 legacy=0x7f869c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.796+0000 7f869b7fe640 1 -- 172.21.15.40:0/1705093650 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c0d1fe0 con 0x7f869c0cf2e0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f869b7fe640 1 --2- 172.21.15.40:0/1705093650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c0cf2e0 0x7f869c06d4d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f868c0098a0 tx=0x7f868c02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85917e1e0fc579d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f869a7fc640 1 -- 172.21.15.40:0/1705093650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c037070 con 0x7f869c0cf2e0 2024-04-26T21:21:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f869a7fc640 1 -- 172.21.15.40:0/1705093650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f868c032440 con 0x7f869c0cf2e0 2024-04-26T21:21:11.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f869a7fc640 1 -- 172.21.15.40:0/1705093650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c032810 con 0x7f869c0cf2e0 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f86a1d0a640 1 -- 172.21.15.40:0/1705093650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c0cf2e0 msgr2=0x7f869c06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.797+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/1705093650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c0cf2e0 0x7f869c06d4d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f868c0098a0 tx=0x7f868c02a2e0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.798+0000 7f86a1d0a640 1 -- 172.21.15.40:0/1705093650 shutdown_connections 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.798+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/1705093650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c0cf2e0 0x7f869c06d4d0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.798+0000 7f86a1d0a640 1 -- 172.21.15.40:0/1705093650 >> 172.21.15.40:0/1705093650 conn(0x7f869c0b4a60 msgr2=0x7f869c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.798+0000 7f86a1d0a640 1 -- 172.21.15.40:0/1705093650 shutdown_connections 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.798+0000 7f86a1d0a640 1 -- 172.21.15.40:0/1705093650 wait complete. 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.799+0000 7f86a1d0a640 1 Processor -- start 2024-04-26T21:21:11.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.799+0000 7f86a1d0a640 1 -- start start 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c0cf2e0 0x7f869c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 0x7f869c1e9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f869c0cf6c0 con 0x7f869c06e490 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f869c06da10 con 0x7f869c0d11b0 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f869c0cf840 con 0x7f869c0cf2e0 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48960/0 (socket says 172.21.15.40:48960) 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 -- 172.21.15.40:0/3107141242 learned_addr learned my addr 172.21.15.40:0/3107141242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:11.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 -- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c0cf2e0 msgr2=0x7f869c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c0cf2e0 0x7f869c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 -- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 msgr2=0x7f869c1e9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869bfff640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 0x7f869c1e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 0x7f869c1e9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 -- 172.21.15.40:0/3107141242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c1f58a0 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869bfff640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 0x7f869c1e9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f869b7fe640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f868c000c00 tx=0x7f868c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c037040 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f868c004230 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869c1ea140 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c03c670 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.800+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869c0c6d60 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.802+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869c06cdd0 con 0x7f869c06e490 2024-04-26T21:21:11.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.802+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f868c03a070 con 0x7f869c06e490 2024-04-26T21:21:11.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.802+0000 7f8683fff640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 0x7f868806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:11.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.802+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f868c0ac320 con 0x7f869c06e490 2024-04-26T21:21:11.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.803+0000 7f869affd640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 0x7f868806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:11.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.803+0000 7f869affd640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 0x7f868806edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8690000f60 tx=0x7f8690007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:11.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.807+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f869c06cdd0 con 0x7f869c06e490 2024-04-26T21:21:11.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.992+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f869c0b8ef0 con 0x7f869c06e490 2024-04-26T21:21:11.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.993+0000 7f8683fff640 1 -- 172.21.15.40:0/3107141242 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7f868c0a9140 con 0x7f869c06e490 2024-04-26T21:21:11.995 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:11.995 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 msgr2=0x7f868806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 0x7f868806edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8690000f60 tx=0x7f8690007a00 comp rx=0 tx=0).stop 2024-04-26T21:21:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 msgr2=0x7f869c1f5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f868c000c00 tx=0x7f868c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 shutdown_connections 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f868806c8f0 0x7f868806edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f869c0d11b0 0x7f869c1e9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f869c0cf2e0 0x7f869c1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 --2- 172.21.15.40:0/3107141242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f869c06e490 0x7f869c1f5360 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 >> 172.21.15.40:0/3107141242 conn(0x7f869c0b4a60 msgr2=0x7f869c070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 shutdown_connections 2024-04-26T21:21:11.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:11.996+0000 7f86a1d0a640 1 -- 172.21.15.40:0/3107141242 wait complete. 2024-04-26T21:21:12.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:20:43.597608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:20:44.259933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:12.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:12.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.112+0000 7f1f9b3bf640 1 Processor -- start 2024-04-26T21:21:12.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.112+0000 7f1f9b3bf640 1 -- start start 2024-04-26T21:21:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9406f340 0x7f1f940b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f940d0300 0x7f1f9406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f940b3e50 0x7f1f940720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9406fc60 con 0x7f1f940d0300 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1f940b46a0 con 0x7f1f940b3e50 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9b3bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940b5db0 con 0x7f1f9406f340 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f940d0300 0x7f1f9406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f940d0300 0x7f1f9406f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48974/0 (socket says 172.21.15.40:48974) 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 -- 172.21.15.40:0/91036427 learned_addr learned my addr 172.21.15.40:0/91036427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9995c640 1 --2- 172.21.15.40:0/91036427 >> v2:172.21.15.184:3300/0 conn(0x7f1f940b3e50 0x7f1f940720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9915b640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9406f340 0x7f1f940b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 -- 172.21.15.40:0/91036427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9406f340 msgr2=0x7f1f940b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9406f340 0x7f1f940b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 -- 172.21.15.40:0/91036427 >> v2:172.21.15.184:3300/0 conn(0x7f1f940b3e50 msgr2=0x7f1f940720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 --2- 172.21.15.40:0/91036427 >> v2:172.21.15.184:3300/0 conn(0x7f1f940b3e50 0x7f1f940720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.113+0000 7f1f9895a640 1 -- 172.21.15.40:0/91036427 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f940b6090 con 0x7f1f940d0300 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.114+0000 7f1f9895a640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f9406f720 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1f94072080 tx=0x7f1f8402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eb4a0ac6e9d100 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.114+0000 7f1f8bfff640 1 -- 172.21.15.40:0/91036427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84037070 con 0x7f1f940d0300 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.114+0000 7f1f8bfff640 1 -- 172.21.15.40:0/91036427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8402ae00 con 0x7f1f940d0300 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.114+0000 7f1f8bfff640 1 -- 172.21.15.40:0/91036427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84032660 con 0x7f1f940d0300 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/91036427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 msgr2=0x7f1f9406f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f9406f720 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1f94072080 tx=0x7f1f8402a0b0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/91036427 shutdown_connections 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/91036427 >> v2:172.21.15.184:3300/0 conn(0x7f1f940b3e50 0x7f1f940720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f9406f720 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/91036427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9406f340 0x7f1f940b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/91036427 >> 172.21.15.40:0/91036427 conn(0x7f1f940cd8a0 msgr2=0x7f1f940cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/91036427 shutdown_connections 2024-04-26T21:21:12.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.115+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/91036427 wait complete. 2024-04-26T21:21:12.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.116+0000 7f1f9b3bf640 1 Processor -- start 2024-04-26T21:21:12.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.116+0000 7f1f9b3bf640 1 -- start start 2024-04-26T21:21:12.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 0x7f1f9415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 0x7f1f94139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940b42e0 con 0x7f1f940d0300 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940b6650 con 0x7f1f9406f340 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9b3bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940b4460 con 0x7f1f940b3e50 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48976/0 (socket says 172.21.15.40:48976) 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 -- 172.21.15.40:0/2260931836 learned_addr learned my addr 172.21.15.40:0/2260931836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9895a640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 0x7f1f94139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9915b640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 0x7f1f9415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 -- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 msgr2=0x7f1f94139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 0x7f1f94139da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 -- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 msgr2=0x7f1f9415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 0x7f1f9415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 -- 172.21.15.40:0/2260931836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9413a710 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9895a640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 0x7f1f94139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9915b640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 0x7f1f9415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.117+0000 7f1f9995c640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00cc30 tx=0x7f1f8c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.118+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00ce40 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.118+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c003990 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.118+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9413a9f0 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.118+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c003d40 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.118+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9413afb0 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.119+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9406a290 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.119+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f8c004000 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.120+0000 7f1f89ffb640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 0x7f1f7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.120+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f1f8c08bfc0 con 0x7f1f940d0300 2024-04-26T21:21:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.120+0000 7f1f9915b640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 0x7f1f7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.120+0000 7f1f9915b640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 0x7f1f7806edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006fd0 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.124+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f8c014030 con 0x7f1f940d0300 2024-04-26T21:21:12.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.309+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f94140680 con 0x7f1f940d0300 2024-04-26T21:21:12.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.310+0000 7f1f89ffb640 1 -- 172.21.15.40:0/2260931836 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f1f8c05a3f0 con 0x7f1f940d0300 2024-04-26T21:21:12.311 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 msgr2=0x7f1f7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 0x7f1f7806edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006fd0 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 msgr2=0x7f1f94136bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00cc30 tx=0x7f1f8c00a5c0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 shutdown_connections 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1f7806c8f0 0x7f1f7806edb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f940d0300 0x7f1f94136bb0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f940b3e50 0x7f1f94139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 --2- 172.21.15.40:0/2260931836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9406f340 0x7f1f9415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 >> 172.21.15.40:0/2260931836 conn(0x7f1f940cd8a0 msgr2=0x7f1f940b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 shutdown_connections 2024-04-26T21:21:12.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.312+0000 7f1f9b3bf640 1 -- 172.21.15.40:0/2260931836 wait complete. 2024-04-26T21:21:12.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.427+0000 7f81e4138640 1 Processor -- start 2024-04-26T21:21:12.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.427+0000 7f81e4138640 1 -- start start 2024-04-26T21:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.427+0000 7f81e4138640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81dc15f490 0x7f81dc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e4138640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 0x7f81dc0ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e4138640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81dc0d4240 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e4138640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc0d31a0 con 0x7f81dc15f490 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e4138640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0d3480 con 0x7f81dc0d22f0 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81dc15f490 0x7f81dc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81dc15f490 0x7f81dc0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48984/0 (socket says 172.21.15.40:48984) 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 -- 172.21.15.40:0/2423450464 learned_addr learned my addr 172.21.15.40:0/2423450464 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e26d5640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 0x7f81dc0ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 -- 172.21.15.40:0/2423450464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 msgr2=0x7f81dc0ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 0x7f81dc0ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 -- 172.21.15.40:0/2423450464 >> v1:172.21.15.184:6789/0 conn(0x7f81dc0d0ff0 legacy=0x7f81dc0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e1ed4640 1 -- 172.21.15.40:0/2423450464 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0d3760 con 0x7f81dc15f490 2024-04-26T21:21:12.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.428+0000 7f81e26d5640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 0x7f81dc0ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:12.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e1ed4640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc15f490 0x7f81dc0d0ab0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009e50 tx=0x7f81d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb32a3430647556 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e0ed2640 1 -- 172.21.15.40:0/2423450464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4036070 con 0x7f81dc15f490 2024-04-26T21:21:12.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e0ed2640 1 -- 172.21.15.40:0/2423450464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d4031600 con 0x7f81dc15f490 2024-04-26T21:21:12.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e0ed2640 1 -- 172.21.15.40:0/2423450464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40319d0 con 0x7f81dc15f490 2024-04-26T21:21:12.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e4138640 1 -- 172.21.15.40:0/2423450464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc15f490 msgr2=0x7f81dc0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e4138640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc15f490 0x7f81dc0d0ab0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009e50 tx=0x7f81d402a710 comp rx=0 tx=0).stop 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e4138640 1 -- 172.21.15.40:0/2423450464 shutdown_connections 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e4138640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc0d22f0 0x7f81dc0ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.429+0000 7f81e4138640 1 --2- 172.21.15.40:0/2423450464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc15f490 0x7f81dc0d0ab0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.430+0000 7f81e4138640 1 -- 172.21.15.40:0/2423450464 >> 172.21.15.40:0/2423450464 conn(0x7f81dc09e530 msgr2=0x7f81dc0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.430+0000 7f81e4138640 1 -- 172.21.15.40:0/2423450464 shutdown_connections 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.430+0000 7f81e4138640 1 -- 172.21.15.40:0/2423450464 wait complete. 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.431+0000 7f81e4138640 1 Processor -- start 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.431+0000 7f81e4138640 1 -- start start 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81dc0d22f0 0x7f81dc14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc15f490 0x7f81dc1504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc15f870 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0d3d20 con 0x7f81dc0d22f0 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2dc0 con 0x7f81dc15f490 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48992/0 (socket says 172.21.15.40:48992) 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 -- 172.21.15.40:0/1244360932 learned_addr learned my addr 172.21.15.40:0/1244360932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e16d3640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81dc0d22f0 0x7f81dc14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e26d5640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc15f490 0x7f81dc1504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 -- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc15f490 msgr2=0x7f81dc1504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc15f490 0x7f81dc1504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 -- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81dc0d22f0 msgr2=0x7f81dc14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81dc0d22f0 0x7f81dc14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 -- 172.21.15.40:0/1244360932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0ba920 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e1ed4640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f81d4000f80 tx=0x7f81d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4036050 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d4042070 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc0bac00 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4009510 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.432+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1f91a0 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.433+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc069ae0 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.434+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81d4007c70 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.434+0000 7f81da7fc640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 0x7f81c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.434+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f81d40ac9a0 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.434+0000 7f81e16d3640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 0x7f81c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.435+0000 7f81e16d3640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 0x7f81c806ede0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1f8e90 tx=0x7f81d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.439+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81dc069ae0 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.622+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81dc0d5900 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.622+0000 7f81da7fc640 1 -- 172.21.15.40:0/1244360932 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f81d407add0 con 0x7f81dc0d0ff0 2024-04-26T21:21:12.623 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:12.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 msgr2=0x7f81c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 0x7f81c806ede0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1f8e90 tx=0x7f81d0009040 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 msgr2=0x7f81dc0b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f81d4000f80 tx=0x7f81d4004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 shutdown_connections 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c806c920 0x7f81c806ede0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81dc15f490 0x7f81dc1504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81dc0d22f0 0x7f81dc14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 --2- 172.21.15.40:0/1244360932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81dc0d0ff0 0x7f81dc0b7e10 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 >> 172.21.15.40:0/1244360932 conn(0x7f81dc09e530 msgr2=0x7f81dc15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 shutdown_connections 2024-04-26T21:21:12.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.624+0000 7f81e4138640 1 -- 172.21.15.40:0/1244360932 wait complete. 2024-04-26T21:21:12.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.738+0000 7fc13896a640 1 Processor -- start 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.738+0000 7fc13896a640 1 -- start start 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.739+0000 7fc13896a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 0x7fc1340d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.739+0000 7fc13896a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 0x7fc1340d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.739+0000 7fc13896a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1340d4240 con 0x7fc1340d2120 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.739+0000 7fc13896a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1340d2570 con 0x7fc1340d0ff0 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.739+0000 7fc13896a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1340d31a0 con 0x7fc13415f490 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc132d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1340d2120 0x7fc1340ba3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38572/0 (socket says 172.21.15.40:38572) 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc132d9d640 1 -- 172.21.15.40:0/1044261617 learned_addr learned my addr 172.21.15.40:0/1044261617 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13259c640 1 --2- 172.21.15.40:0/1044261617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 0x7fc1340d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1044261617 >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 0x7fc1340d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224951873 0 0) 0x7fc1340d4240 con 0x7fc1340d2120 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc11c003520 con 0x7fc1340d2120 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254076190 0 0) 0x7fc11c003520 con 0x7fc1340d2120 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13259c640 1 -- 172.21.15.40:0/1044261617 >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 msgr2=0x7fc1340d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13259c640 1 --2- 172.21.15.40:0/1044261617 >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 0x7fc1340d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13259c640 1 -- 172.21.15.40:0/1044261617 >> v1:172.21.15.40:6789/0 conn(0x7fc1340d2120 legacy=0x7fc1340ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc13259c640 1 -- 172.21.15.40:0/1044261617 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340d3480 con 0x7fc13415f490 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.740+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1044261617 >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 0x7fc1340d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.741+0000 7fc13259c640 1 --2- 172.21.15.40:0/1044261617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 0x7fc1340d0ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1240097a0 tx=0x7fc12402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480aa546c5a82a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.741+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037070 con 0x7fc13415f490 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.741+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12402aba0 con 0x7fc13415f490 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.741+0000 7fc13159a640 1 -- 172.21.15.40:0/1044261617 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0019f0 con 0x7fc13415f490 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 -- 172.21.15.40:0/1044261617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 msgr2=0x7fc1340d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 --2- 172.21.15.40:0/1044261617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 0x7fc1340d0ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1240097a0 tx=0x7fc12402a110 comp rx=0 tx=0).stop 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 -- 172.21.15.40:0/1044261617 shutdown_connections 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 --2- 172.21.15.40:0/1044261617 >> v2:172.21.15.184:3300/0 conn(0x7fc1340d0ff0 0x7fc1340d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 --2- 172.21.15.40:0/1044261617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13415f490 0x7fc1340d0ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 -- 172.21.15.40:0/1044261617 >> 172.21.15.40:0/1044261617 conn(0x7fc13409e530 msgr2=0x7fc1340d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 -- 172.21.15.40:0/1044261617 shutdown_connections 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.742+0000 7fc13896a640 1 -- 172.21.15.40:0/1044261617 wait complete. 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.743+0000 7fc13896a640 1 Processor -- start 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.743+0000 7fc13896a640 1 -- start start 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 0x7fc13406e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415f490 0x7fc13406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340d3d20 con 0x7fc1340d0ff0 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340d3a40 con 0x7fc13415f490 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13896a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc13415f870 con 0x7fc1340d2120 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48996/0 (socket says 172.21.15.40:48996) 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 -- 172.21.15.40:0/1602714727 learned_addr learned my addr 172.21.15.40:0/1602714727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 0x7fc13406e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc132d9d640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415f490 0x7fc13406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 -- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 msgr2=0x7fc13406e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 0x7fc13406e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 -- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415f490 msgr2=0x7fc13406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415f490 0x7fc13406e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc13259c640 1 -- 172.21.15.40:0/1602714727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340cd090 con 0x7fc1340d0ff0 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.744+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 0x7fc13406e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc13259c640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fc124035040 tx=0x7fc1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124041070 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12403c660 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1340cd370 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037040 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.745+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1340c9c20 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.746+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13415d750 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.746+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc124044070 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.747+0000 7fc122ffd640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 0x7fc11406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.747+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fc1240ac430 con 0x7fc1340d0ff0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.747+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 0x7fc11406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.747+0000 7fc131d9b640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 0x7fc11406edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc1340a2500 tx=0x7fc128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.751+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc13415d750 con 0x7fc1340d0ff0 2024-04-26T21:21:12.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.939+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc13406a290 con 0x7fc1340d0ff0 2024-04-26T21:21:12.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.939+0000 7fc122ffd640 1 -- 172.21.15.40:0/1602714727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7fc12407a980 con 0x7fc1340d0ff0 2024-04-26T21:21:12.940 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 msgr2=0x7fc11406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 0x7fc11406edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc1340a2500 tx=0x7fc128008040 comp rx=0 tx=0).stop 2024-04-26T21:21:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 msgr2=0x7fc134155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fc124035040 tx=0x7fc1240046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 shutdown_connections 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc11406c8f0 0x7fc11406edb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc13415f490 0x7fc13406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1340d2120 0x7fc13406e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 --2- 172.21.15.40:0/1602714727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1340d0ff0 0x7fc134155ab0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 >> 172.21.15.40:0/1602714727 conn(0x7fc13409e530 msgr2=0x7fc13415e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 shutdown_connections 2024-04-26T21:21:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:12.941+0000 7fc13896a640 1 -- 172.21.15.40:0/1602714727 wait complete. 2024-04-26T21:21:12.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:21:13.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.053+0000 7fb12a919640 1 Processor -- start 2024-04-26T21:21:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb12a919640 1 -- start start 2024-04-26T21:21:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb12a919640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1240d3ee0 0x7fb1240cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb12a919640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1240d03a0 con 0x7fb12415f370 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb12a919640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1240d0680 con 0x7fb12415f750 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb12a919640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb1240d0960 con 0x7fb1240d3ee0 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.054+0000 7fb1237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1240d3ee0 0x7fb1240cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb1237fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1240d3ee0 0x7fb1240cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49008/0 (socket says 172.21.15.40:49008) 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb1237fe640 1 -- 172.21.15.40:0/1157759460 learned_addr learned my addr 172.21.15.40:0/1157759460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb123fff640 1 --1- 172.21.15.40:0/1157759460 >> v1:172.21.15.40:6789/0 conn(0x7fb12415f370 0x7fb1240d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38582/0 (socket says 172.21.15.40:38582) 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb1237fe640 1 -- 172.21.15.40:0/1157759460 >> v1:172.21.15.184:6789/0 conn(0x7fb12415f750 legacy=0x7fb1240b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb1237fe640 1 -- 172.21.15.40:0/1157759460 >> v1:172.21.15.40:6789/0 conn(0x7fb12415f370 legacy=0x7fb1240d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb1237fe640 1 -- 172.21.15.40:0/1157759460 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240d0c40 con 0x7fb1240d3ee0 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.055+0000 7fb122ffd640 1 -- 172.21.15.40:0/1157759460 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2274829470 0 0) 0x7fb1240d03a0 con 0x7fb12415f370 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb1237fe640 1 --2- 172.21.15.40:0/1157759460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240cfd70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098a0 tx=0x7fb11802a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b697c2aa71b5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb122ffd640 1 -- 172.21.15.40:0/1157759460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118037070 con 0x7fb1240d3ee0 2024-04-26T21:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb122ffd640 1 -- 172.21.15.40:0/1157759460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb118032640 con 0x7fb1240d3ee0 2024-04-26T21:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb122ffd640 1 -- 172.21.15.40:0/1157759460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118032a10 con 0x7fb1240d3ee0 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 -- 172.21.15.40:0/1157759460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 msgr2=0x7fb1240cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 --2- 172.21.15.40:0/1157759460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240cfd70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098a0 tx=0x7fb11802a320 comp rx=0 tx=0).stop 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 -- 172.21.15.40:0/1157759460 shutdown_connections 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 --2- 172.21.15.40:0/1157759460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240cfd70 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 -- 172.21.15.40:0/1157759460 >> 172.21.15.40:0/1157759460 conn(0x7fb12408ec10 msgr2=0x7fb12409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 -- 172.21.15.40:0/1157759460 shutdown_connections 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.056+0000 7fb12a919640 1 -- 172.21.15.40:0/1157759460 wait complete. 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.057+0000 7fb12a919640 1 Processor -- start 2024-04-26T21:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 -- start start 2024-04-26T21:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb12415f370 0x7fb12414efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb12415f750 0x7fb1240cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240b4950 con 0x7fb1240d3ee0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240b47d0 con 0x7fb12415f750 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb12a919640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240b4ad0 con 0x7fb12415f370 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb123fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb123fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49020/0 (socket says 172.21.15.40:49020) 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.058+0000 7fb123fff640 1 -- 172.21.15.40:0/4240804533 learned_addr learned my addr 172.21.15.40:0/4240804533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb128eb6640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb12415f750 0x7fb1240cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb1237fe640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb12415f370 0x7fb12414efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 -- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb12415f370 msgr2=0x7fb12414efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb12415f370 0x7fb12414efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 -- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb12415f750 msgr2=0x7fb1240cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb12415f750 0x7fb1240cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 -- 172.21.15.40:0/4240804533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240ce0f0 con 0x7fb1240d3ee0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb123fff640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c4d0 tx=0x7fb11400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114018070 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1140074e0 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114004a20 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1240ce370 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.059+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1240ce930 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.060+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12406a290 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.061+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb11401b070 con 0x7fb1240d3ee0 2024-04-26T21:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.061+0000 7fb120ff9640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.061+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fb11408ade0 con 0x7fb1240d3ee0 2024-04-26T21:21:13.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.061+0000 7fb1237fe640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.062+0000 7fb1237fe640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 0x7fb11006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb118034000 tx=0x7fb11802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.066+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1240ce0f0 con 0x7fb1240d3ee0 2024-04-26T21:21:13.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.261+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1240b6650 con 0x7fb1240d3ee0 2024-04-26T21:21:13.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.262+0000 7fb120ff9640 1 -- 172.21.15.40:0/4240804533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7fb1140592a0 con 0x7fb1240d3ee0 2024-04-26T21:21:13.263 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 msgr2=0x7fb11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 0x7fb11006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb118034000 tx=0x7fb11802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 msgr2=0x7fb1240c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c4d0 tx=0x7fb11400c990 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 shutdown_connections 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb12415f750 0x7fb1240cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb12415f370 0x7fb12414efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 --2- 172.21.15.40:0/4240804533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1240d3ee0 0x7fb1240c9510 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.263+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 >> 172.21.15.40:0/4240804533 conn(0x7fb12408ec10 msgr2=0x7fb1240d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.264+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 shutdown_connections 2024-04-26T21:21:13.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.264+0000 7fb12a919640 1 -- 172.21.15.40:0/4240804533 wait complete. 2024-04-26T21:21:13.275 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:21:13.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:13.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:13.308 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:21:13.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:13.308 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:21:13.366 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:13.366 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:13.366 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:13.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:13.382 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:13.382 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:13.382 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:13.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:13.425 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 Processor -- start 2024-04-26T21:21:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 -- start start 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 0x7fb9b40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 0x7fb9b40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b40d03a0 con 0x7fb9b415f750 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b40b4d10 con 0x7fb9b415f370 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.525+0000 7fb9b9cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b40d0680 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb9b415f750 0x7fb9b40d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38596/0 (socket says 172.21.15.40:38596) 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b2ffd640 1 -- 172.21.15.40:0/3217136043 learned_addr learned my addr 172.21.15.40:0/3217136043 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b3fff640 1 --2- 172.21.15.40:0/3217136043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 0x7fb9b40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b37fe640 1 --2- 172.21.15.40:0/3217136043 >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 0x7fb9b40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894741235 0 0) 0x7fb9b40d03a0 con 0x7fb9b415f750 2024-04-26T21:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9a0003520 con 0x7fb9b415f750 2024-04-26T21:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260785009 0 0) 0x7fb9a0003520 con 0x7fb9b415f750 2024-04-26T21:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b3fff640 1 -- 172.21.15.40:0/3217136043 >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 msgr2=0x7fb9b40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b3fff640 1 --2- 172.21.15.40:0/3217136043 >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 0x7fb9b40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b3fff640 1 -- 172.21.15.40:0/3217136043 >> v1:172.21.15.40:6789/0 conn(0x7fb9b415f750 legacy=0x7fb9b40d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b3fff640 1 -- 172.21.15.40:0/3217136043 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b40d0960 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.526+0000 7fb9b37fe640 1 --2- 172.21.15.40:0/3217136043 >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 0x7fb9b40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.527+0000 7fb9b3fff640 1 --2- 172.21.15.40:0/3217136043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 0x7fb9b40b47d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009d90 tx=0x7fb9ac02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f958f4109a56dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.527+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c001670 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.527+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac007540 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.527+0000 7fb9b27fc640 1 -- 172.21.15.40:0/3217136043 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac007910 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/3217136043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 msgr2=0x7fb9b40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/3217136043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 0x7fb9b40b47d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009d90 tx=0x7fb9ac02a5f0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/3217136043 shutdown_connections 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/3217136043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b40d3ee0 0x7fb9b40b47d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/3217136043 >> v2:172.21.15.184:3300/0 conn(0x7fb9b415f370 0x7fb9b40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/3217136043 >> 172.21.15.40:0/3217136043 conn(0x7fb9b408ec10 msgr2=0x7fb9b409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/3217136043 shutdown_connections 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.528+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/3217136043 wait complete. 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.529+0000 7fb9b9cb0640 1 Processor -- start 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.529+0000 7fb9b9cb0640 1 -- start start 2024-04-26T21:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b415f370 0x7fb9b414b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b415f750 0x7fb9b414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b40b51c0 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b40d0f20 con 0x7fb9b415f750 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b9cb0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b40b5340 con 0x7fb9b415f370 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49030/0 (socket says 172.21.15.40:49030) 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 -- 172.21.15.40:0/2730368035 learned_addr learned my addr 172.21.15.40:0/2730368035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b3fff640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b415f750 0x7fb9b414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b2ffd640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b415f370 0x7fb9b414b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 -- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b415f370 msgr2=0x7fb9b414b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b415f370 0x7fb9b414b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 -- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b415f750 msgr2=0x7fb9b414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b415f750 0x7fb9b414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.530+0000 7fb9b37fe640 1 -- 172.21.15.40:0/2730368035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b414f220 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9b37fe640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ec10 tx=0x7fb9a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4017070 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a400cbd0 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4013470 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b40cae30 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.531+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b40cb3f0 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.532+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b406a290 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.532+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9a401a070 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.533+0000 7fb9abfff640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.533+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fb9a408be80 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.533+0000 7fb9b2ffd640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.533+0000 7fb9b2ffd640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 0x7fb99406edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb99c001eb0 tx=0x7fb99c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.537+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9a405a210 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.761+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b40695b0 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.761+0000 7fb9abfff640 1 -- 172.21.15.40:0/2730368035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5191 (secure 0 0 0) 0x7fb9a4059b10 con 0x7fb9b40d3ee0 2024-04-26T21:21:13.763 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:13.763 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:21:05.985678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 msgr2=0x7fb99406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 0x7fb99406edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb99c001eb0 tx=0x7fb99c009040 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 msgr2=0x7fb9b40b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ec10 tx=0x7fb9a400c520 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 shutdown_connections 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9b415f750 0x7fb9b414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9b415f370 0x7fb9b414b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 --2- 172.21.15.40:0/2730368035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b40d3ee0 0x7fb9b40b9600 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.763+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 >> 172.21.15.40:0/2730368035 conn(0x7fb9b408ec10 msgr2=0x7fb9b40b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.764+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 shutdown_connections 2024-04-26T21:21:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.764+0000 7fb9b9cb0640 1 -- 172.21.15.40:0/2730368035 wait complete. 2024-04-26T21:21:13.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:13.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.875+0000 7f96b0f24640 1 Processor -- start 2024-04-26T21:21:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.875+0000 7f96b0f24640 1 -- start start 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96b0f24640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96ac0d3ee0 0x7f96ac0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96b0f24640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0d03a0 con 0x7f96ac15f370 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96b0f24640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0d0680 con 0x7f96ac15f750 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96b0f24640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96ac0d0960 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96a9d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96ac0d3ee0 0x7f96ac0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96aa59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f96ac15f370 0x7f96ac0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38602/0 (socket says 172.21.15.40:38602) 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.876+0000 7f96aa59c640 1 -- 172.21.15.40:0/1380288610 learned_addr learned my addr 172.21.15.40:0/1380288610 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.877+0000 7f96a9d9b640 1 -- 172.21.15.40:0/1380288610 >> v1:172.21.15.184:6789/0 conn(0x7f96ac15f750 legacy=0x7f96ac0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.877+0000 7f96a9d9b640 1 -- 172.21.15.40:0/1380288610 >> v1:172.21.15.40:6789/0 conn(0x7f96ac15f370 legacy=0x7f96ac0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.877+0000 7f96a9d9b640 1 -- 172.21.15.40:0/1380288610 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0d0c40 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.877+0000 7f96a9d9b640 1 --2- 172.21.15.40:0/1380288610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac0cfd70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097d0 tx=0x7f96a002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37997c439cf6fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96a959a640 1 -- 172.21.15.40:0/1380288610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037070 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96a959a640 1 -- 172.21.15.40:0/1380288610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a002ad90 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96a959a640 1 -- 172.21.15.40:0/1380288610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0032660 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 -- 172.21.15.40:0/1380288610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 msgr2=0x7f96ac0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 --2- 172.21.15.40:0/1380288610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac0cfd70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097d0 tx=0x7f96a002a100 comp rx=0 tx=0).stop 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 -- 172.21.15.40:0/1380288610 shutdown_connections 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 --2- 172.21.15.40:0/1380288610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac0cfd70 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 -- 172.21.15.40:0/1380288610 >> 172.21.15.40:0/1380288610 conn(0x7f96ac08ec10 msgr2=0x7f96ac09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 -- 172.21.15.40:0/1380288610 shutdown_connections 2024-04-26T21:21:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.878+0000 7f96b0f24640 1 -- 172.21.15.40:0/1380288610 wait complete. 2024-04-26T21:21:13.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 Processor -- start 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 -- start start 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ac15f370 0x7f96ac0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96ac15f750 0x7f96ac0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.880+0000 7f96b0f24640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0b4950 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96b0f24640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0b47d0 con 0x7f96ac15f370 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96b0f24640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac0b4ad0 con 0x7f96ac15f750 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49042/0 (socket says 172.21.15.40:49042) 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 -- 172.21.15.40:0/698214243 learned_addr learned my addr 172.21.15.40:0/698214243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96a9d9b640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ac15f370 0x7f96ac0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aad9d640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96ac15f750 0x7f96ac0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 -- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96ac15f750 msgr2=0x7f96ac0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96ac15f750 0x7f96ac0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 -- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ac15f370 msgr2=0x7f96ac0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:13.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ac15f370 0x7f96ac0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:13.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 -- 172.21.15.40:0/698214243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0ce2a0 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96aa59c640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c7b0 tx=0x7f96a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4018070 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a40072f0 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac0ce580 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40047e0 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.881+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac0cdf90 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.882+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac06a290 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.883+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a401b070 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.883+0000 7f969affd640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.883+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f96a408afd0 con 0x7f96ac0d3ee0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.884+0000 7f96a9d9b640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.884+0000 7f96a9d9b640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:13.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:13.887+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96a4058d10 con 0x7f96ac0d3ee0 2024-04-26T21:21:14.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.073+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96ac0ce750 con 0x7f96ac0d3ee0 2024-04-26T21:21:14.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.074+0000 7f969affd640 1 -- 172.21.15.40:0/698214243 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7f96a405d010 con 0x7f96ac0d3ee0 2024-04-26T21:21:14.076 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:14.076 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 msgr2=0x7f968c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 msgr2=0x7f96ac159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c7b0 tx=0x7f96a400cc70 comp rx=0 tx=0).stop 2024-04-26T21:21:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 shutdown_connections 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96ac15f750 0x7f96ac0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ac15f370 0x7f96ac0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 --2- 172.21.15.40:0/698214243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0d3ee0 0x7f96ac159740 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 >> 172.21.15.40:0/698214243 conn(0x7f96ac08ec10 msgr2=0x7f96ac0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 shutdown_connections 2024-04-26T21:21:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.076+0000 7f96b0f24640 1 -- 172.21.15.40:0/698214243 wait complete. 2024-04-26T21:21:14.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:20:43.597608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:20:44.259933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:14.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.191+0000 7f13ebd66640 1 Processor -- start 2024-04-26T21:21:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.191+0000 7f13ebd66640 1 -- start start 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13ebd66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13ebd66640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e409c790 con 0x7f13e40d1170 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13ebd66640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e4163ec0 con 0x7f13e406ea30 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13ebd66640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13e406eec0 con 0x7f13e41631f0 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13e9301640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f13e40d1170 0x7f13e41635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38606/0 (socket says 172.21.15.40:38606) 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13e9301640 1 -- 172.21.15.40:0/2894825018 learned_addr learned my addr 172.21.15.40:0/2894825018 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:14.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.192+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517194128 0 0) 0x7f13e409c790 con 0x7f13e40d1170 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13cc003520 con 0x7f13e40d1170 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936649279 0 0) 0x7f13e4163ec0 con 0x7f13e406ea30 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e409c790 con 0x7f13e406ea30 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3131849038 0 0) 0x7f13cc003520 con 0x7f13e40d1170 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 msgr2=0x7f13e40b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13e40b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 >> v1:172.21.15.184:6789/0 conn(0x7f13e406ea30 legacy=0x7f13e40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e40d1a40 con 0x7f13e40d1170 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f13d8001670 con 0x7f13e40d1170 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 >> v1:172.21.15.40:6789/0 conn(0x7f13e40d1170 legacy=0x7f13e41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc003b70 0x7f13cc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc004800 0x7f13cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13cc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc00baa0 con 0x7f13cc003b70 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc00bc50 con 0x7f13cc004800 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc00be50 con 0x7f13e41631f0 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc003b70 0x7f13cc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13ea303640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13cc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9301640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc004800 0x7f13cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 -- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 msgr2=0x7f13cc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e41631f0 0x7f13cc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 -- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc004800 msgr2=0x7f13cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc004800 0x7f13cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9b02640 1 -- 172.21.15.40:0/2894825018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc003520 con 0x7f13cc003b70 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.193+0000 7f13e9301640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc004800 0x7f13cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.194+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc003b70 0x7f13cc003f70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009fc0 tx=0x7f13cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f8db73483ea2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.194+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4009c10 con 0x7f13cc003b70 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.194+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d4036070 con 0x7f13cc003b70 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.194+0000 7f13e8b00640 1 -- 172.21.15.40:0/2894825018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4031880 con 0x7f13cc003b70 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 -- 172.21.15.40:0/2894825018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc003b70 msgr2=0x7f13cc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2894825018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13cc003b70 0x7f13cc003f70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009fc0 tx=0x7f13cc005830 comp rx=0 tx=0).stop 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ea303640 1 -- 172.21.15.40:0/2894825018 reap_dead start 2024-04-26T21:21:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 -- 172.21.15.40:0/2894825018 shutdown_connections 2024-04-26T21:21:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 -- 172.21.15.40:0/2894825018 >> 172.21.15.40:0/2894825018 conn(0x7f13e409ba40 msgr2=0x7f13e409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 -- 172.21.15.40:0/2894825018 shutdown_connections 2024-04-26T21:21:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.195+0000 7f13ebd66640 1 -- 172.21.15.40:0/2894825018 wait complete. 2024-04-26T21:21:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.196+0000 7f13ebd66640 1 Processor -- start 2024-04-26T21:21:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.196+0000 7f13ebd66640 1 -- start start 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc003b70 0x7f13e414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e4154340 0x7f13e4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13e40d2000 con 0x7f13e4161950 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13e40d15f0 con 0x7f13cc003b70 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ebd66640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40d0230 con 0x7f13e4154340 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49052/0 (socket says 172.21.15.40:49052) 2024-04-26T21:21:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 -- 172.21.15.40:0/2865704639 learned_addr learned my addr 172.21.15.40:0/2865704639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13e9301640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e4154340 0x7f13e4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc003b70 0x7f13e414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 -- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e4154340 msgr2=0x7f13e4157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e4154340 0x7f13e4157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 -- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc003b70 msgr2=0x7f13e414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc003b70 0x7f13e414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 -- 172.21.15.40:0/2865704639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e414a7b0 con 0x7f13e4161950 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13ea303640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f13e40d2ef0 tx=0x7f13dc00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc005030 con 0x7f13e4161950 2024-04-26T21:21:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc0051f0 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.197+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0069e0 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.198+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e40c6ac0 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.198+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e40c7080 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.199+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e406a290 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.199+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13dc006b80 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.199+0000 7f13e27fc640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 0x7f13d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.200+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f13dc08ae80 con 0x7f13e4161950 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.200+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 0x7f13d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.200+0000 7f13e9b02640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 0x7f13d006ede0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f13d4035000 tx=0x7f13d402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.204+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13dc0592b0 con 0x7f13e4161950 2024-04-26T21:21:14.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.387+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13e40b8f10 con 0x7f13e4161950 2024-04-26T21:21:14.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.387+0000 7f13e27fc640 1 -- 172.21.15.40:0/2865704639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f13dc058bb0 con 0x7f13e4161950 2024-04-26T21:21:14.388 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:14.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 msgr2=0x7f13d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 0x7f13d006ede0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f13d4035000 tx=0x7f13d402f000 comp rx=0 tx=0).stop 2024-04-26T21:21:14.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 msgr2=0x7f13e4161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f13e40d2ef0 tx=0x7f13dc00e820 comp rx=0 tx=0).stop 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 shutdown_connections 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f13d006c920 0x7f13d006ede0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13e4161950 0x7f13e4161e00 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e4154340 0x7f13e4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 --2- 172.21.15.40:0/2865704639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13cc003b70 0x7f13e414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 >> 172.21.15.40:0/2865704639 conn(0x7f13e409ba40 msgr2=0x7f13e415fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 shutdown_connections 2024-04-26T21:21:14.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.389+0000 7f13ebd66640 1 -- 172.21.15.40:0/2865704639 wait complete. 2024-04-26T21:21:14.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.503+0000 7efdd816a640 1 Processor -- start 2024-04-26T21:21:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.503+0000 7efdd816a640 1 -- start start 2024-04-26T21:21:14.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd816a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00d3820 0x7efdd00b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd816a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdd00b60b0 0x7efdd00d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd816a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd00d3f50 con 0x7efdd015f680 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd816a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdd015fa60 con 0x7efdd00b60b0 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd816a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdd00b5090 con 0x7efdd00d3820 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd6707640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdd00b60b0 0x7efdd00d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd6707640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdd00b60b0 0x7efdd00d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49064/0 (socket says 172.21.15.40:49064) 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.504+0000 7efdd6707640 1 -- 172.21.15.40:0/254025279 learned_addr learned my addr 172.21.15.40:0/254025279 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd5705640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00d3820 0x7efdd00b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd6707640 1 -- 172.21.15.40:0/254025279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00d3820 msgr2=0x7efdd00b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd6707640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00d3820 0x7efdd00b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd6707640 1 -- 172.21.15.40:0/254025279 >> v1:172.21.15.184:6789/0 conn(0x7efdd015f680 legacy=0x7efdd0163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd6707640 1 -- 172.21.15.40:0/254025279 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00b5370 con 0x7efdd00b60b0 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.505+0000 7efdd6707640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd00b60b0 0x7efdd00d4f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009e50 tx=0x7efdc002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0c2e7a8b51a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd4f04640 1 -- 172.21.15.40:0/254025279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0036070 con 0x7efdd00b60b0 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd4f04640 1 -- 172.21.15.40:0/254025279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc0031600 con 0x7efdd00b60b0 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd4f04640 1 -- 172.21.15.40:0/254025279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00319d0 con 0x7efdd00b60b0 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 -- 172.21.15.40:0/254025279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd00b60b0 msgr2=0x7efdd00d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd00b60b0 0x7efdd00d4f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009e50 tx=0x7efdc002a710 comp rx=0 tx=0).stop 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 -- 172.21.15.40:0/254025279 shutdown_connections 2024-04-26T21:21:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd00b60b0 0x7efdd00d4f20 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 --2- 172.21.15.40:0/254025279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00d3820 0x7efdd00b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 -- 172.21.15.40:0/254025279 >> 172.21.15.40:0/254025279 conn(0x7efdd008eca0 msgr2=0x7efdd009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 -- 172.21.15.40:0/254025279 shutdown_connections 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.506+0000 7efdd816a640 1 -- 172.21.15.40:0/254025279 wait complete. 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.507+0000 7efdd816a640 1 Processor -- start 2024-04-26T21:21:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.507+0000 7efdd816a640 1 -- start start 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00b60b0 0x7efdd00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd00d3820 0x7efdd00cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00d55e0 con 0x7efdd015f680 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00d5460 con 0x7efdd00d3820 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd816a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdd00d5760 con 0x7efdd00b60b0 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49074/0 (socket says 172.21.15.40:49074) 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 -- 172.21.15.40:0/4260672650 learned_addr learned my addr 172.21.15.40:0/4260672650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd5705640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd00d3820 0x7efdd00cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd5f06640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00b60b0 0x7efdd00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 -- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00b60b0 msgr2=0x7efdd00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00b60b0 0x7efdd00cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 -- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd00d3820 msgr2=0x7efdd00cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd00d3820 0x7efdd00cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.508+0000 7efdd6707640 1 -- 172.21.15.40:0/4260672650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00cf100 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdd6707640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009e50 tx=0x7efdc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0036050 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc0042070 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0009510 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd00cb4a0 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.509+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd00cba60 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.510+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd00cc340 con 0x7efdd015f680 2024-04-26T21:21:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.510+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdc0007bd0 con 0x7efdd015f680 2024-04-26T21:21:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.511+0000 7efdce7fc640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.511+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7efdc00ac9c0 con 0x7efdd015f680 2024-04-26T21:21:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.511+0000 7efdd5f06640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.511+0000 7efdd5f06640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 0x7efdbc06edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efdc8007fd0 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.515+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdc00aee20 con 0x7efdd015f680 2024-04-26T21:21:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.699+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efdd0075640 con 0x7efdd015f680 2024-04-26T21:21:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.699+0000 7efdce7fc640 1 -- 172.21.15.40:0/4260672650 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7efdc007adf0 con 0x7efdd015f680 2024-04-26T21:21:14.700 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:14.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 msgr2=0x7efdbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 0x7efdbc06edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efdc8007fd0 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 msgr2=0x7efdd01f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009e50 tx=0x7efdc0004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 shutdown_connections 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efdbc06c8f0 0x7efdbc06edb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdd015f680 0x7efdd01f81f0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd00d3820 0x7efdd00cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 --2- 172.21.15.40:0/4260672650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdd00b60b0 0x7efdd00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 >> 172.21.15.40:0/4260672650 conn(0x7efdd008eca0 msgr2=0x7efdd00b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 shutdown_connections 2024-04-26T21:21:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.701+0000 7efdd816a640 1 -- 172.21.15.40:0/4260672650 wait complete. 2024-04-26T21:21:14.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.816+0000 7f6e67b04640 1 Processor -- start 2024-04-26T21:21:14.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.816+0000 7f6e67b04640 1 -- start start 2024-04-26T21:21:14.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e67b04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f5b0 0x7f6e600b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e67b04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e600b55b0 0x7f6e600d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e67b04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e600b6aa0 con 0x7f6e6015f990 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e67b04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6e600d3eb0 con 0x7f6e600b55b0 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e67b04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600d4190 con 0x7f6e6015f5b0 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e660a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e600b55b0 0x7f6e600d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e6509f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6e6015f990 0x7f6e600d0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38622/0 (socket says 172.21.15.40:38622) 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e6509f640 1 -- 172.21.15.40:0/1236419527 learned_addr learned my addr 172.21.15.40:0/1236419527 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e658a0640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f5b0 0x7f6e600b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e660a1640 1 -- 172.21.15.40:0/1236419527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f5b0 msgr2=0x7f6e600b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e660a1640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f5b0 0x7f6e600b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e660a1640 1 -- 172.21.15.40:0/1236419527 >> v1:172.21.15.40:6789/0 conn(0x7f6e6015f990 legacy=0x7f6e600d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.817+0000 7f6e660a1640 1 -- 172.21.15.40:0/1236419527 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600d0200 con 0x7f6e600b55b0 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.818+0000 7f6e660a1640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e600b55b0 0x7f6e600d3340 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009d90 tx=0x7f6e5802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6da8dc011f369b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.818+0000 7f6e6489e640 1 -- 172.21.15.40:0/1236419527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58031600 con 0x7f6e600b55b0 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.818+0000 7f6e6489e640 1 -- 172.21.15.40:0/1236419527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e580317a0 con 0x7f6e600b55b0 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.818+0000 7f6e6489e640 1 -- 172.21.15.40:0/1236419527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58031b70 con 0x7f6e600b55b0 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 -- 172.21.15.40:0/1236419527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e600b55b0 msgr2=0x7f6e600d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e600b55b0 0x7f6e600d3340 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009d90 tx=0x7f6e5802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 -- 172.21.15.40:0/1236419527 shutdown_connections 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e600b55b0 0x7f6e600d3340 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 --2- 172.21.15.40:0/1236419527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f5b0 0x7f6e600b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 -- 172.21.15.40:0/1236419527 >> 172.21.15.40:0/1236419527 conn(0x7f6e6008ec40 msgr2=0x7f6e6009e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 -- 172.21.15.40:0/1236419527 shutdown_connections 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.819+0000 7f6e67b04640 1 -- 172.21.15.40:0/1236419527 wait complete. 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.820+0000 7f6e67b04640 1 Processor -- start 2024-04-26T21:21:14.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.820+0000 7f6e67b04640 1 -- start start 2024-04-26T21:21:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 0x7f6e600cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 0x7f6e600ba460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600d1290 con 0x7f6e6015f5b0 2024-04-26T21:21:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600d1110 con 0x7f6e600b55b0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e67b04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600d3880 con 0x7f6e6015f990 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49102/0 (socket says 172.21.15.40:49102) 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 -- 172.21.15.40:0/4053044320 learned_addr learned my addr 172.21.15.40:0/4053044320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e658a0640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 0x7f6e600cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e660a1640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 0x7f6e600ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 -- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 msgr2=0x7f6e600ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 0x7f6e600ba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 -- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 msgr2=0x7f6e600cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 0x7f6e600cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 -- 172.21.15.40:0/4053044320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600caa30 con 0x7f6e6015f5b0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e658a0640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 0x7f6e600cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e660a1640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 0x7f6e600ba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.821+0000 7f6e6509f640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000c3c0 tx=0x7f6e5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.822+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50017070 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.822+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e50005ce0 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.822+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e600cad10 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.822+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500072f0 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.822+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e600c8ff0 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.823+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6006a290 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.823+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e5001a070 con 0x7f6e6015f5b0 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.824+0000 7f6e567fc640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.824+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f6e5008ae90 con 0x7f6e6015f5b0 2024-04-26T21:21:14.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.824+0000 7f6e658a0640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.824+0000 7f6e658a0640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:14.828+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e50059350 con 0x7f6e6015f5b0 2024-04-26T21:21:15.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.016+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e600caee0 con 0x7f6e6015f5b0 2024-04-26T21:21:15.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.016+0000 7f6e567fc640 1 -- 172.21.15.40:0/4053044320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f6e50058c50 con 0x7f6e6015f5b0 2024-04-26T21:21:15.018 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:15.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 msgr2=0x7f6e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).stop 2024-04-26T21:21:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 msgr2=0x7f6e600b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000c3c0 tx=0x7f6e5000c880 comp rx=0 tx=0).stop 2024-04-26T21:21:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 shutdown_connections 2024-04-26T21:21:15.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6015f990 0x7f6e600ba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6015f5b0 0x7f6e600b5d30 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 --2- 172.21.15.40:0/4053044320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e600b55b0 0x7f6e600cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 >> 172.21.15.40:0/4053044320 conn(0x7f6e6008ec40 msgr2=0x7f6e600b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 shutdown_connections 2024-04-26T21:21:15.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.019+0000 7f6e67b04640 1 -- 172.21.15.40:0/4053044320 wait complete. 2024-04-26T21:21:15.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:15.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.130+0000 7f11af30a640 1 Processor -- start 2024-04-26T21:21:15.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 -- start start 2024-04-26T21:21:15.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f11a815f370 0x7f11a80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11a815f750 0x7f11a80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a80d37d0 0x7f11a80d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80b4d10 con 0x7f11a815f750 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80d4160 con 0x7f11a815f370 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.131+0000 7f11af30a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80d03a0 con 0x7f11a80d37d0 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11a815f750 0x7f11a80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11a815f750 0x7f11a80b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49110/0 (socket says 172.21.15.40:49110) 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 -- 172.21.15.40:0/571134052 learned_addr learned my addr 172.21.15.40:0/571134052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 -- 172.21.15.40:0/571134052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a80d37d0 msgr2=0x7f11a80d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/571134052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a80d37d0 0x7f11a80d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 -- 172.21.15.40:0/571134052 >> v2:172.21.15.184:3300/0 conn(0x7f11a815f370 msgr2=0x7f11a80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ad0a6640 1 --2- 172.21.15.40:0/571134052 >> v2:172.21.15.184:3300/0 conn(0x7f11a815f370 0x7f11a80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/571134052 >> v2:172.21.15.184:3300/0 conn(0x7f11a815f370 0x7f11a80cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.132+0000 7f11ac8a5640 1 -- 172.21.15.40:0/571134052 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80d0680 con 0x7f11a815f750 2024-04-26T21:21:15.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/571134052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a815f750 0x7f11a80b47d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f11a80d2d10 tx=0x7f119802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eee176d26d0e411 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f119ffff640 1 -- 172.21.15.40:0/571134052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198037070 con 0x7f11a815f750 2024-04-26T21:21:15.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f119ffff640 1 -- 172.21.15.40:0/571134052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1198032440 con 0x7f11a815f750 2024-04-26T21:21:15.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f119ffff640 1 -- 172.21.15.40:0/571134052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198032810 con 0x7f11a815f750 2024-04-26T21:21:15.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 -- 172.21.15.40:0/571134052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a815f750 msgr2=0x7f11a80b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 --2- 172.21.15.40:0/571134052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a815f750 0x7f11a80b47d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f11a80d2d10 tx=0x7f119802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 -- 172.21.15.40:0/571134052 shutdown_connections 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 --2- 172.21.15.40:0/571134052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a80d37d0 0x7f11a80d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 --2- 172.21.15.40:0/571134052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a815f750 0x7f11a80b47d0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 --2- 172.21.15.40:0/571134052 >> v2:172.21.15.184:3300/0 conn(0x7f11a815f370 0x7f11a80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 -- 172.21.15.40:0/571134052 >> 172.21.15.40:0/571134052 conn(0x7f11a808ec10 msgr2=0x7f11a809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 -- 172.21.15.40:0/571134052 shutdown_connections 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.133+0000 7f11af30a640 1 -- 172.21.15.40:0/571134052 wait complete. 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 Processor -- start 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 -- start start 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a815f370 0x7f11a8161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 0x7f11a8162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d0f20 con 0x7f11a80d37d0 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d0c40 con 0x7f11a815f750 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11af30a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80d1bd0 con 0x7f11a815f370 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11ad0a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11ad0a6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49112/0 (socket says 172.21.15.40:49112) 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.135+0000 7f11ad0a6640 1 -- 172.21.15.40:0/238797813 learned_addr learned my addr 172.21.15.40:0/238797813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:15.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a815f370 0x7f11a8161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad8a7640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 0x7f11a8162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 -- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a815f370 msgr2=0x7f11a8161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a815f370 0x7f11a8161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 -- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 msgr2=0x7f11a8162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 0x7f11a8162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 -- 172.21.15.40:0/238797813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80c9a10 con 0x7f11a80d37d0 2024-04-26T21:21:15.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad8a7640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 0x7f11a8162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11ad0a6640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c820 tx=0x7f11a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4015070 con 0x7f11a80d37d0 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a4004240 con 0x7f11a80d37d0 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a80c9cf0 con 0x7f11a80d37d0 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a400ce10 con 0x7f11a80d37d0 2024-04-26T21:21:15.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.136+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80cc640 con 0x7f11a80d37d0 2024-04-26T21:21:15.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.137+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a806a290 con 0x7f11a80d37d0 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.138+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11a4007830 con 0x7f11a80d37d0 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.138+0000 7f119dffb640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.138+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f11a408b930 con 0x7f11a80d37d0 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.139+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.139+0000 7f11ac8a5640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 0x7f118c06edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1198034000 tx=0x7f119802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.143+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11a40597a0 con 0x7f11a80d37d0 2024-04-26T21:21:15.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.365+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a80754c0 con 0x7f11a80d37d0 2024-04-26T21:21:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.365+0000 7f119dffb640 1 -- 172.21.15.40:0/238797813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5192 (secure 0 0 0) 0x7f11a405daa0 con 0x7f11a80d37d0 2024-04-26T21:21:15.367 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:15.367 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:21:13.987062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":10,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 91 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 msgr2=0x7f118c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 0x7f118c06edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1198034000 tx=0x7f119802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 msgr2=0x7f11a815d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c820 tx=0x7f11a400cce0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 shutdown_connections 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11a815f750 0x7f11a8162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11a815f370 0x7f11a8161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 --2- 172.21.15.40:0/238797813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11a80d37d0 0x7f11a815d000 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 >> 172.21.15.40:0/238797813 conn(0x7f11a808ec10 msgr2=0x7f11a815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 shutdown_connections 2024-04-26T21:21:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.368+0000 7f11af30a640 1 -- 172.21.15.40:0/238797813 wait complete. 2024-04-26T21:21:15.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:15.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.479+0000 7f5173cf9640 1 Processor -- start 2024-04-26T21:21:15.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.479+0000 7f5173cf9640 1 -- start start 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f516c15b4e0 0x7f516c071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b8c0 0x7f516c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f516c073820 0x7f516c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f516c0ce230 con 0x7f516c073820 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f516c0733e0 con 0x7f516c15b4e0 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5173cf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f516c072f00 con 0x7f516c15b8c0 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f516c073820 0x7f516c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f516c073820 0x7f516c06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49116/0 (socket says 172.21.15.40:49116) 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 -- 172.21.15.40:0/1332423210 learned_addr learned my addr 172.21.15.40:0/1332423210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5171294640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b8c0 0x7f516c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5171a95640 1 --2- 172.21.15.40:0/1332423210 >> v2:172.21.15.184:3300/0 conn(0x7f516c15b4e0 0x7f516c071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 -- 172.21.15.40:0/1332423210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b8c0 msgr2=0x7f516c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b8c0 0x7f516c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 -- 172.21.15.40:0/1332423210 >> v2:172.21.15.184:3300/0 conn(0x7f516c15b4e0 msgr2=0x7f516c071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 --2- 172.21.15.40:0/1332423210 >> v2:172.21.15.184:3300/0 conn(0x7f516c15b4e0 0x7f516c071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.480+0000 7f5172296640 1 -- 172.21.15.40:0/1332423210 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c072660 con 0x7f516c073820 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.481+0000 7f5172296640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c06ebb0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b3ba0 tx=0x7f516402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc18ba1ba685d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.481+0000 7f5170a93640 1 -- 172.21.15.40:0/1332423210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164036070 con 0x7f516c073820 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.481+0000 7f5170a93640 1 -- 172.21.15.40:0/1332423210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164031600 con 0x7f516c073820 2024-04-26T21:21:15.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.481+0000 7f5170a93640 1 -- 172.21.15.40:0/1332423210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640319d0 con 0x7f516c073820 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.481+0000 7f5173cf9640 1 -- 172.21.15.40:0/1332423210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 msgr2=0x7f516c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c06ebb0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b3ba0 tx=0x7f516402a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 -- 172.21.15.40:0/1332423210 shutdown_connections 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c06ebb0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 --2- 172.21.15.40:0/1332423210 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b8c0 0x7f516c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 --2- 172.21.15.40:0/1332423210 >> v2:172.21.15.184:3300/0 conn(0x7f516c15b4e0 0x7f516c071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 -- 172.21.15.40:0/1332423210 >> 172.21.15.40:0/1332423210 conn(0x7f516c0d0d30 msgr2=0x7f516c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 -- 172.21.15.40:0/1332423210 shutdown_connections 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.482+0000 7f5173cf9640 1 -- 172.21.15.40:0/1332423210 wait complete. 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.483+0000 7f5173cf9640 1 Processor -- start 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.483+0000 7f5173cf9640 1 -- start start 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b4e0 0x7f516c0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 0x7f516c0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0b3e80 con 0x7f516c073820 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f516c072c20 con 0x7f516c15b8c0 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f516c06f0f0 con 0x7f516c15b4e0 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49132/0 (socket says 172.21.15.40:49132) 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 -- 172.21.15.40:0/2392648559 learned_addr learned my addr 172.21.15.40:0/2392648559 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:15.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171294640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b4e0 0x7f516c0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5172296640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 0x7f516c0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 -- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b4e0 msgr2=0x7f516c0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b4e0 0x7f516c0afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 -- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 msgr2=0x7f516c0affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 0x7f516c0affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 -- 172.21.15.40:0/2392648559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c1509a0 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5172296640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 0x7f516c0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5171a95640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f516800d900 tx=0x7f516800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800b040 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516800aac0 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168011460 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c157050 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.484+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c141950 con 0x7f516c073820 2024-04-26T21:21:15.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.485+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c06a290 con 0x7f516c073820 2024-04-26T21:21:15.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.486+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5168011600 con 0x7f516c073820 2024-04-26T21:21:15.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.486+0000 7f51627fc640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.487+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f516808bfe0 con 0x7f516c073820 2024-04-26T21:21:15.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.487+0000 7f5171294640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.487+0000 7f5171294640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.491+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5168015030 con 0x7f516c073820 2024-04-26T21:21:15.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.677+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f516c06d2b0 con 0x7f516c073820 2024-04-26T21:21:15.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.678+0000 7f51627fc640 1 -- 172.21.15.40:0/2392648559 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7f5168060030 con 0x7f516c073820 2024-04-26T21:21:15.679 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:15.679 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 msgr2=0x7f515006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c008040 comp rx=0 tx=0).stop 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 msgr2=0x7f516c150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f516800d900 tx=0x7f516800ddc0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 shutdown_connections 2024-04-26T21:21:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f516c15b8c0 0x7f516c0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f516c15b4e0 0x7f516c0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 --2- 172.21.15.40:0/2392648559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f516c073820 0x7f516c150460 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 >> 172.21.15.40:0/2392648559 conn(0x7f516c0d0d30 msgr2=0x7f516c0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 shutdown_connections 2024-04-26T21:21:15.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.680+0000 7f5173cf9640 1 -- 172.21.15.40:0/2392648559 wait complete. 2024-04-26T21:21:15.692 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:21:15.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.792+0000 7fa6c2a72640 1 Processor -- start 2024-04-26T21:21:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.792+0000 7fa6c2a72640 1 -- start start 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c2a72640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6bc0d11b0 0x7fa6bc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c2a72640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6bc0d1740 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c2a72640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6bc0d1a20 con 0x7fa6bc0cf2e0 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c2a72640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa6bc06e710 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6bbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6bc0d11b0 0x7fa6bc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c100f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa6bc06e2c0 0x7fa6bc15ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38632/0 (socket says 172.21.15.40:38632) 2024-04-26T21:21:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6c100f640 1 -- 172.21.15.40:0/859588292 learned_addr learned my addr 172.21.15.40:0/859588292 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6bbfff640 1 -- 172.21.15.40:0/859588292 >> v1:172.21.15.184:6789/0 conn(0x7fa6bc0cf2e0 legacy=0x7fa6bc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6bbfff640 1 -- 172.21.15.40:0/859588292 >> v1:172.21.15.40:6789/0 conn(0x7fa6bc06e2c0 legacy=0x7fa6bc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.793+0000 7fa6bbfff640 1 -- 172.21.15.40:0/859588292 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc0d1d00 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.794+0000 7fa6bbfff640 1 --2- 172.21.15.40:0/859588292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc0d11b0 0x7fa6bc06dd80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00097d0 tx=0x7fa6b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cbad625c9d0126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.794+0000 7fa6bb7fe640 1 -- 172.21.15.40:0/859588292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0037070 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.794+0000 7fa6bb7fe640 1 -- 172.21.15.40:0/859588292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b0032440 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.794+0000 7fa6bb7fe640 1 -- 172.21.15.40:0/859588292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0032810 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 -- 172.21.15.40:0/859588292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc0d11b0 msgr2=0x7fa6bc06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/859588292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc0d11b0 0x7fa6bc06dd80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00097d0 tx=0x7fa6b002a270 comp rx=0 tx=0).stop 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 -- 172.21.15.40:0/859588292 shutdown_connections 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/859588292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc0d11b0 0x7fa6bc06dd80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 -- 172.21.15.40:0/859588292 >> 172.21.15.40:0/859588292 conn(0x7fa6bc0b4a60 msgr2=0x7fa6bc0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 -- 172.21.15.40:0/859588292 shutdown_connections 2024-04-26T21:21:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.795+0000 7fa6c2a72640 1 -- 172.21.15.40:0/859588292 wait complete. 2024-04-26T21:21:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.796+0000 7fa6c2a72640 1 Processor -- start 2024-04-26T21:21:15.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.796+0000 7fa6c2a72640 1 -- start start 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6bc0cf2e0 0x7fa6bc0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 0x7fa6bc0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc0cf6c0 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc0d22c0 con 0x7fa6bc0cf2e0 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c2a72640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc0cf840 con 0x7fa6bc0d11b0 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49148/0 (socket says 172.21.15.40:49148) 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 -- 172.21.15.40:0/91017513 learned_addr learned my addr 172.21.15.40:0/91017513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:15.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6bbfff640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6bc0cf2e0 0x7fa6bc0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c100f640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 0x7fa6bc0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 -- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 msgr2=0x7fa6bc0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 0x7fa6bc0b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 -- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6bc0cf2e0 msgr2=0x7fa6bc0b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6bc0cf2e0 0x7fa6bc0b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c080e640 1 -- 172.21.15.40:0/91017513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc1e9cd0 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.797+0000 7fa6c100f640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 0x7fa6bc0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6c080e640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c3f0 tx=0x7fa6ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0052d0 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac005490 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc1e9fb0 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac007260 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.798+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc1ea570 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.799+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc1ea7b0 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.799+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6ac007400 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.800+0000 7fa6b97fa640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 0x7fa6a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.800+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fa6ac08b010 con 0x7fa6bc06e2c0 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.800+0000 7fa6bbfff640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 0x7fa6a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.801+0000 7fa6bbfff640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 0x7fa6a806edb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0034000 tx=0x7fa6b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:15.804+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6bc1ea7b0 con 0x7fa6bc06e2c0 2024-04-26T21:21:16.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.028+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6bc06a290 con 0x7fa6bc06e2c0 2024-04-26T21:21:16.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.028+0000 7fa6b97fa640 1 -- 172.21.15.40:0/91017513 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5192 (secure 0 0 0) 0x7fa6ac059440 con 0x7fa6bc06e2c0 2024-04-26T21:21:16.030 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:16.030 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:21:13.987062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":10,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:16.030 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 91 2024-04-26T21:21:16.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 msgr2=0x7fa6a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 0x7fa6a806edb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0034000 tx=0x7fa6b002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 msgr2=0x7fa6bc14e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c3f0 tx=0x7fa6ac00c8b0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 shutdown_connections 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a806c8f0 0x7fa6a806edb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6bc0d11b0 0x7fa6bc0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6bc0cf2e0 0x7fa6bc0b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 --2- 172.21.15.40:0/91017513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6bc06e2c0 0x7fa6bc14e430 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 >> 172.21.15.40:0/91017513 conn(0x7fa6bc0b4a60 msgr2=0x7fa6bc070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 shutdown_connections 2024-04-26T21:21:16.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.030+0000 7fa6c2a72640 1 -- 172.21.15.40:0/91017513 wait complete. 2024-04-26T21:21:16.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.142+0000 7fe892ef1640 1 Processor -- start 2024-04-26T21:21:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.142+0000 7fe892ef1640 1 -- start start 2024-04-26T21:21:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe892ef1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe88c15f720 0x7fe88c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe892ef1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 0x7fe88c0d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe892ef1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88c0b2510 con 0x7fe88c15f340 2024-04-26T21:21:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe892ef1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c0b27f0 con 0x7fe88c15f720 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe892ef1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c0b2ad0 con 0x7fe88c0d5610 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe88c15f720 0x7fe88c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe88c15f720 0x7fe88c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49158/0 (socket says 172.21.15.40:49158) 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 -- 172.21.15.40:0/917937596 learned_addr learned my addr 172.21.15.40:0/917937596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 -- 172.21.15.40:0/917937596 >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 msgr2=0x7fe88c0d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe89148e640 1 --2- 172.21.15.40:0/917937596 >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 0x7fe88c0d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 --2- 172.21.15.40:0/917937596 >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 0x7fe88c0d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 -- 172.21.15.40:0/917937596 >> v1:172.21.15.184:6789/0 conn(0x7fe88c15f340 legacy=0x7fe88c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe883fff640 1 -- 172.21.15.40:0/917937596 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0b2db0 con 0x7fe88c15f720 2024-04-26T21:21:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.143+0000 7fe89148e640 1 --2- 172.21.15.40:0/917937596 >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 0x7fe88c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.144+0000 7fe883fff640 1 --2- 172.21.15.40:0/917937596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c15f720 0x7fe88c0b1e80 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0d4c20 tx=0x7fe87c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6abd2d3fab7514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.144+0000 7fe8837fe640 1 -- 172.21.15.40:0/917937596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c037070 con 0x7fe88c15f720 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.144+0000 7fe8837fe640 1 -- 172.21.15.40:0/917937596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c032440 con 0x7fe88c15f720 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.144+0000 7fe8837fe640 1 -- 172.21.15.40:0/917937596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c032810 con 0x7fe88c15f720 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 -- 172.21.15.40:0/917937596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c15f720 msgr2=0x7fe88c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 --2- 172.21.15.40:0/917937596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c15f720 0x7fe88c0b1e80 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0d4c20 tx=0x7fe87c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 -- 172.21.15.40:0/917937596 shutdown_connections 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 --2- 172.21.15.40:0/917937596 >> v2:172.21.15.184:3300/0 conn(0x7fe88c0d5610 0x7fe88c0d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 --2- 172.21.15.40:0/917937596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c15f720 0x7fe88c0b1e80 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 -- 172.21.15.40:0/917937596 >> 172.21.15.40:0/917937596 conn(0x7fe88c08ec70 msgr2=0x7fe88c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 -- 172.21.15.40:0/917937596 shutdown_connections 2024-04-26T21:21:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.145+0000 7fe892ef1640 1 -- 172.21.15.40:0/917937596 wait complete. 2024-04-26T21:21:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.146+0000 7fe892ef1640 1 Processor -- start 2024-04-26T21:21:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.146+0000 7fe892ef1640 1 -- start start 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 0x7fe88c160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe88c15f720 0x7fe88c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b3940 con 0x7fe88c0d5610 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b37c0 con 0x7fe88c15f340 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe892ef1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0b3ac0 con 0x7fe88c15f720 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49166/0 (socket says 172.21.15.40:49166) 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 -- 172.21.15.40:0/2602135550 learned_addr learned my addr 172.21.15.40:0/2602135550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe883fff640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 0x7fe88c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe89148e640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe88c15f720 0x7fe88c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 -- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe88c15f720 msgr2=0x7fe88c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe88c15f720 0x7fe88c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 -- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 msgr2=0x7fe88c160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 0x7fe88c160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 -- 172.21.15.40:0/2602135550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c160fd0 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe883fff640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 0x7fe88c160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.147+0000 7fe890c8d640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c410 tx=0x7fe88800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.148+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888017070 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.148+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8880035f0 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.148+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c1612b0 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.148+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880039e0 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.148+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c161870 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.149+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c06a290 con 0x7fe88c0d5610 2024-04-26T21:21:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.149+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe88801a070 con 0x7fe88c0d5610 2024-04-26T21:21:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.149+0000 7fe8817fa640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 0x7fe87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.150+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7fe88808aff0 con 0x7fe88c0d5610 2024-04-26T21:21:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.150+0000 7fe883fff640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 0x7fe87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.150+0000 7fe883fff640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 0x7fe87806edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034000 tx=0x7fe87c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.154+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe888059420 con 0x7fe88c0d5610 2024-04-26T21:21:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.377+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe88c0bb090 con 0x7fe88c0d5610 2024-04-26T21:21:16.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.378+0000 7fe8817fa640 1 -- 172.21.15.40:0/2602135550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5192 (secure 0 0 0) 0x7fe888058d20 con 0x7fe88c0d5610 2024-04-26T21:21:16.379 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:16.379 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:20:45.315641+0000","modified":"2024-04-26T21:21:13.987062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":10,"addr":"172.21.15.40:6835/3716452175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3716452175},{"type":"v1","addr":"172.21.15.40:6835","nonce":3716452175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":10}]} 2024-04-26T21:21:16.379 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 91 2024-04-26T21:21:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 msgr2=0x7fe87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 0x7fe87806edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034000 tx=0x7fe87c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:16.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 msgr2=0x7fe88c14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c410 tx=0x7fe88800c8d0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 shutdown_connections 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe87806c8f0 0x7fe87806edb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe88c15f720 0x7fe88c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe88c15f340 0x7fe88c160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 --2- 172.21.15.40:0/2602135550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0d5610 0x7fe88c14e610 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 >> 172.21.15.40:0/2602135550 conn(0x7fe88c08ec70 msgr2=0x7fe88c0d4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 shutdown_connections 2024-04-26T21:21:16.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.380+0000 7fe892ef1640 1 -- 172.21.15.40:0/2602135550 wait complete. 2024-04-26T21:21:16.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:16.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.491+0000 7f3d67d22640 1 Processor -- start 2024-04-26T21:21:16.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.491+0000 7f3d67d22640 1 -- start start 2024-04-26T21:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d67d22640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d600d3d00 0x7f3d600d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d67d22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d600d5330 0x7f3d600cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d67d22640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600d05f0 con 0x7f3d6015f3e0 2024-04-26T21:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d67d22640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f3d600d4190 con 0x7f3d600d3d00 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d67d22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d600d08d0 con 0x7f3d600d5330 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d65abe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d6015f3e0 0x7f3d600cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38644/0 (socket says 172.21.15.40:38644) 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d65abe640 1 -- 172.21.15.40:0/3695561554 learned_addr learned my addr 172.21.15.40:0/3695561554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d652bd640 1 --2- 172.21.15.40:0/3695561554 >> v2:172.21.15.184:3300/0 conn(0x7f3d600d3d00 0x7f3d600d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d662bf640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d600d5330 0x7f3d600cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d64abc640 1 -- 172.21.15.40:0/3695561554 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273382796 0 0) 0x7f3d600d05f0 con 0x7f3d6015f3e0 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.492+0000 7f3d64abc640 1 -- 172.21.15.40:0/3695561554 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d4c003520 con 0x7f3d6015f3e0 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.493+0000 7f3d652bd640 1 -- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d600d5330 msgr2=0x7f3d600cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.493+0000 7f3d652bd640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d600d5330 0x7f3d600cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.493+0000 7f3d652bd640 1 -- 172.21.15.40:0/3695561554 >> v1:172.21.15.40:6789/0 conn(0x7f3d6015f3e0 legacy=0x7f3d600cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.493+0000 7f3d652bd640 1 -- 172.21.15.40:0/3695561554 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d600d0bb0 con 0x7f3d600d3d00 2024-04-26T21:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.493+0000 7f3d652bd640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d3d00 0x7f3d600d4df0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3d500097a0 tx=0x7f3d5002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e357ccb5a5aeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d64abc640 1 -- 172.21.15.40:0/3695561554 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50037070 con 0x7f3d600d3d00 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d64abc640 1 -- 172.21.15.40:0/3695561554 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d50032440 con 0x7f3d600d3d00 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d64abc640 1 -- 172.21.15.40:0/3695561554 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500327f0 con 0x7f3d600d3d00 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 -- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d3d00 msgr2=0x7f3d600d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d3d00 0x7f3d600d4df0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3d500097a0 tx=0x7f3d5002a180 comp rx=0 tx=0).stop 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 -- 172.21.15.40:0/3695561554 shutdown_connections 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d600d5330 0x7f3d600cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 --2- 172.21.15.40:0/3695561554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d3d00 0x7f3d600d4df0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 -- 172.21.15.40:0/3695561554 >> 172.21.15.40:0/3695561554 conn(0x7f3d6008ebf0 msgr2=0x7f3d6009e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 -- 172.21.15.40:0/3695561554 shutdown_connections 2024-04-26T21:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.494+0000 7f3d67d22640 1 -- 172.21.15.40:0/3695561554 wait complete. 2024-04-26T21:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.495+0000 7f3d67d22640 1 Processor -- start 2024-04-26T21:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.495+0000 7f3d67d22640 1 -- start start 2024-04-26T21:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 0x7f3d600b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d6015f3e0 0x7f3d600b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6015f7c0 con 0x7f3d600d3d00 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600d5780 con 0x7f3d600d5330 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d67d22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6015f940 con 0x7f3d6015f3e0 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d65abe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d65abe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49176/0 (socket says 172.21.15.40:49176) 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d65abe640 1 -- 172.21.15.40:0/2556658835 learned_addr learned my addr 172.21.15.40:0/2556658835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d652bd640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 0x7f3d600b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d662bf640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d6015f3e0 0x7f3d600b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d65abe640 1 -- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d6015f3e0 msgr2=0x7f3d600b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.496+0000 7f3d65abe640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d6015f3e0 0x7f3d600b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d65abe640 1 -- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 msgr2=0x7f3d600b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d65abe640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 0x7f3d600b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d65abe640 1 -- 172.21.15.40:0/2556658835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d600b4de0 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d652bd640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 0x7f3d600b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d65abe640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c001460 tx=0x7f3d5c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c017070 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00ac00 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c012400 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d600b50c0 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.497+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601f4980 con 0x7f3d600d3d00 2024-04-26T21:21:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.498+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d600b55e0 con 0x7f3d600d3d00 2024-04-26T21:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.498+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5c01a070 con 0x7f3d600d3d00 2024-04-26T21:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.499+0000 7f3d567fc640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 0x7f3d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.499+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f3d5c08b990 con 0x7f3d600d3d00 2024-04-26T21:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.499+0000 7f3d652bd640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 0x7f3d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.500+0000 7f3d652bd640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 0x7f3d4406edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3d50034000 tx=0x7f3d5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.503+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d600b55e0 con 0x7f3d600d3d00 2024-04-26T21:21:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.691+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d6006a290 con 0x7f3d600d3d00 2024-04-26T21:21:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.692+0000 7f3d567fc640 1 -- 172.21.15.40:0/2556658835 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16679 (secure 0 0 0) 0x7f3d5c059d30 con 0x7f3d600d3d00 2024-04-26T21:21:16.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:16.693 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:20:46.955853+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:20:43.597608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-26T21:20:44.259933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.694+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 msgr2=0x7f3d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.694+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 0x7f3d4406edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3d50034000 tx=0x7f3d5002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.694+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 msgr2=0x7f3d601f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.694+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c001460 tx=0x7f3d5c00cf70 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 shutdown_connections 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f3d4406c8f0 0x7f3d4406edb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d6015f3e0 0x7f3d600b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d600d5330 0x7f3d600b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 --2- 172.21.15.40:0/2556658835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d600d3d00 0x7f3d601f4440 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 >> 172.21.15.40:0/2556658835 conn(0x7f3d6008ebf0 msgr2=0x7f3d600d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 shutdown_connections 2024-04-26T21:21:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.695+0000 7f3d67d22640 1 -- 172.21.15.40:0/2556658835 wait complete. 2024-04-26T21:21:16.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:21:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.807+0000 7f7fd7778640 1 Processor -- start 2024-04-26T21:21:16.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.807+0000 7f7fd7778640 1 -- start start 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd7778640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7fd006e490 0x7f7fd015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd7778640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd00d1740 con 0x7f7fd00d11b0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd7778640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd00d1a20 con 0x7f7fd00cf2e0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd7778640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd00d1d00 con 0x7f7fd006e490 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd4d13640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7fd00d11b0 0x7f7fd006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38658/0 (socket says 172.21.15.40:38658) 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd4d13640 1 -- 172.21.15.40:0/3582228428 learned_addr learned my addr 172.21.15.40:0/3582228428 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fd5d15640 1 --2- 172.21.15.40:0/3582228428 >> v2:172.21.15.184:3300/0 conn(0x7f7fd006e490 0x7f7fd015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.808+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197435058 0 0) 0x7f7fd00d1740 con 0x7f7fd00d11b0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fb8003520 con 0x7f7fd00d11b0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849341588 0 0) 0x7f7fb8003520 con 0x7f7fd00d11b0 2024-04-26T21:21:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 >> v2:172.21.15.184:3300/0 conn(0x7f7fd006e490 msgr2=0x7f7fd015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 --2- 172.21.15.40:0/3582228428 >> v2:172.21.15.184:3300/0 conn(0x7f7fd006e490 0x7f7fd015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 >> v1:172.21.15.184:6789/0 conn(0x7f7fd00cf2e0 legacy=0x7f7fd006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd00d1fe0 con 0x7f7fd00d11b0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7fc4001670 con 0x7f7fd00d11b0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 >> v1:172.21.15.40:6789/0 conn(0x7f7fd00d11b0 legacy=0x7f7fd006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 0x7f7fb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb8004800 0x7f7fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80051c0 0x7f7fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00d1d00 con 0x7f7fb80039b0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800c060 con 0x7f7fb8004800 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb800c260 con 0x7f7fb80051c0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 0x7f7fb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd4d13640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb8004800 0x7f7fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5d15640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80051c0 0x7f7fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 -- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80051c0 msgr2=0x7f7fb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80051c0 0x7f7fb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 -- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb8004800 msgr2=0x7f7fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb8004800 0x7f7fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5514640 1 -- 172.21.15.40:0/3582228428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb8003520 con 0x7f7fb80039b0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd4d13640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb8004800 0x7f7fb8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.809+0000 7f7fd5d15640 1 -- 172.21.15.40:0/3582228428 reap_dead start 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.810+0000 7f7fd5514640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 0x7f7fb8003dd0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8001e50 tx=0x7f7fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e46d187fddce0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.810+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8001a70 con 0x7f7fb80039b0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.810+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc802ae40 con 0x7f7fb80039b0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.810+0000 7f7fcffff640 1 -- 172.21.15.40:0/3582228428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003bb0 con 0x7f7fb80039b0 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 -- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 msgr2=0x7f7fb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 0x7f7fb8003dd0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8001e50 tx=0x7f7fb8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:16.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 -- 172.21.15.40:0/3582228428 shutdown_connections 2024-04-26T21:21:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 --2- 172.21.15.40:0/3582228428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fb80039b0 0x7f7fb8003dd0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 -- 172.21.15.40:0/3582228428 >> 172.21.15.40:0/3582228428 conn(0x7f7fd00b4a60 msgr2=0x7f7fd00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 -- 172.21.15.40:0/3582228428 shutdown_connections 2024-04-26T21:21:16.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.811+0000 7f7fd7778640 1 -- 172.21.15.40:0/3582228428 wait complete. 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.812+0000 7f7fd7778640 1 Processor -- start 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.812+0000 7f7fd7778640 1 -- start start 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80039b0 0x7f7fd01f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd00cdc50 0x7f7fd00c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fd00cb960 0x7f7fd00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00cf840 con 0x7f7fd00cb960 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00cf6c0 con 0x7f7fd00cdc50 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd006dd80 con 0x7f7fb80039b0 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fd00cb960 0x7f7fd00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fd00cb960 0x7f7fd00cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49204/0 (socket says 172.21.15.40:49204) 2024-04-26T21:21:16.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 -- 172.21.15.40:0/388839612 learned_addr learned my addr 172.21.15.40:0/388839612 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd4d13640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd00cdc50 0x7f7fd00c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5514640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80039b0 0x7f7fd01f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 -- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80039b0 msgr2=0x7f7fd01f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb80039b0 0x7f7fd01f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 -- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd00cdc50 msgr2=0x7f7fd00c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd00cdc50 0x7f7fd00c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 -- 172.21.15.40:0/388839612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01f5ae0 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd4d13640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd00cdc50 0x7f7fd00c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd5d15640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fd00cb960 0x7f7fd00cbe10 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7fd006e8a0 tx=0x7f7fc0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0019020 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc0003610 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fd7778640 1 -- 172.21.15.40:0/388839612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd01f5dc0 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.813+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0003a00 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.814+0000 7f7fd7778640 1 -- 172.21.15.40:0/388839612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd01f6380 con 0x7f7fd00cb960 2024-04-26T21:21:16.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.814+0000 7f7fd7778640 1 -- 172.21.15.40:0/388839612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd006a290 con 0x7f7fd00cb960 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.815+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fc001c020 con 0x7f7fd00cb960 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.815+0000 7f7fcdffb640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.815+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f7fc008bd40 con 0x7f7fd00cb960 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.816+0000 7f7fd5514640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.816+0000 7f7fd5514640 1 --2- 172.21.15.40:0/388839612 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8032000 tx=0x7f7fc802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:16.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:16.820+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fc0023030 con 0x7f7fd00cb960 2024-04-26T21:21:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.048+0000 7f7fd7778640 1 -- 172.21.15.40:0/388839612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7fd006dae0 con 0x7f7fd00cb960 2024-04-26T21:21:17.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.048+0000 7f7fcdffb640 1 -- 172.21.15.40:0/388839612 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v91) ==== 316+0+0 (secure 0 0 0) 0x7f7fc005a170 con 0x7f7fd00cb960 2024-04-26T21:21:17.049 INFO:teuthology.orchestra.run.smithi040.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-04-26T21:21:17.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:21:17.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-04-26T21:21:17.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.155+0000 7f703475f640 1 Processor -- start 2024-04-26T21:21:17.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.155+0000 7f703475f640 1 -- start start 2024-04-26T21:21:17.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.156+0000 7f703475f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f702c143840 0x7f702c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.156+0000 7f703475f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f702c0b4520 0x7f702c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.156+0000 7f703475f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702c15dca0 con 0x7f702c0b53f0 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.156+0000 7f703475f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f702c143c20 con 0x7f702c0b4520 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.156+0000 7f703475f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f702c15df80 con 0x7f702c143840 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7031cfa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f702c0b4520 0x7f702c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7032cfc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f702c0b53f0 0x7f702c160e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38674/0 (socket says 172.21.15.40:38674) 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7032cfc640 1 -- 172.21.15.40:0/1270428379 learned_addr learned my addr 172.21.15.40:0/1270428379 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f70324fb640 1 --2- 172.21.15.40:0/1270428379 >> v2:172.21.15.184:3300/0 conn(0x7f702c143840 0x7f702c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f70314f9640 1 -- 172.21.15.40:0/1270428379 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667463162 0 0) 0x7f702c15dca0 con 0x7f702c0b53f0 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f70314f9640 1 -- 172.21.15.40:0/1270428379 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7014003520 con 0x7f702c0b53f0 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7031cfa640 1 -- 172.21.15.40:0/1270428379 >> v2:172.21.15.184:3300/0 conn(0x7f702c143840 msgr2=0x7f702c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7031cfa640 1 --2- 172.21.15.40:0/1270428379 >> v2:172.21.15.184:3300/0 conn(0x7f702c143840 0x7f702c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7031cfa640 1 -- 172.21.15.40:0/1270428379 >> v1:172.21.15.40:6789/0 conn(0x7f702c0b53f0 legacy=0x7f702c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.157+0000 7f7031cfa640 1 -- 172.21.15.40:0/1270428379 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c15e260 con 0x7f702c0b4520 2024-04-26T21:21:17.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f7031cfa640 1 --2- 172.21.15.40:0/1270428379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c0b4520 0x7f702c0b4eb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f702c076130 tx=0x7f702002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443130e96240aae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f70314f9640 1 -- 172.21.15.40:0/1270428379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020037070 con 0x7f702c0b4520 2024-04-26T21:21:17.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f70314f9640 1 -- 172.21.15.40:0/1270428379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7020032440 con 0x7f702c0b4520 2024-04-26T21:21:17.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f70314f9640 1 -- 172.21.15.40:0/1270428379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020032810 con 0x7f702c0b4520 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 -- 172.21.15.40:0/1270428379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c0b4520 msgr2=0x7f702c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 --2- 172.21.15.40:0/1270428379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c0b4520 0x7f702c0b4eb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f702c076130 tx=0x7f702002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 -- 172.21.15.40:0/1270428379 shutdown_connections 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 --2- 172.21.15.40:0/1270428379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c0b4520 0x7f702c0b4eb0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 --2- 172.21.15.40:0/1270428379 >> v2:172.21.15.184:3300/0 conn(0x7f702c143840 0x7f702c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 -- 172.21.15.40:0/1270428379 >> 172.21.15.40:0/1270428379 conn(0x7f702c0cd8a0 msgr2=0x7f702c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 -- 172.21.15.40:0/1270428379 shutdown_connections 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.158+0000 7f703475f640 1 -- 172.21.15.40:0/1270428379 wait complete. 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 Processor -- start 2024-04-26T21:21:17.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 -- start start 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f702c0b4520 0x7f702c145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702c0b53f0 0x7f702c146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f702c15da40 con 0x7f702c143840 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f702c15d8c0 con 0x7f702c0b53f0 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.160+0000 7f703475f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f702c0d0000 con 0x7f702c0b4520 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49220/0 (socket says 172.21.15.40:49220) 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 -- 172.21.15.40:0/2900759772 learned_addr learned my addr 172.21.15.40:0/2900759772 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7031cfa640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702c0b53f0 0x7f702c146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f70324fb640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f702c0b4520 0x7f702c145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 -- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f702c0b4520 msgr2=0x7f702c145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f702c0b4520 0x7f702c145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 -- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702c0b53f0 msgr2=0x7f702c146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702c0b53f0 0x7f702c146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 -- 172.21.15.40:0/2900759772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c1f54d0 con 0x7f702c143840 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f7032cfc640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cb40 tx=0x7f701c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c017070 con 0x7f702c143840 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c00aca0 con 0x7f702c143840 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c142230 con 0x7f702c143840 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c012620 con 0x7f702c143840 2024-04-26T21:21:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.161+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c072500 con 0x7f702c143840 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.162+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c06a290 con 0x7f702c143840 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.163+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f701c01a070 con 0x7f702c143840 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.163+0000 7f702affd640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.163+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5709+0+0 (secure 0 0 0) 0x7f701c08b8c0 con 0x7f702c143840 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.163+0000 7f70324fb640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.164+0000 7f70324fb640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7024009aa0 tx=0x7f7024009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.167+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f701c059730 con 0x7f702c143840 2024-04-26T21:21:17.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.391+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f702c142c10 con 0x7f702c143840 2024-04-26T21:21:17.393 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:21:17.392+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:21:17.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.393+0000 7f702affd640 1 -- 172.21.15.40:0/2900759772 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 191+0+0 (secure 0 0 0) 0x7f701c05da30 con 0x7f702c143840 2024-04-26T21:21:17.394 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:21:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 msgr2=0x7f701806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7024009aa0 tx=0x7f7024009190 comp rx=0 tx=0).stop 2024-04-26T21:21:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 msgr2=0x7f702c0af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cb40 tx=0x7f701c00a060 comp rx=0 tx=0).stop 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 shutdown_connections 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f702c143840 0x7f702c0af640 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702c0b53f0 0x7f702c146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 --2- 172.21.15.40:0/2900759772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f702c0b4520 0x7f702c145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 >> 172.21.15.40:0/2900759772 conn(0x7f702c0cd8a0 msgr2=0x7f702c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 shutdown_connections 2024-04-26T21:21:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.397+0000 7f703475f640 1 -- 172.21.15.40:0/2900759772 wait complete. 2024-04-26T21:21:17.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:21:17.434 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:21:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.512+0000 7f436de45640 1 Processor -- start 2024-04-26T21:21:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.512+0000 7f436de45640 1 -- start start 2024-04-26T21:21:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f436de45640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 0x7f436806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f436de45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f436de45640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436809c700 con 0x7f43680b3210 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f436de45640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f43680b4480 con 0x7f43680d0010 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f436de45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f436806e050 con 0x7f43680b3c20 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f43677fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 0x7f436806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:48726/0 (socket says 172.21.15.40:48726) 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f43677fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 0x7f436806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:48372/0 (socket says 172.21.15.40:48372) 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 -- 172.21.15.40:0/230270460 learned_addr learned my addr 172.21.15.40:0/230270460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4367fff640 1 --1- 172.21.15.40:0/230270460 >> v1:172.21.15.184:6789/0 conn(0x7f43680b3210 0x7f436815fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:48100/0 (socket says 172.21.15.40:48100) 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f43667fc640 1 -- 172.21.15.40:0/230270460 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3029913878 0 0) 0x7f436809c700 con 0x7f43680b3210 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 -- 172.21.15.40:0/230270460 >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 msgr2=0x7f436806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 --2- 172.21.15.40:0/230270460 >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 0x7f436806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 -- 172.21.15.40:0/230270460 >> v1:172.21.15.184:6789/0 conn(0x7f43680b3210 legacy=0x7f436815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.513+0000 7f4366ffd640 1 -- 172.21.15.40:0/230270460 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43680d1c50 con 0x7f43680b3c20 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.514+0000 7f4366ffd640 1 --2- 172.21.15.40:0/230270460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f435c009870 tx=0x7f435c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6d18ed954bc11b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.514+0000 7f43667fc640 1 -- 172.21.15.40:0/230270460 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c037070 con 0x7f43680b3c20 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.514+0000 7f43667fc640 1 -- 172.21.15.40:0/230270460 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c032440 con 0x7f43680b3c20 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.514+0000 7f43667fc640 1 -- 172.21.15.40:0/230270460 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c032810 con 0x7f43680b3c20 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 -- 172.21.15.40:0/230270460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 msgr2=0x7f43680b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 --2- 172.21.15.40:0/230270460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f435c009870 tx=0x7f435c02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 -- 172.21.15.40:0/230270460 shutdown_connections 2024-04-26T21:21:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 --2- 172.21.15.40:0/230270460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680b3c20 0x7f43680b2cd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 --2- 172.21.15.40:0/230270460 >> v2:172.21.15.184:3300/0 conn(0x7f43680d0010 0x7f436806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 -- 172.21.15.40:0/230270460 >> 172.21.15.40:0/230270460 conn(0x7f436809ba40 msgr2=0x7f436809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 -- 172.21.15.40:0/230270460 shutdown_connections 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.515+0000 7f436de45640 1 -- 172.21.15.40:0/230270460 wait complete. 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.516+0000 7f436de45640 1 Processor -- start 2024-04-26T21:21:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.516+0000 7f436de45640 1 -- start start 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43680b3210 0x7f43681e9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 0x7f436809ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43680b3820 con 0x7f43680b3c20 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43680b36a0 con 0x7f43680b3210 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f436de45640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f43680b39a0 con 0x7f43680d0010 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49264/0 (socket says 172.21.15.40:49264) 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f4366ffd640 1 -- 172.21.15.40:0/1259856884 learned_addr learned my addr 172.21.15.40:0/1259856884 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f4367fff640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 0x7f436809ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.517+0000 7f43677fe640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43680b3210 0x7f43681e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 -- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 msgr2=0x7f436809ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 0x7f436809ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 -- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43680b3210 msgr2=0x7f43681e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43680b3210 0x7f43681e9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 -- 172.21.15.40:0/1259856884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436809e300 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4367fff640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 0x7f436809ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f4366ffd640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c037040 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c042070 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681e9650 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c030880 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.518+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681e9c10 con 0x7f43680b3c20 2024-04-26T21:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.519+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436806a290 con 0x7f43680b3c20 2024-04-26T21:21:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.520+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f435c030a20 con 0x7f43680b3c20 2024-04-26T21:21:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.520+0000 7f434ffff640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 0x7f435406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.520+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5795+0+0 (secure 0 0 0) 0x7f435c0ac3b0 con 0x7f43680b3c20 2024-04-26T21:21:17.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.520+0000 7f43677fe640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 0x7f435406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.521+0000 7f43677fe640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 0x7f435406edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f43680b0ba0 tx=0x7f4358008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.524+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f436809e300 con 0x7f43680b3c20 2024-04-26T21:21:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:17.750+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4368068e00 con 0x7f43680b3c20 2024-04-26T21:21:18.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.395+0000 7f434ffff640 1 -- 172.21.15.40:0/1259856884 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) ==== 104+0+0 (secure 0 0 0) 0x7f435c045360 con 0x7f43680b3c20 2024-04-26T21:21:18.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 msgr2=0x7f435406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 0x7f435406edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f43680b0ba0 tx=0x7f4358008040 comp rx=0 tx=0).stop 2024-04-26T21:21:18.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 msgr2=0x7f43681ea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 shutdown_connections 2024-04-26T21:21:18.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f435406c8f0 0x7f435406edb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43680d0010 0x7f436809ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43680b3c20 0x7f43681ea6f0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 --2- 172.21.15.40:0/1259856884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43680b3210 0x7f43681e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.397+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 >> 172.21.15.40:0/1259856884 conn(0x7f436809ba40 msgr2=0x7f436809be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:18.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.398+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 shutdown_connections 2024-04-26T21:21:18.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.398+0000 7f436de45640 1 -- 172.21.15.40:0/1259856884 wait complete. 2024-04-26T21:21:18.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:21:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.561+0000 7fe76f6d1640 1 Processor -- start 2024-04-26T21:21:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 -- start start 2024-04-26T21:21:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe76806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe7680d3a30 0x7fe76809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7680d1240 con 0x7fe76809bf80 2024-04-26T21:21:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe7680d1e10 con 0x7fe7680d3a30 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.562+0000 7fe76f6d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680d14e0 con 0x7fe7680cee20 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe76dc6e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe76809bf80 0x7fe7681630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38680/0 (socket says 172.21.15.40:38680) 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe76dc6e640 1 -- 172.21.15.40:0/713925791 learned_addr learned my addr 172.21.15.40:0/713925791 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/713925791 >> v2:172.21.15.184:3300/0 conn(0x7fe7680d3a30 0x7fe76809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe76d46d640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe76806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 786098122 0 0) 0x7fe7680d1240 con 0x7fe76809bf80 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe754003520 con 0x7fe76809bf80 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1151081936 0 0) 0x7fe754003520 con 0x7fe76809bf80 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 msgr2=0x7fe76806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe76806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 >> v2:172.21.15.184:3300/0 conn(0x7fe7680d3a30 msgr2=0x7fe76809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 --2- 172.21.15.40:0/713925791 >> v2:172.21.15.184:3300/0 conn(0x7fe7680d3a30 0x7fe76809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680d17c0 con 0x7fe76809bf80 2024-04-26T21:21:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fe7600019f0 con 0x7fe76809bf80 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 >> v1:172.21.15.40:6789/0 conn(0x7fe76809bf80 legacy=0x7fe7681630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe7540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754003f30 0x7fe754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe754004d40 0x7fe7540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680d1e10 con 0x7fe754003f30 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680d14e0 con 0x7fe754004d40 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.563+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe75400b9f0 con 0x7fe7680cee20 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754003f30 0x7fe754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76d46d640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe7540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76dc6e640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe754004d40 0x7fe7540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 -- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 msgr2=0x7fe7540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe7680cee20 0x7fe7540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 -- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe754004d40 msgr2=0x7fe7540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe754004d40 0x7fe7540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.564+0000 7fe76cc6c640 1 -- 172.21.15.40:0/713925791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe754003520 con 0x7fe754003f30 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754003f30 0x7fe754004800 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe758009ac0 tx=0x7fe754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ffa759e6a3d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758036070 con 0x7fe754003f30 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe758031400 con 0x7fe754003f30 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe75ffff640 1 -- 172.21.15.40:0/713925791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758009690 con 0x7fe754003f30 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 -- 172.21.15.40:0/713925791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754003f30 msgr2=0x7fe754004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/713925791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754003f30 0x7fe754004800 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe758009ac0 tx=0x7fe754005830 comp rx=0 tx=0).stop 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76dc6e640 1 -- 172.21.15.40:0/713925791 reap_dead start 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 -- 172.21.15.40:0/713925791 shutdown_connections 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 -- 172.21.15.40:0/713925791 >> 172.21.15.40:0/713925791 conn(0x7fe7680d2f10 msgr2=0x7fe7680d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 -- 172.21.15.40:0/713925791 shutdown_connections 2024-04-26T21:21:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.565+0000 7fe76f6d1640 1 -- 172.21.15.40:0/713925791 wait complete. 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 Processor -- start 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 -- start start 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe754003f30 0x7fe76814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe76815c880 0x7fe768160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680cf200 con 0x7fe76815cd00 2024-04-26T21:21:18.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe76809c7a0 con 0x7fe76815c880 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76f6d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680cf380 con 0x7fe754003f30 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76dc6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76dc6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49284/0 (socket says 172.21.15.40:49284) 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.567+0000 7fe76dc6e640 1 -- 172.21.15.40:0/2543895834 learned_addr learned my addr 172.21.15.40:0/2543895834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76cc6c640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe76815c880 0x7fe768160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76d46d640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe754003f30 0x7fe76814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 -- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe754003f30 msgr2=0x7fe76814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe754003f30 0x7fe76814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 -- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe76815c880 msgr2=0x7fe768160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe76815c880 0x7fe768160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 -- 172.21.15.40:0/2543895834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76815d260 con 0x7fe76815cd00 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76dc6e640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe760002230 tx=0x7fe760007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760019070 con 0x7fe76815cd00 2024-04-26T21:21:18.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76000ac80 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76815d540 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760013610 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.568+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76815db00 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.569+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76806a290 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.570+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe76001c070 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.570+0000 7fe75dffb640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 0x7fe74c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.570+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5795+0+0 (secure 0 0 0) 0x7fe76008b630 con 0x7fe76815cd00 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.570+0000 7fe76d46d640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 0x7fe74c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:18.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.571+0000 7fe76d46d640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 0x7fe74c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe764009ae0 tx=0x7fe764009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:18.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.575+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7600593b0 con 0x7fe76815cd00 2024-04-26T21:21:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:18.759+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7680b8e90 con 0x7fe76815cd00 2024-04-26T21:21:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.755+0000 7fe75dffb640 1 -- 172.21.15.40:0/2543895834 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7fe76005d6b0 con 0x7fe76815cd00 2024-04-26T21:21:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:21:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.758+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 msgr2=0x7fe74c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.758+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 0x7fe74c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe764009ae0 tx=0x7fe764009190 comp rx=0 tx=0).stop 2024-04-26T21:21:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.758+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 msgr2=0x7fe7681614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.758+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe760002230 tx=0x7fe760007a60 comp rx=0 tx=0).stop 2024-04-26T21:21:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 shutdown_connections 2024-04-26T21:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe74c06c920 0x7fe74c06ede0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe76815cd00 0x7fe7681614c0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe76815c880 0x7fe768160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 --2- 172.21.15.40:0/2543895834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe754003f30 0x7fe76814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.759+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 >> 172.21.15.40:0/2543895834 conn(0x7fe7680d2f10 msgr2=0x7fe7680b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.761+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 shutdown_connections 2024-04-26T21:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.761+0000 7fe76f6d1640 1 -- 172.21.15.40:0/2543895834 wait complete. 2024-04-26T21:21:19.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:21:19.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.887+0000 7f9c270c4640 1 Processor -- start 2024-04-26T21:21:19.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.887+0000 7f9c270c4640 1 -- start start 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c270c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c200d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c270c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9c2006e330 0x7f9c200d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c270c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c200d0140 con 0x7f9c200d16d0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c270c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9c200721a0 con 0x7f9c2006e330 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c270c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c200d1b20 con 0x7f9c201454c0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c25661640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c200d16d0 0x7f9c2015c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38682/0 (socket says 172.21.15.40:38682) 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c25661640 1 -- 172.21.15.40:0/2126764855 learned_addr learned my addr 172.21.15.40:0/2126764855 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c1ffff640 1 --2- 172.21.15.40:0/2126764855 >> v2:172.21.15.184:3300/0 conn(0x7f9c2006e330 0x7f9c200d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.888+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 152090082 0 0) 0x7f9c200d0140 con 0x7f9c200d16d0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c08003520 con 0x7f9c200d16d0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c200d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 69769832 0 0) 0x7f9c08003520 con 0x7f9c200d16d0 2024-04-26T21:21:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 msgr2=0x7f9c200d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c200d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 >> v2:172.21.15.184:3300/0 conn(0x7f9c2006e330 msgr2=0x7f9c200d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 --2- 172.21.15.40:0/2126764855 >> v2:172.21.15.184:3300/0 conn(0x7f9c2006e330 0x7f9c200d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2006e830 con 0x7f9c200d16d0 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f9c10001670 con 0x7f9c200d16d0 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 >> v1:172.21.15.40:6789/0 conn(0x7f9c200d16d0 legacy=0x7f9c2015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c080039b0 0x7f9c08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c08004d40 0x7f9c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200d1b20 con 0x7f9c08004d40 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0800b9c0 con 0x7f9c080039b0 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0800bb40 con 0x7f9c201454c0 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c08004d40 0x7f9c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1ffff640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c080039b0 0x7f9c08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 -- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 msgr2=0x7f9c08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 -- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c080039b0 msgr2=0x7f9c08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c080039b0 0x7f9c08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c25661640 1 -- 172.21.15.40:0/2126764855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c08003520 con 0x7f9c08004d40 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c080039b0 0x7f9c08003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.889+0000 7f9c1ffff640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c201454c0 0x7f9c08004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.890+0000 7f9c25661640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c08004d40 0x7f9c080051c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f9c10007fd0 tx=0x7f9c08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21edc4dd7a78e6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.890+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007d50 con 0x7f9c08004d40 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.890+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c10005ad0 con 0x7f9c08004d40 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.890+0000 7f9c1f7fe640 1 -- 172.21.15.40:0/2126764855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007070 con 0x7f9c08004d40 2024-04-26T21:21:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 -- 172.21.15.40:0/2126764855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c08004d40 msgr2=0x7f9c080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2126764855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c08004d40 0x7f9c080051c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f9c10007fd0 tx=0x7f9c08005830 comp rx=0 tx=0).stop 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c25661640 1 -- 172.21.15.40:0/2126764855 reap_dead start 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 -- 172.21.15.40:0/2126764855 shutdown_connections 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 -- 172.21.15.40:0/2126764855 >> 172.21.15.40:0/2126764855 conn(0x7f9c2006c690 msgr2=0x7f9c2006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 -- 172.21.15.40:0/2126764855 shutdown_connections 2024-04-26T21:21:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.891+0000 7f9c270c4640 1 -- 172.21.15.40:0/2126764855 wait complete. 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.892+0000 7f9c270c4640 1 Processor -- start 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.892+0000 7f9c270c4640 1 -- start start 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c08004d40 0x7f9c2015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 0x7f9c2015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2006edf0 con 0x7f9c2015d8a0 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200ceca0 con 0x7f9c2013ff80 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c270c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2006f0d0 con 0x7f9c08004d40 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49288/0 (socket says 172.21.15.40:49288) 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 -- 172.21.15.40:0/2638252727 learned_addr learned my addr 172.21.15.40:0/2638252727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c1ffff640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 0x7f9c2015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c08004d40 0x7f9c2015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 -- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c08004d40 msgr2=0x7f9c2015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c08004d40 0x7f9c2015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 -- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 msgr2=0x7f9c2015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 0x7f9c2015b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 -- 172.21.15.40:0/2638252727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2013c7f0 con 0x7f9c2015d8a0 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c1ffff640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 0x7f9c2015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:19.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.893+0000 7f9c25661640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f9c10000c00 tx=0x7f9c2006ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.894+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007d50 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.894+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c10007280 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.894+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c20140480 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.894+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10001cb0 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.894+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c200cca80 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.895+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2015e260 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.895+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c10046020 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.895+0000 7f9c1d7fa640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 0x7f9c0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.896+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5357+0+0 (secure 0 0 0) 0x7f9c100ab4f0 con 0x7f9c2015d8a0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.896+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 0x7f9c0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.896+0000 7f9c24e60640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 0x7f9c0c06ede0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9c201406e0 tx=0x7f9c1800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:19.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:19.900+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c1004d020 con 0x7f9c2015d8a0 2024-04-26T21:21:20.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.089+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c2015d540 con 0x7f9c2015d8a0 2024-04-26T21:21:20.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.756+0000 7f9c1d7fa640 1 -- 172.21.15.40:0/2638252727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f9c2015d540 con 0x7f9c2015d8a0 2024-04-26T21:21:20.758 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:21:20.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.762+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 msgr2=0x7f9c0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:20.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.762+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 0x7f9c0c06ede0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9c201406e0 tx=0x7f9c1800d040 comp rx=0 tx=0).stop 2024-04-26T21:21:20.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.762+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 msgr2=0x7f9c2015dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:20.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.762+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f9c10000c00 tx=0x7f9c2006ea00 comp rx=0 tx=0).stop 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 shutdown_connections 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9c0c06c920 0x7f9c0c06ede0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c2015d8a0 0x7f9c2015dd20 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c2013ff80 0x7f9c2015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 --2- 172.21.15.40:0/2638252727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c08004d40 0x7f9c2015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.763+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 >> 172.21.15.40:0/2638252727 conn(0x7f9c2006c690 msgr2=0x7f9c20140970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.764+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 shutdown_connections 2024-04-26T21:21:20.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.765+0000 7f9c270c4640 1 -- 172.21.15.40:0/2638252727 wait complete. 2024-04-26T21:21:20.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-26T21:21:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.889+0000 7ff1b677f640 1 Processor -- start 2024-04-26T21:21:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.889+0000 7ff1b677f640 1 -- start start 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b677f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b015f4f0 0x7ff1b0073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b677f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1b0074030 0x7ff1b00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b677f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b015c530 con 0x7ff1b015b930 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b677f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b015c810 con 0x7ff1b0074030 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b677f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b015caf0 con 0x7ff1b015f4f0 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1b0074030 0x7ff1b00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1b0074030 0x7ff1b00cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40222/0 (socket says 172.21.15.40:40222) 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 -- 172.21.15.40:0/2770318369 learned_addr learned my addr 172.21.15.40:0/2770318369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 -- 172.21.15.40:0/2770318369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b015f4f0 msgr2=0x7ff1b0073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 --2- 172.21.15.40:0/2770318369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b015f4f0 0x7ff1b0073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 -- 172.21.15.40:0/2770318369 >> v1:172.21.15.184:6789/0 conn(0x7ff1b015b930 legacy=0x7ff1b015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.890+0000 7ff1b4d1c640 1 -- 172.21.15.40:0/2770318369 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b015cdd0 con 0x7ff1b0074030 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1b4d1c640 1 --2- 172.21.15.40:0/2770318369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b0074030 0x7ff1b00cf260 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009e50 tx=0x7ff1a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e127f0ac05b74ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1aeffd640 1 -- 172.21.15.40:0/2770318369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8036070 con 0x7ff1b0074030 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1aeffd640 1 -- 172.21.15.40:0/2770318369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8031600 con 0x7ff1b0074030 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1aeffd640 1 -- 172.21.15.40:0/2770318369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80319d0 con 0x7ff1b0074030 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1b677f640 1 -- 172.21.15.40:0/2770318369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b0074030 msgr2=0x7ff1b00cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.891+0000 7ff1b677f640 1 --2- 172.21.15.40:0/2770318369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b0074030 0x7ff1b00cf260 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009e50 tx=0x7ff1a802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 -- 172.21.15.40:0/2770318369 shutdown_connections 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 --2- 172.21.15.40:0/2770318369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b0074030 0x7ff1b00cf260 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 --2- 172.21.15.40:0/2770318369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b015f4f0 0x7ff1b0073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 -- 172.21.15.40:0/2770318369 >> 172.21.15.40:0/2770318369 conn(0x7ff1b00cd8a0 msgr2=0x7ff1b00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 -- 172.21.15.40:0/2770318369 shutdown_connections 2024-04-26T21:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.892+0000 7ff1b677f640 1 -- 172.21.15.40:0/2770318369 wait complete. 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.893+0000 7ff1b677f640 1 Processor -- start 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.893+0000 7ff1b677f640 1 -- start start 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 0x7ff1b01f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 0x7ff1b00afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b006f340 con 0x7ff1b015b930 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b006fdd0 con 0x7ff1b015f4f0 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b677f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b006f4c0 con 0x7ff1b0074030 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40224/0 (socket says 172.21.15.40:40224) 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 -- 172.21.15.40:0/3128849576 learned_addr learned my addr 172.21.15.40:0/3128849576 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b4d1c640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 0x7ff1b00afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1affff640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 0x7ff1b01f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 -- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 msgr2=0x7ff1b01f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 0x7ff1b01f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 -- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 msgr2=0x7ff1b00afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 0x7ff1b00afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 -- 172.21.15.40:0/3128849576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b01f32b0 con 0x7ff1b015b930 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1b4d1c640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 0x7ff1b00afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1affff640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 0x7ff1b01f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1af7fe640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b410 tx=0x7ff1a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4015020 con 0x7ff1b015b930 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.894+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a4011850 con 0x7ff1b015b930 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.895+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b00aaa70 con 0x7ff1b015b930 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.895+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011c40 con 0x7ff1b015b930 2024-04-26T21:21:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.895+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b00ab030 con 0x7ff1b015b930 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.895+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b006a290 con 0x7ff1b015b930 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.896+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1a4018020 con 0x7ff1b015b930 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.897+0000 7ff1acff9640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 0x7ff19c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.897+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7ff1a408b730 con 0x7ff1b015b930 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.897+0000 7ff1affff640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 0x7ff19c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.897+0000 7ff1affff640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 0x7ff19c06ede0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0003eb0 tx=0x7ff1a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:20.901+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1a401f020 con 0x7ff1b015b930 2024-04-26T21:21:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.089+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7ff1b006d2b0 con 0x7ff1b015b930 2024-04-26T21:21:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.092+0000 7ff1acff9640 1 -- 172.21.15.40:0/3128849576 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v61) ==== 111+0+0 (secure 0 0 0) 0x7ff1a40172c0 con 0x7ff1b015b930 2024-04-26T21:21:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 msgr2=0x7ff19c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 0x7ff19c06ede0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0003eb0 tx=0x7ff1a0009040 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 msgr2=0x7ff1b01f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b410 tx=0x7ff1a400b8d0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 shutdown_connections 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff19c06c920 0x7ff19c06ede0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1b015f4f0 0x7ff1b00afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1b015b930 0x7ff1b01f5a60 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 --2- 172.21.15.40:0/3128849576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1b0074030 0x7ff1b01f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 >> 172.21.15.40:0/3128849576 conn(0x7ff1b00cd8a0 msgr2=0x7ff1b00b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 shutdown_connections 2024-04-26T21:21:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.094+0000 7ff1b677f640 1 -- 172.21.15.40:0/3128849576 wait complete. 2024-04-26T21:21:21.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:21:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.207+0000 7fb450566640 1 Processor -- start 2024-04-26T21:21:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.207+0000 7fb450566640 1 -- start start 2024-04-26T21:21:21.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb450566640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4480afe10 0x7fb4480cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb450566640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb44815d4c0 0x7fb4480b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb450566640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb448074430 con 0x7fb448069ae0 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb450566640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb4480746f0 con 0x7fb44815d4c0 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb450566640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4480ae140 con 0x7fb4480afe10 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.208+0000 7fb44db01640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb44815d4c0 0x7fb4480b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44eb03640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb448069ae0 0x7fb4480d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55150/0 (socket says 172.21.15.40:55150) 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44eb03640 1 -- 172.21.15.40:0/3574540725 learned_addr learned my addr 172.21.15.40:0/3574540725 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44e302640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4480afe10 0x7fb4480cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44db01640 1 -- 172.21.15.40:0/3574540725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4480afe10 msgr2=0x7fb4480cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44db01640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4480afe10 0x7fb4480cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44db01640 1 -- 172.21.15.40:0/3574540725 >> v1:172.21.15.40:6789/0 conn(0x7fb448069ae0 legacy=0x7fb4480d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44d300640 1 -- 172.21.15.40:0/3574540725 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362770654 0 0) 0x7fb448074430 con 0x7fb448069ae0 2024-04-26T21:21:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.209+0000 7fb44db01640 1 -- 172.21.15.40:0/3574540725 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4480ae420 con 0x7fb44815d4c0 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb44db01640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb44815d4c0 0x7fb4480b01f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb4380097a0 tx=0x7fb43802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c054e57889fa541b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb44d300640 1 -- 172.21.15.40:0/3574540725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438037070 con 0x7fb44815d4c0 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb44d300640 1 -- 172.21.15.40:0/3574540725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb438032440 con 0x7fb44815d4c0 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb44d300640 1 -- 172.21.15.40:0/3574540725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438032810 con 0x7fb44815d4c0 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 -- 172.21.15.40:0/3574540725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb44815d4c0 msgr2=0x7fb4480b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb44815d4c0 0x7fb4480b01f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb4380097a0 tx=0x7fb43802a180 comp rx=0 tx=0).stop 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 -- 172.21.15.40:0/3574540725 shutdown_connections 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb44815d4c0 0x7fb4480b01f0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 --2- 172.21.15.40:0/3574540725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4480afe10 0x7fb4480cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 -- 172.21.15.40:0/3574540725 >> 172.21.15.40:0/3574540725 conn(0x7fb448073910 msgr2=0x7fb448073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 -- 172.21.15.40:0/3574540725 shutdown_connections 2024-04-26T21:21:21.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.210+0000 7fb450566640 1 -- 172.21.15.40:0/3574540725 wait complete. 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 Processor -- start 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 -- start start 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb448069ae0 0x7fb448154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 0x7fb44814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb44806f8e0 con 0x7fb4480afe10 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4480d1140 con 0x7fb448069ae0 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb450566640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4480ce100 con 0x7fb44815d4c0 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.212+0000 7fb44db01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40252/0 (socket says 172.21.15.40:40252) 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 -- 172.21.15.40:0/4179167890 learned_addr learned my addr 172.21.15.40:0/4179167890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44eb03640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 0x7fb44814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44e302640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb448069ae0 0x7fb448154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 -- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 msgr2=0x7fb44814d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 0x7fb44814d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 -- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb448069ae0 msgr2=0x7fb448154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb448069ae0 0x7fb448154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 -- 172.21.15.40:0/4179167890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4480cee70 con 0x7fb4480afe10 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44eb03640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 0x7fb44814d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:21.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb44db01640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438037050 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb438038040 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438004470 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4480cf150 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.213+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4480cf710 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.214+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4480b40b0 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.215+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb438030980 con 0x7fb4480afe10 2024-04-26T21:21:21.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.215+0000 7fb43effd640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 0x7fb42c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.215+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fb4380ac3c0 con 0x7fb4480afe10 2024-04-26T21:21:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.215+0000 7fb44e302640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 0x7fb42c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.216+0000 7fb44e302640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 0x7fb42c06edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb44000a970 tx=0x7fb440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.219+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4480b40b0 con 0x7fb4480afe10 2024-04-26T21:21:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.404+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb44806a290 con 0x7fb4480afe10 2024-04-26T21:21:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.408+0000 7fb43effd640 1 -- 172.21.15.40:0/4179167890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v62) ==== 104+0+0 (secure 0 0 0) 0x7fb43807ab80 con 0x7fb4480afe10 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 msgr2=0x7fb42c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 0x7fb42c06edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb44000a970 tx=0x7fb440008040 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 msgr2=0x7fb44814ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb4380046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 shutdown_connections 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb42c06c8f0 0x7fb42c06edb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb44815d4c0 0x7fb44814d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4480afe10 0x7fb44814ce20 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 --2- 172.21.15.40:0/4179167890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb448069ae0 0x7fb448154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 >> 172.21.15.40:0/4179167890 conn(0x7fb448073910 msgr2=0x7fb4480b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 shutdown_connections 2024-04-26T21:21:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.410+0000 7fb450566640 1 -- 172.21.15.40:0/4179167890 wait complete. 2024-04-26T21:21:21.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-26T21:21:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.523+0000 7f57da0f8640 1 Processor -- start 2024-04-26T21:21:21.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.523+0000 7f57da0f8640 1 -- start start 2024-04-26T21:21:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57da0f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 0x7f57d40d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57da0f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 0x7f57d40ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57da0f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d40d4240 con 0x7f57d40d0ff0 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57da0f8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f57d40d31a0 con 0x7f57d40d22f0 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57da0f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40d3480 con 0x7f57d415f490 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57d37fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 0x7f57d40d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.524+0000 7f57d3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 0x7f57d40ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d2ffd640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f57d40d0ff0 0x7f57d40d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:53148/0 (socket says 172.21.15.40:53148) 2024-04-26T21:21:21.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d2ffd640 1 -- 172.21.15.40:0/1671468479 learned_addr learned my addr 172.21.15.40:0/1671468479 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d37fe640 1 -- 172.21.15.40:0/1671468479 >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 msgr2=0x7f57d40ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d37fe640 1 --2- 172.21.15.40:0/1671468479 >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 0x7f57d40ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d37fe640 1 -- 172.21.15.40:0/1671468479 >> v1:172.21.15.184:6789/0 conn(0x7f57d40d0ff0 legacy=0x7f57d40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d37fe640 1 -- 172.21.15.40:0/1671468479 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40d3760 con 0x7f57d415f490 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.525+0000 7f57d3fff640 1 --2- 172.21.15.40:0/1671468479 >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 0x7f57d40ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57d37fe640 1 --2- 172.21.15.40:0/1671468479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 0x7f57d40d0ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097a0 tx=0x7f57c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee356995b07bcc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57d27fc640 1 -- 172.21.15.40:0/1671468479 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4037070 con 0x7f57d415f490 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57d27fc640 1 -- 172.21.15.40:0/1671468479 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4032440 con 0x7f57d415f490 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57d27fc640 1 -- 172.21.15.40:0/1671468479 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4032810 con 0x7f57d415f490 2024-04-26T21:21:21.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 -- 172.21.15.40:0/1671468479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 msgr2=0x7f57d40d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 --2- 172.21.15.40:0/1671468479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 0x7f57d40d0ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097a0 tx=0x7f57c402a180 comp rx=0 tx=0).stop 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 -- 172.21.15.40:0/1671468479 shutdown_connections 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 --2- 172.21.15.40:0/1671468479 >> v2:172.21.15.184:3300/0 conn(0x7f57d40d22f0 0x7f57d40ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 --2- 172.21.15.40:0/1671468479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d415f490 0x7f57d40d0ab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 -- 172.21.15.40:0/1671468479 >> 172.21.15.40:0/1671468479 conn(0x7f57d409e530 msgr2=0x7f57d40d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.526+0000 7f57da0f8640 1 -- 172.21.15.40:0/1671468479 shutdown_connections 2024-04-26T21:21:21.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.527+0000 7f57da0f8640 1 -- 172.21.15.40:0/1671468479 wait complete. 2024-04-26T21:21:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.528+0000 7f57da0f8640 1 Processor -- start 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.528+0000 7f57da0f8640 1 -- start start 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.528+0000 7f57da0f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40d0ff0 0x7f57d40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d415f490 0x7f57d40b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57d415f870 con 0x7f57d40d22f0 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d3d20 con 0x7f57d415f490 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40d2dc0 con 0x7f57d40d0ff0 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40260/0 (socket says 172.21.15.40:40260) 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 -- 172.21.15.40:0/3941943579 learned_addr learned my addr 172.21.15.40:0/3941943579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d37fe640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40d0ff0 0x7f57d40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d3fff640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d415f490 0x7f57d40b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 -- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40d0ff0 msgr2=0x7f57d40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40d0ff0 0x7f57d40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 -- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d415f490 msgr2=0x7f57d40b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d415f490 0x7f57d40b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 -- 172.21.15.40:0/3941943579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40b9730 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57d2ffd640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c440 tx=0x7f57c800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8017070 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c8005ce0 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d40b9a10 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80072f0 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.529+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d40b9430 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.530+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d406a290 con 0x7f57d40d22f0 2024-04-26T21:21:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.531+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57c801a070 con 0x7f57d40d22f0 2024-04-26T21:21:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.531+0000 7f57bbfff640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 0x7f57c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.531+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f57c808aa50 con 0x7f57d40d22f0 2024-04-26T21:21:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.532+0000 7f57d37fe640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 0x7f57c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.532+0000 7f57d37fe640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 0x7f57c006edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f57c4034000 tx=0x7f57c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.536+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57c8058b30 con 0x7f57d40d22f0 2024-04-26T21:21:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.720+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f57d415fb60 con 0x7f57d40d22f0 2024-04-26T21:21:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.726+0000 7f57bbfff640 1 -- 172.21.15.40:0/3941943579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v63) ==== 102+0+0 (secure 0 0 0) 0x7f57c805ce30 con 0x7f57d40d22f0 2024-04-26T21:21:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 msgr2=0x7f57c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 0x7f57c006edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f57c4034000 tx=0x7f57c402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 msgr2=0x7f57d415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c440 tx=0x7f57c800c840 comp rx=0 tx=0).stop 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 shutdown_connections 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f57c006c8f0 0x7f57c006edb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d415f490 0x7f57d40b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40d22f0 0x7f57d415d6a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 --2- 172.21.15.40:0/3941943579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57d40d0ff0 0x7f57d40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 >> 172.21.15.40:0/3941943579 conn(0x7f57d409e530 msgr2=0x7f57d40b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 shutdown_connections 2024-04-26T21:21:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.728+0000 7f57da0f8640 1 -- 172.21.15.40:0/3941943579 wait complete. 2024-04-26T21:21:21.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-26T21:21:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.842+0000 7f374fd5f640 1 Processor -- start 2024-04-26T21:21:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 -- start start 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374815f680 0x7f37480d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 0x7f37480b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37480b60b0 0x7f37480d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f374815fa60 con 0x7f37480b60b0 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f37480d3f50 con 0x7f3748163aa0 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.843+0000 7f374fd5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37480b5090 con 0x7f374815f680 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37480b60b0 0x7f37480d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37480b60b0 0x7f37480d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40264/0 (socket says 172.21.15.40:40264) 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 -- 172.21.15.40:0/2187093308 learned_addr learned my addr 172.21.15.40:0/2187093308 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374dafb640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374815f680 0x7f37480d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374d2fa640 1 --2- 172.21.15.40:0/2187093308 >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 0x7f37480b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 -- 172.21.15.40:0/2187093308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374815f680 msgr2=0x7f37480d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374815f680 0x7f37480d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 -- 172.21.15.40:0/2187093308 >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 msgr2=0x7f37480b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 --2- 172.21.15.40:0/2187093308 >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 0x7f37480b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374e2fc640 1 -- 172.21.15.40:0/2187093308 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37480b5370 con 0x7f37480b60b0 2024-04-26T21:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.844+0000 7f374d2fa640 1 --2- 172.21.15.40:0/2187093308 >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 0x7f37480b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374e2fc640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480b60b0 0x7f37480d4f20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f374808fae0 tx=0x7f374002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9731edb9c1174c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374caf9640 1 -- 172.21.15.40:0/2187093308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740036070 con 0x7f37480b60b0 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374caf9640 1 -- 172.21.15.40:0/2187093308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740031600 con 0x7f37480b60b0 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374caf9640 1 -- 172.21.15.40:0/2187093308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37400319d0 con 0x7f37480b60b0 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 -- 172.21.15.40:0/2187093308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480b60b0 msgr2=0x7f37480d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480b60b0 0x7f37480d4f20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f374808fae0 tx=0x7f374002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 -- 172.21.15.40:0/2187093308 shutdown_connections 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480b60b0 0x7f37480d4f20 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 --2- 172.21.15.40:0/2187093308 >> v2:172.21.15.184:3300/0 conn(0x7f3748163aa0 0x7f37480b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 --2- 172.21.15.40:0/2187093308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f374815f680 0x7f37480d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 -- 172.21.15.40:0/2187093308 >> 172.21.15.40:0/2187093308 conn(0x7f374808eca0 msgr2=0x7f374809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.845+0000 7f374fd5f640 1 -- 172.21.15.40:0/2187093308 shutdown_connections 2024-04-26T21:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.846+0000 7f374fd5f640 1 -- 172.21.15.40:0/2187093308 wait complete. 2024-04-26T21:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 Processor -- start 2024-04-26T21:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 -- start start 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37480b60b0 0x7f37480b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 0x7f37480cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37480d55e0 con 0x7f3748163aa0 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37480d5460 con 0x7f374815f680 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374fd5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f37480d5760 con 0x7f37480b60b0 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.847+0000 7f374e2fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40276/0 (socket says 172.21.15.40:40276) 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 -- 172.21.15.40:0/752225949 learned_addr learned my addr 172.21.15.40:0/752225949 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374dafb640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37480b60b0 0x7f37480b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374d2fa640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 0x7f37480cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 -- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37480b60b0 msgr2=0x7f37480b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37480b60b0 0x7f37480b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 -- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 msgr2=0x7f37480cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 0x7f37480cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 -- 172.21.15.40:0/752225949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37480cf140 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374d2fa640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 0x7f37480cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374e2fc640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3740034040 tx=0x7f3740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740036050 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740042070 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37480b83d0 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740009510 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.848+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37480b8990 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.849+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374806a290 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.850+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3740007bd0 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.850+0000 7f373e7fc640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.850+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f37400ac600 con 0x7f3748163aa0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.850+0000 7f374dafb640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.851+0000 7f374dafb640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 0x7f372c06edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3744006fd0 tx=0x7f3744008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:21.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:21.854+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f374007ad30 con 0x7f3748163aa0 2024-04-26T21:21:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.041+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f37480b4cd0 con 0x7f3748163aa0 2024-04-26T21:21:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.044+0000 7f373e7fc640 1 -- 172.21.15.40:0/752225949 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v64) ==== 105+0+0 (secure 0 0 0) 0x7f374007a630 con 0x7f3748163aa0 2024-04-26T21:21:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 msgr2=0x7f372c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 0x7f372c06edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3744006fd0 tx=0x7f3744008040 comp rx=0 tx=0).stop 2024-04-26T21:21:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 msgr2=0x7f3748153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3740034040 tx=0x7f3740004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 shutdown_connections 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3748163aa0 0x7f3748153660 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f374815f680 0x7f37480cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 --2- 172.21.15.40:0/752225949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37480b60b0 0x7f37480b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 >> 172.21.15.40:0/752225949 conn(0x7f374808eca0 msgr2=0x7f37480b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 shutdown_connections 2024-04-26T21:21:22.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.047+0000 7f374fd5f640 1 -- 172.21.15.40:0/752225949 wait complete. 2024-04-26T21:21:22.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-26T21:21:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.162+0000 7fb4c6524640 1 Processor -- start 2024-04-26T21:21:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.162+0000 7fb4c6524640 1 -- start start 2024-04-26T21:21:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4c6524640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4c015f680 0x7fb4c00d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4c6524640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4c00d4f20 0x7fb4c00d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4c6524640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4c00d58c0 con 0x7fb4c0163aa0 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4c6524640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c015fa60 con 0x7fb4c015f680 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4c6524640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c00d3f50 con 0x7fb4c00d4f20 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4bffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4c015f680 0x7fb4c00d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4bffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb4c015f680 0x7fb4c00d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40290/0 (socket says 172.21.15.40:40290) 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.163+0000 7fb4bffff640 1 -- 172.21.15.40:0/847749291 learned_addr learned my addr 172.21.15.40:0/847749291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.164+0000 7fb4bffff640 1 -- 172.21.15.40:0/847749291 >> v2:172.21.15.184:3300/0 conn(0x7fb4c00d4f20 msgr2=0x7fb4c00d5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.164+0000 7fb4bffff640 1 --2- 172.21.15.40:0/847749291 >> v2:172.21.15.184:3300/0 conn(0x7fb4c00d4f20 0x7fb4c00d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.164+0000 7fb4bffff640 1 -- 172.21.15.40:0/847749291 >> v1:172.21.15.184:6789/0 conn(0x7fb4c0163aa0 legacy=0x7fb4c00b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.164+0000 7fb4bffff640 1 -- 172.21.15.40:0/847749291 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c00b5090 con 0x7fb4c015f680 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.164+0000 7fb4bffff640 1 --2- 172.21.15.40:0/847749291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c015f680 0x7fb4c00d3820 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cf840 tx=0x7fb4b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75dae51914b750c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4beffd640 1 -- 172.21.15.40:0/847749291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0037070 con 0x7fb4c015f680 2024-04-26T21:21:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4beffd640 1 -- 172.21.15.40:0/847749291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0032440 con 0x7fb4c015f680 2024-04-26T21:21:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4beffd640 1 -- 172.21.15.40:0/847749291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0032810 con 0x7fb4c015f680 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 -- 172.21.15.40:0/847749291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c015f680 msgr2=0x7fb4c00d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 --2- 172.21.15.40:0/847749291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c015f680 0x7fb4c00d3820 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cf840 tx=0x7fb4b002a230 comp rx=0 tx=0).stop 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 -- 172.21.15.40:0/847749291 shutdown_connections 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 --2- 172.21.15.40:0/847749291 >> v2:172.21.15.184:3300/0 conn(0x7fb4c00d4f20 0x7fb4c00d5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 --2- 172.21.15.40:0/847749291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c015f680 0x7fb4c00d3820 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 -- 172.21.15.40:0/847749291 >> 172.21.15.40:0/847749291 conn(0x7fb4c008eca0 msgr2=0x7fb4c009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 -- 172.21.15.40:0/847749291 shutdown_connections 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.165+0000 7fb4c6524640 1 -- 172.21.15.40:0/847749291 wait complete. 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.166+0000 7fb4c6524640 1 Processor -- start 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.166+0000 7fb4c6524640 1 -- start start 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c015f680 0x7fb4c00ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c0163aa0 0x7fb4c014d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0163f30 con 0x7fb4c00d4f20 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c00b6280 con 0x7fb4c015f680 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4c6524640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c00cfd80 con 0x7fb4c0163aa0 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40294/0 (socket says 172.21.15.40:40294) 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.167+0000 7fb4bffff640 1 -- 172.21.15.40:0/1263131572 learned_addr learned my addr 172.21.15.40:0/1263131572 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 -- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c0163aa0 msgr2=0x7fb4c014d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c0163aa0 0x7fb4c014d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 -- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c015f680 msgr2=0x7fb4c00ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c015f680 0x7fb4c00ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 -- 172.21.15.40:0/1263131572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c00cf130 con 0x7fb4c00d4f20 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bffff640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002a270 tx=0x7fb4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0041070 con 0x7fb4c00d4f20 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b003c890 con 0x7fb4c00d4f20 2024-04-26T21:21:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0037040 con 0x7fb4c00d4f20 2024-04-26T21:21:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c014df50 con 0x7fb4c00d4f20 2024-04-26T21:21:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.168+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c014e510 con 0x7fb4c00d4f20 2024-04-26T21:21:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.169+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c006a290 con 0x7fb4c00d4f20 2024-04-26T21:21:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.169+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4b0044070 con 0x7fb4c00d4f20 2024-04-26T21:21:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.170+0000 7fb4bcff9640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 0x7fb4ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.170+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fb4b00ac4d0 con 0x7fb4c00d4f20 2024-04-26T21:21:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.170+0000 7fb4bf7fe640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 0x7fb4ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.171+0000 7fb4bf7fe640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 0x7fb4ac06ede0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cc880 tx=0x7fb4b4006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.174+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4b007adb0 con 0x7fb4c00d4f20 2024-04-26T21:21:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.361+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7fb4c00b4cd0 con 0x7fb4c00d4f20 2024-04-26T21:21:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.365+0000 7fb4bcff9640 1 -- 172.21.15.40:0/1263131572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v65) ==== 107+0+0 (secure 0 0 0) 0x7fb4b007a6b0 con 0x7fb4c00d4f20 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 msgr2=0x7fb4ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 0x7fb4ac06ede0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cc880 tx=0x7fb4b4006ba0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 msgr2=0x7fb4c00cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002a270 tx=0x7fb4b00046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 shutdown_connections 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb4ac06c920 0x7fb4ac06ede0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c0163aa0 0x7fb4c014d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c015f680 0x7fb4c00ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 --2- 172.21.15.40:0/1263131572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4c00d4f20 0x7fb4c00cc340 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 >> 172.21.15.40:0/1263131572 conn(0x7fb4c008eca0 msgr2=0x7fb4c00b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 shutdown_connections 2024-04-26T21:21:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.367+0000 7fb4c6524640 1 -- 172.21.15.40:0/1263131572 wait complete. 2024-04-26T21:21:22.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-04-26T21:21:22.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.479+0000 7f01d0538640 1 Processor -- start 2024-04-26T21:21:22.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.479+0000 7f01d0538640 1 -- start start 2024-04-26T21:21:22.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01d0538640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01c815f720 0x7f01c80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01d0538640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 0x7f01c80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01d0538640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c80b2510 con 0x7f01c815f340 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01d0538640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f01c80b27f0 con 0x7f01c815f720 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01d0538640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f01c80b2ad0 con 0x7f01c80d5610 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01cdad3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01c815f720 0x7f01c80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01cdad3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01c815f720 0x7f01c80b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40308/0 (socket says 172.21.15.40:40308) 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01cdad3640 1 -- 172.21.15.40:0/1921474370 learned_addr learned my addr 172.21.15.40:0/1921474370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.480+0000 7f01cead5640 1 --2- 172.21.15.40:0/1921474370 >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 0x7f01c80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cdad3640 1 -- 172.21.15.40:0/1921474370 >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 msgr2=0x7f01c80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cdad3640 1 --2- 172.21.15.40:0/1921474370 >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 0x7f01c80d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cdad3640 1 -- 172.21.15.40:0/1921474370 >> v1:172.21.15.184:6789/0 conn(0x7f01c815f340 legacy=0x7f01c80d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cdad3640 1 -- 172.21.15.40:0/1921474370 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c80b2db0 con 0x7f01c815f720 2024-04-26T21:21:22.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cead5640 1 --2- 172.21.15.40:0/1921474370 >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 0x7f01c80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:22.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cdad3640 1 --2- 172.21.15.40:0/1921474370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b1e80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f01c80d4c20 tx=0x7f01bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f6491efe9fd178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cd2d2640 1 -- 172.21.15.40:0/1921474370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc037070 con 0x7f01c815f720 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.481+0000 7f01cd2d2640 1 -- 172.21.15.40:0/1921474370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc032440 con 0x7f01c815f720 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01cd2d2640 1 -- 172.21.15.40:0/1921474370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc032810 con 0x7f01c815f720 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 -- 172.21.15.40:0/1921474370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 msgr2=0x7f01c80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 --2- 172.21.15.40:0/1921474370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b1e80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f01c80d4c20 tx=0x7f01bc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 -- 172.21.15.40:0/1921474370 shutdown_connections 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 --2- 172.21.15.40:0/1921474370 >> v2:172.21.15.184:3300/0 conn(0x7f01c80d5610 0x7f01c80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 --2- 172.21.15.40:0/1921474370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b1e80 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 -- 172.21.15.40:0/1921474370 >> 172.21.15.40:0/1921474370 conn(0x7f01c808ec70 msgr2=0x7f01c809e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 -- 172.21.15.40:0/1921474370 shutdown_connections 2024-04-26T21:21:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.482+0000 7f01d0538640 1 -- 172.21.15.40:0/1921474370 wait complete. 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.483+0000 7f01d0538640 1 Processor -- start 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.483+0000 7f01d0538640 1 -- start start 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 0x7f01c815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01c815f340 0x7f01c80b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80b3940 con 0x7f01c815f720 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80b37c0 con 0x7f01c80d5610 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01d0538640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f01c80b3ac0 con 0x7f01c815f340 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01cead5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01cead5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40310/0 (socket says 172.21.15.40:40310) 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01cead5640 1 -- 172.21.15.40:0/3983311539 learned_addr learned my addr 172.21.15.40:0/3983311539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01ce2d4640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 0x7f01c815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.484+0000 7f01cdad3640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01c815f340 0x7f01c80b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 -- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01c815f340 msgr2=0x7f01c80b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01c815f340 0x7f01c80b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 -- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 msgr2=0x7f01c815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 0x7f01c815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 -- 172.21.15.40:0/3983311539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c80b8690 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01ce2d4640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 0x7f01c815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01cead5640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f01b800ced0 tx=0x7f01b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8003b90 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8003d50 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c80b8970 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8004260 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.485+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c80b8f30 con 0x7f01c815f720 2024-04-26T21:21:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.486+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c806a290 con 0x7f01c815f720 2024-04-26T21:21:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.486+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01b8004400 con 0x7f01c815f720 2024-04-26T21:21:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.487+0000 7f01c6ffd640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.487+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f01b808c1d0 con 0x7f01c815f720 2024-04-26T21:21:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.487+0000 7f01ce2d4640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.488+0000 7f01ce2d4640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 0x7f01b406edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f01c0003eb0 tx=0x7f01c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.491+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01b805a900 con 0x7f01c815f720 2024-04-26T21:21:22.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.745+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f01c80b2020 con 0x7f01c815f720 2024-04-26T21:21:22.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.775+0000 7f01c6ffd640 1 -- 172.21.15.40:0/3983311539 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v489) ==== 148+0+0 (secure 0 0 0) 0x7f01b808ae20 con 0x7f01c815f720 2024-04-26T21:21:22.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 msgr2=0x7f01b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 0x7f01b406edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f01c0003eb0 tx=0x7f01c0009040 comp rx=0 tx=0).stop 2024-04-26T21:21:22.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 msgr2=0x7f01c80b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f01b800ced0 tx=0x7f01b800a710 comp rx=0 tx=0).stop 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 shutdown_connections 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01c815f720 0x7f01c80b9b30 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01c815f340 0x7f01c80b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 --2- 172.21.15.40:0/3983311539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c80d5610 0x7f01c815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 >> 172.21.15.40:0/3983311539 conn(0x7f01c808ec70 msgr2=0x7f01c80b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 shutdown_connections 2024-04-26T21:21:22.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.777+0000 7f01d0538640 1 -- 172.21.15.40:0/3983311539 wait complete. 2024-04-26T21:21:22.789 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-26T21:21:22.789 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2024-04-26T21:21:22.789 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-26T21:21:22.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.889+0000 7fd494fb6640 1 Processor -- start 2024-04-26T21:21:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.889+0000 7fd494fb6640 1 -- start start 2024-04-26T21:21:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd494fb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4900cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd494fb6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd4900d3ee0 0x7fd4900b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd494fb6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4900d03a0 con 0x7fd49015f750 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd494fb6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fd4900b4d10 con 0x7fd4900d3ee0 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd494fb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4900d0680 con 0x7fd49015f370 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd48dd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd49015f750 0x7fd4900d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55166/0 (socket says 172.21.15.40:55166) 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.890+0000 7fd48dd9b640 1 -- 172.21.15.40:0/1441539053 learned_addr learned my addr 172.21.15.40:0/1441539053 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910786495 0 0) 0x7fd4900d03a0 con 0x7fd49015f750 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/1441539053 >> v2:172.21.15.184:3300/0 conn(0x7fd4900d3ee0 0x7fd4900b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd478003520 con 0x7fd49015f750 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48e59c640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4900cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 402230366 0 0) 0x7fd478003520 con 0x7fd49015f750 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 msgr2=0x7fd4900cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4900cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 >> v2:172.21.15.184:3300/0 conn(0x7fd4900d3ee0 msgr2=0x7fd4900b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 --2- 172.21.15.40:0/1441539053 >> v2:172.21.15.184:3300/0 conn(0x7fd4900d3ee0 0x7fd4900b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4900d0960 con 0x7fd49015f750 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fd484001670 con 0x7fd49015f750 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 >> v1:172.21.15.40:6789/0 conn(0x7fd49015f750 legacy=0x7fd4900d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478003f30 0x7fd478004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd478004d40 0x7fd4780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4900b4d10 con 0x7fd478004d40 2024-04-26T21:21:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4900d0680 con 0x7fd478003f30 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd47800ba10 con 0x7fd49015f370 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd478004d40 0x7fd4780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48e59c640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48dd9b640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478003f30 0x7fd478004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 -- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 msgr2=0x7fd4780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd49015f370 0x7fd4780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 -- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478003f30 msgr2=0x7fd478004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478003f30 0x7fd478004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48ed9d640 1 -- 172.21.15.40:0/1441539053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd478003520 con 0x7fd478004d40 2024-04-26T21:21:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.891+0000 7fd48dd9b640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478003f30 0x7fd478004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.892+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd478004d40 0x7fd4780051c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd480009af0 tx=0x7fd478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8179ea12d3b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.892+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480036070 con 0x7fd478004d40 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.892+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd480031400 con 0x7fd478004d40 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.892+0000 7fd48d59a640 1 -- 172.21.15.40:0/1441539053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480009690 con 0x7fd478004d40 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 -- 172.21.15.40:0/1441539053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd478004d40 msgr2=0x7fd4780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 --2- 172.21.15.40:0/1441539053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd478004d40 0x7fd4780051c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd480009af0 tx=0x7fd478005830 comp rx=0 tx=0).stop 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd48ed9d640 1 -- 172.21.15.40:0/1441539053 reap_dead start 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 -- 172.21.15.40:0/1441539053 shutdown_connections 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 -- 172.21.15.40:0/1441539053 >> 172.21.15.40:0/1441539053 conn(0x7fd49008ec10 msgr2=0x7fd49009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 -- 172.21.15.40:0/1441539053 shutdown_connections 2024-04-26T21:21:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.893+0000 7fd494fb6640 1 -- 172.21.15.40:0/1441539053 wait complete. 2024-04-26T21:21:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.894+0000 7fd494fb6640 1 Processor -- start 2024-04-26T21:21:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.894+0000 7fd494fb6640 1 -- start start 2024-04-26T21:21:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478004d40 0x7fd49015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4900b8f30 0x7fd4901f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4900cfd70 con 0x7fd4900ca860 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4900d0f20 con 0x7fd478004d40 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd494fb6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd4900cfef0 con 0x7fd4900b8f30 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40326/0 (socket says 172.21.15.40:40326) 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 -- 172.21.15.40:0/2860388333 learned_addr learned my addr 172.21.15.40:0/2860388333 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48e59c640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478004d40 0x7fd49015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48dd9b640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4900b8f30 0x7fd4901f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 -- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4900b8f30 msgr2=0x7fd4901f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4900b8f30 0x7fd4901f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 -- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478004d40 msgr2=0x7fd49015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478004d40 0x7fd49015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.895+0000 7fd48ed9d640 1 -- 172.21.15.40:0/2860388333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4901f5f50 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd48ed9d640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd480009c10 tx=0x7fd480004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48003f020 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd480003610 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4900cae00 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480036040 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.896+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4900cb3c0 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.897+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49006a290 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.897+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd480042020 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.898+0000 7fd47effd640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 0x7fd47006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.898+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fd4800aab60 con 0x7fd4900ca860 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.898+0000 7fd48e59c640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 0x7fd47006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.898+0000 7fd48e59c640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 0x7fd47006ede0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd4900cbd20 tx=0x7fd48800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:22.902+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd48004a020 con 0x7fd4900ca860 2024-04-26T21:21:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.090+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4901e4c60 con 0x7fd4900ca860 2024-04-26T21:21:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.091+0000 7fd47effd640 1 -- 172.21.15.40:0/2860388333 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+13030 (secure 0 0 0) 0x7fd480079320 con 0x7fd4900ca860 2024-04-26T21:21:23.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:23.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":137,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:20.756202+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3716452175":"2024-04-27T21:21:17.392935+0000","172.21.15.40:6834/3716452175":"2024-04-27T21:21:17.392935+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:23.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 msgr2=0x7fd47006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 0x7fd47006ede0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd4900cbd20 tx=0x7fd48800b040 comp rx=0 tx=0).stop 2024-04-26T21:21:23.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 msgr2=0x7fd49006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd480009c10 tx=0x7fd480004730 comp rx=0 tx=0).stop 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 shutdown_connections 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd47006c920 0x7fd47006ede0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4900ca860 0x7fd49006d0f0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4900b8f30 0x7fd4901f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 --2- 172.21.15.40:0/2860388333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd478004d40 0x7fd49015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 >> 172.21.15.40:0/2860388333 conn(0x7fd49008ec10 msgr2=0x7fd4900b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 shutdown_connections 2024-04-26T21:21:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.093+0000 7fd494fb6640 1 -- 172.21.15.40:0/2860388333 wait complete. 2024-04-26T21:21:23.105 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:23.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:23.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.208+0000 7f73c44d1640 1 Processor -- start 2024-04-26T21:21:23.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.208+0000 7f73c44d1640 1 -- start start 2024-04-26T21:21:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c44d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73bc0b4eb0 0x7f73bc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c44d1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73bc15dca0 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c44d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73bc15df80 con 0x7f73bc143840 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c44d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f73bc143c20 con 0x7f73bc0b4eb0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c2a6e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f73bc0ceff0 0x7f73bc0cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55180/0 (socket says 172.21.15.40:55180) 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c2a6e640 1 -- 172.21.15.40:0/3822659917 learned_addr learned my addr 172.21.15.40:0/3822659917 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/3822659917 >> v2:172.21.15.184:3300/0 conn(0x7f73bc0b4eb0 0x7f73bc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059686304 0 0) 0x7f73bc15dca0 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a8003520 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 295138297 0 0) 0x7f73a8003520 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 >> v2:172.21.15.184:3300/0 conn(0x7f73bc0b4eb0 msgr2=0x7f73bc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 --2- 172.21.15.40:0/3822659917 >> v2:172.21.15.184:3300/0 conn(0x7f73bc0b4eb0 0x7f73bc0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 >> v1:172.21.15.184:6789/0 conn(0x7f73bc143840 legacy=0x7f73bc0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.209+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc15e260 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f73b40019f0 con 0x7f73bc0ceff0 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 >> v1:172.21.15.40:6789/0 conn(0x7f73bc0ceff0 legacy=0x7f73bc0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80039b0 0x7f73a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80051c0 0x7f73a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc143c20 con 0x7f73a8004800 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a800c090 con 0x7f73a80039b0 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73a800c290 con 0x7f73a80051c0 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c226d640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80039b0 0x7f73a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c2a6e640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80051c0 0x7f73a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 -- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80051c0 msgr2=0x7f73a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73a80051c0 0x7f73a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 -- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80039b0 msgr2=0x7f73a8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80039b0 0x7f73a8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c1a6c640 1 -- 172.21.15.40:0/3822659917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a8003520 con 0x7f73a8004800 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c226d640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a80039b0 0x7f73a8003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.210+0000 7f73c2a6e640 1 -- 172.21.15.40:0/3822659917 reap_dead start 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73a8004c80 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009ac0 tx=0x7f73a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e512078271c639 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac036070 con 0x7f73a8004800 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac031400 con 0x7f73a8004800 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c126b640 1 -- 172.21.15.40:0/3822659917 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac009690 con 0x7f73a8004800 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c44d1640 1 -- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 msgr2=0x7f73a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c44d1640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73a8004c80 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009ac0 tx=0x7f73a8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c44d1640 1 -- 172.21.15.40:0/3822659917 shutdown_connections 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c44d1640 1 --2- 172.21.15.40:0/3822659917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73a8004c80 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.211+0000 7f73c44d1640 1 -- 172.21.15.40:0/3822659917 >> 172.21.15.40:0/3822659917 conn(0x7f73bc0cd8a0 msgr2=0x7f73bc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.212+0000 7f73c44d1640 1 -- 172.21.15.40:0/3822659917 shutdown_connections 2024-04-26T21:21:23.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.212+0000 7f73c44d1640 1 -- 172.21.15.40:0/3822659917 wait complete. 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 Processor -- start 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 -- start start 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73bc150ce0 0x7f73bc150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73bc1508c0 0x7f73bc140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc15da40 con 0x7f73a8004800 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc15d8c0 con 0x7f73bc1508c0 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.213+0000 7f73c44d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc0b5330 con 0x7f73bc150ce0 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40338/0 (socket says 172.21.15.40:40338) 2024-04-26T21:21:23.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 -- 172.21.15.40:0/4183341908 learned_addr learned my addr 172.21.15.40:0/4183341908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 -- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73bc150ce0 msgr2=0x7f73bc150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73bc150ce0 0x7f73bc150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 -- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73bc1508c0 msgr2=0x7f73bc140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73bc1508c0 0x7f73bc140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 -- 172.21.15.40:0/4183341908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc1462a0 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c226d640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e480 tx=0x7f73b800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b80052e0 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b8007020 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc146580 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8007210 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.214+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc146b40 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.215+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73bc06a290 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.216+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73b801a020 con 0x7f73a8004800 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.216+0000 7f73b2ffd640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 0x7f73a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.216+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f73b808ad70 con 0x7f73a8004800 2024-04-26T21:21:23.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.216+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 0x7f73a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.217+0000 7f73c1a6c640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 0x7f73a006edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009ac0 tx=0x7f73ac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.220+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73b8059530 con 0x7f73a8004800 2024-04-26T21:21:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.407+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73bc0ce420 con 0x7f73a8004800 2024-04-26T21:21:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.408+0000 7f73b2ffd640 1 -- 172.21.15.40:0/4183341908 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7f73b8058e30 con 0x7f73a8004800 2024-04-26T21:21:23.409 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 msgr2=0x7f73a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 0x7f73a006edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009ac0 tx=0x7f73ac02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 msgr2=0x7f73bc145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e480 tx=0x7f73b800e940 comp rx=0 tx=0).stop 2024-04-26T21:21:23.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 shutdown_connections 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f73a006c8f0 0x7f73a006edb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73bc1508c0 0x7f73bc140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73bc150ce0 0x7f73bc150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 --2- 172.21.15.40:0/4183341908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73a8004800 0x7f73bc145b10 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 >> 172.21.15.40:0/4183341908 conn(0x7f73bc0cd8a0 msgr2=0x7f73bc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 shutdown_connections 2024-04-26T21:21:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.410+0000 7f73c44d1640 1 -- 172.21.15.40:0/4183341908 wait complete. 2024-04-26T21:21:23.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-04-26T21:21:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.522+0000 7f4d6efa4640 1 Processor -- start 2024-04-26T21:21:23.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.522+0000 7f4d6efa4640 1 -- start start 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6efa4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d680cf2e0 0x7f4d6806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6efa4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4d6806e490 0x7f4d6815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6efa4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d680d1740 con 0x7f4d680d11b0 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6efa4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4d680d1a20 con 0x7f4d680cf2e0 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6efa4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4d680d1d00 con 0x7f4d6806e490 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d680cf2e0 0x7f4d6806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d680cf2e0 0x7f4d6806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40346/0 (socket says 172.21.15.40:40346) 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 -- 172.21.15.40:0/1966517074 learned_addr learned my addr 172.21.15.40:0/1966517074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6d541640 1 --2- 172.21.15.40:0/1966517074 >> v2:172.21.15.184:3300/0 conn(0x7f4d6806e490 0x7f4d6815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 -- 172.21.15.40:0/1966517074 >> v2:172.21.15.184:3300/0 conn(0x7f4d6806e490 msgr2=0x7f4d6815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 --2- 172.21.15.40:0/1966517074 >> v2:172.21.15.184:3300/0 conn(0x7f4d6806e490 0x7f4d6815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 -- 172.21.15.40:0/1966517074 >> v1:172.21.15.184:6789/0 conn(0x7f4d680d11b0 legacy=0x7f4d6806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.523+0000 7f4d6cd40640 1 -- 172.21.15.40:0/1966517074 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d680d1fe0 con 0x7f4d680cf2e0 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.524+0000 7f4d6cd40640 1 --2- 172.21.15.40:0/1966517074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6806d4d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009870 tx=0x7f4d5c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6e4a3f13e323c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.524+0000 7f4d677fe640 1 -- 172.21.15.40:0/1966517074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c037070 con 0x7f4d680cf2e0 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.524+0000 7f4d677fe640 1 -- 172.21.15.40:0/1966517074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c032440 con 0x7f4d680cf2e0 2024-04-26T21:21:23.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.524+0000 7f4d677fe640 1 -- 172.21.15.40:0/1966517074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c032810 con 0x7f4d680cf2e0 2024-04-26T21:21:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 -- 172.21.15.40:0/1966517074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 msgr2=0x7f4d6806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/1966517074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6806d4d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009870 tx=0x7f4d5c02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 -- 172.21.15.40:0/1966517074 shutdown_connections 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/1966517074 >> v2:172.21.15.184:3300/0 conn(0x7f4d6806e490 0x7f4d6815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/1966517074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6806d4d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 -- 172.21.15.40:0/1966517074 >> 172.21.15.40:0/1966517074 conn(0x7f4d680b4a60 msgr2=0x7f4d680b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 -- 172.21.15.40:0/1966517074 shutdown_connections 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.525+0000 7f4d6efa4640 1 -- 172.21.15.40:0/1966517074 wait complete. 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.526+0000 7f4d6efa4640 1 Processor -- start 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.526+0000 7f4d6efa4640 1 -- start start 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d6806e490 0x7f4d680b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 0x7f4d6814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d680cf6c0 con 0x7f4d680cf2e0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6806da10 con 0x7f4d680d11b0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6efa4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d680cf840 con 0x7f4d6806e490 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40350/0 (socket says 172.21.15.40:40350) 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 -- 172.21.15.40:0/2174470526 learned_addr learned my addr 172.21.15.40:0/2174470526 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6cd40640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d6806e490 0x7f4d680b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6d541640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 0x7f4d6814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 -- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d6806e490 msgr2=0x7f4d680b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d6806e490 0x7f4d680b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 -- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 msgr2=0x7f4d6814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 0x7f4d6814f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 -- 172.21.15.40:0/2174470526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d680b6fa0 con 0x7f4d680cf2e0 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d6d541640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 0x7f4d6814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:23.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.527+0000 7f4d67fff640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c3c0 tx=0x7f4d5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.528+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540052d0 con 0x7f4d680cf2e0 2024-04-26T21:21:23.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.528+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d54005490 con 0x7f4d680cf2e0 2024-04-26T21:21:23.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.528+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d680b7280 con 0x7f4d680cf2e0 2024-04-26T21:21:23.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.528+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54007260 con 0x7f4d680cf2e0 2024-04-26T21:21:23.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.528+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d680b7840 con 0x7f4d680cf2e0 2024-04-26T21:21:23.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.529+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6806a290 con 0x7f4d680cf2e0 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.529+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d54007400 con 0x7f4d680cf2e0 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.530+0000 7f4d657fa640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 0x7f4d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.530+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7f4d5408aca0 con 0x7f4d680cf2e0 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.530+0000 7f4d6cd40640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 0x7f4d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.530+0000 7f4d6cd40640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 0x7f4d4c06edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c034000 tx=0x7f4d5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.534+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d54058d80 con 0x7f4d680cf2e0 2024-04-26T21:21:23.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:23.791+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f4d680b8ef0 con 0x7f4d680cf2e0 2024-04-26T21:21:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.781+0000 7f4d657fa640 1 -- 172.21.15.40:0/2174470526 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v491) ==== 154+0+0 (secure 0 0 0) 0x7f4d5405d080 con 0x7f4d680cf2e0 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 msgr2=0x7f4d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 0x7f4d4c06edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c034000 tx=0x7f4d5c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 msgr2=0x7f4d6809cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c3c0 tx=0x7f4d5400c880 comp rx=0 tx=0).stop 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 shutdown_connections 2024-04-26T21:21:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d4c06c8f0 0x7f4d4c06edb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d680d11b0 0x7f4d6814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680cf2e0 0x7f4d6809cf10 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 --2- 172.21.15.40:0/2174470526 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d6806e490 0x7f4d680b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 >> 172.21.15.40:0/2174470526 conn(0x7f4d680b4a60 msgr2=0x7f4d6806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 shutdown_connections 2024-04-26T21:21:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.784+0000 7f4d6efa4640 1 -- 172.21.15.40:0/2174470526 wait complete. 2024-04-26T21:21:24.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:21:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.908+0000 7fa355f04640 1 Processor -- start 2024-04-26T21:21:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.908+0000 7fa355f04640 1 -- start start 2024-04-26T21:21:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa355f04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3500d0010 0x7fa35006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa355f04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35009c700 con 0x7fa3500b33e0 2024-04-26T21:21:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa355f04640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3500d1c50 con 0x7fa3500b3c20 2024-04-26T21:21:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa355f04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa3500b4480 con 0x7fa3500d0010 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3500d0010 0x7fa35006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa3500b33e0 0x7fa350163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55192/0 (socket says 172.21.15.40:55192) 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34ffff640 1 -- 172.21.15.40:0/1188899329 learned_addr learned my addr 172.21.15.40:0/1188899329 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34f7fe640 1 -- 172.21.15.40:0/1188899329 >> v1:172.21.15.184:6789/0 conn(0x7fa3500b3c20 legacy=0x7fa3500b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34f7fe640 1 -- 172.21.15.40:0/1188899329 >> v1:172.21.15.40:6789/0 conn(0x7fa3500b33e0 legacy=0x7fa350163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34f7fe640 1 -- 172.21.15.40:0/1188899329 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3500d1f30 con 0x7fa3500d0010 2024-04-26T21:21:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.909+0000 7fa34e7fc640 1 -- 172.21.15.40:0/1188899329 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569335566 0 0) 0x7fa35009c700 con 0x7fa3500b33e0 2024-04-26T21:21:24.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.910+0000 7fa34f7fe640 1 --2- 172.21.15.40:0/1188899329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500d0010 0x7fa35006db10 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3400097a0 tx=0x7fa34002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b668250a052e53d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.910+0000 7fa34e7fc640 1 -- 172.21.15.40:0/1188899329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340037070 con 0x7fa3500d0010 2024-04-26T21:21:24.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.910+0000 7fa34e7fc640 1 -- 172.21.15.40:0/1188899329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa340032440 con 0x7fa3500d0010 2024-04-26T21:21:24.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.910+0000 7fa34e7fc640 1 -- 172.21.15.40:0/1188899329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340032810 con 0x7fa3500d0010 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 -- 172.21.15.40:0/1188899329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500d0010 msgr2=0x7fa35006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 --2- 172.21.15.40:0/1188899329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500d0010 0x7fa35006db10 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3400097a0 tx=0x7fa34002a230 comp rx=0 tx=0).stop 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 -- 172.21.15.40:0/1188899329 shutdown_connections 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 --2- 172.21.15.40:0/1188899329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500d0010 0x7fa35006db10 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 -- 172.21.15.40:0/1188899329 >> 172.21.15.40:0/1188899329 conn(0x7fa35009ba40 msgr2=0x7fa35009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 -- 172.21.15.40:0/1188899329 shutdown_connections 2024-04-26T21:21:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.911+0000 7fa355f04640 1 -- 172.21.15.40:0/1188899329 wait complete. 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.912+0000 7fa355f04640 1 Processor -- start 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.912+0000 7fa355f04640 1 -- start start 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 0x7fa35015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 0x7fa35015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006e050 con 0x7fa3500b3c20 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500b3a20 con 0x7fa3500d0010 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa355f04640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa350163990 con 0x7fa3500b33e0 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40358/0 (socket says 172.21.15.40:40358) 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 -- 172.21.15.40:0/27642156 learned_addr learned my addr 172.21.15.40:0/27642156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34f7fe640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 0x7fa35015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34ffff640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 0x7fa35015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 -- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 msgr2=0x7fa35015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 0x7fa35015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 -- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 msgr2=0x7fa35015a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 0x7fa35015a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34effd640 1 -- 172.21.15.40:0/27642156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35015aa00 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34f7fe640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 0x7fa35015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.913+0000 7fa34ffff640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 0x7fa35015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa34effd640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c450 tx=0x7fa34400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3440052d0 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa344005470 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344012870 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35015e400 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.914+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35015eca0 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.915+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35006a290 con 0x7fa3500b3c20 2024-04-26T21:21:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.915+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa344012a10 con 0x7fa3500b3c20 2024-04-26T21:21:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.916+0000 7fa337fff640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 0x7fa33c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.916+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fa34408b4d0 con 0x7fa3500b3c20 2024-04-26T21:21:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.916+0000 7fa34f7fe640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 0x7fa33c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.917+0000 7fa34f7fe640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 0x7fa33c06ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa340034000 tx=0x7fa34002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:24.920+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa35015aa00 con 0x7fa3500b3c20 2024-04-26T21:21:25.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.107+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa350068e00 con 0x7fa3500b3c20 2024-04-26T21:21:25.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.111+0000 7fa337fff640 1 -- 172.21.15.40:0/27642156 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v66)=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7fa344059b70 con 0x7fa3500b3c20 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 msgr2=0x7fa33c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 0x7fa33c06ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa340034000 tx=0x7fa34002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 msgr2=0x7fa35015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c450 tx=0x7fa34400c910 comp rx=0 tx=0).stop 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 shutdown_connections 2024-04-26T21:21:25.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa33c06c920 0x7fa33c06ede0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3500d0010 0x7fa35015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3500b3c20 0x7fa35015ddb0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 --2- 172.21.15.40:0/27642156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3500b33e0 0x7fa35015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 >> 172.21.15.40:0/27642156 conn(0x7fa35009ba40 msgr2=0x7fa3500afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 shutdown_connections 2024-04-26T21:21:25.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.113+0000 7fa355f04640 1 -- 172.21.15.40:0/27642156 wait complete. 2024-04-26T21:21:25.126 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:21:25.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:25.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:25.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:21:25.223 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:25.224 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:25.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:25.240 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:25.240 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:25.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:25.258 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:25.258 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:25.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:25.295 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:25.295 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:25.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:25.315 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:21:25.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:25.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.435+0000 7fe82c807640 1 Processor -- start 2024-04-26T21:21:25.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.435+0000 7fe82c807640 1 -- start start 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.435+0000 7fe82c807640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8280cf2e0 0x7fe82806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe82c807640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8280d11b0 0x7fe82806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe82c807640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe82815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe82c807640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe82806e750 con 0x7fe8280d11b0 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe82c807640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe8280d1740 con 0x7fe8280cf2e0 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe82c807640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8280d1a20 con 0x7fe82806e2c0 2024-04-26T21:21:25.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8280d11b0 0x7fe82806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8280d11b0 0x7fe82806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40370/0 (socket says 172.21.15.40:40370) 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 -- 172.21.15.40:0/2735686057 learned_addr learned my addr 172.21.15.40:0/2735686057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 -- 172.21.15.40:0/2735686057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 msgr2=0x7fe82815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 --2- 172.21.15.40:0/2735686057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe82815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 -- 172.21.15.40:0/2735686057 >> v2:172.21.15.184:3300/0 conn(0x7fe8280cf2e0 msgr2=0x7fe82806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 --2- 172.21.15.40:0/2735686057 >> v2:172.21.15.184:3300/0 conn(0x7fe8280cf2e0 0x7fe82806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.436+0000 7fe825d9b640 1 -- 172.21.15.40:0/2735686057 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8280d1d00 con 0x7fe8280d11b0 2024-04-26T21:21:25.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe825d9b640 1 --2- 172.21.15.40:0/2735686057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280d11b0 0x7fe82806dd80 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009870 tx=0x7fe81c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9848e4d1d22c0274 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82559a640 1 -- 172.21.15.40:0/2735686057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c037070 con 0x7fe8280d11b0 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82559a640 1 -- 172.21.15.40:0/2735686057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c032440 con 0x7fe8280d11b0 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82559a640 1 -- 172.21.15.40:0/2735686057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c032810 con 0x7fe8280d11b0 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 -- 172.21.15.40:0/2735686057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280d11b0 msgr2=0x7fe82806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 --2- 172.21.15.40:0/2735686057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280d11b0 0x7fe82806dd80 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009870 tx=0x7fe81c02a210 comp rx=0 tx=0).stop 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 -- 172.21.15.40:0/2735686057 shutdown_connections 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 --2- 172.21.15.40:0/2735686057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe82815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 --2- 172.21.15.40:0/2735686057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280d11b0 0x7fe82806dd80 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 --2- 172.21.15.40:0/2735686057 >> v2:172.21.15.184:3300/0 conn(0x7fe8280cf2e0 0x7fe82806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.437+0000 7fe82c807640 1 -- 172.21.15.40:0/2735686057 >> 172.21.15.40:0/2735686057 conn(0x7fe8280b4a60 msgr2=0x7fe8280b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.438+0000 7fe82c807640 1 -- 172.21.15.40:0/2735686057 shutdown_connections 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.438+0000 7fe82c807640 1 -- 172.21.15.40:0/2735686057 wait complete. 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.439+0000 7fe82c807640 1 Processor -- start 2024-04-26T21:21:25.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.439+0000 7fe82c807640 1 -- start start 2024-04-26T21:21:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe8280b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 0x7fe8281eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe82806da10 con 0x7fe8280cf2e0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8280d22c0 con 0x7fe8280d11b0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe8280cf6c0 con 0x7fe82806e2c0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40386/0 (socket says 172.21.15.40:40386) 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 -- 172.21.15.40:0/407732426 learned_addr learned my addr 172.21.15.40:0/407732426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82659c640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe8280b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe826d9d640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 0x7fe8281eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 -- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 msgr2=0x7fe8280b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe8280b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 -- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 msgr2=0x7fe8281eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 0x7fe8281eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 -- 172.21.15.40:0/407732426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8280b5e00 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe826d9d640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 0x7fe8281eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe825d9b640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0090b0 tx=0x7fe81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c041070 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81c03c870 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c037040 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.440+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8280b60e0 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.441+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8280b66a0 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.441+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82806e0e0 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.442+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe81c044070 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.442+0000 7fe816ffd640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.442+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fe81c0ac320 con 0x7fe8280cf2e0 2024-04-26T21:21:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.443+0000 7fe82659c640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.443+0000 7fe82659c640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 0x7fe80806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe8281f87c0 tx=0x7fe820009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.447+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe82806e0e0 con 0x7fe8280cf2e0 2024-04-26T21:21:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.671+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8280b8ef0 con 0x7fe8280cf2e0 2024-04-26T21:21:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.671+0000 7fe816ffd640 1 -- 172.21.15.40:0/407732426 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+3875 (secure 0 0 0) 0x7fe81c07ac00 con 0x7fe8280cf2e0 2024-04-26T21:21:25.674 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:25.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/3367581680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3367581680},{"type":"v1","addr":"172.21.15.184:6836","nonce":3367581680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2338538029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2338538029},{"type":"v1","addr":"172.21.15.40:6833","nonce":2338538029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":9000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2328397078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":2328397078},{"type":"v1","addr":"172.21.15.184:6837","nonce":2328397078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":9405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2565929669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2565929669},{"type":"v1","addr":"172.21.15.40:6835","nonce":2565929669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[]} 2024-04-26T21:21:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 93 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 msgr2=0x7fe80806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 0x7fe80806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe8281f87c0 tx=0x7fe820009110 comp rx=0 tx=0).stop 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 msgr2=0x7fe8280a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0090b0 tx=0x7fe81c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 shutdown_connections 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8280d11b0 0x7fe8281eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8280cf2e0 0x7fe8280a0970 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 --2- 172.21.15.40:0/407732426 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe82806e2c0 0x7fe8280b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 >> 172.21.15.40:0/407732426 conn(0x7fe8280b4a60 msgr2=0x7fe8280b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 shutdown_connections 2024-04-26T21:21:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.674+0000 7fe82c807640 1 -- 172.21.15.40:0/407732426 wait complete. 2024-04-26T21:21:25.687 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:21:25.687 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:21:25.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:21:25.687 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:21:25.688 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:21:25.688 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:21:25.688 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:21:25.688 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:21:25.688 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:21:25.688 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:21:25.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:21:25.689 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:21:25.689 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:21:25.689 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:21:25.689 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:21:25.689 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:21:25.696 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:21:25.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:21:25.697 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:21:25.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:21:25.701 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:21:25.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:21:25.701 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:21:25.702 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:21:25.739 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:21:25.739 DEBUG:teuthology.parallel:result is None 2024-04-26T21:21:25.740 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:21:25.740 DEBUG:teuthology.parallel:result is None 2024-04-26T21:21:25.743 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:21:25.743 DEBUG:teuthology.parallel:result is None 2024-04-26T21:21:25.743 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:21:25.744 DEBUG:teuthology.parallel:result is None 2024-04-26T21:21:25.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:21:25.815 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-26T21:21:25.816 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:21:25.822 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2024-04-26T21:21:25.824 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2024-04-26T21:21:25.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 Processor -- start 2024-04-26T21:21:25.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 -- start start 2024-04-26T21:21:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19815f370 0x7fc1980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1980d3ee0 0x7fc1980b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1980d03a0 con 0x7fc19815f750 2024-04-26T21:21:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1980b4d10 con 0x7fc19815f370 2024-04-26T21:21:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.850+0000 7fc19f0da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1980d0680 con 0x7fc1980d3ee0 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19815f370 0x7fc1980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc19815f370 0x7fc1980cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40432/0 (socket says 172.21.15.40:40432) 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 -- 172.21.15.40:0/4103388695 learned_addr learned my addr 172.21.15.40:0/4103388695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19d677640 1 --2- 172.21.15.40:0/4103388695 >> v2:172.21.15.184:3300/0 conn(0x7fc1980d3ee0 0x7fc1980b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 -- 172.21.15.40:0/4103388695 >> v2:172.21.15.184:3300/0 conn(0x7fc1980d3ee0 msgr2=0x7fc1980b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 --2- 172.21.15.40:0/4103388695 >> v2:172.21.15.184:3300/0 conn(0x7fc1980d3ee0 0x7fc1980b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 -- 172.21.15.40:0/4103388695 >> v1:172.21.15.184:6789/0 conn(0x7fc19815f750 legacy=0x7fc1980d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.851+0000 7fc19ce76640 1 -- 172.21.15.40:0/4103388695 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980d0960 con 0x7fc19815f370 2024-04-26T21:21:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc19ce76640 1 --2- 172.21.15.40:0/4103388695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980cfd70 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc1980c8cf0 tx=0x7fc18c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87b13584e7cab81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc1977fe640 1 -- 172.21.15.40:0/4103388695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037070 con 0x7fc19815f370 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc1977fe640 1 -- 172.21.15.40:0/4103388695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c032440 con 0x7fc19815f370 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc1977fe640 1 -- 172.21.15.40:0/4103388695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c032810 con 0x7fc19815f370 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc19f0da640 1 -- 172.21.15.40:0/4103388695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 msgr2=0x7fc1980cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc19f0da640 1 --2- 172.21.15.40:0/4103388695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980cfd70 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc1980c8cf0 tx=0x7fc18c02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.852+0000 7fc19f0da640 1 -- 172.21.15.40:0/4103388695 shutdown_connections 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.853+0000 7fc19f0da640 1 --2- 172.21.15.40:0/4103388695 >> v2:172.21.15.184:3300/0 conn(0x7fc1980d3ee0 0x7fc1980b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.853+0000 7fc19f0da640 1 --2- 172.21.15.40:0/4103388695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980cfd70 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.853+0000 7fc19f0da640 1 -- 172.21.15.40:0/4103388695 >> 172.21.15.40:0/4103388695 conn(0x7fc19808ec10 msgr2=0x7fc19809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.853+0000 7fc19f0da640 1 -- 172.21.15.40:0/4103388695 shutdown_connections 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.853+0000 7fc19f0da640 1 -- 172.21.15.40:0/4103388695 wait complete. 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.854+0000 7fc19f0da640 1 Processor -- start 2024-04-26T21:21:25.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.854+0000 7fc19f0da640 1 -- start start 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 0x7fc198161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b51c0 con 0x7fc19815f370 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980d0f20 con 0x7fc1980d3ee0 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19f0da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980b5340 con 0x7fc19815f750 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc197fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19d677640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 0x7fc198161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:47492/0 (socket says 172.21.15.40:47492) 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 -- 172.21.15.40:0/3672446079 learned_addr learned my addr 172.21.15.40:0/3672446079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 -- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 msgr2=0x7fc198161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 0x7fc198161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 -- 172.21.15.40:0/3672446079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 msgr2=0x7fc1980b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 -- 172.21.15.40:0/3672446079 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980b8330 con 0x7fc1980d3ee0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19d677640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 0x7fc198161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc19ce76640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02a2b0 tx=0x7fc18c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c041020 con 0x7fc1980d3ee0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.855+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c0306c0 con 0x7fc1980d3ee0 2024-04-26T21:21:25.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.856+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1980b9100 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.856+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037040 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.856+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981f5240 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.857+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19806a290 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.857+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc18c044020 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.857+0000 7fc1957fa640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.857+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4943+0+0 (secure 0 0 0) 0x7fc18c0abf90 con 0x7fc1980d3ee0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.858+0000 7fc197fff640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.858+0000 7fc197fff640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc19809e5a0 tx=0x7fc184009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:25.863+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc18c04b030 con 0x7fc1980d3ee0 2024-04-26T21:21:26.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.064+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc1980d3900 con 0x7fc1980d3ee0 2024-04-26T21:21:26.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.800+0000 7fc1957fa640 1 -- 172.21.15.40:0/3672446079 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7fc1980b9100 con 0x7fc1980d3ee0 2024-04-26T21:21:26.803 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-26T21:21:26.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 msgr2=0x7fc17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:26.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc19809e5a0 tx=0x7fc184009040 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 msgr2=0x7fc1980b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02a2b0 tx=0x7fc18c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 shutdown_connections 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc19815f750 0x7fc198161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc19815f370 0x7fc1980b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 --2- 172.21.15.40:0/3672446079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1980d3ee0 0x7fc1980b7df0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 >> 172.21.15.40:0/3672446079 conn(0x7fc19808ec10 msgr2=0x7fc19815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 shutdown_connections 2024-04-26T21:21:26.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:26.804+0000 7fc19f0da640 1 -- 172.21.15.40:0/3672446079 wait complete. 2024-04-26T21:21:26.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:21:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.107+0000 7f5260e2c640 1 Processor -- start 2024-04-26T21:21:27.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.107+0000 7f5260e2c640 1 -- start start 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.108+0000 7f5260e2c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f525c0d5610 0x7f525c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.108+0000 7f5260e2c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f525c0b2510 con 0x7f525c15f720 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.108+0000 7f5260e2c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f525c0b27f0 con 0x7f525c15f340 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.108+0000 7f5260e2c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f525c0b2ad0 con 0x7f525c0d5610 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f525c0d5610 0x7f525c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f525c0d5610 0x7f525c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40444/0 (socket says 172.21.15.40:40444) 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f5259d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f525c15f720 0x7f525c0b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55204/0 (socket says 172.21.15.40:55204) 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 -- 172.21.15.40:0/3483018688 learned_addr learned my addr 172.21.15.40:0/3483018688 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 -- 172.21.15.40:0/3483018688 >> v1:172.21.15.184:6789/0 conn(0x7f525c15f340 legacy=0x7f525c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525959a640 1 -- 172.21.15.40:0/3483018688 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331156209 0 0) 0x7f525c0b2510 con 0x7f525c15f720 2024-04-26T21:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 -- 172.21.15.40:0/3483018688 >> v1:172.21.15.40:6789/0 conn(0x7f525c15f720 legacy=0x7f525c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.109+0000 7f525ad9d640 1 -- 172.21.15.40:0/3483018688 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c0b2db0 con 0x7f525c0d5610 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f525ad9d640 1 --2- 172.21.15.40:0/3483018688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c0b1e80 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e50 tx=0x7f524c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2d58e7a39cd7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f525959a640 1 -- 172.21.15.40:0/3483018688 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c036070 con 0x7f525c0d5610 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f525959a640 1 -- 172.21.15.40:0/3483018688 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c031600 con 0x7f525c0d5610 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f525959a640 1 -- 172.21.15.40:0/3483018688 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c0319d0 con 0x7f525c0d5610 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f5260e2c640 1 -- 172.21.15.40:0/3483018688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 msgr2=0x7f525c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3483018688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c0b1e80 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e50 tx=0x7f524c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f5260e2c640 1 -- 172.21.15.40:0/3483018688 shutdown_connections 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3483018688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c0b1e80 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.110+0000 7f5260e2c640 1 -- 172.21.15.40:0/3483018688 >> 172.21.15.40:0/3483018688 conn(0x7f525c08ec70 msgr2=0x7f525c09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.111+0000 7f5260e2c640 1 -- 172.21.15.40:0/3483018688 shutdown_connections 2024-04-26T21:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.111+0000 7f5260e2c640 1 -- 172.21.15.40:0/3483018688 wait complete. 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.112+0000 7f5260e2c640 1 Processor -- start 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.112+0000 7f5260e2c640 1 -- start start 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f525c15f340 0x7f525c14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f525c15f720 0x7f525c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f525c0d40d0 con 0x7f525c0d5610 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f525c0d3f50 con 0x7f525c15f720 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5260e2c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f525c0b3cc0 con 0x7f525c15f340 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40452/0 (socket says 172.21.15.40:40452) 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 -- 172.21.15.40:0/3106015708 learned_addr learned my addr 172.21.15.40:0/3106015708 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5259d9b640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f525c15f340 0x7f525c14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525ad9d640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f525c15f720 0x7f525c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 -- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f525c15f340 msgr2=0x7f525c14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f525c15f340 0x7f525c14ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 -- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f525c15f720 msgr2=0x7f525c14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f525c15f720 0x7f525c14b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 -- 172.21.15.40:0/3106015708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c0caca0 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f525a59c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f525400c430 tx=0x7f525400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254011790 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.113+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5254017070 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.114+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254011bb0 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.114+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c0caf80 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.114+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c0cb540 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.114+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c06a290 con 0x7f525c0d5610 2024-04-26T21:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.115+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f525401c460 con 0x7f525c0d5610 2024-04-26T21:21:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.115+0000 7f5252ffd640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 0x7f523c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.116+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f525408b740 con 0x7f525c0d5610 2024-04-26T21:21:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.116+0000 7f5259d9b640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 0x7f523c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.116+0000 7f5259d9b640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 0x7f523c06edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f525c0d1cc0 tx=0x7f5244007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.120+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5254059e20 con 0x7f525c0d5610 2024-04-26T21:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.375+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f525c0cb710 con 0x7f525c0d5610 2024-04-26T21:21:27.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.375+0000 7f5252ffd640 1 -- 172.21.15.40:0/3106015708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7f5254059720 con 0x7f525c0d5610 2024-04-26T21:21:27.377 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:27.377 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmQgmnERAAlnKDKcmnZ8T3nLIbXbBLwQ==", 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmUKoJHRAAxyi+Sb0nz3kolo1c6XO3/g==", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-04-26T21:21:27.379 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxm5euXGBAAhIu+goQs6LNGfRVWqZjQdQ==", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmoEPPIxAA9FI1s3wEXWpoQAvR5UJgQw==", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:21:27.380 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBrGCxmlNxUBhAAPbCJ8fc6lNIskNFsESMxzw==", 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.381 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBuGCxmG5SFBBAAES64cDlmlxbGL9yde4nJoQ==", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBxGCxmurTiAxAAkL9djF/U/aD1mwRsqmjLqw==", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.382 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB0GCxmPwreAhAAzF9QB/7W7MdN8FXjJ2BpyQ==", 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB6GCxmJwc9BRAAHNRARBNFvrq+qvEixYWLjQ==", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQB9GCxmoCgdAxAARiQVZ/FvkwYLKSzQo/IpcA==", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.384 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCAGCxmbsJCARAAFJkai1k/FCdljAzPBf8KlA==", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQCDGCxmM1OxARAAishkJb1yC5a/u9ZJ8Q3cDg==", 2024-04-26T21:21:27.385 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg==", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T21:21:27.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmrqlQMBAA94AQVCE2Zu7MMS8NAHuaKA==", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIGBBLRAAfSBKtYl4MRJydgSeCt/UNw==", 2024-04-26T21:21:27.387 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmhUq9NRAAUtPn8sfKJg18066RWro9Ag==", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBmGCxmlyIMMhAAO3oYiUjsoPW9xwNNxgXkEQ==", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-26T21:21:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmquF6DhAAwZRdMjvA+MpWkbkf437tMg==", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBnGCxmIg3aChAA5sQU29xNn8+3GvitYhuVgQ==", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-26T21:21:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-26T21:21:27.391 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 msgr2=0x7f523c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 0x7f523c06edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f525c0d1cc0 tx=0x7f5244007c00 comp rx=0 tx=0).stop 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 msgr2=0x7f525c1f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f525400c430 tx=0x7f525400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 shutdown_connections 2024-04-26T21:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f523c06c8f0 0x7f523c06edb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f525c15f720 0x7f525c14b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f525c15f340 0x7f525c14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 --2- 172.21.15.40:0/3106015708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f525c0d5610 0x7f525c1f4590 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 >> 172.21.15.40:0/3106015708 conn(0x7f525c08ec70 msgr2=0x7f525c0b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 shutdown_connections 2024-04-26T21:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.377+0000 7f5260e2c640 1 -- 172.21.15.40:0/3106015708 wait complete. 2024-04-26T21:21:27.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.492+0000 7f10ad989640 1 Processor -- start 2024-04-26T21:21:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.492+0000 7f10ad989640 1 -- start start 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10ad989640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10a815f750 0x7f10a80b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10ad989640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 0x7f10a80d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10ad989640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a80d2440 con 0x7f10a80d3320 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10ad989640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f10a80d0030 con 0x7f10a815f750 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10ad989640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f10a80d2720 con 0x7f10a80d51a0 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10a6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10a815f750 0x7f10a80b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10a6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10a815f750 0x7f10a80b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40466/0 (socket says 172.21.15.40:40466) 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.493+0000 7f10a6ffd640 1 -- 172.21.15.40:0/42343836 learned_addr learned my addr 172.21.15.40:0/42343836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a77fe640 1 --2- 172.21.15.40:0/42343836 >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 0x7f10a80d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a6ffd640 1 -- 172.21.15.40:0/42343836 >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 msgr2=0x7f10a80d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a6ffd640 1 --2- 172.21.15.40:0/42343836 >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 0x7f10a80d5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a6ffd640 1 -- 172.21.15.40:0/42343836 >> v1:172.21.15.184:6789/0 conn(0x7f10a80d3320 legacy=0x7f10a80d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a6ffd640 1 -- 172.21.15.40:0/42343836 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80d2a00 con 0x7f10a815f750 2024-04-26T21:21:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a77fe640 1 --2- 172.21.15.40:0/42343836 >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 0x7f10a80d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.494+0000 7f10a6ffd640 1 --2- 172.21.15.40:0/42343836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a815f750 0x7f10a80b50e0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f10a808e4a0 tx=0x7f10a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ba98634fec6428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10a5ffb640 1 -- 172.21.15.40:0/42343836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037070 con 0x7f10a815f750 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10a5ffb640 1 -- 172.21.15.40:0/42343836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0032440 con 0x7f10a815f750 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10a5ffb640 1 -- 172.21.15.40:0/42343836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0032810 con 0x7f10a815f750 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 -- 172.21.15.40:0/42343836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a815f750 msgr2=0x7f10a80b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 --2- 172.21.15.40:0/42343836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a815f750 0x7f10a80b50e0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f10a808e4a0 tx=0x7f10a002a230 comp rx=0 tx=0).stop 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 -- 172.21.15.40:0/42343836 shutdown_connections 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 --2- 172.21.15.40:0/42343836 >> v2:172.21.15.184:3300/0 conn(0x7f10a80d51a0 0x7f10a80d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 --2- 172.21.15.40:0/42343836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a815f750 0x7f10a80b50e0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 -- 172.21.15.40:0/42343836 >> 172.21.15.40:0/42343836 conn(0x7f10a808ec20 msgr2=0x7f10a809e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 -- 172.21.15.40:0/42343836 shutdown_connections 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.495+0000 7f10ad989640 1 -- 172.21.15.40:0/42343836 wait complete. 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.496+0000 7f10ad989640 1 Processor -- start 2024-04-26T21:21:27.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 -- start start 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 0x7f10a8160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 0x7f10a80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80d0850 con 0x7f10a80d3320 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80d06d0 con 0x7f10a80d51a0 2024-04-26T21:21:27.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10ad989640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f10a80d09d0 con 0x7f10a815f750 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40480/0 (socket says 172.21.15.40:40480) 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.497+0000 7f10a6ffd640 1 -- 172.21.15.40:0/2336780743 learned_addr learned my addr 172.21.15.40:0/2336780743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a67fc640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 0x7f10a8160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 -- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 msgr2=0x7f10a80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a77fe640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 0x7f10a80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 0x7f10a80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 -- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 msgr2=0x7f10a8160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 0x7f10a8160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 -- 172.21.15.40:0/2336780743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a8160ef0 con 0x7f10a80d3320 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a67fc640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 0x7f10a8160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a77fe640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 0x7f10a80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10a6ffd640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037040 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0042070 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a8161190 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0030bb0 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.498+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a8161750 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.499+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a80d3bb0 con 0x7f10a80d3320 2024-04-26T21:21:27.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.499+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10a0047460 con 0x7f10a80d3320 2024-04-26T21:21:27.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.500+0000 7f109ffff640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 0x7f108806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.500+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f10a00ac3c0 con 0x7f10a80d3320 2024-04-26T21:21:27.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.500+0000 7f10a67fc640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 0x7f108806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.501+0000 7f10a67fc640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 0x7f108806edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f10a806de10 tx=0x7f1090009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.504+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10a80d3bb0 con 0x7f10a80d3320 2024-04-26T21:21:27.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.691+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10a80c88b0 con 0x7f10a80d3320 2024-04-26T21:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.692+0000 7f109ffff640 1 -- 172.21.15.40:0/2336780743 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+12901 (secure 0 0 0) 0x7f10a0045360 con 0x7f10a80d3320 2024-04-26T21:21:27.694 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:27.694 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":141,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:26.813951+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 msgr2=0x7f108806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 0x7f108806edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f10a806de10 tx=0x7f1090009040 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 msgr2=0x7f10a80ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a00046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 shutdown_connections 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f108806c8f0 0x7f108806edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f10a815f750 0x7f10a80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10a80d51a0 0x7f10a8160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 --2- 172.21.15.40:0/2336780743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10a80d3320 0x7f10a80ba570 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 >> 172.21.15.40:0/2336780743 conn(0x7f10a808ec20 msgr2=0x7f10a806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 shutdown_connections 2024-04-26T21:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.694+0000 7f10ad989640 1 -- 172.21.15.40:0/2336780743 wait complete. 2024-04-26T21:21:27.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:27.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:27.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.813+0000 7f0324ff0640 1 Processor -- start 2024-04-26T21:21:27.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.813+0000 7f0324ff0640 1 -- start start 2024-04-26T21:21:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 0x7f03200d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03200d0ff0 0x7f03200d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200d2120 0x7f03200ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f03200d2570 con 0x7f03200d0ff0 2024-04-26T21:21:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f03200d4240 con 0x7f032015f490 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f0324ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f03200d31a0 con 0x7f03200d2120 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03200d0ff0 0x7f03200d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03200d0ff0 0x7f03200d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40484/0 (socket says 172.21.15.40:40484) 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 -- 172.21.15.40:0/645815067 learned_addr learned my addr 172.21.15.40:0/645815067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031e59c640 1 --2- 172.21.15.40:0/645815067 >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 0x7f03200d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031ed9d640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200d2120 0x7f03200ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 -- 172.21.15.40:0/645815067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200d2120 msgr2=0x7f03200ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200d2120 0x7f03200ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 -- 172.21.15.40:0/645815067 >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 msgr2=0x7f03200d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 --2- 172.21.15.40:0/645815067 >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 0x7f03200d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031dd9b640 1 -- 172.21.15.40:0/645815067 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200d3480 con 0x7f03200d0ff0 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.814+0000 7f031e59c640 1 --2- 172.21.15.40:0/645815067 >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 0x7f03200d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f031dd9b640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d0ff0 0x7f03200d1be0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f03200b7cf0 tx=0x7f031002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a37eac8f14f781e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f031d59a640 1 -- 172.21.15.40:0/645815067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310037070 con 0x7f03200d0ff0 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f031d59a640 1 -- 172.21.15.40:0/645815067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0310032440 con 0x7f03200d0ff0 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f031d59a640 1 -- 172.21.15.40:0/645815067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310032810 con 0x7f03200d0ff0 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f0324ff0640 1 -- 172.21.15.40:0/645815067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d0ff0 msgr2=0x7f03200d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f0324ff0640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d0ff0 0x7f03200d1be0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f03200b7cf0 tx=0x7f031002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f0324ff0640 1 -- 172.21.15.40:0/645815067 shutdown_connections 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.815+0000 7f0324ff0640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03200d2120 0x7f03200ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.816+0000 7f0324ff0640 1 --2- 172.21.15.40:0/645815067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d0ff0 0x7f03200d1be0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.816+0000 7f0324ff0640 1 --2- 172.21.15.40:0/645815067 >> v2:172.21.15.184:3300/0 conn(0x7f032015f490 0x7f03200d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.816+0000 7f0324ff0640 1 -- 172.21.15.40:0/645815067 >> 172.21.15.40:0/645815067 conn(0x7f032009e530 msgr2=0x7f03200d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.816+0000 7f0324ff0640 1 -- 172.21.15.40:0/645815067 shutdown_connections 2024-04-26T21:21:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.816+0000 7f0324ff0640 1 -- 172.21.15.40:0/645815067 wait complete. 2024-04-26T21:21:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.817+0000 7f0324ff0640 1 Processor -- start 2024-04-26T21:21:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.817+0000 7f0324ff0640 1 -- start start 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d0ff0 0x7f032015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032015f490 0x7f03200cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03200d3d20 con 0x7f03200d2120 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03200d3a40 con 0x7f03200d0ff0 2024-04-26T21:21:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f0324ff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f032015f870 con 0x7f032015f490 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40490/0 (socket says 172.21.15.40:40490) 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 -- 172.21.15.40:0/4200634327 learned_addr learned my addr 172.21.15.40:0/4200634327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031e59c640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d0ff0 0x7f032015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031ed9d640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032015f490 0x7f03200cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 -- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032015f490 msgr2=0x7f03200cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032015f490 0x7f03200cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 -- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d0ff0 msgr2=0x7f032015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d0ff0 0x7f032015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 -- 172.21.15.40:0/4200634327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200c96e0 con 0x7f03200d2120 2024-04-26T21:21:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.818+0000 7f031dd9b640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f031002a220 tx=0x7f03100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.820+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310041020 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.820+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03100306c0 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.820+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310037040 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.820+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03200cf1e0 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.820+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03200cf7a0 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.821+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0320068520 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.822+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0310044020 con 0x7f03200d2120 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.822+0000 7f030effd640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.822+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f03100abed0 con 0x7f03200d2120 2024-04-26T21:21:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.822+0000 7f031e59c640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.823+0000 7f031e59c640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0314006e80 tx=0x7f0314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:27.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:27.826+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0320068520 con 0x7f03200d2120 2024-04-26T21:21:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.011+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f032015de50 con 0x7f03200d2120 2024-04-26T21:21:28.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.011+0000 7f030effd640 1 -- 172.21.15.40:0/4200634327 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7f032015de50 con 0x7f03200d2120 2024-04-26T21:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 msgr2=0x7f030006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0314006e80 tx=0x7f0314008040 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 msgr2=0x7f0320152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f031002a220 tx=0x7f03100046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 shutdown_connections 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f032015f490 0x7f03200cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03200d2120 0x7f0320152280 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 --2- 172.21.15.40:0/4200634327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03200d0ff0 0x7f032015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.013+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 >> 172.21.15.40:0/4200634327 conn(0x7f032009e530 msgr2=0x7f03200cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.014+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 shutdown_connections 2024-04-26T21:21:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.014+0000 7f0324ff0640 1 -- 172.21.15.40:0/4200634327 wait complete. 2024-04-26T21:21:28.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:21:28.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.126+0000 7fa799fa4640 1 Processor -- start 2024-04-26T21:21:28.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.126+0000 7fa799fa4640 1 -- start start 2024-04-26T21:21:28.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.126+0000 7fa799fa4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 0x7fa794146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.126+0000 7fa799fa4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa7941465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.126+0000 7fa799fa4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7940d05b0 con 0x7fa794140350 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa799fa4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa7940d0890 con 0x7fa79413ff70 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa799fa4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa79406e450 con 0x7fa7940cff20 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa792ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa794140350 0x7fa7940cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55220/0 (socket says 172.21.15.40:55220) 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa792ffd640 1 -- 172.21.15.40:0/124914048 learned_addr learned my addr 172.21.15.40:0/124914048 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa7937fe640 1 --2- 172.21.15.40:0/124914048 >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 0x7fa794146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa793fff640 1 --2- 172.21.15.40:0/124914048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa7941465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451711948 0 0) 0x7fa7940d05b0 con 0x7fa794140350 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa77c003520 con 0x7fa794140350 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747141732 0 0) 0x7fa77c003520 con 0x7fa794140350 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa793fff640 1 -- 172.21.15.40:0/124914048 >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 msgr2=0x7fa794146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa793fff640 1 --2- 172.21.15.40:0/124914048 >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 0x7fa794146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa793fff640 1 -- 172.21.15.40:0/124914048 >> v1:172.21.15.40:6789/0 conn(0x7fa794140350 legacy=0x7fa7940cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa793fff640 1 -- 172.21.15.40:0/124914048 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79406e730 con 0x7fa7940cff20 2024-04-26T21:21:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.127+0000 7fa7937fe640 1 --2- 172.21.15.40:0/124914048 >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 0x7fa794146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.128+0000 7fa793fff640 1 --2- 172.21.15.40:0/124914048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa7941465c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009d90 tx=0x7fa78c02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815b9e8dc6a08da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.128+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788001670 con 0x7fa7940cff20 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.128+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c007540 con 0x7fa7940cff20 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.128+0000 7fa7927fc640 1 -- 172.21.15.40:0/124914048 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0078f0 con 0x7fa7940cff20 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 -- 172.21.15.40:0/124914048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 msgr2=0x7fa7941465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 --2- 172.21.15.40:0/124914048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa7941465c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009d90 tx=0x7fa78c02a560 comp rx=0 tx=0).stop 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 -- 172.21.15.40:0/124914048 shutdown_connections 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 --2- 172.21.15.40:0/124914048 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa7941465c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 --2- 172.21.15.40:0/124914048 >> v2:172.21.15.184:3300/0 conn(0x7fa79413ff70 0x7fa794146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 -- 172.21.15.40:0/124914048 >> 172.21.15.40:0/124914048 conn(0x7fa7940d0d30 msgr2=0x7fa7940d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 -- 172.21.15.40:0/124914048 shutdown_connections 2024-04-26T21:21:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.129+0000 7fa799fa4640 1 -- 172.21.15.40:0/124914048 wait complete. 2024-04-26T21:21:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.130+0000 7fa799fa4640 1 Processor -- start 2024-04-26T21:21:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.130+0000 7fa799fa4640 1 -- start start 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa794155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79413ff70 0x7fa7940730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa79406efe0 con 0x7fa794140350 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa79406ee60 con 0x7fa79413ff70 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa799fa4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa79406f160 con 0x7fa7940cff20 2024-04-26T21:21:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40498/0 (socket says 172.21.15.40:40498) 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 -- 172.21.15.40:0/2044606122 learned_addr learned my addr 172.21.15.40:0/2044606122 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa792ffd640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79413ff70 0x7fa7940730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa7937fe640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa794155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 -- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 msgr2=0x7fa794155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa794155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 -- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79413ff70 msgr2=0x7fa7940730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79413ff70 0x7fa7940730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa793fff640 1 -- 172.21.15.40:0/2044606122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa794072070 con 0x7fa794140350 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.131+0000 7fa7937fe640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa794155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa793fff640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009eb0 tx=0x7fa78c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c040070 con 0x7fa794140350 2024-04-26T21:21:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c031c40 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7940722e0 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c009450 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.132+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7940728a0 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.133+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79406a290 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.133+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa78c039040 con 0x7fa794140350 2024-04-26T21:21:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.134+0000 7fa77bfff640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 0x7fa78006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.134+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7fa78c0acba0 con 0x7fa794140350 2024-04-26T21:21:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.135+0000 7fa7937fe640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 0x7fa78006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.137+0000 7fa7937fe640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 0x7fa78006edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa784009ac0 tx=0x7fa784008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.139+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa78c07b320 con 0x7fa794140350 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.325+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa794072d40 con 0x7fa794140350 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.325+0000 7fa77bfff640 1 -- 172.21.15.40:0/2044606122 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7fa78c07ac20 con 0x7fa794140350 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-26T21:21:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-26T21:21:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-26T21:21:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-26T21:21:28.328 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 msgr2=0x7fa78006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 0x7fa78006edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa784009ac0 tx=0x7fa784008040 comp rx=0 tx=0).stop 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 msgr2=0x7fa79413e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009eb0 tx=0x7fa78c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 shutdown_connections 2024-04-26T21:21:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa78006c8f0 0x7fa78006edb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa794140350 0x7fa79413e5b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa79413ff70 0x7fa7940730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 --2- 172.21.15.40:0/2044606122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7940cff20 0x7fa794155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 >> 172.21.15.40:0/2044606122 conn(0x7fa7940d0d30 msgr2=0x7fa79413a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 shutdown_connections 2024-04-26T21:21:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.327+0000 7fa799fa4640 1 -- 172.21.15.40:0/2044606122 wait complete. 2024-04-26T21:21:28.340 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:21:28.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:21:28.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.443+0000 7f359aff0640 1 Processor -- start 2024-04-26T21:21:28.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.443+0000 7f359aff0640 1 -- start start 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f359aff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 0x7f35940d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f359aff0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35940d33a0 con 0x7f35940b6020 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f359aff0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35940d3680 con 0x7f35940d2140 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f359aff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35940b4970 con 0x7f3594162a10 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f3593fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f35940b6020 0x7f35940d1a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55232/0 (socket says 172.21.15.40:55232) 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f3593fff640 1 -- 172.21.15.40:0/2881831375 learned_addr learned my addr 172.21.15.40:0/2881831375 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f3598d8c640 1 --2- 172.21.15.40:0/2881831375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 0x7f35940d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733867657 0 0) 0x7f35940d33a0 con 0x7f35940b6020 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f357c003520 con 0x7f35940b6020 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131529689 0 0) 0x7f35940d3680 con 0x7f35940d2140 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.444+0000 7f3598d8c640 1 -- 172.21.15.40:0/2881831375 >> v1:172.21.15.184:6789/0 conn(0x7f35940d2140 legacy=0x7f35940cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.445+0000 7f3598d8c640 1 -- 172.21.15.40:0/2881831375 >> v1:172.21.15.40:6789/0 conn(0x7f35940b6020 legacy=0x7f35940d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.445+0000 7f3598d8c640 1 -- 172.21.15.40:0/2881831375 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35940d3960 con 0x7f3594162a10 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.445+0000 7f3598d8c640 1 --2- 172.21.15.40:0/2881831375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 0x7f35940d2dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f358c009dc0 tx=0x7f358c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b9fd5f966c778f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.445+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c007740 con 0x7f3594162a10 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c0078e0 con 0x7f3594162a10 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f35937fe640 1 -- 172.21.15.40:0/2881831375 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c007c90 con 0x7f3594162a10 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 -- 172.21.15.40:0/2881831375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 msgr2=0x7f35940d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 --2- 172.21.15.40:0/2881831375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 0x7f35940d2dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f358c009dc0 tx=0x7f358c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 -- 172.21.15.40:0/2881831375 shutdown_connections 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 --2- 172.21.15.40:0/2881831375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3594162a10 0x7f35940d2dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 -- 172.21.15.40:0/2881831375 >> 172.21.15.40:0/2881831375 conn(0x7f359409e530 msgr2=0x7f35940d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 -- 172.21.15.40:0/2881831375 shutdown_connections 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.446+0000 7f359aff0640 1 -- 172.21.15.40:0/2881831375 wait complete. 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.447+0000 7f359aff0640 1 Processor -- start 2024-04-26T21:21:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.447+0000 7f359aff0640 1 -- start start 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35940b6020 0x7f3594161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 0x7f359415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3594162df0 con 0x7f35940d2140 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35940d3f20 con 0x7f3594162a10 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f359aff0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35940b6600 con 0x7f35940b6020 2024-04-26T21:21:28.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f3593fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f3593fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40500/0 (socket says 172.21.15.40:40500) 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.448+0000 7f3593fff640 1 -- 172.21.15.40:0/2758551823 learned_addr learned my addr 172.21.15.40:0/2758551823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f359958d640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 0x7f359415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 -- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35940b6020 msgr2=0x7f3594161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3598d8c640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35940b6020 0x7f3594161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35940b6020 0x7f3594161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 -- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 msgr2=0x7f359415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 0x7f359415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 -- 172.21.15.40:0/2758551823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35940ca010 con 0x7f35940d2140 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f359958d640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 0x7f359415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f3593fff640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f358800c460 tx=0x7f358800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588017070 con 0x7f35940d2140 2024-04-26T21:21:28.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3588005ce0 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35940b9520 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588007290 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.449+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35940b9ae0 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.450+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35940b9ce0 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.451+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f358801a070 con 0x7f35940d2140 2024-04-26T21:21:28.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.451+0000 7f35917fa640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.451+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4857+0+0 (secure 0 0 0) 0x7f358808ab40 con 0x7f35940d2140 2024-04-26T21:21:28.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.451+0000 7f3598d8c640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.452+0000 7f3598d8c640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 0x7f358006ede0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.455+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35940b9ce0 con 0x7f35940d2140 2024-04-26T21:21:28.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.639+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f35940b7ad0 con 0x7f35940d2140 2024-04-26T21:21:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.822+0000 7f35917fa640 1 -- 172.21.15.40:0/2758551823 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7f35880592d0 con 0x7f35940d2140 2024-04-26T21:21:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-26T21:21:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 msgr2=0x7f358006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 0x7f358006ede0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f358c034000 tx=0x7f358c02f000 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 msgr2=0x7f359415bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f358800c460 tx=0x7f358800c920 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 shutdown_connections 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3594162a10 0x7f359415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35940d2140 0x7f359415bfe0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 --2- 172.21.15.40:0/2758551823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35940b6020 0x7f3594161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.824+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 >> 172.21.15.40:0/2758551823 conn(0x7f359409e530 msgr2=0x7f35940d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.825+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 shutdown_connections 2024-04-26T21:21:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.825+0000 7f359aff0640 1 -- 172.21.15.40:0/2758551823 wait complete. 2024-04-26T21:21:28.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:21:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.946+0000 7f90408b1640 1 Processor -- start 2024-04-26T21:21:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.946+0000 7f90408b1640 1 -- start start 2024-04-26T21:21:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f90408b1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f903c1459a0 0x7f903c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f90408b1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c0b6300 0x7f903c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f90408b1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f903c0b4480 con 0x7f903c15d030 2024-04-26T21:21:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f90408b1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f903c0b4760 con 0x7f903c1459a0 2024-04-26T21:21:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f90408b1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f903c0b4a40 con 0x7f903c0b6300 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f903a59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f903c1459a0 0x7f903c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f9033fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f903c15d030 0x7f903c0b5bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:53218/0 (socket says 172.21.15.40:53218) 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.947+0000 7f9033fff640 1 -- 172.21.15.40:0/430653409 learned_addr learned my addr 172.21.15.40:0/430653409 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903ad9d640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c0b6300 0x7f903c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903a59c640 1 -- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c0b6300 msgr2=0x7f903c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903a59c640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c0b6300 0x7f903c0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903a59c640 1 -- 172.21.15.40:0/430653409 >> v1:172.21.15.184:6789/0 conn(0x7f903c15d030 legacy=0x7f903c0b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903a59c640 1 -- 172.21.15.40:0/430653409 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c0b4d20 con 0x7f903c1459a0 2024-04-26T21:21:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.948+0000 7f903a59c640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c0b3de0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f902c0097a0 tx=0x7f902c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2806dcdf9502e376 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f9039d9b640 1 -- 172.21.15.40:0/430653409 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c037070 con 0x7f903c1459a0 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f9039d9b640 1 -- 172.21.15.40:0/430653409 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c032440 con 0x7f903c1459a0 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f9039d9b640 1 -- 172.21.15.40:0/430653409 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c032810 con 0x7f903c1459a0 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 -- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 msgr2=0x7f903c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c0b3de0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f902c0097a0 tx=0x7f902c02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 -- 172.21.15.40:0/430653409 shutdown_connections 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c0b6300 0x7f903c0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 --2- 172.21.15.40:0/430653409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c0b3de0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 -- 172.21.15.40:0/430653409 >> 172.21.15.40:0/430653409 conn(0x7f903c0d0d30 msgr2=0x7f903c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 -- 172.21.15.40:0/430653409 shutdown_connections 2024-04-26T21:21:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.949+0000 7f90408b1640 1 -- 172.21.15.40:0/430653409 wait complete. 2024-04-26T21:21:28.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.950+0000 7f90408b1640 1 Processor -- start 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 -- start start 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 0x7f903c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f903c072630 con 0x7f903c0b6300 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f903c15d470 con 0x7f903c1459a0 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f90408b1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f903c0727b0 con 0x7f903c15d030 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f903a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f903a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40510/0 (socket says 172.21.15.40:40510) 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.951+0000 7f903a59c640 1 -- 172.21.15.40:0/2279310206 learned_addr learned my addr 172.21.15.40:0/2279310206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 -- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 msgr2=0x7f903c0b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903ad9d640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 0x7f903c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f9033fff640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 0x7f903c0b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 -- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 msgr2=0x7f903c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 -- 172.21.15.40:0/2279310206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c12c110 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f9033fff640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903ad9d640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 0x7f903c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f903a59c640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f902c02a1c0 tx=0x7f902c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c037040 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c042070 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c12c3f0 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c030bb0 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.952+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c0b2380 con 0x7f903c0b6300 2024-04-26T21:21:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.953+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903c0cd550 con 0x7f903c0b6300 2024-04-26T21:21:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.954+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f902c047460 con 0x7f903c0b6300 2024-04-26T21:21:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.954+0000 7f9032ffd640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 0x7f901c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.954+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5227+0+0 (secure 0 0 0) 0x7f902c0ac490 con 0x7f903c0b6300 2024-04-26T21:21:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.954+0000 7f9033fff640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 0x7f901c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.955+0000 7f9033fff640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 0x7f901c06edb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f903c0cd2a0 tx=0x7f9024009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:28.959+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f903c0cd550 con 0x7f903c0b6300 2024-04-26T21:21:29.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.146+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f903c06a290 con 0x7f903c0b6300 2024-04-26T21:21:29.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.828+0000 7f9032ffd640 1 -- 172.21.15.40:0/2279310206 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7f902c07ab30 con 0x7f903c0b6300 2024-04-26T21:21:29.830 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-26T21:21:29.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.831+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 msgr2=0x7f901c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.831+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 0x7f901c06edb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f903c0cd2a0 tx=0x7f9024009040 comp rx=0 tx=0).stop 2024-04-26T21:21:29.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.831+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 msgr2=0x7f903c12a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.831+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f902c02a1c0 tx=0x7f902c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 shutdown_connections 2024-04-26T21:21:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f901c06c8f0 0x7f901c06edb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f903c15d030 0x7f903c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f903c1459a0 0x7f903c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 --2- 172.21.15.40:0/2279310206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f903c0b6300 0x7f903c12a160 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 >> 172.21.15.40:0/2279310206 conn(0x7f903c0d0d30 msgr2=0x7f903c0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:29.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 shutdown_connections 2024-04-26T21:21:29.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.832+0000 7f90408b1640 1 -- 172.21.15.40:0/2279310206 wait complete. 2024-04-26T21:21:29.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:21:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.969+0000 7f97e0faf640 1 Processor -- start 2024-04-26T21:21:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 -- start start 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97dc0d3820 0x7f97dc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc0b60b0 0x7f97dc0d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc0d3c90 con 0x7f97dc15f680 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc15fa60 con 0x7f97dc0d3820 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97e0faf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0d3f50 con 0x7f97dc0b60b0 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.970+0000 7f97d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97dc0d3820 0x7f97dc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97dc0d3820 0x7f97dc0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40512/0 (socket says 172.21.15.40:40512) 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 -- 172.21.15.40:0/174060023 learned_addr learned my addr 172.21.15.40:0/174060023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97da59c640 1 --1- 172.21.15.40:0/174060023 >> v1:172.21.15.40:6789/0 conn(0x7f97dc15f680 0x7f97dc163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55234/0 (socket says 172.21.15.40:55234) 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97dad9d640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc0b60b0 0x7f97dc0d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 -- 172.21.15.40:0/174060023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc0b60b0 msgr2=0x7f97dc0d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc0b60b0 0x7f97dc0d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 -- 172.21.15.40:0/174060023 >> v1:172.21.15.40:6789/0 conn(0x7f97dc15f680 legacy=0x7f97dc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d9d9b640 1 -- 172.21.15.40:0/174060023 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640656800 0 0) 0x7f97dc0d3c90 con 0x7f97dc15f680 2024-04-26T21:21:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.971+0000 7f97d3fff640 1 -- 172.21.15.40:0/174060023 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0b5090 con 0x7f97dc0d3820 2024-04-26T21:21:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97d3fff640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0b5b70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009870 tx=0x7f97c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f6c67974e85583 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97d9d9b640 1 -- 172.21.15.40:0/174060023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4037070 con 0x7f97dc0d3820 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97d9d9b640 1 -- 172.21.15.40:0/174060023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c4032440 con 0x7f97dc0d3820 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97d9d9b640 1 -- 172.21.15.40:0/174060023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4032810 con 0x7f97dc0d3820 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 -- 172.21.15.40:0/174060023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 msgr2=0x7f97dc0b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0b5b70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009870 tx=0x7f97c402a270 comp rx=0 tx=0).stop 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 -- 172.21.15.40:0/174060023 shutdown_connections 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc0b60b0 0x7f97dc0d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 --2- 172.21.15.40:0/174060023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0b5b70 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 -- 172.21.15.40:0/174060023 >> 172.21.15.40:0/174060023 conn(0x7f97dc08eca0 msgr2=0x7f97dc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 -- 172.21.15.40:0/174060023 shutdown_connections 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.972+0000 7f97e0faf640 1 -- 172.21.15.40:0/174060023 wait complete. 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.974+0000 7f97e0faf640 1 Processor -- start 2024-04-26T21:21:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.974+0000 7f97e0faf640 1 -- start start 2024-04-26T21:21:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.974+0000 7f97e0faf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc0b60b0 0x7f97dc1f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc15f680 0x7f97dc0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0d55e0 con 0x7f97dc0d3820 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0d5460 con 0x7f97dc0b60b0 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0d5760 con 0x7f97dc15f680 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40520/0 (socket says 172.21.15.40:40520) 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 -- 172.21.15.40:0/4160364541 learned_addr learned my addr 172.21.15.40:0/4160364541 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97da59c640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc0b60b0 0x7f97dc1f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97dad9d640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc15f680 0x7f97dc0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 -- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc15f680 msgr2=0x7f97dc0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc15f680 0x7f97dc0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 -- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc0b60b0 msgr2=0x7f97dc1f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc0b60b0 0x7f97dc1f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 -- 172.21.15.40:0/4160364541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0d0ec0 con 0x7f97dc0d3820 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d3fff640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009870 tx=0x7f97c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4037040 con 0x7f97dc0d3820 2024-04-26T21:21:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c4004030 con 0x7f97dc0d3820 2024-04-26T21:21:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4004440 con 0x7f97dc0d3820 2024-04-26T21:21:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc0d1160 con 0x7f97dc0d3820 2024-04-26T21:21:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.975+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc0d1720 con 0x7f97dc0d3820 2024-04-26T21:21:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.976+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc06a290 con 0x7f97dc0d3820 2024-04-26T21:21:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.977+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97c403a070 con 0x7f97dc0d3820 2024-04-26T21:21:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.977+0000 7f97d2ffd640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.978+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5609+0+0 (secure 0 0 0) 0x7f97c40ac0f0 con 0x7f97dc0d3820 2024-04-26T21:21:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.978+0000 7f97da59c640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.978+0000 7f97da59c640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f97cc000f60 tx=0x7f97cc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:29.982+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97c4041020 con 0x7f97dc0d3820 2024-04-26T21:21:30.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.206+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f97dc0b4cd0 con 0x7f97dc0d3820 2024-04-26T21:21:30.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.829+0000 7f97d2ffd640 1 -- 172.21.15.40:0/4160364541 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v96) ==== 174+0+0 (secure 0 0 0) 0x7f97c4079e90 con 0x7f97dc0d3820 2024-04-26T21:21:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 22 and data pool 23 2024-04-26T21:21:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.834+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 msgr2=0x7f97bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.834+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f97cc000f60 tx=0x7f97cc007d50 comp rx=0 tx=0).stop 2024-04-26T21:21:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.834+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 msgr2=0x7f97dc0d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.834+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009870 tx=0x7f97c40046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 shutdown_connections 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f97dc15f680 0x7f97dc0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97dc0d3820 0x7f97dc0d0980 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 --2- 172.21.15.40:0/4160364541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc0b60b0 0x7f97dc1f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.835+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 >> 172.21.15.40:0/4160364541 conn(0x7f97dc08eca0 msgr2=0x7f97dc06cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:30.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.838+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 shutdown_connections 2024-04-26T21:21:30.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.840+0000 7f97e0faf640 1 -- 172.21.15.40:0/4160364541 wait complete. 2024-04-26T21:21:30.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.971+0000 7f7df8a09640 1 Processor -- start 2024-04-26T21:21:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.971+0000 7f7df8a09640 1 -- start start 2024-04-26T21:21:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df8a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7df415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df8a09640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df40d1740 con 0x7f7df40d11b0 2024-04-26T21:21:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df8a09640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df40d1a20 con 0x7f7df40cf2e0 2024-04-26T21:21:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df8a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40d1d00 con 0x7f7df406e490 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7debfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7df40d11b0 0x7f7df406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36450/0 (socket says 172.21.15.40:36450) 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7debfff640 1 -- 172.21.15.40:0/1971695422 learned_addr learned my addr 172.21.15.40:0/1971695422 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084835115 0 0) 0x7f7df40d1740 con 0x7f7df40d11b0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7de0003520 con 0x7f7df40d11b0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7df415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1552807148 0 0) 0x7f7de0003520 con 0x7f7df40d11b0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 msgr2=0x7f7df415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7df415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 >> v1:172.21.15.184:6789/0 conn(0x7f7df40cf2e0 legacy=0x7f7df406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df40d1fe0 con 0x7f7df40d11b0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f7ddc001670 con 0x7f7df40d11b0 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 >> v1:172.21.15.40:6789/0 conn(0x7f7df40d11b0 legacy=0x7f7df406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7de00039e0 0x7f7de0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0004800 0x7f7de0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40d1d00 con 0x7f7de0004800 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7de000ba70 con 0x7f7de00039e0 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7de000bc70 con 0x7f7df406e490 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.972+0000 7f7debfff640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0004800 0x7f7de0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7df259c640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7de00039e0 0x7f7de0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 -- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 msgr2=0x7f7de00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7de00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 -- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7de00039e0 msgr2=0x7f7de0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7de00039e0 0x7f7de0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 -- 172.21.15.40:0/1971695422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de0003520 con 0x7f7de0004800 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7df259c640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7de00039e0 0x7f7de0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7df406e490 0x7f7de00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.973+0000 7f7debfff640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0004800 0x7f7de0004c80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc001eb0 tx=0x7f7de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adef1c7da2da637 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc007d30 con 0x7f7de0004800 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc005af0 con 0x7f7de0004800 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df1d9b640 1 -- 172.21.15.40:0/1971695422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc007070 con 0x7f7de0004800 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 -- 172.21.15.40:0/1971695422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0004800 msgr2=0x7f7de0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 --2- 172.21.15.40:0/1971695422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0004800 0x7f7de0004c80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc001eb0 tx=0x7f7de0005830 comp rx=0 tx=0).stop 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df2d9d640 1 -- 172.21.15.40:0/1971695422 reap_dead start 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 -- 172.21.15.40:0/1971695422 shutdown_connections 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 -- 172.21.15.40:0/1971695422 >> 172.21.15.40:0/1971695422 conn(0x7f7df40b4a60 msgr2=0x7f7df40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 -- 172.21.15.40:0/1971695422 shutdown_connections 2024-04-26T21:21:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.974+0000 7f7df8a09640 1 -- 172.21.15.40:0/1971695422 wait complete. 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.975+0000 7f7df8a09640 1 Processor -- start 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.975+0000 7f7df8a09640 1 -- start start 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 0x7f7df415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df415b330 0x7f7df40b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40cf840 con 0x7f7df415df20 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40cf6c0 con 0x7f7df415b330 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df8a09640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40ce660 con 0x7f7de0004800 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58316/0 (socket says 172.21.15.40:58316) 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df2d9d640 1 -- 172.21.15.40:0/12138907 learned_addr learned my addr 172.21.15.40:0/12138907 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7debfff640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df415b330 0x7f7df40b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.976+0000 7f7df259c640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 0x7f7df415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 -- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 msgr2=0x7f7df415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 0x7f7df415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 -- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df415b330 msgr2=0x7f7df40b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df415b330 0x7f7df40b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 -- 172.21.15.40:0/12138907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df415b0a0 con 0x7f7df415df20 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df259c640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 0x7f7df415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df2d9d640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7df406e8a0 tx=0x7f7dec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec016470 con 0x7f7df415df20 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dec01c070 con 0x7f7df415df20 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df415d140 con 0x7f7df415df20 2024-04-26T21:21:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec016850 con 0x7f7df415df20 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.977+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df415d700 con 0x7f7df415df20 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.978+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df40b4600 con 0x7f7df415df20 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.978+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dec005d10 con 0x7f7df415df20 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.979+0000 7f7deaffd640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 0x7f7dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.979+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f7dec08c300 con 0x7f7df415df20 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.979+0000 7f7df259c640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 0x7f7dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.980+0000 7f7df259c640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 0x7f7dd406edb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7de4007fd0 tx=0x7f7de4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:30.983+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7df40b4600 con 0x7f7df415df20 2024-04-26T21:21:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.173+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7df406a290 con 0x7f7df415df20 2024-04-26T21:21:31.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.174+0000 7f7deaffd640 1 -- 172.21.15.40:0/12138907 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16684 (secure 0 0 0) 0x7f7df415b0a0 con 0x7f7df415df20 2024-04-26T21:21:31.176 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:31.176 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":144,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:30.208949+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 msgr2=0x7f7dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 0x7f7dd406edb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7de4007fd0 tx=0x7f7de4009040 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 msgr2=0x7f7df415e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f7df406e8a0 tx=0x7f7dec0115a0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 shutdown_connections 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7dd406c8f0 0x7f7dd406edb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7df415df20 0x7f7df415e3a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7df415b330 0x7f7df40b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 --2- 172.21.15.40:0/12138907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7de0004800 0x7f7df415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 >> 172.21.15.40:0/12138907 conn(0x7f7df40b4a60 msgr2=0x7f7df40c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 shutdown_connections 2024-04-26T21:21:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.177+0000 7f7df8a09640 1 -- 172.21.15.40:0/12138907 wait complete. 2024-04-26T21:21:31.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.289+0000 7fb9a05a4640 1 Processor -- start 2024-04-26T21:21:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.289+0000 7fb9a05a4640 1 -- start start 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb9a05a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb9980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb9a05a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9980d3ee0 0x7fb9980b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb9a05a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9980d03a0 con 0x7fb99815f750 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb9a05a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb9980b4d10 con 0x7fb9980d3ee0 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb9a05a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9980d0680 con 0x7fb99815f370 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99db3f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb99815f750 0x7fb9980d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36462/0 (socket says 172.21.15.40:36462) 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99db3f640 1 -- 172.21.15.40:0/3476878768 learned_addr learned my addr 172.21.15.40:0/3476878768 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> v2:172.21.15.184:3300/0 conn(0x7fb9980d3ee0 0x7fb9980b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99e340640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb9980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637857682 0 0) 0x7fb9980d03a0 con 0x7fb99815f750 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.290+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb984003520 con 0x7fb99815f750 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 589552848 0 0) 0x7fb984003520 con 0x7fb99815f750 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 msgr2=0x7fb9980cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb9980cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 >> v2:172.21.15.184:3300/0 conn(0x7fb9980d3ee0 msgr2=0x7fb9980b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 --2- 172.21.15.40:0/3476878768 >> v2:172.21.15.184:3300/0 conn(0x7fb9980d3ee0 0x7fb9980b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9980d0960 con 0x7fb99815f750 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fb988001670 con 0x7fb99815f750 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> v2:172.21.15.184:3300/0 conn(0x7fb9980d3ee0 0x7fb9980b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 >> v1:172.21.15.40:6789/0 conn(0x7fb99815f750 legacy=0x7fb9980d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb984004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb984004d40 0x7fb9840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9980b4d10 con 0x7fb984004d40 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9980d0680 con 0x7fb9840039b0 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb98400b9c0 con 0x7fb99815f370 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb984004d40 0x7fb9840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99db3f640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb984004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99e340640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 -- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 msgr2=0x7fb984004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb99815f370 0x7fb984004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 -- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9840039b0 msgr2=0x7fb984003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.291+0000 7fb99eb41640 1 -- 172.21.15.40:0/3476878768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb984003520 con 0x7fb984004d40 2024-04-26T21:21:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.292+0000 7fb99eb41640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb984004d40 0x7fb9840051c0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e350 tx=0x7fb984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71482fd6af9655b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.292+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990009de0 con 0x7fb984004d40 2024-04-26T21:21:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.292+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99002e660 con 0x7fb984004d40 2024-04-26T21:21:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.292+0000 7fb99d33e640 1 -- 172.21.15.40:0/3476878768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99002ea10 con 0x7fb984004d40 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 -- 172.21.15.40:0/3476878768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb984004d40 msgr2=0x7fb9840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/3476878768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb984004d40 0x7fb9840051c0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e350 tx=0x7fb984005830 comp rx=0 tx=0).stop 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb99eb41640 1 -- 172.21.15.40:0/3476878768 reap_dead start 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 -- 172.21.15.40:0/3476878768 shutdown_connections 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 -- 172.21.15.40:0/3476878768 >> 172.21.15.40:0/3476878768 conn(0x7fb99808ec10 msgr2=0x7fb99809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 -- 172.21.15.40:0/3476878768 shutdown_connections 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.293+0000 7fb9a05a4640 1 -- 172.21.15.40:0/3476878768 wait complete. 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.294+0000 7fb9a05a4640 1 Processor -- start 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.294+0000 7fb9a05a4640 1 -- start start 2024-04-26T21:21:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb984004d40 0x7fb99815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 0x7fb9981f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb9980cfd70 con 0x7fb9981e6b50 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9980d0f20 con 0x7fb9981e6fd0 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb9a05a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb9980cfef0 con 0x7fb984004d40 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58342/0 (socket says 172.21.15.40:58342) 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 -- 172.21.15.40:0/164662361 learned_addr learned my addr 172.21.15.40:0/164662361 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99e340640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb984004d40 0x7fb99815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99eb41640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 0x7fb9981f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 -- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb984004d40 msgr2=0x7fb99815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb984004d40 0x7fb99815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 -- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 msgr2=0x7fb9981f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 0x7fb9981f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 -- 172.21.15.40:0/164662361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9981e74c0 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99eb41640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 0x7fb9981f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.295+0000 7fb99db3f640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb988007340 tx=0x7fb988007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.296+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988001670 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.296+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988018070 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.296+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98800fb60 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.296+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9981e77a0 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.296+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981e7d60 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.297+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99806a290 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.297+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb98800fd00 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.298+0000 7fb98effd640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.298+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fb98808bb80 con 0x7fb9981e6b50 2024-04-26T21:21:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.298+0000 7fb99e340640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.299+0000 7fb99e340640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 0x7fb97c06edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9980ba350 tx=0x7fb994009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.302+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb988059fb0 con 0x7fb9981e6b50 2024-04-26T21:21:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.488+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9980b6350 con 0x7fb9981e6b50 2024-04-26T21:21:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.489+0000 7fb98effd640 1 -- 172.21.15.40:0/164662361 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16684 (secure 0 0 0) 0x7fb9880598b0 con 0x7fb9981e6b50 2024-04-26T21:21:31.491 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:31.491 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":144,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:30.208949+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 msgr2=0x7fb97c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 0x7fb97c06edb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9980ba350 tx=0x7fb994009190 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 msgr2=0x7fb9981f8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb988007340 tx=0x7fb988007740 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 shutdown_connections 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9981e6fd0 0x7fb9981f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9981e6b50 0x7fb9981f8510 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 --2- 172.21.15.40:0/164662361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb984004d40 0x7fb99815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 >> 172.21.15.40:0/164662361 conn(0x7fb99808ec10 msgr2=0x7fb99806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 shutdown_connections 2024-04-26T21:21:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.491+0000 7fb9a05a4640 1 -- 172.21.15.40:0/164662361 wait complete. 2024-04-26T21:21:31.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:21:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.606+0000 7f8b0c546640 1 Processor -- start 2024-04-26T21:21:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.606+0000 7f8b0c546640 1 -- start start 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.607+0000 7f8b0c546640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b040d3820 0x7f8b040b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.607+0000 7f8b0c546640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b040b60b0 0x7f8b040d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.607+0000 7f8b0c546640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b040d3c90 con 0x7f8b0415f680 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.607+0000 7f8b0c546640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8b0415fa60 con 0x7f8b040b60b0 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.607+0000 7f8b0c546640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8b040d3f50 con 0x7f8b040d3820 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b040b60b0 0x7f8b040d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b040b60b0 0x7f8b040d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58354/0 (socket says 172.21.15.40:58354) 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 -- 172.21.15.40:0/2307332279 learned_addr learned my addr 172.21.15.40:0/2307332279 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b09ae1640 1 --2- 172.21.15.40:0/2307332279 >> v2:172.21.15.184:3300/0 conn(0x7f8b040d3820 0x7f8b040b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 -- 172.21.15.40:0/2307332279 >> v2:172.21.15.184:3300/0 conn(0x7f8b040d3820 msgr2=0x7f8b040b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 --2- 172.21.15.40:0/2307332279 >> v2:172.21.15.184:3300/0 conn(0x7f8b040d3820 0x7f8b040b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 -- 172.21.15.40:0/2307332279 >> v1:172.21.15.184:6789/0 conn(0x7f8b0415f680 legacy=0x7f8b04163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.608+0000 7f8b0aae3640 1 -- 172.21.15.40:0/2307332279 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040b5090 con 0x7f8b040b60b0 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0aae3640 1 --2- 172.21.15.40:0/2307332279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040d4f20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f8b04163040 tx=0x7f8afc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5dcb40b2227652 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b092e0640 1 -- 172.21.15.40:0/2307332279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc036070 con 0x7f8b040b60b0 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b092e0640 1 -- 172.21.15.40:0/2307332279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc031600 con 0x7f8b040b60b0 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b092e0640 1 -- 172.21.15.40:0/2307332279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0319d0 con 0x7f8b040b60b0 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 -- 172.21.15.40:0/2307332279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 msgr2=0x7f8b040d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 --2- 172.21.15.40:0/2307332279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040d4f20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f8b04163040 tx=0x7f8afc02a730 comp rx=0 tx=0).stop 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 -- 172.21.15.40:0/2307332279 shutdown_connections 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 --2- 172.21.15.40:0/2307332279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040d4f20 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 --2- 172.21.15.40:0/2307332279 >> v2:172.21.15.184:3300/0 conn(0x7f8b040d3820 0x7f8b040b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 -- 172.21.15.40:0/2307332279 >> 172.21.15.40:0/2307332279 conn(0x7f8b0408eca0 msgr2=0x7f8b0409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 -- 172.21.15.40:0/2307332279 shutdown_connections 2024-04-26T21:21:31.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.609+0000 7f8b0c546640 1 -- 172.21.15.40:0/2307332279 wait complete. 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 Processor -- start 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 -- start start 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 0x7f8b0415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 0x7f8b040cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040d55e0 con 0x7f8b040b60b0 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040d5460 con 0x7f8b0415f680 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0c546640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b040d5760 con 0x7f8b040d3820 2024-04-26T21:21:31.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0a2e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0a2e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58362/0 (socket says 172.21.15.40:58362) 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.611+0000 7f8b0a2e2640 1 -- 172.21.15.40:0/159782375 learned_addr learned my addr 172.21.15.40:0/159782375 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 -- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 msgr2=0x7f8b0415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b09ae1640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 0x7f8b0415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0aae3640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 0x7f8b040cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 0x7f8b0415dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 -- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 msgr2=0x7f8b040cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 0x7f8b040cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 -- 172.21.15.40:0/159782375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040c9c10 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0aae3640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 0x7f8b040cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b09ae1640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 0x7f8b0415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0a2e2640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c3f0 tx=0x7f8b0000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000052d0 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b00005490 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b041622b0 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00007260 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.612+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b04162870 con 0x7f8b040b60b0 2024-04-26T21:21:31.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.613+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0406a290 con 0x7f8b040b60b0 2024-04-26T21:21:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.614+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b00007400 con 0x7f8b040b60b0 2024-04-26T21:21:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.614+0000 7f8afaffd640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.614+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f8b0008af80 con 0x7f8b040b60b0 2024-04-26T21:21:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.614+0000 7f8b09ae1640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.615+0000 7f8b09ae1640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 0x7f8ae806edb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8b040c91e0 tx=0x7f8af4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.618+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b040c9c10 con 0x7f8b040b60b0 2024-04-26T21:21:31.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:31.853+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8b04162a40 con 0x7f8b040b60b0 2024-04-26T21:21:32.804 INFO:tasks.ceph.mds.c.smithi040.stderr:2024-04-26T21:21:32.803+0000 7f623737f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:21:32.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.849+0000 7f8afaffd640 1 -- 172.21.15.40:0/159782375 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7f8b00059440 con 0x7f8b040b60b0 2024-04-26T21:21:32.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:21:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 msgr2=0x7f8ae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 0x7f8ae806edb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8b040c91e0 tx=0x7f8af4008040 comp rx=0 tx=0).stop 2024-04-26T21:21:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 msgr2=0x7f8b040ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c3f0 tx=0x7f8b0000c8b0 comp rx=0 tx=0).stop 2024-04-26T21:21:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 shutdown_connections 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0415f680 0x7f8b040cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d3820 0x7f8b0415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 --2- 172.21.15.40:0/159782375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040b60b0 0x7f8b040ca690 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 >> 172.21.15.40:0/159782375 conn(0x7f8b0408eca0 msgr2=0x7f8b040b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 shutdown_connections 2024-04-26T21:21:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:32.852+0000 7f8b0c546640 1 -- 172.21.15.40:0/159782375 wait complete. 2024-04-26T21:21:32.855 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:21:32.868 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:21:32.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:21:33.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.052+0000 7f6a5cb89640 1 Processor -- start 2024-04-26T21:21:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.053+0000 7f6a5cb89640 1 -- start start 2024-04-26T21:21:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.053+0000 7f6a5cb89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 0x7f6a580b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.053+0000 7f6a5cb89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580d3f50 con 0x7f6a5815f680 2024-04-26T21:21:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.053+0000 7f6a5cb89640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580b5090 con 0x7f6a580b60b0 2024-04-26T21:21:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.053+0000 7f6a5cb89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5815fa60 con 0x7f6a580d3820 2024-04-26T21:21:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6a5815f680 0x7f6a58163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36468/0 (socket says 172.21.15.40:36468) 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5659c640 1 -- 172.21.15.40:0/501144564 learned_addr learned my addr 172.21.15.40:0/501144564 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a55d9b640 1 --2- 172.21.15.40:0/501144564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 0x7f6a580b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698683650 0 0) 0x7f6a580d3f50 con 0x7f6a5815f680 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a44003520 con 0x7f6a5815f680 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404855399 0 0) 0x7f6a580b5090 con 0x7f6a580b60b0 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a580d3f50 con 0x7f6a580b60b0 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611917608 0 0) 0x7f6a44003520 con 0x7f6a5815f680 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a55d9b640 1 -- 172.21.15.40:0/501144564 >> v1:172.21.15.184:6789/0 conn(0x7f6a580b60b0 legacy=0x7f6a580d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a55d9b640 1 -- 172.21.15.40:0/501144564 >> v1:172.21.15.40:6789/0 conn(0x7f6a5815f680 legacy=0x7f6a58163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.054+0000 7f6a55d9b640 1 -- 172.21.15.40:0/501144564 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580b5370 con 0x7f6a580d3820 2024-04-26T21:21:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.055+0000 7f6a55d9b640 1 --2- 172.21.15.40:0/501144564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 0x7f6a580b5b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a400097d0 tx=0x7f6a4002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830df5148674a766 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.055+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40037070 con 0x7f6a580d3820 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.055+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4002ac30 con 0x7f6a580d3820 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.055+0000 7f6a5559a640 1 -- 172.21.15.40:0/501144564 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50001670 con 0x7f6a580d3820 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 -- 172.21.15.40:0/501144564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 msgr2=0x7f6a580b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/501144564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 0x7f6a580b5b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a400097d0 tx=0x7f6a4002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 -- 172.21.15.40:0/501144564 shutdown_connections 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/501144564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580d3820 0x7f6a580b5b70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 -- 172.21.15.40:0/501144564 >> 172.21.15.40:0/501144564 conn(0x7f6a5808eca0 msgr2=0x7f6a5809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 -- 172.21.15.40:0/501144564 shutdown_connections 2024-04-26T21:21:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.056+0000 7f6a5cb89640 1 -- 172.21.15.40:0/501144564 wait complete. 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.057+0000 7f6a5cb89640 1 Processor -- start 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.057+0000 7f6a5cb89640 1 -- start start 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 0x7f6a580d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 0x7f6a580ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580d57b0 con 0x7f6a5815f680 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580d5630 con 0x7f6a580d3820 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5cb89640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6a580d5930 con 0x7f6a580b60b0 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58370/0 (socket says 172.21.15.40:58370) 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 -- 172.21.15.40:0/362999781 learned_addr learned my addr 172.21.15.40:0/362999781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a55d9b640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 0x7f6a580ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5659c640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 0x7f6a580d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 -- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 msgr2=0x7f6a580d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 0x7f6a580d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 -- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 msgr2=0x7f6a580ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 0x7f6a580ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a56d9d640 1 -- 172.21.15.40:0/362999781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580d1eb0 con 0x7f6a5815f680 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a55d9b640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 0x7f6a580ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.058+0000 7f6a5659c640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 0x7f6a580d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a56d9d640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c790 tx=0x7f6a4800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48003b70 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48003d10 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a580d0e10 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800aa50 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.059+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a580d13d0 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.060+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5806a290 con 0x7f6a5815f680 2024-04-26T21:21:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.060+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a4800abf0 con 0x7f6a5815f680 2024-04-26T21:21:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.061+0000 7f6a4effd640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.061+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f6a4808bb10 con 0x7f6a5815f680 2024-04-26T21:21:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.061+0000 7f6a5659c640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.062+0000 7f6a5659c640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 0x7f6a3806edb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6a500017e0 tx=0x7f6a50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.065+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a48059f40 con 0x7f6a5815f680 2024-04-26T21:21:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.289+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6a5806eca0 con 0x7f6a5815f680 2024-04-26T21:21:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.858+0000 7f6a4effd640 1 -- 172.21.15.40:0/362999781 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7f6a48059840 con 0x7f6a5815f680 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 msgr2=0x7f6a3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 0x7f6a3806edb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6a500017e0 tx=0x7f6a50009040 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 msgr2=0x7f6a580cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c790 tx=0x7f6a4800cc50 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 shutdown_connections 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a5815f680 0x7f6a580cedd0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a580d3820 0x7f6a580ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 --2- 172.21.15.40:0/362999781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a580b60b0 0x7f6a580d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 >> 172.21.15.40:0/362999781 conn(0x7f6a5808eca0 msgr2=0x7f6a580b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 shutdown_connections 2024-04-26T21:21:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.861+0000 7f6a5cb89640 1 -- 172.21.15.40:0/362999781 wait complete. 2024-04-26T21:21:33.876 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:21:33.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.981+0000 7f49e02cc640 1 Processor -- start 2024-04-26T21:21:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.981+0000 7f49e02cc640 1 -- start start 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.982+0000 7f49e02cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49d815f370 0x7f49d80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.982+0000 7f49e02cc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49d815f750 0x7f49d80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.982+0000 7f49e02cc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80d03a0 con 0x7f49d80d37d0 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.982+0000 7f49e02cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80b4d10 con 0x7f49d815f370 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.982+0000 7f49e02cc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80d0680 con 0x7f49d815f750 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49d815f370 0x7f49d80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49d815f370 0x7f49d80cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58374/0 (socket says 172.21.15.40:58374) 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 -- 172.21.15.40:0/462266407 learned_addr learned my addr 172.21.15.40:0/462266407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49dd867640 1 --2- 172.21.15.40:0/462266407 >> v2:172.21.15.184:3300/0 conn(0x7f49d815f750 0x7f49d80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 -- 172.21.15.40:0/462266407 >> v2:172.21.15.184:3300/0 conn(0x7f49d815f750 msgr2=0x7f49d80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 --2- 172.21.15.40:0/462266407 >> v2:172.21.15.184:3300/0 conn(0x7f49d815f750 0x7f49d80b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 -- 172.21.15.40:0/462266407 >> v1:172.21.15.184:6789/0 conn(0x7f49d80d37d0 legacy=0x7f49d80d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.983+0000 7f49de068640 1 -- 172.21.15.40:0/462266407 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80d0960 con 0x7f49d815f370 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49de068640 1 --2- 172.21.15.40:0/462266407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f370 0x7f49d80cfd70 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f49d80d02b0 tx=0x7f49d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe6b93a417dfce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49dd066640 1 -- 172.21.15.40:0/462266407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4037070 con 0x7f49d815f370 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49dd066640 1 -- 172.21.15.40:0/462266407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d4032440 con 0x7f49d815f370 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49dd066640 1 -- 172.21.15.40:0/462266407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4032810 con 0x7f49d815f370 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 -- 172.21.15.40:0/462266407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f370 msgr2=0x7f49d80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 --2- 172.21.15.40:0/462266407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f370 0x7f49d80cfd70 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f49d80d02b0 tx=0x7f49d402a270 comp rx=0 tx=0).stop 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 -- 172.21.15.40:0/462266407 shutdown_connections 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 --2- 172.21.15.40:0/462266407 >> v2:172.21.15.184:3300/0 conn(0x7f49d815f750 0x7f49d80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 --2- 172.21.15.40:0/462266407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f370 0x7f49d80cfd70 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.984+0000 7f49e02cc640 1 -- 172.21.15.40:0/462266407 >> 172.21.15.40:0/462266407 conn(0x7f49d808ec10 msgr2=0x7f49d809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:33.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.985+0000 7f49e02cc640 1 -- 172.21.15.40:0/462266407 shutdown_connections 2024-04-26T21:21:33.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.985+0000 7f49e02cc640 1 -- 172.21.15.40:0/462266407 wait complete. 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 Processor -- start 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 -- start start 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 0x7f49d80baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49d815f370 0x7f49d80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80b61d0 con 0x7f49d815f750 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80d0f20 con 0x7f49d80d37d0 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.986+0000 7f49e02cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80b6350 con 0x7f49d815f370 2024-04-26T21:21:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58388/0 (socket says 172.21.15.40:58388) 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 -- 172.21.15.40:0/2022936591 learned_addr learned my addr 172.21.15.40:0/2022936591 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de068640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 0x7f49d80baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49dd867640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49d815f370 0x7f49d80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 -- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49d815f370 msgr2=0x7f49d80ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49d815f370 0x7f49d80ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 -- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 msgr2=0x7f49d80baaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 0x7f49d80baaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 -- 172.21.15.40:0/2022936591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80cd530 con 0x7f49d815f750 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de068640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 0x7f49d80baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49de869640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c760 tx=0x7f49d000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0016020 con 0x7f49d815f750 2024-04-26T21:21:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0006d50 con 0x7f49d815f750 2024-04-26T21:21:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d80cd810 con 0x7f49d815f750 2024-04-26T21:21:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0003bb0 con 0x7f49d815f750 2024-04-26T21:21:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.987+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d80b7df0 con 0x7f49d815f750 2024-04-26T21:21:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.988+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d806a290 con 0x7f49d815f750 2024-04-26T21:21:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.989+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49d00050d0 con 0x7f49d815f750 2024-04-26T21:21:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.989+0000 7f49ceffd640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 0x7f49bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.989+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f49d008b620 con 0x7f49d815f750 2024-04-26T21:21:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.990+0000 7f49de068640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 0x7f49bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.990+0000 7f49de068640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 0x7f49bc06ede0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f49d4034000 tx=0x7f49d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:33.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:33.994+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49d0020020 con 0x7f49d815f750 2024-04-26T21:21:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.215+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d806e9b0 con 0x7f49d815f750 2024-04-26T21:21:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.216+0000 7f49ceffd640 1 -- 172.21.15.40:0/2022936591 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7f49d0059ae0 con 0x7f49d815f750 2024-04-26T21:21:34.218 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:34.218 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 msgr2=0x7f49bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 0x7f49bc06ede0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f49d4034000 tx=0x7f49d402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 msgr2=0x7f49d80cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c760 tx=0x7f49d000cb60 comp rx=0 tx=0).stop 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 shutdown_connections 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c920 0x7f49bc06ede0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d815f750 0x7f49d80cad30 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49d815f370 0x7f49d80ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 --2- 172.21.15.40:0/2022936591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d80d37d0 0x7f49d80baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 >> 172.21.15.40:0/2022936591 conn(0x7f49d808ec10 msgr2=0x7f49d809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 shutdown_connections 2024-04-26T21:21:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.218+0000 7f49e02cc640 1 -- 172.21.15.40:0/2022936591 wait complete. 2024-04-26T21:21:34.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.329+0000 7f1cf74dc640 1 Processor -- start 2024-04-26T21:21:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.329+0000 7f1cf74dc640 1 -- start start 2024-04-26T21:21:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf74dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf015f720 0x7f1cf00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf74dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 0x7f1cf00d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf74dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00b2510 con 0x7f1cf015f340 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf74dc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf00b27f0 con 0x7f1cf015f720 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf74dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf00b2ad0 con 0x7f1cf00d5610 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf4a77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf015f720 0x7f1cf00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf4a77640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf015f720 0x7f1cf00b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58402/0 (socket says 172.21.15.40:58402) 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf4a77640 1 -- 172.21.15.40:0/3678806725 learned_addr learned my addr 172.21.15.40:0/3678806725 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.330+0000 7f1cf5a79640 1 --2- 172.21.15.40:0/3678806725 >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 0x7f1cf00d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf4a77640 1 -- 172.21.15.40:0/3678806725 >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 msgr2=0x7f1cf00d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf4a77640 1 --2- 172.21.15.40:0/3678806725 >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 0x7f1cf00d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf4a77640 1 -- 172.21.15.40:0/3678806725 >> v1:172.21.15.184:6789/0 conn(0x7f1cf015f340 legacy=0x7f1cf00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf4a77640 1 -- 172.21.15.40:0/3678806725 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00b2db0 con 0x7f1cf015f720 2024-04-26T21:21:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf5a79640 1 --2- 172.21.15.40:0/3678806725 >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 0x7f1cf00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1cf4a77640 1 --2- 172.21.15.40:0/3678806725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f720 0x7f1cf00b1e80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00d4c20 tx=0x7f1ce402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dafa166745d81d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.331+0000 7f1ceffff640 1 -- 172.21.15.40:0/3678806725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037070 con 0x7f1cf015f720 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1ceffff640 1 -- 172.21.15.40:0/3678806725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce4032440 con 0x7f1cf015f720 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1ceffff640 1 -- 172.21.15.40:0/3678806725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4032810 con 0x7f1cf015f720 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 -- 172.21.15.40:0/3678806725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f720 msgr2=0x7f1cf00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/3678806725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f720 0x7f1cf00b1e80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00d4c20 tx=0x7f1ce402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 -- 172.21.15.40:0/3678806725 shutdown_connections 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/3678806725 >> v2:172.21.15.184:3300/0 conn(0x7f1cf00d5610 0x7f1cf00d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/3678806725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f720 0x7f1cf00b1e80 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 -- 172.21.15.40:0/3678806725 >> 172.21.15.40:0/3678806725 conn(0x7f1cf008ec70 msgr2=0x7f1cf009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 -- 172.21.15.40:0/3678806725 shutdown_connections 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.332+0000 7f1cf74dc640 1 -- 172.21.15.40:0/3678806725 wait complete. 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.333+0000 7f1cf74dc640 1 Processor -- start 2024-04-26T21:21:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.333+0000 7f1cf74dc640 1 -- start start 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf00d5610 0x7f1cf00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cf015f720 0x7f1cf00b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00b3940 con 0x7f1cf015f340 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00b37c0 con 0x7f1cf00d5610 2024-04-26T21:21:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf74dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf00b3ac0 con 0x7f1cf015f720 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58406/0 (socket says 172.21.15.40:58406) 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 -- 172.21.15.40:0/1009454834 learned_addr learned my addr 172.21.15.40:0/1009454834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf5a79640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cf015f720 0x7f1cf00b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf5278640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf00d5610 0x7f1cf00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 -- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cf015f720 msgr2=0x7f1cf00b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cf015f720 0x7f1cf00b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 -- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf00d5610 msgr2=0x7f1cf00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf00d5610 0x7f1cf00ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.334+0000 7f1cf4a77640 1 -- 172.21.15.40:0/1009454834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00b9410 con 0x7f1cf015f340 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cf4a77640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1ce402a220 tx=0x7f1ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4041020 con 0x7f1cf015f340 2024-04-26T21:21:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce40306c0 con 0x7f1cf015f340 2024-04-26T21:21:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037040 con 0x7f1cf015f340 2024-04-26T21:21:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf00b96f0 con 0x7f1cf015f340 2024-04-26T21:21:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.335+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00cc6a0 con 0x7f1cf015f340 2024-04-26T21:21:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.336+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf006a290 con 0x7f1cf015f340 2024-04-26T21:21:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.337+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce4044020 con 0x7f1cf015f340 2024-04-26T21:21:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.337+0000 7f1cedffb640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.337+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1ce40ac3b0 con 0x7f1cf015f340 2024-04-26T21:21:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.337+0000 7f1cf5278640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.338+0000 7f1cf5278640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8001640 tx=0x7f1ce8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.342+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ce404b030 con 0x7f1cf015f340 2024-04-26T21:21:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.530+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cf00d3020 con 0x7f1cf015f340 2024-04-26T21:21:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.531+0000 7f1cedffb640 1 -- 172.21.15.40:0/1009454834 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7f1ce407a7e0 con 0x7f1cf015f340 2024-04-26T21:21:34.533 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 msgr2=0x7f1cdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8001640 tx=0x7f1ce8009040 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 msgr2=0x7f1cf00b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1ce402a220 tx=0x7f1ce40046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 shutdown_connections 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1cf015f720 0x7f1cf00b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf015f340 0x7f1cf00b8800 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 --2- 172.21.15.40:0/1009454834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf00d5610 0x7f1cf00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 >> 172.21.15.40:0/1009454834 conn(0x7f1cf008ec70 msgr2=0x7f1cf00d4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 shutdown_connections 2024-04-26T21:21:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.534+0000 7f1cf74dc640 1 -- 172.21.15.40:0/1009454834 wait complete. 2024-04-26T21:21:34.547 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:21:34.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:34.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.647+0000 7f6d12a25640 1 Processor -- start 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.647+0000 7f6d12a25640 1 -- start start 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d12a25640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0c15f1e0 0x7f6d0c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d12a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d12a25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d0c0b4a10 con 0x7f6d0c06d2b0 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d12a25640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0c06eeb0 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d12a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c0b4cf0 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0c15f1e0 0x7f6d0c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0c15f1e0 0x7f6d0c06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:58438/0 (socket says 172.21.15.40:58438) 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:39790/0 (socket says 172.21.15.40:39790) 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 -- 172.21.15.40:0/3441829768 learned_addr learned my addr 172.21.15.40:0/3441829768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d10fc2640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6d0c06d2b0 0x7f6d0c0b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50742/0 (socket says 172.21.15.40:50742) 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 -- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 msgr2=0x7f6d0c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 -- 172.21.15.40:0/3441829768 >> v1:172.21.15.184:6789/0 conn(0x7f6d0c06d2b0 legacy=0x7f6d0c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 -- 172.21.15.40:0/3441829768 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c0b4fd0 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0affd640 1 -- 172.21.15.40:0/3441829768 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2681022878 0 0) 0x7f6d0c0b4a10 con 0x7f6d0c06d2b0 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0b7fe640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0bfff640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c15f1e0 0x7f6d0c06e970 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009870 tx=0x7f6d0002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c307420696e5c1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0affd640 1 -- 172.21.15.40:0/3441829768 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00037070 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0affd640 1 -- 172.21.15.40:0/3441829768 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d00032440 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.648+0000 7f6d0affd640 1 -- 172.21.15.40:0/3441829768 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00032810 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 -- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c15f1e0 msgr2=0x7f6d0c06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c15f1e0 0x7f6d0c06e970 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009870 tx=0x7f6d0002a270 comp rx=0 tx=0).stop 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 -- 172.21.15.40:0/3441829768 shutdown_connections 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c0b44e0 0x7f6d0c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 --2- 172.21.15.40:0/3441829768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c15f1e0 0x7f6d0c06e970 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 -- 172.21.15.40:0/3441829768 >> 172.21.15.40:0/3441829768 conn(0x7f6d0c0728a0 msgr2=0x7f6d0c072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 -- 172.21.15.40:0/3441829768 shutdown_connections 2024-04-26T21:21:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.649+0000 7f6d12a25640 1 -- 172.21.15.40:0/3441829768 wait complete. 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.650+0000 7f6d12a25640 1 Processor -- start 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.650+0000 7f6d12a25640 1 -- start start 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c06d2b0 0x7f6d0c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c15f1e0 0x7f6d0c1f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0aedf0 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0b5590 con 0x7f6d0c06d2b0 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d12a25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c15f5c0 con 0x7f6d0c15f1e0 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58418/0 (socket says 172.21.15.40:58418) 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.651+0000 7f6d0b7fe640 1 -- 172.21.15.40:0/1322960557 learned_addr learned my addr 172.21.15.40:0/1322960557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d10fc2640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c15f1e0 0x7f6d0c1f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0bfff640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c06d2b0 0x7f6d0c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 -- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c15f1e0 msgr2=0x7f6d0c1f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c15f1e0 0x7f6d0c1f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 -- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c06d2b0 msgr2=0x7f6d0c15a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c06d2b0 0x7f6d0c15a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 -- 172.21.15.40:0/1322960557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c15c190 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d0b7fe640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00ebe0 tx=0x7f6cfc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc017070 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00cc30 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc013470 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c15c470 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.652+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c15fa90 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.653+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0c15f880 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.654+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cfc01a070 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.654+0000 7f6d08ff9640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 0x7f6cf806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.654+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f6cfc08bcc0 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.655+0000 7f6d0bfff640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 0x7f6cf806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:34.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.655+0000 7f6d0bfff640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 0x7f6cf806ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6d00034000 tx=0x7f6d0002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:34.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.659+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6cfc059b30 con 0x7f6d0c0b44e0 2024-04-26T21:21:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.838+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d0c073710 con 0x7f6cf806c920 2024-04-26T21:21:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.845+0000 7f6d08ff9640 1 -- 172.21.15.40:0/1322960557 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f6d000040c0 con 0x7f6cf806c920 2024-04-26T21:21:34.850 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 msgr2=0x7f6cf806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 0x7f6cf806ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6d00034000 tx=0x7f6d0002e000 comp rx=0 tx=0).stop 2024-04-26T21:21:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 msgr2=0x7f6d0c15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00ebe0 tx=0x7f6cfc00c520 comp rx=0 tx=0).stop 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 shutdown_connections 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6cf806c920 0x7f6cf806ede0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c15f1e0 0x7f6d0c1f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d0c0b44e0 0x7f6d0c15b940 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 --2- 172.21.15.40:0/1322960557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c06d2b0 0x7f6d0c15a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 >> 172.21.15.40:0/1322960557 conn(0x7f6d0c0728a0 msgr2=0x7f6d0c162de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 shutdown_connections 2024-04-26T21:21:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.848+0000 7f6d12a25640 1 -- 172.21.15.40:0/1322960557 wait complete. 2024-04-26T21:21:34.861 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-26T21:21:33.057015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273148,"kb_used_data":15336,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701572,"statfs":{"total":773094113280,"available":772814409728,"internally_reserved":0,"allocated":15704064,"data_stored":7420988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.231955"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.853332+0000","last_change":"2024-04-26T21:21:30.848314+0000","last_active":"2024-04-26T21:21:30.853332+0000","last_peered":"2024-04-26T21:21:30.853332+0000","last_clean":"2024-04-26T21:21:30.853332+0000","last_became_active":"2024-04-26T21:21:30.848014+0000","last_became_peered":"2024-04-26T21:21:30.848014+0000","last_unstale":"2024-04-26T21:21:30.853332+0000","last_undegraded":"2024-04-26T21:21:30.853332+0000","last_fullsized":"2024-04-26T21:21:30.853332+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866140+0000","last_change":"2024-04-26T21:21:30.862186+0000","last_active":"2024-04-26T21:21:30.866140+0000","last_peered":"2024-04-26T21:21:30.866140+0000","last_clean":"2024-04-26T21:21:30.866140+0000","last_became_active":"2024-04-26T21:21:30.861934+0000","last_became_peered":"2024-04-26T21:21:30.861934+0000","last_unstale":"2024-04-26T21:21:30.866140+0000","last_undegraded":"2024-04-26T21:21:30.866140+0000","last_fullsized":"2024-04-26T21:21:30.866140+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:33.945984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854145+0000","last_change":"2024-04-26T21:21:30.855060+0000","last_active":"2024-04-26T21:21:31.854145+0000","last_peered":"2024-04-26T21:21:31.854145+0000","last_clean":"2024-04-26T21:21:31.854145+0000","last_became_active":"2024-04-26T21:21:30.854740+0000","last_became_peered":"2024-04-26T21:21:30.854740+0000","last_unstale":"2024-04-26T21:21:31.854145+0000","last_undegraded":"2024-04-26T21:21:31.854145+0000","last_fullsized":"2024-04-26T21:21:31.854145+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:08.119391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.842111+0000","last_change":"2024-04-26T21:21:29.842111+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.842111+0000","last_undegraded":"2024-04-26T21:21:29.842111+0000","last_fullsized":"2024-04-26T21:21:29.842111+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866327+0000","last_change":"2024-04-26T21:21:30.862866+0000","last_active":"2024-04-26T21:21:30.866327+0000","last_peered":"2024-04-26T21:21:30.866327+0000","last_clean":"2024-04-26T21:21:30.866327+0000","last_became_active":"2024-04-26T21:21:30.862517+0000","last_became_peered":"2024-04-26T21:21:30.862517+0000","last_unstale":"2024-04-26T21:21:30.866327+0000","last_undegraded":"2024-04-26T21:21:30.866327+0000","last_fullsized":"2024-04-26T21:21:30.866327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:26.736661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241074+0000","last_change":"2024-04-26T21:21:30.848419+0000","last_active":"2024-04-26T21:21:32.241074+0000","last_peered":"2024-04-26T21:21:32.241074+0000","last_clean":"2024-04-26T21:21:32.241074+0000","last_became_active":"2024-04-26T21:21:30.848149+0000","last_became_peered":"2024-04-26T21:21:30.848149+0000","last_unstale":"2024-04-26T21:21:32.241074+0000","last_undegraded":"2024-04-26T21:21:32.241074+0000","last_fullsized":"2024-04-26T21:21:32.241074+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:19.093492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530322+0000","last_change":"2024-04-26T21:21:30.848910+0000","last_active":"2024-04-26T21:21:32.530322+0000","last_peered":"2024-04-26T21:21:32.530322+0000","last_clean":"2024-04-26T21:21:32.530322+0000","last_became_active":"2024-04-26T21:21:30.848596+0000","last_became_peered":"2024-04-26T21:21:30.848596+0000","last_unstale":"2024-04-26T21:21:32.530322+0000","last_undegraded":"2024-04-26T21:21:32.530322+0000","last_fullsized":"2024-04-26T21:21:32.530322+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:01.023665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.868646+0000","last_change":"2024-04-26T21:21:30.863893+0000","last_active":"2024-04-26T21:21:30.868646+0000","last_peered":"2024-04-26T21:21:30.868646+0000","last_clean":"2024-04-26T21:21:30.868646+0000","last_became_active":"2024-04-26T21:21:30.863610+0000","last_became_peered":"2024-04-26T21:21:30.863610+0000","last_unstale":"2024-04-26T21:21:30.868646+0000","last_undegraded":"2024-04-26T21:21:30.868646+0000","last_fullsized":"2024-04-26T21:21:30.868646+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:10.395400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866131+0000","last_change":"2024-04-26T21:21:30.862388+0000","last_active":"2024-04-26T21:21:30.866131+0000","last_peered":"2024-04-26T21:21:30.866131+0000","last_clean":"2024-04-26T21:21:30.866131+0000","last_became_active":"2024-04-26T21:21:30.861676+0000","last_became_peered":"2024-04-26T21:21:30.861676+0000","last_unstale":"2024-04-26T21:21:30.866131+0000","last_undegraded":"2024-04-26T21:21:30.866131+0000","last_fullsized":"2024-04-26T21:21:30.866131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:35.647397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.863462+0000","last_change":"2024-04-26T21:21:30.859687+0000","last_active":"2024-04-26T21:21:30.863462+0000","last_peered":"2024-04-26T21:21:30.863462+0000","last_clean":"2024-04-26T21:21:30.863462+0000","last_became_active":"2024-04-26T21:21:30.859357+0000","last_became_peered":"2024-04-26T21:21:30.859357+0000","last_unstale":"2024-04-26T21:21:30.863462+0000","last_undegraded":"2024-04-26T21:21:30.863462+0000","last_fullsized":"2024-04-26T21:21:30.863462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:35.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.868717+0000","last_change":"2024-04-26T21:21:30.864946+0000","last_active":"2024-04-26T21:21:30.868717+0000","last_peered":"2024-04-26T21:21:30.868717+0000","last_clean":"2024-04-26T21:21:30.868717+0000","last_became_active":"2024-04-26T21:21:30.864531+0000","last_became_peered":"2024-04-26T21:21:30.864531+0000","last_unstale":"2024-04-26T21:21:30.868717+0000","last_undegraded":"2024-04-26T21:21:30.868717+0000","last_fullsized":"2024-04-26T21:21:30.868717+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:43.289270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866078+0000","last_change":"2024-04-26T21:21:30.861937+0000","last_active":"2024-04-26T21:21:30.866078+0000","last_peered":"2024-04-26T21:21:30.866078+0000","last_clean":"2024-04-26T21:21:30.866078+0000","last_became_active":"2024-04-26T21:21:30.861536+0000","last_became_peered":"2024-04-26T21:21:30.861536+0000","last_unstale":"2024-04-26T21:21:30.866078+0000","last_undegraded":"2024-04-26T21:21:30.866078+0000","last_fullsized":"2024-04-26T21:21:30.866078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:21.624243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854148+0000","last_change":"2024-04-26T21:21:30.847985+0000","last_active":"2024-04-26T21:21:31.854148+0000","last_peered":"2024-04-26T21:21:31.854148+0000","last_clean":"2024-04-26T21:21:31.854148+0000","last_became_active":"2024-04-26T21:21:30.847632+0000","last_became_peered":"2024-04-26T21:21:30.847632+0000","last_unstale":"2024-04-26T21:21:31.854148+0000","last_undegraded":"2024-04-26T21:21:31.854148+0000","last_fullsized":"2024-04-26T21:21:31.854148+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:17.557172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853791+0000","last_change":"2024-04-26T21:21:30.845080+0000","last_active":"2024-04-26T21:21:31.853791+0000","last_peered":"2024-04-26T21:21:31.853791+0000","last_clean":"2024-04-26T21:21:31.853791+0000","last_became_active":"2024-04-26T21:21:30.844092+0000","last_became_peered":"2024-04-26T21:21:30.844092+0000","last_unstale":"2024-04-26T21:21:31.853791+0000","last_undegraded":"2024-04-26T21:21:31.853791+0000","last_fullsized":"2024-04-26T21:21:31.853791+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:32.230659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852470+0000","last_change":"2024-04-26T21:21:30.847476+0000","last_active":"2024-04-26T21:21:30.852470+0000","last_peered":"2024-04-26T21:21:30.852470+0000","last_clean":"2024-04-26T21:21:30.852470+0000","last_became_active":"2024-04-26T21:21:30.847138+0000","last_became_peered":"2024-04-26T21:21:30.847138+0000","last_unstale":"2024-04-26T21:21:30.852470+0000","last_undegraded":"2024-04-26T21:21:30.852470+0000","last_fullsized":"2024-04-26T21:21:30.852470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:42.147439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.529916+0000","last_change":"2024-04-26T21:21:30.845831+0000","last_active":"2024-04-26T21:21:32.529916+0000","last_peered":"2024-04-26T21:21:32.529916+0000","last_clean":"2024-04-26T21:21:32.529916+0000","last_became_active":"2024-04-26T21:21:30.845594+0000","last_became_peered":"2024-04-26T21:21:30.845594+0000","last_unstale":"2024-04-26T21:21:32.529916+0000","last_undegraded":"2024-04-26T21:21:32.529916+0000","last_fullsized":"2024-04-26T21:21:32.529916+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241034+0000","last_change":"2024-04-26T21:21:30.844268+0000","last_active":"2024-04-26T21:21:32.241034+0000","last_peered":"2024-04-26T21:21:32.241034+0000","last_clean":"2024-04-26T21:21:32.241034+0000","last_became_active":"2024-04-26T21:21:30.843996+0000","last_became_peered":"2024-04-26T21:21:30.843996+0000","last_unstale":"2024-04-26T21:21:32.241034+0000","last_undegraded":"2024-04-26T21:21:32.241034+0000","last_fullsized":"2024-04-26T21:21:32.241034+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:33.654093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852717+0000","last_change":"2024-04-26T21:21:30.848232+0000","last_active":"2024-04-26T21:21:31.852717+0000","last_peered":"2024-04-26T21:21:31.852717+0000","last_clean":"2024-04-26T21:21:31.852717+0000","last_became_active":"2024-04-26T21:21:30.847922+0000","last_became_peered":"2024-04-26T21:21:30.847922+0000","last_unstale":"2024-04-26T21:21:31.852717+0000","last_undegraded":"2024-04-26T21:21:31.852717+0000","last_fullsized":"2024-04-26T21:21:31.852717+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:42.147439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241138+0000","last_change":"2024-04-26T21:21:30.850019+0000","last_active":"2024-04-26T21:21:32.241138+0000","last_peered":"2024-04-26T21:21:32.241138+0000","last_clean":"2024-04-26T21:21:32.241138+0000","last_became_active":"2024-04-26T21:21:30.849712+0000","last_became_peered":"2024-04-26T21:21:30.849712+0000","last_unstale":"2024-04-26T21:21:32.241138+0000","last_undegraded":"2024-04-26T21:21:32.241138+0000","last_fullsized":"2024-04-26T21:21:32.241138+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:07.677134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854087+0000","last_change":"2024-04-26T21:21:30.846346+0000","last_active":"2024-04-26T21:21:31.854087+0000","last_peered":"2024-04-26T21:21:31.854087+0000","last_clean":"2024-04-26T21:21:31.854087+0000","last_became_active":"2024-04-26T21:21:30.845908+0000","last_became_peered":"2024-04-26T21:21:30.845908+0000","last_unstale":"2024-04-26T21:21:31.854087+0000","last_undegraded":"2024-04-26T21:21:31.854087+0000","last_fullsized":"2024-04-26T21:21:31.854087+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:34.952418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240899+0000","last_change":"2024-04-26T21:21:30.845657+0000","last_active":"2024-04-26T21:21:32.240899+0000","last_peered":"2024-04-26T21:21:32.240899+0000","last_clean":"2024-04-26T21:21:32.240899+0000","last_became_active":"2024-04-26T21:21:30.845370+0000","last_became_peered":"2024-04-26T21:21:30.845370+0000","last_unstale":"2024-04-26T21:21:32.240899+0000","last_undegraded":"2024-04-26T21:21:32.240899+0000","last_fullsized":"2024-04-26T21:21:32.240899+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:37.466259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.863299+0000","last_change":"2024-04-26T21:21:30.859279+0000","last_active":"2024-04-26T21:21:30.863299+0000","last_peered":"2024-04-26T21:21:30.863299+0000","last_clean":"2024-04-26T21:21:30.863299+0000","last_became_active":"2024-04-26T21:21:30.858948+0000","last_became_peered":"2024-04-26T21:21:30.858948+0000","last_unstale":"2024-04-26T21:21:30.863299+0000","last_undegraded":"2024-04-26T21:21:30.863299+0000","last_fullsized":"2024-04-26T21:21:30.863299+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:22.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240942+0000","last_change":"2024-04-26T21:21:30.860430+0000","last_active":"2024-04-26T21:21:32.240942+0000","last_peered":"2024-04-26T21:21:32.240942+0000","last_clean":"2024-04-26T21:21:32.240942+0000","last_became_active":"2024-04-26T21:21:30.860068+0000","last_became_peered":"2024-04-26T21:21:30.860068+0000","last_unstale":"2024-04-26T21:21:32.240942+0000","last_undegraded":"2024-04-26T21:21:32.240942+0000","last_fullsized":"2024-04-26T21:21:32.240942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:06.273802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.864057+0000","last_change":"2024-04-26T21:21:30.858291+0000","last_active":"2024-04-26T21:21:30.864057+0000","last_peered":"2024-04-26T21:21:30.864057+0000","last_clean":"2024-04-26T21:21:30.864057+0000","last_became_active":"2024-04-26T21:21:30.857995+0000","last_became_peered":"2024-04-26T21:21:30.857995+0000","last_unstale":"2024-04-26T21:21:30.864057+0000","last_undegraded":"2024-04-26T21:21:30.864057+0000","last_fullsized":"2024-04-26T21:21:30.864057+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:55.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854453+0000","last_change":"2024-04-26T21:21:30.856008+0000","last_active":"2024-04-26T21:21:31.854453+0000","last_peered":"2024-04-26T21:21:31.854453+0000","last_clean":"2024-04-26T21:21:31.854453+0000","last_became_active":"2024-04-26T21:21:30.855721+0000","last_became_peered":"2024-04-26T21:21:30.855721+0000","last_unstale":"2024-04-26T21:21:31.854453+0000","last_undegraded":"2024-04-26T21:21:31.854453+0000","last_fullsized":"2024-04-26T21:21:31.854453+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:22.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530275+0000","last_change":"2024-04-26T21:21:30.853863+0000","last_active":"2024-04-26T21:21:32.530275+0000","last_peered":"2024-04-26T21:21:32.530275+0000","last_clean":"2024-04-26T21:21:32.530275+0000","last_became_active":"2024-04-26T21:21:30.853517+0000","last_became_peered":"2024-04-26T21:21:30.853517+0000","last_unstale":"2024-04-26T21:21:32.530275+0000","last_undegraded":"2024-04-26T21:21:32.530275+0000","last_fullsized":"2024-04-26T21:21:32.530275+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:34.021913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854217+0000","last_change":"2024-04-26T21:21:30.846757+0000","last_active":"2024-04-26T21:21:31.854217+0000","last_peered":"2024-04-26T21:21:31.854217+0000","last_clean":"2024-04-26T21:21:31.854217+0000","last_became_active":"2024-04-26T21:21:30.846210+0000","last_became_peered":"2024-04-26T21:21:30.846210+0000","last_unstale":"2024-04-26T21:21:31.854217+0000","last_undegraded":"2024-04-26T21:21:31.854217+0000","last_fullsized":"2024-04-26T21:21:31.854217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:47.461550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.841630+0000","last_change":"2024-04-26T21:21:29.841630+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.841630+0000","last_undegraded":"2024-04-26T21:21:29.841630+0000","last_fullsized":"2024-04-26T21:21:29.841630+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853952+0000","last_change":"2024-04-26T21:21:30.846888+0000","last_active":"2024-04-26T21:21:31.853952+0000","last_peered":"2024-04-26T21:21:31.853952+0000","last_clean":"2024-04-26T21:21:31.853952+0000","last_became_active":"2024-04-26T21:21:30.846192+0000","last_became_peered":"2024-04-26T21:21:30.846192+0000","last_unstale":"2024-04-26T21:21:31.853952+0000","last_undegraded":"2024-04-26T21:21:31.853952+0000","last_fullsized":"2024-04-26T21:21:31.853952+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:45.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852209+0000","last_change":"2024-04-26T21:21:30.848354+0000","last_active":"2024-04-26T21:21:31.852209+0000","last_peered":"2024-04-26T21:21:31.852209+0000","last_clean":"2024-04-26T21:21:31.852209+0000","last_became_active":"2024-04-26T21:21:30.848034+0000","last_became_peered":"2024-04-26T21:21:30.848034+0000","last_unstale":"2024-04-26T21:21:31.852209+0000","last_undegraded":"2024-04-26T21:21:31.852209+0000","last_fullsized":"2024-04-26T21:21:31.852209+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:34.678906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852866+0000","last_change":"2024-04-26T21:21:30.854934+0000","last_active":"2024-04-26T21:21:31.852866+0000","last_peered":"2024-04-26T21:21:31.852866+0000","last_clean":"2024-04-26T21:21:31.852866+0000","last_became_active":"2024-04-26T21:21:30.854650+0000","last_became_peered":"2024-04-26T21:21:30.854650+0000","last_unstale":"2024-04-26T21:21:31.852866+0000","last_undegraded":"2024-04-26T21:21:31.852866+0000","last_fullsized":"2024-04-26T21:21:31.852866+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:00.687704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852475+0000","last_change":"2024-04-26T21:21:30.845721+0000","last_active":"2024-04-26T21:21:31.852475+0000","last_peered":"2024-04-26T21:21:31.852475+0000","last_clean":"2024-04-26T21:21:31.852475+0000","last_became_active":"2024-04-26T21:21:30.845448+0000","last_became_peered":"2024-04-26T21:21:30.845448+0000","last_unstale":"2024-04-26T21:21:31.852475+0000","last_undegraded":"2024-04-26T21:21:31.852475+0000","last_fullsized":"2024-04-26T21:21:31.852475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:45.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530435+0000","last_change":"2024-04-26T21:21:30.852633+0000","last_active":"2024-04-26T21:21:32.530435+0000","last_peered":"2024-04-26T21:21:32.530435+0000","last_clean":"2024-04-26T21:21:32.530435+0000","last_became_active":"2024-04-26T21:21:30.852359+0000","last_became_peered":"2024-04-26T21:21:30.852359+0000","last_unstale":"2024-04-26T21:21:32.530435+0000","last_undegraded":"2024-04-26T21:21:32.530435+0000","last_fullsized":"2024-04-26T21:21:32.530435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:46.023515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866612+0000","last_change":"2024-04-26T21:21:29.849064+0000","last_active":"2024-04-26T21:21:30.866612+0000","last_peered":"2024-04-26T21:21:30.866612+0000","last_clean":"2024-04-26T21:21:30.866612+0000","last_became_active":"2024-04-26T21:21:29.848773+0000","last_became_peered":"2024-04-26T21:21:29.848773+0000","last_unstale":"2024-04-26T21:21:30.866612+0000","last_undegraded":"2024-04-26T21:21:30.866612+0000","last_fullsized":"2024-04-26T21:21:30.866612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:44.231241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852689+0000","last_change":"2024-04-26T21:21:30.850740+0000","last_active":"2024-04-26T21:21:31.852689+0000","last_peered":"2024-04-26T21:21:31.852689+0000","last_clean":"2024-04-26T21:21:31.852689+0000","last_became_active":"2024-04-26T21:21:30.850412+0000","last_became_peered":"2024-04-26T21:21:30.850412+0000","last_unstale":"2024-04-26T21:21:31.852689+0000","last_undegraded":"2024-04-26T21:21:31.852689+0000","last_fullsized":"2024-04-26T21:21:31.852689+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:11.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852252+0000","last_change":"2024-04-26T21:21:29.858154+0000","last_active":"2024-04-26T21:21:31.852252+0000","last_peered":"2024-04-26T21:21:31.852252+0000","last_clean":"2024-04-26T21:21:31.852252+0000","last_became_active":"2024-04-26T21:21:29.857824+0000","last_became_peered":"2024-04-26T21:21:29.857824+0000","last_unstale":"2024-04-26T21:21:31.852252+0000","last_undegraded":"2024-04-26T21:21:31.852252+0000","last_fullsized":"2024-04-26T21:21:31.852252+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:42.285348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240902+0000","last_change":"2024-04-26T21:21:30.846692+0000","last_active":"2024-04-26T21:21:32.240902+0000","last_peered":"2024-04-26T21:21:32.240902+0000","last_clean":"2024-04-26T21:21:32.240902+0000","last_became_active":"2024-04-26T21:21:30.846426+0000","last_became_peered":"2024-04-26T21:21:30.846426+0000","last_unstale":"2024-04-26T21:21:32.240902+0000","last_undegraded":"2024-04-26T21:21:32.240902+0000","last_fullsized":"2024-04-26T21:21:32.240902+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:24.786622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.858833+0000","last_change":"2024-04-26T21:21:29.858317+0000","last_active":"2024-04-26T21:21:29.858833+0000","last_peered":"2024-04-26T21:21:29.858833+0000","last_clean":"2024-04-26T21:21:29.858833+0000","last_became_active":"2024-04-26T21:21:29.857984+0000","last_became_peered":"2024-04-26T21:21:29.857984+0000","last_unstale":"2024-04-26T21:21:29.858833+0000","last_undegraded":"2024-04-26T21:21:29.858833+0000","last_fullsized":"2024-04-26T21:21:29.858833+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:03.039910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854313+0000","last_change":"2024-04-26T21:21:30.842973+0000","last_active":"2024-04-26T21:21:31.854313+0000","last_peered":"2024-04-26T21:21:31.854313+0000","last_clean":"2024-04-26T21:21:31.854313+0000","last_became_active":"2024-04-26T21:21:30.842630+0000","last_became_peered":"2024-04-26T21:21:30.842630+0000","last_unstale":"2024-04-26T21:21:31.854313+0000","last_undegraded":"2024-04-26T21:21:31.854313+0000","last_fullsized":"2024-04-26T21:21:31.854313+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:55.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.850486+0000","last_change":"2024-04-26T21:21:29.849851+0000","last_active":"2024-04-26T21:21:29.850486+0000","last_peered":"2024-04-26T21:21:29.850486+0000","last_clean":"2024-04-26T21:21:29.850486+0000","last_became_active":"2024-04-26T21:21:29.849582+0000","last_became_peered":"2024-04-26T21:21:29.849582+0000","last_unstale":"2024-04-26T21:21:29.850486+0000","last_undegraded":"2024-04-26T21:21:29.850486+0000","last_fullsized":"2024-04-26T21:21:29.850486+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:19.930530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530412+0000","last_change":"2024-04-26T21:21:30.847374+0000","last_active":"2024-04-26T21:21:32.530412+0000","last_peered":"2024-04-26T21:21:32.530412+0000","last_clean":"2024-04-26T21:21:32.530412+0000","last_became_active":"2024-04-26T21:21:30.847099+0000","last_became_peered":"2024-04-26T21:21:30.847099+0000","last_unstale":"2024-04-26T21:21:32.530412+0000","last_undegraded":"2024-04-26T21:21:32.530412+0000","last_fullsized":"2024-04-26T21:21:32.530412+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:05.878186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854092+0000","last_change":"2024-04-26T21:21:29.847980+0000","last_active":"2024-04-26T21:21:31.854092+0000","last_peered":"2024-04-26T21:21:31.854092+0000","last_clean":"2024-04-26T21:21:31.854092+0000","last_became_active":"2024-04-26T21:21:29.847681+0000","last_became_peered":"2024-04-26T21:21:29.847681+0000","last_unstale":"2024-04-26T21:21:31.854092+0000","last_undegraded":"2024-04-26T21:21:31.854092+0000","last_fullsized":"2024-04-26T21:21:31.854092+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:33.932902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.840439+0000","last_change":"2024-04-26T21:21:29.840439+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.840439+0000","last_undegraded":"2024-04-26T21:21:29.840439+0000","last_fullsized":"2024-04-26T21:21:29.840439+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.848942+0000","last_change":"2024-04-26T21:21:29.848308+0000","last_active":"2024-04-26T21:21:29.848942+0000","last_peered":"2024-04-26T21:21:29.848942+0000","last_clean":"2024-04-26T21:21:29.848942+0000","last_became_active":"2024-04-26T21:21:29.848020+0000","last_became_peered":"2024-04-26T21:21:29.848020+0000","last_unstale":"2024-04-26T21:21:29.848942+0000","last_undegraded":"2024-04-26T21:21:29.848942+0000","last_fullsized":"2024-04-26T21:21:29.848942+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:22.653947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.839523+0000","last_change":"2024-04-26T21:21:29.839523+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.839523+0000","last_undegraded":"2024-04-26T21:21:29.839523+0000","last_fullsized":"2024-04-26T21:21:29.839523+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852959+0000","last_change":"2024-04-26T21:21:29.854926+0000","last_active":"2024-04-26T21:21:30.852959+0000","last_peered":"2024-04-26T21:21:30.852959+0000","last_clean":"2024-04-26T21:21:30.852959+0000","last_became_active":"2024-04-26T21:21:29.854562+0000","last_became_peered":"2024-04-26T21:21:29.854562+0000","last_unstale":"2024-04-26T21:21:30.852959+0000","last_undegraded":"2024-04-26T21:21:30.852959+0000","last_fullsized":"2024-04-26T21:21:30.852959+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:16.553250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241054+0000","last_change":"2024-04-26T21:21:30.849254+0000","last_active":"2024-04-26T21:21:32.241054+0000","last_peered":"2024-04-26T21:21:32.241054+0000","last_clean":"2024-04-26T21:21:32.241054+0000","last_became_active":"2024-04-26T21:21:30.848998+0000","last_became_peered":"2024-04-26T21:21:30.848998+0000","last_unstale":"2024-04-26T21:21:32.241054+0000","last_undegraded":"2024-04-26T21:21:32.241054+0000","last_fullsized":"2024-04-26T21:21:32.241054+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:04.951704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530071+0000","last_change":"2024-04-26T21:21:29.845962+0000","last_active":"2024-04-26T21:21:32.530071+0000","last_peered":"2024-04-26T21:21:32.530071+0000","last_clean":"2024-04-26T21:21:32.530071+0000","last_became_active":"2024-04-26T21:21:29.845617+0000","last_became_peered":"2024-04-26T21:21:29.845617+0000","last_unstale":"2024-04-26T21:21:32.530071+0000","last_undegraded":"2024-04-26T21:21:32.530071+0000","last_fullsized":"2024-04-26T21:21:32.530071+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:48.064648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852944+0000","last_change":"2024-04-26T21:21:30.850203+0000","last_active":"2024-04-26T21:21:31.852944+0000","last_peered":"2024-04-26T21:21:31.852944+0000","last_clean":"2024-04-26T21:21:31.852944+0000","last_became_active":"2024-04-26T21:21:30.849924+0000","last_became_peered":"2024-04-26T21:21:30.849924+0000","last_unstale":"2024-04-26T21:21:31.852944+0000","last_undegraded":"2024-04-26T21:21:31.852944+0000","last_fullsized":"2024-04-26T21:21:31.852944+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:01.023665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241080+0000","last_change":"2024-04-26T21:21:29.845311+0000","last_active":"2024-04-26T21:21:32.241080+0000","last_peered":"2024-04-26T21:21:32.241080+0000","last_clean":"2024-04-26T21:21:32.241080+0000","last_became_active":"2024-04-26T21:21:29.844638+0000","last_became_peered":"2024-04-26T21:21:29.844638+0000","last_unstale":"2024-04-26T21:21:32.241080+0000","last_undegraded":"2024-04-26T21:21:32.241080+0000","last_fullsized":"2024-04-26T21:21:32.241080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:34.777820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852933+0000","last_change":"2024-04-26T21:21:30.855332+0000","last_active":"2024-04-26T21:21:31.852933+0000","last_peered":"2024-04-26T21:21:31.852933+0000","last_clean":"2024-04-26T21:21:31.852933+0000","last_became_active":"2024-04-26T21:21:30.855051+0000","last_became_peered":"2024-04-26T21:21:30.855051+0000","last_unstale":"2024-04-26T21:21:31.852933+0000","last_undegraded":"2024-04-26T21:21:31.852933+0000","last_fullsized":"2024-04-26T21:21:31.852933+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:23.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241128+0000","last_change":"2024-04-26T21:21:29.841586+0000","last_active":"2024-04-26T21:21:32.241128+0000","last_peered":"2024-04-26T21:21:32.241128+0000","last_clean":"2024-04-26T21:21:32.241128+0000","last_became_active":"2024-04-26T21:21:29.841260+0000","last_became_peered":"2024-04-26T21:21:29.841260+0000","last_unstale":"2024-04-26T21:21:32.241128+0000","last_undegraded":"2024-04-26T21:21:32.241128+0000","last_fullsized":"2024-04-26T21:21:32.241128+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:31.542242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852708+0000","last_change":"2024-04-26T21:21:30.860265+0000","last_active":"2024-04-26T21:21:31.852708+0000","last_peered":"2024-04-26T21:21:31.852708+0000","last_clean":"2024-04-26T21:21:31.852708+0000","last_became_active":"2024-04-26T21:21:30.859431+0000","last_became_peered":"2024-04-26T21:21:30.859431+0000","last_unstale":"2024-04-26T21:21:31.852708+0000","last_undegraded":"2024-04-26T21:21:31.852708+0000","last_fullsized":"2024-04-26T21:21:31.852708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:41.198489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853048+0000","last_change":"2024-04-26T21:21:29.850661+0000","last_active":"2024-04-26T21:21:31.853048+0000","last_peered":"2024-04-26T21:21:31.853048+0000","last_clean":"2024-04-26T21:21:31.853048+0000","last_became_active":"2024-04-26T21:21:29.850358+0000","last_became_peered":"2024-04-26T21:21:29.850358+0000","last_unstale":"2024-04-26T21:21:31.853048+0000","last_undegraded":"2024-04-26T21:21:31.853048+0000","last_fullsized":"2024-04-26T21:21:31.853048+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:25.732739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852571+0000","last_change":"2024-04-26T21:21:30.863866+0000","last_active":"2024-04-26T21:21:31.852571+0000","last_peered":"2024-04-26T21:21:31.852571+0000","last_clean":"2024-04-26T21:21:31.852571+0000","last_became_active":"2024-04-26T21:21:30.863458+0000","last_became_peered":"2024-04-26T21:21:30.863458+0000","last_unstale":"2024-04-26T21:21:31.852571+0000","last_undegraded":"2024-04-26T21:21:31.852571+0000","last_fullsized":"2024-04-26T21:21:31.852571+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:28.992696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.851493+0000","last_change":"2024-04-26T21:21:29.850924+0000","last_active":"2024-04-26T21:21:29.851493+0000","last_peered":"2024-04-26T21:21:29.851493+0000","last_clean":"2024-04-26T21:21:29.851493+0000","last_became_active":"2024-04-26T21:21:29.850609+0000","last_became_peered":"2024-04-26T21:21:29.850609+0000","last_unstale":"2024-04-26T21:21:29.851493+0000","last_undegraded":"2024-04-26T21:21:29.851493+0000","last_fullsized":"2024-04-26T21:21:29.851493+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:14.251065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530297+0000","last_change":"2024-04-26T21:21:30.852635+0000","last_active":"2024-04-26T21:21:32.530297+0000","last_peered":"2024-04-26T21:21:32.530297+0000","last_clean":"2024-04-26T21:21:32.530297+0000","last_became_active":"2024-04-26T21:21:30.850418+0000","last_became_peered":"2024-04-26T21:21:30.850418+0000","last_unstale":"2024-04-26T21:21:32.530297+0000","last_undegraded":"2024-04-26T21:21:32.530297+0000","last_fullsized":"2024-04-26T21:21:32.530297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:23.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852878+0000","last_change":"2024-04-26T21:21:29.844229+0000","last_active":"2024-04-26T21:21:31.852878+0000","last_peered":"2024-04-26T21:21:31.852878+0000","last_clean":"2024-04-26T21:21:31.852878+0000","last_became_active":"2024-04-26T21:21:29.843919+0000","last_became_peered":"2024-04-26T21:21:29.843919+0000","last_unstale":"2024-04-26T21:21:31.852878+0000","last_undegraded":"2024-04-26T21:21:31.852878+0000","last_fullsized":"2024-04-26T21:21:31.852878+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:12.167753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.839002+0000","last_change":"2024-04-26T21:21:29.839002+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.839002+0000","last_undegraded":"2024-04-26T21:21:29.839002+0000","last_fullsized":"2024-04-26T21:21:29.839002+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.858582+0000","last_change":"2024-04-26T21:21:29.857888+0000","last_active":"2024-04-26T21:21:29.858582+0000","last_peered":"2024-04-26T21:21:29.858582+0000","last_clean":"2024-04-26T21:21:29.858582+0000","last_became_active":"2024-04-26T21:21:29.857413+0000","last_became_peered":"2024-04-26T21:21:29.857413+0000","last_unstale":"2024-04-26T21:21:29.858582+0000","last_undegraded":"2024-04-26T21:21:29.858582+0000","last_fullsized":"2024-04-26T21:21:29.858582+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:01.948548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853713+0000","last_change":"2024-04-26T21:21:30.854343+0000","last_active":"2024-04-26T21:21:31.853713+0000","last_peered":"2024-04-26T21:21:31.853713+0000","last_clean":"2024-04-26T21:21:31.853713+0000","last_became_active":"2024-04-26T21:21:30.853942+0000","last_became_peered":"2024-04-26T21:21:30.853942+0000","last_unstale":"2024-04-26T21:21:31.853713+0000","last_undegraded":"2024-04-26T21:21:31.853713+0000","last_fullsized":"2024-04-26T21:21:31.853713+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:42.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":22,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852515+0000","last_change":"2024-04-26T21:21:29.847898+0000","last_active":"2024-04-26T21:21:31.852515+0000","last_peered":"2024-04-26T21:21:31.852515+0000","last_clean":"2024-04-26T21:21:31.852515+0000","last_became_active":"2024-04-26T21:21:29.847617+0000","last_became_peered":"2024-04-26T21:21:29.847617+0000","last_unstale":"2024-04-26T21:21:31.852515+0000","last_undegraded":"2024-04-26T21:21:31.852515+0000","last_fullsized":"2024-04-26T21:21:31.852515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:32.942062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.839596+0000","last_change":"2024-04-26T21:21:29.839596+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.839596+0000","last_undegraded":"2024-04-26T21:21:29.839596+0000","last_fullsized":"2024-04-26T21:21:29.839596+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854396+0000","last_change":"2024-04-26T21:21:29.857918+0000","last_active":"2024-04-26T21:21:31.854396+0000","last_peered":"2024-04-26T21:21:31.854396+0000","last_clean":"2024-04-26T21:21:31.854396+0000","last_became_active":"2024-04-26T21:21:29.857585+0000","last_became_peered":"2024-04-26T21:21:29.857585+0000","last_unstale":"2024-04-26T21:21:31.854396+0000","last_undegraded":"2024-04-26T21:21:31.854396+0000","last_fullsized":"2024-04-26T21:21:31.854396+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:05.269879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852817+0000","last_change":"2024-04-26T21:21:30.848158+0000","last_active":"2024-04-26T21:21:31.852817+0000","last_peered":"2024-04-26T21:21:31.852817+0000","last_clean":"2024-04-26T21:21:31.852817+0000","last_became_active":"2024-04-26T21:21:30.847868+0000","last_became_peered":"2024-04-26T21:21:30.847868+0000","last_unstale":"2024-04-26T21:21:31.852817+0000","last_undegraded":"2024-04-26T21:21:31.852817+0000","last_fullsized":"2024-04-26T21:21:31.852817+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:34.021913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854355+0000","last_change":"2024-04-26T21:21:29.852123+0000","last_active":"2024-04-26T21:21:31.854355+0000","last_peered":"2024-04-26T21:21:31.854355+0000","last_clean":"2024-04-26T21:21:31.854355+0000","last_became_active":"2024-04-26T21:21:29.851249+0000","last_became_peered":"2024-04-26T21:21:29.851249+0000","last_unstale":"2024-04-26T21:21:31.854355+0000","last_undegraded":"2024-04-26T21:21:31.854355+0000","last_fullsized":"2024-04-26T21:21:31.854355+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:36.462336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-26T21:21:29.841185+0000","last_change":"2024-04-26T21:21:29.841185+0000","last_active":"2024-04-26T21:21:29.824405+0000","last_peered":"2024-04-26T21:21:29.824405+0000","last_clean":"2024-04-26T21:21:29.824405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T21:21:29.841185+0000","last_undegraded":"2024-04-26T21:21:29.841185+0000","last_fullsized":"2024-04-26T21:21:29.841185+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866319+0000","last_change":"2024-04-26T21:21:29.846155+0000","last_active":"2024-04-26T21:21:30.866319+0000","last_peered":"2024-04-26T21:21:30.866319+0000","last_clean":"2024-04-26T21:21:30.866319+0000","last_became_active":"2024-04-26T21:21:29.845829+0000","last_became_peered":"2024-04-26T21:21:29.845829+0000","last_unstale":"2024-04-26T21:21:30.866319+0000","last_undegraded":"2024-04-26T21:21:30.866319+0000","last_fullsized":"2024-04-26T21:21:30.866319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:05.058957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530470+0000","last_change":"2024-04-26T21:21:30.851131+0000","last_active":"2024-04-26T21:21:32.530470+0000","last_peered":"2024-04-26T21:21:32.530470+0000","last_clean":"2024-04-26T21:21:32.530470+0000","last_became_active":"2024-04-26T21:21:30.850850+0000","last_became_peered":"2024-04-26T21:21:30.850850+0000","last_unstale":"2024-04-26T21:21:32.530470+0000","last_undegraded":"2024-04-26T21:21:32.530470+0000","last_fullsized":"2024-04-26T21:21:32.530470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:53.964354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842631+0000","last_change":"2024-04-26T21:21:29.851145+0000","last_active":"2024-04-26T21:21:30.842631+0000","last_peered":"2024-04-26T21:21:30.842631+0000","last_clean":"2024-04-26T21:21:30.842631+0000","last_became_active":"2024-04-26T21:21:29.850835+0000","last_became_peered":"2024-04-26T21:21:29.850835+0000","last_unstale":"2024-04-26T21:21:30.842631+0000","last_undegraded":"2024-04-26T21:21:30.842631+0000","last_fullsized":"2024-04-26T21:21:30.842631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:10.881969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853772+0000","last_change":"2024-04-26T21:21:30.846083+0000","last_active":"2024-04-26T21:21:31.853772+0000","last_peered":"2024-04-26T21:21:31.853772+0000","last_clean":"2024-04-26T21:21:31.853772+0000","last_became_active":"2024-04-26T21:21:30.845721+0000","last_became_peered":"2024-04-26T21:21:30.845721+0000","last_unstale":"2024-04-26T21:21:31.853772+0000","last_undegraded":"2024-04-26T21:21:31.853772+0000","last_fullsized":"2024-04-26T21:21:31.853772+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:39.101882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.855975+0000","last_change":"2024-04-26T21:21:29.855288+0000","last_active":"2024-04-26T21:21:29.855975+0000","last_peered":"2024-04-26T21:21:29.855975+0000","last_clean":"2024-04-26T21:21:29.855975+0000","last_became_active":"2024-04-26T21:21:29.854909+0000","last_became_peered":"2024-04-26T21:21:29.854909+0000","last_unstale":"2024-04-26T21:21:29.855975+0000","last_undegraded":"2024-04-26T21:21:29.855975+0000","last_fullsized":"2024-04-26T21:21:29.855975+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:47.062363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530354+0000","last_change":"2024-04-26T21:21:30.852301+0000","last_active":"2024-04-26T21:21:32.530354+0000","last_peered":"2024-04-26T21:21:32.530354+0000","last_clean":"2024-04-26T21:21:32.530354+0000","last_became_active":"2024-04-26T21:21:30.850064+0000","last_became_peered":"2024-04-26T21:21:30.850064+0000","last_unstale":"2024-04-26T21:21:32.530354+0000","last_undegraded":"2024-04-26T21:21:32.530354+0000","last_fullsized":"2024-04-26T21:21:32.530354+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:27.634083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.840422+0000","last_change":"2024-04-26T21:21:29.846529+0000","last_active":"2024-04-26T21:21:30.840422+0000","last_peered":"2024-04-26T21:21:30.840422+0000","last_clean":"2024-04-26T21:21:30.840422+0000","last_became_active":"2024-04-26T21:21:29.844512+0000","last_became_peered":"2024-04-26T21:21:29.844512+0000","last_unstale":"2024-04-26T21:21:30.840422+0000","last_undegraded":"2024-04-26T21:21:30.840422+0000","last_fullsized":"2024-04-26T21:21:30.840422+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:23.867196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866162+0000","last_change":"2024-04-26T21:21:30.861743+0000","last_active":"2024-04-26T21:21:30.866162+0000","last_peered":"2024-04-26T21:21:30.866162+0000","last_clean":"2024-04-26T21:21:30.866162+0000","last_became_active":"2024-04-26T21:21:30.860060+0000","last_became_peered":"2024-04-26T21:21:30.860060+0000","last_unstale":"2024-04-26T21:21:30.866162+0000","last_undegraded":"2024-04-26T21:21:30.866162+0000","last_fullsized":"2024-04-26T21:21:30.866162+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.850046+0000","last_change":"2024-04-26T21:21:29.849421+0000","last_active":"2024-04-26T21:21:29.850046+0000","last_peered":"2024-04-26T21:21:29.850046+0000","last_clean":"2024-04-26T21:21:29.850046+0000","last_became_active":"2024-04-26T21:21:29.849137+0000","last_became_peered":"2024-04-26T21:21:29.849137+0000","last_unstale":"2024-04-26T21:21:29.850046+0000","last_undegraded":"2024-04-26T21:21:29.850046+0000","last_fullsized":"2024-04-26T21:21:29.850046+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:36.336615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241127+0000","last_change":"2024-04-26T21:21:30.860369+0000","last_active":"2024-04-26T21:21:32.241127+0000","last_peered":"2024-04-26T21:21:32.241127+0000","last_clean":"2024-04-26T21:21:32.241127+0000","last_became_active":"2024-04-26T21:21:30.860044+0000","last_became_peered":"2024-04-26T21:21:30.860044+0000","last_unstale":"2024-04-26T21:21:32.241127+0000","last_undegraded":"2024-04-26T21:21:32.241127+0000","last_fullsized":"2024-04-26T21:21:32.241127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:50.382317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842467+0000","last_change":"2024-04-26T21:21:29.847624+0000","last_active":"2024-04-26T21:21:30.842467+0000","last_peered":"2024-04-26T21:21:30.842467+0000","last_clean":"2024-04-26T21:21:30.842467+0000","last_became_active":"2024-04-26T21:21:29.847237+0000","last_became_peered":"2024-04-26T21:21:29.847237+0000","last_unstale":"2024-04-26T21:21:30.842467+0000","last_undegraded":"2024-04-26T21:21:30.842467+0000","last_fullsized":"2024-04-26T21:21:30.842467+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:27.988774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852896+0000","last_change":"2024-04-26T21:21:30.850778+0000","last_active":"2024-04-26T21:21:31.852896+0000","last_peered":"2024-04-26T21:21:31.852896+0000","last_clean":"2024-04-26T21:21:31.852896+0000","last_became_active":"2024-04-26T21:21:30.849972+0000","last_became_peered":"2024-04-26T21:21:30.849972+0000","last_unstale":"2024-04-26T21:21:31.852896+0000","last_undegraded":"2024-04-26T21:21:31.852896+0000","last_fullsized":"2024-04-26T21:21:31.852896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:53.964354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852940+0000","last_change":"2024-04-26T21:21:29.858034+0000","last_active":"2024-04-26T21:21:31.852940+0000","last_peered":"2024-04-26T21:21:31.852940+0000","last_clean":"2024-04-26T21:21:31.852940+0000","last_became_active":"2024-04-26T21:21:29.857698+0000","last_became_peered":"2024-04-26T21:21:29.857698+0000","last_unstale":"2024-04-26T21:21:31.852940+0000","last_undegraded":"2024-04-26T21:21:31.852940+0000","last_fullsized":"2024-04-26T21:21:31.852940+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:18.666803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530396+0000","last_change":"2024-04-26T21:21:30.862583+0000","last_active":"2024-04-26T21:21:32.530396+0000","last_peered":"2024-04-26T21:21:32.530396+0000","last_clean":"2024-04-26T21:21:32.530396+0000","last_became_active":"2024-04-26T21:21:30.862288+0000","last_became_peered":"2024-04-26T21:21:30.862288+0000","last_unstale":"2024-04-26T21:21:32.530396+0000","last_undegraded":"2024-04-26T21:21:32.530396+0000","last_fullsized":"2024-04-26T21:21:32.530396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:43.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854373+0000","last_change":"2024-04-26T21:21:29.847434+0000","last_active":"2024-04-26T21:21:31.854373+0000","last_peered":"2024-04-26T21:21:31.854373+0000","last_clean":"2024-04-26T21:21:31.854373+0000","last_became_active":"2024-04-26T21:21:29.847060+0000","last_became_peered":"2024-04-26T21:21:29.847060+0000","last_unstale":"2024-04-26T21:21:31.854373+0000","last_undegraded":"2024-04-26T21:21:31.854373+0000","last_fullsized":"2024-04-26T21:21:31.854373+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:23.782700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241047+0000","last_change":"2024-04-26T21:21:30.856877+0000","last_active":"2024-04-26T21:21:32.241047+0000","last_peered":"2024-04-26T21:21:32.241047+0000","last_clean":"2024-04-26T21:21:32.241047+0000","last_became_active":"2024-04-26T21:21:30.856490+0000","last_became_peered":"2024-04-26T21:21:30.856490+0000","last_unstale":"2024-04-26T21:21:32.241047+0000","last_undegraded":"2024-04-26T21:21:32.241047+0000","last_fullsized":"2024-04-26T21:21:32.241047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:14.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852406+0000","last_change":"2024-04-26T21:21:29.857449+0000","last_active":"2024-04-26T21:21:30.852406+0000","last_peered":"2024-04-26T21:21:30.852406+0000","last_clean":"2024-04-26T21:21:30.852406+0000","last_became_active":"2024-04-26T21:21:29.857121+0000","last_became_peered":"2024-04-26T21:21:29.857121+0000","last_unstale":"2024-04-26T21:21:30.852406+0000","last_undegraded":"2024-04-26T21:21:30.852406+0000","last_fullsized":"2024-04-26T21:21:30.852406+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:07.115469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852696+0000","last_change":"2024-04-26T21:21:30.848153+0000","last_active":"2024-04-26T21:21:31.852696+0000","last_peered":"2024-04-26T21:21:31.852696+0000","last_clean":"2024-04-26T21:21:31.852696+0000","last_became_active":"2024-04-26T21:21:30.847848+0000","last_became_peered":"2024-04-26T21:21:30.847848+0000","last_unstale":"2024-04-26T21:21:31.852696+0000","last_undegraded":"2024-04-26T21:21:31.852696+0000","last_fullsized":"2024-04-26T21:21:31.852696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:11.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852974+0000","last_change":"2024-04-26T21:21:29.845722+0000","last_active":"2024-04-26T21:21:31.852974+0000","last_peered":"2024-04-26T21:21:31.852974+0000","last_clean":"2024-04-26T21:21:31.852974+0000","last_became_active":"2024-04-26T21:21:29.845257+0000","last_became_peered":"2024-04-26T21:21:29.845257+0000","last_unstale":"2024-04-26T21:21:31.852974+0000","last_undegraded":"2024-04-26T21:21:31.852974+0000","last_fullsized":"2024-04-26T21:21:31.852974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:34.598815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852759+0000","last_change":"2024-04-26T21:21:30.850942+0000","last_active":"2024-04-26T21:21:31.852759+0000","last_peered":"2024-04-26T21:21:31.852759+0000","last_clean":"2024-04-26T21:21:31.852759+0000","last_became_active":"2024-04-26T21:21:30.850664+0000","last_became_peered":"2024-04-26T21:21:30.850664+0000","last_unstale":"2024-04-26T21:21:31.852759+0000","last_undegraded":"2024-04-26T21:21:31.852759+0000","last_fullsized":"2024-04-26T21:21:31.852759+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852133+0000","last_change":"2024-04-26T21:21:29.851281+0000","last_active":"2024-04-26T21:21:31.852133+0000","last_peered":"2024-04-26T21:21:31.852133+0000","last_clean":"2024-04-26T21:21:31.852133+0000","last_became_active":"2024-04-26T21:21:29.851015+0000","last_became_peered":"2024-04-26T21:21:29.851015+0000","last_unstale":"2024-04-26T21:21:31.852133+0000","last_undegraded":"2024-04-26T21:21:31.852133+0000","last_fullsized":"2024-04-26T21:21:31.852133+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:09.391478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852309+0000","last_change":"2024-04-26T21:21:30.856847+0000","last_active":"2024-04-26T21:21:31.852309+0000","last_peered":"2024-04-26T21:21:31.852309+0000","last_clean":"2024-04-26T21:21:31.852309+0000","last_became_active":"2024-04-26T21:21:30.856144+0000","last_became_peered":"2024-04-26T21:21:30.856144+0000","last_unstale":"2024-04-26T21:21:31.852309+0000","last_undegraded":"2024-04-26T21:21:31.852309+0000","last_fullsized":"2024-04-26T21:21:31.852309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:48.213837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-26T21:21:29.848469+0000","last_change":"2024-04-26T21:21:29.845176+0000","last_active":"2024-04-26T21:21:29.848469+0000","last_peered":"2024-04-26T21:21:29.848469+0000","last_clean":"2024-04-26T21:21:29.848469+0000","last_became_active":"2024-04-26T21:21:29.844897+0000","last_became_peered":"2024-04-26T21:21:29.844897+0000","last_unstale":"2024-04-26T21:21:29.848469+0000","last_undegraded":"2024-04-26T21:21:29.848469+0000","last_fullsized":"2024-04-26T21:21:29.848469+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:07.655442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530030+0000","last_change":"2024-04-26T21:21:30.851341+0000","last_active":"2024-04-26T21:21:32.530030+0000","last_peered":"2024-04-26T21:21:32.530030+0000","last_clean":"2024-04-26T21:21:32.530030+0000","last_became_active":"2024-04-26T21:21:30.851041+0000","last_became_peered":"2024-04-26T21:21:30.851041+0000","last_unstale":"2024-04-26T21:21:32.530030+0000","last_undegraded":"2024-04-26T21:21:32.530030+0000","last_fullsized":"2024-04-26T21:21:32.530030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.853068+0000","last_change":"2024-04-26T21:21:29.854796+0000","last_active":"2024-04-26T21:21:30.853068+0000","last_peered":"2024-04-26T21:21:30.853068+0000","last_clean":"2024-04-26T21:21:30.853068+0000","last_became_active":"2024-04-26T21:21:29.854515+0000","last_became_peered":"2024-04-26T21:21:29.854515+0000","last_unstale":"2024-04-26T21:21:30.853068+0000","last_undegraded":"2024-04-26T21:21:30.853068+0000","last_fullsized":"2024-04-26T21:21:30.853068+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:46.457627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853924+0000","last_change":"2024-04-26T21:21:30.848268+0000","last_active":"2024-04-26T21:21:31.853924+0000","last_peered":"2024-04-26T21:21:31.853924+0000","last_clean":"2024-04-26T21:21:31.853924+0000","last_became_active":"2024-04-26T21:21:30.847987+0000","last_became_peered":"2024-04-26T21:21:30.847987+0000","last_unstale":"2024-04-26T21:21:31.853924+0000","last_undegraded":"2024-04-26T21:21:31.853924+0000","last_fullsized":"2024-04-26T21:21:31.853924+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:05.721301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852005+0000","last_change":"2024-04-26T21:21:29.852143+0000","last_active":"2024-04-26T21:21:31.852005+0000","last_peered":"2024-04-26T21:21:31.852005+0000","last_clean":"2024-04-26T21:21:31.852005+0000","last_became_active":"2024-04-26T21:21:29.851645+0000","last_became_peered":"2024-04-26T21:21:29.851645+0000","last_unstale":"2024-04-26T21:21:31.852005+0000","last_undegraded":"2024-04-26T21:21:31.852005+0000","last_fullsized":"2024-04-26T21:21:31.852005+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:34.643475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852743+0000","last_change":"2024-04-26T21:21:30.855681+0000","last_active":"2024-04-26T21:21:31.852743+0000","last_peered":"2024-04-26T21:21:31.852743+0000","last_clean":"2024-04-26T21:21:31.852743+0000","last_became_active":"2024-04-26T21:21:30.855301+0000","last_became_peered":"2024-04-26T21:21:30.855301+0000","last_unstale":"2024-04-26T21:21:31.852743+0000","last_undegraded":"2024-04-26T21:21:31.852743+0000","last_fullsized":"2024-04-26T21:21:31.852743+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852803+0000","last_change":"2024-04-26T21:21:29.850531+0000","last_active":"2024-04-26T21:21:31.852803+0000","last_peered":"2024-04-26T21:21:31.852803+0000","last_clean":"2024-04-26T21:21:31.852803+0000","last_became_active":"2024-04-26T21:21:29.850243+0000","last_became_peered":"2024-04-26T21:21:29.850243+0000","last_unstale":"2024-04-26T21:21:31.852803+0000","last_undegraded":"2024-04-26T21:21:31.852803+0000","last_fullsized":"2024-04-26T21:21:31.852803+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:20.694957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":389,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842530+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:30.842530+0000","last_peered":"2024-04-26T21:21:30.842530+0000","last_clean":"2024-04-26T21:21:30.842530+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:30.842530+0000","last_undegraded":"2024-04-26T21:21:30.842530+0000","last_fullsized":"2024-04-26T21:21:30.842530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2420736,"data_stored":1369849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1806336,"data_stored":777149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1822720,"data_stored":784031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1830912,"data_stored":785109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":783461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37124,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334716,"statfs":{"total":96636764160,"available":96598749184,"internally_reserved":0,"allocated":1818624,"data_stored":783517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":2408448,"data_stored":1373795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1777664,"data_stored":764077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:34.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:34.999+0000 7f4331a07640 1 Processor -- start 2024-04-26T21:21:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 -- start start 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f432c15f750 0x7f432c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f432c0d51a0 0x7f432c0d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f432c0d2440 con 0x7f432c0d3320 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d0030 con 0x7f432c15f750 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f4331a07640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d2720 con 0x7f432c0d51a0 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.000+0000 7f432affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f432c15f750 0x7f432c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f432c15f750 0x7f432c0b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58420/0 (socket says 172.21.15.40:58420) 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 -- 172.21.15.40:0/2816364281 learned_addr learned my addr 172.21.15.40:0/2816364281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f432c0d3320 0x7f432c0d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36474/0 (socket says 172.21.15.40:36474) 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 -- 172.21.15.40:0/2816364281 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d51a0 msgr2=0x7f432c0d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 --2- 172.21.15.40:0/2816364281 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d51a0 0x7f432c0d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 -- 172.21.15.40:0/2816364281 >> v1:172.21.15.40:6789/0 conn(0x7f432c0d3320 legacy=0x7f432c0d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 -- 172.21.15.40:0/2816364281 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0d2a00 con 0x7f432c15f750 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.001+0000 7f432affd640 1 --2- 172.21.15.40:0/2816364281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c15f750 0x7f432c0b50e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f432c08e4a0 tx=0x7f431c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759cea2acb317251 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4329ffb640 1 -- 172.21.15.40:0/2816364281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c036070 con 0x7f432c15f750 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4329ffb640 1 -- 172.21.15.40:0/2816364281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c031600 con 0x7f432c15f750 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4329ffb640 1 -- 172.21.15.40:0/2816364281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0319d0 con 0x7f432c15f750 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 -- 172.21.15.40:0/2816364281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c15f750 msgr2=0x7f432c0b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 --2- 172.21.15.40:0/2816364281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c15f750 0x7f432c0b50e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f432c08e4a0 tx=0x7f431c02a730 comp rx=0 tx=0).stop 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 -- 172.21.15.40:0/2816364281 shutdown_connections 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 --2- 172.21.15.40:0/2816364281 >> v2:172.21.15.184:3300/0 conn(0x7f432c0d51a0 0x7f432c0d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 --2- 172.21.15.40:0/2816364281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c15f750 0x7f432c0b50e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 -- 172.21.15.40:0/2816364281 >> 172.21.15.40:0/2816364281 conn(0x7f432c08ec20 msgr2=0x7f432c09e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 -- 172.21.15.40:0/2816364281 shutdown_connections 2024-04-26T21:21:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.002+0000 7f4331a07640 1 -- 172.21.15.40:0/2816364281 wait complete. 2024-04-26T21:21:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.003+0000 7f4331a07640 1 Processor -- start 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 -- start start 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0d51a0 0x7f432c1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 0x7f432c0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d0850 con 0x7f432c0d3320 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0d06d0 con 0x7f432c15f750 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f4331a07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0d09d0 con 0x7f432c0d51a0 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f432affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f432affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58428/0 (socket says 172.21.15.40:58428) 2024-04-26T21:21:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.004+0000 7f432affd640 1 -- 172.21.15.40:0/4168623983 learned_addr learned my addr 172.21.15.40:0/4168623983 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 -- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0d51a0 msgr2=0x7f432c1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0d51a0 0x7f432c1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 -- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 msgr2=0x7f432c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432b7fe640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 0x7f432c0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 0x7f432c0b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 -- 172.21.15.40:0/4168623983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0b8a10 con 0x7f432c0d3320 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432b7fe640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 0x7f432c0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f432affd640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f431c034040 tx=0x7f431c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c036050 con 0x7f432c0d3320 2024-04-26T21:21:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c042070 con 0x7f432c0d3320 2024-04-26T21:21:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c0b8cf0 con 0x7f432c0d3320 2024-04-26T21:21:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c009510 con 0x7f432c0d3320 2024-04-26T21:21:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.005+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c0b9750 con 0x7f432c0d3320 2024-04-26T21:21:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.006+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c0b9b90 con 0x7f432c0d3320 2024-04-26T21:21:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.007+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431c007c70 con 0x7f432c0d3320 2024-04-26T21:21:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.007+0000 7f4323fff640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.007+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f431c0acae0 con 0x7f432c0d3320 2024-04-26T21:21:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.007+0000 7f432a7fc640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.008+0000 7f432a7fc640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4314000f60 tx=0x7f4314007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.012+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f431c045360 con 0x7f432c0d3320 2024-04-26T21:21:35.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.191+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f432c160070 con 0x7f430c06c8f0 2024-04-26T21:21:35.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.198+0000 7f4323fff640 1 -- 172.21.15.40:0/4168623983 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f432c160070 con 0x7f430c06c8f0 2024-04-26T21:21:35.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 msgr2=0x7f430c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4314000f60 tx=0x7f4314007a00 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 msgr2=0x7f432c15fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f431c034040 tx=0x7f431c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 shutdown_connections 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f430c06c8f0 0x7f430c06edb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f432c15f750 0x7f432c0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f432c0d51a0 0x7f432c1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 --2- 172.21.15.40:0/4168623983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f432c0d3320 0x7f432c15fb30 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 >> 172.21.15.40:0/4168623983 conn(0x7f432c08ec20 msgr2=0x7f432c15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 shutdown_connections 2024-04-26T21:21:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.200+0000 7f4331a07640 1 -- 172.21.15.40:0/4168623983 wait complete. 2024-04-26T21:21:35.213 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-04-26T21:21:35.057551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273184,"kb_used_data":15376,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701536,"statfs":{"total":773094113280,"available":772814372864,"internally_reserved":0,"allocated":15745024,"data_stored":7440372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001231"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.853332+0000","last_change":"2024-04-26T21:21:30.848314+0000","last_active":"2024-04-26T21:21:30.853332+0000","last_peered":"2024-04-26T21:21:30.853332+0000","last_clean":"2024-04-26T21:21:30.853332+0000","last_became_active":"2024-04-26T21:21:30.848014+0000","last_became_peered":"2024-04-26T21:21:30.848014+0000","last_unstale":"2024-04-26T21:21:30.853332+0000","last_undegraded":"2024-04-26T21:21:30.853332+0000","last_fullsized":"2024-04-26T21:21:30.853332+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866140+0000","last_change":"2024-04-26T21:21:30.862186+0000","last_active":"2024-04-26T21:21:30.866140+0000","last_peered":"2024-04-26T21:21:30.866140+0000","last_clean":"2024-04-26T21:21:30.866140+0000","last_became_active":"2024-04-26T21:21:30.861934+0000","last_became_peered":"2024-04-26T21:21:30.861934+0000","last_unstale":"2024-04-26T21:21:30.866140+0000","last_undegraded":"2024-04-26T21:21:30.866140+0000","last_fullsized":"2024-04-26T21:21:30.866140+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:33.945984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854145+0000","last_change":"2024-04-26T21:21:30.855060+0000","last_active":"2024-04-26T21:21:31.854145+0000","last_peered":"2024-04-26T21:21:31.854145+0000","last_clean":"2024-04-26T21:21:31.854145+0000","last_became_active":"2024-04-26T21:21:30.854740+0000","last_became_peered":"2024-04-26T21:21:30.854740+0000","last_unstale":"2024-04-26T21:21:31.854145+0000","last_undegraded":"2024-04-26T21:21:31.854145+0000","last_fullsized":"2024-04-26T21:21:31.854145+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:08.119391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158309+0000","last_change":"2024-04-26T21:21:30.854714+0000","last_active":"2024-04-26T21:21:33.158309+0000","last_peered":"2024-04-26T21:21:33.158309+0000","last_clean":"2024-04-26T21:21:33.158309+0000","last_became_active":"2024-04-26T21:21:30.854423+0000","last_became_peered":"2024-04-26T21:21:30.854423+0000","last_unstale":"2024-04-26T21:21:33.158309+0000","last_undegraded":"2024-04-26T21:21:33.158309+0000","last_fullsized":"2024-04-26T21:21:33.158309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:03.788542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866327+0000","last_change":"2024-04-26T21:21:30.862866+0000","last_active":"2024-04-26T21:21:30.866327+0000","last_peered":"2024-04-26T21:21:30.866327+0000","last_clean":"2024-04-26T21:21:30.866327+0000","last_became_active":"2024-04-26T21:21:30.862517+0000","last_became_peered":"2024-04-26T21:21:30.862517+0000","last_unstale":"2024-04-26T21:21:30.866327+0000","last_undegraded":"2024-04-26T21:21:30.866327+0000","last_fullsized":"2024-04-26T21:21:30.866327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:26.736661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241074+0000","last_change":"2024-04-26T21:21:30.848419+0000","last_active":"2024-04-26T21:21:32.241074+0000","last_peered":"2024-04-26T21:21:32.241074+0000","last_clean":"2024-04-26T21:21:32.241074+0000","last_became_active":"2024-04-26T21:21:30.848149+0000","last_became_peered":"2024-04-26T21:21:30.848149+0000","last_unstale":"2024-04-26T21:21:32.241074+0000","last_undegraded":"2024-04-26T21:21:32.241074+0000","last_fullsized":"2024-04-26T21:21:32.241074+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:19.093492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530322+0000","last_change":"2024-04-26T21:21:30.848910+0000","last_active":"2024-04-26T21:21:32.530322+0000","last_peered":"2024-04-26T21:21:32.530322+0000","last_clean":"2024-04-26T21:21:32.530322+0000","last_became_active":"2024-04-26T21:21:30.848596+0000","last_became_peered":"2024-04-26T21:21:30.848596+0000","last_unstale":"2024-04-26T21:21:32.530322+0000","last_undegraded":"2024-04-26T21:21:32.530322+0000","last_fullsized":"2024-04-26T21:21:32.530322+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:01.023665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.868646+0000","last_change":"2024-04-26T21:21:30.863893+0000","last_active":"2024-04-26T21:21:30.868646+0000","last_peered":"2024-04-26T21:21:30.868646+0000","last_clean":"2024-04-26T21:21:30.868646+0000","last_became_active":"2024-04-26T21:21:30.863610+0000","last_became_peered":"2024-04-26T21:21:30.863610+0000","last_unstale":"2024-04-26T21:21:30.868646+0000","last_undegraded":"2024-04-26T21:21:30.868646+0000","last_fullsized":"2024-04-26T21:21:30.868646+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:10.395400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866131+0000","last_change":"2024-04-26T21:21:30.862388+0000","last_active":"2024-04-26T21:21:30.866131+0000","last_peered":"2024-04-26T21:21:30.866131+0000","last_clean":"2024-04-26T21:21:30.866131+0000","last_became_active":"2024-04-26T21:21:30.861676+0000","last_became_peered":"2024-04-26T21:21:30.861676+0000","last_unstale":"2024-04-26T21:21:30.866131+0000","last_undegraded":"2024-04-26T21:21:30.866131+0000","last_fullsized":"2024-04-26T21:21:30.866131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:35.647397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.863462+0000","last_change":"2024-04-26T21:21:30.859687+0000","last_active":"2024-04-26T21:21:30.863462+0000","last_peered":"2024-04-26T21:21:30.863462+0000","last_clean":"2024-04-26T21:21:30.863462+0000","last_became_active":"2024-04-26T21:21:30.859357+0000","last_became_peered":"2024-04-26T21:21:30.859357+0000","last_unstale":"2024-04-26T21:21:30.863462+0000","last_undegraded":"2024-04-26T21:21:30.863462+0000","last_fullsized":"2024-04-26T21:21:30.863462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:35.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.868717+0000","last_change":"2024-04-26T21:21:30.864946+0000","last_active":"2024-04-26T21:21:30.868717+0000","last_peered":"2024-04-26T21:21:30.868717+0000","last_clean":"2024-04-26T21:21:30.868717+0000","last_became_active":"2024-04-26T21:21:30.864531+0000","last_became_peered":"2024-04-26T21:21:30.864531+0000","last_unstale":"2024-04-26T21:21:30.868717+0000","last_undegraded":"2024-04-26T21:21:30.868717+0000","last_fullsized":"2024-04-26T21:21:30.868717+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:43.289270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866078+0000","last_change":"2024-04-26T21:21:30.861937+0000","last_active":"2024-04-26T21:21:30.866078+0000","last_peered":"2024-04-26T21:21:30.866078+0000","last_clean":"2024-04-26T21:21:30.866078+0000","last_became_active":"2024-04-26T21:21:30.861536+0000","last_became_peered":"2024-04-26T21:21:30.861536+0000","last_unstale":"2024-04-26T21:21:30.866078+0000","last_undegraded":"2024-04-26T21:21:30.866078+0000","last_fullsized":"2024-04-26T21:21:30.866078+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:21.624243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854148+0000","last_change":"2024-04-26T21:21:30.847985+0000","last_active":"2024-04-26T21:21:31.854148+0000","last_peered":"2024-04-26T21:21:31.854148+0000","last_clean":"2024-04-26T21:21:31.854148+0000","last_became_active":"2024-04-26T21:21:30.847632+0000","last_became_peered":"2024-04-26T21:21:30.847632+0000","last_unstale":"2024-04-26T21:21:31.854148+0000","last_undegraded":"2024-04-26T21:21:31.854148+0000","last_fullsized":"2024-04-26T21:21:31.854148+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:17.557172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853791+0000","last_change":"2024-04-26T21:21:30.845080+0000","last_active":"2024-04-26T21:21:31.853791+0000","last_peered":"2024-04-26T21:21:31.853791+0000","last_clean":"2024-04-26T21:21:31.853791+0000","last_became_active":"2024-04-26T21:21:30.844092+0000","last_became_peered":"2024-04-26T21:21:30.844092+0000","last_unstale":"2024-04-26T21:21:31.853791+0000","last_undegraded":"2024-04-26T21:21:31.853791+0000","last_fullsized":"2024-04-26T21:21:31.853791+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:32.230659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852470+0000","last_change":"2024-04-26T21:21:30.847476+0000","last_active":"2024-04-26T21:21:30.852470+0000","last_peered":"2024-04-26T21:21:30.852470+0000","last_clean":"2024-04-26T21:21:30.852470+0000","last_became_active":"2024-04-26T21:21:30.847138+0000","last_became_peered":"2024-04-26T21:21:30.847138+0000","last_unstale":"2024-04-26T21:21:30.852470+0000","last_undegraded":"2024-04-26T21:21:30.852470+0000","last_fullsized":"2024-04-26T21:21:30.852470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:42.147439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.529916+0000","last_change":"2024-04-26T21:21:30.845831+0000","last_active":"2024-04-26T21:21:32.529916+0000","last_peered":"2024-04-26T21:21:32.529916+0000","last_clean":"2024-04-26T21:21:32.529916+0000","last_became_active":"2024-04-26T21:21:30.845594+0000","last_became_peered":"2024-04-26T21:21:30.845594+0000","last_unstale":"2024-04-26T21:21:32.529916+0000","last_undegraded":"2024-04-26T21:21:32.529916+0000","last_fullsized":"2024-04-26T21:21:32.529916+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241034+0000","last_change":"2024-04-26T21:21:30.844268+0000","last_active":"2024-04-26T21:21:32.241034+0000","last_peered":"2024-04-26T21:21:32.241034+0000","last_clean":"2024-04-26T21:21:32.241034+0000","last_became_active":"2024-04-26T21:21:30.843996+0000","last_became_peered":"2024-04-26T21:21:30.843996+0000","last_unstale":"2024-04-26T21:21:32.241034+0000","last_undegraded":"2024-04-26T21:21:32.241034+0000","last_fullsized":"2024-04-26T21:21:32.241034+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:33.654093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852717+0000","last_change":"2024-04-26T21:21:30.848232+0000","last_active":"2024-04-26T21:21:31.852717+0000","last_peered":"2024-04-26T21:21:31.852717+0000","last_clean":"2024-04-26T21:21:31.852717+0000","last_became_active":"2024-04-26T21:21:30.847922+0000","last_became_peered":"2024-04-26T21:21:30.847922+0000","last_unstale":"2024-04-26T21:21:31.852717+0000","last_undegraded":"2024-04-26T21:21:31.852717+0000","last_fullsized":"2024-04-26T21:21:31.852717+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:42.147439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241138+0000","last_change":"2024-04-26T21:21:30.850019+0000","last_active":"2024-04-26T21:21:32.241138+0000","last_peered":"2024-04-26T21:21:32.241138+0000","last_clean":"2024-04-26T21:21:32.241138+0000","last_became_active":"2024-04-26T21:21:30.849712+0000","last_became_peered":"2024-04-26T21:21:30.849712+0000","last_unstale":"2024-04-26T21:21:32.241138+0000","last_undegraded":"2024-04-26T21:21:32.241138+0000","last_fullsized":"2024-04-26T21:21:32.241138+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:07.677134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854087+0000","last_change":"2024-04-26T21:21:30.846346+0000","last_active":"2024-04-26T21:21:31.854087+0000","last_peered":"2024-04-26T21:21:31.854087+0000","last_clean":"2024-04-26T21:21:31.854087+0000","last_became_active":"2024-04-26T21:21:30.845908+0000","last_became_peered":"2024-04-26T21:21:30.845908+0000","last_unstale":"2024-04-26T21:21:31.854087+0000","last_undegraded":"2024-04-26T21:21:31.854087+0000","last_fullsized":"2024-04-26T21:21:31.854087+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:34.952418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240899+0000","last_change":"2024-04-26T21:21:30.845657+0000","last_active":"2024-04-26T21:21:32.240899+0000","last_peered":"2024-04-26T21:21:32.240899+0000","last_clean":"2024-04-26T21:21:32.240899+0000","last_became_active":"2024-04-26T21:21:30.845370+0000","last_became_peered":"2024-04-26T21:21:30.845370+0000","last_unstale":"2024-04-26T21:21:32.240899+0000","last_undegraded":"2024-04-26T21:21:32.240899+0000","last_fullsized":"2024-04-26T21:21:32.240899+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:37.466259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.863299+0000","last_change":"2024-04-26T21:21:30.859279+0000","last_active":"2024-04-26T21:21:30.863299+0000","last_peered":"2024-04-26T21:21:30.863299+0000","last_clean":"2024-04-26T21:21:30.863299+0000","last_became_active":"2024-04-26T21:21:30.858948+0000","last_became_peered":"2024-04-26T21:21:30.858948+0000","last_unstale":"2024-04-26T21:21:30.863299+0000","last_undegraded":"2024-04-26T21:21:30.863299+0000","last_fullsized":"2024-04-26T21:21:30.863299+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:22.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240942+0000","last_change":"2024-04-26T21:21:30.860430+0000","last_active":"2024-04-26T21:21:32.240942+0000","last_peered":"2024-04-26T21:21:32.240942+0000","last_clean":"2024-04-26T21:21:32.240942+0000","last_became_active":"2024-04-26T21:21:30.860068+0000","last_became_peered":"2024-04-26T21:21:30.860068+0000","last_unstale":"2024-04-26T21:21:32.240942+0000","last_undegraded":"2024-04-26T21:21:32.240942+0000","last_fullsized":"2024-04-26T21:21:32.240942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:06.273802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.864057+0000","last_change":"2024-04-26T21:21:30.858291+0000","last_active":"2024-04-26T21:21:30.864057+0000","last_peered":"2024-04-26T21:21:30.864057+0000","last_clean":"2024-04-26T21:21:30.864057+0000","last_became_active":"2024-04-26T21:21:30.857995+0000","last_became_peered":"2024-04-26T21:21:30.857995+0000","last_unstale":"2024-04-26T21:21:30.864057+0000","last_undegraded":"2024-04-26T21:21:30.864057+0000","last_fullsized":"2024-04-26T21:21:30.864057+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:55.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854453+0000","last_change":"2024-04-26T21:21:30.856008+0000","last_active":"2024-04-26T21:21:31.854453+0000","last_peered":"2024-04-26T21:21:31.854453+0000","last_clean":"2024-04-26T21:21:31.854453+0000","last_became_active":"2024-04-26T21:21:30.855721+0000","last_became_peered":"2024-04-26T21:21:30.855721+0000","last_unstale":"2024-04-26T21:21:31.854453+0000","last_undegraded":"2024-04-26T21:21:31.854453+0000","last_fullsized":"2024-04-26T21:21:31.854453+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:22.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530275+0000","last_change":"2024-04-26T21:21:30.853863+0000","last_active":"2024-04-26T21:21:32.530275+0000","last_peered":"2024-04-26T21:21:32.530275+0000","last_clean":"2024-04-26T21:21:32.530275+0000","last_became_active":"2024-04-26T21:21:30.853517+0000","last_became_peered":"2024-04-26T21:21:30.853517+0000","last_unstale":"2024-04-26T21:21:32.530275+0000","last_undegraded":"2024-04-26T21:21:32.530275+0000","last_fullsized":"2024-04-26T21:21:32.530275+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:34.021913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854217+0000","last_change":"2024-04-26T21:21:30.846757+0000","last_active":"2024-04-26T21:21:31.854217+0000","last_peered":"2024-04-26T21:21:31.854217+0000","last_clean":"2024-04-26T21:21:31.854217+0000","last_became_active":"2024-04-26T21:21:30.846210+0000","last_became_peered":"2024-04-26T21:21:30.846210+0000","last_unstale":"2024-04-26T21:21:31.854217+0000","last_undegraded":"2024-04-26T21:21:31.854217+0000","last_fullsized":"2024-04-26T21:21:31.854217+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:47.461550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158358+0000","last_change":"2024-04-26T21:21:30.852488+0000","last_active":"2024-04-26T21:21:33.158358+0000","last_peered":"2024-04-26T21:21:33.158358+0000","last_clean":"2024-04-26T21:21:33.158358+0000","last_became_active":"2024-04-26T21:21:30.852243+0000","last_became_peered":"2024-04-26T21:21:30.852243+0000","last_unstale":"2024-04-26T21:21:33.158358+0000","last_undegraded":"2024-04-26T21:21:33.158358+0000","last_fullsized":"2024-04-26T21:21:33.158358+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:16.906559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853952+0000","last_change":"2024-04-26T21:21:30.846888+0000","last_active":"2024-04-26T21:21:31.853952+0000","last_peered":"2024-04-26T21:21:31.853952+0000","last_clean":"2024-04-26T21:21:31.853952+0000","last_became_active":"2024-04-26T21:21:30.846192+0000","last_became_peered":"2024-04-26T21:21:30.846192+0000","last_unstale":"2024-04-26T21:21:31.853952+0000","last_undegraded":"2024-04-26T21:21:31.853952+0000","last_fullsized":"2024-04-26T21:21:31.853952+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:08:45.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852209+0000","last_change":"2024-04-26T21:21:30.848354+0000","last_active":"2024-04-26T21:21:31.852209+0000","last_peered":"2024-04-26T21:21:31.852209+0000","last_clean":"2024-04-26T21:21:31.852209+0000","last_became_active":"2024-04-26T21:21:30.848034+0000","last_became_peered":"2024-04-26T21:21:30.848034+0000","last_unstale":"2024-04-26T21:21:31.852209+0000","last_undegraded":"2024-04-26T21:21:31.852209+0000","last_fullsized":"2024-04-26T21:21:31.852209+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:34.678906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852866+0000","last_change":"2024-04-26T21:21:30.854934+0000","last_active":"2024-04-26T21:21:31.852866+0000","last_peered":"2024-04-26T21:21:31.852866+0000","last_clean":"2024-04-26T21:21:31.852866+0000","last_became_active":"2024-04-26T21:21:30.854650+0000","last_became_peered":"2024-04-26T21:21:30.854650+0000","last_unstale":"2024-04-26T21:21:31.852866+0000","last_undegraded":"2024-04-26T21:21:31.852866+0000","last_fullsized":"2024-04-26T21:21:31.852866+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:00.687704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852475+0000","last_change":"2024-04-26T21:21:30.845721+0000","last_active":"2024-04-26T21:21:31.852475+0000","last_peered":"2024-04-26T21:21:31.852475+0000","last_clean":"2024-04-26T21:21:31.852475+0000","last_became_active":"2024-04-26T21:21:30.845448+0000","last_became_peered":"2024-04-26T21:21:30.845448+0000","last_unstale":"2024-04-26T21:21:31.852475+0000","last_undegraded":"2024-04-26T21:21:31.852475+0000","last_fullsized":"2024-04-26T21:21:31.852475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:45.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530435+0000","last_change":"2024-04-26T21:21:30.852633+0000","last_active":"2024-04-26T21:21:32.530435+0000","last_peered":"2024-04-26T21:21:32.530435+0000","last_clean":"2024-04-26T21:21:32.530435+0000","last_became_active":"2024-04-26T21:21:30.852359+0000","last_became_peered":"2024-04-26T21:21:30.852359+0000","last_unstale":"2024-04-26T21:21:32.530435+0000","last_undegraded":"2024-04-26T21:21:32.530435+0000","last_fullsized":"2024-04-26T21:21:32.530435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:46.023515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":16,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866612+0000","last_change":"2024-04-26T21:21:29.849064+0000","last_active":"2024-04-26T21:21:30.866612+0000","last_peered":"2024-04-26T21:21:30.866612+0000","last_clean":"2024-04-26T21:21:30.866612+0000","last_became_active":"2024-04-26T21:21:29.848773+0000","last_became_peered":"2024-04-26T21:21:29.848773+0000","last_unstale":"2024-04-26T21:21:30.866612+0000","last_undegraded":"2024-04-26T21:21:30.866612+0000","last_fullsized":"2024-04-26T21:21:30.866612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:04:44.231241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852689+0000","last_change":"2024-04-26T21:21:30.850740+0000","last_active":"2024-04-26T21:21:31.852689+0000","last_peered":"2024-04-26T21:21:31.852689+0000","last_clean":"2024-04-26T21:21:31.852689+0000","last_became_active":"2024-04-26T21:21:30.850412+0000","last_became_peered":"2024-04-26T21:21:30.850412+0000","last_unstale":"2024-04-26T21:21:31.852689+0000","last_undegraded":"2024-04-26T21:21:31.852689+0000","last_fullsized":"2024-04-26T21:21:31.852689+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:11.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852252+0000","last_change":"2024-04-26T21:21:29.858154+0000","last_active":"2024-04-26T21:21:31.852252+0000","last_peered":"2024-04-26T21:21:31.852252+0000","last_clean":"2024-04-26T21:21:31.852252+0000","last_became_active":"2024-04-26T21:21:29.857824+0000","last_became_peered":"2024-04-26T21:21:29.857824+0000","last_unstale":"2024-04-26T21:21:31.852252+0000","last_undegraded":"2024-04-26T21:21:31.852252+0000","last_fullsized":"2024-04-26T21:21:31.852252+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:42.285348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.240902+0000","last_change":"2024-04-26T21:21:30.846692+0000","last_active":"2024-04-26T21:21:32.240902+0000","last_peered":"2024-04-26T21:21:32.240902+0000","last_clean":"2024-04-26T21:21:32.240902+0000","last_became_active":"2024-04-26T21:21:30.846426+0000","last_became_peered":"2024-04-26T21:21:30.846426+0000","last_unstale":"2024-04-26T21:21:32.240902+0000","last_undegraded":"2024-04-26T21:21:32.240902+0000","last_fullsized":"2024-04-26T21:21:32.240902+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:24.786622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158507+0000","last_change":"2024-04-26T21:21:29.858317+0000","last_active":"2024-04-26T21:21:33.158507+0000","last_peered":"2024-04-26T21:21:33.158507+0000","last_clean":"2024-04-26T21:21:33.158507+0000","last_became_active":"2024-04-26T21:21:29.857984+0000","last_became_peered":"2024-04-26T21:21:29.857984+0000","last_unstale":"2024-04-26T21:21:33.158507+0000","last_undegraded":"2024-04-26T21:21:33.158507+0000","last_fullsized":"2024-04-26T21:21:33.158507+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:09.003586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854313+0000","last_change":"2024-04-26T21:21:30.842973+0000","last_active":"2024-04-26T21:21:31.854313+0000","last_peered":"2024-04-26T21:21:31.854313+0000","last_clean":"2024-04-26T21:21:31.854313+0000","last_became_active":"2024-04-26T21:21:30.842630+0000","last_became_peered":"2024-04-26T21:21:30.842630+0000","last_unstale":"2024-04-26T21:21:31.854313+0000","last_undegraded":"2024-04-26T21:21:31.854313+0000","last_fullsized":"2024-04-26T21:21:31.854313+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:55.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158465+0000","last_change":"2024-04-26T21:21:29.849851+0000","last_active":"2024-04-26T21:21:33.158465+0000","last_peered":"2024-04-26T21:21:33.158465+0000","last_clean":"2024-04-26T21:21:33.158465+0000","last_became_active":"2024-04-26T21:21:29.849582+0000","last_became_peered":"2024-04-26T21:21:29.849582+0000","last_unstale":"2024-04-26T21:21:33.158465+0000","last_undegraded":"2024-04-26T21:21:33.158465+0000","last_fullsized":"2024-04-26T21:21:33.158465+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:33:14.971322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530412+0000","last_change":"2024-04-26T21:21:30.847374+0000","last_active":"2024-04-26T21:21:32.530412+0000","last_peered":"2024-04-26T21:21:32.530412+0000","last_clean":"2024-04-26T21:21:32.530412+0000","last_became_active":"2024-04-26T21:21:30.847099+0000","last_became_peered":"2024-04-26T21:21:30.847099+0000","last_unstale":"2024-04-26T21:21:32.530412+0000","last_undegraded":"2024-04-26T21:21:32.530412+0000","last_fullsized":"2024-04-26T21:21:32.530412+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:05.878186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854092+0000","last_change":"2024-04-26T21:21:29.847980+0000","last_active":"2024-04-26T21:21:31.854092+0000","last_peered":"2024-04-26T21:21:31.854092+0000","last_clean":"2024-04-26T21:21:31.854092+0000","last_became_active":"2024-04-26T21:21:29.847681+0000","last_became_peered":"2024-04-26T21:21:29.847681+0000","last_unstale":"2024-04-26T21:21:31.854092+0000","last_undegraded":"2024-04-26T21:21:31.854092+0000","last_fullsized":"2024-04-26T21:21:31.854092+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:33.932902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158423+0000","last_change":"2024-04-26T21:21:30.850411+0000","last_active":"2024-04-26T21:21:33.158423+0000","last_peered":"2024-04-26T21:21:33.158423+0000","last_clean":"2024-04-26T21:21:33.158423+0000","last_became_active":"2024-04-26T21:21:30.850128+0000","last_became_peered":"2024-04-26T21:21:30.850128+0000","last_unstale":"2024-04-26T21:21:33.158423+0000","last_undegraded":"2024-04-26T21:21:33.158423+0000","last_fullsized":"2024-04-26T21:21:33.158423+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:00.913571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158103+0000","last_change":"2024-04-26T21:21:29.848308+0000","last_active":"2024-04-26T21:21:33.158103+0000","last_peered":"2024-04-26T21:21:33.158103+0000","last_clean":"2024-04-26T21:21:33.158103+0000","last_became_active":"2024-04-26T21:21:29.848020+0000","last_became_peered":"2024-04-26T21:21:29.848020+0000","last_unstale":"2024-04-26T21:21:33.158103+0000","last_undegraded":"2024-04-26T21:21:33.158103+0000","last_fullsized":"2024-04-26T21:21:33.158103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:38:57.124715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158140+0000","last_change":"2024-04-26T21:21:30.848649+0000","last_active":"2024-04-26T21:21:33.158140+0000","last_peered":"2024-04-26T21:21:33.158140+0000","last_clean":"2024-04-26T21:21:33.158140+0000","last_became_active":"2024-04-26T21:21:30.846358+0000","last_became_peered":"2024-04-26T21:21:30.846358+0000","last_unstale":"2024-04-26T21:21:33.158140+0000","last_undegraded":"2024-04-26T21:21:33.158140+0000","last_fullsized":"2024-04-26T21:21:33.158140+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:27.201427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852959+0000","last_change":"2024-04-26T21:21:29.854926+0000","last_active":"2024-04-26T21:21:30.852959+0000","last_peered":"2024-04-26T21:21:30.852959+0000","last_clean":"2024-04-26T21:21:30.852959+0000","last_became_active":"2024-04-26T21:21:29.854562+0000","last_became_peered":"2024-04-26T21:21:29.854562+0000","last_unstale":"2024-04-26T21:21:30.852959+0000","last_undegraded":"2024-04-26T21:21:30.852959+0000","last_fullsized":"2024-04-26T21:21:30.852959+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:16.553250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241054+0000","last_change":"2024-04-26T21:21:30.849254+0000","last_active":"2024-04-26T21:21:32.241054+0000","last_peered":"2024-04-26T21:21:32.241054+0000","last_clean":"2024-04-26T21:21:32.241054+0000","last_became_active":"2024-04-26T21:21:30.848998+0000","last_became_peered":"2024-04-26T21:21:30.848998+0000","last_unstale":"2024-04-26T21:21:32.241054+0000","last_undegraded":"2024-04-26T21:21:32.241054+0000","last_fullsized":"2024-04-26T21:21:32.241054+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:04.951704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530071+0000","last_change":"2024-04-26T21:21:29.845962+0000","last_active":"2024-04-26T21:21:32.530071+0000","last_peered":"2024-04-26T21:21:32.530071+0000","last_clean":"2024-04-26T21:21:32.530071+0000","last_became_active":"2024-04-26T21:21:29.845617+0000","last_became_peered":"2024-04-26T21:21:29.845617+0000","last_unstale":"2024-04-26T21:21:32.530071+0000","last_undegraded":"2024-04-26T21:21:32.530071+0000","last_fullsized":"2024-04-26T21:21:32.530071+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:48.064648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852944+0000","last_change":"2024-04-26T21:21:30.850203+0000","last_active":"2024-04-26T21:21:31.852944+0000","last_peered":"2024-04-26T21:21:31.852944+0000","last_clean":"2024-04-26T21:21:31.852944+0000","last_became_active":"2024-04-26T21:21:30.849924+0000","last_became_peered":"2024-04-26T21:21:30.849924+0000","last_unstale":"2024-04-26T21:21:31.852944+0000","last_undegraded":"2024-04-26T21:21:31.852944+0000","last_fullsized":"2024-04-26T21:21:31.852944+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:01.023665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241080+0000","last_change":"2024-04-26T21:21:29.845311+0000","last_active":"2024-04-26T21:21:32.241080+0000","last_peered":"2024-04-26T21:21:32.241080+0000","last_clean":"2024-04-26T21:21:32.241080+0000","last_became_active":"2024-04-26T21:21:29.844638+0000","last_became_peered":"2024-04-26T21:21:29.844638+0000","last_unstale":"2024-04-26T21:21:32.241080+0000","last_undegraded":"2024-04-26T21:21:32.241080+0000","last_fullsized":"2024-04-26T21:21:32.241080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:34.777820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852933+0000","last_change":"2024-04-26T21:21:30.855332+0000","last_active":"2024-04-26T21:21:31.852933+0000","last_peered":"2024-04-26T21:21:31.852933+0000","last_clean":"2024-04-26T21:21:31.852933+0000","last_became_active":"2024-04-26T21:21:30.855051+0000","last_became_peered":"2024-04-26T21:21:30.855051+0000","last_unstale":"2024-04-26T21:21:31.852933+0000","last_undegraded":"2024-04-26T21:21:31.852933+0000","last_fullsized":"2024-04-26T21:21:31.852933+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:23.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241128+0000","last_change":"2024-04-26T21:21:29.841586+0000","last_active":"2024-04-26T21:21:32.241128+0000","last_peered":"2024-04-26T21:21:32.241128+0000","last_clean":"2024-04-26T21:21:32.241128+0000","last_became_active":"2024-04-26T21:21:29.841260+0000","last_became_peered":"2024-04-26T21:21:29.841260+0000","last_unstale":"2024-04-26T21:21:32.241128+0000","last_undegraded":"2024-04-26T21:21:32.241128+0000","last_fullsized":"2024-04-26T21:21:32.241128+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:31.542242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852708+0000","last_change":"2024-04-26T21:21:30.860265+0000","last_active":"2024-04-26T21:21:31.852708+0000","last_peered":"2024-04-26T21:21:31.852708+0000","last_clean":"2024-04-26T21:21:31.852708+0000","last_became_active":"2024-04-26T21:21:30.859431+0000","last_became_peered":"2024-04-26T21:21:30.859431+0000","last_unstale":"2024-04-26T21:21:31.852708+0000","last_undegraded":"2024-04-26T21:21:31.852708+0000","last_fullsized":"2024-04-26T21:21:31.852708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:41.198489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853048+0000","last_change":"2024-04-26T21:21:29.850661+0000","last_active":"2024-04-26T21:21:31.853048+0000","last_peered":"2024-04-26T21:21:31.853048+0000","last_clean":"2024-04-26T21:21:31.853048+0000","last_became_active":"2024-04-26T21:21:29.850358+0000","last_became_peered":"2024-04-26T21:21:29.850358+0000","last_unstale":"2024-04-26T21:21:31.853048+0000","last_undegraded":"2024-04-26T21:21:31.853048+0000","last_fullsized":"2024-04-26T21:21:31.853048+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:25.732739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852571+0000","last_change":"2024-04-26T21:21:30.863866+0000","last_active":"2024-04-26T21:21:31.852571+0000","last_peered":"2024-04-26T21:21:31.852571+0000","last_clean":"2024-04-26T21:21:31.852571+0000","last_became_active":"2024-04-26T21:21:30.863458+0000","last_became_peered":"2024-04-26T21:21:30.863458+0000","last_unstale":"2024-04-26T21:21:31.852571+0000","last_undegraded":"2024-04-26T21:21:31.852571+0000","last_fullsized":"2024-04-26T21:21:31.852571+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:28.992696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158378+0000","last_change":"2024-04-26T21:21:29.850924+0000","last_active":"2024-04-26T21:21:33.158378+0000","last_peered":"2024-04-26T21:21:33.158378+0000","last_clean":"2024-04-26T21:21:33.158378+0000","last_became_active":"2024-04-26T21:21:29.850609+0000","last_became_peered":"2024-04-26T21:21:29.850609+0000","last_unstale":"2024-04-26T21:21:33.158378+0000","last_undegraded":"2024-04-26T21:21:33.158378+0000","last_fullsized":"2024-04-26T21:21:33.158378+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:41.983686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530297+0000","last_change":"2024-04-26T21:21:30.852635+0000","last_active":"2024-04-26T21:21:32.530297+0000","last_peered":"2024-04-26T21:21:32.530297+0000","last_clean":"2024-04-26T21:21:32.530297+0000","last_became_active":"2024-04-26T21:21:30.850418+0000","last_became_peered":"2024-04-26T21:21:30.850418+0000","last_unstale":"2024-04-26T21:21:32.530297+0000","last_undegraded":"2024-04-26T21:21:32.530297+0000","last_fullsized":"2024-04-26T21:21:32.530297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:23.900738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852878+0000","last_change":"2024-04-26T21:21:29.844229+0000","last_active":"2024-04-26T21:21:31.852878+0000","last_peered":"2024-04-26T21:21:31.852878+0000","last_clean":"2024-04-26T21:21:31.852878+0000","last_became_active":"2024-04-26T21:21:29.843919+0000","last_became_peered":"2024-04-26T21:21:29.843919+0000","last_unstale":"2024-04-26T21:21:31.852878+0000","last_undegraded":"2024-04-26T21:21:31.852878+0000","last_fullsized":"2024-04-26T21:21:31.852878+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:25:12.167753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158258+0000","last_change":"2024-04-26T21:21:30.849530+0000","last_active":"2024-04-26T21:21:33.158258+0000","last_peered":"2024-04-26T21:21:33.158258+0000","last_clean":"2024-04-26T21:21:33.158258+0000","last_became_active":"2024-04-26T21:21:30.849259+0000","last_became_peered":"2024-04-26T21:21:30.849259+0000","last_unstale":"2024-04-26T21:21:33.158258+0000","last_undegraded":"2024-04-26T21:21:33.158258+0000","last_fullsized":"2024-04-26T21:21:33.158258+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:38.333875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158150+0000","last_change":"2024-04-26T21:21:29.857888+0000","last_active":"2024-04-26T21:21:33.158150+0000","last_peered":"2024-04-26T21:21:33.158150+0000","last_clean":"2024-04-26T21:21:33.158150+0000","last_became_active":"2024-04-26T21:21:29.857413+0000","last_became_peered":"2024-04-26T21:21:29.857413+0000","last_unstale":"2024-04-26T21:21:33.158150+0000","last_undegraded":"2024-04-26T21:21:33.158150+0000","last_fullsized":"2024-04-26T21:21:33.158150+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:08.376909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853713+0000","last_change":"2024-04-26T21:21:30.854343+0000","last_active":"2024-04-26T21:21:31.853713+0000","last_peered":"2024-04-26T21:21:31.853713+0000","last_clean":"2024-04-26T21:21:31.853713+0000","last_became_active":"2024-04-26T21:21:30.853942+0000","last_became_peered":"2024-04-26T21:21:30.853942+0000","last_unstale":"2024-04-26T21:21:31.853713+0000","last_undegraded":"2024-04-26T21:21:31.853713+0000","last_fullsized":"2024-04-26T21:21:31.853713+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:42.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":22,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852515+0000","last_change":"2024-04-26T21:21:29.847898+0000","last_active":"2024-04-26T21:21:31.852515+0000","last_peered":"2024-04-26T21:21:31.852515+0000","last_clean":"2024-04-26T21:21:31.852515+0000","last_became_active":"2024-04-26T21:21:29.847617+0000","last_became_peered":"2024-04-26T21:21:29.847617+0000","last_unstale":"2024-04-26T21:21:31.852515+0000","last_undegraded":"2024-04-26T21:21:31.852515+0000","last_fullsized":"2024-04-26T21:21:31.852515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:32.942062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158720+0000","last_change":"2024-04-26T21:21:30.853780+0000","last_active":"2024-04-26T21:21:33.158720+0000","last_peered":"2024-04-26T21:21:33.158720+0000","last_clean":"2024-04-26T21:21:33.158720+0000","last_became_active":"2024-04-26T21:21:30.852998+0000","last_became_peered":"2024-04-26T21:21:30.852998+0000","last_unstale":"2024-04-26T21:21:33.158720+0000","last_undegraded":"2024-04-26T21:21:33.158720+0000","last_fullsized":"2024-04-26T21:21:33.158720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:12.283413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854396+0000","last_change":"2024-04-26T21:21:29.857918+0000","last_active":"2024-04-26T21:21:31.854396+0000","last_peered":"2024-04-26T21:21:31.854396+0000","last_clean":"2024-04-26T21:21:31.854396+0000","last_became_active":"2024-04-26T21:21:29.857585+0000","last_became_peered":"2024-04-26T21:21:29.857585+0000","last_unstale":"2024-04-26T21:21:31.854396+0000","last_undegraded":"2024-04-26T21:21:31.854396+0000","last_fullsized":"2024-04-26T21:21:31.854396+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:05.269879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852817+0000","last_change":"2024-04-26T21:21:30.848158+0000","last_active":"2024-04-26T21:21:31.852817+0000","last_peered":"2024-04-26T21:21:31.852817+0000","last_clean":"2024-04-26T21:21:31.852817+0000","last_became_active":"2024-04-26T21:21:30.847868+0000","last_became_peered":"2024-04-26T21:21:30.847868+0000","last_unstale":"2024-04-26T21:21:31.852817+0000","last_undegraded":"2024-04-26T21:21:31.852817+0000","last_fullsized":"2024-04-26T21:21:31.852817+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:34.021913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854355+0000","last_change":"2024-04-26T21:21:29.852123+0000","last_active":"2024-04-26T21:21:31.854355+0000","last_peered":"2024-04-26T21:21:31.854355+0000","last_clean":"2024-04-26T21:21:31.854355+0000","last_became_active":"2024-04-26T21:21:29.851249+0000","last_became_peered":"2024-04-26T21:21:29.851249+0000","last_unstale":"2024-04-26T21:21:31.854355+0000","last_undegraded":"2024-04-26T21:21:31.854355+0000","last_fullsized":"2024-04-26T21:21:31.854355+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:36.462336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158461+0000","last_change":"2024-04-26T21:21:30.859535+0000","last_active":"2024-04-26T21:21:33.158461+0000","last_peered":"2024-04-26T21:21:33.158461+0000","last_clean":"2024-04-26T21:21:33.158461+0000","last_became_active":"2024-04-26T21:21:30.859225+0000","last_became_peered":"2024-04-26T21:21:30.859225+0000","last_unstale":"2024-04-26T21:21:33.158461+0000","last_undegraded":"2024-04-26T21:21:33.158461+0000","last_fullsized":"2024-04-26T21:21:33.158461+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:18.513571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866319+0000","last_change":"2024-04-26T21:21:29.846155+0000","last_active":"2024-04-26T21:21:30.866319+0000","last_peered":"2024-04-26T21:21:30.866319+0000","last_clean":"2024-04-26T21:21:30.866319+0000","last_became_active":"2024-04-26T21:21:29.845829+0000","last_became_peered":"2024-04-26T21:21:29.845829+0000","last_unstale":"2024-04-26T21:21:30.866319+0000","last_undegraded":"2024-04-26T21:21:30.866319+0000","last_fullsized":"2024-04-26T21:21:30.866319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:05.058957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530470+0000","last_change":"2024-04-26T21:21:30.851131+0000","last_active":"2024-04-26T21:21:32.530470+0000","last_peered":"2024-04-26T21:21:32.530470+0000","last_clean":"2024-04-26T21:21:32.530470+0000","last_became_active":"2024-04-26T21:21:30.850850+0000","last_became_peered":"2024-04-26T21:21:30.850850+0000","last_unstale":"2024-04-26T21:21:32.530470+0000","last_undegraded":"2024-04-26T21:21:32.530470+0000","last_fullsized":"2024-04-26T21:21:32.530470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:53.964354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842631+0000","last_change":"2024-04-26T21:21:29.851145+0000","last_active":"2024-04-26T21:21:30.842631+0000","last_peered":"2024-04-26T21:21:30.842631+0000","last_clean":"2024-04-26T21:21:30.842631+0000","last_became_active":"2024-04-26T21:21:29.850835+0000","last_became_peered":"2024-04-26T21:21:29.850835+0000","last_unstale":"2024-04-26T21:21:30.842631+0000","last_undegraded":"2024-04-26T21:21:30.842631+0000","last_fullsized":"2024-04-26T21:21:30.842631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:10.881969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853772+0000","last_change":"2024-04-26T21:21:30.846083+0000","last_active":"2024-04-26T21:21:31.853772+0000","last_peered":"2024-04-26T21:21:31.853772+0000","last_clean":"2024-04-26T21:21:31.853772+0000","last_became_active":"2024-04-26T21:21:30.845721+0000","last_became_peered":"2024-04-26T21:21:30.845721+0000","last_unstale":"2024-04-26T21:21:31.853772+0000","last_undegraded":"2024-04-26T21:21:31.853772+0000","last_fullsized":"2024-04-26T21:21:31.853772+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:39.101882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158491+0000","last_change":"2024-04-26T21:21:29.855288+0000","last_active":"2024-04-26T21:21:33.158491+0000","last_peered":"2024-04-26T21:21:33.158491+0000","last_clean":"2024-04-26T21:21:33.158491+0000","last_became_active":"2024-04-26T21:21:29.854909+0000","last_became_peered":"2024-04-26T21:21:29.854909+0000","last_unstale":"2024-04-26T21:21:33.158491+0000","last_undegraded":"2024-04-26T21:21:33.158491+0000","last_fullsized":"2024-04-26T21:21:33.158491+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:18.175199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530354+0000","last_change":"2024-04-26T21:21:30.852301+0000","last_active":"2024-04-26T21:21:32.530354+0000","last_peered":"2024-04-26T21:21:32.530354+0000","last_clean":"2024-04-26T21:21:32.530354+0000","last_became_active":"2024-04-26T21:21:30.850064+0000","last_became_peered":"2024-04-26T21:21:30.850064+0000","last_unstale":"2024-04-26T21:21:32.530354+0000","last_undegraded":"2024-04-26T21:21:32.530354+0000","last_fullsized":"2024-04-26T21:21:32.530354+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:27.634083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.840422+0000","last_change":"2024-04-26T21:21:29.846529+0000","last_active":"2024-04-26T21:21:30.840422+0000","last_peered":"2024-04-26T21:21:30.840422+0000","last_clean":"2024-04-26T21:21:30.840422+0000","last_became_active":"2024-04-26T21:21:29.844512+0000","last_became_peered":"2024-04-26T21:21:29.844512+0000","last_unstale":"2024-04-26T21:21:30.840422+0000","last_undegraded":"2024-04-26T21:21:30.840422+0000","last_fullsized":"2024-04-26T21:21:30.840422+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:23.867196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.866162+0000","last_change":"2024-04-26T21:21:30.861743+0000","last_active":"2024-04-26T21:21:30.866162+0000","last_peered":"2024-04-26T21:21:30.866162+0000","last_clean":"2024-04-26T21:21:30.866162+0000","last_became_active":"2024-04-26T21:21:30.860060+0000","last_became_peered":"2024-04-26T21:21:30.860060+0000","last_unstale":"2024-04-26T21:21:30.866162+0000","last_undegraded":"2024-04-26T21:21:30.866162+0000","last_fullsized":"2024-04-26T21:21:30.866162+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158162+0000","last_change":"2024-04-26T21:21:29.849421+0000","last_active":"2024-04-26T21:21:33.158162+0000","last_peered":"2024-04-26T21:21:33.158162+0000","last_clean":"2024-04-26T21:21:33.158162+0000","last_became_active":"2024-04-26T21:21:29.849137+0000","last_became_peered":"2024-04-26T21:21:29.849137+0000","last_unstale":"2024-04-26T21:21:33.158162+0000","last_undegraded":"2024-04-26T21:21:33.158162+0000","last_fullsized":"2024-04-26T21:21:33.158162+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:25.251390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241127+0000","last_change":"2024-04-26T21:21:30.860369+0000","last_active":"2024-04-26T21:21:32.241127+0000","last_peered":"2024-04-26T21:21:32.241127+0000","last_clean":"2024-04-26T21:21:32.241127+0000","last_became_active":"2024-04-26T21:21:30.860044+0000","last_became_peered":"2024-04-26T21:21:30.860044+0000","last_unstale":"2024-04-26T21:21:32.241127+0000","last_undegraded":"2024-04-26T21:21:32.241127+0000","last_fullsized":"2024-04-26T21:21:32.241127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:50.382317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842467+0000","last_change":"2024-04-26T21:21:29.847624+0000","last_active":"2024-04-26T21:21:30.842467+0000","last_peered":"2024-04-26T21:21:30.842467+0000","last_clean":"2024-04-26T21:21:30.842467+0000","last_became_active":"2024-04-26T21:21:29.847237+0000","last_became_peered":"2024-04-26T21:21:29.847237+0000","last_unstale":"2024-04-26T21:21:30.842467+0000","last_undegraded":"2024-04-26T21:21:30.842467+0000","last_fullsized":"2024-04-26T21:21:30.842467+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:27.988774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852896+0000","last_change":"2024-04-26T21:21:30.850778+0000","last_active":"2024-04-26T21:21:31.852896+0000","last_peered":"2024-04-26T21:21:31.852896+0000","last_clean":"2024-04-26T21:21:31.852896+0000","last_became_active":"2024-04-26T21:21:30.849972+0000","last_became_peered":"2024-04-26T21:21:30.849972+0000","last_unstale":"2024-04-26T21:21:31.852896+0000","last_undegraded":"2024-04-26T21:21:31.852896+0000","last_fullsized":"2024-04-26T21:21:31.852896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:53.964354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852940+0000","last_change":"2024-04-26T21:21:29.858034+0000","last_active":"2024-04-26T21:21:31.852940+0000","last_peered":"2024-04-26T21:21:31.852940+0000","last_clean":"2024-04-26T21:21:31.852940+0000","last_became_active":"2024-04-26T21:21:29.857698+0000","last_became_peered":"2024-04-26T21:21:29.857698+0000","last_unstale":"2024-04-26T21:21:31.852940+0000","last_undegraded":"2024-04-26T21:21:31.852940+0000","last_fullsized":"2024-04-26T21:21:31.852940+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:18.666803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530396+0000","last_change":"2024-04-26T21:21:30.862583+0000","last_active":"2024-04-26T21:21:32.530396+0000","last_peered":"2024-04-26T21:21:32.530396+0000","last_clean":"2024-04-26T21:21:32.530396+0000","last_became_active":"2024-04-26T21:21:30.862288+0000","last_became_peered":"2024-04-26T21:21:30.862288+0000","last_unstale":"2024-04-26T21:21:32.530396+0000","last_undegraded":"2024-04-26T21:21:32.530396+0000","last_fullsized":"2024-04-26T21:21:32.530396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:43.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.854373+0000","last_change":"2024-04-26T21:21:29.847434+0000","last_active":"2024-04-26T21:21:31.854373+0000","last_peered":"2024-04-26T21:21:31.854373+0000","last_clean":"2024-04-26T21:21:31.854373+0000","last_became_active":"2024-04-26T21:21:29.847060+0000","last_became_peered":"2024-04-26T21:21:29.847060+0000","last_unstale":"2024-04-26T21:21:31.854373+0000","last_undegraded":"2024-04-26T21:21:31.854373+0000","last_fullsized":"2024-04-26T21:21:31.854373+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:23.782700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.241047+0000","last_change":"2024-04-26T21:21:30.856877+0000","last_active":"2024-04-26T21:21:32.241047+0000","last_peered":"2024-04-26T21:21:32.241047+0000","last_clean":"2024-04-26T21:21:32.241047+0000","last_became_active":"2024-04-26T21:21:30.856490+0000","last_became_peered":"2024-04-26T21:21:30.856490+0000","last_unstale":"2024-04-26T21:21:32.241047+0000","last_undegraded":"2024-04-26T21:21:32.241047+0000","last_fullsized":"2024-04-26T21:21:32.241047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:14.909542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.852406+0000","last_change":"2024-04-26T21:21:29.857449+0000","last_active":"2024-04-26T21:21:30.852406+0000","last_peered":"2024-04-26T21:21:30.852406+0000","last_clean":"2024-04-26T21:21:30.852406+0000","last_became_active":"2024-04-26T21:21:29.857121+0000","last_became_peered":"2024-04-26T21:21:29.857121+0000","last_unstale":"2024-04-26T21:21:30.852406+0000","last_undegraded":"2024-04-26T21:21:30.852406+0000","last_fullsized":"2024-04-26T21:21:30.852406+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:07.115469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852696+0000","last_change":"2024-04-26T21:21:30.848153+0000","last_active":"2024-04-26T21:21:31.852696+0000","last_peered":"2024-04-26T21:21:31.852696+0000","last_clean":"2024-04-26T21:21:31.852696+0000","last_became_active":"2024-04-26T21:21:30.847848+0000","last_became_peered":"2024-04-26T21:21:30.847848+0000","last_unstale":"2024-04-26T21:21:31.852696+0000","last_undegraded":"2024-04-26T21:21:31.852696+0000","last_fullsized":"2024-04-26T21:21:31.852696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:11.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852974+0000","last_change":"2024-04-26T21:21:29.845722+0000","last_active":"2024-04-26T21:21:31.852974+0000","last_peered":"2024-04-26T21:21:31.852974+0000","last_clean":"2024-04-26T21:21:31.852974+0000","last_became_active":"2024-04-26T21:21:29.845257+0000","last_became_peered":"2024-04-26T21:21:29.845257+0000","last_unstale":"2024-04-26T21:21:31.852974+0000","last_undegraded":"2024-04-26T21:21:31.852974+0000","last_fullsized":"2024-04-26T21:21:31.852974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:34.598815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852759+0000","last_change":"2024-04-26T21:21:30.850942+0000","last_active":"2024-04-26T21:21:31.852759+0000","last_peered":"2024-04-26T21:21:31.852759+0000","last_clean":"2024-04-26T21:21:31.852759+0000","last_became_active":"2024-04-26T21:21:30.850664+0000","last_became_peered":"2024-04-26T21:21:30.850664+0000","last_unstale":"2024-04-26T21:21:31.852759+0000","last_undegraded":"2024-04-26T21:21:31.852759+0000","last_fullsized":"2024-04-26T21:21:31.852759+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852133+0000","last_change":"2024-04-26T21:21:29.851281+0000","last_active":"2024-04-26T21:21:31.852133+0000","last_peered":"2024-04-26T21:21:31.852133+0000","last_clean":"2024-04-26T21:21:31.852133+0000","last_became_active":"2024-04-26T21:21:29.851015+0000","last_became_peered":"2024-04-26T21:21:29.851015+0000","last_unstale":"2024-04-26T21:21:31.852133+0000","last_undegraded":"2024-04-26T21:21:31.852133+0000","last_fullsized":"2024-04-26T21:21:31.852133+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:09.391478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852309+0000","last_change":"2024-04-26T21:21:30.856847+0000","last_active":"2024-04-26T21:21:31.852309+0000","last_peered":"2024-04-26T21:21:31.852309+0000","last_clean":"2024-04-26T21:21:31.852309+0000","last_became_active":"2024-04-26T21:21:30.856144+0000","last_became_peered":"2024-04-26T21:21:30.856144+0000","last_unstale":"2024-04-26T21:21:31.852309+0000","last_undegraded":"2024-04-26T21:21:31.852309+0000","last_fullsized":"2024-04-26T21:21:31.852309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:48.213837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:33.158229+0000","last_change":"2024-04-26T21:21:29.845176+0000","last_active":"2024-04-26T21:21:33.158229+0000","last_peered":"2024-04-26T21:21:33.158229+0000","last_clean":"2024-04-26T21:21:33.158229+0000","last_became_active":"2024-04-26T21:21:29.844897+0000","last_became_peered":"2024-04-26T21:21:29.844897+0000","last_unstale":"2024-04-26T21:21:33.158229+0000","last_undegraded":"2024-04-26T21:21:33.158229+0000","last_fullsized":"2024-04-26T21:21:33.158229+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:26.127392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:32.530030+0000","last_change":"2024-04-26T21:21:30.851341+0000","last_active":"2024-04-26T21:21:32.530030+0000","last_peered":"2024-04-26T21:21:32.530030+0000","last_clean":"2024-04-26T21:21:32.530030+0000","last_became_active":"2024-04-26T21:21:30.851041+0000","last_became_peered":"2024-04-26T21:21:30.851041+0000","last_unstale":"2024-04-26T21:21:32.530030+0000","last_undegraded":"2024-04-26T21:21:32.530030+0000","last_fullsized":"2024-04-26T21:21:32.530030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:25.350756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.853068+0000","last_change":"2024-04-26T21:21:29.854796+0000","last_active":"2024-04-26T21:21:30.853068+0000","last_peered":"2024-04-26T21:21:30.853068+0000","last_clean":"2024-04-26T21:21:30.853068+0000","last_became_active":"2024-04-26T21:21:29.854515+0000","last_became_peered":"2024-04-26T21:21:29.854515+0000","last_unstale":"2024-04-26T21:21:30.853068+0000","last_undegraded":"2024-04-26T21:21:30.853068+0000","last_fullsized":"2024-04-26T21:21:30.853068+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:46.457627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.853924+0000","last_change":"2024-04-26T21:21:30.848268+0000","last_active":"2024-04-26T21:21:31.853924+0000","last_peered":"2024-04-26T21:21:31.853924+0000","last_clean":"2024-04-26T21:21:31.853924+0000","last_became_active":"2024-04-26T21:21:30.847987+0000","last_became_peered":"2024-04-26T21:21:30.847987+0000","last_unstale":"2024-04-26T21:21:31.853924+0000","last_undegraded":"2024-04-26T21:21:31.853924+0000","last_fullsized":"2024-04-26T21:21:31.853924+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:05.721301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852005+0000","last_change":"2024-04-26T21:21:29.852143+0000","last_active":"2024-04-26T21:21:31.852005+0000","last_peered":"2024-04-26T21:21:31.852005+0000","last_clean":"2024-04-26T21:21:31.852005+0000","last_became_active":"2024-04-26T21:21:29.851645+0000","last_became_peered":"2024-04-26T21:21:29.851645+0000","last_unstale":"2024-04-26T21:21:31.852005+0000","last_undegraded":"2024-04-26T21:21:31.852005+0000","last_fullsized":"2024-04-26T21:21:31.852005+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:34.643475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852743+0000","last_change":"2024-04-26T21:21:30.855681+0000","last_active":"2024-04-26T21:21:31.852743+0000","last_peered":"2024-04-26T21:21:31.852743+0000","last_clean":"2024-04-26T21:21:31.852743+0000","last_became_active":"2024-04-26T21:21:30.855301+0000","last_became_peered":"2024-04-26T21:21:30.855301+0000","last_unstale":"2024-04-26T21:21:31.852743+0000","last_undegraded":"2024-04-26T21:21:31.852743+0000","last_fullsized":"2024-04-26T21:21:31.852743+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:29.824405+0000","last_clean_scrub_stamp":"2024-04-26T21:21:29.824405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:09.563691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-26T21:21:31.852803+0000","last_change":"2024-04-26T21:21:29.850531+0000","last_active":"2024-04-26T21:21:31.852803+0000","last_peered":"2024-04-26T21:21:31.852803+0000","last_clean":"2024-04-26T21:21:31.852803+0000","last_became_active":"2024-04-26T21:21:29.850243+0000","last_became_peered":"2024-04-26T21:21:29.850243+0000","last_unstale":"2024-04-26T21:21:31.852803+0000","last_undegraded":"2024-04-26T21:21:31.852803+0000","last_fullsized":"2024-04-26T21:21:31.852803+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:21:28.820483+0000","last_clean_scrub_stamp":"2024-04-26T21:21:28.820483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:33:20.694957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":389,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-04-26T21:21:30.842530+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:30.842530+0000","last_peered":"2024-04-26T21:21:30.842530+0000","last_clean":"2024-04-26T21:21:30.842530+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:30.842530+0000","last_undegraded":"2024-04-26T21:21:30.842530+0000","last_fullsized":"2024-04-26T21:21:30.842530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33548,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338292,"statfs":{"total":96636764160,"available":96602411008,"internally_reserved":0,"allocated":2420736,"data_stored":1369849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1806336,"data_stored":777149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1822720,"data_stored":784031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1830912,"data_stored":785109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":783461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37124,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334716,"statfs":{"total":96636764160,"available":96598749184,"internally_reserved":0,"allocated":1818624,"data_stored":783517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":2408448,"data_stored":1373795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37116,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334724,"statfs":{"total":96636764160,"available":96598757376,"internally_reserved":0,"allocated":1818624,"data_stored":783461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:35.217 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:21:35.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:21:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.359+0000 7f0f07e99640 1 Processor -- start 2024-04-26T21:21:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.359+0000 7f0f07e99640 1 -- start start 2024-04-26T21:21:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.360+0000 7f0f07e99640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f0015f6e0 0x7f0f000b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.360+0000 7f0f07e99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.360+0000 7f0f07e99640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f000b2580 con 0x7f0f000d56a0 2024-04-26T21:21:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.360+0000 7f0f07e99640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f0f000d5b10 con 0x7f0f0015f6e0 2024-04-26T21:21:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.360+0000 7f0f07e99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f000b2860 con 0x7f0f000d3d10 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05434640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f0015f6e0 0x7f0f000b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f0015f6e0 0x7f0f000b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:58486/0 (socket says 172.21.15.40:58486) 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f06436640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f0f000d56a0 0x7f0f000b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:50754/0 (socket says 172.21.15.40:50754) 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05434640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.40:39802/0 (socket says 172.21.15.40:39802) 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 -- 172.21.15.40:0/4185187774 learned_addr learned my addr 172.21.15.40:0/4185187774 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 -- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 msgr2=0x7f0f000d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 -- 172.21.15.40:0/4185187774 >> v1:172.21.15.184:6789/0 conn(0x7f0f000d56a0 legacy=0x7f0f000b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05c35640 1 -- 172.21.15.40:0/4185187774 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f000b2b40 con 0x7f0f0015f6e0 2024-04-26T21:21:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f04c33640 1 -- 172.21.15.40:0/4185187774 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497381912 0 0) 0x7f0f000b2580 con 0x7f0f000d56a0 2024-04-26T21:21:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.361+0000 7f0f05434640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f05c35640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0015f6e0 0x7f0f000b1ea0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009870 tx=0x7f0ef802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90bb0d298dd06a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f04c33640 1 -- 172.21.15.40:0/4185187774 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8037070 con 0x7f0f0015f6e0 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f04c33640 1 -- 172.21.15.40:0/4185187774 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef8032440 con 0x7f0f0015f6e0 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f04c33640 1 -- 172.21.15.40:0/4185187774 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8032810 con 0x7f0f0015f6e0 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f07e99640 1 -- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0015f6e0 msgr2=0x7f0f000b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.362+0000 7f0f07e99640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0015f6e0 0x7f0f000b1ea0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009870 tx=0x7f0ef802a270 comp rx=0 tx=0).stop 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 -- 172.21.15.40:0/4185187774 shutdown_connections 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d3d10 0x7f0f000d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 --2- 172.21.15.40:0/4185187774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0015f6e0 0x7f0f000b1ea0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 -- 172.21.15.40:0/4185187774 >> 172.21.15.40:0/4185187774 conn(0x7f0f0008ec20 msgr2=0x7f0f0009e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 -- 172.21.15.40:0/4185187774 shutdown_connections 2024-04-26T21:21:35.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.363+0000 7f0f07e99640 1 -- 172.21.15.40:0/4185187774 wait complete. 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.364+0000 7f0f07e99640 1 Processor -- start 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.364+0000 7f0f07e99640 1 -- start start 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.364+0000 7f0f07e99640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f000d3d10 0x7f0f00154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.364+0000 7f0f07e99640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d56a0 0x7f0f00162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.364+0000 7f0f07e99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f07e99640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f000d4170 con 0x7f0f0015f6e0 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f07e99640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f000b4630 con 0x7f0f000d3d10 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f07e99640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0015fd90 con 0x7f0f000d56a0 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f05c35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f000d3d10 0x7f0f00154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58444/0 (socket says 172.21.15.40:58444) 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 -- 172.21.15.40:0/805609881 learned_addr learned my addr 172.21.15.40:0/805609881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f05434640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d56a0 0x7f0f00162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 -- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d56a0 msgr2=0x7f0f00162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d56a0 0x7f0f00162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 -- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f000d3d10 msgr2=0x7f0f00154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f000d3d10 0x7f0f00154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 -- 172.21.15.40:0/805609881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f000cacb0 con 0x7f0f0015f6e0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f06436640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c7e0 tx=0x7f0ef000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0018070 con 0x7f0f0015f6e0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef000a640 con 0x7f0f0015f6e0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f000caf90 con 0x7f0f0015f6e0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000aa30 con 0x7f0f0015f6e0 2024-04-26T21:21:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.365+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f000cb550 con 0x7f0f0015f6e0 2024-04-26T21:21:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.366+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0006a290 con 0x7f0f0015f6e0 2024-04-26T21:21:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.367+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef000abd0 con 0x7f0f0015f6e0 2024-04-26T21:21:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.367+0000 7f0efe7fc640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 0x7f0eec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.368+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f0ef008b140 con 0x7f0f0015f6e0 2024-04-26T21:21:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.368+0000 7f0f05c35640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 0x7f0eec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.368+0000 7f0f05c35640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 0x7f0eec06ede0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8034000 tx=0x7f0ef802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.373+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ef0058e10 con 0x7f0f0015f6e0 2024-04-26T21:21:35.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.631+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0f000d5480 con 0x7f0f0015f6e0 2024-04-26T21:21:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.633+0000 7f0efe7fc640 1 -- 172.21.15.40:0/805609881 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7f0ef005d110 con 0x7f0f0015f6e0 2024-04-26T21:21:35.634 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-26T21:21:35.635 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnGCxmcOHBJxAArg6DIMCIyaLpqVR7rAy/Zg== 2024-04-26T21:21:35.635 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-26T21:21:35.635 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-26T21:21:35.635 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:21:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 msgr2=0x7f0eec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 0x7f0eec06ede0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8034000 tx=0x7f0ef802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 msgr2=0x7f0f001625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c7e0 tx=0x7f0ef000cca0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 shutdown_connections 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0eec06c920 0x7f0eec06ede0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0015f6e0 0x7f0f001625a0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f000d56a0 0x7f0f00162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 --2- 172.21.15.40:0/805609881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f000d3d10 0x7f0f00154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 >> 172.21.15.40:0/805609881 conn(0x7f0f0008ec20 msgr2=0x7f0f000d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 shutdown_connections 2024-04-26T21:21:35.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.636+0000 7f0f07e99640 1 -- 172.21.15.40:0/805609881 wait complete. 2024-04-26T21:21:35.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.747+0000 7f38c3672640 1 Processor -- start 2024-04-26T21:21:35.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 -- start start 2024-04-26T21:21:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38bc09c400 0x7f38bc0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f38bc070520 0x7f38bc0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc0d14c0 con 0x7f38bc070d20 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0d1290 con 0x7f38bc09c400 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.748+0000 7f38c3672640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0d1760 con 0x7f38bc070520 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38bc09c400 0x7f38bc0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38bc09c400 0x7f38bc0cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58460/0 (socket says 172.21.15.40:58460) 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 -- 172.21.15.40:0/2740007827 learned_addr learned my addr 172.21.15.40:0/2740007827 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c1c0f640 1 --2- 172.21.15.40:0/2740007827 >> v2:172.21.15.184:3300/0 conn(0x7f38bc070520 0x7f38bc0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 -- 172.21.15.40:0/2740007827 >> v2:172.21.15.184:3300/0 conn(0x7f38bc070520 msgr2=0x7f38bc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 --2- 172.21.15.40:0/2740007827 >> v2:172.21.15.184:3300/0 conn(0x7f38bc070520 0x7f38bc0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 -- 172.21.15.40:0/2740007827 >> v1:172.21.15.184:6789/0 conn(0x7f38bc070d20 legacy=0x7f38bc071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.749+0000 7f38c0c0d640 1 -- 172.21.15.40:0/2740007827 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0d1a40 con 0x7f38bc09c400 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c0c0d640 1 --2- 172.21.15.40:0/2740007827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc09c400 0x7f38bc0cd5a0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0cf980 tx=0x7f38ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5ebac8b8f95367 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38b3fff640 1 -- 172.21.15.40:0/2740007827 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037070 con 0x7f38bc09c400 2024-04-26T21:21:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38b3fff640 1 -- 172.21.15.40:0/2740007827 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac032440 con 0x7f38bc09c400 2024-04-26T21:21:35.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38b3fff640 1 -- 172.21.15.40:0/2740007827 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac032810 con 0x7f38bc09c400 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 -- 172.21.15.40:0/2740007827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc09c400 msgr2=0x7f38bc0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 --2- 172.21.15.40:0/2740007827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc09c400 0x7f38bc0cd5a0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0cf980 tx=0x7f38ac02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 -- 172.21.15.40:0/2740007827 shutdown_connections 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 --2- 172.21.15.40:0/2740007827 >> v2:172.21.15.184:3300/0 conn(0x7f38bc070520 0x7f38bc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 --2- 172.21.15.40:0/2740007827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc09c400 0x7f38bc0cd5a0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 -- 172.21.15.40:0/2740007827 >> 172.21.15.40:0/2740007827 conn(0x7f38bc0b4a40 msgr2=0x7f38bc0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 -- 172.21.15.40:0/2740007827 shutdown_connections 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.750+0000 7f38c3672640 1 -- 172.21.15.40:0/2740007827 wait complete. 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 Processor -- start 2024-04-26T21:21:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 -- start start 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38bc070520 0x7f38bc0b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38bc09c400 0x7f38bc09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0cf350 con 0x7f38bc070d20 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0d2000 con 0x7f38bc09c400 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c3672640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc0d2520 con 0x7f38bc070520 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c0c0d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c0c0d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58472/0 (socket says 172.21.15.40:58472) 2024-04-26T21:21:35.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.752+0000 7f38c0c0d640 1 -- 172.21.15.40:0/1391592198 learned_addr learned my addr 172.21.15.40:0/1391592198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c1c0f640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38bc09c400 0x7f38bc09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 -- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38bc070520 msgr2=0x7f38bc0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38bc070520 0x7f38bc0b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 -- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38bc09c400 msgr2=0x7f38bc09e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38bc09c400 0x7f38bc09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 -- 172.21.15.40:0/1391592198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0b7e70 con 0x7f38bc070d20 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c0c0d640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f38ac02a2b0 tx=0x7f38ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac041070 con 0x7f38bc070d20 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac03c890 con 0x7f38bc070d20 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037040 con 0x7f38bc070d20 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc0a0870 con 0x7f38bc070d20 2024-04-26T21:21:35.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.753+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc09cf40 con 0x7f38bc070d20 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.754+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc0a0d90 con 0x7f38bc070d20 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.755+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38ac044070 con 0x7f38bc070d20 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.755+0000 7f38b1ffb640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.755+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f38ac0ac820 con 0x7f38bc070d20 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.755+0000 7f38c140e640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.756+0000 7f38c140e640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f38b8000f60 tx=0x7f38b80079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.759+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38bc0a0d90 con 0x7f38bc070d20 2024-04-26T21:21:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.985+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38bc0693d0 con 0x7f38bc070d20 2024-04-26T21:21:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.986+0000 7f38b1ffb640 1 -- 172.21.15.40:0/1391592198 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7f38ac07ad70 con 0x7f38bc070d20 2024-04-26T21:21:35.987 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 msgr2=0x7f38a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f38b8000f60 tx=0x7f38b80079d0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 msgr2=0x7f38bc1624e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f38ac02a2b0 tx=0x7f38ac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 shutdown_connections 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38bc09c400 0x7f38bc09e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc070d20 0x7f38bc1624e0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 --2- 172.21.15.40:0/1391592198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38bc070520 0x7f38bc0b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 >> 172.21.15.40:0/1391592198 conn(0x7f38bc0b4a40 msgr2=0x7f38bc06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 shutdown_connections 2024-04-26T21:21:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:35.988+0000 7f38c3672640 1 -- 172.21.15.40:0/1391592198 wait complete. 2024-04-26T21:21:36.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:21:30.208910+0000', 'modified': '2024-04-26T21:21:33.291754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9501}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9501': {'gid': 9501, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3970947174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 3970947174}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3970947174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9501, 'qdb_cluster': [9501]} 2024-04-26T21:21:36.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:21:36.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:21:36.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.507+0000 7f14b6093640 1 Processor -- start 2024-04-26T21:21:36.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.507+0000 7f14b6093640 1 -- start start 2024-04-26T21:21:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14b015d380 0x7f14b00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b56d0 0x7f14b00d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b0160d50 0x7f14b01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f14b00d0be0 con 0x7f14b0160d50 2024-04-26T21:21:36.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f14b00d0ec0 con 0x7f14b015d380 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14b6093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14b00d11a0 con 0x7f14b00b56d0 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b0160d50 0x7f14b01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b0160d50 0x7f14b01611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58478/0 (socket says 172.21.15.40:58478) 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 -- 172.21.15.40:0/742323912 learned_addr learned my addr 172.21.15.40:0/742323912 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14aeffd640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b56d0 0x7f14b00d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14af7fe640 1 --2- 172.21.15.40:0/742323912 >> v2:172.21.15.184:3300/0 conn(0x7f14b015d380 0x7f14b00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 -- 172.21.15.40:0/742323912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b56d0 msgr2=0x7f14b00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b56d0 0x7f14b00d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 -- 172.21.15.40:0/742323912 >> v2:172.21.15.184:3300/0 conn(0x7f14b015d380 msgr2=0x7f14b00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 --2- 172.21.15.40:0/742323912 >> v2:172.21.15.184:3300/0 conn(0x7f14b015d380 0x7f14b00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.508+0000 7f14affff640 1 -- 172.21.15.40:0/742323912 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b00d1480 con 0x7f14b0160d50 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.509+0000 7f14affff640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0160d50 0x7f14b01611e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f14b0073bf0 tx=0x7f14a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaee0534b57b21ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.509+0000 7f14ae7fc640 1 -- 172.21.15.40:0/742323912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8036070 con 0x7f14b0160d50 2024-04-26T21:21:36.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.509+0000 7f14ae7fc640 1 -- 172.21.15.40:0/742323912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8031600 con 0x7f14b0160d50 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.509+0000 7f14ae7fc640 1 -- 172.21.15.40:0/742323912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a80319d0 con 0x7f14b0160d50 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 -- 172.21.15.40:0/742323912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0160d50 msgr2=0x7f14b01611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0160d50 0x7f14b01611e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f14b0073bf0 tx=0x7f14a802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 -- 172.21.15.40:0/742323912 shutdown_connections 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0160d50 0x7f14b01611e0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 --2- 172.21.15.40:0/742323912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b56d0 0x7f14b00d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 --2- 172.21.15.40:0/742323912 >> v2:172.21.15.184:3300/0 conn(0x7f14b015d380 0x7f14b00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 -- 172.21.15.40:0/742323912 >> 172.21.15.40:0/742323912 conn(0x7f14b00b4a60 msgr2=0x7f14b00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 -- 172.21.15.40:0/742323912 shutdown_connections 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.510+0000 7f14b6093640 1 -- 172.21.15.40:0/742323912 wait complete. 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.511+0000 7f14b6093640 1 Processor -- start 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.511+0000 7f14b6093640 1 -- start start 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 0x7f14b0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b0160d50 0x7f14b006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14b00cedc0 con 0x7f14b00b56d0 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14b00d1ee0 con 0x7f14b015d380 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14b6093640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14b00b5dc0 con 0x7f14b0160d50 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58486/0 (socket says 172.21.15.40:58486) 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 -- 172.21.15.40:0/833607944 learned_addr learned my addr 172.21.15.40:0/833607944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14affff640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b0160d50 0x7f14b006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14aeffd640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 0x7f14b0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 -- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b0160d50 msgr2=0x7f14b006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b0160d50 0x7f14b006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 -- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 msgr2=0x7f14b0156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.512+0000 7f14af7fe640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 0x7f14b0156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f14af7fe640 1 -- 172.21.15.40:0/833607944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b006de50 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f14aeffd640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 0x7f14b0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f14af7fe640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9b0 tx=0x7f14a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f1497fff640 1 -- 172.21.15.40:0/833607944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0017070 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f1497fff640 1 -- 172.21.15.40:0/833607944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a000c9a0 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b006e130 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f1497fff640 1 -- 172.21.15.40:0/833607944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a000cd90 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.513+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b006e6f0 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.514+0000 7f1497fff640 1 -- 172.21.15.40:0/833607944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14a001a070 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.515+0000 7f1497fff640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 0x7f149c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.515+0000 7f1497fff640 1 -- 172.21.15.40:0/833607944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f14a008be00 con 0x7f14b00b56d0 2024-04-26T21:21:36.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.515+0000 7f14aeffd640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 0x7f149c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.516+0000 7f14aeffd640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 0x7f149c06edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f14b01f8ab0 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.516+0000 7f14b8db3740 1 Processor -- start 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.516+0000 7f14b8db3740 1 -- start start 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.516+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 0x56541ced0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541cea6120 0x56541ceec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14b8db3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56541ce0a740 con 0x56541ceedce0 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56541cda63a0 con 0x56541cea6120 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56541ce06700 con 0x56541ced0440 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58490/0 (socket says 172.21.15.40:58490) 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 -- 172.21.15.40:0/675578731 learned_addr learned my addr 172.21.15.40:0/675578731 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14af7fe640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 0x56541ced0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14affff640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541cea6120 0x56541ceec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 -- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 msgr2=0x56541ced0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 0x56541ced0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 -- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541cea6120 msgr2=0x56541ceec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541cea6120 0x56541ceec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 -- 172.21.15.40:0/675578731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541cdc2750 con 0x56541ceedce0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14af7fe640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 0x56541ced0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14aeffd640 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f14a4011d40 tx=0x7f14a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6121ccd3df971b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14957fa640 1 -- 172.21.15.40:0/675578731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4011d70 con 0x56541ceedce0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14957fa640 1 -- 172.21.15.40:0/675578731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4026070 con 0x56541ceedce0 2024-04-26T21:21:36.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.517+0000 7f14957fa640 1 -- 172.21.15.40:0/675578731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4021640 con 0x56541ceedce0 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 -- 172.21.15.40:0/675578731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 msgr2=0x56541ceee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f14a4011d40 tx=0x7f14a4014000 comp rx=0 tx=0).stop 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 -- 172.21.15.40:0/675578731 shutdown_connections 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541cea6120 0x56541ceec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541ceee230 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 --2- 172.21.15.40:0/675578731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ced0440 0x56541ced0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 -- 172.21.15.40:0/675578731 >> 172.21.15.40:0/675578731 conn(0x56541cea2a50 msgr2=0x56541ceeb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 -- 172.21.15.40:0/675578731 shutdown_connections 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.518+0000 7f14b8db3740 1 -- 172.21.15.40:0/675578731 wait complete. 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.519+0000 7f14b8db3740 1 -- start start 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.519+0000 7f14b8db3740 1 Processor -- start 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.519+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541ced4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.519+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ceedce0 0x56541ce9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.519+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14b8db3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56541cea6ec0 con 0x56541cecc9b0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56541cd91fe0 con 0x56541ced0440 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56541cea0980 con 0x56541ceedce0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14b8db3740 10 client.0 _ll_register_callbacks cb 0x56541cda8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541ced4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58504/0 (socket says 172.21.15.40:58504) 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14aeffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ceedce0 0x56541ce9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 -- 172.21.15.40:0/4042632594 learned_addr learned my addr 172.21.15.40:0/4042632594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 -- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ceedce0 msgr2=0x56541ce9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ceedce0 0x56541ce9f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 -- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 msgr2=0x56541ced4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541ced4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 -- 172.21.15.40:0/4042632594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541cb1c2d0 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14affff640 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f14a802ac90 tx=0x7f14a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d2df06c315736c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f148e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8036040 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f14a803d430 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.520+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a4026070 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f14a804f050 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 10 client.9636 fetch_fsmap learned FSMap version 99 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56541cbe66b0 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4026070 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f14a8052020 con 0x56541cecc9b0 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 10 client.9636 fetch_fsmap finished waiting for FSMap version 99 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 10 client.9636 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 20 client.9636 populate_metadata read hostname 'smithi040' 2024-04-26T21:21:36.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cefb800 0x56541cee2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 4 client.9636 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 --> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541ce18350 con 0x56541cefb800 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14af7fe640 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cefb800 0x56541cee2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f14af7fe640 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cefb800 0x56541cee2920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5356e6540efcec5d server_cookie=bdfc8562a34307b0 in_seq=0 out_seq=0 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.521+0000 7f148e7fc640 10 client.9636 ms_handle_connect on v2:172.21.15.40:6833/3970947174 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f148e7fc640 1 -- 172.21.15.40:0/4042632594 <== mds.0 v2:172.21.15.40:6833/3970947174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56541ce18350 con 0x56541cefb800 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f148e7fc640 10 client.9636 handle_command_reply: tid=0 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 client.9636 shutdown 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 msgr2=0x56541ce9fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f14a802ac90 tx=0x7f14a8004ab0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 shutdown_connections 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cefb800 0x56541cee2920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541cecc9b0 0x56541ce9fbb0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541ceedce0 0x56541ce9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f148e7fc640 0 client.9636 ms_handle_reset on v2:172.21.15.40:6833/3970947174 2024-04-26T21:21:36.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 --2- 172.21.15.40:0/4042632594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541ced4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 >> 172.21.15.40:0/4042632594 conn(0x56541cea6120 msgr2=0x56541ceee7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 shutdown_connections 2024-04-26T21:21:36.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 1 -- 172.21.15.40:0/4042632594 wait complete. 2024-04-26T21:21:36.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.522+0000 7f14b8db3740 20 client.9636 trim_cache size 0 max 16384 2024-04-26T21:21:36.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 Processor -- start 2024-04-26T21:21:36.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 -- start start 2024-04-26T21:21:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cea6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14b00cac60 con 0x56541ceedce0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14b00c7420 con 0x56541ced0440 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.541+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f14b00ca610 con 0x56541cecc9b0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58512/0 (socket says 172.21.15.40:58512) 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 -- 172.21.15.40:0/2399451969 learned_addr learned my addr 172.21.15.40:0/2399451969 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14af7fe640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cea6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14aeffd640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 -- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 msgr2=0x56541cea6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cea6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 -- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 msgr2=0x56541cea6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 -- 172.21.15.40:0/2399451969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541ced0b90 con 0x56541ceedce0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14aeffd640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f14affff640 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009c80 tx=0x7f14a803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2afacbc6c2fdd00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f1494ff9640 1 -- 172.21.15.40:0/2399451969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8058070 con 0x56541ceedce0 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f1494ff9640 1 -- 172.21.15.40:0/2399451969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8037040 con 0x56541ceedce0 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.542+0000 7f1494ff9640 1 -- 172.21.15.40:0/2399451969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8036040 con 0x56541ceedce0 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- 172.21.15.40:0/2399451969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 msgr2=0x56541cecae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009c80 tx=0x7f14a803e000 comp rx=0 tx=0).stop 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- 172.21.15.40:0/2399451969 shutdown_connections 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cecae20 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 --2- 172.21.15.40:0/2399451969 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cea6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- 172.21.15.40:0/2399451969 >> 172.21.15.40:0/2399451969 conn(0x56541cea6120 msgr2=0x56541cea58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- 172.21.15.40:0/2399451969 shutdown_connections 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- 172.21.15.40:0/2399451969 wait complete. 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.543+0000 7f14b8db3740 1 -- start start 2024-04-26T21:21:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 Processor -- start 2024-04-26T21:21:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cf64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56541cefb680 con 0x56541ceedce0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56541cefc460 con 0x56541ced0440 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x56541cefbca0 con 0x56541cecc9b0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14b8db3740 10 client.0 _ll_register_callbacks cb 0x7f14a0009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58514/0 (socket says 172.21.15.40:58514) 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.544+0000 7f14affff640 1 -- 172.21.15.40:0/3996329451 learned_addr learned my addr 172.21.15.40:0/3996329451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14af7fe640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cf64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14aeffd640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 -- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 msgr2=0x56541cf64db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cf64db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 -- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 msgr2=0x56541cea6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 -- 172.21.15.40:0/3996329451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541ceec200 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14af7fe640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cf64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14aeffd640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14affff640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c090 tx=0x7f14a8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a874fa368e019d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8061060 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f14a8004030 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a8037040 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f14a8067050 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14b8db3740 10 client.9642 fetch_fsmap learned FSMap version 99 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56541ceeed70 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8037040 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f14a8008070 con 0x56541ceedce0 2024-04-26T21:21:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.545+0000 7f14b8db3740 10 client.9642 fetch_fsmap finished waiting for FSMap version 99 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 10 client.9642 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 20 client.9642 populate_metadata read hostname 'smithi040' 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cecb570 0x56541cea3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 4 client.9642 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 --> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] -- command(tid 0: {"prefix": "status"}) -- 0x56541ce121c0 con 0x56541cecb570 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14af7fe640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cecb570 0x56541cea3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f148e7fc640 10 client.9642 ms_handle_connect on v2:172.21.15.40:6833/3970947174 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14af7fe640 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cecb570 0x56541cea3e60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adddca6940b2fce2 server_cookie=80dd4becfdd4893c in_seq=0 out_seq=0 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f148e7fc640 1 -- 172.21.15.40:0/3996329451 <== mds.0 v2:172.21.15.40:6833/3970947174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56541ce121c0 con 0x56541cecb570 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f148e7fc640 10 client.9642 handle_command_reply: tid=0 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 client.9642 shutdown 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 msgr2=0x56541cea6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c090 tx=0x7f14a8009810 comp rx=0 tx=0).stop 2024-04-26T21:21:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 shutdown_connections 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:6833/3970947174,v1:172.21.15.40:6835/3970947174] conn(0x56541cecb570 0x56541cea3e60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56541ceedce0 0x56541cea6a40 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56541ced0440 0x56541cea6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 --2- 172.21.15.40:0/3996329451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56541cecc9b0 0x56541cf64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f148e7fc640 0 client.9642 ms_handle_reset on v2:172.21.15.40:6833/3970947174 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 >> 172.21.15.40:0/3996329451 conn(0x56541cea6120 msgr2=0x56541cea5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 shutdown_connections 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.546+0000 7f14b8db3740 1 -- 172.21.15.40:0/3996329451 wait complete. 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b8db3740 20 client.9642 trim_cache size 0 max 16384 2024-04-26T21:21:36.554 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "eee7569e-7e8d-4976-9855-170ac833730d", 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 9501, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 5.7149952989999999, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 99, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 145, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 145, 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 10.730013574999999 2024-04-26T21:21:36.555 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 msgr2=0x7f149c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 0x7f149c06edb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f14b01f8ab0 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 msgr2=0x7f14b006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9b0 tx=0x7f14a000ee70 comp rx=0 tx=0).stop 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 shutdown_connections 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f149c06c8f0 0x7f149c06edb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b0160d50 0x7f14b006d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b015d380 0x7f14b0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 --2- 172.21.15.40:0/833607944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b00b56d0 0x7f14b006eef0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 >> 172.21.15.40:0/833607944 conn(0x7f14b00b4a60 msgr2=0x7f14b00b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 shutdown_connections 2024-04-26T21:21:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.547+0000 7f14b6093640 1 -- 172.21.15.40:0/833607944 wait complete. 2024-04-26T21:21:36.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:21:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.659+0000 7f066fd0a640 1 Processor -- start 2024-04-26T21:21:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.659+0000 7f066fd0a640 1 -- start start 2024-04-26T21:21:36.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066fd0a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06680d3ee0 0x7f06680cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066fd0a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f066815f750 0x7f06680b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066fd0a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06680d03a0 con 0x7f066815f370 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066fd0a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f06680b4d10 con 0x7f06680d3ee0 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066fd0a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f06680d0680 con 0x7f066815f750 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06680d3ee0 0x7f06680cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06680d3ee0 0x7f06680cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58530/0 (socket says 172.21.15.40:58530) 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066daa6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f066815f370 0x7f06680d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36482/0 (socket says 172.21.15.40:36482) 2024-04-26T21:21:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 -- 172.21.15.40:0/979965561 learned_addr learned my addr 172.21.15.40:0/979965561 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066e2a7640 1 --2- 172.21.15.40:0/979965561 >> v2:172.21.15.184:3300/0 conn(0x7f066815f750 0x7f06680b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 -- 172.21.15.40:0/979965561 >> v2:172.21.15.184:3300/0 conn(0x7f066815f750 msgr2=0x7f06680b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 --2- 172.21.15.40:0/979965561 >> v2:172.21.15.184:3300/0 conn(0x7f066815f750 0x7f06680b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 -- 172.21.15.40:0/979965561 >> v1:172.21.15.40:6789/0 conn(0x7f066815f370 legacy=0x7f06680d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066caa4640 1 -- 172.21.15.40:0/979965561 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705383166 0 0) 0x7f06680d03a0 con 0x7f066815f370 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.660+0000 7f066d2a5640 1 -- 172.21.15.40:0/979965561 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06680d0960 con 0x7f06680d3ee0 2024-04-26T21:21:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.661+0000 7f066d2a5640 1 --2- 172.21.15.40:0/979965561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06680cfd70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d06056b7886002f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.661+0000 7f066caa4640 1 -- 172.21.15.40:0/979965561 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658037070 con 0x7f06680d3ee0 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.661+0000 7f066caa4640 1 -- 172.21.15.40:0/979965561 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658032440 con 0x7f06680d3ee0 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.661+0000 7f066caa4640 1 -- 172.21.15.40:0/979965561 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658032810 con 0x7f06680d3ee0 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 -- 172.21.15.40:0/979965561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 msgr2=0x7f06680cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 --2- 172.21.15.40:0/979965561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06680cfd70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802a2a0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 -- 172.21.15.40:0/979965561 shutdown_connections 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 --2- 172.21.15.40:0/979965561 >> v2:172.21.15.184:3300/0 conn(0x7f066815f750 0x7f06680b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 --2- 172.21.15.40:0/979965561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06680cfd70 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 -- 172.21.15.40:0/979965561 >> 172.21.15.40:0/979965561 conn(0x7f066808ec10 msgr2=0x7f066809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 -- 172.21.15.40:0/979965561 shutdown_connections 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.662+0000 7f066fd0a640 1 -- 172.21.15.40:0/979965561 wait complete. 2024-04-26T21:21:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.663+0000 7f066fd0a640 1 Processor -- start 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.663+0000 7f066fd0a640 1 -- start start 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066815f370 0x7f06681f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066815f750 0x7f066806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06680b51c0 con 0x7f06680d3ee0 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06680d0f20 con 0x7f066815f370 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f06680b5340 con 0x7f066815f750 2024-04-26T21:21:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58536/0 (socket says 172.21.15.40:58536) 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 -- 172.21.15.40:0/2987893397 learned_addr learned my addr 172.21.15.40:0/2987893397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066d2a5640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066815f370 0x7f06681f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066e2a7640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066815f750 0x7f066806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 -- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066815f750 msgr2=0x7f066806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066815f750 0x7f066806dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 -- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066815f370 msgr2=0x7f06681f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066815f370 0x7f06681f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 -- 172.21.15.40:0/2987893397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06681f9110 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066daa6640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f066400c500 tx=0x7f066400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664018070 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0664007580 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664004a20 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.664+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066814e200 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.665+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066814e7c0 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.665+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066806a290 con 0x7f06680d3ee0 2024-04-26T21:21:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.666+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f066401b070 con 0x7f06680d3ee0 2024-04-26T21:21:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.666+0000 7f065e7fc640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 0x7f064c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.666+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f066408aec0 con 0x7f06680d3ee0 2024-04-26T21:21:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.666+0000 7f066d2a5640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 0x7f064c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.667+0000 7f066d2a5640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 0x7f064c06edb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0658034000 tx=0x7f065802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.670+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06681f9110 con 0x7f06680d3ee0 2024-04-26T21:21:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.899+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f06680d14a0 con 0x7f06680d3ee0 2024-04-26T21:21:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.899+0000 7f065e7fc640 1 -- 172.21.15.40:0/2987893397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f06640592e0 con 0x7f06680d3ee0 2024-04-26T21:21:36.901 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 msgr2=0x7f064c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 0x7f064c06edb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0658034000 tx=0x7f065802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 msgr2=0x7f06681f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f066400c500 tx=0x7f066400c9c0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 shutdown_connections 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f064c06c8f0 0x7f064c06edb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f066815f750 0x7f066806dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f066815f370 0x7f06681f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 --2- 172.21.15.40:0/2987893397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06680d3ee0 0x7f06681f8bd0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 >> 172.21.15.40:0/2987893397 conn(0x7f066808ec10 msgr2=0x7f06680d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 shutdown_connections 2024-04-26T21:21:36.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:36.902+0000 7f066fd0a640 1 -- 172.21.15.40:0/2987893397 wait complete. 2024-04-26T21:21:36.913 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:21:36.913 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:21:36.913 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:21:36.914 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:21:36.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:36.914 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-26T21:21:36.930 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:21:36.930 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:21:36.930 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:21:36.930 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-26T21:21:36.931 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2815:1dff:fe36:8c7b/64 scope link 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.932 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 12:18:cd:67:1a:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::1018:cdff:fe67:1a58/64 scope link 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:a1:bc:79:25:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f8a1:bcff:fe79:2573/64 scope link 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:36.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:36.933 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:21:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-26T21:21:36.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:36.995 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-26T21:21:37.050 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-26T21:21:37.051 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:21:37.051 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:21:37.051 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:21:37.051 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:21:37.051 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:21:37.051 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-26T21:21:37.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:21:37.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:21:37.051 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:21:37.052 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-26T21:21:37.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:21:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:37.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:37.176 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:21:37.192 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:21:37.192 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:21:37.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:21:37.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:37.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-26T21:21:37.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:21:37.257 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:37.313 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:21:37.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:21:37.313 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:37.368 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:21:37.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:37.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:21:37.452 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:21:37.452 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:21:37.452 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:21:37.461 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:21:37.461 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:21:37.461 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:21:37.461 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:21:37.461 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:21:37.466 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:21:37.467 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:21:37.467 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:21:37.467 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:21:37.467 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:21:37.467 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:21:37.468 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:21:37.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:37.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:37.535 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:21:37.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:21:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:37.636 INFO:teuthology.orchestra.run.smithi040.stdout:client7121 2024-04-26T21:21:37.637 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:37.637 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:37.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:21:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.792+0000 7f8b08dc9640 1 Processor -- start 2024-04-26T21:21:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.792+0000 7f8b08dc9640 1 -- start start 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b08dc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 0x7f8b0406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b08dc9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b0406e2c0 0x7f8b0415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b08dc9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b040d1740 con 0x7f8b040cf2e0 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b08dc9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8b0406e750 con 0x7f8b0406e2c0 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b08dc9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b040d1a20 con 0x7f8b040d11b0 2024-04-26T21:21:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b0259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b040cf2e0 0x7f8b0406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36490/0 (socket says 172.21.15.40:36490) 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b0259c640 1 -- 172.21.15.40:0/25523104 learned_addr learned my addr 172.21.15.40:0/25523104 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8afbfff640 1 --2- 172.21.15.40:0/25523104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 0x7f8b0406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b02d9d640 1 --2- 172.21.15.40:0/25523104 >> v2:172.21.15.184:3300/0 conn(0x7f8b0406e2c0 0x7f8b0415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b01d9b640 1 -- 172.21.15.40:0/25523104 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884905315 0 0) 0x7f8b040d1740 con 0x7f8b040cf2e0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8b01d9b640 1 -- 172.21.15.40:0/25523104 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8af0003520 con 0x7f8b040cf2e0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8afbfff640 1 -- 172.21.15.40:0/25523104 >> v2:172.21.15.184:3300/0 conn(0x7f8b0406e2c0 msgr2=0x7f8b0415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8afbfff640 1 --2- 172.21.15.40:0/25523104 >> v2:172.21.15.184:3300/0 conn(0x7f8b0406e2c0 0x7f8b0415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8afbfff640 1 -- 172.21.15.40:0/25523104 >> v1:172.21.15.40:6789/0 conn(0x7f8b040cf2e0 legacy=0x7f8b0406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.793+0000 7f8afbfff640 1 -- 172.21.15.40:0/25523104 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040d1d00 con 0x7f8b040d11b0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.794+0000 7f8afbfff640 1 --2- 172.21.15.40:0/25523104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 0x7f8b0406dd80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0097a0 tx=0x7f8aec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c18dd5c388caf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.794+0000 7f8b01d9b640 1 -- 172.21.15.40:0/25523104 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec037070 con 0x7f8b040d11b0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.794+0000 7f8b01d9b640 1 -- 172.21.15.40:0/25523104 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aec032440 con 0x7f8b040d11b0 2024-04-26T21:21:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.794+0000 7f8b01d9b640 1 -- 172.21.15.40:0/25523104 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0327f0 con 0x7f8b040d11b0 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 -- 172.21.15.40:0/25523104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 msgr2=0x7f8b0406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/25523104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 0x7f8b0406dd80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0097a0 tx=0x7f8aec02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 -- 172.21.15.40:0/25523104 shutdown_connections 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/25523104 >> v2:172.21.15.184:3300/0 conn(0x7f8b0406e2c0 0x7f8b0415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/25523104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b040d11b0 0x7f8b0406dd80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 -- 172.21.15.40:0/25523104 >> 172.21.15.40:0/25523104 conn(0x7f8b040b4a60 msgr2=0x7f8b040b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 -- 172.21.15.40:0/25523104 shutdown_connections 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.795+0000 7f8b08dc9640 1 -- 172.21.15.40:0/25523104 wait complete. 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.796+0000 7f8b08dc9640 1 Processor -- start 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.796+0000 7f8b08dc9640 1 -- start start 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0406e2c0 0x7f8b0415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 0x7f8b040a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040cf6c0 con 0x7f8b040cf2e0 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040d22c0 con 0x7f8b040d11b0 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b08dc9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8b040cf840 con 0x7f8b0406e2c0 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8afbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8afbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58542/0 (socket says 172.21.15.40:58542) 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8afbfff640 1 -- 172.21.15.40:0/855703851 learned_addr learned my addr 172.21.15.40:0/855703851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b0259c640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0406e2c0 0x7f8b0415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.797+0000 7f8b02d9d640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 0x7f8b040a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 -- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0406e2c0 msgr2=0x7f8b0415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0406e2c0 0x7f8b0415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 -- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 msgr2=0x7f8b040a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 0x7f8b040a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 -- 172.21.15.40:0/855703851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040aff70 con 0x7f8b040cf2e0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8b02d9d640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 0x7f8b040a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afbfff640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000c00 tx=0x7f8aec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec041020 con 0x7f8b040cf2e0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aec03d620 con 0x7f8b040cf2e0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec037040 con 0x7f8b040cf2e0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b040b0250 con 0x7f8b040cf2e0 2024-04-26T21:21:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.798+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b040b0810 con 0x7f8b040cf2e0 2024-04-26T21:21:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.799+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0406a290 con 0x7f8b040cf2e0 2024-04-26T21:21:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.799+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8aec044020 con 0x7f8b040cf2e0 2024-04-26T21:21:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.800+0000 7f8afaffd640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 0x7f8ae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.800+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8aec0ac350 con 0x7f8b040cf2e0 2024-04-26T21:21:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.800+0000 7f8b0259c640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 0x7f8ae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.801+0000 7f8b0259c640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 0x7f8ae406edb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005880 tx=0x7f8af4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:37.804+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8aec04b030 con 0x7f8b040cf2e0 2024-04-26T21:21:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.031+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b040b09e0 con 0x7f8b040cf2e0 2024-04-26T21:21:38.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.031+0000 7f8afaffd640 1 -- 172.21.15.40:0/855703851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f8aec07a790 con 0x7f8b040cf2e0 2024-04-26T21:21:38.032 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 msgr2=0x7f8ae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 0x7f8ae406edb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005880 tx=0x7f8af4009040 comp rx=0 tx=0).stop 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 msgr2=0x7f8b041605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000c00 tx=0x7f8aec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 shutdown_connections 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8ae406c8f0 0x7f8ae406edb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b040d11b0 0x7f8b040a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b040cf2e0 0x7f8b041605b0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:38.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 --2- 172.21.15.40:0/855703851 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0406e2c0 0x7f8b0415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:38.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 >> 172.21.15.40:0/855703851 conn(0x7f8b040b4a60 msgr2=0x7f8b040b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:38.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 shutdown_connections 2024-04-26T21:21:38.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:38.033+0000 7f8b08dc9640 1 -- 172.21.15.40:0/855703851 wait complete. 2024-04-26T21:21:38.045 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:21:38.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:21:38.046 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:21:38.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.046 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2024-04-26T21:21:38.065 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:38.066 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42175sec preferred_lft 42175sec 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ad8:d1a3:97c5:ce71/64 scope link noprefixroute 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.067 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::cc7a:c5ff:fe40:effa/64 scope link 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 16:58:d5:45:67:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::1458:d5ff:fe45:678e/64 scope link 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 36:c4:31:2a:04:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::34c4:31ff:fe2a:449/64 scope link 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:21:38.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.068 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:21:38.132 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-04-26T21:21:38.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.133 DEBUG:teuthology.orchestra.run.smithi184:> route 2024-04-26T21:21:38.191 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2024-04-26T21:21:38.191 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:21:38.191 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:21:38.192 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:21:38.192 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:21:38.192 DEBUG:teuthology.orchestra.run.smithi184:> ') 2024-04-26T21:21:38.283 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:21:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:21:38.314 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:21:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:38.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.320 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:21:38.335 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:21:38.335 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:21:38.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:21:38.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.336 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2024-04-26T21:21:38.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:21:38.399 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:38.454 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:21:38.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:21:38.455 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:38.509 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T21:21:38.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.509 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:21:38.591 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:21:38.591 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:21:38.591 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:21:38.599 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:21:38.599 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:21:38.599 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:21:38.599 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:21:38.599 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:invalid new device string format 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.184:3300,172.21.15.184:3301" 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790" 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.184:6789,172.21.15.184:6790:/ 2024-04-26T21:21:38.604 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=eee7569e-7e8d-4976-9855-170ac833730d" will pass to kernel 2024-04-26T21:21:38.605 INFO:teuthology.orchestra.run.smithi184.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:21:38.605 INFO:teuthology.orchestra.run.smithi184.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:21:38.605 INFO:teuthology.orchestra.run.smithi184.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:21:38.605 INFO:teuthology.orchestra.run.smithi184.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:21:38.605 INFO:teuthology.orchestra.run.smithi184.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:21:38.606 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:21:38.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:38.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:38.677 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:21:38.766 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:21:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:21:38.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-04-26T21:21:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:21:38.773 INFO:teuthology.orchestra.run.smithi184.stdout:client8161 2024-04-26T21:21:38.773 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:38.773 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:38.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:21:38.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:38.834 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:21:38.870 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:21:38.870 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:38.870 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:38.871 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:38.893 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:38.893 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:38.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.097+0000 7f6eb2864640 1 Processor -- start 2024-04-26T21:21:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.097+0000 7f6eb2864640 1 -- start start 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb2864640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6eac148a30 0x7f6eac073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb2864640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6eac0b3af0 0x7f6eac0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb2864640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac0cfa80 con 0x7f6eac160640 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb2864640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6eac0cfd60 con 0x7f6eac0b3af0 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb2864640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6eac0d0040 con 0x7f6eac148a30 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eab7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6eac160640 0x7f6eac1636a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36498/0 (socket says 172.21.15.40:36498) 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eab7fe640 1 -- 172.21.15.40:0/2226087059 learned_addr learned my addr 172.21.15.40:0/2226087059 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.098+0000 7f6eb0e01640 1 --2- 172.21.15.40:0/2226087059 >> v2:172.21.15.40:3300/0 conn(0x7f6eac0b3af0 0x7f6eac0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eabfff640 1 --2- 172.21.15.40:0/2226087059 >> v2:172.21.15.184:3300/0 conn(0x7f6eac148a30 0x7f6eac073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eaaffd640 1 -- 172.21.15.40:0/2226087059 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930038663 0 0) 0x7f6eac0cfa80 con 0x7f6eac160640 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eaaffd640 1 -- 172.21.15.40:0/2226087059 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e94003520 con 0x7f6eac160640 2024-04-26T21:21:39.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eb0e01640 1 -- 172.21.15.40:0/2226087059 >> v2:172.21.15.184:3300/0 conn(0x7f6eac148a30 msgr2=0x7f6eac073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eb0e01640 1 --2- 172.21.15.40:0/2226087059 >> v2:172.21.15.184:3300/0 conn(0x7f6eac148a30 0x7f6eac073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eb0e01640 1 -- 172.21.15.40:0/2226087059 >> v1:172.21.15.40:6789/0 conn(0x7f6eac160640 legacy=0x7f6eac1636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.099+0000 7f6eb0e01640 1 -- 172.21.15.40:0/2226087059 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac162ea0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb0e01640 1 --2- 172.21.15.40:0/2226087059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b3fa0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0cf6d0 tx=0x7f6ea402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fd9111c48b954c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eaaffd640 1 -- 172.21.15.40:0/2226087059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4007740 con 0x7f6eac0b3af0 2024-04-26T21:21:39.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eaaffd640 1 -- 172.21.15.40:0/2226087059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea40078e0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eaaffd640 1 -- 172.21.15.40:0/2226087059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4007cb0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 -- 172.21.15.40:0/2226087059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 msgr2=0x7f6eac0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 --2- 172.21.15.40:0/2226087059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b3fa0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0cf6d0 tx=0x7f6ea402a5f0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 -- 172.21.15.40:0/2226087059 shutdown_connections 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 --2- 172.21.15.40:0/2226087059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b3fa0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 --2- 172.21.15.40:0/2226087059 >> v2:172.21.15.184:3300/0 conn(0x7f6eac148a30 0x7f6eac073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 -- 172.21.15.40:0/2226087059 >> 172.21.15.40:0/2226087059 conn(0x7f6eac0ce8b0 msgr2=0x7f6eac0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 -- 172.21.15.40:0/2226087059 shutdown_connections 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.100+0000 7f6eb2864640 1 -- 172.21.15.40:0/2226087059 wait complete. 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.102+0000 7f6eb2864640 1 Processor -- start 2024-04-26T21:21:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.102+0000 7f6eb2864640 1 -- start start 2024-04-26T21:21:39.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.102+0000 7f6eb2864640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eac148a30 0x7f6eac142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 0x7f6eac0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac074410 con 0x7f6eac0b3af0 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac163db0 con 0x7f6eac160640 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6eac074590 con 0x7f6eac148a30 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58562/0 (socket says 172.21.15.40:58562) 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 -- 172.21.15.40:0/3588234710 learned_addr learned my addr 172.21.15.40:0/3588234710 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eab7fe640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eac148a30 0x7f6eac142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb0e01640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 0x7f6eac0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 -- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eac148a30 msgr2=0x7f6eac142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eac148a30 0x7f6eac142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 -- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 msgr2=0x7f6eac0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 0x7f6eac0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 -- 172.21.15.40:0/3588234710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac0b2770 con 0x7f6eac0b3af0 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb0e01640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 0x7f6eac0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eabfff640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e9e0 tx=0x7f6ea000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0017070 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea000c970 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac0b2a10 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000cd60 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.103+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac143140 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.104+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac06a290 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.105+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea001a070 con 0x7f6eac0b3af0 2024-04-26T21:21:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.105+0000 7f6ea8ff9640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 0x7f6e9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.105+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea008ba80 con 0x7f6eac0b3af0 2024-04-26T21:21:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.105+0000 7f6eab7fe640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 0x7f6e9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.106+0000 7f6eab7fe640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 0x7f6e9806ede0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0701b0 tx=0x7f6e9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.110+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ea0059eb0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.296+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6eac0739f0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.297+0000 7f6ea8ff9640 1 -- 172.21.15.40:0/3588234710 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7f6ea00597b0 con 0x7f6eac0b3af0 2024-04-26T21:21:39.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:39.300 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 msgr2=0x7f6e9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 0x7f6e9806ede0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0701b0 tx=0x7f6e9c009040 comp rx=0 tx=0).stop 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 msgr2=0x7f6eac0b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e9e0 tx=0x7f6ea000eea0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 shutdown_connections 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6e9806c920 0x7f6e9806ede0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eac160640 0x7f6eac0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eac148a30 0x7f6eac142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 --2- 172.21.15.40:0/3588234710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eac0b3af0 0x7f6eac0b7a50 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 >> 172.21.15.40:0/3588234710 conn(0x7f6eac0ce8b0 msgr2=0x7f6eac06f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 shutdown_connections 2024-04-26T21:21:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.299+0000 7f6eb2864640 1 -- 172.21.15.40:0/3588234710 wait complete. 2024-04-26T21:21:39.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:21:28.644565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:21:29.151922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:39.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.416+0000 7f0d3c719640 1 Processor -- start 2024-04-26T21:21:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.416+0000 7f0d3c719640 1 -- start start 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3c719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d3406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3c719640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3409c700 con 0x7f0d340b33e0 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3c719640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d340d1c50 con 0x7f0d340b3c20 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3c719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d340b4480 con 0x7f0d340d0010 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3acb6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d340b33e0 0x7f0d34163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36506/0 (socket says 172.21.15.40:36506) 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3acb6640 1 -- 172.21.15.40:0/2476957942 learned_addr learned my addr 172.21.15.40:0/2476957942 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d3406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812799616 0 0) 0x7f0d3409c700 con 0x7f0d340b33e0 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d20003520 con 0x7f0d340b33e0 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006045314 0 0) 0x7f0d340d1c50 con 0x7f0d340b3c20 2024-04-26T21:21:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d3409c700 con 0x7f0d340b3c20 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584409743 0 0) 0x7f0d20003520 con 0x7f0d340b33e0 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 msgr2=0x7f0d3406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d3406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 >> v1:172.21.15.184:6789/0 conn(0x7f0d340b3c20 legacy=0x7f0d340b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d340d1f30 con 0x7f0d340b33e0 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f0d2c0019f0 con 0x7f0d340b33e0 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d3406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 >> v1:172.21.15.40:6789/0 conn(0x7f0d340b33e0 legacy=0x7f0d34163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d20003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.417+0000 7f0d394b3640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d200040b0 0x7f0d20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d394b3640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d20004d40 0x7f0d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d340b4480 con 0x7f0d200040b0 2024-04-26T21:21:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2000ba70 con 0x7f0d20004d40 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2000bc70 con 0x7f0d340d0010 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d200040b0 0x7f0d20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d20003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d3acb6640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d20004d40 0x7f0d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 -- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 msgr2=0x7f0d20003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d340d0010 0x7f0d20003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 -- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d20004d40 msgr2=0x7f0d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d20004d40 0x7f0d200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.418+0000 7f0d39cb4640 1 -- 172.21.15.40:0/2476957942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d20003520 con 0x7f0d200040b0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d39cb4640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d200040b0 0x7f0d20004800 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f0d240018d0 tx=0x7f0d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02bf4d36affc189 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240014f0 con 0x7f0d200040b0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d240052d0 con 0x7f0d200040b0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d394b3640 1 -- 172.21.15.40:0/2476957942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2402aca0 con 0x7f0d200040b0 2024-04-26T21:21:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 -- 172.21.15.40:0/2476957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d200040b0 msgr2=0x7f0d20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 --2- 172.21.15.40:0/2476957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d200040b0 0x7f0d20004800 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f0d240018d0 tx=0x7f0d20005830 comp rx=0 tx=0).stop 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3acb6640 1 -- 172.21.15.40:0/2476957942 reap_dead start 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 -- 172.21.15.40:0/2476957942 shutdown_connections 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 -- 172.21.15.40:0/2476957942 >> 172.21.15.40:0/2476957942 conn(0x7f0d3409ba40 msgr2=0x7f0d3409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 -- 172.21.15.40:0/2476957942 shutdown_connections 2024-04-26T21:21:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.419+0000 7f0d3c719640 1 -- 172.21.15.40:0/2476957942 wait complete. 2024-04-26T21:21:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 Processor -- start 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 -- start start 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d200040b0 0x7f0d341f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d341f8440 0x7f0d3409d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3406db10 con 0x7f0d3409dd80 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d340b3a80 con 0x7f0d200040b0 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3c719640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3406dc90 con 0x7f0d341f8440 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3acb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3acb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58590/0 (socket says 172.21.15.40:58590) 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.421+0000 7f0d3acb6640 1 -- 172.21.15.40:0/584981434 learned_addr learned my addr 172.21.15.40:0/584981434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d200040b0 0x7f0d341f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d39cb4640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d341f8440 0x7f0d3409d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 -- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d341f8440 msgr2=0x7f0d3409d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d341f8440 0x7f0d3409d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 -- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d200040b0 msgr2=0x7f0d341f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d200040b0 0x7f0d341f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 -- 172.21.15.40:0/584981434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3409e300 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3acb6640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f0d34073ee0 tx=0x7f0d2c007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00ab70 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c019070 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3415c730 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0133f0 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.422+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3415ccf0 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.423+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3406a290 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.423+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d2c013590 con 0x7f0d3409dd80 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.424+0000 7f0d2affd640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 0x7f0d1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.424+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f0d2c08bf90 con 0x7f0d3409dd80 2024-04-26T21:21:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.424+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 0x7f0d1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.425+0000 7f0d3a4b5640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 0x7f0d1806edb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ad0 tx=0x7f0d30009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.428+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d2c059d70 con 0x7f0d3409dd80 2024-04-26T21:21:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.615+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d340b85c0 con 0x7f0d3409dd80 2024-04-26T21:21:39.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.616+0000 7f0d2affd640 1 -- 172.21.15.40:0/584981434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f0d2c05e070 con 0x7f0d3409dd80 2024-04-26T21:21:39.617 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 msgr2=0x7f0d1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 0x7f0d1806edb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ad0 tx=0x7f0d30009190 comp rx=0 tx=0).stop 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 msgr2=0x7f0d3409f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f0d34073ee0 tx=0x7f0d2c007b60 comp rx=0 tx=0).stop 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 shutdown_connections 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f0d1806c8f0 0x7f0d1806edb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d3409dd80 0x7f0d3409f380 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d341f8440 0x7f0d3409d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 --2- 172.21.15.40:0/584981434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d200040b0 0x7f0d341f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 >> 172.21.15.40:0/584981434 conn(0x7f0d3409ba40 msgr2=0x7f0d340b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 shutdown_connections 2024-04-26T21:21:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.618+0000 7f0d3c719640 1 -- 172.21.15.40:0/584981434 wait complete. 2024-04-26T21:21:39.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:39.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 Processor -- start 2024-04-26T21:21:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 -- start start 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5de00d0300 0x7f5de00b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de00b46a0 con 0x7f5de006f340 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de00b5db0 con 0x7f5de00b3e50 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.733+0000 7f5de506b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5de00b6090 con 0x7f5de00d0300 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dded9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5de006f340 0x7f5de006f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36520/0 (socket says 172.21.15.40:36520) 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dde59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5de00d0300 0x7f5de00b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dded9d640 1 -- 172.21.15.40:0/112891032 learned_addr learned my addr 172.21.15.40:0/112891032 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dde59c640 1 -- 172.21.15.40:0/112891032 >> v1:172.21.15.184:6789/0 conn(0x7f5de00b3e50 legacy=0x7f5de00720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dde59c640 1 -- 172.21.15.40:0/112891032 >> v1:172.21.15.40:6789/0 conn(0x7f5de006f340 legacy=0x7f5de006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dde59c640 1 -- 172.21.15.40:0/112891032 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00b6370 con 0x7f5de00d0300 2024-04-26T21:21:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.734+0000 7f5dddd9b640 1 -- 172.21.15.40:0/112891032 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983008389 0 0) 0x7f5de00b46a0 con 0x7f5de006f340 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5dde59c640 1 --2- 172.21.15.40:0/112891032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de00d0300 0x7f5de00b6b70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80097a0 tx=0x7f5dc802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bacef053e6f824 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5dddd9b640 1 -- 172.21.15.40:0/112891032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8037070 con 0x7f5de00d0300 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5dddd9b640 1 -- 172.21.15.40:0/112891032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc802adc0 con 0x7f5de00d0300 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5dddd9b640 1 -- 172.21.15.40:0/112891032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8032660 con 0x7f5de00d0300 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 -- 172.21.15.40:0/112891032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de00d0300 msgr2=0x7f5de00b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 --2- 172.21.15.40:0/112891032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de00d0300 0x7f5de00b6b70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80097a0 tx=0x7f5dc802a0b0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 -- 172.21.15.40:0/112891032 shutdown_connections 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 --2- 172.21.15.40:0/112891032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de00d0300 0x7f5de00b6b70 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 -- 172.21.15.40:0/112891032 >> 172.21.15.40:0/112891032 conn(0x7f5de00cd8a0 msgr2=0x7f5de00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 -- 172.21.15.40:0/112891032 shutdown_connections 2024-04-26T21:21:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.735+0000 7f5de506b640 1 -- 172.21.15.40:0/112891032 wait complete. 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.737+0000 7f5de506b640 1 Processor -- start 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.737+0000 7f5de506b640 1 -- start start 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.737+0000 7f5de506b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.737+0000 7f5de506b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 0x7f5de01f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 0x7f5de0156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00b4460 con 0x7f5de006f340 2024-04-26T21:21:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00b42e0 con 0x7f5de00d0300 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f5de0142a30 con 0x7f5de00b3e50 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58606/0 (socket says 172.21.15.40:58606) 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 -- 172.21.15.40:0/3832014108 learned_addr learned my addr 172.21.15.40:0/3832014108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dde59c640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 0x7f5de01f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5ddf59e640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 0x7f5de0156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 -- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 msgr2=0x7f5de01f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 0x7f5de01f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 -- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 msgr2=0x7f5de0156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 0x7f5de0156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 -- 172.21.15.40:0/3832014108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de015ef30 con 0x7f5de006f340 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dde59c640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 0x7f5de01f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5ddf59e640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 0x7f5de0156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dded9d640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c430 tx=0x7f5dd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80074e0 con 0x7f5de006f340 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd80047e0 con 0x7f5de006f340 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de0157070 con 0x7f5de006f340 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004bd0 con 0x7f5de006f340 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.738+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de015f1a0 con 0x7f5de006f340 2024-04-26T21:21:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.739+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de006a290 con 0x7f5de006f340 2024-04-26T21:21:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.740+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dd8011070 con 0x7f5de006f340 2024-04-26T21:21:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.741+0000 7f5dd77fe640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 0x7f5dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.741+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f5dd808b780 con 0x7f5de006f340 2024-04-26T21:21:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.741+0000 7f5dde59c640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 0x7f5dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.741+0000 7f5dde59c640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 0x7f5dc006edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8034000 tx=0x7f5dc802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.745+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5de0157070 con 0x7f5de006f340 2024-04-26T21:21:39.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.932+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5de00cbe40 con 0x7f5de006f340 2024-04-26T21:21:39.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.932+0000 7f5dd77fe640 1 -- 172.21.15.40:0/3832014108 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f5dd800e280 con 0x7f5de006f340 2024-04-26T21:21:39.933 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 msgr2=0x7f5dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 0x7f5dc006edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8034000 tx=0x7f5dc802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 msgr2=0x7f5de015c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c430 tx=0x7f5dd800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 shutdown_connections 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f5dc006c8f0 0x7f5dc006edb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5de00d0300 0x7f5de0156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5de00b3e50 0x7f5de01f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 --2- 172.21.15.40:0/3832014108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5de006f340 0x7f5de015c6e0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 >> 172.21.15.40:0/3832014108 conn(0x7f5de00cd8a0 msgr2=0x7f5de0072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 shutdown_connections 2024-04-26T21:21:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:39.934+0000 7f5de506b640 1 -- 172.21.15.40:0/3832014108 wait complete. 2024-04-26T21:21:39.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.048+0000 7f1d805b4640 1 Processor -- start 2024-04-26T21:21:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.048+0000 7f1d805b4640 1 -- start start 2024-04-26T21:21:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d805b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d781631f0 0x7f1d780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d805b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 0x7f1d781635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d805b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7809c790 con 0x7f1d780b3300 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d805b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d780d15f0 con 0x7f1d781631f0 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d805b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1d78163ec0 con 0x7f1d780d1170 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7e350640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d781631f0 0x7f1d780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7eb51640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d780b3300 0x7f1d780b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36534/0 (socket says 172.21.15.40:36534) 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7eb51640 1 -- 172.21.15.40:0/3499241872 learned_addr learned my addr 172.21.15.40:0/3499241872 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7db4f640 1 --2- 172.21.15.40:0/3499241872 >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 0x7f1d781635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7e350640 1 -- 172.21.15.40:0/3499241872 >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 msgr2=0x7f1d781635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7e350640 1 --2- 172.21.15.40:0/3499241872 >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 0x7f1d781635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7e350640 1 -- 172.21.15.40:0/3499241872 >> v1:172.21.15.40:6789/0 conn(0x7f1d780b3300 legacy=0x7f1d780b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7d34e640 1 -- 172.21.15.40:0/3499241872 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770793122 0 0) 0x7f1d7809c790 con 0x7f1d780b3300 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7e350640 1 -- 172.21.15.40:0/3499241872 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d780d1a40 con 0x7f1d781631f0 2024-04-26T21:21:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.049+0000 7f1d7db4f640 1 --2- 172.21.15.40:0/3499241872 >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 0x7f1d781635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d7e350640 1 --2- 172.21.15.40:0/3499241872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d780b2000 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1d780734b0 tx=0x7f1d7402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46293e1362e3bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d7d34e640 1 -- 172.21.15.40:0/3499241872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74037070 con 0x7f1d781631f0 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d7d34e640 1 -- 172.21.15.40:0/3499241872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d74032440 con 0x7f1d781631f0 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d7d34e640 1 -- 172.21.15.40:0/3499241872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7402ae60 con 0x7f1d781631f0 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d805b4640 1 -- 172.21.15.40:0/3499241872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 msgr2=0x7f1d780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.050+0000 7f1d805b4640 1 --2- 172.21.15.40:0/3499241872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d780b2000 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1d780734b0 tx=0x7f1d7402a110 comp rx=0 tx=0).stop 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 -- 172.21.15.40:0/3499241872 shutdown_connections 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 --2- 172.21.15.40:0/3499241872 >> v2:172.21.15.184:3300/0 conn(0x7f1d780d1170 0x7f1d781635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 --2- 172.21.15.40:0/3499241872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d780b2000 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 -- 172.21.15.40:0/3499241872 >> 172.21.15.40:0/3499241872 conn(0x7f1d7809ba40 msgr2=0x7f1d7809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 -- 172.21.15.40:0/3499241872 shutdown_connections 2024-04-26T21:21:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.051+0000 7f1d805b4640 1 -- 172.21.15.40:0/3499241872 wait complete. 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.052+0000 7f1d805b4640 1 Processor -- start 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.052+0000 7f1d805b4640 1 -- start start 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.052+0000 7f1d805b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 0x7f1d7815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.052+0000 7f1d805b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 0x7f1d781f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780d0230 con 0x7f1d781631f0 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780d2000 con 0x7f1d780d1170 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1d781473b0 con 0x7f1d780b3300 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58628/0 (socket says 172.21.15.40:58628) 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 -- 172.21.15.40:0/1807481075 learned_addr learned my addr 172.21.15.40:0/1807481075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7e350640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 0x7f1d7815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7db4f640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 0x7f1d781f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 -- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 msgr2=0x7f1d7815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 0x7f1d7815fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 -- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 msgr2=0x7f1d781f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 0x7f1d781f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 -- 172.21.15.40:0/1807481075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d78147690 con 0x7f1d781631f0 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7e350640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 0x7f1d7815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7db4f640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 0x7f1d781f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d7eb51640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c7d0 tx=0x7f1d7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70018070 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7000a640 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000aa70 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d78147970 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.053+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d78147f30 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.054+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7806a290 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.055+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d7000ac10 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.055+0000 7f1d6effd640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 0x7f1d5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.055+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1d7008b120 con 0x7f1d781631f0 2024-04-26T21:21:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.055+0000 7f1d7e350640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 0x7f1d5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.056+0000 7f1d7e350640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 0x7f1d5c06ede0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1d74034000 tx=0x7f1d7402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.060+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d700594d0 con 0x7f1d781631f0 2024-04-26T21:21:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.243+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d78148100 con 0x7f1d781631f0 2024-04-26T21:21:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.243+0000 7f1d6effd640 1 -- 172.21.15.40:0/1807481075 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f1d70058dd0 con 0x7f1d781631f0 2024-04-26T21:21:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 msgr2=0x7f1d5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 0x7f1d5c06ede0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1d74034000 tx=0x7f1d7402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:40.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 msgr2=0x7f1d78162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c7d0 tx=0x7f1d7000cc90 comp rx=0 tx=0).stop 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 shutdown_connections 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1d5c06c920 0x7f1d5c06ede0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d781631f0 0x7f1d78162590 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d780d1170 0x7f1d781f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 --2- 172.21.15.40:0/1807481075 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d780b3300 0x7f1d7815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 >> 172.21.15.40:0/1807481075 conn(0x7f1d7809ba40 msgr2=0x7f1d7806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 shutdown_connections 2024-04-26T21:21:40.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.245+0000 7f1d805b4640 1 -- 172.21.15.40:0/1807481075 wait complete. 2024-04-26T21:21:40.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:21:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.358+0000 7f6f94609640 1 Processor -- start 2024-04-26T21:21:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.358+0000 7f6f94609640 1 -- start start 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.359+0000 7f6f94609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f8c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.359+0000 7f6f94609640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f8c0d3ee0 0x7f6f8c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.359+0000 7f6f94609640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c0d03a0 con 0x7f6f8c15f750 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.359+0000 7f6f94609640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c0b4d10 con 0x7f6f8c0d3ee0 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.359+0000 7f6f94609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0d0680 con 0x7f6f8c15f370 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f91ba4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6f8c15f750 0x7f6f8c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36548/0 (socket says 172.21.15.40:36548) 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f91ba4640 1 -- 172.21.15.40:0/885652863 learned_addr learned my addr 172.21.15.40:0/885652863 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f92ba6640 1 --2- 172.21.15.40:0/885652863 >> v2:172.21.15.184:3300/0 conn(0x7f6f8c0d3ee0 0x7f6f8c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f923a5640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f8c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996261503 0 0) 0x7f6f8c0d03a0 con 0x7f6f8c15f750 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f78003520 con 0x7f6f8c15f750 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209006513 0 0) 0x7f6f78003520 con 0x7f6f8c15f750 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 msgr2=0x7f6f8c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f8c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 >> v2:172.21.15.184:3300/0 conn(0x7f6f8c0d3ee0 msgr2=0x7f6f8c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 --2- 172.21.15.40:0/885652863 >> v2:172.21.15.184:3300/0 conn(0x7f6f8c0d3ee0 0x7f6f8c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0d0960 con 0x7f6f8c15f750 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f6f7c001670 con 0x7f6f8c15f750 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 >> v1:172.21.15.40:6789/0 conn(0x7f6f8c15f750 legacy=0x7f6f8c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780039b0 0x7f6f78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f78004800 0x7f6f78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0b4d10 con 0x7f6f780039b0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0d0680 con 0x7f6f78004800 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7800b9c0 con 0x7f6f8c15f370 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f923a5640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780039b0 0x7f6f78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f91ba4640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f78004800 0x7f6f78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.360+0000 7f6f92ba6640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 -- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 msgr2=0x7f6f780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c15f370 0x7f6f780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 -- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f78004800 msgr2=0x7f6f78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f78004800 0x7f6f78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 -- 172.21.15.40:0/885652863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f78003520 con 0x7f6f780039b0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f91ba4640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f78004800 0x7f6f78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f923a5640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780039b0 0x7f6f78003dd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009af0 tx=0x7f6f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbb68df1eda3373 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88036070 con 0x7f6f780039b0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f88031400 con 0x7f6f780039b0 2024-04-26T21:21:40.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.361+0000 7f6f913a3640 1 -- 172.21.15.40:0/885652863 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88009690 con 0x7f6f780039b0 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 -- 172.21.15.40:0/885652863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780039b0 msgr2=0x7f6f78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 --2- 172.21.15.40:0/885652863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f780039b0 0x7f6f78003dd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009af0 tx=0x7f6f78005830 comp rx=0 tx=0).stop 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f92ba6640 1 -- 172.21.15.40:0/885652863 reap_dead start 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 -- 172.21.15.40:0/885652863 shutdown_connections 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 -- 172.21.15.40:0/885652863 >> 172.21.15.40:0/885652863 conn(0x7f6f8c08ec10 msgr2=0x7f6f8c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 -- 172.21.15.40:0/885652863 shutdown_connections 2024-04-26T21:21:40.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.362+0000 7f6f94609640 1 -- 172.21.15.40:0/885652863 wait complete. 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.363+0000 7f6f94609640 1 Processor -- start 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.363+0000 7f6f94609640 1 -- start start 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 0x7f6f8c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 0x7f6f8c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cfd70 con 0x7f6f8c0ca930 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0d0f20 con 0x7f6f780039b0 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f94609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cfef0 con 0x7f6f8c162b30 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58652/0 (socket says 172.21.15.40:58652) 2024-04-26T21:21:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 -- 172.21.15.40:0/2982549798 learned_addr learned my addr 172.21.15.40:0/2982549798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f923a5640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 0x7f6f8c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f91ba4640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 0x7f6f8c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 -- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 msgr2=0x7f6f8c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 0x7f6f8c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 -- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 msgr2=0x7f6f8c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 0x7f6f8c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f92ba6640 1 -- 172.21.15.40:0/2982549798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c15d270 con 0x7f6f8c0ca930 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f923a5640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 0x7f6f8c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.364+0000 7f6f91ba4640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 0x7f6f8c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f92ba6640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0b7cf0 tx=0x7f6f8400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84015430 con 0x7f6f8c0ca930 2024-04-26T21:21:40.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f840155f0 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840159e0 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c0ccaf0 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.365+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c0cd0b0 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.366+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c068460 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.366+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f84015b80 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.367+0000 7f6f82ffd640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 0x7f6f7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.367+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8408c910 con 0x7f6f8c0ca930 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.367+0000 7f6f923a5640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 0x7f6f7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.368+0000 7f6f923a5640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 0x7f6f7006ede0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6f88033000 tx=0x7f6f88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.371+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f8c068460 con 0x7f6f8c0ca930 2024-04-26T21:21:40.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.569+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f8c06a290 con 0x7f6f8c0ca930 2024-04-26T21:21:40.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.570+0000 7f6f82ffd640 1 -- 172.21.15.40:0/2982549798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f6f84019030 con 0x7f6f8c0ca930 2024-04-26T21:21:40.571 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 msgr2=0x7f6f7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 0x7f6f7006ede0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6f88033000 tx=0x7f6f88008030 comp rx=0 tx=0).stop 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 msgr2=0x7f6f8c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0b7cf0 tx=0x7f6f8400f580 comp rx=0 tx=0).stop 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 shutdown_connections 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6f7006c920 0x7f6f7006ede0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0ca930 0x7f6f8c0cadb0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f8c162b30 0x7f6f8c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 --2- 172.21.15.40:0/2982549798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f780039b0 0x7f6f8c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 >> 172.21.15.40:0/2982549798 conn(0x7f6f8c08ec10 msgr2=0x7f6f8c0b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 shutdown_connections 2024-04-26T21:21:40.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.572+0000 7f6f94609640 1 -- 172.21.15.40:0/2982549798 wait complete. 2024-04-26T21:21:40.583 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:21:40.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:40.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:40.612 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:21:40.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:40.613 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:21:40.668 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:40.668 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:40.668 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:40.688 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:40.688 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:40.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.845+0000 7f910322a640 1 Processor -- start 2024-04-26T21:21:40.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.845+0000 7f910322a640 1 -- start start 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f910322a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90fc0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f910322a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc0b4530 con 0x7f90fc0b61a0 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f910322a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc0b4810 con 0x7f90fc145980 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f910322a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc160ac0 con 0x7f90fc160630 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f90fbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f90fc0b61a0 0x7f90fc0720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36318/0 (socket says 172.21.15.40:36318) 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f90fbfff640 1 -- 172.21.15.40:0/4184358024 learned_addr learned my addr 172.21.15.40:0/4184358024 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f91017c7640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90fc0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329519210 0 0) 0x7f90fc0b4530 con 0x7f90fc0b61a0 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.846+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e8003520 con 0x7f90fc0b61a0 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3687260970 0 0) 0x7f90e8003520 con 0x7f90fc0b61a0 2024-04-26T21:21:40.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 msgr2=0x7f90fc0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90fc0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 >> v1:172.21.15.184:6789/0 conn(0x7f90fc145980 legacy=0x7f90fc0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc0b4af0 con 0x7f90fc0b61a0 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f90ec001690 con 0x7f90fc0b61a0 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 >> v1:172.21.15.40:6789/0 conn(0x7f90fc0b61a0 legacy=0x7f90fc0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e8003f20 0x7f90e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8004d40 0x7f90e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90e800baa0 con 0x7f90e8003f20 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90e800bc50 con 0x7f90e8004d40 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90e800be50 con 0x7f90fc160630 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e8003f20 0x7f90e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f9100fc6640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f91017c7640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8004d40 0x7f90e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 -- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 msgr2=0x7f90e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90fc160630 0x7f90e80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 -- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8004d40 msgr2=0x7f90e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8004d40 0x7f90e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.847+0000 7f90fbfff640 1 -- 172.21.15.40:0/4184358024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8003520 con 0x7f90e8003f20 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f90fbfff640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e8003f20 0x7f90e8004800 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c210 tx=0x7f90e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcedb1973e98ab47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec007d30 con 0x7f90e8003f20 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ec005ad0 con 0x7f90e8003f20 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f90fb7fe640 1 -- 172.21.15.40:0/4184358024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec007070 con 0x7f90e8003f20 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f910322a640 1 -- 172.21.15.40:0/4184358024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e8003f20 msgr2=0x7f90e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f910322a640 1 --2- 172.21.15.40:0/4184358024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e8003f20 0x7f90e8004800 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c210 tx=0x7f90e8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.848+0000 7f91017c7640 1 -- 172.21.15.40:0/4184358024 reap_dead start 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.849+0000 7f910322a640 1 -- 172.21.15.40:0/4184358024 shutdown_connections 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.849+0000 7f910322a640 1 -- 172.21.15.40:0/4184358024 >> 172.21.15.40:0/4184358024 conn(0x7f90fc0d0d30 msgr2=0x7f90fc0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.849+0000 7f910322a640 1 -- 172.21.15.40:0/4184358024 shutdown_connections 2024-04-26T21:21:40.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.849+0000 7f910322a640 1 -- 172.21.15.40:0/4184358024 wait complete. 2024-04-26T21:21:40.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.850+0000 7f910322a640 1 Processor -- start 2024-04-26T21:21:40.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.850+0000 7f910322a640 1 -- start start 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e8003f20 0x7f90fc0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc15b9c0 0x7f90fc1e5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0b3de0 con 0x7f90fc0cfea0 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0b5140 con 0x7f90fc15b9c0 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0b3f60 con 0x7f90e8003f20 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59644/0 (socket says 172.21.15.40:59644) 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 -- 172.21.15.40:0/1796646488 learned_addr learned my addr 172.21.15.40:0/1796646488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f9100fc6640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e8003f20 0x7f90fc0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f90fbfff640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc15b9c0 0x7f90fc1e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 -- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e8003f20 msgr2=0x7f90fc0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e8003f20 0x7f90fc0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 -- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc15b9c0 msgr2=0x7f90fc1e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc15b9c0 0x7f90fc1e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 -- 172.21.15.40:0/1796646488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc136e60 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f91017c7640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f90e4011860 tx=0x7f90e4011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4004240 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e401b070 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc1e61b0 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.851+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e400f740 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.852+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc0cf6a0 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.853+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc06a290 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.853+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90e4012040 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.854+0000 7f90f97fa640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 0x7f90dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.854+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f90e408b770 con 0x7f90fc0cfea0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.854+0000 7f9100fc6640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 0x7f90dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.855+0000 7f9100fc6640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 0x7f90dc06ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f90fc1371b0 tx=0x7f90f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:40.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:40.858+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90e4059550 con 0x7f90fc0cfea0 2024-04-26T21:21:41.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.044+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90fc15bda0 con 0x7f90fc0cfea0 2024-04-26T21:21:41.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.045+0000 7f90f97fa640 1 -- 172.21.15.40:0/1796646488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7f90e405d850 con 0x7f90fc0cfea0 2024-04-26T21:21:41.047 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:41.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 msgr2=0x7f90dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 0x7f90dc06ede0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f90fc1371b0 tx=0x7f90f4009040 comp rx=0 tx=0).stop 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 msgr2=0x7f90fc0b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f90e4011860 tx=0x7f90e4011d20 comp rx=0 tx=0).stop 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 shutdown_connections 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f90dc06c920 0x7f90dc06ede0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90fc0cfea0 0x7f90fc0b24c0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90fc15b9c0 0x7f90fc1e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 --2- 172.21.15.40:0/1796646488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e8003f20 0x7f90fc0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 >> 172.21.15.40:0/1796646488 conn(0x7f90fc0d0d30 msgr2=0x7f90fc144c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 shutdown_connections 2024-04-26T21:21:41.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.048+0000 7f910322a640 1 -- 172.21.15.40:0/1796646488 wait complete. 2024-04-26T21:21:41.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:21:28.644565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:21:29.151922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:41.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.164+0000 7feea89af640 1 Processor -- start 2024-04-26T21:21:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.164+0000 7feea89af640 1 -- start start 2024-04-26T21:21:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea89af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feea4161080 0x7feea4161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea89af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea41619d0 con 0x7feea415e7e0 2024-04-26T21:21:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea89af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40cdd40 con 0x7feea4161cb0 2024-04-26T21:21:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea89af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7feea40d2280 con 0x7feea4161080 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feea415e7e0 0x7feea40ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36332/0 (socket says 172.21.15.40:36332) 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea259c640 1 -- 172.21.15.40:0/3277971199 learned_addr learned my addr 172.21.15.40:0/3277971199 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3277971199 >> v2:172.21.15.184:3300/0 conn(0x7feea4161080 0x7feea4161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4157286977 0 0) 0x7fee94001110 con 0x7feea415e7e0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee90003520 con 0x7feea415e7e0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199582951 0 0) 0x7feea40cdd40 con 0x7feea4161cb0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee94001110 con 0x7feea4161cb0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2937706421 0 0) 0x7fee90003520 con 0x7feea415e7e0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 >> v2:172.21.15.184:3300/0 conn(0x7feea4161080 msgr2=0x7feea4161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 --2- 172.21.15.40:0/3277971199 >> v2:172.21.15.184:3300/0 conn(0x7feea4161080 0x7feea4161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 >> v1:172.21.15.184:6789/0 conn(0x7feea4161cb0 legacy=0x7feea40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40cdfe0 con 0x7feea415e7e0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fee94001840 con 0x7feea415e7e0 2024-04-26T21:21:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 >> v1:172.21.15.40:6789/0 conn(0x7feea415e7e0 legacy=0x7feea40ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee90003b40 0x7fee90003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feea40d2280 con 0x7fee900051c0 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee9000c0f0 con 0x7fee90004800 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.165+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fee9000c2f0 con 0x7fee90003b40 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea259c640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee90003b40 0x7fee90003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7fee9bfff640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 -- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee90003b40 msgr2=0x7fee90003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee90003b40 0x7fee90003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 -- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee90004800 msgr2=0x7fee90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 -- 172.21.15.40:0/3277971199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40cdfe0 con 0x7fee900051c0 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.166+0000 7feea2d9d640 1 -- 172.21.15.40:0/3277971199 reap_dead start 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 0x7fee9000b9c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009fc0 tx=0x7fee90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4fb8defaa7f957 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c035020 con 0x7fee900051c0 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c003bd0 con 0x7fee900051c0 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea1d9b640 1 -- 172.21.15.40:0/3277971199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c009c10 con 0x7fee900051c0 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 -- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 msgr2=0x7fee9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 0x7fee9000b9c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009fc0 tx=0x7fee90005830 comp rx=0 tx=0).stop 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 -- 172.21.15.40:0/3277971199 shutdown_connections 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 --2- 172.21.15.40:0/3277971199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 -- 172.21.15.40:0/3277971199 >> 172.21.15.40:0/3277971199 conn(0x7feea409ba40 msgr2=0x7feea409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 -- 172.21.15.40:0/3277971199 shutdown_connections 2024-04-26T21:21:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.167+0000 7feea89af640 1 -- 172.21.15.40:0/3277971199 wait complete. 2024-04-26T21:21:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 Processor -- start 2024-04-26T21:21:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 -- start start 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee900051c0 0x7feea414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feea415d870 0x7feea415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feea40ce880 con 0x7feea415fe50 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feea40ce5a0 con 0x7fee900051c0 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.169+0000 7feea89af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7feea4161490 con 0x7feea415d870 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59654/0 (socket says 172.21.15.40:59654) 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 -- 172.21.15.40:0/3028770479 learned_addr learned my addr 172.21.15.40:0/3028770479 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7fee9bfff640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feea415d870 0x7feea415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea259c640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee900051c0 0x7feea414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 -- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feea415d870 msgr2=0x7feea415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feea415d870 0x7feea415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 -- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee900051c0 msgr2=0x7feea414e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee900051c0 0x7feea414e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 -- 172.21.15.40:0/3028770479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea415be70 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea2d9d640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e230 tx=0x7fee9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c035030 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c043070 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea415c150 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02d070 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.170+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea415c710 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.171+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea406a290 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.172+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee9c005d10 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.172+0000 7fee9affd640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.172+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fee9c0ace60 con 0x7feea415fe50 2024-04-26T21:21:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.173+0000 7feea259c640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.173+0000 7feea259c640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 0x7fee8406ede0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fee940010e0 tx=0x7fee94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.176+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee9c07b3b0 con 0x7feea415fe50 2024-04-26T21:21:41.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.364+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feea4068e00 con 0x7feea415fe50 2024-04-26T21:21:41.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.364+0000 7fee9affd640 1 -- 172.21.15.40:0/3028770479 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fee9c07acb0 con 0x7feea415fe50 2024-04-26T21:21:41.365 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 msgr2=0x7fee8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 0x7fee8406ede0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fee940010e0 tx=0x7fee94009040 comp rx=0 tx=0).stop 2024-04-26T21:21:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 msgr2=0x7feea41602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e230 tx=0x7fee9c009750 comp rx=0 tx=0).stop 2024-04-26T21:21:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 shutdown_connections 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feea415fe50 0x7feea41602d0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feea415d870 0x7feea415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 --2- 172.21.15.40:0/3028770479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee900051c0 0x7feea414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 >> 172.21.15.40:0/3028770479 conn(0x7feea409ba40 msgr2=0x7feea40708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 shutdown_connections 2024-04-26T21:21:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.366+0000 7feea89af640 1 -- 172.21.15.40:0/3028770479 wait complete. 2024-04-26T21:21:41.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.481+0000 7fa0a0464640 1 Processor -- start 2024-04-26T21:21:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.481+0000 7fa0a0464640 1 -- start start 2024-04-26T21:21:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa0a0464640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa0a0464640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0980d0ff0 0x7fa0980d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa0a0464640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0980d4240 con 0x7fa0980d2120 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa0a0464640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa0980d2570 con 0x7fa0980d0ff0 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa0a0464640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa0980d31a0 con 0x7fa09815f490 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09ea01640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa0980d2120 0x7fa0980ba3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36344/0 (socket says 172.21.15.40:36344) 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09ea01640 1 -- 172.21.15.40:0/2045142276 learned_addr learned my addr 172.21.15.40:0/2045142276 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09e200640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/2045142276 >> v2:172.21.15.184:3300/0 conn(0x7fa0980d0ff0 0x7fa0980d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d1fe640 1 -- 172.21.15.40:0/2045142276 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336136650 0 0) 0x7fa0980d4240 con 0x7fa0980d2120 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d1fe640 1 -- 172.21.15.40:0/2045142276 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa084003520 con 0x7fa0980d2120 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d9ff640 1 -- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 msgr2=0x7fa0980d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d9ff640 1 -- 172.21.15.40:0/2045142276 >> v1:172.21.15.40:6789/0 conn(0x7fa0980d2120 legacy=0x7fa0980ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09d9ff640 1 -- 172.21.15.40:0/2045142276 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0980d3480 con 0x7fa0980d0ff0 2024-04-26T21:21:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.482+0000 7fa09e200640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.483+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d0ff0 0x7fa0980d1be0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0880097a0 tx=0x7fa08802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81ad9d1153c4729 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.483+0000 7fa09d1fe640 1 -- 172.21.15.40:0/2045142276 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088037070 con 0x7fa0980d0ff0 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.483+0000 7fa09d1fe640 1 -- 172.21.15.40:0/2045142276 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa088032440 con 0x7fa0980d0ff0 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.483+0000 7fa09d1fe640 1 -- 172.21.15.40:0/2045142276 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880327f0 con 0x7fa0980d0ff0 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 -- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d0ff0 msgr2=0x7fa0980d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d0ff0 0x7fa0980d1be0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0880097a0 tx=0x7fa08802a180 comp rx=0 tx=0).stop 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 -- 172.21.15.40:0/2045142276 shutdown_connections 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d0ff0 0x7fa0980d1be0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 --2- 172.21.15.40:0/2045142276 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 -- 172.21.15.40:0/2045142276 >> 172.21.15.40:0/2045142276 conn(0x7fa09809e530 msgr2=0x7fa0980d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 -- 172.21.15.40:0/2045142276 shutdown_connections 2024-04-26T21:21:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.484+0000 7fa0a0464640 1 -- 172.21.15.40:0/2045142276 wait complete. 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.485+0000 7fa0a0464640 1 Processor -- start 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 -- start start 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d2120 0x7fa0980b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0980d3d20 con 0x7fa0980d0ff0 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0980d3a40 con 0x7fa0980d2120 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa0a0464640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa09815f870 con 0x7fa09815f490 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa09e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa09e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59668/0 (socket says 172.21.15.40:59668) 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.486+0000 7fa09e200640 1 -- 172.21.15.40:0/268108193 learned_addr learned my addr 172.21.15.40:0/268108193 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d2120 0x7fa0980b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09ea01640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 -- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 msgr2=0x7fa0980caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 -- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d2120 msgr2=0x7fa0980b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d2120 0x7fa0980b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 -- 172.21.15.40:0/268108193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0980c9830 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09ea01640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa09e200640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa09400ebe0 tx=0x7fa09400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094017070 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09400cc30 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094013470 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0980b99d0 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.487+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0980b9f90 con 0x7fa0980d0ff0 2024-04-26T21:21:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.488+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09806a290 con 0x7fa0980d0ff0 2024-04-26T21:21:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.488+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa09401a070 con 0x7fa0980d0ff0 2024-04-26T21:21:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.489+0000 7fa08effd640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 0x7fa07c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.489+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fa09408ba70 con 0x7fa0980d0ff0 2024-04-26T21:21:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.489+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 0x7fa07c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.490+0000 7fa09d9ff640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 0x7fa07c06edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa088034000 tx=0x7fa08802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.493+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa094059f30 con 0x7fa0980d0ff0 2024-04-26T21:21:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.678+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa09806e6c0 con 0x7fa0980d0ff0 2024-04-26T21:21:41.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.679+0000 7fa08effd640 1 -- 172.21.15.40:0/268108193 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7fa094059830 con 0x7fa0980d0ff0 2024-04-26T21:21:41.680 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 msgr2=0x7fa07c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 0x7fa07c06edb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa088034000 tx=0x7fa08802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 msgr2=0x7fa0980b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa09400ebe0 tx=0x7fa09400c520 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 shutdown_connections 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa07c06c8f0 0x7fa07c06edb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa09815f490 0x7fa0980caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0980d2120 0x7fa0980b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 --2- 172.21.15.40:0/268108193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0980d0ff0 0x7fa0980b8e60 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 >> 172.21.15.40:0/268108193 conn(0x7fa09809e530 msgr2=0x7fa098075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 shutdown_connections 2024-04-26T21:21:41.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.681+0000 7fa0a0464640 1 -- 172.21.15.40:0/268108193 wait complete. 2024-04-26T21:21:41.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:41.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.795+0000 7f425cc08640 1 Processor -- start 2024-04-26T21:21:41.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 -- start start 2024-04-26T21:21:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42580d0010 0x7f425806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 0x7f42580b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 0x7f42580b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f42580b4480 con 0x7f42580d0010 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f425806e050 con 0x7f42580b3210 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425cc08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f425809c700 con 0x7f42580b3c20 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.796+0000 7f425659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42580d0010 0x7f425806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f42580d0010 0x7f425806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59674/0 (socket says 172.21.15.40:59674) 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 -- 172.21.15.40:0/1834847830 learned_addr learned my addr 172.21.15.40:0/1834847830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f424ffff640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 0x7f42580b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f4256d9d640 1 --2- 172.21.15.40:0/1834847830 >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 0x7f42580b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 -- 172.21.15.40:0/1834847830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 msgr2=0x7f42580b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 0x7f42580b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 -- 172.21.15.40:0/1834847830 >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 msgr2=0x7f42580b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 --2- 172.21.15.40:0/1834847830 >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 0x7f42580b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 -- 172.21.15.40:0/1834847830 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580d1c50 con 0x7f42580d0010 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f424ffff640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 0x7f42580b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f4256d9d640 1 --2- 172.21.15.40:0/1834847830 >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 0x7f42580b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.797+0000 7f425659c640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f425806db10 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f42580741b0 tx=0x7f424802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa72c081fde32ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f4255d9b640 1 -- 172.21.15.40:0/1834847830 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f42580d0010 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f4255d9b640 1 -- 172.21.15.40:0/1834847830 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4248032440 con 0x7f42580d0010 2024-04-26T21:21:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f4255d9b640 1 -- 172.21.15.40:0/1834847830 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248032810 con 0x7f42580d0010 2024-04-26T21:21:41.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 -- 172.21.15.40:0/1834847830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 msgr2=0x7f425806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f425806db10 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f42580741b0 tx=0x7f424802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 -- 172.21.15.40:0/1834847830 shutdown_connections 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 --2- 172.21.15.40:0/1834847830 >> v2:172.21.15.184:3300/0 conn(0x7f42580b3210 0x7f42580b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3c20 0x7f42580b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 --2- 172.21.15.40:0/1834847830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f425806db10 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 -- 172.21.15.40:0/1834847830 >> 172.21.15.40:0/1834847830 conn(0x7f425809ba40 msgr2=0x7f425809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 -- 172.21.15.40:0/1834847830 shutdown_connections 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.798+0000 7f425cc08640 1 -- 172.21.15.40:0/1834847830 wait complete. 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 Processor -- start 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 -- start start 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3210 0x7f42580ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 0x7f42580cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f425815fe90 con 0x7f42580d0010 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f425815fd10 con 0x7f42580b3c20 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f425cc08640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4258160010 con 0x7f42580b3210 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f4256d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f4256d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59678/0 (socket says 172.21.15.40:59678) 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.800+0000 7f4256d9d640 1 -- 172.21.15.40:0/3436776787 learned_addr learned my addr 172.21.15.40:0/3436776787 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f424ffff640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 0x7f42580cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f425659c640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3210 0x7f42580ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 -- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3210 msgr2=0x7f42580ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3210 0x7f42580ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 -- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 msgr2=0x7f42580cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 0x7f42580cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 -- 172.21.15.40:0/3436776787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580cd330 con 0x7f42580d0010 2024-04-26T21:21:41.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f424ffff640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 0x7f42580cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f4256d9d640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f4250009fa0 tx=0x7f425000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425001a070 con 0x7f42580d0010 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250003bb0 con 0x7f42580d0010 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42580cd610 con 0x7f42580d0010 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250004030 con 0x7f42580d0010 2024-04-26T21:21:41.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.801+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42580cdbd0 con 0x7f42580d0010 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.802+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425806a290 con 0x7f42580d0010 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.803+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f425001d070 con 0x7f42580d0010 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.803+0000 7f424effd640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 0x7f423806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.803+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f425008be50 con 0x7f42580d0010 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.803+0000 7f425659c640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 0x7f423806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:41.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.804+0000 7f425659c640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 0x7f423806edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.807+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4250010070 con 0x7f42580d0010 2024-04-26T21:21:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.991+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4258068e00 con 0x7f42580d0010 2024-04-26T21:21:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.992+0000 7f424effd640 1 -- 172.21.15.40:0/3436776787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7f425005a280 con 0x7f42580d0010 2024-04-26T21:21:41.993 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 msgr2=0x7f423806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 0x7f423806edb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 msgr2=0x7f42580b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f4250009fa0 tx=0x7f425000d560 comp rx=0 tx=0).stop 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 shutdown_connections 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f423806c8f0 0x7f423806edb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42580d0010 0x7f42580b02e0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42580b3c20 0x7f42580cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 --2- 172.21.15.40:0/3436776787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42580b3210 0x7f42580ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:41.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 >> 172.21.15.40:0/3436776787 conn(0x7f425809ba40 msgr2=0x7f42581603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:41.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 shutdown_connections 2024-04-26T21:21:41.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:41.994+0000 7f425cc08640 1 -- 172.21.15.40:0/3436776787 wait complete. 2024-04-26T21:21:42.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 Processor -- start 2024-04-26T21:21:42.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 -- start start 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f24145980 0x7f8f24160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b3de0 0x7f8f240b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f240b5fd0 0x7f8f2415c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8f240b6440 con 0x7f8f240b5fd0 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f8f240b5140 con 0x7f8f24145980 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.107+0000 7f8f2a37f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f240b4260 con 0x7f8f240b3de0 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f240b5fd0 0x7f8f2415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8f240b5fd0 0x7f8f2415c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59690/0 (socket says 172.21.15.40:59690) 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 -- 172.21.15.40:0/746785586 learned_addr learned my addr 172.21.15.40:0/746785586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f23fff640 1 --2- 172.21.15.40:0/746785586 >> v2:172.21.15.184:3300/0 conn(0x7f8f24145980 0x7f8f24160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f237fe640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b3de0 0x7f8f240b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 -- 172.21.15.40:0/746785586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b3de0 msgr2=0x7f8f240b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b3de0 0x7f8f240b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 -- 172.21.15.40:0/746785586 >> v2:172.21.15.184:3300/0 conn(0x7f8f24145980 msgr2=0x7f8f24160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 --2- 172.21.15.40:0/746785586 >> v2:172.21.15.184:3300/0 conn(0x7f8f24145980 0x7f8f24160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.108+0000 7f8f2891c640 1 -- 172.21.15.40:0/746785586 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f240b4530 con 0x7f8f240b5fd0 2024-04-26T21:21:42.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2891c640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f240b5fd0 0x7f8f2415c510 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406ff10 tx=0x7f8f1c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c793510e1777c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f22ffd640 1 -- 172.21.15.40:0/746785586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c036070 con 0x7f8f240b5fd0 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f22ffd640 1 -- 172.21.15.40:0/746785586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c031600 con 0x7f8f240b5fd0 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f22ffd640 1 -- 172.21.15.40:0/746785586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0319d0 con 0x7f8f240b5fd0 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 -- 172.21.15.40:0/746785586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f240b5fd0 msgr2=0x7f8f2415c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f240b5fd0 0x7f8f2415c510 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406ff10 tx=0x7f8f1c02a610 comp rx=0 tx=0).stop 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 -- 172.21.15.40:0/746785586 shutdown_connections 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f240b5fd0 0x7f8f2415c510 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/746785586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b3de0 0x7f8f240b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/746785586 >> v2:172.21.15.184:3300/0 conn(0x7f8f24145980 0x7f8f24160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 -- 172.21.15.40:0/746785586 >> 172.21.15.40:0/746785586 conn(0x7f8f240d0d30 msgr2=0x7f8f240d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 -- 172.21.15.40:0/746785586 shutdown_connections 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.109+0000 7f8f2a37f640 1 -- 172.21.15.40:0/746785586 wait complete. 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.110+0000 7f8f2a37f640 1 Processor -- start 2024-04-26T21:21:42.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 -- start start 2024-04-26T21:21:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f240b3de0 0x7f8f2415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b5fd0 0x7f8f2415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f240b4dd0 con 0x7f8f24145980 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f240b4af0 con 0x7f8f240b3de0 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.111+0000 7f8f2a37f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2415ca50 con 0x7f8f240b5fd0 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59692/0 (socket says 172.21.15.40:59692) 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 -- 172.21.15.40:0/3740191067 learned_addr learned my addr 172.21.15.40:0/3740191067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:42.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f23fff640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f240b3de0 0x7f8f2415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f237fe640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b5fd0 0x7f8f2415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 -- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b5fd0 msgr2=0x7f8f2415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b5fd0 0x7f8f2415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 -- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f240b3de0 msgr2=0x7f8f2415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f240b3de0 0x7f8f2415b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 -- 172.21.15.40:0/3740191067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2415e630 con 0x7f8f24145980 2024-04-26T21:21:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2891c640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009f40 tx=0x7f8f1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c036050 con 0x7f8f24145980 2024-04-26T21:21:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c042070 con 0x7f8f24145980 2024-04-26T21:21:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2415e910 con 0x7f8f24145980 2024-04-26T21:21:42.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c009510 con 0x7f8f24145980 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.112+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2415eed0 con 0x7f8f24145980 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.113+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2406a290 con 0x7f8f24145980 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.114+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f1c007bf0 con 0x7f8f24145980 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.114+0000 7f8f20ff9640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 0x7f8f1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.114+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f8f1c0ac8b0 con 0x7f8f24145980 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.114+0000 7f8f23fff640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 0x7f8f1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.115+0000 7f8f23fff640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 0x7f8f1006edb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8f241f8690 tx=0x7f8f14009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.118+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f1c07acd0 con 0x7f8f24145980 2024-04-26T21:21:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.315+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f24073cc0 con 0x7f8f24145980 2024-04-26T21:21:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.315+0000 7f8f20ff9640 1 -- 172.21.15.40:0/3740191067 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f8f1c07a5d0 con 0x7f8f24145980 2024-04-26T21:21:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 msgr2=0x7f8f1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 0x7f8f1006edb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8f241f8690 tx=0x7f8f14009190 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 msgr2=0x7f8f2413f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009f40 tx=0x7f8f1c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 shutdown_connections 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f8f1006c8f0 0x7f8f1006edb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f24145980 0x7f8f2413f790 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f240b5fd0 0x7f8f2415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.317+0000 7f8f2a37f640 1 --2- 172.21.15.40:0/3740191067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f240b3de0 0x7f8f2415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.318+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 >> 172.21.15.40:0/3740191067 conn(0x7f8f240d0d30 msgr2=0x7f8f240b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.318+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 shutdown_connections 2024-04-26T21:21:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.318+0000 7f8f2a37f640 1 -- 172.21.15.40:0/3740191067 wait complete. 2024-04-26T21:21:42.329 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:21:42.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:42.330 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T21:21:42.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:21:42.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:42.362 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:21:42.420 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:42.421 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:42.421 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:42.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:42.437 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:21:42.437 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:42.437 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:42.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:42.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.576+0000 7fab73efc640 1 Processor -- start 2024-04-26T21:21:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 -- start start 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab6c0d3ee0 0x7fab6c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab6c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6c0d03a0 con 0x7fab6c15f370 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fab6c0b4d10 con 0x7fab6c0d3ee0 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.577+0000 7fab73efc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab6c0d0680 con 0x7fab6c15f750 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab71c98640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fab6c15f370 0x7fab6c0d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36358/0 (socket says 172.21.15.40:36358) 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab71c98640 1 -- 172.21.15.40:0/839893545 learned_addr learned my addr 172.21.15.40:0/839893545 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab71497640 1 --2- 172.21.15.40:0/839893545 >> v2:172.21.15.184:3300/0 conn(0x7fab6c0d3ee0 0x7fab6c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab72499640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab6c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3563137848 0 0) 0x7fab6c0d03a0 con 0x7fab6c15f370 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab58003520 con 0x7fab6c15f370 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2344079829 0 0) 0x7fab58003520 con 0x7fab6c15f370 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 msgr2=0x7fab6c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab6c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 >> v2:172.21.15.184:3300/0 conn(0x7fab6c0d3ee0 msgr2=0x7fab6c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 --2- 172.21.15.40:0/839893545 >> v2:172.21.15.184:3300/0 conn(0x7fab6c0d3ee0 0x7fab6c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c0d0960 con 0x7fab6c15f370 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fab68001670 con 0x7fab6c15f370 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 >> v1:172.21.15.40:6789/0 conn(0x7fab6c15f370 legacy=0x7fab6c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab580039b0 0x7fab58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58004800 0x7fab58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c0b4d10 con 0x7fab58004800 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c0d0680 con 0x7fab580039b0 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab5800b9c0 con 0x7fab6c15f750 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab71497640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58004800 0x7fab58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab71c98640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab580039b0 0x7fab58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.578+0000 7fab72499640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 -- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 msgr2=0x7fab580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c15f750 0x7fab580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 -- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab580039b0 msgr2=0x7fab58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab580039b0 0x7fab58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 -- 172.21.15.40:0/839893545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab58003520 con 0x7fab58004800 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab71497640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58004800 0x7fab58004c80 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fab6c08e680 tx=0x7fab58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd785cb35c3e7c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c036070 con 0x7fab58004800 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c031400 con 0x7fab58004800 2024-04-26T21:21:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.579+0000 7fab70c96640 1 -- 172.21.15.40:0/839893545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c009690 con 0x7fab58004800 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 -- 172.21.15.40:0/839893545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58004800 msgr2=0x7fab58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 --2- 172.21.15.40:0/839893545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58004800 0x7fab58004c80 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fab6c08e680 tx=0x7fab58005830 comp rx=0 tx=0).stop 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab72499640 1 -- 172.21.15.40:0/839893545 reap_dead start 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 -- 172.21.15.40:0/839893545 shutdown_connections 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 -- 172.21.15.40:0/839893545 >> 172.21.15.40:0/839893545 conn(0x7fab6c08ec10 msgr2=0x7fab6c09e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 -- 172.21.15.40:0/839893545 shutdown_connections 2024-04-26T21:21:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.580+0000 7fab73efc640 1 -- 172.21.15.40:0/839893545 wait complete. 2024-04-26T21:21:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.581+0000 7fab73efc640 1 Processor -- start 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.581+0000 7fab73efc640 1 -- start start 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58004800 0x7fab6c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c0b9fc0 0x7fab6c0ba440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c0b47d0 con 0x7fab6c15d910 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c0d0f20 con 0x7fab58004800 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab73efc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fab6c0b4950 con 0x7fab6c0b9fc0 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71497640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71497640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59700/0 (socket says 172.21.15.40:59700) 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71497640 1 -- 172.21.15.40:0/1340950991 learned_addr learned my addr 172.21.15.40:0/1340950991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab72499640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c0b9fc0 0x7fab6c0ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71c98640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58004800 0x7fab6c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71497640 1 -- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c0b9fc0 msgr2=0x7fab6c0ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.582+0000 7fab71497640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c0b9fc0 0x7fab6c0ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab71497640 1 -- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58004800 msgr2=0x7fab6c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab71497640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58004800 0x7fab6c15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab71497640 1 -- 172.21.15.40:0/1340950991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c15d670 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab71497640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fab5c034040 tx=0x7fab5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c041020 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c03c470 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c036040 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c0ce180 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.583+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c0ce740 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.584+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6c06a290 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.584+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab5c044020 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.585+0000 7fab627fc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 0x7fab5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.585+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fab5c0ac130 con 0x7fab6c15d910 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.585+0000 7fab71c98640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 0x7fab5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.586+0000 7fab71c98640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 0x7fab5006ede0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fab680015d0 tx=0x7fab680078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.589+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab5c04b030 con 0x7fab6c15d910 2024-04-26T21:21:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.813+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab6c075fb0 con 0x7fab6c15d910 2024-04-26T21:21:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.814+0000 7fab627fc640 1 -- 172.21.15.40:0/1340950991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7fab5c079fa0 con 0x7fab6c15d910 2024-04-26T21:21:42.815 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:42.815 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 msgr2=0x7fab5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 0x7fab5006ede0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fab680015d0 tx=0x7fab680078f0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 msgr2=0x7fab6c0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fab5c034040 tx=0x7fab5c004730 comp rx=0 tx=0).stop 2024-04-26T21:21:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 shutdown_connections 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fab5006c920 0x7fab5006ede0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab6c0b9fc0 0x7fab6c0ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6c15d910 0x7fab6c0cd880 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 --2- 172.21.15.40:0/1340950991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab58004800 0x7fab6c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 >> 172.21.15.40:0/1340950991 conn(0x7fab6c08ec10 msgr2=0x7fab6c0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 shutdown_connections 2024-04-26T21:21:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.816+0000 7fab73efc640 1 -- 172.21.15.40:0/1340950991 wait complete. 2024-04-26T21:21:42.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.928+0000 7f81daa23640 1 Processor -- start 2024-04-26T21:21:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81daa23640 1 -- start start 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81daa23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81d40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81daa23640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d40b4530 con 0x7f81d4145980 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81daa23640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d40b4810 con 0x7f81d40b61a0 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81daa23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81d4160ac0 con 0x7f81d4160630 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.929+0000 7f81d3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81d4145980 0x7f81d40b5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36368/0 (socket says 172.21.15.40:36368) 2024-04-26T21:21:42.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d3fff640 1 -- 172.21.15.40:0/1553572642 learned_addr learned my addr 172.21.15.40:0/1553572642 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081288815 0 0) 0x7f81d40b4530 con 0x7f81d4145980 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81d40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81bc003520 con 0x7f81d4145980 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4075269889 0 0) 0x7f81bc003520 con 0x7f81d4145980 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 msgr2=0x7f81d40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81d40b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 >> v1:172.21.15.184:6789/0 conn(0x7f81d40b61a0 legacy=0x7f81d40720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d40b4af0 con 0x7f81d4145980 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f81c4001690 con 0x7f81d4145980 2024-04-26T21:21:42.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 >> v1:172.21.15.40:6789/0 conn(0x7f81d4145980 legacy=0x7f81d40b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81bc003f20 0x7f81bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc004d40 0x7f81bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc00dcb0 con 0x7f81bc003f20 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc00de60 con 0x7f81bc004d40 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81bc00e060 con 0x7f81d4160630 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d37fe640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81bc003f20 0x7f81bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d3fff640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.930+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc004d40 0x7f81bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 -- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 msgr2=0x7f81bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d4160630 0x7f81bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 -- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc004d40 msgr2=0x7f81bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc004d40 0x7f81bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 -- 172.21.15.40:0/1553572642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc003520 con 0x7f81bc003f20 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc004d40 0x7f81bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d37fe640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81bc003f20 0x7f81bc004800 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001860 tx=0x7f81bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2821d08ee7b6805 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8037070 con 0x7f81bc003f20 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c802abf0 con 0x7f81bc003f20 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.931+0000 7f81d2ffd640 1 -- 172.21.15.40:0/1553572642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8001430 con 0x7f81bc003f20 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 -- 172.21.15.40:0/1553572642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81bc003f20 msgr2=0x7f81bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 --2- 172.21.15.40:0/1553572642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81bc003f20 0x7f81bc004800 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001860 tx=0x7f81bc005830 comp rx=0 tx=0).stop 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81d8fc0640 1 -- 172.21.15.40:0/1553572642 reap_dead start 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 -- 172.21.15.40:0/1553572642 shutdown_connections 2024-04-26T21:21:42.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 -- 172.21.15.40:0/1553572642 >> 172.21.15.40:0/1553572642 conn(0x7f81d40d0d30 msgr2=0x7f81d40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:42.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 -- 172.21.15.40:0/1553572642 shutdown_connections 2024-04-26T21:21:42.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.932+0000 7f81daa23640 1 -- 172.21.15.40:0/1553572642 wait complete. 2024-04-26T21:21:42.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.933+0000 7f81daa23640 1 Processor -- start 2024-04-26T21:21:42.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.933+0000 7f81daa23640 1 -- start start 2024-04-26T21:21:42.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 0x7f81d40afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 0x7f81d415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81d40b3de0 con 0x7f81d41f0bf0 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81d40b5140 con 0x7f81bc003f20 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81daa23640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f81d40b3f60 con 0x7f81d415ed30 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81d8fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81d8fc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59712/0 (socket says 172.21.15.40:59712) 2024-04-26T21:21:42.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81d8fc0640 1 -- 172.21.15.40:0/2530094546 learned_addr learned my addr 172.21.15.40:0/2530094546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81d3fff640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 0x7f81d40afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.934+0000 7f81d37fe640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 0x7f81d415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 -- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 msgr2=0x7f81d415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 0x7f81d415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 -- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 msgr2=0x7f81d40afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 0x7f81d40afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 -- 172.21.15.40:0/2530094546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d40b2300 con 0x7f81d41f0bf0 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d3fff640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 0x7f81d40afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d37fe640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 0x7f81d415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d8fc0640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f81d40b6fc0 tx=0x7f81cc010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc010e60 con 0x7f81d41f0bf0 2024-04-26T21:21:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81cc003990 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc003da0 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d40b25e0 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.935+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d40b2ba0 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.936+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d406a290 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.936+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81cc005d10 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.937+0000 7f81d0ff9640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 0x7f81c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.937+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f81cc08c260 con 0x7f81d41f0bf0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.937+0000 7f81d3fff640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 0x7f81c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.938+0000 7f81d3fff640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 0x7f81c006ede0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f81d415fdc0 tx=0x7f81c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:42.941+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81cc025020 con 0x7f81d41f0bf0 2024-04-26T21:21:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.128+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81d4073cc0 con 0x7f81d41f0bf0 2024-04-26T21:21:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.129+0000 7f81d0ff9640 1 -- 172.21.15.40:0/2530094546 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7f81cc05a690 con 0x7f81d41f0bf0 2024-04-26T21:21:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:43.130 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 msgr2=0x7f81c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 0x7f81c006ede0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f81d415fdc0 tx=0x7f81c4009040 comp rx=0 tx=0).stop 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 msgr2=0x7f81d41f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f81d40b6fc0 tx=0x7f81cc010590 comp rx=0 tx=0).stop 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 shutdown_connections 2024-04-26T21:21:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f81c006c920 0x7f81c006ede0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d41f0bf0 0x7f81d41f10a0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81d415ed30 0x7f81d415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 --2- 172.21.15.40:0/2530094546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81bc003f20 0x7f81d40afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 >> 172.21.15.40:0/2530094546 conn(0x7f81d40d0d30 msgr2=0x7f81d4071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 shutdown_connections 2024-04-26T21:21:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.131+0000 7f81daa23640 1 -- 172.21.15.40:0/2530094546 wait complete. 2024-04-26T21:21:43.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:11:48.932717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:21:28.644565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:21:29.151922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:43.144 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.247+0000 7f91e3a2a640 1 Processor -- start 2024-04-26T21:21:43.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.247+0000 7f91e3a2a640 1 -- start start 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e3a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc0d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e3a2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e3a2a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91dc0d4240 con 0x7f91dc15f490 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e3a2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91dc0d2580 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e3a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0d31a0 con 0x7f91dc0d1be0 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59714/0 (socket says 172.21.15.40:59714) 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e17c6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91dc15f490 0x7f91dc0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36372/0 (socket says 172.21.15.40:36372) 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 -- 172.21.15.40:0/1202125350 learned_addr learned my addr 172.21.15.40:0/1202125350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 -- 172.21.15.40:0/1202125350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 msgr2=0x7f91dc0d2040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:21:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 --2- 172.21.15.40:0/1202125350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc0d2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 -- 172.21.15.40:0/1202125350 >> v1:172.21.15.40:6789/0 conn(0x7f91dc15f490 legacy=0x7f91dc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91e1fc7640 1 -- 172.21.15.40:0/1202125350 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc0d3480 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.248+0000 7f91d3fff640 1 -- 172.21.15.40:0/1202125350 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021584190 0 0) 0x7f91dc0d4240 con 0x7f91dc15f490 2024-04-26T21:21:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e1fc7640 1 --2- 172.21.15.40:0/1202125350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009e50 tx=0x7f91d402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31fc1a6474996ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91d3fff640 1 -- 172.21.15.40:0/1202125350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4036070 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91d3fff640 1 -- 172.21.15.40:0/1202125350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d4031600 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91d3fff640 1 -- 172.21.15.40:0/1202125350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40319d0 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1202125350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 msgr2=0x7f91dc0ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1202125350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009e50 tx=0x7f91d402a730 comp rx=0 tx=0).stop 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1202125350 shutdown_connections 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1202125350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc0ba7f0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1202125350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc0d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.249+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1202125350 >> 172.21.15.40:0/1202125350 conn(0x7f91dc09e530 msgr2=0x7f91dc0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.250+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1202125350 shutdown_connections 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.250+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1202125350 wait complete. 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 Processor -- start 2024-04-26T21:21:43.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 -- start start 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc1e98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 0x7f91dc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0d3d20 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0d3a40 con 0x7f91dc15f490 2024-04-26T21:21:43.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.251+0000 7f91e3a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0d11c0 con 0x7f91dc0d1be0 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59730/0 (socket says 172.21.15.40:59730) 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 -- 172.21.15.40:0/1920615844 learned_addr learned my addr 172.21.15.40:0/1920615844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e1fc7640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 0x7f91dc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e0fc5640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc1e98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 -- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 msgr2=0x7f91dc1e98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc1e98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 -- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 msgr2=0x7f91dc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 0x7f91dc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 -- 172.21.15.40:0/1920615844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1583b0 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e1fc7640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 0x7f91dc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e0fc5640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc1e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:43.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e17c6640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c460 tx=0x7f91d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8018070 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8007500 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc158690 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.252+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8004a20 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.253+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc156220 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.253+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc069ae0 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.254+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d801b070 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.254+0000 7f91d1ffb640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 0x7f91c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.255+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f91d808ae50 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.255+0000 7f91e0fc5640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 0x7f91c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.255+0000 7f91e0fc5640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 0x7f91c006ede0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f91dc1f8b80 tx=0x7f91cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.259+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91dc069ae0 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.446+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91dc0d6030 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.446+0000 7f91d1ffb640 1 -- 172.21.15.40:0/1920615844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7f91dc0d6030 con 0x7f91dc0ba3a0 2024-04-26T21:21:43.448 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.448+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 msgr2=0x7f91c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.448+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 0x7f91c006ede0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f91dc1f8b80 tx=0x7f91cc008040 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 msgr2=0x7f91dc155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c460 tx=0x7f91d800c920 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 shutdown_connections 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f91c006c920 0x7f91c006ede0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc15f490 0x7f91dc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc0d1be0 0x7f91dc1e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 --2- 172.21.15.40:0/1920615844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91dc0ba3a0 0x7f91dc155ce0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 >> 172.21.15.40:0/1920615844 conn(0x7f91dc09e530 msgr2=0x7f91dc0d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 shutdown_connections 2024-04-26T21:21:43.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.449+0000 7f91e3a2a640 1 -- 172.21.15.40:0/1920615844 wait complete. 2024-04-26T21:21:43.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:43.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 Processor -- start 2024-04-26T21:21:43.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 -- start start 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 0x7f1c9c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 0x7f1c9c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9c15dca0 con 0x7f1c9c0b4520 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c143c20 con 0x7f1c9c143840 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.564+0000 7f1ca15c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c15df80 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1c9c0b4520 0x7f1c9c0b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36374/0 (socket says 172.21.15.40:36374) 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9a7fc640 1 -- 172.21.15.40:0/3407579930 learned_addr learned my addr 172.21.15.40:0/3407579930 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3407579930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 0x7f1c9c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9affd640 1 --2- 172.21.15.40:0/3407579930 >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 0x7f1c9c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116664720 0 0) 0x7f1c9c15dca0 con 0x7f1c9c0b4520 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c84003520 con 0x7f1c9c0b4520 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3407579930 >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 msgr2=0x7f1c9c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3407579930 >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 0x7f1c9c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33549207 0 0) 0x7f1c84003520 con 0x7f1c9c0b4520 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3407579930 >> v1:172.21.15.40:6789/0 conn(0x7f1c9c0b4520 legacy=0x7f1c9c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3407579930 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c15e260 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.565+0000 7f1c9affd640 1 --2- 172.21.15.40:0/3407579930 >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 0x7f1c9c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.566+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3407579930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 0x7f1c9c0ceff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed57602a5190dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.566+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c037070 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.566+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8c02aba0 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.566+0000 7f1c99ffb640 1 -- 172.21.15.40:0/3407579930 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90001670 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3407579930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 msgr2=0x7f1c9c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3407579930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 0x7f1c9c0ceff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c02a110 comp rx=0 tx=0).stop 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3407579930 shutdown_connections 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3407579930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b55c0 0x7f1c9c0ceff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3407579930 >> v2:172.21.15.184:3300/0 conn(0x7f1c9c143840 0x7f1c9c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3407579930 >> 172.21.15.40:0/3407579930 conn(0x7f1c9c0cd8a0 msgr2=0x7f1c9c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3407579930 shutdown_connections 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.567+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3407579930 wait complete. 2024-04-26T21:21:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.568+0000 7f1ca15c6640 1 Processor -- start 2024-04-26T21:21:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.568+0000 7f1ca15c6640 1 -- start start 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b4520 0x7f1c9c139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c9c0b55c0 0x7f1c9c143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c15da40 con 0x7f1c9c143840 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c15d8c0 con 0x7f1c9c0b55c0 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1ca15c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0cf530 con 0x7f1c9c0b4520 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59738/0 (socket says 172.21.15.40:59738) 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3390659664 learned_addr learned my addr 172.21.15.40:0/3390659664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9affd640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b4520 0x7f1c9c139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9a7fc640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c9c0b55c0 0x7f1c9c143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b4520 msgr2=0x7f1c9c139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b4520 0x7f1c9c139610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c9c0b55c0 msgr2=0x7f1c9c143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c9c0b55c0 0x7f1c9c143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 -- 172.21.15.40:0/3390659664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c073f30 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.569+0000 7f1c9b7fe640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.570+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c041020 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.570+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8c03d620 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.570+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c146690 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.570+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c037040 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.570+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c139b50 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.571+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c06a290 con 0x7f1c9c143840 2024-04-26T21:21:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.571+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c8c044020 con 0x7f1c9c143840 2024-04-26T21:21:43.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.571+0000 7f1c837fe640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 0x7f1c8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.572+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f1c8c0ac000 con 0x7f1c9c143840 2024-04-26T21:21:43.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.572+0000 7f1c9affd640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 0x7f1c8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.572+0000 7f1c9affd640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 0x7f1c8806ede0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c94007440 tx=0x7f1c9400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.576+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c8c04b030 con 0x7f1c9c143840 2024-04-26T21:21:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.760+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c9c0ce420 con 0x7f1c9c143840 2024-04-26T21:21:43.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.760+0000 7f1c837fe640 1 -- 172.21.15.40:0/3390659664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) ==== 104+0+11 (secure 0 0 0) 0x7f1c8c07a430 con 0x7f1c9c143840 2024-04-26T21:21:43.761 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 msgr2=0x7f1c8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 0x7f1c8806ede0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1c94007440 tx=0x7f1c9400b040 comp rx=0 tx=0).stop 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 msgr2=0x7f1c9c0af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0097a0 tx=0x7f1c8c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 shutdown_connections 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1c8806c920 0x7f1c8806ede0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9c143840 0x7f1c9c0af990 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c9c0b55c0 0x7f1c9c143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 --2- 172.21.15.40:0/3390659664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c9c0b4520 0x7f1c9c139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 >> 172.21.15.40:0/3390659664 conn(0x7f1c9c0cd8a0 msgr2=0x7f1c9c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 shutdown_connections 2024-04-26T21:21:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.762+0000 7f1ca15c6640 1 -- 172.21.15.40:0/3390659664 wait complete. 2024-04-26T21:21:43.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 Processor -- start 2024-04-26T21:21:43.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 -- start start 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 0x7ff4e00d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4e00d0f40 0x7ff4e00d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e00b6aa0 con 0x7ff4e015f5b0 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e00d3eb0 con 0x7ff4e00d0f40 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.878+0000 7ff4e5499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e00d4190 con 0x7ff4e015f990 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4deffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff4e015f5b0 0x7ff4e00b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36378/0 (socket says 172.21.15.40:36378) 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4deffd640 1 -- 172.21.15.40:0/2571500946 learned_addr learned my addr 172.21.15.40:0/2571500946 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4df7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4e00d0f40 0x7ff4e00d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4de7fc640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 0x7ff4e00d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4df7fe640 1 -- 172.21.15.40:0/2571500946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 msgr2=0x7ff4e00d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4df7fe640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 0x7ff4e00d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4df7fe640 1 -- 172.21.15.40:0/2571500946 >> v1:172.21.15.40:6789/0 conn(0x7ff4e015f5b0 legacy=0x7ff4e00b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4df7fe640 1 -- 172.21.15.40:0/2571500946 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e00d0200 con 0x7ff4e00d0f40 2024-04-26T21:21:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.879+0000 7ff4de7fc640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 0x7ff4e00d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4df7fe640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e00d0f40 0x7ff4e00d3340 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009d90 tx=0x7ff4d802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71040472b9719cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4ddffb640 1 -- 172.21.15.40:0/2571500946 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8007740 con 0x7ff4e00d0f40 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4ddffb640 1 -- 172.21.15.40:0/2571500946 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d80078e0 con 0x7ff4e00d0f40 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4ddffb640 1 -- 172.21.15.40:0/2571500946 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8007cb0 con 0x7ff4e00d0f40 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 -- 172.21.15.40:0/2571500946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e00d0f40 msgr2=0x7ff4e00d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e00d0f40 0x7ff4e00d3340 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009d90 tx=0x7ff4d802a530 comp rx=0 tx=0).stop 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 -- 172.21.15.40:0/2571500946 shutdown_connections 2024-04-26T21:21:43.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e00d0f40 0x7ff4e00d3340 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 --2- 172.21.15.40:0/2571500946 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e015f990 0x7ff4e00d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 -- 172.21.15.40:0/2571500946 >> 172.21.15.40:0/2571500946 conn(0x7ff4e008ec40 msgr2=0x7ff4e009e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 -- 172.21.15.40:0/2571500946 shutdown_connections 2024-04-26T21:21:43.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.880+0000 7ff4e5499640 1 -- 172.21.15.40:0/2571500946 wait complete. 2024-04-26T21:21:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 Processor -- start 2024-04-26T21:21:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 -- start start 2024-04-26T21:21:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e00d0f40 0x7ff4e0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e015f5b0 0x7ff4e00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e00b5900 con 0x7ff4e015f990 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e00b5780 con 0x7ff4e015f5b0 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4e5499640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e00d3880 con 0x7ff4e00d0f40 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59754/0 (socket says 172.21.15.40:59754) 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.882+0000 7ff4df7fe640 1 -- 172.21.15.40:0/3254520493 learned_addr learned my addr 172.21.15.40:0/3254520493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4deffd640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e00d0f40 0x7ff4e0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4de7fc640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e015f5b0 0x7ff4e00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 -- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e00d0f40 msgr2=0x7ff4e0153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e00d0f40 0x7ff4e0153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 -- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e015f5b0 msgr2=0x7ff4e00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e015f5b0 0x7ff4e00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 -- 172.21.15.40:0/3254520493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e00cbae0 con 0x7ff4e015f990 2024-04-26T21:21:43.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4df7fe640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009eb0 tx=0x7ff4d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8040070 con 0x7ff4e015f990 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8031c90 con 0x7ff4e015f990 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e00cc3a0 con 0x7ff4e015f990 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8045680 con 0x7ff4e015f990 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.883+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e00cc960 con 0x7ff4e015f990 2024-04-26T21:21:43.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.884+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e006a290 con 0x7ff4e015f990 2024-04-26T21:21:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.885+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4d8039040 con 0x7ff4e015f990 2024-04-26T21:21:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.885+0000 7ff4c77fe640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.885+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7ff4d80ad480 con 0x7ff4e015f990 2024-04-26T21:21:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.885+0000 7ff4deffd640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.886+0000 7ff4deffd640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 0x7ff4cc06ede0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0000f60 tx=0x7ff4d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:43.890+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4d8043360 con 0x7ff4e015f990 2024-04-26T21:21:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.075+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4e01636c0 con 0x7ff4e015f990 2024-04-26T21:21:44.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.075+0000 7ff4c77fe640 1 -- 172.21.15.40:0/3254520493 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) ==== 100+0+11 (secure 0 0 0) 0x7ff4d807ba00 con 0x7ff4e015f990 2024-04-26T21:21:44.076 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 msgr2=0x7ff4cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 0x7ff4cc06ede0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0000f60 tx=0x7ff4d0009040 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 msgr2=0x7ff4e00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009eb0 tx=0x7ff4d8004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 shutdown_connections 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e015f990 0x7ff4e00d22e0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e015f5b0 0x7ff4e00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 --2- 172.21.15.40:0/3254520493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff4e00d0f40 0x7ff4e0153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.077+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 >> 172.21.15.40:0/3254520493 conn(0x7ff4e008ec40 msgr2=0x7ff4e006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.078+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 shutdown_connections 2024-04-26T21:21:44.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.078+0000 7ff4e5499640 1 -- 172.21.15.40:0/3254520493 wait complete. 2024-04-26T21:21:44.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.189+0000 7f2653d4e640 1 Processor -- start 2024-04-26T21:21:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.189+0000 7f2653d4e640 1 -- start start 2024-04-26T21:21:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f2653d4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f264c06e490 0x7f264c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f2653d4e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0d1740 con 0x7f264c0d11b0 2024-04-26T21:21:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f2653d4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0d1a20 con 0x7f264c0cf2e0 2024-04-26T21:21:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f2653d4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f264c0d1d00 con 0x7f264c06e490 2024-04-26T21:21:44.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26512e9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f264c0d11b0 0x7f264c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36394/0 (socket says 172.21.15.40:36394) 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26522eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f264c06e490 0x7f264c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26512e9640 1 -- 172.21.15.40:0/1696589374 learned_addr learned my addr 172.21.15.40:0/1696589374 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26522eb640 1 -- 172.21.15.40:0/1696589374 >> v1:172.21.15.184:6789/0 conn(0x7f264c0cf2e0 legacy=0x7f264c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f2650ae8640 1 -- 172.21.15.40:0/1696589374 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563766902 0 0) 0x7f264c0d1740 con 0x7f264c0d11b0 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26522eb640 1 -- 172.21.15.40:0/1696589374 >> v1:172.21.15.40:6789/0 conn(0x7f264c0d11b0 legacy=0x7f264c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.190+0000 7f26522eb640 1 -- 172.21.15.40:0/1696589374 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0d1fe0 con 0x7f264c06e490 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.191+0000 7f26522eb640 1 --2- 172.21.15.40:0/1696589374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c15ebe0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2644009dc0 tx=0x7f264402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd30d2432bb82144 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.191+0000 7f2650ae8640 1 -- 172.21.15.40:0/1696589374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644007740 con 0x7f264c06e490 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.191+0000 7f2650ae8640 1 -- 172.21.15.40:0/1696589374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26440078e0 con 0x7f264c06e490 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.191+0000 7f2650ae8640 1 -- 172.21.15.40:0/1696589374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644007cb0 con 0x7f264c06e490 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 -- 172.21.15.40:0/1696589374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 msgr2=0x7f264c15ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 --2- 172.21.15.40:0/1696589374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c15ebe0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2644009dc0 tx=0x7f264402a730 comp rx=0 tx=0).stop 2024-04-26T21:21:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 -- 172.21.15.40:0/1696589374 shutdown_connections 2024-04-26T21:21:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 --2- 172.21.15.40:0/1696589374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c15ebe0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 -- 172.21.15.40:0/1696589374 >> 172.21.15.40:0/1696589374 conn(0x7f264c0b4a60 msgr2=0x7f264c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 -- 172.21.15.40:0/1696589374 shutdown_connections 2024-04-26T21:21:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.192+0000 7f2653d4e640 1 -- 172.21.15.40:0/1696589374 wait complete. 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.193+0000 7f2653d4e640 1 Processor -- start 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.193+0000 7f2653d4e640 1 -- start start 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 0x7f264c153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 0x7f264c0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cf840 con 0x7f264c06e490 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cf6c0 con 0x7f264c0cf2e0 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0ce660 con 0x7f264c0d11b0 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59782/0 (socket says 172.21.15.40:59782) 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 -- 172.21.15.40:0/3904843417 learned_addr learned my addr 172.21.15.40:0/3904843417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26522eb640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 0x7f264c0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 -- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 msgr2=0x7f264c0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26512e9640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 0x7f264c153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 0x7f264c0c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 -- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 msgr2=0x7f264c153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 0x7f264c153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 -- 172.21.15.40:0/3904843417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c09fdd0 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26522eb640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 0x7f264c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26512e9640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 0x7f264c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2651aea640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f264800c480 tx=0x7f264800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480052d0 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648005470 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c0a00b0 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.194+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648012870 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.195+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c0a0670 con 0x7f264c06e490 2024-04-26T21:21:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.196+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c06a290 con 0x7f264c06e490 2024-04-26T21:21:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.196+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2648012a10 con 0x7f264c06e490 2024-04-26T21:21:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.196+0000 7f26427fc640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.196+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f264808b790 con 0x7f264c06e490 2024-04-26T21:21:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.197+0000 7f26512e9640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.197+0000 7f26512e9640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 0x7f263006edb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f264c15d950 tx=0x7f263c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.201+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2648059c50 con 0x7f264c06e490 2024-04-26T21:21:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.424+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f264c160ae0 con 0x7f264c06e490 2024-04-26T21:21:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.424+0000 7f26427fc640 1 -- 172.21.15.40:0/3904843417 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7f2648059550 con 0x7f264c06e490 2024-04-26T21:21:44.425 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:44.425 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 msgr2=0x7f263006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 0x7f263006edb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f264c15d950 tx=0x7f263c009040 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 msgr2=0x7f264c09e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f264800c480 tx=0x7f264800c940 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 shutdown_connections 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f264c0d11b0 0x7f264c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f264c0cf2e0 0x7f264c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 --2- 172.21.15.40:0/3904843417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f264c06e490 0x7f264c09e0a0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 >> 172.21.15.40:0/3904843417 conn(0x7f264c0b4a60 msgr2=0x7f264c071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.426+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 shutdown_connections 2024-04-26T21:21:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.427+0000 7f2653d4e640 1 -- 172.21.15.40:0/3904843417 wait complete. 2024-04-26T21:21:44.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:44.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.538+0000 7f78003da640 1 Processor -- start 2024-04-26T21:21:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 -- start start 2024-04-26T21:21:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77f815f4a0 0x7f77f80b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77f80afe30 0x7f77f80b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80b6320 0x7f77f8162f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f77f815f880 con 0x7f77f80afe30 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f77f80af9a0 con 0x7f77f815f4a0 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.539+0000 7f78003da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77f80aed90 con 0x7f77f80b6320 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77f80afe30 0x7f77f80b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77f80afe30 0x7f77f80b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59794/0 (socket says 172.21.15.40:59794) 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 -- 172.21.15.40:0/2645042554 learned_addr learned my addr 172.21.15.40:0/2645042554 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fe176640 1 --2- 172.21.15.40:0/2645042554 >> v2:172.21.15.184:3300/0 conn(0x7f77f815f4a0 0x7f77f80b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fe977640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80b6320 0x7f77f8162f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 -- 172.21.15.40:0/2645042554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80b6320 msgr2=0x7f77f8162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80b6320 0x7f77f8162f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 -- 172.21.15.40:0/2645042554 >> v2:172.21.15.184:3300/0 conn(0x7f77f815f4a0 msgr2=0x7f77f80b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 --2- 172.21.15.40:0/2645042554 >> v2:172.21.15.184:3300/0 conn(0x7f77f815f4a0 0x7f77f80b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.540+0000 7f77fd975640 1 -- 172.21.15.40:0/2645042554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f80af070 con 0x7f77f80afe30 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f77fd975640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f80afe30 0x7f77f80b4600 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f77f8072980 tx=0x7f77e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa690296fec4c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f77fd174640 1 -- 172.21.15.40:0/2645042554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8037070 con 0x7f77f80afe30 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f77fd174640 1 -- 172.21.15.40:0/2645042554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e8032440 con 0x7f77f80afe30 2024-04-26T21:21:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f77fd174640 1 -- 172.21.15.40:0/2645042554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8032810 con 0x7f77f80afe30 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f78003da640 1 -- 172.21.15.40:0/2645042554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f80afe30 msgr2=0x7f77f80b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.541+0000 7f78003da640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f80afe30 0x7f77f80b4600 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f77f8072980 tx=0x7f77e802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 -- 172.21.15.40:0/2645042554 shutdown_connections 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80b6320 0x7f77f8162f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 --2- 172.21.15.40:0/2645042554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f80afe30 0x7f77f80b4600 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 --2- 172.21.15.40:0/2645042554 >> v2:172.21.15.184:3300/0 conn(0x7f77f815f4a0 0x7f77f80b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 -- 172.21.15.40:0/2645042554 >> 172.21.15.40:0/2645042554 conn(0x7f77f80b6800 msgr2=0x7f77f80738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 -- 172.21.15.40:0/2645042554 shutdown_connections 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.542+0000 7f78003da640 1 -- 172.21.15.40:0/2645042554 wait complete. 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 Processor -- start 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 -- start start 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80afe30 0x7f77f81570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f80b6320 0x7f77f815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77f80aea80 con 0x7f77f815f4a0 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77f80af630 con 0x7f77f80b6320 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.543+0000 7f78003da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f77f81634c0 con 0x7f77f80afe30 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59798/0 (socket says 172.21.15.40:59798) 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 -- 172.21.15.40:0/1409327090 learned_addr learned my addr 172.21.15.40:0/1409327090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe176640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80afe30 0x7f77f81570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fd975640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f80b6320 0x7f77f815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 -- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80afe30 msgr2=0x7f77f81570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80afe30 0x7f77f81570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 -- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f80b6320 msgr2=0x7f77f815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f80b6320 0x7f77f815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 -- 172.21.15.40:0/1409327090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f8162570 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77fe977640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e7d0 tx=0x7f77f000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f000c940 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f8162810 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f000cd30 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.544+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f815da90 con 0x7f77f815f4a0 2024-04-26T21:21:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.545+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f806a290 con 0x7f77f815f4a0 2024-04-26T21:21:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.546+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77f001a070 con 0x7f77f815f4a0 2024-04-26T21:21:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.546+0000 7f77eeffd640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 0x7f77dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.546+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f77f008ba50 con 0x7f77f815f4a0 2024-04-26T21:21:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.547+0000 7f77fe176640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 0x7f77dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.547+0000 7f77fe176640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 0x7f77dc06edb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f77f4011340 tx=0x7f77f400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.551+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77f0059820 con 0x7f77f815f4a0 2024-04-26T21:21:44.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.739+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77f806d4b0 con 0x7f77f815f4a0 2024-04-26T21:21:44.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.740+0000 7f77eeffd640 1 -- 172.21.15.40:0/1409327090 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7f77f005db20 con 0x7f77f815f4a0 2024-04-26T21:21:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:44.742 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 msgr2=0x7f77dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 0x7f77dc06edb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f77f4011340 tx=0x7f77f400b040 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 msgr2=0x7f77f8162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e7d0 tx=0x7f77f000ec90 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 shutdown_connections 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f77dc06c8f0 0x7f77dc06edb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77f815f4a0 0x7f77f8162030 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f80b6320 0x7f77f815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.742+0000 7f78003da640 1 --2- 172.21.15.40:0/1409327090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f80afe30 0x7f77f81570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.743+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 >> 172.21.15.40:0/1409327090 conn(0x7f77f80b6800 msgr2=0x7f77f80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.743+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 shutdown_connections 2024-04-26T21:21:44.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.743+0000 7f78003da640 1 -- 172.21.15.40:0/1409327090 wait complete. 2024-04-26T21:21:44.754 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:21:44.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:44.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 Processor -- start 2024-04-26T21:21:44.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 -- start start 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b415f340 0x7f86b40b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d5610 0x7f86b40d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b40b2510 con 0x7f86b415f720 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40b27f0 con 0x7f86b415f340 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.855+0000 7f86b927e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40b2ad0 con 0x7f86b40d5610 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b415f340 0x7f86b40b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b27fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86b415f720 0x7f86b40d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36402/0 (socket says 172.21.15.40:36402) 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86b415f340 0x7f86b40b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59806/0 (socket says 172.21.15.40:59806) 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b27fc640 1 -- 172.21.15.40:0/1279201313 learned_addr learned my addr 172.21.15.40:0/1279201313 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 -- 172.21.15.40:0/1279201313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d5610 msgr2=0x7f86b40d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/1279201313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d5610 0x7f86b40d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 -- 172.21.15.40:0/1279201313 >> v1:172.21.15.40:6789/0 conn(0x7f86b415f720 legacy=0x7f86b40d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.856+0000 7f86b2ffd640 1 -- 172.21.15.40:0/1279201313 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40b2db0 con 0x7f86b415f340 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/1279201313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f340 0x7f86b40b1e80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097a0 tx=0x7f86a402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b249d01d4ed80519 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b1ffb640 1 -- 172.21.15.40:0/1279201313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86b415f340 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b1ffb640 1 -- 172.21.15.40:0/1279201313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a402ae50 con 0x7f86b415f340 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b1ffb640 1 -- 172.21.15.40:0/1279201313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4032660 con 0x7f86b415f340 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 -- 172.21.15.40:0/1279201313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f340 msgr2=0x7f86b40b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 --2- 172.21.15.40:0/1279201313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f340 0x7f86b40b1e80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097a0 tx=0x7f86a402a100 comp rx=0 tx=0).stop 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 -- 172.21.15.40:0/1279201313 shutdown_connections 2024-04-26T21:21:44.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 --2- 172.21.15.40:0/1279201313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b40d5610 0x7f86b40d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 --2- 172.21.15.40:0/1279201313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f340 0x7f86b40b1e80 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 -- 172.21.15.40:0/1279201313 >> 172.21.15.40:0/1279201313 conn(0x7f86b408ec70 msgr2=0x7f86b409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:44.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 -- 172.21.15.40:0/1279201313 shutdown_connections 2024-04-26T21:21:44.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.857+0000 7f86b927e640 1 -- 172.21.15.40:0/1279201313 wait complete. 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 Processor -- start 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 -- start start 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 0x7f86b40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 0x7f86b41f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40b3940 con 0x7f86b415f720 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40b37c0 con 0x7f86b40d5610 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.859+0000 7f86b927e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40b3ac0 con 0x7f86b415f340 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59814/0 (socket says 172.21.15.40:59814) 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 -- 172.21.15.40:0/4235776835 learned_addr learned my addr 172.21.15.40:0/4235776835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b27fc640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 0x7f86b41f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 0x7f86b40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 -- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 msgr2=0x7f86b41f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 0x7f86b41f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 -- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 msgr2=0x7f86b40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 0x7f86b40b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 -- 172.21.15.40:0/4235776835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40ca630 con 0x7f86b415f720 2024-04-26T21:21:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b27fc640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 0x7f86b41f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 0x7f86b40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b37fe640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00b9e0 tx=0x7f86ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac019070 con 0x7f86b415f720 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac00cb20 con 0x7f86b415f720 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b40ca910 con 0x7f86b415f720 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac014440 con 0x7f86b415f720 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.860+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b40b9770 con 0x7f86b415f720 2024-04-26T21:21:44.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.861+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b40caae0 con 0x7f86b415f720 2024-04-26T21:21:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.862+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86ac01c070 con 0x7f86b415f720 2024-04-26T21:21:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.862+0000 7f869b7fe640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.862+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f86ac08b890 con 0x7f86b415f720 2024-04-26T21:21:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.863+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.863+0000 7f86b2ffd640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034000 tx=0x7f86a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:44.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:44.867+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86ac059cb0 con 0x7f86b415f720 2024-04-26T21:21:45.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.090+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86b406a290 con 0x7f86b415f720 2024-04-26T21:21:45.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.091+0000 7f869b7fe640 1 -- 172.21.15.40:0/4235776835 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7f86ac0595b0 con 0x7f86b415f720 2024-04-26T21:21:45.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:45.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:45.092 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:45.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 msgr2=0x7f86a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034000 tx=0x7f86a402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:45.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 msgr2=0x7f86b406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00b9e0 tx=0x7f86ac00bea0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 shutdown_connections 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f86a006c8f0 0x7f86a006edb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b415f720 0x7f86b406dad0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86b415f340 0x7f86b41f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 --2- 172.21.15.40:0/4235776835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86b40d5610 0x7f86b40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 >> 172.21.15.40:0/4235776835 conn(0x7f86b408ec70 msgr2=0x7f86b40b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 shutdown_connections 2024-04-26T21:21:45.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.093+0000 7f86b927e640 1 -- 172.21.15.40:0/4235776835 wait complete. 2024-04-26T21:21:45.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:21:45.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.205+0000 7fb7079da640 1 Processor -- start 2024-04-26T21:21:45.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.205+0000 7fb7079da640 1 -- start start 2024-04-26T21:21:45.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 0x7fb700146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb700140350 0x7fb7001465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 0x7fb7000cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000d05b0 con 0x7fb700140350 2024-04-26T21:21:45.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000d0890 con 0x7fb70013ff70 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb7079da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb70006e450 con 0x7fb7000cf810 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb700140350 0x7fb7001465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb700140350 0x7fb7001465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59822/0 (socket says 172.21.15.40:59822) 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 -- 172.21.15.40:0/2577450169 learned_addr learned my addr 172.21.15.40:0/2577450169 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb705776640 1 --2- 172.21.15.40:0/2577450169 >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 0x7fb700146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb705f77640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 0x7fb7000cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 -- 172.21.15.40:0/2577450169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 msgr2=0x7fb7000cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 0x7fb7000cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 -- 172.21.15.40:0/2577450169 >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 msgr2=0x7fb700146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 --2- 172.21.15.40:0/2577450169 >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 0x7fb700146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb704f75640 1 -- 172.21.15.40:0/2577450169 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70006e730 con 0x7fb700140350 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb705776640 1 --2- 172.21.15.40:0/2577450169 >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 0x7fb700146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:45.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.206+0000 7fb705f77640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 0x7fb7000cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.207+0000 7fb704f75640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb700140350 0x7fb7001465c0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb700074200 tx=0x7fb6f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0214574d02c6936 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.207+0000 7fb6f7fff640 1 -- 172.21.15.40:0/2577450169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0037070 con 0x7fb700140350 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.207+0000 7fb6f7fff640 1 -- 172.21.15.40:0/2577450169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f0032440 con 0x7fb700140350 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.207+0000 7fb6f7fff640 1 -- 172.21.15.40:0/2577450169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0032810 con 0x7fb700140350 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 -- 172.21.15.40:0/2577450169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb700140350 msgr2=0x7fb7001465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb700140350 0x7fb7001465c0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb700074200 tx=0x7fb6f002a140 comp rx=0 tx=0).stop 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 -- 172.21.15.40:0/2577450169 shutdown_connections 2024-04-26T21:21:45.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7000cf810 0x7fb7000cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 --2- 172.21.15.40:0/2577450169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb700140350 0x7fb7001465c0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 --2- 172.21.15.40:0/2577450169 >> v2:172.21.15.184:3300/0 conn(0x7fb70013ff70 0x7fb700146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 -- 172.21.15.40:0/2577450169 >> 172.21.15.40:0/2577450169 conn(0x7fb7000d0d30 msgr2=0x7fb7000d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 -- 172.21.15.40:0/2577450169 shutdown_connections 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.208+0000 7fb7079da640 1 -- 172.21.15.40:0/2577450169 wait complete. 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.209+0000 7fb7079da640 1 Processor -- start 2024-04-26T21:21:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.209+0000 7fb7079da640 1 -- start start 2024-04-26T21:21:45.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7000cf810 0x7fb7001397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 0x7fb70013dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb700142a40 con 0x7fb70013ff70 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7001428c0 con 0x7fb7000cf810 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb7079da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb700142bc0 con 0x7fb700140350 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59824/0 (socket says 172.21.15.40:59824) 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 -- 172.21.15.40:0/3984131730 learned_addr learned my addr 172.21.15.40:0/3984131730 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb705776640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7000cf810 0x7fb7001397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb705f77640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 0x7fb70013dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 -- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 msgr2=0x7fb70013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 0x7fb70013dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 -- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7000cf810 msgr2=0x7fb7001397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7000cf810 0x7fb7001397a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 -- 172.21.15.40:0/3984131730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb700139ce0 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb705f77640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 0x7fb70013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb704f75640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002a180 tx=0x7fb6f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.210+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0041020 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.211+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f00306c0 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.211+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0037040 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.211+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7000b49a0 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.211+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7000b4f60 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.212+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70006a290 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.212+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6f0044020 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.213+0000 7fb6f5ffb640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 0x7fb6e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.213+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7fb6f00abfe0 con 0x7fb70013ff70 2024-04-26T21:21:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.213+0000 7fb705776640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 0x7fb6e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.213+0000 7fb705776640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 0x7fb6e406edb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc006e80 tx=0x7fb6fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.217+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6f004b070 con 0x7fb70013ff70 2024-04-26T21:21:45.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.440+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7000b13f0 con 0x7fb70013ff70 2024-04-26T21:21:45.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.441+0000 7fb6f5ffb640 1 -- 172.21.15.40:0/3984131730 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5191 (secure 0 0 0) 0x7fb700139ce0 con 0x7fb70013ff70 2024-04-26T21:21:45.443 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:45.443 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:21:30.208910+0000","modified":"2024-04-26T21:21:33.291754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3970947174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3970947174},{"type":"v1","addr":"172.21.15.40:6835","nonce":3970947174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":11}]} 2024-04-26T21:21:45.443 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 99 2024-04-26T21:21:45.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 msgr2=0x7fb6e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 0x7fb6e406edb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc006e80 tx=0x7fb6fc008040 comp rx=0 tx=0).stop 2024-04-26T21:21:45.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 msgr2=0x7fb70013d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002a180 tx=0x7fb6f00046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 shutdown_connections 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6e406c8f0 0x7fb6e406edb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb700140350 0x7fb70013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb70013ff70 0x7fb70013d840 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 --2- 172.21.15.40:0/3984131730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7000cf810 0x7fb7001397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 >> 172.21.15.40:0/3984131730 conn(0x7fb7000d0d30 msgr2=0x7fb7000b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 shutdown_connections 2024-04-26T21:21:45.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.443+0000 7fb7079da640 1 -- 172.21.15.40:0/3984131730 wait complete. 2024-04-26T21:21:45.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:45.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.555+0000 7efdbc72b640 1 Processor -- start 2024-04-26T21:21:45.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.555+0000 7efdbc72b640 1 -- start start 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbc72b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdb415fa60 0x7efdb40d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbc72b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb40d3b80 con 0x7efdb40b4290 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbc72b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb40d3e60 con 0x7efdb415f680 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbc72b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdb40d4140 con 0x7efdb415fa60 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdb9cc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdb415fa60 0x7efdb40d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbacc8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efdb40b4290 0x7efdb40d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36404/0 (socket says 172.21.15.40:36404) 2024-04-26T21:21:45.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.556+0000 7efdbacc8640 1 -- 172.21.15.40:0/856513739 learned_addr learned my addr 172.21.15.40:0/856513739 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.557+0000 7efdb9cc6640 1 -- 172.21.15.40:0/856513739 >> v1:172.21.15.184:6789/0 conn(0x7efdb415f680 legacy=0x7efdb40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.557+0000 7efdb9cc6640 1 -- 172.21.15.40:0/856513739 >> v1:172.21.15.40:6789/0 conn(0x7efdb40b4290 legacy=0x7efdb40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.557+0000 7efdb94c5640 1 -- 172.21.15.40:0/856513739 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476825439 0 0) 0x7efdb40d3b80 con 0x7efdb40b4290 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.557+0000 7efdb9cc6640 1 -- 172.21.15.40:0/856513739 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb40d0270 con 0x7efdb415fa60 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.557+0000 7efdb9cc6640 1 --2- 172.21.15.40:0/856513739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415fa60 0x7efdb40d5e50 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7efda80097d0 tx=0x7efda802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0c0dd51291511f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdb94c5640 1 -- 172.21.15.40:0/856513739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8037070 con 0x7efdb415fa60 2024-04-26T21:21:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdb94c5640 1 -- 172.21.15.40:0/856513739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8032440 con 0x7efdb415fa60 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdb94c5640 1 -- 172.21.15.40:0/856513739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8032810 con 0x7efdb415fa60 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 -- 172.21.15.40:0/856513739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415fa60 msgr2=0x7efdb40d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 --2- 172.21.15.40:0/856513739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415fa60 0x7efdb40d5e50 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7efda80097d0 tx=0x7efda802a270 comp rx=0 tx=0).stop 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 -- 172.21.15.40:0/856513739 shutdown_connections 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 --2- 172.21.15.40:0/856513739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415fa60 0x7efdb40d5e50 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 -- 172.21.15.40:0/856513739 >> 172.21.15.40:0/856513739 conn(0x7efdb409e5a0 msgr2=0x7efdb40d5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 -- 172.21.15.40:0/856513739 shutdown_connections 2024-04-26T21:21:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.558+0000 7efdbc72b640 1 -- 172.21.15.40:0/856513739 wait complete. 2024-04-26T21:21:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.559+0000 7efdbc72b640 1 Processor -- start 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.559+0000 7efdbc72b640 1 -- start start 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 0x7efdb415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 0x7efdb40b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40d4420 con 0x7efdb415f680 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40b2610 con 0x7efdb415fa60 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbc72b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7efdb40d2300 con 0x7efdb40b4290 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdb9cc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdb9cc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59848/0 (socket says 172.21.15.40:59848) 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdb9cc6640 1 -- 172.21.15.40:0/1978089492 learned_addr learned my addr 172.21.15.40:0/1978089492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdbacc8640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 0x7efdb40b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdba4c7640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 0x7efdb415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdb9cc6640 1 -- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 msgr2=0x7efdb415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.560+0000 7efdb9cc6640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 0x7efdb415db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb9cc6640 1 -- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 msgr2=0x7efdb40b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb9cc6640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 0x7efdb40b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb9cc6640 1 -- 172.21.15.40:0/1978089492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41f61c0 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdbacc8640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 0x7efdb40b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdba4c7640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 0x7efdb415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb9cc6640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7efda8035040 tx=0x7efda80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8037050 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8038040 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8004490 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb4162f80 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.561+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41f6760 con 0x7efdb415f680 2024-04-26T21:21:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.562+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb406a290 con 0x7efdb415f680 2024-04-26T21:21:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.562+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efda8030900 con 0x7efdb415f680 2024-04-26T21:21:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.563+0000 7efdb2ffd640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.563+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7efda80ac710 con 0x7efdb415f680 2024-04-26T21:21:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.563+0000 7efdba4c7640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.564+0000 7efdba4c7640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efdb415e990 tx=0x7efdac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.567+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efda807abd0 con 0x7efdb415f680 2024-04-26T21:21:45.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.752+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdb41f6930 con 0x7efdb415f680 2024-04-26T21:21:45.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.754+0000 7efdb2ffd640 1 -- 172.21.15.40:0/1978089492 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16671 (secure 0 0 0) 0x7efda807a4d0 con 0x7efdb415f680 2024-04-26T21:21:45.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:45.755 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":145,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:31.843500+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:21:28.644565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-26T21:21:29.151922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 msgr2=0x7efda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efdb415e990 tx=0x7efdac009040 comp rx=0 tx=0).stop 2024-04-26T21:21:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 msgr2=0x7efdb4162580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7efda8035040 tx=0x7efda80046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 shutdown_connections 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdb415fa60 0x7efdb40b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb415f680 0x7efdb4162580 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 --2- 172.21.15.40:0/1978089492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efdb40b4290 0x7efdb415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.755+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 >> 172.21.15.40:0/1978089492 conn(0x7efdb409e5a0 msgr2=0x7efdb406da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.756+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 shutdown_connections 2024-04-26T21:21:45.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.756+0000 7efdbc72b640 1 -- 172.21.15.40:0/1978089492 wait complete. 2024-04-26T21:21:45.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:21:45.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.869+0000 7f00f5b22640 1 Processor -- start 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.869+0000 7f00f5b22640 1 -- start start 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f015f370 0x7f00f00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00f015f750 0x7f00f00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00f00d37d0 0x7f00f00d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f00f00b4d10 con 0x7f00f015f750 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f00f00d4120 con 0x7f00f00d37d0 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00f5b22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d03a0 con 0x7f00f015f370 2024-04-26T21:21:45.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00e6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00f015f750 0x7f00f00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00e6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00f015f750 0x7f00f00b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59858/0 (socket says 172.21.15.40:59858) 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.870+0000 7f00e6ffd640 1 -- 172.21.15.40:0/2913823503 learned_addr learned my addr 172.21.15.40:0/2913823503 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00effff640 1 --2- 172.21.15.40:0/2913823503 >> v2:172.21.15.184:3300/0 conn(0x7f00f00d37d0 0x7f00f00d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f015f370 0x7f00f00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 -- 172.21.15.40:0/2913823503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f015f370 msgr2=0x7f00f00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f015f370 0x7f00f00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 -- 172.21.15.40:0/2913823503 >> v2:172.21.15.184:3300/0 conn(0x7f00f00d37d0 msgr2=0x7f00f00d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:45.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 --2- 172.21.15.40:0/2913823503 >> v2:172.21.15.184:3300/0 conn(0x7f00f00d37d0 0x7f00f00d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 -- 172.21.15.40:0/2913823503 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00d0680 con 0x7f00f015f750 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00e6ffd640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b47d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f00f00d02b0 tx=0x7f00d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac832d0efe06302 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.871+0000 7f00eeffd640 1 -- 172.21.15.40:0/2913823503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037070 con 0x7f00f015f750 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00eeffd640 1 -- 172.21.15.40:0/2913823503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8032440 con 0x7f00f015f750 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00eeffd640 1 -- 172.21.15.40:0/2913823503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8032810 con 0x7f00f015f750 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 -- 172.21.15.40:0/2913823503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 msgr2=0x7f00f00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b47d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f00f00d02b0 tx=0x7f00d802a140 comp rx=0 tx=0).stop 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 -- 172.21.15.40:0/2913823503 shutdown_connections 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 --2- 172.21.15.40:0/2913823503 >> v2:172.21.15.184:3300/0 conn(0x7f00f00d37d0 0x7f00f00d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b47d0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 --2- 172.21.15.40:0/2913823503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f015f370 0x7f00f00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 -- 172.21.15.40:0/2913823503 >> 172.21.15.40:0/2913823503 conn(0x7f00f008ec10 msgr2=0x7f00f009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 -- 172.21.15.40:0/2913823503 shutdown_connections 2024-04-26T21:21:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.872+0000 7f00f5b22640 1 -- 172.21.15.40:0/2913823503 wait complete. 2024-04-26T21:21:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.873+0000 7f00f5b22640 1 Processor -- start 2024-04-26T21:21:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.873+0000 7f00f5b22640 1 -- start start 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f00d37d0 0x7f00f00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 0x7f00f01f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d0f20 con 0x7f00f015f750 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d0c40 con 0x7f00f015f370 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00f5b22640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d1bd0 con 0x7f00f00d37d0 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59872/0 (socket says 172.21.15.40:59872) 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.874+0000 7f00effff640 1 -- 172.21.15.40:0/18148424 learned_addr learned my addr 172.21.15.40:0/18148424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00e6ffd640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 0x7f00f01f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f00d37d0 0x7f00f00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 -- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f00d37d0 msgr2=0x7f00f00ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f00d37d0 0x7f00f00ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 -- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 msgr2=0x7f00f01f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 0x7f00f01f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 -- 172.21.15.40:0/18148424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f01f4c60 con 0x7f00f015f750 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00e6ffd640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 0x7f00f01f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00effff640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f00e8007e60 tx=0x7f00e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8014620 con 0x7f00f015f750 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e8019070 con 0x7f00f015f750 2024-04-26T21:21:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f00b90e0 con 0x7f00f015f750 2024-04-26T21:21:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8014a20 con 0x7f00f015f750 2024-04-26T21:21:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.875+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f00cc2c0 con 0x7f00f015f750 2024-04-26T21:21:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.876+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f006a290 con 0x7f00f015f750 2024-04-26T21:21:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.876+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00e8014bc0 con 0x7f00f015f750 2024-04-26T21:21:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.877+0000 7f00ecff9640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 0x7f00d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.877+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5709+0+0 (secure 0 0 0) 0x7f00e808c3f0 con 0x7f00f015f750 2024-04-26T21:21:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.877+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 0x7f00d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.878+0000 7f00ef7fe640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 0x7f00d006edb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:45.881+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00e805a820 con 0x7f00f015f750 2024-04-26T21:21:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.107+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f00f00d15e0 con 0x7f00f015f750 2024-04-26T21:21:46.108 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-26T21:21:46.108+0000 7f067b959640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:21:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.109+0000 7f00ecff9640 1 -- 172.21.15.40:0/18148424 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7f00e805a120 con 0x7f00f015f750 2024-04-26T21:21:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.113+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 msgr2=0x7f00d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.113+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 0x7f00d006edb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00e0006fd0 tx=0x7f00e0008040 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 msgr2=0x7f00f00b8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f00e8007e60 tx=0x7f00e800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 shutdown_connections 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f00d006c8f0 0x7f00d006edb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00f015f750 0x7f00f00b8890 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00f015f370 0x7f00f01f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 --2- 172.21.15.40:0/18148424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00f00d37d0 0x7f00f00ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 >> 172.21.15.40:0/18148424 conn(0x7f00f008ec10 msgr2=0x7f00f00d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 shutdown_connections 2024-04-26T21:21:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.114+0000 7f00f5b22640 1 -- 172.21.15.40:0/18148424 wait complete. 2024-04-26T21:21:46.116 INFO:tasks.ceph.mds.c.smithi040.stderr: -2242> 2024-04-26T21:21:32.803+0000 7f623737f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:21:46.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:21:46.144 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-26T21:21:46.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.229+0000 7fc21242a640 1 Processor -- start 2024-04-26T21:21:46.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.229+0000 7fc21242a640 1 -- start start 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc21242a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc20c0d0010 0x7fc20c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc21242a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 0x7fc20c163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc21242a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc20c09c700 con 0x7fc20c0b3c20 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc21242a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c0b4480 con 0x7fc20c0d0010 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc21242a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c06e050 con 0x7fc20c0b33e0 2024-04-26T21:21:46.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc20c0b3c20 0x7fc20c0b2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36418/0 (socket says 172.21.15.40:36418) 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20b7fe640 1 -- 172.21.15.40:0/483757027 learned_addr learned my addr 172.21.15.40:0/483757027 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc2109c7640 1 --2- 172.21.15.40:0/483757027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 0x7fc20c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20bfff640 1 --2- 172.21.15.40:0/483757027 >> v2:172.21.15.184:3300/0 conn(0x7fc20c0d0010 0x7fc20c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920798952 0 0) 0x7fc20c09c700 con 0x7fc20c0b3c20 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f4003520 con 0x7fc20c0b3c20 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc2109c7640 1 -- 172.21.15.40:0/483757027 >> v2:172.21.15.184:3300/0 conn(0x7fc20c0d0010 msgr2=0x7fc20c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc2109c7640 1 --2- 172.21.15.40:0/483757027 >> v2:172.21.15.184:3300/0 conn(0x7fc20c0d0010 0x7fc20c06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc2109c7640 1 -- 172.21.15.40:0/483757027 >> v1:172.21.15.40:6789/0 conn(0x7fc20c0b3c20 legacy=0x7fc20c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1209365784 0 0) 0x7fc1f4003520 con 0x7fc20c0b3c20 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.230+0000 7fc2109c7640 1 -- 172.21.15.40:0/483757027 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c0d1c50 con 0x7fc20c0b33e0 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.231+0000 7fc2109c7640 1 --2- 172.21.15.40:0/483757027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 0x7fc20c163280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc204009d90 tx=0x7fc20402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f38500e3740ac7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.231+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200001670 con 0x7fc20c0b33e0 2024-04-26T21:21:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.231+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204007540 con 0x7fc20c0b33e0 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.231+0000 7fc20affd640 1 -- 172.21.15.40:0/483757027 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204007910 con 0x7fc20c0b33e0 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 -- 172.21.15.40:0/483757027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 msgr2=0x7fc20c163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 --2- 172.21.15.40:0/483757027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 0x7fc20c163280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc204009d90 tx=0x7fc20402a5f0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 -- 172.21.15.40:0/483757027 shutdown_connections 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 --2- 172.21.15.40:0/483757027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b33e0 0x7fc20c163280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 --2- 172.21.15.40:0/483757027 >> v2:172.21.15.184:3300/0 conn(0x7fc20c0d0010 0x7fc20c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 -- 172.21.15.40:0/483757027 >> 172.21.15.40:0/483757027 conn(0x7fc20c09ba40 msgr2=0x7fc20c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 -- 172.21.15.40:0/483757027 shutdown_connections 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.232+0000 7fc21242a640 1 -- 172.21.15.40:0/483757027 wait complete. 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.233+0000 7fc21242a640 1 Processor -- start 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.233+0000 7fc21242a640 1 -- start start 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b3c20 0x7fc20c09eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c0d0010 0x7fc20c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0b39d0 con 0x7fc20c0b33e0 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0b3850 con 0x7fc20c0d0010 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc21242a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c1637c0 con 0x7fc20c0b3c20 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59896/0 (socket says 172.21.15.40:59896) 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 -- 172.21.15.40:0/1491197976 learned_addr learned my addr 172.21.15.40:0/1491197976 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20b7fe640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b3c20 0x7fc20c09eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc2109c7640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c0d0010 0x7fc20c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 -- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b3c20 msgr2=0x7fc20c09eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b3c20 0x7fc20c09eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 -- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c0d0010 msgr2=0x7fc20c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c0d0010 0x7fc20c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:46.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.234+0000 7fc20bfff640 1 -- 172.21.15.40:0/1491197976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c09ced0 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc20bfff640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e870 tx=0x7fc1fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc017070 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc00c940 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00cd30 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c09d1b0 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.235+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c09d770 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.236+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c06a290 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.236+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1fc01a070 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.237+0000 7fc208ff9640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.237+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5795+0+0 (secure 0 0 0) 0x7fc1fc08bb20 con 0x7fc20c0b33e0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.237+0000 7fc20b7fe640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:46.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.237+0000 7fc20b7fe640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 0x7fc1f806edb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06db40 tx=0x7fc200009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:46.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.241+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1fc059e50 con 0x7fc20c0b33e0 2024-04-26T21:21:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:46.466+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc20c068e00 con 0x7fc20c0b33e0 2024-04-26T21:21:47.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.111+0000 7fc208ff9640 1 -- 172.21.15.40:0/1491197976 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) ==== 104+0+0 (secure 0 0 0) 0x7fc1fc059750 con 0x7fc20c0b33e0 2024-04-26T21:21:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 msgr2=0x7fc1f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 0x7fc1f806edb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06db40 tx=0x7fc200009040 comp rx=0 tx=0).stop 2024-04-26T21:21:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 msgr2=0x7fc20c150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e870 tx=0x7fc1fc00ed30 comp rx=0 tx=0).stop 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 shutdown_connections 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc20c0d0010 0x7fc20c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc20c0b3c20 0x7fc20c09eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 --2- 172.21.15.40:0/1491197976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc20c0b33e0 0x7fc20c150390 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 >> 172.21.15.40:0/1491197976 conn(0x7fc20c09ba40 msgr2=0x7fc20c15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 shutdown_connections 2024-04-26T21:21:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.114+0000 7fc21242a640 1 -- 172.21.15.40:0/1491197976 wait complete. 2024-04-26T21:21:47.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:21:47.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.284+0000 7fa6b94ea640 1 Processor -- start 2024-04-26T21:21:47.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.284+0000 7fa6b94ea640 1 -- start start 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b94ea640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa6b409c150 0x7fa6b406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b94ea640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b40d1240 con 0x7fa6b40d3a30 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b94ea640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b40d14e0 con 0x7fa6b40cee20 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b94ea640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b40d1e10 con 0x7fa6b409c150 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa6b40d3a30 0x7fa6b415eb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36428/0 (socket says 172.21.15.40:36428) 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b37fe640 1 -- 172.21.15.40:0/2824372686 learned_addr learned my addr 172.21.15.40:0/2824372686 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b27fc640 1 --2- 172.21.15.40:0/2824372686 >> v2:172.21.15.184:3300/0 conn(0x7fa6b409c150 0x7fa6b406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135237271 0 0) 0x7fa6b40d1240 con 0x7fa6b40d3a30 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa69c003520 con 0x7fa6b40d3a30 2024-04-26T21:21:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150409352 0 0) 0x7fa6b40d14e0 con 0x7fa6b40cee20 2024-04-26T21:21:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.285+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b40d1240 con 0x7fa6b40cee20 2024-04-26T21:21:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.286+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1250605989 0 0) 0x7fa69c003520 con 0x7fa6b40d3a30 2024-04-26T21:21:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.286+0000 7fa6b27fc640 1 -- 172.21.15.40:0/2824372686 >> v1:172.21.15.184:6789/0 conn(0x7fa6b40cee20 legacy=0x7fa6b409ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.286+0000 7fa6b27fc640 1 -- 172.21.15.40:0/2824372686 >> v1:172.21.15.40:6789/0 conn(0x7fa6b40d3a30 legacy=0x7fa6b415eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.286+0000 7fa6b27fc640 1 -- 172.21.15.40:0/2824372686 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40d17c0 con 0x7fa6b409c150 2024-04-26T21:21:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.286+0000 7fa6b27fc640 1 --2- 172.21.15.40:0/2824372686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b409c150 0x7fa6b406d5c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097a0 tx=0x7fa6a802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17b82b2619896e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8037070 con 0x7fa6b409c150 2024-04-26T21:21:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a802ac50 con 0x7fa6b409c150 2024-04-26T21:21:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b1ffb640 1 -- 172.21.15.40:0/2824372686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0019f0 con 0x7fa6b409c150 2024-04-26T21:21:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 -- 172.21.15.40:0/2824372686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b409c150 msgr2=0x7fa6b406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/2824372686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b409c150 0x7fa6b406d5c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097a0 tx=0x7fa6a802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 -- 172.21.15.40:0/2824372686 shutdown_connections 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/2824372686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b409c150 0x7fa6b406d5c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 -- 172.21.15.40:0/2824372686 >> 172.21.15.40:0/2824372686 conn(0x7fa6b40d2f10 msgr2=0x7fa6b40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 -- 172.21.15.40:0/2824372686 shutdown_connections 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.287+0000 7fa6b94ea640 1 -- 172.21.15.40:0/2824372686 wait complete. 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 Processor -- start 2024-04-26T21:21:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 -- start start 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6b409c150 0x7fa6b40c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b40cee20 0x7fa6b41600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40cf200 con 0x7fa6b40d3a30 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b409c7a0 con 0x7fa6b40cee20 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b94ea640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b40cf380 con 0x7fa6b409c150 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59912/0 (socket says 172.21.15.40:59912) 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.289+0000 7fa6b37fe640 1 -- 172.21.15.40:0/753980464 learned_addr learned my addr 172.21.15.40:0/753980464 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b27fc640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b40cee20 0x7fa6b41600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b2ffd640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6b409c150 0x7fa6b40c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 -- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6b409c150 msgr2=0x7fa6b40c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6b409c150 0x7fa6b40c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 -- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b40cee20 msgr2=0x7fa6b41600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b40cee20 0x7fa6b41600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 -- 172.21.15.40:0/753980464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40c7f70 con 0x7fa6b40d3a30 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b37fe640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00cbe0 tx=0x7fa6ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac017070 con 0x7fa6b40d3a30 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac00ace0 con 0x7fa6b40d3a30 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b40c8250 con 0x7fa6b40d3a30 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac012640 con 0x7fa6b40d3a30 2024-04-26T21:21:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.290+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b40c8810 con 0x7fa6b40d3a30 2024-04-26T21:21:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.291+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b406a290 con 0x7fa6b40d3a30 2024-04-26T21:21:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.292+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6ac01a070 con 0x7fa6b40d3a30 2024-04-26T21:21:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.292+0000 7fa69b7fe640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 0x7fa6a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.292+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5795+0+0 (secure 0 0 0) 0x7fa6ac08b9e0 con 0x7fa6b40d3a30 2024-04-26T21:21:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.292+0000 7fa6b2ffd640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 0x7fa6a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.293+0000 7fa6b2ffd640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 0x7fa6a006edb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40011e0 tx=0x7fa6a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.297+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6ac059e40 con 0x7fa6b40d3a30 2024-04-26T21:21:47.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:47.481+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6b40b4820 con 0x7fa6b40d3a30 2024-04-26T21:21:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.112+0000 7fa69b7fe640 1 -- 172.21.15.40:0/753980464 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7fa6ac059740 con 0x7fa6b40d3a30 2024-04-26T21:21:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:21:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 msgr2=0x7fa6a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 0x7fa6a006edb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40011e0 tx=0x7fa6a4009040 comp rx=0 tx=0).stop 2024-04-26T21:21:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 msgr2=0x7fa6b41f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00cbe0 tx=0x7fa6ac00a430 comp rx=0 tx=0).stop 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 shutdown_connections 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa6a006c8f0 0x7fa6a006edb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b40d3a30 0x7fa6b41f4b40 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6b40cee20 0x7fa6b41600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 --2- 172.21.15.40:0/753980464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6b409c150 0x7fa6b40c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 >> 172.21.15.40:0/753980464 conn(0x7fa6b40d2f10 msgr2=0x7fa6b40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 shutdown_connections 2024-04-26T21:21:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.116+0000 7fa6b94ea640 1 -- 172.21.15.40:0/753980464 wait complete. 2024-04-26T21:21:48.131 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:21:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.241+0000 7f6dd9e28640 1 Processor -- start 2024-04-26T21:21:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.241+0000 7f6dd9e28640 1 -- start start 2024-04-26T21:21:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd9e28640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 0x7f6dd40d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd9e28640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40b6020 0x7f6dd40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd9e28640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd40d33a0 con 0x7f6dd4162a10 2024-04-26T21:21:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd9e28640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd40b4970 con 0x7f6dd40b6020 2024-04-26T21:21:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd9e28640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd40d3680 con 0x7f6dd40d2140 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40b6020 0x7f6dd40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6dd40b6020 0x7f6dd40cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59914/0 (socket says 172.21.15.40:59914) 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.242+0000 7f6dd3fff640 1 -- 172.21.15.40:0/2612155851 learned_addr learned my addr 172.21.15.40:0/2612155851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd3fff640 1 -- 172.21.15.40:0/2612155851 >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 msgr2=0x7f6dd40d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd2ffd640 1 --2- 172.21.15.40:0/2612155851 >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 0x7f6dd40d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd3fff640 1 --2- 172.21.15.40:0/2612155851 >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 0x7f6dd40d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd3fff640 1 -- 172.21.15.40:0/2612155851 >> v1:172.21.15.184:6789/0 conn(0x7f6dd4162a10 legacy=0x7f6dd40d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd3fff640 1 -- 172.21.15.40:0/2612155851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd40d3960 con 0x7f6dd40b6020 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd2ffd640 1 --2- 172.21.15.40:0/2612155851 >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 0x7f6dd40d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd3fff640 1 --2- 172.21.15.40:0/2612155851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40b6020 0x7f6dd40cffb0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6dd408f0b0 tx=0x7f6dcc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fafcc1f9e45c368 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.243+0000 7f6dd27fc640 1 -- 172.21.15.40:0/2612155851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc036070 con 0x7f6dd40b6020 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd27fc640 1 -- 172.21.15.40:0/2612155851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc031600 con 0x7f6dd40b6020 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd27fc640 1 -- 172.21.15.40:0/2612155851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0319d0 con 0x7f6dd40b6020 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 -- 172.21.15.40:0/2612155851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40b6020 msgr2=0x7f6dd40cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/2612155851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40b6020 0x7f6dd40cffb0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6dd408f0b0 tx=0x7f6dcc02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 -- 172.21.15.40:0/2612155851 shutdown_connections 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/2612155851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40b6020 0x7f6dd40cffb0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/2612155851 >> v2:172.21.15.184:3300/0 conn(0x7f6dd40d2140 0x7f6dd40d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 -- 172.21.15.40:0/2612155851 >> 172.21.15.40:0/2612155851 conn(0x7f6dd409e530 msgr2=0x7f6dd40d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 -- 172.21.15.40:0/2612155851 shutdown_connections 2024-04-26T21:21:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.244+0000 7f6dd9e28640 1 -- 172.21.15.40:0/2612155851 wait complete. 2024-04-26T21:21:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.245+0000 7f6dd9e28640 1 Processor -- start 2024-04-26T21:21:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.245+0000 7f6dd9e28640 1 -- start start 2024-04-26T21:21:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40b6020 0x7f6dd40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd4162a10 0x7f6dd40cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4162df0 con 0x7f6dd40d2140 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40d3f20 con 0x7f6dd40b6020 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd9e28640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd40b6600 con 0x7f6dd4162a10 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59920/0 (socket says 172.21.15.40:59920) 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd2ffd640 1 -- 172.21.15.40:0/4047715216 learned_addr learned my addr 172.21.15.40:0/4047715216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd3fff640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd4162a10 0x7f6dd40cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.246+0000 7f6dd37fe640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40b6020 0x7f6dd40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 -- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd4162a10 msgr2=0x7f6dd40cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd4162a10 0x7f6dd40cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 -- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40b6020 msgr2=0x7f6dd40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40b6020 0x7f6dd40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 -- 172.21.15.40:0/4047715216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd40ce360 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd2ffd640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c7f0 tx=0x7f6dc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8015070 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc8007a30 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8007e00 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd40cc510 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.247+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd415abd0 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.248+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd406a290 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.249+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dc8018070 con 0x7f6dd40d2140 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.249+0000 7f6dbbfff640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.249+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5357+0+0 (secure 0 0 0) 0x7f6dc808b8c0 con 0x7f6dd40d2140 2024-04-26T21:21:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.249+0000 7f6dd37fe640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.250+0000 7f6dd37fe640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 0x7f6dc006edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4006fd0 tx=0x7f6dc4009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.254+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dc8059890 con 0x7f6dd40d2140 2024-04-26T21:21:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:48.437+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6dd40b6ac0 con 0x7f6dd40d2140 2024-04-26T21:21:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.114+0000 7f6dbbfff640 1 -- 172.21.15.40:0/4047715216 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f6dc805db90 con 0x7f6dd40d2140 2024-04-26T21:21:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-26T21:21:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.119+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 msgr2=0x7f6dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.119+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 0x7f6dc006edb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4006fd0 tx=0x7f6dc4009060 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.119+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 msgr2=0x7f6dd415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.119+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c7f0 tx=0x7f6dc800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 shutdown_connections 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f6dc006c8f0 0x7f6dc006edb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6dd4162a10 0x7f6dd40cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6dd40d2140 0x7f6dd415a290 secure :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c7f0 tx=0x7f6dc800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 --2- 172.21.15.40:0/4047715216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6dd40b6020 0x7f6dd40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.120+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 >> 172.21.15.40:0/4047715216 conn(0x7f6dd409e530 msgr2=0x7f6dd406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.121+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 shutdown_connections 2024-04-26T21:21:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.121+0000 7f6dd9e28640 1 -- 172.21.15.40:0/4047715216 wait complete. 2024-04-26T21:21:49.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:21:49.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.246+0000 7f4d39722640 1 Processor -- start 2024-04-26T21:21:49.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.246+0000 7f4d39722640 1 -- start start 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d39722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d3415d340 0x7f4d3409c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d39722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d340d1940 0x7f4d340702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d39722640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d340cee60 con 0x7f4d340b2420 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d39722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4d340ce9d0 con 0x7f4d3415d340 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d39722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d340d0ff0 con 0x7f4d340d1940 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d32ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d3415d340 0x7f4d3409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d32ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d3415d340 0x7f4d3409c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59934/0 (socket says 172.21.15.40:59934) 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d32ffd640 1 -- 172.21.15.40:0/2939540208 learned_addr learned my addr 172.21.15.40:0/2939540208 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.247+0000 7f4d337fe640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d340d1940 0x7f4d340702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.248+0000 7f4d32ffd640 1 -- 172.21.15.40:0/2939540208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d340d1940 msgr2=0x7f4d340702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.248+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d340d1940 0x7f4d340702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.248+0000 7f4d32ffd640 1 -- 172.21.15.40:0/2939540208 >> v1:172.21.15.184:6789/0 conn(0x7f4d340b2420 legacy=0x7f4d340d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.248+0000 7f4d32ffd640 1 -- 172.21.15.40:0/2939540208 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d340d0d10 con 0x7f4d3415d340 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.248+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d3409c560 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88039c92aa2c90e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d31ffb640 1 -- 172.21.15.40:0/2939540208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c037070 con 0x7f4d3415d340 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d31ffb640 1 -- 172.21.15.40:0/2939540208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c032440 con 0x7f4d3415d340 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d31ffb640 1 -- 172.21.15.40:0/2939540208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c032810 con 0x7f4d3415d340 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 -- 172.21.15.40:0/2939540208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 msgr2=0x7f4d3409c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d3409c560 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009870 tx=0x7f4d2c02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 -- 172.21.15.40:0/2939540208 shutdown_connections 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d340d1940 0x7f4d340702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 --2- 172.21.15.40:0/2939540208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d3409c560 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 -- 172.21.15.40:0/2939540208 >> 172.21.15.40:0/2939540208 conn(0x7f4d3409ba40 msgr2=0x7f4d3409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 -- 172.21.15.40:0/2939540208 shutdown_connections 2024-04-26T21:21:49.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.249+0000 7f4d39722640 1 -- 172.21.15.40:0/2939540208 wait complete. 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.250+0000 7f4d39722640 1 Processor -- start 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 -- start start 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 0x7f4d340b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d3414ec70 0x7f4d34151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340b2090 con 0x7f4d3415d340 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3415d720 con 0x7f4d340d1940 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d39722640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4d340b2210 con 0x7f4d3414ec70 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59948/0 (socket says 172.21.15.40:59948) 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.251+0000 7f4d327fc640 1 -- 172.21.15.40:0/891033798 learned_addr learned my addr 172.21.15.40:0/891033798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 0x7f4d340b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d337fe640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d3414ec70 0x7f4d34151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 -- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d3414ec70 msgr2=0x7f4d34151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d3414ec70 0x7f4d34151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 -- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 msgr2=0x7f4d340b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 0x7f4d340b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 -- 172.21.15.40:0/891033798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d340b8050 con 0x7f4d3415d340 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 0x7f4d340b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d327fc640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c450 tx=0x7f4d2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280052d0 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28005470 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d340b8330 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28012870 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.252+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d340b88f0 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.253+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d340b8d20 con 0x7f4d3415d340 2024-04-26T21:21:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.254+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d28012a10 con 0x7f4d3415d340 2024-04-26T21:21:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.254+0000 7f4d1b7fe640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.254+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f4d2808b450 con 0x7f4d3415d340 2024-04-26T21:21:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.254+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.255+0000 7f4d32ffd640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c034000 tx=0x7f4d2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.259+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d28059b80 con 0x7f4d3415d340 2024-04-26T21:21:49.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.442+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4d340693d0 con 0x7f4d3415d340 2024-04-26T21:21:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.445+0000 7f4d1b7fe640 1 -- 172.21.15.40:0/891033798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7f4d28059480 con 0x7f4d3415d340 2024-04-26T21:21:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.447+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 msgr2=0x7f4d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.447+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c034000 tx=0x7f4d2c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.447+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 msgr2=0x7f4d34151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.447+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c450 tx=0x7f4d2800c910 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 shutdown_connections 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d3414ec70 0x7f4d34151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3415d340 0x7f4d34151440 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 --2- 172.21.15.40:0/891033798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d340d1940 0x7f4d340b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 >> 172.21.15.40:0/891033798 conn(0x7f4d3409ba40 msgr2=0x7f4d340d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 shutdown_connections 2024-04-26T21:21:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.448+0000 7f4d39722640 1 -- 172.21.15.40:0/891033798 wait complete. 2024-04-26T21:21:49.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-04-26T21:21:49.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.559+0000 7f8553ff7640 1 Processor -- start 2024-04-26T21:21:49.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.559+0000 7f8553ff7640 1 -- start start 2024-04-26T21:21:49.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.560+0000 7f8553ff7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f854c09ba40 0x7f854c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.560+0000 7f8553ff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 0x7f854c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.560+0000 7f8553ff7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854c15fce0 con 0x7f854c09be20 2024-04-26T21:21:49.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.560+0000 7f8553ff7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f854c160610 con 0x7f854c09ba40 2024-04-26T21:21:49.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.560+0000 7f8553ff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f854c0709b0 con 0x7f854c070570 2024-04-26T21:21:49.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551d93640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f854c09ba40 0x7f854c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8552594640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f854c09be20 0x7f854c162f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:41806/0 (socket says 172.21.15.40:41806) 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551592640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 0x7f854c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8552594640 1 -- 172.21.15.40:0/1186243558 learned_addr learned my addr 172.21.15.40:0/1186243558 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8550d91640 1 -- 172.21.15.40:0/1186243558 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903333163 0 0) 0x7f854c15fce0 con 0x7f854c09be20 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8550d91640 1 -- 172.21.15.40:0/1186243558 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8534003520 con 0x7f854c09be20 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551592640 1 -- 172.21.15.40:0/1186243558 >> v2:172.21.15.184:3300/0 conn(0x7f854c09ba40 msgr2=0x7f854c15f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551592640 1 --2- 172.21.15.40:0/1186243558 >> v2:172.21.15.184:3300/0 conn(0x7f854c09ba40 0x7f854c15f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551592640 1 -- 172.21.15.40:0/1186243558 >> v1:172.21.15.184:6789/0 conn(0x7f854c09be20 legacy=0x7f854c162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.561+0000 7f8551592640 1 -- 172.21.15.40:0/1186243558 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c15ffc0 con 0x7f854c070570 2024-04-26T21:21:49.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.562+0000 7f8551592640 1 --2- 172.21.15.40:0/1186243558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 0x7f854c0b5530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85400097a0 tx=0x7f854002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c2d4061ad42d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.562+0000 7f8550d91640 1 -- 172.21.15.40:0/1186243558 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540037070 con 0x7f854c070570 2024-04-26T21:21:49.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.562+0000 7f8550d91640 1 -- 172.21.15.40:0/1186243558 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540032440 con 0x7f854c070570 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.562+0000 7f8550d91640 1 -- 172.21.15.40:0/1186243558 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540032810 con 0x7f854c070570 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 -- 172.21.15.40:0/1186243558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 msgr2=0x7f854c0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 --2- 172.21.15.40:0/1186243558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 0x7f854c0b5530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85400097a0 tx=0x7f854002a180 comp rx=0 tx=0).stop 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 -- 172.21.15.40:0/1186243558 shutdown_connections 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 --2- 172.21.15.40:0/1186243558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c070570 0x7f854c0b5530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 --2- 172.21.15.40:0/1186243558 >> v2:172.21.15.184:3300/0 conn(0x7f854c09ba40 0x7f854c15f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 -- 172.21.15.40:0/1186243558 >> 172.21.15.40:0/1186243558 conn(0x7f854c0b4a10 msgr2=0x7f854c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 -- 172.21.15.40:0/1186243558 shutdown_connections 2024-04-26T21:21:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.563+0000 7f8553ff7640 1 -- 172.21.15.40:0/1186243558 wait complete. 2024-04-26T21:21:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.564+0000 7f8553ff7640 1 Processor -- start 2024-04-26T21:21:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.564+0000 7f8553ff7640 1 -- start start 2024-04-26T21:21:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c070570 0x7f854c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c09be20 0x7f854c154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f854c0d2050 con 0x7f854c09ba40 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f854c09c2b0 con 0x7f854c070570 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8553ff7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f854c0d21d0 con 0x7f854c09be20 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59962/0 (socket says 172.21.15.40:59962) 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 -- 172.21.15.40:0/619161120 learned_addr learned my addr 172.21.15.40:0/619161120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551d93640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c070570 0x7f854c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8552594640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c09be20 0x7f854c154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 -- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c09be20 msgr2=0x7f854c154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c09be20 0x7f854c154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 -- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c070570 msgr2=0x7f854c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c070570 0x7f854c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 -- 172.21.15.40:0/619161120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c152ba0 con 0x7f854c09ba40 2024-04-26T21:21:49.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.565+0000 7f8551592640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f85400098c0 tx=0x7f85400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.566+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540041030 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.566+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854003d4e0 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.566+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c155300 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.566+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540037040 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.566+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1558c0 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.567+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c155b00 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.567+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8540044070 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.567+0000 7f854a7fc640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 0x7f853806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.568+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f85400abd80 con 0x7f854c09ba40 2024-04-26T21:21:49.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.568+0000 7f8551d93640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 0x7f853806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:49.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.568+0000 7f8551d93640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 0x7f853806edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8544009b40 tx=0x7f8544009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:49.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.572+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8540079e70 con 0x7f854c09ba40 2024-04-26T21:21:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:49.835+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f854c06a290 con 0x7f854c09ba40 2024-04-26T21:21:50.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.115+0000 7f854a7fc640 1 -- 172.21.15.40:0/619161120 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v516) ==== 151+0+0 (secure 0 0 0) 0x7f854007e170 con 0x7f854c09ba40 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 msgr2=0x7f853806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 0x7f853806edb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8544009b40 tx=0x7f8544009190 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 msgr2=0x7f854c1541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f85400098c0 tx=0x7f85400046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 shutdown_connections 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f853806c8f0 0x7f853806edb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f854c09be20 0x7f854c154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f854c09ba40 0x7f854c1541c0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 --2- 172.21.15.40:0/619161120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854c070570 0x7f854c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 >> 172.21.15.40:0/619161120 conn(0x7f854c0b4a10 msgr2=0x7f854c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:50.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 shutdown_connections 2024-04-26T21:21:50.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:50.118+0000 7f8553ff7640 1 -- 172.21.15.40:0/619161120 wait complete. 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... skipped 'Require FUSE client to inject client release failure' 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner: 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner:Ran 10 tests in 558.110s 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner: 2024-04-26T21:21:50.131 INFO:tasks.cephfs_test_runner:OK (skipped=4) 2024-04-26T21:21:50.132 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T21:21:50.142 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T21:21:50.156 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T21:21:50.156 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:50.156 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:50.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:50.173 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-26T21:21:50.173 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:50.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17997 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:21:50.229 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:50.229 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:50.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:50.247 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-04-26T21:21:50.248 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:50.307 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056110,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14148 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:21:50.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.307 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:21:50.322 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:21:50.323 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:21:50.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:21:50.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:21:50.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:21:50.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.539 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-26T21:21:50.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.595 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-26T21:21:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-26T21:21:50.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:21:50.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.664 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:21:50.681 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:21:50.681 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:21:50.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:21:50.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.750 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:21:50.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.819 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:21:50.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.910 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2024-04-26T21:21:50.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:50.925 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2024-04-26T21:21:50.993 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2024-04-26T21:21:50.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:21:50.995 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T21:21:51.007 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:21:51.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.112+0000 7f519d4da640 1 Processor -- start 2024-04-26T21:21:51.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.112+0000 7f519d4da640 1 -- start start 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f519d4da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f5198146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f519d4da640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5198140350 0x7f51981465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f519d4da640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51980d05b0 con 0x7f51980cf810 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f519d4da640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f51980d0390 con 0x7f5198140350 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f519d4da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f51980d0890 con 0x7f519813ff70 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5198140350 0x7f51981465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5198140350 0x7f51981465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57316/0 (socket says 172.21.15.40:57316) 2024-04-26T21:21:51.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 -- 172.21.15.40:0/3249865791 learned_addr learned my addr 172.21.15.40:0/3249865791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 -- 172.21.15.40:0/3249865791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 msgr2=0x7f5198146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f5196ffd640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f5198146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f5198146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 -- 172.21.15.40:0/3249865791 >> v1:172.21.15.184:6789/0 conn(0x7f51980cf810 legacy=0x7f51980cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f51967fc640 1 -- 172.21.15.40:0/3249865791 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519806e450 con 0x7f5198140350 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.113+0000 7f5196ffd640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f5198146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.114+0000 7f51967fc640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5198140350 0x7f51981465c0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1e18fae77cfe3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.114+0000 7f5195ffb640 1 -- 172.21.15.40:0/3249865791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c037070 con 0x7f5198140350 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.114+0000 7f5195ffb640 1 -- 172.21.15.40:0/3249865791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c032440 con 0x7f5198140350 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.114+0000 7f5195ffb640 1 -- 172.21.15.40:0/3249865791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c032810 con 0x7f5198140350 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 -- 172.21.15.40:0/3249865791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5198140350 msgr2=0x7f51981465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5198140350 0x7f51981465c0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f518c009870 tx=0x7f518c02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 -- 172.21.15.40:0/3249865791 shutdown_connections 2024-04-26T21:21:51.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5198140350 0x7f51981465c0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 --2- 172.21.15.40:0/3249865791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f5198146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 -- 172.21.15.40:0/3249865791 >> 172.21.15.40:0/3249865791 conn(0x7f51980d0d30 msgr2=0x7f51980d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 -- 172.21.15.40:0/3249865791 shutdown_connections 2024-04-26T21:21:51.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.115+0000 7f519d4da640 1 -- 172.21.15.40:0/3249865791 wait complete. 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.116+0000 7f519d4da640 1 Processor -- start 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.116+0000 7f519d4da640 1 -- start start 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f519815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5198140350 0x7f519815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f519806edd0 con 0x7f51980cf810 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f519806ec50 con 0x7f5198140350 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f519d4da640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f519806ef50 con 0x7f519813ff70 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57318/0 (socket says 172.21.15.40:57318) 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 -- 172.21.15.40:0/944625564 learned_addr learned my addr 172.21.15.40:0/944625564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f51967fc640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f519815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f51977fe640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5198140350 0x7f519815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 -- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 msgr2=0x7f519815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f519815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 -- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5198140350 msgr2=0x7f519815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5198140350 0x7f519815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.117+0000 7f5196ffd640 1 -- 172.21.15.40:0/944625564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519815f8d0 con 0x7f51980cf810 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.118+0000 7f5196ffd640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f519000c7f0 tx=0x7f519000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.119+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190015070 con 0x7f51980cf810 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.119+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5190007a30 con 0x7f51980cf810 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.119+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5198160410 con 0x7f51980cf810 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.119+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190007e00 con 0x7f51980cf810 2024-04-26T21:21:51.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.119+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519815fb70 con 0x7f51980cf810 2024-04-26T21:21:51.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.120+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519806a290 con 0x7f51980cf810 2024-04-26T21:21:51.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.120+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5190018070 con 0x7f51980cf810 2024-04-26T21:21:51.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.121+0000 7f517f7fe640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 0x7f518406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.121+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f519008b770 con 0x7f51980cf810 2024-04-26T21:21:51.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.121+0000 7f51967fc640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 0x7f518406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.121+0000 7f51967fc640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 0x7f518406ede0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f518c034000 tx=0x7f518c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.125+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5190059860 con 0x7f51980cf810 2024-04-26T21:21:51.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.306+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51980b45e0 con 0x7f518406c920 2024-04-26T21:21:51.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.307+0000 7f517f7fe640 1 -- 172.21.15.40:0/944625564 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7f51980b45e0 con 0x7f518406c920 2024-04-26T21:21:51.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:51.309 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 msgr2=0x7f518406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 0x7f518406ede0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f518c034000 tx=0x7f518c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 msgr2=0x7f5198144e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f519000c7f0 tx=0x7f519000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 shutdown_connections 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.309+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f518406c920 0x7f518406ede0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5198140350 0x7f519815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f519813ff70 0x7f519815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 --2- 172.21.15.40:0/944625564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51980cf810 0x7f5198144e40 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 >> 172.21.15.40:0/944625564 conn(0x7f51980d0d30 msgr2=0x7f51980b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 shutdown_connections 2024-04-26T21:21:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.310+0000 7f519d4da640 1 -- 172.21.15.40:0/944625564 wait complete. 2024-04-26T21:21:51.321 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-04-26T21:21:51.061683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":273256,"kb_used_data":15496,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701464,"statfs":{"total":773094113280,"available":772814299136,"internally_reserved":0,"allocated":15867904,"data_stored":7505182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4191,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001497"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":393,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-26T21:21:46.846407+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:46.846407+0000","last_peered":"2024-04-26T21:21:46.846407+0000","last_clean":"2024-04-26T21:21:46.846407+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:46.846407+0000","last_undegraded":"2024-04-26T21:21:46.846407+0000","last_fullsized":"2024-04-26T21:21:46.846407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2445312,"data_stored":1384011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":790112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1843200,"data_stored":792971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33544,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338296,"statfs":{"total":96636764160,"available":96602415104,"internally_reserved":0,"allocated":2420736,"data_stored":1379876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:51.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:51.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.421+0000 7fb717d66640 1 Processor -- start 2024-04-26T21:21:51.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 -- start start 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb71015f370 0x7fb7100cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb71015f750 0x7fb7100b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7100d03a0 con 0x7fb7100d37d0 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100b4d10 con 0x7fb71015f370 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb717d66640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100d0680 con 0x7fb71015f750 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb715b02640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb71015f370 0x7fb7100cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb716303640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7100d37d0 0x7fb7100d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60114/0 (socket says 172.21.15.40:60114) 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.422+0000 7fb715b02640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb71015f370 0x7fb7100cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57326/0 (socket says 172.21.15.40:57326) 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb716303640 1 -- 172.21.15.40:0/2581570094 learned_addr learned my addr 172.21.15.40:0/2581570094 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb715301640 1 --2- 172.21.15.40:0/2581570094 >> v2:172.21.15.184:3300/0 conn(0x7fb71015f750 0x7fb7100b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb715b02640 1 -- 172.21.15.40:0/2581570094 >> v2:172.21.15.184:3300/0 conn(0x7fb71015f750 msgr2=0x7fb7100b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb715b02640 1 --2- 172.21.15.40:0/2581570094 >> v2:172.21.15.184:3300/0 conn(0x7fb71015f750 0x7fb7100b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb715b02640 1 -- 172.21.15.40:0/2581570094 >> v1:172.21.15.40:6789/0 conn(0x7fb7100d37d0 legacy=0x7fb7100d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb715b02640 1 -- 172.21.15.40:0/2581570094 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7100d0960 con 0x7fb71015f370 2024-04-26T21:21:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.423+0000 7fb714b00640 1 -- 172.21.15.40:0/2581570094 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3652466511 0 0) 0x7fb7100d03a0 con 0x7fb7100d37d0 2024-04-26T21:21:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb715b02640 1 --2- 172.21.15.40:0/2581570094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100cfd70 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb7100d02b0 tx=0x7fb70802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34e7e843f71c90a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb714b00640 1 -- 172.21.15.40:0/2581570094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708037070 con 0x7fb71015f370 2024-04-26T21:21:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb714b00640 1 -- 172.21.15.40:0/2581570094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708032440 con 0x7fb71015f370 2024-04-26T21:21:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb714b00640 1 -- 172.21.15.40:0/2581570094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708032810 con 0x7fb71015f370 2024-04-26T21:21:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 -- 172.21.15.40:0/2581570094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 msgr2=0x7fb7100cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 --2- 172.21.15.40:0/2581570094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100cfd70 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb7100d02b0 tx=0x7fb70802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 -- 172.21.15.40:0/2581570094 shutdown_connections 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 --2- 172.21.15.40:0/2581570094 >> v2:172.21.15.184:3300/0 conn(0x7fb71015f750 0x7fb7100b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 --2- 172.21.15.40:0/2581570094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100cfd70 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 -- 172.21.15.40:0/2581570094 >> 172.21.15.40:0/2581570094 conn(0x7fb71008ec10 msgr2=0x7fb71009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 -- 172.21.15.40:0/2581570094 shutdown_connections 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.424+0000 7fb717d66640 1 -- 172.21.15.40:0/2581570094 wait complete. 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 Processor -- start 2024-04-26T21:21:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 -- start start 2024-04-26T21:21:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7100d37d0 0x7fb7100cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71015f750 0x7fb7100c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100b61d0 con 0x7fb71015f370 2024-04-26T21:21:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100d0f20 con 0x7fb71015f750 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb717d66640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb7100b6350 con 0x7fb7100d37d0 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb715301640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb715301640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57338/0 (socket says 172.21.15.40:57338) 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.426+0000 7fb715301640 1 -- 172.21.15.40:0/289312382 learned_addr learned my addr 172.21.15.40:0/289312382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb716303640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71015f750 0x7fb7100c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 -- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7100d37d0 msgr2=0x7fb7100cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7100d37d0 0x7fb7100cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 -- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71015f750 msgr2=0x7fb7100c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71015f750 0x7fb7100c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 -- 172.21.15.40:0/289312382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb710160950 con 0x7fb71015f370 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb715301640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fb70000e9b0 tx=0x7fb70000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700017070 con 0x7fb71015f370 2024-04-26T21:21:51.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70000c9a0 con 0x7fb71015f370 2024-04-26T21:21:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb710160c30 con 0x7fb71015f370 2024-04-26T21:21:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000cd90 con 0x7fb71015f370 2024-04-26T21:21:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.427+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101611f0 con 0x7fb71015f370 2024-04-26T21:21:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.428+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71006a290 con 0x7fb71015f370 2024-04-26T21:21:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.429+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb70001a070 con 0x7fb71015f370 2024-04-26T21:21:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.429+0000 7fb7067fc640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 0x7fb6f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.429+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fb70008bb50 con 0x7fb71015f370 2024-04-26T21:21:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.429+0000 7fb715b02640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 0x7fb6f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.430+0000 7fb715b02640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 0x7fb6f406edb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb708034000 tx=0x7fb70802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.434+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb70005a310 con 0x7fb71015f370 2024-04-26T21:21:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.613+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7100cbbb0 con 0x7fb6f406c8f0 2024-04-26T21:21:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.614+0000 7fb7067fc640 1 -- 172.21.15.40:0/289312382 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7fb708032590 con 0x7fb6f406c8f0 2024-04-26T21:21:51.615 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 msgr2=0x7fb6f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 0x7fb6f406edb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb708034000 tx=0x7fb70802c000 comp rx=0 tx=0).stop 2024-04-26T21:21:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 msgr2=0x7fb7100c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fb70000e9b0 tx=0x7fb70000ee70 comp rx=0 tx=0).stop 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 shutdown_connections 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb6f406c8f0 0x7fb6f406edb0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb71015f750 0x7fb7100c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb71015f370 0x7fb7100c8d20 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 --2- 172.21.15.40:0/289312382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7100d37d0 0x7fb7100cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 >> 172.21.15.40:0/289312382 conn(0x7fb71008ec10 msgr2=0x7fb71009e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 shutdown_connections 2024-04-26T21:21:51.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.616+0000 7fb717d66640 1 -- 172.21.15.40:0/289312382 wait complete. 2024-04-26T21:21:51.628 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-04-26T21:21:51.061683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":273256,"kb_used_data":15496,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701464,"statfs":{"total":773094113280,"available":772814299136,"internally_reserved":0,"allocated":15867904,"data_stored":7505182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4191,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001497"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":393,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-26T21:21:46.846407+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:46.846407+0000","last_peered":"2024-04-26T21:21:46.846407+0000","last_clean":"2024-04-26T21:21:46.846407+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:46.846407+0000","last_undegraded":"2024-04-26T21:21:46.846407+0000","last_fullsized":"2024-04-26T21:21:46.846407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2445312,"data_stored":1384011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":790112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1843200,"data_stored":792971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33544,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338296,"statfs":{"total":96636764160,"available":96602415104,"internally_reserved":0,"allocated":2420736,"data_stored":1379876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:51.628 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:21:51.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.728+0000 7f6964a2a640 1 Processor -- start 2024-04-26T21:21:51.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.728+0000 7f6964a2a640 1 -- start start 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f6964a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f6964a2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f696015f720 0x7f69600d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f6964a2a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69600b2510 con 0x7f69600d4f00 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f6964a2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f69600b27f0 con 0x7f696015f720 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f6964a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69600b2ad0 con 0x7f696015f340 2024-04-26T21:21:51.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f696015f720 0x7f69600d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f696015f720 0x7f69600d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57354/0 (socket says 172.21.15.40:57354) 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 -- 172.21.15.40:0/2375554261 learned_addr learned my addr 172.21.15.40:0/2375554261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 -- 172.21.15.40:0/2375554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 msgr2=0x7f69600b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 --2- 172.21.15.40:0/2375554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 -- 172.21.15.40:0/2375554261 >> v1:172.21.15.184:6789/0 conn(0x7f69600d4f00 legacy=0x7f69600d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.729+0000 7f695dd9b640 1 -- 172.21.15.40:0/2375554261 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600b2db0 con 0x7f696015f720 2024-04-26T21:21:51.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f695dd9b640 1 --2- 172.21.15.40:0/2375554261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696015f720 0x7f69600d3d40 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f6954009870 tx=0x7f695402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283c551300f9fd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f695d59a640 1 -- 172.21.15.40:0/2375554261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954037070 con 0x7f696015f720 2024-04-26T21:21:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f695d59a640 1 -- 172.21.15.40:0/2375554261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954032440 con 0x7f696015f720 2024-04-26T21:21:51.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f695d59a640 1 -- 172.21.15.40:0/2375554261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954032810 con 0x7f696015f720 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f6964a2a640 1 -- 172.21.15.40:0/2375554261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696015f720 msgr2=0x7f69600d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.730+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2375554261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696015f720 0x7f69600d3d40 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f6954009870 tx=0x7f695402a180 comp rx=0 tx=0).stop 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 -- 172.21.15.40:0/2375554261 shutdown_connections 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2375554261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f696015f720 0x7f69600d3d40 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2375554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 -- 172.21.15.40:0/2375554261 >> 172.21.15.40:0/2375554261 conn(0x7f696008ec70 msgr2=0x7f696009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 -- 172.21.15.40:0/2375554261 shutdown_connections 2024-04-26T21:21:51.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.731+0000 7f6964a2a640 1 -- 172.21.15.40:0/2375554261 wait complete. 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.732+0000 7f6964a2a640 1 Processor -- start 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.732+0000 7f6964a2a640 1 -- start start 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 0x7f69600b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69600b3980 con 0x7f69600d4f00 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69600b3800 con 0x7f696015f720 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f69600b3b00 con 0x7f696015f340 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57360/0 (socket says 172.21.15.40:57360) 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 -- 172.21.15.40:0/2617000532 learned_addr learned my addr 172.21.15.40:0/2617000532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695dd9b640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695ed9d640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 0x7f69600b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 -- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 msgr2=0x7f69600d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 -- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 msgr2=0x7f69600b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 0x7f69600b9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 -- 172.21.15.40:0/2617000532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600cc400 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695ed9d640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 0x7f69600b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f695e59c640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f695800b4d0 tx=0x7f695800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958014070 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958010870 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69601f8ea0 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958010c60 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.733+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601f9460 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.734+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696006a290 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.735+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6958017070 con 0x7f69600d4f00 2024-04-26T21:21:51.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.735+0000 7f694effd640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 0x7f694006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.735+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f695808b680 con 0x7f69600d4f00 2024-04-26T21:21:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.735+0000 7f695dd9b640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 0x7f694006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.736+0000 7f695dd9b640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 0x7f694006edb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6954034000 tx=0x7f695402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:51.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.740+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6958059db0 con 0x7f69600d4f00 2024-04-26T21:21:51.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.920+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69600c8b30 con 0x7f694006c8f0 2024-04-26T21:21:51.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.921+0000 7f694effd640 1 -- 172.21.15.40:0/2617000532 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7f69600c8b30 con 0x7f694006c8f0 2024-04-26T21:21:51.922 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:51.923 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 msgr2=0x7f694006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 0x7f694006edb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6954034000 tx=0x7f695402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 msgr2=0x7f69600d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f695800b4d0 tx=0x7f695800b990 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 shutdown_connections 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f694006c8f0 0x7f694006edb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f696015f720 0x7f69600b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f696015f340 0x7f69600d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 --2- 172.21.15.40:0/2617000532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69600d4f00 0x7f69600d0ac0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:51.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 >> 172.21.15.40:0/2617000532 conn(0x7f696008ec70 msgr2=0x7f696006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 shutdown_connections 2024-04-26T21:21:51.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:51.924+0000 7f6964a2a640 1 -- 172.21.15.40:0/2617000532 wait complete. 2024-04-26T21:21:51.935 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-04-26T21:21:51.061683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":273256,"kb_used_data":15496,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701464,"statfs":{"total":773094113280,"available":772814299136,"internally_reserved":0,"allocated":15867904,"data_stored":7505182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4191,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001497"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":393,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-26T21:21:46.846407+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:46.846407+0000","last_peered":"2024-04-26T21:21:46.846407+0000","last_clean":"2024-04-26T21:21:46.846407+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:46.846407+0000","last_undegraded":"2024-04-26T21:21:46.846407+0000","last_fullsized":"2024-04-26T21:21:46.846407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2445312,"data_stored":1384011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":790112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1843200,"data_stored":792971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33544,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338296,"statfs":{"total":96636764160,"available":96602415104,"internally_reserved":0,"allocated":2420736,"data_stored":1379876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":789542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:51.936 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.036+0000 7fb2c6a07640 1 Processor -- start 2024-04-26T21:21:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.036+0000 7fb2c6a07640 1 -- start start 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2c6a07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2c00d0380 0x7fb2c00b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2c6a07640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c00737c0 con 0x7fb2c0145b00 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2c6a07640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c0073aa0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2c6a07640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c0073d80 con 0x7fb2c00d0380 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2c00d0380 0x7fb2c00b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2c00d0380 0x7fb2c00b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57372/0 (socket says 172.21.15.40:57372) 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb2c0145b00 0x7fb2c0072fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60118/0 (socket says 172.21.15.40:60118) 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/1749461103 learned_addr learned my addr 172.21.15.40:0/1749461103 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/1749461103 >> v1:172.21.15.184:6789/0 conn(0x7fb2c00b5f20 legacy=0x7fb2c00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/1749461103 >> v1:172.21.15.40:6789/0 conn(0x7fb2c0145b00 legacy=0x7fb2c0072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2bf7fe640 1 -- 172.21.15.40:0/1749461103 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c00b4200 con 0x7fb2c00d0380 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.037+0000 7fb2beffd640 1 -- 172.21.15.40:0/1749461103 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861125059 0 0) 0x7fb2c00737c0 con 0x7fb2c0145b00 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.038+0000 7fb2bf7fe640 1 --2- 172.21.15.40:0/1749461103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00d0380 0x7fb2c00b59e0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009870 tx=0x7fb2a802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1674cf0443b5077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.038+0000 7fb2beffd640 1 -- 172.21.15.40:0/1749461103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8037070 con 0x7fb2c00d0380 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.038+0000 7fb2beffd640 1 -- 172.21.15.40:0/1749461103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a802ae50 con 0x7fb2c00d0380 2024-04-26T21:21:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.038+0000 7fb2beffd640 1 -- 172.21.15.40:0/1749461103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8032860 con 0x7fb2c00d0380 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 -- 172.21.15.40:0/1749461103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00d0380 msgr2=0x7fb2c00b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/1749461103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00d0380 0x7fb2c00b59e0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009870 tx=0x7fb2a802a300 comp rx=0 tx=0).stop 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 -- 172.21.15.40:0/1749461103 shutdown_connections 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/1749461103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00d0380 0x7fb2c00b59e0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 -- 172.21.15.40:0/1749461103 >> 172.21.15.40:0/1749461103 conn(0x7fb2c00ce8b0 msgr2=0x7fb2c00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 -- 172.21.15.40:0/1749461103 shutdown_connections 2024-04-26T21:21:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.039+0000 7fb2c6a07640 1 -- 172.21.15.40:0/1749461103 wait complete. 2024-04-26T21:21:52.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.040+0000 7fb2c6a07640 1 Processor -- start 2024-04-26T21:21:52.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.040+0000 7fb2c6a07640 1 -- start start 2024-04-26T21:21:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c00d0380 0x7fb2c015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c0145b00 0x7fb2c00b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c0070860 con 0x7fb2c00b5f20 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c00706e0 con 0x7fb2c0145b00 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c00709e0 con 0x7fb2c00d0380 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57374/0 (socket says 172.21.15.40:57374) 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 -- 172.21.15.40:0/3959830984 learned_addr learned my addr 172.21.15.40:0/3959830984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 -- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c00d0380 msgr2=0x7fb2c015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c00d0380 0x7fb2c015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 -- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c0145b00 msgr2=0x7fb2c00b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c0145b00 0x7fb2c00b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 -- 172.21.15.40:0/3959830984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c013feb0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bffff640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c420 tx=0x7fb2b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00074e0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b00047e0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004c10 con 0x7fb2c00b5f20 2024-04-26T21:21:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.041+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c013ee10 con 0x7fb2c00b5f20 2024-04-26T21:21:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.042+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c013f6b0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.042+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c013f200 con 0x7fb2c00b5f20 2024-04-26T21:21:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.043+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b0011070 con 0x7fb2c00b5f20 2024-04-26T21:21:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.043+0000 7fb2bcff9640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.043+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fb2b008b4c0 con 0x7fb2c00b5f20 2024-04-26T21:21:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.044+0000 7fb2bf7fe640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.044+0000 7fb2bf7fe640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 0x7fb2a006edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8034000 tx=0x7fb2a802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:52.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.048+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2b0059b70 con 0x7fb2c00b5f20 2024-04-26T21:21:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.234+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2c006a290 con 0x7fb2c00b5f20 2024-04-26T21:21:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.235+0000 7fb2bcff9640 1 -- 172.21.15.40:0/3959830984 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13030 (secure 0 0 0) 0x7fb2b0059470 con 0x7fb2c00b5f20 2024-04-26T21:21:52.236 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:52.236 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":148,"fsid":"eee7569e-7e8d-4976-9855-170ac833730d","created":"2024-04-26T21:11:36.784540+0000","modified":"2024-04-26T21:21:49.113583+0000","last_up_change":"2024-04-26T21:11:47.612550+0000","last_in_change":"2024-04-26T21:11:38.885359+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:11:48.932717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed5d3890-562f-46b3-b113-08a83bd94972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6817","nonce":2954344348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6819","nonce":2954344348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6823","nonce":2954344348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2954344348},{"type":"v1","addr":"172.21.15.40:6821","nonce":2954344348}]},"public_addr":"172.21.15.40:6817/2954344348","cluster_addr":"172.21.15.40:6819/2954344348","heartbeat_back_addr":"172.21.15.40:6823/2954344348","heartbeat_front_addr":"172.21.15.40:6821/2954344348","state":["exists","up"]},{"osd":1,"uuid":"c03230cc-d7f6-411b-b073-2cc11a7b77d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6801","nonce":2244954825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6803","nonce":2244954825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6807","nonce":2244954825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2244954825},{"type":"v1","addr":"172.21.15.40:6805","nonce":2244954825}]},"public_addr":"172.21.15.40:6801/2244954825","cluster_addr":"172.21.15.40:6803/2244954825","heartbeat_back_addr":"172.21.15.40:6807/2244954825","heartbeat_front_addr":"172.21.15.40:6805/2244954825","state":["exists","up"]},{"osd":2,"uuid":"000d1a68-5653-4e07-85cc-c9dfbf4ebba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6825","nonce":3577609501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6827","nonce":3577609501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6831","nonce":3577609501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3577609501},{"type":"v1","addr":"172.21.15.40:6829","nonce":3577609501}]},"public_addr":"172.21.15.40:6825/3577609501","cluster_addr":"172.21.15.40:6827/3577609501","heartbeat_back_addr":"172.21.15.40:6831/3577609501","heartbeat_front_addr":"172.21.15.40:6829/3577609501","state":["exists","up"]},{"osd":3,"uuid":"f4e0d16b-3813-4bbd-8b55-eed7726eb89c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6809","nonce":592958762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6811","nonce":592958762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6815","nonce":592958762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":592958762},{"type":"v1","addr":"172.21.15.40:6813","nonce":592958762}]},"public_addr":"172.21.15.40:6809/592958762","cluster_addr":"172.21.15.40:6811/592958762","heartbeat_back_addr":"172.21.15.40:6815/592958762","heartbeat_front_addr":"172.21.15.40:6813/592958762","state":["exists","up"]},{"osd":4,"uuid":"f1884ac6-1116-4ae6-8df5-baaa810c6939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6817","nonce":1784643509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6819","nonce":1784643509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6823","nonce":1784643509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1784643509},{"type":"v1","addr":"172.21.15.184:6821","nonce":1784643509}]},"public_addr":"172.21.15.184:6817/1784643509","cluster_addr":"172.21.15.184:6819/1784643509","heartbeat_back_addr":"172.21.15.184:6823/1784643509","heartbeat_front_addr":"172.21.15.184:6821/1784643509","state":["exists","up"]},{"osd":5,"uuid":"0254928f-2523-4319-92f6-71fed0af94c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6801","nonce":2962645716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6803","nonce":2962645716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6811","nonce":2962645716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2962645716},{"type":"v1","addr":"172.21.15.184:6807","nonce":2962645716}]},"public_addr":"172.21.15.184:6801/2962645716","cluster_addr":"172.21.15.184:6803/2962645716","heartbeat_back_addr":"172.21.15.184:6811/2962645716","heartbeat_front_addr":"172.21.15.184:6807/2962645716","state":["exists","up"]},{"osd":6,"uuid":"5808a23b-e704-4ab8-a08a-ca416254c458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6825","nonce":3739826890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6827","nonce":3739826890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6831","nonce":3739826890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3739826890},{"type":"v1","addr":"172.21.15.184:6829","nonce":3739826890}]},"public_addr":"172.21.15.184:6825/3739826890","cluster_addr":"172.21.15.184:6827/3739826890","heartbeat_back_addr":"172.21.15.184:6831/3739826890","heartbeat_front_addr":"172.21.15.184:6829/3739826890","state":["exists","up"]},{"osd":7,"uuid":"a490edd1-3e4d-4620-a20b-200d529a06e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6806","nonce":2214404971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6810","nonce":2214404971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6815","nonce":2214404971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2214404971},{"type":"v1","addr":"172.21.15.184:6813","nonce":2214404971}]},"public_addr":"172.21.15.184:6806/2214404971","cluster_addr":"172.21.15.184:6810/2214404971","heartbeat_back_addr":"172.21.15.184:6815/2214404971","heartbeat_front_addr":"172.21.15.184:6813/2214404971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.063901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.763833+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096697+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.943050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.072295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:43.968775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.096274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:11:44.119033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3970947174":"2024-04-27T21:21:46.108875+0000","172.21.15.40:6833/3970947174":"2024-04-27T21:21:46.108875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 msgr2=0x7fb2a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 0x7fb2a006edb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8034000 tx=0x7fb2a802c000 comp rx=0 tx=0).stop 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 msgr2=0x7fb2c0140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000c420 tx=0x7fb2b000c8e0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 shutdown_connections 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2c0145b00 0x7fb2c00b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2c00d0380 0x7fb2c015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 --2- 172.21.15.40:0/3959830984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2c00b5f20 0x7fb2c0140300 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 >> 172.21.15.40:0/3959830984 conn(0x7fb2c00ce8b0 msgr2=0x7fb2c0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 shutdown_connections 2024-04-26T21:21:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:52.237+0000 7fb2c6a07640 1 -- 172.21.15.40:0/3959830984 wait complete. 2024-04-26T21:21:53.250 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T21:21:53.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.366+0000 7fc1f8299640 1 Processor -- start 2024-04-26T21:21:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.366+0000 7fc1f8299640 1 -- start start 2024-04-26T21:21:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1f0162a10 0x7fc1f00d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 0x7fc1f00d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 0x7fc1f0163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f00b4970 con 0x7fc1f0162a10 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f00d33a0 con 0x7fc1f00b6020 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f8299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f00d3680 con 0x7fc1f00d1f70 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f6035640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1f0162a10 0x7fc1f00d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f6035640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1f0162a10 0x7fc1f00d2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57378/0 (socket says 172.21.15.40:57378) 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.367+0000 7fc1f6035640 1 -- 172.21.15.40:0/2963791960 learned_addr learned my addr 172.21.15.40:0/2963791960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f5834640 1 --2- 172.21.15.40:0/2963791960 >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 0x7fc1f00d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6836640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 0x7fc1f0163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6035640 1 -- 172.21.15.40:0/2963791960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 msgr2=0x7fc1f0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 0x7fc1f0163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6035640 1 -- 172.21.15.40:0/2963791960 >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 msgr2=0x7fc1f00d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2963791960 >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 0x7fc1f00d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6035640 1 -- 172.21.15.40:0/2963791960 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f00d3960 con 0x7fc1f0162a10 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f5834640 1 --2- 172.21.15.40:0/2963791960 >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 0x7fc1f00d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.368+0000 7fc1f6836640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 0x7fc1f0163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f0162a10 0x7fc1f00d2dc0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc1f008f0b0 tx=0x7fc1e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710fd48bc85995b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f5033640 1 -- 172.21.15.40:0/2963791960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8037070 con 0x7fc1f0162a10 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f5033640 1 -- 172.21.15.40:0/2963791960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e8032440 con 0x7fc1f0162a10 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f5033640 1 -- 172.21.15.40:0/2963791960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8032810 con 0x7fc1f0162a10 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 -- 172.21.15.40:0/2963791960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f0162a10 msgr2=0x7fc1f00d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f0162a10 0x7fc1f00d2dc0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc1f008f0b0 tx=0x7fc1e802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 -- 172.21.15.40:0/2963791960 shutdown_connections 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f00d1f70 0x7fc1f0163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2963791960 >> v2:172.21.15.184:3300/0 conn(0x7fc1f00b6020 0x7fc1f00d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2963791960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f0162a10 0x7fc1f00d2dc0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 -- 172.21.15.40:0/2963791960 >> 172.21.15.40:0/2963791960 conn(0x7fc1f009e530 msgr2=0x7fc1f00d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 -- 172.21.15.40:0/2963791960 shutdown_connections 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.369+0000 7fc1f8299640 1 -- 172.21.15.40:0/2963791960 wait complete. 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 Processor -- start 2024-04-26T21:21:53.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 -- start start 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f00b6020 0x7fc1f015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 0x7fc1f00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f0162a10 0x7fc1f00b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0162df0 con 0x7fc1f00d1f70 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f00d3f20 con 0x7fc1f00b6020 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.371+0000 7fc1f8299640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f00b6600 con 0x7fc1f0162a10 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 0x7fc1f00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 0x7fc1f00b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57386/0 (socket says 172.21.15.40:57386) 2024-04-26T21:21:53.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 -- 172.21.15.40:0/2285411542 learned_addr learned my addr 172.21.15.40:0/2285411542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f00b6020 0x7fc1f015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f6836640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f0162a10 0x7fc1f00b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 -- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f0162a10 msgr2=0x7fc1f00b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1f0162a10 0x7fc1f00b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 -- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f00b6020 msgr2=0x7fc1f015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f00b6020 0x7fc1f015c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f015c5a0 con 0x7fc1f00d1f70 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f5834640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 0x7fc1f00b2900 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d800 tx=0x7fc1e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0018070 con 0x7fc1f00d1f70 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e0004030 con 0x7fc1f00d1f70 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f00b90b0 con 0x7fc1f00d1f70 2024-04-26T21:21:53.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0004400 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.372+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f00b9670 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.373+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1f00b8e60 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.374+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1e00096a0 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.374+0000 7fc1e6ffd640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1d406c8f0 0x7fc1d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.374+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fc1e008c490 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.374+0000 7fc1e6ffd640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7fc1d40727b0 0x7fc1d4074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1e000fe50 con 0x7fc1d40727b0 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fc1e008c920 con 0x7fc1f00d1f70 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1d406c8f0 0x7fc1d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1f6836640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7fc1d40727b0 0x7fc1d4074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1f6836640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7fc1d40727b0 0x7fc1d4074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.375+0000 7fc1f6035640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1d406c8f0 0x7fc1d406edb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8034000 tx=0x7fc1e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.376+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== osd.0 v2:172.21.15.40:6816/2954344348 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1e000fe50 con 0x7fc1d40727b0 2024-04-26T21:21:53.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.389+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 --> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc1f006a290 con 0x7fc1d40727b0 2024-04-26T21:21:53.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.389+0000 7fc1e6ffd640 1 -- 172.21.15.40:0/2285411542 <== osd.0 v2:172.21.15.40:6816/2954344348 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc1f006a290 con 0x7fc1d40727b0 2024-04-26T21:21:53.391 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:53.391 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:53.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.389+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7fc1d40727b0 msgr2=0x7fc1d4074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:6816/2954344348,v1:172.21.15.40:6817/2954344348] conn(0x7fc1d40727b0 0x7fc1d4074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1d406c8f0 msgr2=0x7fc1d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fc1d406c8f0 0x7fc1d406edb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8034000 tx=0x7fc1e802e000 comp rx=0 tx=0).stop 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 msgr2=0x7fc1f00b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 --2- 172.21.15.40:0/2285411542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1f00d1f70 0x7fc1f00b2900 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d800 tx=0x7fc1e000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f6836640 1 -- 172.21.15.40:0/2285411542 reap_dead start 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 shutdown_connections 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 >> 172.21.15.40:0/2285411542 conn(0x7fc1f009e530 msgr2=0x7fc1f006e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 shutdown_connections 2024-04-26T21:21:53.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.390+0000 7fc1f8299640 1 -- 172.21.15.40:0/2285411542 wait complete. 2024-04-26T21:21:53.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T21:21:53.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.503+0000 7f35a9d0a640 1 Processor -- start 2024-04-26T21:21:53.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.503+0000 7f35a9d0a640 1 -- start start 2024-04-26T21:21:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a9d0a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35a406e490 0x7f35a415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a9d0a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a40d1740 con 0x7f35a40cf2e0 2024-04-26T21:21:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a9d0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a40d1a20 con 0x7f35a40d11b0 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a9d0a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f35a40d1d00 con 0x7f35a406e490 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f35a40cf2e0 0x7f35a406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60124/0 (socket says 172.21.15.40:60124) 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a37fe640 1 -- 172.21.15.40:0/1455252122 learned_addr learned my addr 172.21.15.40:0/1455252122 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35a406e490 0x7f35a415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a3fff640 1 -- 172.21.15.40:0/1455252122 >> v1:172.21.15.184:6789/0 conn(0x7f35a40d11b0 legacy=0x7f35a406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a3fff640 1 -- 172.21.15.40:0/1455252122 >> v1:172.21.15.40:6789/0 conn(0x7f35a40cf2e0 legacy=0x7f35a406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a27fc640 1 -- 172.21.15.40:0/1455252122 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407583964 0 0) 0x7f35a40d1740 con 0x7f35a40cf2e0 2024-04-26T21:21:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.504+0000 7f35a3fff640 1 -- 172.21.15.40:0/1455252122 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a40d1fe0 con 0x7f35a406e490 2024-04-26T21:21:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.505+0000 7f35a3fff640 1 --2- 172.21.15.40:0/1455252122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a406e490 0x7f35a415ebe0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f359c009dc0 tx=0x7f359c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588222e2ad79a001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.505+0000 7f35a27fc640 1 -- 172.21.15.40:0/1455252122 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c007740 con 0x7f35a406e490 2024-04-26T21:21:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.505+0000 7f35a27fc640 1 -- 172.21.15.40:0/1455252122 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c0078e0 con 0x7f35a406e490 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.505+0000 7f35a27fc640 1 -- 172.21.15.40:0/1455252122 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c007cb0 con 0x7f35a406e490 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 -- 172.21.15.40:0/1455252122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a406e490 msgr2=0x7f35a415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/1455252122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a406e490 0x7f35a415ebe0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f359c009dc0 tx=0x7f359c02a730 comp rx=0 tx=0).stop 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 -- 172.21.15.40:0/1455252122 shutdown_connections 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/1455252122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a406e490 0x7f35a415ebe0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 -- 172.21.15.40:0/1455252122 >> 172.21.15.40:0/1455252122 conn(0x7f35a40b4a60 msgr2=0x7f35a40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 -- 172.21.15.40:0/1455252122 shutdown_connections 2024-04-26T21:21:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.506+0000 7f35a9d0a640 1 -- 172.21.15.40:0/1455252122 wait complete. 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.507+0000 7f35a9d0a640 1 Processor -- start 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.507+0000 7f35a9d0a640 1 -- start start 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 0x7f35a409dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 0x7f35a4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35a40cf840 con 0x7f35a40d11b0 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35a40cf6c0 con 0x7f35a40cf2e0 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a9d0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f35a40ce660 con 0x7f35a406e490 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57408/0 (socket says 172.21.15.40:57408) 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a3fff640 1 -- 172.21.15.40:0/16331965 learned_addr learned my addr 172.21.15.40:0/16331965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a37fe640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 0x7f35a409dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.508+0000 7f35a2ffd640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 0x7f35a4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 -- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 msgr2=0x7f35a409dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 0x7f35a409dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 -- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 msgr2=0x7f35a4154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 0x7f35a4154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 -- 172.21.15.40:0/16331965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a409dff0 con 0x7f35a40d11b0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a2ffd640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 0x7f35a4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a37fe640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 0x7f35a409dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a3fff640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f359c02ad20 tx=0x7f359c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c040070 con 0x7f35a40d11b0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c031c40 con 0x7f35a40d11b0 2024-04-26T21:21:53.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a409efd0 con 0x7f35a40d11b0 2024-04-26T21:21:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c003990 con 0x7f35a40d11b0 2024-04-26T21:21:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.509+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a409d2b0 con 0x7f35a40d11b0 2024-04-26T21:21:53.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.510+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a406a290 con 0x7f35a40d11b0 2024-04-26T21:21:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.511+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f359c039040 con 0x7f35a40d11b0 2024-04-26T21:21:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.511+0000 7f358bfff640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 0x7f359006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.511+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f359c0acb10 con 0x7f35a40d11b0 2024-04-26T21:21:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.511+0000 7f35a37fe640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 0x7f359006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.512+0000 7f35a37fe640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 0x7f359006ede0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f60 tx=0x7f3594007c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.516+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f359c07b2d0 con 0x7f35a40d11b0 2024-04-26T21:21:53.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.700+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f35a40b8ef0 con 0x7f359006c920 2024-04-26T21:21:53.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.700+0000 7f358bfff640 1 -- 172.21.15.40:0/16331965 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f35a40b8ef0 con 0x7f359006c920 2024-04-26T21:21:53.701 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T21:21:53.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 msgr2=0x7f359006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 0x7f359006ede0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f60 tx=0x7f3594007c50 comp rx=0 tx=0).stop 2024-04-26T21:21:53.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 msgr2=0x7f35a409e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f359c02ad20 tx=0x7f359c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 shutdown_connections 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f359006c920 0x7f359006ede0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a40d11b0 0x7f35a409e530 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35a40cf2e0 0x7f35a4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 --2- 172.21.15.40:0/16331965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f35a406e490 0x7f35a409dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 >> 172.21.15.40:0/16331965 conn(0x7f35a40b4a60 msgr2=0x7f35a4162ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 shutdown_connections 2024-04-26T21:21:53.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.702+0000 7f35a9d0a640 1 -- 172.21.15.40:0/16331965 wait complete. 2024-04-26T21:21:53.714 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T21:21:53.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:53.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.818+0000 7f897ddb4640 1 Processor -- start 2024-04-26T21:21:53.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.818+0000 7f897ddb4640 1 -- start start 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f897ddb4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89780cee20 0x7f897806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f897ddb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f897809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f897ddb4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89780d1240 con 0x7f897809bf80 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f897ddb4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f89780d1e10 con 0x7f89780cee20 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f897ddb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89780d14e0 con 0x7f89780d3a30 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f8977fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f897809bf80 0x7f89781630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60132/0 (socket says 172.21.15.40:60132) 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f8977fff640 1 -- 172.21.15.40:0/3594530383 learned_addr learned my addr 172.21.15.40:0/3594530383 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89777fe640 1 --2- 172.21.15.40:0/3594530383 >> v2:172.21.15.184:3300/0 conn(0x7f89780cee20 0x7f897806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f8976ffd640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f897809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752821405 0 0) 0x7f89780d1240 con 0x7f897809bf80 2024-04-26T21:21:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8960003520 con 0x7f897809bf80 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747778 0 0) 0x7f8960003520 con 0x7f897809bf80 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 msgr2=0x7f897809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f897809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 >> v2:172.21.15.184:3300/0 conn(0x7f89780cee20 msgr2=0x7f897806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 --2- 172.21.15.40:0/3594530383 >> v2:172.21.15.184:3300/0 conn(0x7f89780cee20 0x7f897806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.819+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780d17c0 con 0x7f897809bf80 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f89700019f0 con 0x7f897809bf80 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 >> v1:172.21.15.40:6789/0 conn(0x7f897809bf80 legacy=0x7f89781630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89600039b0 0x7f8960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f8960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f89600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89780d1e10 con 0x7f8960004800 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89780d14e0 con 0x7f89600039b0 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f896000b9c0 con 0x7f89780d3a30 2024-04-26T21:21:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f8960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f89777fe640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89600039b0 0x7f8960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8977fff640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f89600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 -- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 msgr2=0x7f89600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f89600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 -- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89600039b0 msgr2=0x7f8960003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89600039b0 0x7f8960003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8976ffd640 1 -- 172.21.15.40:0/3594530383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8960003520 con 0x7f8960004800 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.820+0000 7f8977fff640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89780d3a30 0x7f89600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f8976ffd640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f8960004c80 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f896c009ac0 tx=0x7f8960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3397e54ba5b0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c036070 con 0x7f8960004800 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f896c031400 con 0x7f8960004800 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f89767fc640 1 -- 172.21.15.40:0/3594530383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c009690 con 0x7f8960004800 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 -- 172.21.15.40:0/3594530383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 msgr2=0x7f8960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 --2- 172.21.15.40:0/3594530383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f8960004c80 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f896c009ac0 tx=0x7f8960005830 comp rx=0 tx=0).stop 2024-04-26T21:21:53.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f8977fff640 1 -- 172.21.15.40:0/3594530383 reap_dead start 2024-04-26T21:21:53.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 -- 172.21.15.40:0/3594530383 shutdown_connections 2024-04-26T21:21:53.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 -- 172.21.15.40:0/3594530383 >> 172.21.15.40:0/3594530383 conn(0x7f89780d2f10 msgr2=0x7f89780d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 -- 172.21.15.40:0/3594530383 shutdown_connections 2024-04-26T21:21:53.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.821+0000 7f897ddb4640 1 -- 172.21.15.40:0/3594530383 wait complete. 2024-04-26T21:21:53.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 Processor -- start 2024-04-26T21:21:53.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 -- start start 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f89780b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89780cac80 0x7f897809f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f897809fe80 0x7f8978155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89780cf200 con 0x7f8960004800 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f897809c7a0 con 0x7f89780cac80 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.823+0000 7f897ddb4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f89780cf380 con 0x7f897809fe80 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f89780b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f89780b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57428/0 (socket says 172.21.15.40:57428) 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 -- 172.21.15.40:0/2403651227 learned_addr learned my addr 172.21.15.40:0/2403651227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f8976ffd640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89780cac80 0x7f897809f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f8977fff640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f897809fe80 0x7f8978155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 -- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f897809fe80 msgr2=0x7f8978155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f897809fe80 0x7f8978155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 -- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89780cac80 msgr2=0x7f897809f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89780cac80 0x7f897809f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780a0350 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f8976ffd640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89780cac80 0x7f897809f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f89777fe640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f89780b6980 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8968011a70 tx=0x7f8968011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896800fb50 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f896800fcf0 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89780ca1a0 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896800fea0 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.824+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89780b6ec0 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.825+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89780c9fe0 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8968012040 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c8f0 0x7f896406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f896808bf50 con 0x7f8960004800 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f89640727b0 0x7f8964074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8968025ec0 con 0x7f89640727b0 2024-04-26T21:21:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f896808e5f0 con 0x7f8960004800 2024-04-26T21:21:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f8976ffd640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c8f0 0x7f896406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.826+0000 7f8977fff640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f89640727b0 0x7f8964074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.827+0000 7f8977fff640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f89640727b0 0x7f8964074bd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.827+0000 7f8976ffd640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c8f0 0x7f896406edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f896c033000 tx=0x7f896c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.827+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== osd.1 v2:172.21.15.40:6800/2244954825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8968025ec0 con 0x7f89640727b0 2024-04-26T21:21:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 --> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f897806a290 con 0x7f89640727b0 2024-04-26T21:21:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f895ffff640 1 -- 172.21.15.40:0/2403651227 <== osd.1 v2:172.21.15.40:6800/2244954825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f897806a290 con 0x7f89640727b0 2024-04-26T21:21:53.842 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:53.842 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:53.842 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f89640727b0 msgr2=0x7f8964074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:6800/2244954825,v1:172.21.15.40:6801/2244954825] conn(0x7f89640727b0 0x7f8964074bd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c8f0 msgr2=0x7f896406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f896406c8f0 0x7f896406edb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f896c033000 tx=0x7f896c008030 comp rx=0 tx=0).stop 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 msgr2=0x7f89780b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 --2- 172.21.15.40:0/2403651227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960004800 0x7f89780b6980 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8968011a70 tx=0x7f8968011f30 comp rx=0 tx=0).stop 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f8977fff640 1 -- 172.21.15.40:0/2403651227 reap_dead start 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 shutdown_connections 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 >> 172.21.15.40:0/2403651227 conn(0x7f89780d2f10 msgr2=0x7f89780cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 shutdown_connections 2024-04-26T21:21:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.841+0000 7f897ddb4640 1 -- 172.21.15.40:0/2403651227 wait complete. 2024-04-26T21:21:53.853 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T21:21:53.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.954+0000 7fa4d1693640 1 Processor -- start 2024-04-26T21:21:53.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.954+0000 7fa4d1693640 1 -- start start 2024-04-26T21:21:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4d1693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4cc073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4d1693640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc15e2e0 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4d1693640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc160d70 con 0x7fa4cc161850 2024-04-26T21:21:53.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4d1693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc15e5c0 con 0x7fa4cc146980 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4cb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4cc15ddc0 0x7fa4cc0b6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60146/0 (socket says 172.21.15.40:60146) 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/3380399711 learned_addr learned my addr 172.21.15.40:0/3380399711 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4caffd640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4cc073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817274397 0 0) 0x7fa4cc15e2e0 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b8003520 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4221152151 0 0) 0x7fa4b8003520 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 msgr2=0x7fa4cc073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4cc073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 >> v1:172.21.15.184:6789/0 conn(0x7fa4cc161850 legacy=0x7fa4cc15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc161050 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7fa4c40019f0 con 0x7fa4cc15ddc0 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 >> v1:172.21.15.40:6789/0 conn(0x7fa4cc15ddc0 legacy=0x7fa4cc0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b80039e0 0x7fa4b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b8004800 0x7fa4b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc15e5c0 con 0x7fa4b80039e0 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b800ba70 con 0x7fa4b8004800 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.955+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b800bc20 con 0x7fa4cc146980 2024-04-26T21:21:53.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b80039e0 0x7fa4b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4cb7fe640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4c3fff640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b8004800 0x7fa4b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 -- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 msgr2=0x7fa4b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc146980 0x7fa4b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 -- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b8004800 msgr2=0x7fa4b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b8004800 0x7fa4b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.956+0000 7fa4caffd640 1 -- 172.21.15.40:0/3380399711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8003520 con 0x7fa4b80039e0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4caffd640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b80039e0 0x7fa4b8003dc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009a90 tx=0x7fa4b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9581ebdeb378c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc036070 con 0x7fa4b80039e0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc031400 con 0x7fa4b80039e0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4ca7fc640 1 -- 172.21.15.40:0/3380399711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc009690 con 0x7fa4b80039e0 2024-04-26T21:21:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 -- 172.21.15.40:0/3380399711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b80039e0 msgr2=0x7fa4b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 --2- 172.21.15.40:0/3380399711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b80039e0 0x7fa4b8003dc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009a90 tx=0x7fa4b8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/3380399711 reap_dead start 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 -- 172.21.15.40:0/3380399711 shutdown_connections 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 -- 172.21.15.40:0/3380399711 >> 172.21.15.40:0/3380399711 conn(0x7fa4cc0ce8b0 msgr2=0x7fa4cc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 -- 172.21.15.40:0/3380399711 shutdown_connections 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.957+0000 7fa4d1693640 1 -- 172.21.15.40:0/3380399711 wait complete. 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.958+0000 7fa4d1693640 1 Processor -- start 2024-04-26T21:21:53.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 -- start start 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b80039e0 0x7fa4cc140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 0x7fa4cc0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc146d60 con 0x7fa4cc0cb720 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc1613c0 con 0x7fa4b80039e0 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4d1693640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b7160 con 0x7fa4cc154500 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57454/0 (socket says 172.21.15.40:57454) 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/1049936872 learned_addr learned my addr 172.21.15.40:0/1049936872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.959+0000 7fa4caffd640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b80039e0 0x7fa4cc140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4c3fff640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 0x7fa4cc0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 msgr2=0x7fa4cc0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 0x7fa4cc0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b80039e0 msgr2=0x7fa4cc140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b80039e0 0x7fa4cc140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 -- 172.21.15.40:0/1049936872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc13fcb0 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4c3fff640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 0x7fa4cc0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4cb7fe640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0721f0 tx=0x7fa4c400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4017020 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c400aca0 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc13ff90 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40135f0 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.960+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc140550 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.961+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc06a290 con 0x7fa4cc0cb720 2024-04-26T21:21:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.961+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4c401a020 con 0x7fa4cc0cb720 2024-04-26T21:21:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.962+0000 7fa4c37fe640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.962+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fa4c408c0d0 con 0x7fa4cc0cb720 2024-04-26T21:21:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.962+0000 7fa4caffd640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.963+0000 7fa4caffd640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc033000 tx=0x7fa4bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:53.966+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4c4021020 con 0x7fa4cc0cb720 2024-04-26T21:21:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.150+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa4cc143ff0 con 0x7fa4ac06c8f0 2024-04-26T21:21:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.151+0000 7fa4c37fe640 1 -- 172.21.15.40:0/1049936872 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4cc143ff0 con 0x7fa4ac06c8f0 2024-04-26T21:21:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 msgr2=0x7fa4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc033000 tx=0x7fa4bc008030 comp rx=0 tx=0).stop 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 msgr2=0x7fa4cc0cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0721f0 tx=0x7fa4c400a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 shutdown_connections 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0cb720 0x7fa4cc0cbec0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4cc154500 0x7fa4cc0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 --2- 172.21.15.40:0/1049936872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4b80039e0 0x7fa4cc140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 >> 172.21.15.40:0/1049936872 conn(0x7fa4cc0ce8b0 msgr2=0x7fa4cc15e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 shutdown_connections 2024-04-26T21:21:54.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.153+0000 7fa4d1693640 1 -- 172.21.15.40:0/1049936872 wait complete. 2024-04-26T21:21:54.164 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T21:21:54.165 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.268+0000 7f782706d640 1 Processor -- start 2024-04-26T21:21:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 -- start start 2024-04-26T21:21:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 0x7f7820144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78201612b0 con 0x7f78200d1190 2024-04-26T21:21:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f7820073440 con 0x7f78201437f0 2024-04-26T21:21:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.269+0000 7f782706d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7820161570 con 0x7f7820160e70 2024-04-26T21:21:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f7824e09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 0x7f7820144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f7824e09640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.40:36486/0 (socket says 172.21.15.40:36486) 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f7824e09640 1 -- 172.21.15.40:0/2578956805 learned_addr learned my addr 172.21.15.40:0/2578956805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f782560a640 1 --1- 172.21.15.40:0/2578956805 >> v1:172.21.15.184:6789/0 conn(0x7f78200d1190 0x7f7820073bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55270/0 (socket says 172.21.15.40:55270) 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781ffff640 1 -- 172.21.15.40:0/2578956805 >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 msgr2=0x7f782006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781ffff640 1 --2- 172.21.15.40:0/2578956805 >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781ffff640 1 -- 172.21.15.40:0/2578956805 >> v1:172.21.15.184:6789/0 conn(0x7f78200d1190 legacy=0x7f7820073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781f7fe640 1 -- 172.21.15.40:0/2578956805 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529192097 0 0) 0x7f78201612b0 con 0x7f78200d1190 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f781ffff640 1 -- 172.21.15.40:0/2578956805 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820161850 con 0x7f7820160e70 2024-04-26T21:21:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.270+0000 7f7824e09640 1 --2- 172.21.15.40:0/2578956805 >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.271+0000 7f781ffff640 1 --2- 172.21.15.40:0/2578956805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 0x7f7820144120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd80d223f2c89e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.271+0000 7f781f7fe640 1 -- 172.21.15.40:0/2578956805 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c037070 con 0x7f7820160e70 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.271+0000 7f781f7fe640 1 -- 172.21.15.40:0/2578956805 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c032440 con 0x7f7820160e70 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.271+0000 7f781f7fe640 1 -- 172.21.15.40:0/2578956805 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c032810 con 0x7f7820160e70 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 -- 172.21.15.40:0/2578956805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 msgr2=0x7f7820144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 --2- 172.21.15.40:0/2578956805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 0x7f7820144120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 -- 172.21.15.40:0/2578956805 shutdown_connections 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 --2- 172.21.15.40:0/2578956805 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7820160e70 0x7f7820144120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 --2- 172.21.15.40:0/2578956805 >> v2:172.21.15.184:3300/0 conn(0x7f78201437f0 0x7f782006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 -- 172.21.15.40:0/2578956805 >> 172.21.15.40:0/2578956805 conn(0x7f782006c690 msgr2=0x7f782006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 -- 172.21.15.40:0/2578956805 shutdown_connections 2024-04-26T21:21:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.272+0000 7f782706d640 1 -- 172.21.15.40:0/2578956805 wait complete. 2024-04-26T21:21:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.273+0000 7f782706d640 1 Processor -- start 2024-04-26T21:21:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.273+0000 7f782706d640 1 -- start start 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78200d1190 0x7f782014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 0x7f78201e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7820160e70 0x7f782013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7820144660 con 0x7f78201437f0 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78200736a0 con 0x7f7820160e70 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782706d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f78201447e0 con 0x7f78200d1190 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 0x7f78201e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 0x7f78201e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57464/0 (socket says 172.21.15.40:57464) 2024-04-26T21:21:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 -- 172.21.15.40:0/2694934361 learned_addr learned my addr 172.21.15.40:0/2694934361 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f7824e09640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78200d1190 0x7f782014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782560a640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7820160e70 0x7f782013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 -- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78200d1190 msgr2=0x7f782014f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78200d1190 0x7f782014f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 -- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7820160e70 msgr2=0x7f782013ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7820160e70 0x7f782013ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f781ffff640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782014f740 con 0x7f78201437f0 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f782560a640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7820160e70 0x7f782013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.274+0000 7f7824e09640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78200d1190 0x7f782014f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f781ffff640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 0x7f78201e7b60 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c037050 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c038040 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78201e8630 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c004490 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.275+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201f37d0 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.276+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f782006a290 con 0x7f78201437f0 2024-04-26T21:21:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.276+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f780c03c6d0 con 0x7f78201437f0 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f781d7fa640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f780406c8f0 0x7f780406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f780c0ac3e0 con 0x7f78201437f0 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f781d7fa640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f78040727b0 0x7f7804074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f780c008eb0 con 0x7f78040727b0 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f780c0b3050 con 0x7f78201437f0 2024-04-26T21:21:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f7824e09640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f780406c8f0 0x7f780406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f782560a640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f78040727b0 0x7f7804074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.277+0000 7f782560a640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f78040727b0 0x7f7804074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.278+0000 7f7824e09640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f780406c8f0 0x7f780406edb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7814009630 tx=0x7f7814008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.278+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== osd.2 v2:172.21.15.40:6824/3577609501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f780c008eb0 con 0x7f78040727b0 2024-04-26T21:21:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.291+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 --> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f782006eb80 con 0x7f78040727b0 2024-04-26T21:21:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f781d7fa640 1 -- 172.21.15.40:0/2694934361 <== osd.2 v2:172.21.15.40:6824/3577609501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f782006eb80 con 0x7f78040727b0 2024-04-26T21:21:54.293 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:54.293 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f78040727b0 msgr2=0x7f7804074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:6824/3577609501,v1:172.21.15.40:6825/3577609501] conn(0x7f78040727b0 0x7f7804074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f780406c8f0 msgr2=0x7f780406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f780406c8f0 0x7f780406edb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7814009630 tx=0x7f7814008040 comp rx=0 tx=0).stop 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 msgr2=0x7f78201e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 --2- 172.21.15.40:0/2694934361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78201437f0 0x7f78201e7b60 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782560a640 1 -- 172.21.15.40:0/2694934361 reap_dead start 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 shutdown_connections 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 >> 172.21.15.40:0/2694934361 conn(0x7f782006c690 msgr2=0x7f782006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 shutdown_connections 2024-04-26T21:21:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.292+0000 7f782706d640 1 -- 172.21.15.40:0/2694934361 wait complete. 2024-04-26T21:21:54.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T21:21:54.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.404+0000 7ff9ab7af640 1 Processor -- start 2024-04-26T21:21:54.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.404+0000 7ff9ab7af640 1 -- start start 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9ab7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9ab7af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9a4160d50 0x7ff9a41611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9ab7af640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a40d0be0 con 0x7ff9a40b56d0 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9ab7af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a40d0ec0 con 0x7ff9a4160d50 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9ab7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a40d11a0 con 0x7ff9a415d380 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9a9d4c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9a4160d50 0x7ff9a41611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9a954b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9a8d4a640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff9a40b56d0 0x7ff9a40d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.40:55272/0 (socket says 172.21.15.40:55272) 2024-04-26T21:21:54.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.405+0000 7ff9a8d4a640 1 -- 172.21.15.40:0/3873603022 learned_addr learned my addr 172.21.15.40:0/3873603022 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 msgr2=0x7ff9a40ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff9a9d4c640 1 --2- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/3873603022 >> v1:172.21.15.184:6789/0 conn(0x7ff9a40b56d0 legacy=0x7ff9a40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/3873603022 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a40d1480 con 0x7ff9a4160d50 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff99bfff640 1 -- 172.21.15.40:0/3873603022 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272514650 0 0) 0x7ff9a40d0be0 con 0x7ff9a40b56d0 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.406+0000 7ff9a9d4c640 1 --2- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a4160d50 0x7ff9a41611e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009d90 tx=0x7ff99c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae3291ee3b719f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff99bfff640 1 -- 172.21.15.40:0/3873603022 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c031600 con 0x7ff9a4160d50 2024-04-26T21:21:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff99bfff640 1 -- 172.21.15.40:0/3873603022 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c0317a0 con 0x7ff9a4160d50 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff99bfff640 1 -- 172.21.15.40:0/3873603022 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c031b70 con 0x7ff9a4160d50 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 -- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a4160d50 msgr2=0x7ff9a41611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a4160d50 0x7ff9a41611e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009d90 tx=0x7ff99c02a5d0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 -- 172.21.15.40:0/3873603022 shutdown_connections 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a4160d50 0x7ff9a41611e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/3873603022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 -- 172.21.15.40:0/3873603022 >> 172.21.15.40:0/3873603022 conn(0x7ff9a40b4a60 msgr2=0x7ff9a40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 -- 172.21.15.40:0/3873603022 shutdown_connections 2024-04-26T21:21:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.407+0000 7ff9ab7af640 1 -- 172.21.15.40:0/3873603022 wait complete. 2024-04-26T21:21:54.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.408+0000 7ff9ab7af640 1 Processor -- start 2024-04-26T21:21:54.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.408+0000 7ff9ab7af640 1 -- start start 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 0x7ff9a40ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a40b5dc0 con 0x7ff9a4160d50 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a40cedc0 con 0x7ff9a40b56d0 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9ab7af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a40b5f40 con 0x7ff9a415d380 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9a9d4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9a9d4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57472/0 (socket says 172.21.15.40:57472) 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/1570457388 learned_addr learned my addr 172.21.15.40:0/1570457388 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9a954b640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 0x7ff9a40ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.409+0000 7ff9a8d4a640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 msgr2=0x7ff9a40cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 msgr2=0x7ff9a40ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 0x7ff9a40ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 -- 172.21.15.40:0/1570457388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a40ccf60 con 0x7ff9a4160d50 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a954b640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 0x7ff9a40ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a8d4a640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9a9d4c640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009eb0 tx=0x7ff99c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c040070 con 0x7ff9a4160d50 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c030b60 con 0x7ff9a4160d50 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a40cc9a0 con 0x7ff9a4160d50 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c009450 con 0x7ff9a4160d50 2024-04-26T21:21:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.410+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a40c76c0 con 0x7ff9a4160d50 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.411+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a406a290 con 0x7ff9a4160d50 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.412+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff99c039040 con 0x7ff9a4160d50 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.412+0000 7ff999ffb640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 0x7ff98806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.412+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7ff99c0aca60 con 0x7ff9a4160d50 2024-04-26T21:21:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.412+0000 7ff9a954b640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 0x7ff98806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.413+0000 7ff9a954b640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 0x7ff98806edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009ac0 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.416+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff99c07b220 con 0x7ff9a4160d50 2024-04-26T21:21:54.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.603+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ff9a409c8d0 con 0x7ff98806c8f0 2024-04-26T21:21:54.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.603+0000 7ff999ffb640 1 -- 172.21.15.40:0/1570457388 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff9a409c8d0 con 0x7ff98806c8f0 2024-04-26T21:21:54.604 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T21:21:54.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 msgr2=0x7ff98806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 0x7ff98806edb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009ac0 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 msgr2=0x7ff9a40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009eb0 tx=0x7ff99c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 shutdown_connections 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff98806c8f0 0x7ff98806edb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9a4160d50 0x7ff9a40cf9f0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9a415d380 0x7ff9a40cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 --2- 172.21.15.40:0/1570457388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a40b56d0 0x7ff9a40ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 >> 172.21.15.40:0/1570457388 conn(0x7ff9a40b4a60 msgr2=0x7ff9a4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 shutdown_connections 2024-04-26T21:21:54.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.605+0000 7ff9ab7af640 1 -- 172.21.15.40:0/1570457388 wait complete. 2024-04-26T21:21:54.617 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T21:21:54.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.722+0000 7f49e13e2640 1 Processor -- start 2024-04-26T21:21:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.722+0000 7f49e13e2640 1 -- start start 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49e13e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49dc0cee20 0x7f49dc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49e13e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49dc09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49e13e2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49dc0d1240 con 0x7f49dc09bf80 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49e13e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f49dc0d1e10 con 0x7f49dc0cee20 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49e13e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49dc0d14e0 con 0x7f49dc0d3a30 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49db7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49dc09bf80 0x7f49dc1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60148/0 (socket says 172.21.15.40:60148) 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49db7fe640 1 -- 172.21.15.40:0/64060483 learned_addr learned my addr 172.21.15.40:0/64060483 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49daffd640 1 --2- 172.21.15.40:0/64060483 >> v2:172.21.15.184:3300/0 conn(0x7f49dc0cee20 0x7f49dc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49da7fc640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49dc09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297379094 0 0) 0x7f49dc0d1240 con 0x7f49dc09bf80 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.723+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49c8003520 con 0x7f49dc09bf80 2024-04-26T21:21:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2323046176 0 0) 0x7f49c8003520 con 0x7f49dc09bf80 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 msgr2=0x7f49dc09ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49dc09ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 >> v2:172.21.15.184:3300/0 conn(0x7f49dc0cee20 msgr2=0x7f49dc06d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 --2- 172.21.15.40:0/64060483 >> v2:172.21.15.184:3300/0 conn(0x7f49dc0cee20 0x7f49dc06d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc0d17c0 con 0x7f49dc09bf80 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f49cc0019f0 con 0x7f49dc09bf80 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49da7fc640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49dc09ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 >> v1:172.21.15.40:6789/0 conn(0x7f49dc09bf80 legacy=0x7f49dc1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49c80039b0 0x7f49c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c8004800 0x7f49c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0d1e10 con 0x7f49c80039b0 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0d14e0 con 0x7f49c8004800 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49c800bad0 con 0x7f49dc0d3a30 2024-04-26T21:21:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49c80039b0 0x7f49c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49da7fc640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c8004800 0x7f49c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49db7fe640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 -- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 msgr2=0x7f49c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 -- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c8004800 msgr2=0x7f49c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c8004800 0x7f49c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49daffd640 1 -- 172.21.15.40:0/64060483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c8003520 con 0x7f49c80039b0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.724+0000 7f49db7fe640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc0d3a30 0x7f49c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49daffd640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49c80039b0 0x7f49c8003dd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ba30 tx=0x7f49c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f4298529d45dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4030070 con 0x7f49c80039b0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d402ae10 con 0x7f49c80039b0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49d9ffb640 1 -- 172.21.15.40:0/64060483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d400b600 con 0x7f49c80039b0 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49e13e2640 1 -- 172.21.15.40:0/64060483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49c80039b0 msgr2=0x7f49c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49e13e2640 1 --2- 172.21.15.40:0/64060483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49c80039b0 0x7f49c8003dd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ba30 tx=0x7f49c8005830 comp rx=0 tx=0).stop 2024-04-26T21:21:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.725+0000 7f49db7fe640 1 -- 172.21.15.40:0/64060483 reap_dead start 2024-04-26T21:21:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.726+0000 7f49e13e2640 1 -- 172.21.15.40:0/64060483 shutdown_connections 2024-04-26T21:21:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.726+0000 7f49e13e2640 1 -- 172.21.15.40:0/64060483 >> 172.21.15.40:0/64060483 conn(0x7f49dc0d2f10 msgr2=0x7f49dc0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.726+0000 7f49e13e2640 1 -- 172.21.15.40:0/64060483 shutdown_connections 2024-04-26T21:21:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.726+0000 7f49e13e2640 1 -- 172.21.15.40:0/64060483 wait complete. 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.727+0000 7f49e13e2640 1 Processor -- start 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.727+0000 7f49e13e2640 1 -- start start 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c80039b0 0x7f49dc14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc1586a0 0x7f49dc15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 0x7f49dc161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0cf200 con 0x7f49dc15c8f0 2024-04-26T21:21:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc09c7a0 con 0x7f49c80039b0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f49dc0cf380 con 0x7f49dc1586a0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 0x7f49dc161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 0x7f49dc161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57486/0 (socket says 172.21.15.40:57486) 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 -- 172.21.15.40:0/159590808 learned_addr learned my addr 172.21.15.40:0/159590808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49da7fc640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc1586a0 0x7f49dc15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49daffd640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c80039b0 0x7f49dc14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 -- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc1586a0 msgr2=0x7f49dc15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49dc1586a0 0x7f49dc15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 -- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c80039b0 msgr2=0x7f49dc14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49c80039b0 0x7f49dc14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc15e690 con 0x7f49dc15c8f0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49db7fe640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 0x7f49dc161250 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f49dc073960 tx=0x7f49cc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00eda0 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc018070 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc15d0b0 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc013660 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.728+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc15d670 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.729+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49dc0d29e0 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.730+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49cc013800 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.730+0000 7f49d37fe640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c8f0 0x7f49bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f49cc08bb20 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49daffd640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c8f0 0x7f49bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49d37fe640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7f49bc0727b0 0x7f49bc074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49cc011e50 con 0x7f49bc0727b0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f49cc090240 con 0x7f49dc15c8f0 2024-04-26T21:21:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49da7fc640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7f49bc0727b0 0x7f49bc074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49da7fc640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7f49bc0727b0 0x7f49bc074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.731+0000 7f49daffd640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c8f0 0x7f49bc06edb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f49d4035000 tx=0x7f49d4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.732+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== osd.3 v2:172.21.15.40:6808/592958762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49cc011e50 con 0x7f49bc0727b0 2024-04-26T21:21:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 --> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f49dc06a290 con 0x7f49bc0727b0 2024-04-26T21:21:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49d37fe640 1 -- 172.21.15.40:0/159590808 <== osd.3 v2:172.21.15.40:6808/592958762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f49dc06a290 con 0x7f49bc0727b0 2024-04-26T21:21:54.746 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:54.746 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:54.746 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7f49bc0727b0 msgr2=0x7f49bc074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:6808/592958762,v1:172.21.15.40:6809/592958762] conn(0x7f49bc0727b0 0x7f49bc074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c8f0 msgr2=0x7f49bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f49bc06c8f0 0x7f49bc06edb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f49d4035000 tx=0x7f49d4030000 comp rx=0 tx=0).stop 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 msgr2=0x7f49dc161250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.745+0000 7f49e13e2640 1 --2- 172.21.15.40:0/159590808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49dc15c8f0 0x7f49dc161250 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f49dc073960 tx=0x7f49cc00e530 comp rx=0 tx=0).stop 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.746+0000 7f49db7fe640 1 -- 172.21.15.40:0/159590808 reap_dead start 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.746+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 shutdown_connections 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.746+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 >> 172.21.15.40:0/159590808 conn(0x7f49dc0d2f10 msgr2=0x7f49dc0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.746+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 shutdown_connections 2024-04-26T21:21:54.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.746+0000 7f49e13e2640 1 -- 172.21.15.40:0/159590808 wait complete. 2024-04-26T21:21:54.757 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T21:21:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.858+0000 7f3449f59640 1 Processor -- start 2024-04-26T21:21:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.858+0000 7f3449f59640 1 -- start start 2024-04-26T21:21:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3449f59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f344415f720 0x7f34440b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3449f59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f34440d5610 0x7f34440d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3449f59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34440b2510 con 0x7f344415f340 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3449f59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f34440b27f0 con 0x7f34440d5610 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3449f59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f34440b2ad0 con 0x7f344415f720 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f34440d5610 0x7f34440d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f34440d5610 0x7f34440d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57496/0 (socket says 172.21.15.40:57496) 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 -- 172.21.15.40:0/2238433037 learned_addr learned my addr 172.21.15.40:0/2238433037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3442ffd640 1 --2- 172.21.15.40:0/2238433037 >> v2:172.21.15.184:3300/0 conn(0x7f344415f720 0x7f34440b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 -- 172.21.15.40:0/2238433037 >> v2:172.21.15.184:3300/0 conn(0x7f344415f720 msgr2=0x7f34440b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 --2- 172.21.15.40:0/2238433037 >> v2:172.21.15.184:3300/0 conn(0x7f344415f720 0x7f34440b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.859+0000 7f3443fff640 1 -- 172.21.15.40:0/2238433037 >> v1:172.21.15.184:6789/0 conn(0x7f344415f340 legacy=0x7f34440d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.860+0000 7f3443fff640 1 -- 172.21.15.40:0/2238433037 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34440b2db0 con 0x7f34440d5610 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.860+0000 7f3443fff640 1 --2- 172.21.15.40:0/2238433037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34440d3d40 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f34440cf8a0 tx=0x7f343c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2eca24aafad217 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.860+0000 7f34427fc640 1 -- 172.21.15.40:0/2238433037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c036070 con 0x7f34440d5610 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.860+0000 7f34427fc640 1 -- 172.21.15.40:0/2238433037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c031600 con 0x7f34440d5610 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.860+0000 7f34427fc640 1 -- 172.21.15.40:0/2238433037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0319d0 con 0x7f34440d5610 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 -- 172.21.15.40:0/2238433037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 msgr2=0x7f34440d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 --2- 172.21.15.40:0/2238433037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34440d3d40 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f34440cf8a0 tx=0x7f343c02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 -- 172.21.15.40:0/2238433037 shutdown_connections 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 --2- 172.21.15.40:0/2238433037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34440d3d40 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 --2- 172.21.15.40:0/2238433037 >> v2:172.21.15.184:3300/0 conn(0x7f344415f720 0x7f34440b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 -- 172.21.15.40:0/2238433037 >> 172.21.15.40:0/2238433037 conn(0x7f344408ec70 msgr2=0x7f344409e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 -- 172.21.15.40:0/2238433037 shutdown_connections 2024-04-26T21:21:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.861+0000 7f3449f59640 1 -- 172.21.15.40:0/2238433037 wait complete. 2024-04-26T21:21:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.862+0000 7f3449f59640 1 Processor -- start 2024-04-26T21:21:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.862+0000 7f3449f59640 1 -- start start 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 0x7f34440caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 0x7f34440cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34440b3940 con 0x7f34440d5610 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34440b37c0 con 0x7f344415f340 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3449f59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f34440b3ac0 con 0x7f344415f720 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57508/0 (socket says 172.21.15.40:57508) 2024-04-26T21:21:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 -- 172.21.15.40:0/3437605504 learned_addr learned my addr 172.21.15.40:0/3437605504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3442ffd640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 0x7f34440caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3443fff640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 0x7f34440cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 -- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 msgr2=0x7f34440cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 0x7f34440cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 -- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 msgr2=0x7f34440caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 0x7f34440caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f34437fe640 1 -- 172.21.15.40:0/3437605504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34440b9e50 con 0x7f34440d5610 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3442ffd640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 0x7f34440caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.863+0000 7f3443fff640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 0x7f34440cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f34437fe640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f343400c400 tx=0x7f343400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340052d0 con 0x7f34440d5610 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3434005490 con 0x7f34440d5610 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34440cc4c0 con 0x7f34440d5610 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434007260 con 0x7f34440d5610 2024-04-26T21:21:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.864+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34440cca80 con 0x7f34440d5610 2024-04-26T21:21:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.865+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344406a290 con 0x7f34440d5610 2024-04-26T21:21:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.865+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3434007400 con 0x7f34440d5610 2024-04-26T21:21:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.866+0000 7f342bfff640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 0x7f343006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.866+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f343408ad30 con 0x7f34440d5610 2024-04-26T21:21:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.866+0000 7f3442ffd640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 0x7f343006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.866+0000 7f3442ffd640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 0x7f343006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3438006fd0 tx=0x7f3438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:54.870+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3434059470 con 0x7f34440d5610 2024-04-26T21:21:55.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.055+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f34440d3020 con 0x7f343006c8f0 2024-04-26T21:21:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.055+0000 7f342bfff640 1 -- 172.21.15.40:0/3437605504 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f34440d3020 con 0x7f343006c8f0 2024-04-26T21:21:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 msgr2=0x7f343006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 0x7f343006edb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3438006fd0 tx=0x7f3438008040 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 msgr2=0x7f34441eef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f343400c400 tx=0x7f343400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 shutdown_connections 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f343006c8f0 0x7f343006edb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f344415f720 0x7f34440cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f344415f340 0x7f34440caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 --2- 172.21.15.40:0/3437605504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34440d5610 0x7f34441eef10 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 >> 172.21.15.40:0/3437605504 conn(0x7f344408ec70 msgr2=0x7f34440b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 shutdown_connections 2024-04-26T21:21:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.058+0000 7f3449f59640 1 -- 172.21.15.40:0/3437605504 wait complete. 2024-04-26T21:21:55.070 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T21:21:55.070 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:55.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 Processor -- start 2024-04-26T21:21:55.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 -- start start 2024-04-26T21:21:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a34146a40 0x7f4a340b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4a340b5190 0x7f4a340cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a340d0050 0x7f4a34070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4a34160e80 con 0x7f4a340d0050 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f4a34161160 con 0x7f4a340b5190 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.175+0000 7f4a3a5a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a34161440 con 0x7f4a34146a40 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a340d0050 0x7f4a34070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a340d0050 0x7f4a34070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57516/0 (socket says 172.21.15.40:57516) 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 -- 172.21.15.40:0/1453055747 learned_addr learned my addr 172.21.15.40:0/1453055747 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a337fe640 1 --2- 172.21.15.40:0/1453055747 >> v2:172.21.15.184:3300/0 conn(0x7f4a340b5190 0x7f4a340cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a33fff640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a34146a40 0x7f4a340b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 -- 172.21.15.40:0/1453055747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a34146a40 msgr2=0x7f4a340b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a34146a40 0x7f4a340b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 -- 172.21.15.40:0/1453055747 >> v2:172.21.15.184:3300/0 conn(0x7f4a340b5190 msgr2=0x7f4a340cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 --2- 172.21.15.40:0/1453055747 >> v2:172.21.15.184:3300/0 conn(0x7f4a340b5190 0x7f4a340cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.176+0000 7f4a38b41640 1 -- 172.21.15.40:0/1453055747 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a34161720 con 0x7f4a340d0050 2024-04-26T21:21:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a38b41640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340d0050 0x7f4a34070580 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f4a340715e0 tx=0x7f4a2c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23424fb51ff6577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a32ffd640 1 -- 172.21.15.40:0/1453055747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c036070 con 0x7f4a340d0050 2024-04-26T21:21:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a32ffd640 1 -- 172.21.15.40:0/1453055747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2c031600 con 0x7f4a340d0050 2024-04-26T21:21:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a32ffd640 1 -- 172.21.15.40:0/1453055747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0319d0 con 0x7f4a340d0050 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/1453055747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340d0050 msgr2=0x7f4a34070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340d0050 0x7f4a34070580 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f4a340715e0 tx=0x7f4a2c02a610 comp rx=0 tx=0).stop 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/1453055747 shutdown_connections 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340d0050 0x7f4a34070580 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/1453055747 >> v2:172.21.15.184:3300/0 conn(0x7f4a340b5190 0x7f4a340cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/1453055747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a34146a40 0x7f4a340b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/1453055747 >> 172.21.15.40:0/1453055747 conn(0x7f4a3406c960 msgr2=0x7f4a340b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/1453055747 shutdown_connections 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.177+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/1453055747 wait complete. 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.178+0000 7f4a3a5a4640 1 Processor -- start 2024-04-26T21:21:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 -- start start 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 0x7f4a3413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a340d0050 0x7f4a34140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a34146a40 0x7f4a341f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34146e20 con 0x7f4a340b5190 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34070ac0 con 0x7f4a34146a40 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a3a5a4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3413db10 con 0x7f4a340d0050 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 0x7f4a3413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 0x7f4a3413d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57532/0 (socket says 172.21.15.40:57532) 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.179+0000 7f4a33fff640 1 -- 172.21.15.40:0/3318291418 learned_addr learned my addr 172.21.15.40:0/3318291418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a38b41640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a34146a40 0x7f4a341f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a337fe640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a340d0050 0x7f4a34140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 -- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a340d0050 msgr2=0x7f4a34140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a340d0050 0x7f4a34140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 -- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a34146a40 msgr2=0x7f4a341f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a34146a40 0x7f4a341f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a341f6960 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a38b41640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a34146a40 0x7f4a341f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a337fe640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a340d0050 0x7f4a34140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a33fff640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 0x7f4a3413d530 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400d5f0 tx=0x7f4a2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a240125f0 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a24019070 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341f6c40 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a240129d0 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.180+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a34151080 con 0x7f4a340b5190 2024-04-26T21:21:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.181+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a340693d0 con 0x7f4a340b5190 2024-04-26T21:21:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a24012b70 con 0x7f4a340b5190 2024-04-26T21:21:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4a2006c8f0 0x7f4a2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f4a2408c770 con 0x7f4a340b5190 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f4a200727b0 0x7f4a20074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a20075280 con 0x7f4a200727b0 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.182+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f4a2408cb20 con 0x7f4a340b5190 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.183+0000 7f4a337fe640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4a2006c8f0 0x7f4a2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.183+0000 7f4a38b41640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f4a200727b0 0x7f4a20074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.183+0000 7f4a38b41640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f4a200727b0 0x7f4a20074bd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.183+0000 7f4a337fe640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4a2006c8f0 0x7f4a2006edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4a3415fcc0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.184+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== osd.4 v2:172.21.15.184:6816/1784643509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a20075280 con 0x7f4a200727b0 2024-04-26T21:21:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.197+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 --> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a340d3ba0 con 0x7f4a200727b0 2024-04-26T21:21:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.197+0000 7f4a30ff9640 1 -- 172.21.15.40:0/3318291418 <== osd.4 v2:172.21.15.184:6816/1784643509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a340d3ba0 con 0x7f4a200727b0 2024-04-26T21:21:55.199 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:55.199 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:55.199 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f4a200727b0 msgr2=0x7f4a20074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6816/1784643509,v1:172.21.15.184:6817/1784643509] conn(0x7f4a200727b0 0x7f4a20074bd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4a2006c8f0 msgr2=0x7f4a2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f4a2006c8f0 0x7f4a2006edb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4a3415fcc0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 msgr2=0x7f4a3413d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 --2- 172.21.15.40:0/3318291418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a340b5190 0x7f4a3413d530 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400d5f0 tx=0x7f4a2400dab0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a38b41640 1 -- 172.21.15.40:0/3318291418 reap_dead start 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 shutdown_connections 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 >> 172.21.15.40:0/3318291418 conn(0x7f4a3406c960 msgr2=0x7f4a340d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 shutdown_connections 2024-04-26T21:21:55.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.198+0000 7f4a3a5a4640 1 -- 172.21.15.40:0/3318291418 wait complete. 2024-04-26T21:21:55.209 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T21:21:55.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 Processor -- start 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 -- start start 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e140cf2e0 0x7f9e1406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140d1740 con 0x7f9e1406e490 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140d1a20 con 0x7f9e140d11b0 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.310+0000 7f9e1af5f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f9e140d1d00 con 0x7f9e140cf2e0 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e194fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e1406e490 0x7f9e1415ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60160/0 (socket says 172.21.15.40:60160) 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e194fc640 1 -- 172.21.15.40:0/261839093 learned_addr learned my addr 172.21.15.40:0/261839093 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e18cfb640 1 --2- 172.21.15.40:0/261839093 >> v2:172.21.15.184:3300/0 conn(0x7f9e140cf2e0 0x7f9e1406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490646324 0 0) 0x7f9e140d1740 con 0x7f9e1406e490 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e00003520 con 0x7f9e1406e490 2024-04-26T21:21:55.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602852966 0 0) 0x7f9e140d1a20 con 0x7f9e140d11b0 2024-04-26T21:21:55.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e140d1740 con 0x7f9e140d11b0 2024-04-26T21:21:55.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e18cfb640 1 -- 172.21.15.40:0/261839093 >> v1:172.21.15.184:6789/0 conn(0x7f9e140d11b0 legacy=0x7f9e1406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e18cfb640 1 -- 172.21.15.40:0/261839093 >> v1:172.21.15.40:6789/0 conn(0x7f9e1406e490 legacy=0x7f9e1415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.311+0000 7f9e18cfb640 1 -- 172.21.15.40:0/261839093 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140d1fe0 con 0x7f9e140cf2e0 2024-04-26T21:21:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.312+0000 7f9e18cfb640 1 --2- 172.21.15.40:0/261839093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140cf2e0 0x7f9e1406d4d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009dc0 tx=0x7f9e0c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644ad65f49564394 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.312+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c007740 con 0x7f9e140cf2e0 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.312+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0078e0 con 0x7f9e140cf2e0 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.312+0000 7f9e137fe640 1 -- 172.21.15.40:0/261839093 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c007cb0 con 0x7f9e140cf2e0 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 -- 172.21.15.40:0/261839093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140cf2e0 msgr2=0x7f9e1406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/261839093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140cf2e0 0x7f9e1406d4d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009dc0 tx=0x7f9e0c02a730 comp rx=0 tx=0).stop 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 -- 172.21.15.40:0/261839093 shutdown_connections 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/261839093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140cf2e0 0x7f9e1406d4d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 -- 172.21.15.40:0/261839093 >> 172.21.15.40:0/261839093 conn(0x7f9e140b4a60 msgr2=0x7f9e140b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 -- 172.21.15.40:0/261839093 shutdown_connections 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.313+0000 7f9e1af5f640 1 -- 172.21.15.40:0/261839093 wait complete. 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.314+0000 7f9e1af5f640 1 Processor -- start 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.314+0000 7f9e1af5f640 1 -- start start 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e140cf2e0 0x7f9e141eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140d11b0 0x7f9e1414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140cf6c0 con 0x7f9e1406e490 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1406da10 con 0x7f9e140d11b0 2024-04-26T21:21:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e1af5f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140cf840 con 0x7f9e140cf2e0 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57540/0 (socket says 172.21.15.40:57540) 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 -- 172.21.15.40:0/1398110135 learned_addr learned my addr 172.21.15.40:0/1398110135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e194fc640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140d11b0 0x7f9e1414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e13fff640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e140cf2e0 0x7f9e141eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 -- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e140cf2e0 msgr2=0x7f9e141eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e140cf2e0 0x7f9e141eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 -- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140d11b0 msgr2=0x7f9e1414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140d11b0 0x7f9e1414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 -- 172.21.15.40:0/1398110135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e141eb6f0 con 0x7f9e1406e490 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.315+0000 7f9e18cfb640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009ee0 tx=0x7f9e0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.316+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c03f070 con 0x7f9e1406e490 2024-04-26T21:21:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.316+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c031bc0 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.316+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e141ea110 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.316+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c009450 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.316+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e141ea6d0 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.317+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1406a290 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.317+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e0c03b410 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.317+0000 7f9e117fa640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.318+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f9e0c0acb30 con 0x7f9e1406e490 2024-04-26T21:21:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.318+0000 7f9e13fff640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.318+0000 7f9e13fff640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 0x7f9df406ede0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc007fd0 tx=0x7f9dfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.322+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e0c049060 con 0x7f9e1406e490 2024-04-26T21:21:55.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.506+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f9e1406d5c0 con 0x7f9df406c920 2024-04-26T21:21:55.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.507+0000 7f9e117fa640 1 -- 172.21.15.40:0/1398110135 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9e1406d5c0 con 0x7f9df406c920 2024-04-26T21:21:55.508 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 msgr2=0x7f9df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 0x7f9df406ede0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc007fd0 tx=0x7f9dfc009040 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 msgr2=0x7f9e14148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009ee0 tx=0x7f9e0c004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 shutdown_connections 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e140d11b0 0x7f9e1414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e140cf2e0 0x7f9e141eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 --2- 172.21.15.40:0/1398110135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e1406e490 0x7f9e14148f00 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 >> 172.21.15.40:0/1398110135 conn(0x7f9e140b4a60 msgr2=0x7f9e14070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 shutdown_connections 2024-04-26T21:21:55.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.509+0000 7f9e1af5f640 1 -- 172.21.15.40:0/1398110135 wait complete. 2024-04-26T21:21:55.520 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T21:21:55.520 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.624+0000 7f7ad1124640 1 Processor -- start 2024-04-26T21:21:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.624+0000 7f7ad1124640 1 -- start start 2024-04-26T21:21:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ad1124640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7acc09c150 0x7f7acc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ad1124640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7acc0d1240 con 0x7f7acc0d3a30 2024-04-26T21:21:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ad1124640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7acc0d14e0 con 0x7f7acc0cee20 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ad1124640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0d1e10 con 0x7f7acc09c150 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ac3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7acc09c150 0x7f7acc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7acb59e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7acc0d3a30 0x7f7acc15eb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60168/0 (socket says 172.21.15.40:60168) 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7acb59e640 1 -- 172.21.15.40:0/3644270923 learned_addr learned my addr 172.21.15.40:0/3644270923 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ac3fff640 1 -- 172.21.15.40:0/3644270923 >> v1:172.21.15.184:6789/0 conn(0x7f7acc0cee20 legacy=0x7f7acc09ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ac3fff640 1 -- 172.21.15.40:0/3644270923 >> v1:172.21.15.40:6789/0 conn(0x7f7acc0d3a30 legacy=0x7f7acc15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7ac3fff640 1 -- 172.21.15.40:0/3644270923 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0d17c0 con 0x7f7acc09c150 2024-04-26T21:21:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.625+0000 7f7aca59c640 1 -- 172.21.15.40:0/3644270923 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235720354 0 0) 0x7f7acc0d1240 con 0x7f7acc0d3a30 2024-04-26T21:21:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.626+0000 7f7ac3fff640 1 --2- 172.21.15.40:0/3644270923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc06d5c0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40097a0 tx=0x7f7ab402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb779c44492d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.626+0000 7f7aca59c640 1 -- 172.21.15.40:0/3644270923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4037070 con 0x7f7acc09c150 2024-04-26T21:21:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.626+0000 7f7aca59c640 1 -- 172.21.15.40:0/3644270923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab4032440 con 0x7f7acc09c150 2024-04-26T21:21:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.626+0000 7f7aca59c640 1 -- 172.21.15.40:0/3644270923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4032810 con 0x7f7acc09c150 2024-04-26T21:21:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 -- 172.21.15.40:0/3644270923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 msgr2=0x7f7acc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 --2- 172.21.15.40:0/3644270923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc06d5c0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40097a0 tx=0x7f7ab402a230 comp rx=0 tx=0).stop 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 -- 172.21.15.40:0/3644270923 shutdown_connections 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 --2- 172.21.15.40:0/3644270923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc06d5c0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 -- 172.21.15.40:0/3644270923 >> 172.21.15.40:0/3644270923 conn(0x7f7acc0d2f10 msgr2=0x7f7acc0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 -- 172.21.15.40:0/3644270923 shutdown_connections 2024-04-26T21:21:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.627+0000 7f7ad1124640 1 -- 172.21.15.40:0/3644270923 wait complete. 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.628+0000 7f7ad1124640 1 Processor -- start 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.628+0000 7f7ad1124640 1 -- start start 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7acc0cee20 0x7f7acc15fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7acc0d3a30 0x7f7acc148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0cf200 con 0x7f7acc09c150 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc09c7a0 con 0x7f7acc0d3a30 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ad1124640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc0cf380 con 0x7f7acc0cee20 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc0c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57562/0 (socket says 172.21.15.40:57562) 2024-04-26T21:21:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 -- 172.21.15.40:0/823244096 learned_addr learned my addr 172.21.15.40:0/823244096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acb59e640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7acc0d3a30 0x7f7acc148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7ac3fff640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7acc0cee20 0x7f7acc15fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 -- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7acc0cee20 msgr2=0x7f7acc15fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7acc0cee20 0x7f7acc15fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 -- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7acc0d3a30 msgr2=0x7f7acc148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7acc0d3a30 0x7f7acc148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc149db0 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.629+0000 7f7acad9d640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc0c95f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c450 tx=0x7f7abc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0052d0 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc005470 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc012870 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc15ea20 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc15efe0 con 0x7f7acc09c150 2024-04-26T21:21:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.630+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7acc06a290 con 0x7f7acc09c150 2024-04-26T21:21:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.631+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7abc012a10 con 0x7f7acc09c150 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.631+0000 7f7ac37fe640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7aac06c920 0x7f7aac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.631+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f7abc08b3b0 con 0x7f7acc09c150 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7ac37fe640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7f7aac0727e0 0x7f7aac074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7aac0752b0 con 0x7f7aac0727e0 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f7abc092050 con 0x7f7acc09c150 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7ac3fff640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7aac06c920 0x7f7aac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7acb59e640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7f7aac0727e0 0x7f7aac074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7ac3fff640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7aac06c920 0x7f7aac06ede0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4034000 tx=0x7f7ab402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.632+0000 7f7acb59e640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7f7aac0727e0 0x7f7aac074c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.633+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== osd.5 v2:172.21.15.184:6800/2962645716 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7aac0752b0 con 0x7f7aac0727e0 2024-04-26T21:21:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.646+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 --> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7acc0b4820 con 0x7f7aac0727e0 2024-04-26T21:21:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.647+0000 7f7ac37fe640 1 -- 172.21.15.40:0/823244096 <== osd.5 v2:172.21.15.184:6800/2962645716 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7acc0b4820 con 0x7f7aac0727e0 2024-04-26T21:21:55.648 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:55.648 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7f7aac0727e0 msgr2=0x7f7aac074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6800/2962645716,v1:172.21.15.184:6801/2962645716] conn(0x7f7aac0727e0 0x7f7aac074c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7aac06c920 msgr2=0x7f7aac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f7aac06c920 0x7f7aac06ede0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4034000 tx=0x7f7ab402e000 comp rx=0 tx=0).stop 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 msgr2=0x7f7acc0c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 --2- 172.21.15.40:0/823244096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc09c150 0x7f7acc0c95f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c450 tx=0x7f7abc00c910 comp rx=0 tx=0).stop 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7acb59e640 1 -- 172.21.15.40:0/823244096 reap_dead start 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 shutdown_connections 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 >> 172.21.15.40:0/823244096 conn(0x7f7acc0d2f10 msgr2=0x7f7acc0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 shutdown_connections 2024-04-26T21:21:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.648+0000 7f7ad1124640 1 -- 172.21.15.40:0/823244096 wait complete. 2024-04-26T21:21:55.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T21:21:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.759+0000 7f5479099640 1 Processor -- start 2024-04-26T21:21:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.759+0000 7f5479099640 1 -- start start 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5479099640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f547413ff70 0x7f5474146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5479099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f54741465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5479099640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54740d05b0 con 0x7f5474140350 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5479099640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f54740d0890 con 0x7f547413ff70 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5479099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f547406e450 con 0x7f54740cff20 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f547259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5474140350 0x7f54740cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60184/0 (socket says 172.21.15.40:60184) 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f547259c640 1 -- 172.21.15.40:0/3926000920 learned_addr learned my addr 172.21.15.40:0/3926000920 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f547359e640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f54741465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5472d9d640 1 --2- 172.21.15.40:0/3926000920 >> v2:172.21.15.184:3300/0 conn(0x7f547413ff70 0x7f5474146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3306892883 0 0) 0x7f54740d05b0 con 0x7f5474140350 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.760+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f545c003520 con 0x7f5474140350 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1443889591 0 0) 0x7f545c003520 con 0x7f5474140350 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 msgr2=0x7f54741465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f54741465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 >> v2:172.21.15.184:3300/0 conn(0x7f547413ff70 msgr2=0x7f5474146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 --2- 172.21.15.40:0/3926000920 >> v2:172.21.15.184:3300/0 conn(0x7f547413ff70 0x7f5474146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547406e730 con 0x7f5474140350 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f5468001670 con 0x7f5474140350 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 >> v1:172.21.15.40:6789/0 conn(0x7f5474140350 legacy=0x7f54740cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f545c0039b0 0x7f545c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f545c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f545c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54740d0890 con 0x7f545c004800 2024-04-26T21:21:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f547406e450 con 0x7f545c0039b0 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f545c00b9c0 con 0x7f54740cff20 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f545c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f5472d9d640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f545c0039b0 0x7f545c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547359e640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f545c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 -- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 msgr2=0x7f545c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54740cff20 0x7f545c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 -- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f545c0039b0 msgr2=0x7f545c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f545c0039b0 0x7f545c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.761+0000 7f547259c640 1 -- 172.21.15.40:0/3926000920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c003520 con 0x7f545c004800 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.762+0000 7f547259c640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f545c004c80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f546800c270 tx=0x7f545c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4176929b194cfb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.762+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468007d30 con 0x7f545c004800 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.762+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5468005ad0 con 0x7f545c004800 2024-04-26T21:21:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.762+0000 7f5471d9b640 1 -- 172.21.15.40:0/3926000920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468007070 con 0x7f545c004800 2024-04-26T21:21:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 -- 172.21.15.40:0/3926000920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 msgr2=0x7f545c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 --2- 172.21.15.40:0/3926000920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f545c004c80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f546800c270 tx=0x7f545c005830 comp rx=0 tx=0).stop 2024-04-26T21:21:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f547359e640 1 -- 172.21.15.40:0/3926000920 reap_dead start 2024-04-26T21:21:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 -- 172.21.15.40:0/3926000920 shutdown_connections 2024-04-26T21:21:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 -- 172.21.15.40:0/3926000920 >> 172.21.15.40:0/3926000920 conn(0x7f54740d0d30 msgr2=0x7f54740d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 -- 172.21.15.40:0/3926000920 shutdown_connections 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.763+0000 7f5479099640 1 -- 172.21.15.40:0/3926000920 wait complete. 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.764+0000 7f5479099640 1 Processor -- start 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.764+0000 7f5479099640 1 -- start start 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f547413b510 0x7f5474156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f547415a650 0x7f547415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5474146990 con 0x7f545c004800 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5474146810 con 0x7f547415a650 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5479099640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f547406ec50 con 0x7f547413b510 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57588/0 (socket says 172.21.15.40:57588) 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 -- 172.21.15.40:0/58697190 learned_addr learned my addr 172.21.15.40:0/58697190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 -- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f547413b510 msgr2=0x7f5474156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f547413b510 0x7f5474156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 -- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f547415a650 msgr2=0x7f547415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f547415a650 0x7f547415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 -- 172.21.15.40:0/58697190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741582c0 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.765+0000 7f5472d9d640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f546c009f90 tx=0x7f546c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.766+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c015430 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.766+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546c0155d0 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.766+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741585a0 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.766+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0159a0 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.766+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547415d790 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.767+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547406a290 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.767+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f546c015b40 con 0x7f545c004800 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.768+0000 7f545b7fe640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 0x7f546006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.768+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f546c08ca20 con 0x7f545c004800 2024-04-26T21:21:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.768+0000 7f547259c640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 0x7f546006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.768+0000 7f547259c640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 0x7f546006ede0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5468033000 tx=0x7f5468007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.772+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f546c019030 con 0x7f545c004800 2024-04-26T21:21:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.955+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f54740b45e0 con 0x7f546006c920 2024-04-26T21:21:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.956+0000 7f545b7fe640 1 -- 172.21.15.40:0/58697190 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f54740b45e0 con 0x7f546006c920 2024-04-26T21:21:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T21:21:55.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 msgr2=0x7f546006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 0x7f546006ede0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5468033000 tx=0x7f5468007000 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 msgr2=0x7f54741f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f546c009f90 tx=0x7f546c00f580 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 shutdown_connections 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f546006c920 0x7f546006ede0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f547415a650 0x7f547415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f547413b510 0x7f5474156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 --2- 172.21.15.40:0/58697190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f545c004800 0x7f54741f72c0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 >> 172.21.15.40:0/58697190 conn(0x7f54740d0d30 msgr2=0x7f54740b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 shutdown_connections 2024-04-26T21:21:55.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:55.958+0000 7f5479099640 1 -- 172.21.15.40:0/58697190 wait complete. 2024-04-26T21:21:55.970 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T21:21:55.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:56.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 Processor -- start 2024-04-26T21:21:56.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 -- start start 2024-04-26T21:21:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe62c0cee20 0x7fe62c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0d3a30 0x7fe62c09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe62c09bf80 0x7fe62c1630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe62c0d1e10 con 0x7fe62c0cee20 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7fe62c0d1240 con 0x7fe62c09bf80 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe6315c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c0d14e0 con 0x7fe62c0d3a30 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe62affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe62c0cee20 0x7fe62c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe62affd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe62c0cee20 0x7fe62c06d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57598/0 (socket says 172.21.15.40:57598) 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.074+0000 7fe62affd640 1 -- 172.21.15.40:0/1345824655 learned_addr learned my addr 172.21.15.40:0/1345824655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62b7fe640 1 --2- 172.21.15.40:0/1345824655 >> v2:172.21.15.184:3300/0 conn(0x7fe62c09bf80 0x7fe62c1630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62a7fc640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0d3a30 0x7fe62c09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 -- 172.21.15.40:0/1345824655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0d3a30 msgr2=0x7fe62c09ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0d3a30 0x7fe62c09ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 -- 172.21.15.40:0/1345824655 >> v2:172.21.15.184:3300/0 conn(0x7fe62c09bf80 msgr2=0x7fe62c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 --2- 172.21.15.40:0/1345824655 >> v2:172.21.15.184:3300/0 conn(0x7fe62c09bf80 0x7fe62c1630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 -- 172.21.15.40:0/1345824655 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c0d17c0 con 0x7fe62c0cee20 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.075+0000 7fe62affd640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0cee20 0x7fe62c06d5c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0c69f0 tx=0x7fe61c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcf5464421f47bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe629ffb640 1 -- 172.21.15.40:0/1345824655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c037070 con 0x7fe62c0cee20 2024-04-26T21:21:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe629ffb640 1 -- 172.21.15.40:0/1345824655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61c032440 con 0x7fe62c0cee20 2024-04-26T21:21:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe629ffb640 1 -- 172.21.15.40:0/1345824655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c032810 con 0x7fe62c0cee20 2024-04-26T21:21:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 -- 172.21.15.40:0/1345824655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0cee20 msgr2=0x7fe62c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0cee20 0x7fe62c06d5c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0c69f0 tx=0x7fe61c02a270 comp rx=0 tx=0).stop 2024-04-26T21:21:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 -- 172.21.15.40:0/1345824655 shutdown_connections 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1345824655 >> v2:172.21.15.184:3300/0 conn(0x7fe62c09bf80 0x7fe62c1630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0d3a30 0x7fe62c09ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1345824655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0cee20 0x7fe62c06d5c0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 -- 172.21.15.40:0/1345824655 >> 172.21.15.40:0/1345824655 conn(0x7fe62c0d2f10 msgr2=0x7fe62c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 -- 172.21.15.40:0/1345824655 shutdown_connections 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.076+0000 7fe6315c9640 1 -- 172.21.15.40:0/1345824655 wait complete. 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.077+0000 7fe6315c9640 1 Processor -- start 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.077+0000 7fe6315c9640 1 -- start start 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c09bf80 0x7fe62c15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0cee20 0x7fe62c09cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 0x7fe62c09f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0cf200 con 0x7fe62c0d3a30 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c09c7a0 con 0x7fe62c09bf80 2024-04-26T21:21:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe6315c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c0cf380 con 0x7fe62c0cee20 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 0x7fe62c09f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 0x7fe62c09f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57608/0 (socket says 172.21.15.40:57608) 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe62b7fe640 1 -- 172.21.15.40:0/1032552936 learned_addr learned my addr 172.21.15.40:0/1032552936 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe62affd640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c09bf80 0x7fe62c15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.078+0000 7fe62a7fc640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0cee20 0x7fe62c09cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 -- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0cee20 msgr2=0x7fe62c09cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe62c0cee20 0x7fe62c09cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 -- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c09bf80 msgr2=0x7fe62c15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c09bf80 0x7fe62c15cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c09d3e0 con 0x7fe62c0d3a30 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62affd640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe62c09bf80 0x7fe62c15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe62b7fe640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 0x7fe62c09f4d0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe624007e40 tx=0x7fe62400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624014620 con 0x7fe62c0d3a30 2024-04-26T21:21:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe624019070 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c09d660 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624014a20 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.079+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c09dc20 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.080+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe62c15cff0 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.080+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe624014bc0 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe6137fe640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe61806c8f0 0x7fe61806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fe62408c1f0 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe6137fe640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7fe6180727b0 0x7fe618074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe62400ce50 con 0x7fe6180727b0 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fe624093050 con 0x7fe62c0d3a30 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe62affd640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe61806c8f0 0x7fe61806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.081+0000 7fe62a7fc640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7fe6180727b0 0x7fe618074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.082+0000 7fe62affd640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe61806c8f0 0x7fe61806edb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe61c034000 tx=0x7fe61c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.082+0000 7fe62a7fc640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7fe6180727b0 0x7fe618074bd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.082+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== osd.6 v2:172.21.15.184:6824/3739826890 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe62400ce50 con 0x7fe6180727b0 2024-04-26T21:21:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.096+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 --> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe62c06a290 con 0x7fe6180727b0 2024-04-26T21:21:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.096+0000 7fe6137fe640 1 -- 172.21.15.40:0/1032552936 <== osd.6 v2:172.21.15.184:6824/3739826890 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe62c06a290 con 0x7fe6180727b0 2024-04-26T21:21:56.097 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:56.097 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7fe6180727b0 msgr2=0x7fe618074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6824/3739826890,v1:172.21.15.184:6825/3739826890] conn(0x7fe6180727b0 0x7fe618074bd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe61806c8f0 msgr2=0x7fe61806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fe61806c8f0 0x7fe61806edb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe61c034000 tx=0x7fe61c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 msgr2=0x7fe62c09f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 --2- 172.21.15.40:0/1032552936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe62c0d3a30 0x7fe62c09f4d0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe624007e40 tx=0x7fe62400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe62b7fe640 1 -- 172.21.15.40:0/1032552936 reap_dead start 2024-04-26T21:21:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 shutdown_connections 2024-04-26T21:21:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 >> 172.21.15.40:0/1032552936 conn(0x7fe62c0d2f10 msgr2=0x7fe62c0b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 shutdown_connections 2024-04-26T21:21:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.097+0000 7fe6315c9640 1 -- 172.21.15.40:0/1032552936 wait complete. 2024-04-26T21:21:56.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T21:21:56.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.209+0000 7fd3e8eda640 1 Processor -- start 2024-04-26T21:21:56.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.209+0000 7fd3e8eda640 1 -- start start 2024-04-26T21:21:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e8eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 0x7fd3e406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e8eda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e40d11b0 0x7fd3e406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e8eda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e40d1740 con 0x7fd3e406e2c0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e8eda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e406e750 con 0x7fd3e40d11b0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e8eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e40d1a20 con 0x7fd3e40cf2e0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e1d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e40d11b0 0x7fd3e406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd3e406e2c0 0x7fd3e415ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60194/0 (socket says 172.21.15.40:60194) 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.210+0000 7fd3e2d9d640 1 -- 172.21.15.40:0/523244513 learned_addr learned my addr 172.21.15.40:0/523244513 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e259c640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 0x7fd3e406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e1d9b640 1 -- 172.21.15.40:0/523244513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 msgr2=0x7fd3e406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 0x7fd3e406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e1d9b640 1 -- 172.21.15.40:0/523244513 >> v1:172.21.15.40:6789/0 conn(0x7fd3e406e2c0 legacy=0x7fd3e415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e1d9b640 1 -- 172.21.15.40:0/523244513 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e40d1d00 con 0x7fd3e40d11b0 2024-04-26T21:21:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e159a640 1 -- 172.21.15.40:0/523244513 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544968349 0 0) 0x7fd3e40d1740 con 0x7fd3e406e2c0 2024-04-26T21:21:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e259c640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 0x7fd3e406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e40d11b0 0x7fd3e406dd80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097a0 tx=0x7fd3cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a828e113871cb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e159a640 1 -- 172.21.15.40:0/523244513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc037070 con 0x7fd3e40d11b0 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e159a640 1 -- 172.21.15.40:0/523244513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc032440 con 0x7fd3e40d11b0 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.211+0000 7fd3e159a640 1 -- 172.21.15.40:0/523244513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc032810 con 0x7fd3e40d11b0 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 -- 172.21.15.40:0/523244513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e40d11b0 msgr2=0x7fd3e406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e40d11b0 0x7fd3e406dd80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097a0 tx=0x7fd3cc02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 -- 172.21.15.40:0/523244513 shutdown_connections 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e40d11b0 0x7fd3e406dd80 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/523244513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40cf2e0 0x7fd3e406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 -- 172.21.15.40:0/523244513 >> 172.21.15.40:0/523244513 conn(0x7fd3e40b4a60 msgr2=0x7fd3e40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 -- 172.21.15.40:0/523244513 shutdown_connections 2024-04-26T21:21:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.212+0000 7fd3e8eda640 1 -- 172.21.15.40:0/523244513 wait complete. 2024-04-26T21:21:56.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.213+0000 7fd3e8eda640 1 Processor -- start 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.213+0000 7fd3e8eda640 1 -- start start 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 0x7fd3e415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 0x7fd3e40c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e406da10 con 0x7fd3e406e2c0 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e40d22c0 con 0x7fd3e40cf2e0 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e8eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e40cf6c0 con 0x7fd3e40d11b0 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57624/0 (socket says 172.21.15.40:57624) 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e259c640 1 -- 172.21.15.40:0/2692243872 learned_addr learned my addr 172.21.15.40:0/2692243872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e2d9d640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 0x7fd3e40c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.214+0000 7fd3e259c640 1 -- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 msgr2=0x7fd3e40c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 0x7fd3e415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e259c640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 0x7fd3e40c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e259c640 1 -- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 msgr2=0x7fd3e415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e259c640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 0x7fd3e415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e259c640 1 -- 172.21.15.40:0/2692243872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e415dc80 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e2d9d640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 0x7fd3e40c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 0x7fd3e415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e259c640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cf60 tx=0x7fd3d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d401a070 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d400d840 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e40ccf30 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400dc10 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.215+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e40cd4f0 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.216+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e406a290 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.216+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3d401d070 con 0x7fd3e406e2c0 2024-04-26T21:21:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.217+0000 7fd3daffd640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 0x7fd3c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.217+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7fd3d408bfb0 con 0x7fd3e406e2c0 2024-04-26T21:21:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.217+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 0x7fd3c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.218+0000 7fd3e1d9b640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 0x7fd3c406edb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc034000 tx=0x7fd3cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.221+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3e415dc80 con 0x7fd3e406e2c0 2024-04-26T21:21:56.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.405+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd3e406deb0 con 0x7fd3c406c8f0 2024-04-26T21:21:56.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.406+0000 7fd3daffd640 1 -- 172.21.15.40:0/2692243872 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd3e406deb0 con 0x7fd3c406c8f0 2024-04-26T21:21:56.407 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T21:21:56.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 msgr2=0x7fd3c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 0x7fd3c406edb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc034000 tx=0x7fd3cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 msgr2=0x7fd3e40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cf60 tx=0x7fd3d400cf90 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 shutdown_connections 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7fd3c406c8f0 0x7fd3c406edb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3e40d11b0 0x7fd3e40c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3e40cf2e0 0x7fd3e415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 --2- 172.21.15.40:0/2692243872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e406e2c0 0x7fd3e40cc8c0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.407+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 >> 172.21.15.40:0/2692243872 conn(0x7fd3e40b4a60 msgr2=0x7fd3e4163830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.408+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 shutdown_connections 2024-04-26T21:21:56.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.408+0000 7fd3e8eda640 1 -- 172.21.15.40:0/2692243872 wait complete. 2024-04-26T21:21:56.419 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T21:21:56.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:21:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.523+0000 7f39a0eda640 1 Processor -- start 2024-04-26T21:21:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.523+0000 7f39a0eda640 1 -- start start 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f39a0eda640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f399c0d1930 0x7f399c0d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f39a0eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f39a0eda640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399c0d5e40 con 0x7f399c0c7df0 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f39a0eda640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f399c0d1350 con 0x7f399c0d1930 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f39a0eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f399c0d2250 con 0x7f399c0b6670 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f399c0d1930 0x7f399c0d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399ad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f399c0c7df0 0x7f399c0c8280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60196/0 (socket says 172.21.15.40:60196) 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f399c0d1930 0x7f399c0d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57640/0 (socket says 172.21.15.40:57640) 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399ad9d640 1 -- 172.21.15.40:0/3785539133 learned_addr learned my addr 172.21.15.40:0/3785539133 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f3993fff640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 -- 172.21.15.40:0/3785539133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 msgr2=0x7f399c0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 -- 172.21.15.40:0/3785539133 >> v1:172.21.15.40:6789/0 conn(0x7f399c0c7df0 legacy=0x7f399c0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f399a59c640 1 -- 172.21.15.40:0/3785539133 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c0d0a20 con 0x7f399c0d1930 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.524+0000 7f3999d9b640 1 -- 172.21.15.40:0/3785539133 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390144223 0 0) 0x7f399c0d5e40 con 0x7f399c0c7df0 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.525+0000 7f399a59c640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0d1930 0x7f399c0d5900 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f399c0d14d0 tx=0x7f398c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285f9c7498e608b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.525+0000 7f3999d9b640 1 -- 172.21.15.40:0/3785539133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c037070 con 0x7f399c0d1930 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.525+0000 7f3999d9b640 1 -- 172.21.15.40:0/3785539133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c032440 con 0x7f399c0d1930 2024-04-26T21:21:56.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.525+0000 7f3999d9b640 1 -- 172.21.15.40:0/3785539133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c032810 con 0x7f399c0d1930 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 -- 172.21.15.40:0/3785539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0d1930 msgr2=0x7f399c0d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0d1930 0x7f399c0d5900 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f399c0d14d0 tx=0x7f398c02a180 comp rx=0 tx=0).stop 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 -- 172.21.15.40:0/3785539133 shutdown_connections 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 --2- 172.21.15.40:0/3785539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0d1930 0x7f399c0d5900 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 -- 172.21.15.40:0/3785539133 >> 172.21.15.40:0/3785539133 conn(0x7f399c09e530 msgr2=0x7f399c0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 -- 172.21.15.40:0/3785539133 shutdown_connections 2024-04-26T21:21:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.526+0000 7f39a0eda640 1 -- 172.21.15.40:0/3785539133 wait complete. 2024-04-26T21:21:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.527+0000 7f39a0eda640 1 Processor -- start 2024-04-26T21:21:56.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.527+0000 7f39a0eda640 1 -- start start 2024-04-26T21:21:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 0x7f399c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f399c0d1930 0x7f399c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f399c0c8990 con 0x7f399c0c7df0 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f399c0d0fe0 con 0x7f399c0d1930 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f39a0eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f399c0c8b10 con 0x7f399c0b6670 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 0x7f399c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 0x7f399c0b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57642/0 (socket says 172.21.15.40:57642) 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 -- 172.21.15.40:0/440465814 learned_addr learned my addr 172.21.15.40:0/440465814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 -- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 msgr2=0x7f399c0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f399a59c640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f399c0b6670 0x7f399c0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 -- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f399c0d1930 msgr2=0x7f399c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f399c0d1930 0x7f399c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c0c8d90 con 0x7f399c0c7df0 2024-04-26T21:21:56.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.528+0000 7f3993fff640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 0x7f399c0b2870 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f398400e9b0 tx=0x7f398400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984017070 con 0x7f399c0c7df0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398400c9a0 con 0x7f399c0c7df0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398400cd90 con 0x7f399c0c7df0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c0c9030 con 0x7f399c0c7df0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c0c95f0 con 0x7f399c0c7df0 2024-04-26T21:21:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.529+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f399c0d1f80 con 0x7f399c0c7df0 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.530+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f398401a070 con 0x7f399c0c7df0 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f3992ffd640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f397c06c8f0 0x7f397c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f398408bb80 con 0x7f399c0c7df0 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f3992ffd640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7f397c0727b0 0x7f397c074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f397c075280 con 0x7f397c0727b0 2024-04-26T21:21:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f398408c010 con 0x7f399c0c7df0 2024-04-26T21:21:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f399a59c640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f397c06c8f0 0x7f397c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f399ad9d640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7f397c0727b0 0x7f397c074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.531+0000 7f399a59c640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f397c06c8f0 0x7f397c06edb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f398c034000 tx=0x7f398c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.532+0000 7f399ad9d640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7f397c0727b0 0x7f397c074bd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.532+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== osd.7 v2:172.21.15.184:6804/2214404971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f397c075280 con 0x7f397c0727b0 2024-04-26T21:21:56.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.545+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 --> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f399c0b20f0 con 0x7f397c0727b0 2024-04-26T21:21:56.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f3992ffd640 1 -- 172.21.15.40:0/440465814 <== osd.7 v2:172.21.15.184:6804/2214404971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f399c0b20f0 con 0x7f397c0727b0 2024-04-26T21:21:56.547 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-26T21:21:56.547 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:21:56.547 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7f397c0727b0 msgr2=0x7f397c074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6804/2214404971,v1:172.21.15.184:6806/2214404971] conn(0x7f397c0727b0 0x7f397c074bd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f397c06c8f0 msgr2=0x7f397c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f397c06c8f0 0x7f397c06edb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f398c034000 tx=0x7f398c02e000 comp rx=0 tx=0).stop 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 msgr2=0x7f399c0b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 --2- 172.21.15.40:0/440465814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f399c0c7df0 0x7f399c0b2870 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f398400e9b0 tx=0x7f398400ee70 comp rx=0 tx=0).stop 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f399ad9d640 1 -- 172.21.15.40:0/440465814 reap_dead start 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 shutdown_connections 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 >> 172.21.15.40:0/440465814 conn(0x7f399c09e530 msgr2=0x7f399c0ba8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 shutdown_connections 2024-04-26T21:21:56.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.546+0000 7f39a0eda640 1 -- 172.21.15.40:0/440465814 wait complete. 2024-04-26T21:21:56.557 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T21:21:56.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.658+0000 7f1ea747b640 1 Processor -- start 2024-04-26T21:21:56.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.658+0000 7f1ea747b640 1 -- start start 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea747b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ea00d5610 0x7f1ea00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea747b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea00b2510 con 0x7f1ea015f340 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea747b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea00b27f0 con 0x7f1ea015f720 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea747b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea00b2ad0 con 0x7f1ea00d5610 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5a18640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ea00d5610 0x7f1ea00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5217640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ea015f340 0x7f1ea00d4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60198/0 (socket says 172.21.15.40:60198) 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5a18640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ea00d5610 0x7f1ea00b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57648/0 (socket says 172.21.15.40:57648) 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5217640 1 -- 172.21.15.40:0/185581692 learned_addr learned my addr 172.21.15.40:0/185581692 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:21:56.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5a18640 1 -- 172.21.15.40:0/185581692 >> v1:172.21.15.184:6789/0 conn(0x7f1ea015f720 legacy=0x7f1ea00b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5a18640 1 -- 172.21.15.40:0/185581692 >> v1:172.21.15.40:6789/0 conn(0x7f1ea015f340 legacy=0x7f1ea00d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1ea5a18640 1 -- 172.21.15.40:0/185581692 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00b2db0 con 0x7f1ea00d5610 2024-04-26T21:21:56.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.659+0000 7f1e97fff640 1 -- 172.21.15.40:0/185581692 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825548176 0 0) 0x7f1ea00b2510 con 0x7f1ea015f340 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.660+0000 7f1ea5a18640 1 --2- 172.21.15.40:0/185581692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea00d5610 0x7f1ea00b1e80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009d90 tx=0x7f1e9802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e57133e742aaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.660+0000 7f1e97fff640 1 -- 172.21.15.40:0/185581692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98031600 con 0x7f1ea00d5610 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.660+0000 7f1e97fff640 1 -- 172.21.15.40:0/185581692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e980317a0 con 0x7f1ea00d5610 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.660+0000 7f1e97fff640 1 -- 172.21.15.40:0/185581692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98031b70 con 0x7f1ea00d5610 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.660+0000 7f1ea747b640 1 -- 172.21.15.40:0/185581692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea00d5610 msgr2=0x7f1ea00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 --2- 172.21.15.40:0/185581692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea00d5610 0x7f1ea00b1e80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009d90 tx=0x7f1e9802a610 comp rx=0 tx=0).stop 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 -- 172.21.15.40:0/185581692 shutdown_connections 2024-04-26T21:21:56.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 --2- 172.21.15.40:0/185581692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea00d5610 0x7f1ea00b1e80 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 -- 172.21.15.40:0/185581692 >> 172.21.15.40:0/185581692 conn(0x7f1ea008ec70 msgr2=0x7f1ea009e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 -- 172.21.15.40:0/185581692 shutdown_connections 2024-04-26T21:21:56.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.661+0000 7f1ea747b640 1 -- 172.21.15.40:0/185581692 wait complete. 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.662+0000 7f1ea747b640 1 Processor -- start 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.662+0000 7f1ea747b640 1 -- start start 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea00d5610 0x7f1ea0156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea015f340 0x7f1ea00ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea00d40d0 con 0x7f1ea015f720 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea00d3f50 con 0x7f1ea00d5610 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea00b3cc0 con 0x7f1ea015f340 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57652/0 (socket says 172.21.15.40:57652) 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 -- 172.21.15.40:0/857131191 learned_addr learned my addr 172.21.15.40:0/857131191 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5217640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea00d5610 0x7f1ea0156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea4a16640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea015f340 0x7f1ea00ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 -- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea015f340 msgr2=0x7f1ea00ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea015f340 0x7f1ea00ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 -- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea00d5610 msgr2=0x7f1ea0156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea00d5610 0x7f1ea0156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 -- 172.21.15.40:0/857131191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00c92c0 con 0x7f1ea015f720 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea5a18640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1e9802ac00 tx=0x7f1e98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98040070 con 0x7f1ea015f720 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e98030b60 con 0x7f1ea015f720 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.663+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea00c9540 con 0x7f1ea015f720 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.664+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98045460 con 0x7f1ea015f720 2024-04-26T21:21:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.664+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea00c9b00 con 0x7f1ea015f720 2024-04-26T21:21:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.665+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea006a290 con 0x7f1ea015f720 2024-04-26T21:21:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.665+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e98039040 con 0x7f1ea015f720 2024-04-26T21:21:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.666+0000 7f1e95ffb640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.666+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f1e980ace90 con 0x7f1ea015f720 2024-04-26T21:21:56.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.666+0000 7f1ea5217640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.666+0000 7f1ea5217640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 0x7f1e8406edb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f60 tx=0x7f1e9c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.670+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ea00c92c0 con 0x7f1ea015f720 2024-04-26T21:21:56.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.857+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f1ea00d1f60 con 0x7f1e8406c8f0 2024-04-26T21:21:56.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.858+0000 7f1e95ffb640 1 -- 172.21.15.40:0/857131191 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1ea00d1f60 con 0x7f1e8406c8f0 2024-04-26T21:21:56.859 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 msgr2=0x7f1e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 0x7f1e8406edb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f60 tx=0x7f1e9c007d50 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 msgr2=0x7f1ea00c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1e9802ac00 tx=0x7f1e98004a40 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 shutdown_connections 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea015f720 0x7f1ea00c8d80 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea015f340 0x7f1ea00ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 --2- 172.21.15.40:0/857131191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea00d5610 0x7f1ea0156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 >> 172.21.15.40:0/857131191 conn(0x7f1ea008ec70 msgr2=0x7f1ea00b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 shutdown_connections 2024-04-26T21:21:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.860+0000 7f1ea747b640 1 -- 172.21.15.40:0/857131191 wait complete. 2024-04-26T21:21:56.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:21:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.971+0000 7ff3f70c4640 1 Processor -- start 2024-04-26T21:21:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.971+0000 7ff3f70c4640 1 -- start start 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f70c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f70c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f006e490 0x7ff3f015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f70c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f00d1740 con 0x7ff3f00d11b0 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f70c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f00d1a20 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f70c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f00d1d00 con 0x7ff3f006e490 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57658/0 (socket says 172.21.15.40:57658) 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 -- 172.21.15.40:0/1432530443 learned_addr learned my addr 172.21.15.40:0/1432530443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f5661640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f006e490 0x7ff3f015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 -- 172.21.15.40:0/1432530443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f006e490 msgr2=0x7ff3f015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f006e490 0x7ff3f015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 -- 172.21.15.40:0/1432530443 >> v1:172.21.15.184:6789/0 conn(0x7ff3f00d11b0 legacy=0x7ff3f006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.972+0000 7ff3f4e60640 1 -- 172.21.15.40:0/1432530443 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f00d1fe0 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.973+0000 7ff3f4e60640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0098a0 tx=0x7ff3ec02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b4255fab3616c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.973+0000 7ff3e77fe640 1 -- 172.21.15.40:0/1432530443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec037070 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.973+0000 7ff3e77fe640 1 -- 172.21.15.40:0/1432530443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec032440 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.973+0000 7ff3e77fe640 1 -- 172.21.15.40:0/1432530443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec032810 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 -- 172.21.15.40:0/1432530443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f00cf2e0 msgr2=0x7ff3f006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0098a0 tx=0x7ff3ec02a290 comp rx=0 tx=0).stop 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 -- 172.21.15.40:0/1432530443 shutdown_connections 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f006e490 0x7ff3f015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/1432530443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f006d4d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 -- 172.21.15.40:0/1432530443 >> 172.21.15.40:0/1432530443 conn(0x7ff3f00b4a60 msgr2=0x7ff3f00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 -- 172.21.15.40:0/1432530443 shutdown_connections 2024-04-26T21:21:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.974+0000 7ff3f70c4640 1 -- 172.21.15.40:0/1432530443 wait complete. 2024-04-26T21:21:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.975+0000 7ff3f70c4640 1 Processor -- start 2024-04-26T21:21:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.975+0000 7ff3f70c4640 1 -- start start 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f0148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f00d11b0 0x7ff3f014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f00cf6c0 con 0x7ff3f006e490 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f006da10 con 0x7ff3f00cf2e0 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f70c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f00cf840 con 0x7ff3f00d11b0 2024-04-26T21:21:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57662/0 (socket says 172.21.15.40:57662) 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 -- 172.21.15.40:0/914823823 learned_addr learned my addr 172.21.15.40:0/914823823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f5661640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f00d11b0 0x7ff3f014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3e7fff640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f0148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 -- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f00d11b0 msgr2=0x7ff3f014b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f00d11b0 0x7ff3f014b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 -- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 msgr2=0x7ff3f0148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f0148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 -- 172.21.15.40:0/914823823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01ea440 con 0x7ff3f006e490 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3e7fff640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3f4e60640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.976+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec041020 con 0x7ff3f006e490 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.977+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0306c0 con 0x7ff3f006e490 2024-04-26T21:21:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.977+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec037040 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.977+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01ea720 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.977+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01eace0 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.978+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f006a290 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.978+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3ec044020 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.978+0000 7ff3e57fa640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.979+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7ff3ec0ac050 con 0x7ff3f006e490 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.979+0000 7ff3e7fff640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.979+0000 7ff3e7fff640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00b5cf0 tx=0x7ff3e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:56.983+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3ec04b030 con 0x7ff3f006e490 2024-04-26T21:21:57.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.168+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3f00b8ef0 con 0x7ff3dc06c8f0 2024-04-26T21:21:57.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.170+0000 7ff3e57fa640 1 -- 172.21.15.40:0/914823823 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13471 (secure 0 0 0) 0x7ff3f00b8ef0 con 0x7ff3dc06c8f0 2024-04-26T21:21:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:21:57.171 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 msgr2=0x7ff3dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00b5cf0 tx=0x7ff3e0009040 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 msgr2=0x7ff3f01e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 shutdown_connections 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3f00d11b0 0x7ff3f014b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3f00cf2e0 0x7ff3f0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 --2- 172.21.15.40:0/914823823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3f006e490 0x7ff3f01e9da0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 >> 172.21.15.40:0/914823823 conn(0x7ff3f00b4a60 msgr2=0x7ff3f0071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 shutdown_connections 2024-04-26T21:21:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:21:57.172+0000 7ff3f70c4640 1 -- 172.21.15.40:0/914823823 wait complete. 2024-04-26T21:21:57.183 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-04-26T21:21:57.062874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":273468,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701252,"statfs":{"total":773094113280,"available":772814082048,"internally_reserved":0,"allocated":16064512,"data_stored":7589344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.947793"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":398,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T21:21:49.670387+0000","last_change":"2024-04-26T21:11:51.054596+0000","last_active":"2024-04-26T21:21:49.670387+0000","last_peered":"2024-04-26T21:21:49.670387+0000","last_clean":"2024-04-26T21:21:49.670387+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:49.670387+0000","last_undegraded":"2024-04-26T21:21:49.670387+0000","last_fullsized":"2024-04-26T21:21:49.670387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:11:49.636117+0000","last_clean_scrub_stamp":"2024-04-26T21:11:49.636117+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:29.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2453504,"data_stored":1391444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509564,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37160,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2453504,"data_stored":1391444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:21:57.186 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T21:11:49.636117+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=11, tm_sec=49, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=21, tm_sec=52, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T21:21:57.186 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T21:22:02.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:02.987 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:17.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:22:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.302+0000 7f25c0abd640 1 Processor -- start 2024-04-26T21:22:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.302+0000 7f25c0abd640 1 -- start start 2024-04-26T21:22:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25c0abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25bc146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25c0abd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f25bc0cff20 0x7f25bc1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25c0abd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25bc0d05b0 con 0x7f25bc140350 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25c0abd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 -- 0x7f25bc0d0390 con 0x7f25bc0cff20 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25c0abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25bc0d0890 con 0x7f25bc13ff70 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.303+0000 7f25b9d9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f25bc140350 0x7f25bc0cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36500/0 (socket says 172.21.15.40:36500) 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b9d9b640 1 -- 172.21.15.40:0/1674366361 learned_addr learned my addr 172.21.15.40:0/1674366361 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25bad9d640 1 --2- 172.21.15.40:0/1674366361 >> v2:172.21.15.184:3300/0 conn(0x7f25bc0cff20 0x7f25bc1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25ba59c640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25bc146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942729174 0 0) 0x7f25bc0d05b0 con 0x7f25bc140350 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a4003520 con 0x7f25bc140350 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2613007548 0 0) 0x7f25a4003520 con 0x7f25bc140350 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 msgr2=0x7f25bc146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25bc146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 >> v2:172.21.15.184:3300/0 conn(0x7f25bc0cff20 msgr2=0x7f25bc1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 --2- 172.21.15.40:0/1674366361 >> v2:172.21.15.184:3300/0 conn(0x7f25bc0cff20 0x7f25bc1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc06e450 con 0x7f25bc140350 2024-04-26T21:22:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f25b0001670 con 0x7f25bc140350 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 >> v1:172.21.15.40:6789/0 conn(0x7f25bc140350 legacy=0x7f25bc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25a4003f30 0x7f25a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25a4004d40 0x7f25a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d0390 con 0x7f25a4004d40 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d0890 con 0x7f25a4003f30 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25a400ba10 con 0x7f25bc13ff70 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25bad9d640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25a4004d40 0x7f25a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25ba59c640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.304+0000 7f25b9d9b640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25a4003f30 0x7f25a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 -- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 msgr2=0x7f25a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 -- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25a4003f30 msgr2=0x7f25a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25a4003f30 0x7f25a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 -- 172.21.15.40:0/1674366361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a4003520 con 0x7f25a4004d40 2024-04-26T21:22:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25ba59c640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25bc13ff70 0x7f25a40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25bad9d640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25a4004d40 0x7f25a40051c0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e170 tx=0x7f25a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3c21de1ad39a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.305+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac034020 con 0x7f25a4004d40 2024-04-26T21:22:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ac030430 con 0x7f25a4004d40 2024-04-26T21:22:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25b959a640 1 -- 172.21.15.40:0/1674366361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009c10 con 0x7f25a4004d40 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 -- 172.21.15.40:0/1674366361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25a4004d40 msgr2=0x7f25a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 --2- 172.21.15.40:0/1674366361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25a4004d40 0x7f25a40051c0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e170 tx=0x7f25a4005830 comp rx=0 tx=0).stop 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25bad9d640 1 -- 172.21.15.40:0/1674366361 reap_dead start 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 -- 172.21.15.40:0/1674366361 shutdown_connections 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 -- 172.21.15.40:0/1674366361 >> 172.21.15.40:0/1674366361 conn(0x7f25bc0d0d30 msgr2=0x7f25bc0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 -- 172.21.15.40:0/1674366361 shutdown_connections 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.306+0000 7f25c0abd640 1 -- 172.21.15.40:0/1674366361 wait complete. 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 Processor -- start 2024-04-26T21:22:17.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 -- start start 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 0x7f25bc13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 0x7f25bc1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc146200 con 0x7f25bc1f53a0 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc146080 con 0x7f25bc1f6020 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.308+0000 7f25c0abd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f25bc146380 con 0x7f25a4004d40 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56160/0 (socket says 172.21.15.40:56160) 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 -- 172.21.15.40:0/3026050030 learned_addr learned my addr 172.21.15.40:0/3026050030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25b9d9b640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 0x7f25bc1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25ba59c640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 0x7f25bc13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 -- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 msgr2=0x7f25bc13f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 0x7f25bc13f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 -- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 msgr2=0x7f25bc1f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 0x7f25bc1f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 -- 172.21.15.40:0/3026050030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc1f5950 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25ba59c640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 0x7f25bc13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25b9d9b640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 0x7f25bc1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25bad9d640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e170 tx=0x7f25ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac034040 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ac003b90 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1f5c30 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009c10 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.309+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc157810 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.310+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc1f5e00 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.311+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25ac037020 con 0x7f25bc1f53a0 2024-04-26T21:22:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.311+0000 7f25aaffd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 0x7f259c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.311+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f25ac0ac6b0 con 0x7f25bc1f53a0 2024-04-26T21:22:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.312+0000 7f25ba59c640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 0x7f259c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.312+0000 7f25ba59c640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 0x7f259c06ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009b50 tx=0x7f25b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.316+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25ac049060 con 0x7f25bc1f53a0 2024-04-26T21:22:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.498+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 --> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25bc06a290 con 0x7f259c06c920 2024-04-26T21:22:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.499+0000 7f25aaffd640 1 -- 172.21.15.40:0/3026050030 <== mgr.4105 v2:172.21.15.184:6832/2784003994 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f25bc06a290 con 0x7f259c06c920 2024-04-26T21:22:17.500 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:22:17.501 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-26T21:22:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 msgr2=0x7f259c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 0x7f259c06ede0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009b50 tx=0x7f25b4008040 comp rx=0 tx=0).stop 2024-04-26T21:22:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 msgr2=0x7f25bc144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00e170 tx=0x7f25ac009750 comp rx=0 tx=0).stop 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 shutdown_connections 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f259c06c920 0x7f259c06ede0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25bc1f53a0 0x7f25bc144b40 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25bc1f6020 0x7f25bc1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 --2- 172.21.15.40:0/3026050030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25a4004d40 0x7f25bc13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 >> 172.21.15.40:0/3026050030 conn(0x7f25bc0d0d30 msgr2=0x7f25bc0b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 shutdown_connections 2024-04-26T21:22:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.501+0000 7f25c0abd640 1 -- 172.21.15.40:0/3026050030 wait complete. 2024-04-26T21:22:17.513 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-04-26T21:22:17.066914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":273452,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754701268,"statfs":{"total":773094113280,"available":772814098432,"internally_reserved":0,"allocated":16064512,"data_stored":7589344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002478"},"pg_stats":[{"pgid":"1.0","version":"148'119","reported_seq":409,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-26T21:21:57.505439+0000","last_change":"2024-04-26T21:21:57.505496+0000","last_active":"2024-04-26T21:21:57.505439+0000","last_peered":"2024-04-26T21:21:57.505439+0000","last_clean":"2024-04-26T21:21:57.505439+0000","last_became_active":"2024-04-26T21:11:51.053558+0000","last_became_peered":"2024-04-26T21:11:51.053558+0000","last_unstale":"2024-04-26T21:21:57.505439+0000","last_undegraded":"2024-04-26T21:21:57.505439+0000","last_fullsized":"2024-04-26T21:21:57.505439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'119","last_scrub_stamp":"2024-04-26T21:21:57.505332+0000","last_deep_scrub":"148'119","last_deep_scrub_stamp":"2024-04-26T21:21:57.505332+0000","last_clean_scrub_stamp":"2024-04-26T21:21:57.505332+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:50.711508+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509569,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":2453504,"data_stored":1391444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509569,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37160,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509568,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":2453504,"data_stored":1391444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1859584,"data_stored":801076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:22:17.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T21:22:17.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.613+0000 7f610df9d640 1 Processor -- start 2024-04-26T21:22:17.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f610df9d640 1 -- start start 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f610df9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f61080d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f610df9d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61080d58f0 con 0x7f610815f680 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f610df9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61080d3c90 con 0x7f61080d3820 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f610df9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f610815fa60 con 0x7f61080d4f20 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.614+0000 7f61077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f610815f680 0x7f6108163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36510/0 (socket says 172.21.15.40:36510) 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61077fe640 1 -- 172.21.15.40:0/780655191 learned_addr learned my addr 172.21.15.40:0/780655191 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:22:17.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137493934 0 0) 0x7f61080d58f0 con 0x7f610815f680 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f6107fff640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f61080d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f0003520 con 0x7f610815f680 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 897925606 0 0) 0x7f60f0003520 con 0x7f610815f680 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 msgr2=0x7f61080d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f61080d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 >> v1:172.21.15.184:6789/0 conn(0x7f61080d3820 legacy=0x7f61080b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61080d3f50 con 0x7f610815f680 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1671293776 0 0) 0x7f60f8001670 con 0x7f610815f680 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 >> v1:172.21.15.40:6789/0 conn(0x7f610815f680 legacy=0x7f6108163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00039e0 0x7f60f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f60f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60f0004d40 0x7f60f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f610815fa60 con 0x7f60f00039e0 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60f000ba70 con 0x7f60f0004d40 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f60f000bc70 con 0x7f61080d4f20 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f61077fe640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00039e0 0x7f60f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f6107fff640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60f0004d40 0x7f60f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.615+0000 7f6106ffd640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f60f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 -- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 msgr2=0x7f60f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f60f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 -- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60f0004d40 msgr2=0x7f60f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60f0004d40 0x7f60f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 -- 172.21.15.40:0/780655191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f0003520 con 0x7f60f00039e0 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f6106ffd640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61080d4f20 0x7f60f0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61077fe640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00039e0 0x7f60f0003e00 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f60f8001eb0 tx=0x7f60f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed3f3cc7f06d1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8007d30 con 0x7f60f00039e0 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f80058d0 con 0x7f60f00039e0 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.616+0000 7f61067fc640 1 -- 172.21.15.40:0/780655191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8005ce0 con 0x7f60f00039e0 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 -- 172.21.15.40:0/780655191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00039e0 msgr2=0x7f60f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 --2- 172.21.15.40:0/780655191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00039e0 0x7f60f0003e00 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f60f8001eb0 tx=0x7f60f0005830 comp rx=0 tx=0).stop 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f6107fff640 1 -- 172.21.15.40:0/780655191 reap_dead start 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 -- 172.21.15.40:0/780655191 shutdown_connections 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 -- 172.21.15.40:0/780655191 >> 172.21.15.40:0/780655191 conn(0x7f610808eca0 msgr2=0x7f610809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 -- 172.21.15.40:0/780655191 shutdown_connections 2024-04-26T21:22:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.617+0000 7f610df9d640 1 -- 172.21.15.40:0/780655191 wait complete. 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.618+0000 7f610df9d640 1 Processor -- start 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.618+0000 7f610df9d640 1 -- start start 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60f00039e0 0x7f6108155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61080b7e50 0x7f61080cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61080b6280 con 0x7f61080cb650 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61080b5370 con 0x7f61080b7e50 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f610df9d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f61080d53b0 con 0x7f60f00039e0 2024-04-26T21:22:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56180/0 (socket says 172.21.15.40:56180) 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 -- 172.21.15.40:0/3977062920 learned_addr learned my addr 172.21.15.40:0/3977062920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 -- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60f00039e0 msgr2=0x7f6108155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60f00039e0 0x7f6108155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 -- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61080b7e50 msgr2=0x7f61080cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61080b7e50 0x7f61080cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.619+0000 7f6107fff640 1 -- 172.21.15.40:0/3977062920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61080cbb00 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f6107fff640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f610808ebd0 tx=0x7f610000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000fe10 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f610001d070 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6108161cb0 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100015670 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.620+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61080cd670 con 0x7f61080cb650 2024-04-26T21:22:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.621+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610806a290 con 0x7f61080cb650 2024-04-26T21:22:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.621+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6100015810 con 0x7f61080cb650 2024-04-26T21:22:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.622+0000 7f60effff640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.622+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f610008c0c0 con 0x7f61080cb650 2024-04-26T21:22:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.622+0000 7f61077fe640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.623+0000 7f61077fe640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 0x7f60f406ede0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f60f8033000 tx=0x7f60f8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.626+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6100010040 con 0x7f61080cb650 2024-04-26T21:22:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.810+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f61080baf10 con 0x7f61080cb650 2024-04-26T21:22:17.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.813+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v68)=0 v68) ==== 123+0+0 (secure 0 0 0) 0x7f610005a7f0 con 0x7f61080cb650 2024-04-26T21:22:17.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.814+0000 7f60effff640 1 -- 172.21.15.40:0/3977062920 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f610005a0f0 con 0x7f61080cb650 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 msgr2=0x7f60f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 0x7f60f406ede0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f60f8033000 tx=0x7f60f8009030 comp rx=0 tx=0).stop 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 msgr2=0x7f6108160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f610808ebd0 tx=0x7f610000f580 comp rx=0 tx=0).stop 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 shutdown_connections 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f60f406c920 0x7f60f406ede0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61080cb650 0x7f6108160f90 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61080b7e50 0x7f61080cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 --2- 172.21.15.40:0/3977062920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60f00039e0 0x7f6108155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 >> 172.21.15.40:0/3977062920 conn(0x7f610808eca0 msgr2=0x7f61080b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 shutdown_connections 2024-04-26T21:22:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.816+0000 7f610df9d640 1 -- 172.21.15.40:0/3977062920 wait complete. 2024-04-26T21:22:17.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:22:17.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:22:17.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.928+0000 7f214d615640 1 Processor -- start 2024-04-26T21:22:17.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.928+0000 7f214d615640 1 -- start start 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f214d615640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21480d3e80 0x7f21480d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f214d615640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480b4090 0x7f21480d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f214d615640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21480d3230 con 0x7f214815f510 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f214d615640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f21480cfd10 con 0x7f21480d3e80 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f214d615640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21480cffb0 con 0x7f21480b4090 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f21467fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21480d3e80 0x7f21480d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f2146ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f214815f510 0x7f21480d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36516/0 (socket says 172.21.15.40:36516) 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.929+0000 7f2146ffd640 1 -- 172.21.15.40:0/3672647085 learned_addr learned my addr 172.21.15.40:0/3672647085 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f21467fc640 1 -- 172.21.15.40:0/3672647085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480b4090 msgr2=0x7f21480d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f21467fc640 1 --2- 172.21.15.40:0/3672647085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480b4090 0x7f21480d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f21467fc640 1 -- 172.21.15.40:0/3672647085 >> v1:172.21.15.40:6789/0 conn(0x7f214815f510 legacy=0x7f21480d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f2145ffb640 1 -- 172.21.15.40:0/3672647085 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630608558 0 0) 0x7f21480d3230 con 0x7f214815f510 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f21467fc640 1 -- 172.21.15.40:0/3672647085 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21480d0290 con 0x7f21480d3e80 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f21467fc640 1 --2- 172.21.15.40:0/3672647085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480d3e80 0x7f21480d0d70 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f21300097a0 tx=0x7f213002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63443fef1e737627 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f2145ffb640 1 -- 172.21.15.40:0/3672647085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130037070 con 0x7f21480d3e80 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f2145ffb640 1 -- 172.21.15.40:0/3672647085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2130032440 con 0x7f21480d3e80 2024-04-26T21:22:17.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.930+0000 7f2145ffb640 1 -- 172.21.15.40:0/3672647085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130032830 con 0x7f21480d3e80 2024-04-26T21:22:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 -- 172.21.15.40:0/3672647085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480d3e80 msgr2=0x7f21480d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 --2- 172.21.15.40:0/3672647085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480d3e80 0x7f21480d0d70 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f21300097a0 tx=0x7f213002a180 comp rx=0 tx=0).stop 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 -- 172.21.15.40:0/3672647085 shutdown_connections 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 --2- 172.21.15.40:0/3672647085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480b4090 0x7f21480d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 --2- 172.21.15.40:0/3672647085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480d3e80 0x7f21480d0d70 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 -- 172.21.15.40:0/3672647085 >> 172.21.15.40:0/3672647085 conn(0x7f214809e4d0 msgr2=0x7f21480d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 -- 172.21.15.40:0/3672647085 shutdown_connections 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.931+0000 7f214d615640 1 -- 172.21.15.40:0/3672647085 wait complete. 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.932+0000 7f214d615640 1 Processor -- start 2024-04-26T21:22:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.932+0000 7f214d615640 1 -- start start 2024-04-26T21:22:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480d3e80 0x7f21480a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 0x7f21480ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f214815f8f0 con 0x7f21480b4090 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21480d0850 con 0x7f214815f510 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f214d615640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 -- 0x7f21480b1dc0 con 0x7f21480d3e80 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f2146ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f2146ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56200/0 (socket says 172.21.15.40:56200) 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f2146ffd640 1 -- 172.21.15.40:0/1510236385 learned_addr learned my addr 172.21.15.40:0/1510236385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f2146ffd640 1 -- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480d3e80 msgr2=0x7f21480a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.933+0000 7f21467fc640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480d3e80 0x7f21480a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f2146ffd640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480d3e80 0x7f21480a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f2146ffd640 1 -- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 msgr2=0x7f21480ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f21477fe640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 0x7f21480ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f2146ffd640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 0x7f21480ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f2146ffd640 1 -- 172.21.15.40:0/1510236385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214815ed40 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f21477fe640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 0x7f21480ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f2146ffd640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f213800c7c0 tx=0x7f213800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138018070 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f213800a5e0 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21480c9740 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213800a9f0 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.934+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21480c9d00 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.935+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21480693d0 con 0x7f21480b4090 2024-04-26T21:22:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.936+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f213801b070 con 0x7f21480b4090 2024-04-26T21:22:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.936+0000 7f213f7fe640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.936+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4943+0+0 (secure 0 0 0) 0x7f213808ae10 con 0x7f21480b4090 2024-04-26T21:22:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.937+0000 7f21467fc640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.937+0000 7f21467fc640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 0x7f212806edb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2130034000 tx=0x7f213002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:17.941+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f214815ed40 con 0x7f21480b4090 2024-04-26T21:22:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.167+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21480d58a0 con 0x7f21480b4090 2024-04-26T21:22:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.167+0000 7f213f7fe640 1 -- 172.21.15.40:0/1510236385 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+3877 (secure 0 0 0) 0x7f2138059540 con 0x7f21480b4090 2024-04-26T21:22:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-26T21:22:18.170 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2791503221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2791503221},{"type":"v1","addr":"172.21.15.184:6835","nonce":2791503221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2972465500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2972465500},{"type":"v1","addr":"172.21.15.184:6837","nonce":2972465500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2638999824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2638999824},{"type":"v1","addr":"172.21.15.40:6834","nonce":2638999824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1470631843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":1470631843},{"type":"v1","addr":"172.21.15.40:6835","nonce":1470631843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[]} 2024-04-26T21:22:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 101 2024-04-26T21:22:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.169+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 msgr2=0x7f212806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.169+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 0x7f212806edb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2130034000 tx=0x7f213002e000 comp rx=0 tx=0).stop 2024-04-26T21:22:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.169+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 msgr2=0x7f21480c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.169+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f213800c7c0 tx=0x7f213800cc80 comp rx=0 tx=0).stop 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 shutdown_connections 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:6832/2784003994,v1:172.21.15.184:6833/2784003994] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f214815f510 0x7f21480ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21480d3e80 0x7f21480a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 --2- 172.21.15.40:0/1510236385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21480b4090 0x7f21480c9040 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 >> 172.21.15.40:0/1510236385 conn(0x7f214809e4d0 msgr2=0x7f214806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 shutdown_connections 2024-04-26T21:22:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-26T21:22:18.170+0000 7f214d615640 1 -- 172.21.15.40:0/1510236385 wait complete. 2024-04-26T21:22:18.181 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T21:22:18.182 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T21:22:18.182 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:22:18.182 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.191 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:22:18.191 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:22:18.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.241 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:22:18.241 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:22:18.241 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.253 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:22:18.253 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:22:18.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.305 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:22:18.305 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T21:22:18.306 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T21:22:18.306 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.333 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T21:22:18.334 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T21:22:18.334 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.403 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T21:22:18.403 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T21:22:18.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.476 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T21:22:18.477 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T21:22:18.477 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.547 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T21:22:18.547 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T21:22:18.547 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.586 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T21:22:18.587 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T21:22:18.587 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.665 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T21:22:18.665 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T21:22:18.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.731 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T21:22:18.732 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T21:22:18.732 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.812 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T21:22:18.813 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T21:22:18.813 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T21:22:18.813 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.839 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T21:22:18.839 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T21:22:18.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.881 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T21:22:18.881 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T21:22:18.882 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T21:22:18.882 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.893 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T21:22:18.893 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T21:22:18.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.934 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T21:22:18.934 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T21:22:18.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:22:18.986 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T21:22:18.987 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T21:22:18.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2024-04-26T21:22:19.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:22:19.025 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T21:22:19.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:22:19.223 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T21:22:19.304 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:22:19.304 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T21:22:19.362 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:22:19.363 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T21:22:19.441 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:22:19.441 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T21:22:19.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:22:19.573 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T21:22:19.686 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:22:19.686 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T21:22:19.734 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:22:19.735 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T21:22:19.785 INFO:tasks.ceph:Archiving mon data... 2024-04-26T21:22:19.786 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/data/mon.a.tgz 2024-04-26T21:22:19.787 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-26T21:22:19.802 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.xUUcjXrCc8 2024-04-26T21:22:19.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xUUcjXrCc8 2024-04-26T21:22:19.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.xUUcjXrCc8 2024-04-26T21:22:20.044 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.xUUcjXrCc8 is 440KB 2024-04-26T21:22:20.144 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.xUUcjXrCc8 2024-04-26T21:22:20.159 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/data/mon.b.tgz 2024-04-26T21:22:20.159 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-04-26T21:22:20.177 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.c56rwnU0bt 2024-04-26T21:22:20.178 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.c56rwnU0bt 2024-04-26T21:22:20.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.c56rwnU0bt 2024-04-26T21:22:20.421 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.c56rwnU0bt is 449KB 2024-04-26T21:22:20.521 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.c56rwnU0bt 2024-04-26T21:22:20.536 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/data/mon.c.tgz 2024-04-26T21:22:20.536 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-04-26T21:22:20.590 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.rovNod8n9G 2024-04-26T21:22:20.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rovNod8n9G 2024-04-26T21:22:20.736 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.rovNod8n9G 2024-04-26T21:22:20.816 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.rovNod8n9G is 449KB 2024-04-26T21:22:20.912 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.rovNod8n9G 2024-04-26T21:22:20.927 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T21:22:20.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:22:20.929 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:22:21.014 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:22:21.045 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:22:21.045 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T21:22:21.047 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi040/crash 2024-04-26T21:22:21.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:22:21.082 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi184/crash 2024-04-26T21:22:21.083 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:22:21.107 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T21:22:21.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:21.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:21.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:21.216 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:22:21.253 INFO:tasks.ceph:Compressing logs... 2024-04-26T21:22:21.253 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:22:21.259 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41190.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35130.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.41190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35170.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41190.log.gz 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.35130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43287.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35130.log.gz 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.35170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46488.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43287.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35170.log.gz 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43287.log.gz 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43800.log 2024-04-26T21:22:21.372 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46488.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44923.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43800.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.35083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42323.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44923.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44923.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45508.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42323.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41194.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45508.log.gz 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T21:22:21.373 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.41194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41194.log.gz 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34959.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.34959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41807.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34959.log.gz 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45991.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.41807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41807.log.gz 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45991.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.1.45991.log.gz 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr: -- /var/log/ceph/ceph-mon.c.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-04-26T21:22:21.374 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.35210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42805.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41811.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42805.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42319.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.41811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41811.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44405.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42319.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44405.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46492.log 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.35039.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-04-26T21:22:21.375 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44409.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.46492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46492.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42801.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44409.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45987.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.42801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42801.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43283.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45987.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44927.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43283.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40795.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.44927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44927.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45504.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.40795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40795.log.gz 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43796.log 2024-04-26T21:22:21.376 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.45504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45504.log.gz 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40790.log 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.43796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43796.log.gz 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35250.log 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.40790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40790.log.gz 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.35250.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35250.log.gz 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T21:22:21.377 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-04-26T21:22:21.378 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.34999.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-04-26T21:22:21.478 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T21:22:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79136.log 2024-04-26T21:22:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57380.log 2024-04-26T21:22:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45730.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80397.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79136.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57380.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66986.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45730.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74693.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80397.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47207.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66986.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50500.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74693.log: /var/log/ceph/ceph-client.admin.47207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76411.log 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74693.log.gz 2024-04-26T21:22:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50500.log: 89.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.47207.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66818.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50500.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53579.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66818.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76411.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61304.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66818.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80229.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50250.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53579.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7%/var/log/ceph/ceph-client.admin.80229.log: -- replaced with /var/log/ceph/ceph-client.admin.61304.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60319.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80229.log.gz 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69611.log 2024-04-26T21:22:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60319.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50250.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64161.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60319.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58612.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69611.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63395.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64161.log: /var/log/ceph/ceph-client.admin.58612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55569.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64161.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.63395.log: -- replaced with /var/log/ceph/ceph-client.admin.58612.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63395.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74417.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45898.log 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55569.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-04-26T21:22:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71714.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74417.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69192.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71714.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45898.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49721.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71714.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68510.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70246.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49721.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69192.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49721.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68510.log.gz 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58341.log 2024-04-26T21:22:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40230.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.70246.log.gz -5 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78994.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71997.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58341.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61178.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78994.log: /var/log/ceph/ceph-client.admin.71997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50769.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78994.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71997.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57229.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61178.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73770.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50769.log.gz 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55112.log 2024-04-26T21:22:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73770.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57229.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49324.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73770.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68982.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55112.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49991.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49324.log: /var/log/ceph/ceph-client.admin.68982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49324.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68982.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40588.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49991.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34716.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67664.log 2024-04-26T21:22:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54293.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40588.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34716.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.67664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40840.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67664.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80145.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54293.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40840.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69821.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80145.log.gz 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64203.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47379.log: /var/log/ceph/ceph-client.admin.69821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55993.log 2024-04-26T21:22:21.557 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64203.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69821.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64203.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41609.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55993.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47649.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77483.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67826.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41609.log: /var/log/ceph/ceph-client.admin.77483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69569.log 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41609.log.gz 2024-04-26T21:22:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77483.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67826.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61760.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46024.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69569.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46942.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61760.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67028.log/var/log/ceph/ceph-client.admin.45226.log: 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57617.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67028.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46942.log.gz 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58822.log 2024-04-26T21:22:21.559 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67028.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73666.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58822.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57617.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58822.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80888.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46402.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73666.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58257.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46402.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80888.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61982.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46402.log.gz 2024-04-26T21:22:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61982.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.58257.log.gz --verbose 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.80481.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66049.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61982.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76495.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66049.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80481.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78149.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66049.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59081.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76495.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.78149.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41008.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76765.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78149.log.gz 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52489.log 2024-04-26T21:22:21.561 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59081.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41008.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40145.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52489.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76765.log.gz -5 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.0.57105.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40145.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52489.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75089.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.57105.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40145.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40672.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57105.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56208.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40672.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75089.log.gz 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80649.log 2024-04-26T21:22:21.562 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40672.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53058.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56208.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45772.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56164.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80649.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45772.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53058.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45772.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44343.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56164.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72285.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gz 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79430.log 2024-04-26T21:22:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68247.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44343.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79430.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72285.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67322.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79430.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40924.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67322.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68247.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46688.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40924.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67322.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62186.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40924.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.46688.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59713.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56473.log 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46688.log.gz 2024-04-26T21:22:21.564 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62186.log.gz/var/log/ceph/ceph-client.admin.59713.log: 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66220.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56473.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59713.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55401.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66220.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56473.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75047.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66220.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52741.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.71375.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55401.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75047.log: /var/log/ceph/ceph-client.admin.52741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56389.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75047.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52741.log.gz 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44659.log 2024-04-26T21:22:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56389.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71375.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56389.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67742.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44659.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42772.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42310.log: /var/log/ceph/ceph-client.admin.67742.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42772.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67742.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75005.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50075.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42772.log.gz 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58696.log 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77654.log 88.2% 2024-04-26T21:22:21.566 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75005.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.58696.log: -- replaced with /var/log/ceph/ceph-client.admin.50075.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58696.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61574.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77654.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61803.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61574.log.gz 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40882.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75365.log 2024-04-26T21:22:21.567 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61803.log.gz/var/log/ceph/ceph-client.admin.40882.log: 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62538.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40882.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53449.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62538.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75365.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75428.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53449.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.57743.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62538.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53449.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65527.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57743.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75428.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50639.log 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65527.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57743.log.gz 2024-04-26T21:22:21.568 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76891.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65527.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59899.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50639.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61406.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76891.log: /var/log/ceph/ceph-client.admin.59899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76891.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64797.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2%/var/log/ceph/ceph-client.admin.46234.log: -- replaced with /var/log/ceph/ceph-client.admin.59899.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61406.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66776.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.64797.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.49949.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64797.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65217.log 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66776.log.gz 2024-04-26T21:22:21.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60445.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49949.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65217.log: /var/log/ceph/ceph-client.admin.60445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73566.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65217.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44616.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60445.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71840.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73566.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46276.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67994.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73566.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71840.log.gz 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46900.log 2024-04-26T21:22:21.570 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46276.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49823.log/var/log/ceph/ceph-client.admin.67994.log: 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67994.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49823.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46900.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70419.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44049.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.49823.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59815.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70419.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70871.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70419.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59815.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59815.log.gz 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62312.log 2024-04-26T21:22:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70871.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47026.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70871.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45814.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62312.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72553.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47026.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42142.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79304.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72553.log.gz 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64497.log 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79304.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42142.log.gz -- 2024-04-26T21:22:21.572 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.0.71171.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79304.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59204.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64497.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79659.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.71171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71171.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41942.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59204.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73411.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%/var/log/ceph/ceph-client.admin.41942.log: -- replaced with /var/log/ceph/ceph-client.admin.79659.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41271.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67280.log 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41942.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73411.log.gz 2024-04-26T21:22:21.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76017.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67280.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41271.log.gz -- 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.50811.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67280.log.gz 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49492.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76017.log.gz 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49492.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50811.log.gz 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55951.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49240.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49492.log.gz 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70745.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55951.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69066.log 2024-04-26T21:22:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70745.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55951.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.80733.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70745.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44938.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79617.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69066.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76849.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80733.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79617.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44938.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79617.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51620.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76849.log.gz 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62873.log 2024-04-26T21:22:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72679.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71501.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51620.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72679.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62873.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68468.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72679.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40444.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71501.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68468.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55527.log 2024-04-26T21:22:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48852.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40444.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55527.log: -- /var/log/ceph/ceph-client.admin.41605.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66944.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55527.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41605.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70979.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41605.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71417.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70979.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.66944.log.gz 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72637.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71417.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.70979.log.gz -5 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-04-26T21:22:21.577 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71417.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52344.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72637.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70555.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67660.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70555.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52344.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70555.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.67660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70913.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67660.log.gz 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52573.log 2024-04-26T21:22:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53184.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.70913.log.gz 89.1% 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73327.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53314.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53184.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52573.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40314.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73327.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53184.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73327.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53314.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65385.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40314.log: /var/log/ceph/ceph-client.admin.41944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63017.log 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40314.log.gz 2024-04-26T21:22:21.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65385.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65385.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80313.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63017.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61448.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50415.log: /var/log/ceph/ceph-client.admin.80313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42268.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61448.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.80313.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49763.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61448.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58467.log 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42268.log.gz 2024-04-26T21:22:21.580 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74459.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58467.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.49763.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.43259.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58467.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63491.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74459.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.43259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73075.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43259.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64287.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63491.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57947.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59419.log 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73075.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57947.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64287.log.gz 2024-04-26T21:22:21.581 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57947.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76933.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55485.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46192.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59419.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76933.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75729.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55485.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46192.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53495.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53705.log 2024-04-26T21:22:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53495.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.75729.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77897.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53705.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53495.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69234.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53705.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79388.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69234.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77897.log.gz -5 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.76579.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79388.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69234.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67952.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79388.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59461.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76579.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59461.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67952.log.gz 2024-04-26T21:22:21.583 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51129.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59461.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62975.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77568.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62975.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51129.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79220.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62975.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71756.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62831.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77568.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71756.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79220.log.gz 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65755.log 2024-04-26T21:22:21.584 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71756.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46984.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65755.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62831.log.gzgzip 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55172.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46984.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65755.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56599.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46984.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51536.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56599.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55172.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41865.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51536.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56599.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76201.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51536.log.gz 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42184.log 2024-04-26T21:22:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72427.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41865.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42184.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76201.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62144.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42184.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59587.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62144.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72427.log.gz -5 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68331.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59587.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62144.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72369.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59587.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75215.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73117.log 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68331.log.gz 2024-04-26T21:22:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50373.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72369.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75215.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73117.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49098.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50373.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41267.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74165.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41267.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49098.log.gz -5 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74165.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41267.log.gz 2024-04-26T21:22:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50331.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74165.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46646.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46646.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50331.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73812.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70601.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46646.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40188.log: /var/log/ceph/ceph-client.admin.73812.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73812.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70703.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70601.log: /var/log/ceph/ceph-client.admin.42604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50937.log 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70601.log.gz 2024-04-26T21:22:21.588 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8%/var/log/ceph/ceph-client.admin.70703.log: -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80271.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70703.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50937.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54211.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80271.log: /var/log/ceph/ceph-client.admin.51866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62622.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80271.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.54211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48195.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54211.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78449.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62622.log.gz 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53621.log 2024-04-26T21:22:21.589 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48195.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76369.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.78449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78449.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73159.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53621.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74291.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41264.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73159.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76369.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74291.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73159.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72039.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56557.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41264.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74291.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41264.log.gz 2024-04-26T21:22:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58047.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56557.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72039.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65013.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.42688.log -- replaced with /var/log/ceph/ceph-client.admin.56557.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66688.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58047.log: /var/log/ceph/ceph-client.admin.42688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77981.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58047.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42688.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79574.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66688.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47114.log 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77981.log.gz 2024-04-26T21:22:21.591 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53100.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79574.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47114.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79574.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70331.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47114.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77185.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53100.log: /var/log/ceph/ceph-client.admin.70331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44701.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53100.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70331.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64413.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66562.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77185.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64413.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44701.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59629.log 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64413.log.gz 2024-04-26T21:22:21.592 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66358.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66562.log: /var/log/ceph/ceph-client.admin.59629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58864.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66562.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59629.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45310.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4%/var/log/ceph/ceph-client.admin.58864.log: -- replaced with /var/log/ceph/ceph-client.admin.66358.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40756.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58864.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54215.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45310.log: /var/log/ceph/ceph-client.admin.40756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74627.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45310.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40756.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.54215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63909.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54215.log.gz 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-04-26T21:22:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74627.log: /var/log/ceph/ceph-client.admin.63909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69737.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49626.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63909.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65653.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74627.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62066.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69737.log: /var/log/ceph/ceph-client.admin.65653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60848.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69737.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65653.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62066.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60848.log: /var/log/ceph/ceph-client.admin.54860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53407.log 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60848.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46732.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-04-26T21:22:21.594 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69150.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57109.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53407.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69150.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53407.log.gz -5 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75813.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69150.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61490.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.57109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57109.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68205.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75813.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60235.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61490.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68205.log.gz 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45856.log 2024-04-26T21:22:21.595 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60235.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74123.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73033.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45856.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52386.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61346.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74123.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73033.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65797.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64539.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52386.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61346.log: /var/log/ceph/ceph-client.admin.65797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61346.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65797.log.gz 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73471.log 2024-04-26T21:22:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48810.log: /var/log/ceph/ceph-client.admin.46360.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- 90.7% /var/log/ceph/ceph-client.admin.48153.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64539.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73471.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55790.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73471.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77791.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55790.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48153.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41092.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55790.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80775.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77791.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62270.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77791.log.gz 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41273.log 2024-04-26T21:22:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80775.log: /var/log/ceph/ceph-client.admin.62270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79701.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80775.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62270.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75257.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41273.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79701.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50170.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54902.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75257.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72243.log 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2024-04-26T21:22:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64035.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57187.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72243.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50170.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64035.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28436.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72243.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67112.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64035.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.28436.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.57187.log.gz --verbose 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.48684.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28436.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67112.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48684.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80607.log 2024-04-26T21:22:21.599 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72327.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80607.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80607.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61898.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72327.log: /var/log/ceph/ceph-mgr.y.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72327.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65881.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74543.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61898.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47607.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65881.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74543.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43702.log 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47607.log.gz 2024-04-26T21:22:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66438.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47070.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.43702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43702.log.gz 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60361.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66438.log: /var/log/ceph/ceph-client.admin.47070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60361.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66438.log.gz 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47070.log.gz 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60361.log.gz 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.59671.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64793.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79514.log 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59671.log.gz/var/log/ceph/ceph-client.0.64793.log: 2024-04-26T21:22:21.601 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64329.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.0.64793.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79514.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66262.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80848.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64329.log: /var/log/ceph/ceph-client.admin.66262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80848.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66262.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64329.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80848.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60487.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64371.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45982.log: /var/log/ceph/ceph-client.admin.60487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61718.log 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64371.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60487.log.gz 2024-04-26T21:22:21.602 INFO:teuthology.orchestra.run.smithi040.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48726.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64371.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50681.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61718.log: /var/log/ceph/ceph-client.admin.48726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80355.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0%/var/log/ceph/ceph-client.admin.50681.log: -- replaced with /var/log/ceph/ceph-client.admin.61718.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48726.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63779.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50681.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80355.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49907.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76453.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80355.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50979.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63779.log.gz 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49907.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74963.log 2024-04-26T21:22:21.603 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76453.log: /var/log/ceph/ceph-client.admin.50979.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76453.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50979.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69905.log 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44301.log 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74963.log.gz/var/log/ceph/ceph-client.admin.69905.log: 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66604.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69905.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44301.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44301.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.60806.log 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66604.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66604.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.70375.log 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69108.log 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60806.log.gz 2024-04-26T21:22:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80691.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70375.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75131.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69108.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80691.log: /var/log/ceph/ceph-client.admin.45184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60764.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75131.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80691.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75131.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.40966.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39845.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60764.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64875.log 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-04-26T21:22:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48972.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39845.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74249.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64875.log: /var/log/ceph/ceph-client.admin.48972.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64875.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48972.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45268.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69527.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74249.log: /var/log/ceph/ceph-client.admin.45268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53016.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69527.log: 88.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74249.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45268.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69527.log.gz 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63311.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65443.log 2024-04-26T21:22:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53016.log: /var/log/ceph/ceph-client.admin.63311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55443.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65443.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53016.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63311.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49576.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65443.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52050.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55443.log: /var/log/ceph/ceph-client.admin.49576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42646.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52050.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55443.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49576.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79094.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52050.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65301.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59545.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42646.log.gz 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79094.log: /var/log/ceph/ceph-client.admin.65301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40714.log 2024-04-26T21:22:21.607 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79094.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65301.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73369.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59545.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42226.log 88.8% 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40714.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42478.log 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42226.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73369.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58654.log 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42226.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68756.log 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42478.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58906.log 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58654.log: /var/log/ceph/ceph-client.admin.68756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79935.log 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58906.log: 88.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68756.log.gz 2024-04-26T21:22:21.608 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58654.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58906.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66091.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69024.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79935.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69443.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66091.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53663.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69024.log: /var/log/ceph/ceph-client.admin.69443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58299.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53663.log: 87.6% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69443.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69024.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58425.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53663.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48033.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79178.log 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58299.log.gz 2024-04-26T21:22:21.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58425.log: gzip/var/log/ceph/ceph-client.0.48033.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.0.48033.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.52092.log 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58425.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79178.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50895.log 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46150.log 88.2% 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63993.log 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50895.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44530.log 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46150.log: /var/log/ceph/ceph-client.admin.63993.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63993.log.gz 2024-04-26T21:22:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77833.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65611.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.77833.log: -- replaced with /var/log/ceph/ceph-client.admin.44530.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78107.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77833.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75771.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65611.log: /var/log/ceph/ceph-client.admin.78107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76327.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75771.log: 87.5% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78107.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65611.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75771.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34455.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76327.log.gz 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44175.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62412.log 2024-04-26T21:22:21.611 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34455.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45022.log: /var/log/ceph/ceph-client.admin.44175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77749.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44175.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62412.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68144.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62412.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56683.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77749.log: /var/log/ceph/ceph-client.admin.68144.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77749.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68144.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59001.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77441.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59001.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56683.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59001.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41866.log 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77441.log: /var/log/ceph/ceph-client.admin.53831.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77441.log.gz 2024-04-26T21:22:21.612 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80439.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78527.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80439.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41866.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52699.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80439.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78527.log: /var/log/ceph/ceph-client.admin.52699.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78527.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52699.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79743.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51021.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79743.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63865.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79743.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65343.log 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51021.log: /var/log/ceph/ceph-client.admin.63865.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51021.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63865.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72763.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65343.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63269.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57659.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72763.log: /var/log/ceph/ceph-client.admin.63269.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72763.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63269.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73243.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75572.log 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73243.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57659.log.gz 2024-04-26T21:22:21.614 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72805.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76537.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73243.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75572.log: /var/log/ceph/ceph-client.admin.72805.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75572.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72805.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41607.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48029.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41607.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76537.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60638.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.0.48029.log: -- replaced with /var/log/ceph/ceph-client.admin.41607.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48029.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60638.log.gz 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66646.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-04-26T21:22:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66646.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66134.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66646.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72847.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49198.log: /var/log/ceph/ceph-client.admin.66134.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66134.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66520.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79262.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66520.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.72847.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69695.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66520.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79262.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79262.log.gz 2024-04-26T21:22:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69695.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69695.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68289.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56851.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56259.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56851.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68289.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56851.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46774.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55358.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56259.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46774.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46774.log.gz 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63059.log 2024-04-26T21:22:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56343.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55358.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63059.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63059.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68552.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56343.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71459.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68552.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68552.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51494.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71459.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67910.log 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51494.log.gz 2024-04-26T21:22:21.618 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56431.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56035.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56431.log: /var/log/ceph/ceph-client.admin.67910.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56431.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67910.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44091.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78868.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56035.log.gz/var/log/ceph/ceph-client.admin.44091.log: 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43965.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78868.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44091.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78868.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68714.log 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41272.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43965.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-04-26T21:22:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47505.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65569.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47505.log: /var/log/ceph/ceph-client.admin.68714.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68714.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47505.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70829.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60680.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65569.log: /var/log/ceph/ceph-client.admin.70829.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65569.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70829.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67196.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60680.log: /var/log/ceph/ceph-client.admin.52218.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60680.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63353.log 2024-04-26T21:22:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67196.log: -- /var/log/ceph/ceph-client.admin.76243.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67196.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76243.log: 88.5% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76243.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63353.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56767.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76159.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59503.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56767.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78023.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59503.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76159.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59503.log.gz 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61532.log 2024-04-26T21:22:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50544.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61532.log: /var/log/ceph/ceph-client.admin.78023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78023.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61532.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80019.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57271.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50544.log: /var/log/ceph/ceph-client.admin.80019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50544.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80019.log.gzgzip 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53873.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75173.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53873.log: /var/log/ceph/ceph-client.admin.57271.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57271.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53873.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62454.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72469.log 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75173.log: /var/log/ceph/ceph-client.admin.62454.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75173.log.gz 2024-04-26T21:22:21.622 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57821.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62454.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70289.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57821.log: 87.8% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72469.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57821.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69863.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70289.log: /var/log/ceph/ceph-client.admin.45106.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70289.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67472.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45688.log: /var/log/ceph/ceph-client.admin.69863.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69863.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40504.log 2024-04-26T21:22:21.623 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49865.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67472.log: /var/log/ceph/ceph-client.admin.40504.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67472.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44574.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40504.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44574.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49865.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44574.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62790.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58738.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53142.log: /var/log/ceph/ceph-client.admin.62790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41050.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62790.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78191.log 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41050.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58738.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41050.log.gz 2024-04-26T21:22:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.78191.log: -- /var/log/ceph/ceph-client.admin.54760.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41610.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.78191.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74077.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54760.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64245.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74077.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78445.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64245.log: -- /var/log/ceph/ceph-client.admin.67406.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64245.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.78445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77525.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78445.log.gz 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-04-26T21:22:21.625 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67406.log.gz 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61940.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41945.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77525.log.gz 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.77017.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44827.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1%/var/log/ceph/ceph-client.admin.77017.log: -- replaced with /var/log/ceph/ceph-client.admin.61940.log.gz 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40798.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77017.log.gz/var/log/ceph/ceph-client.admin.44827.log: 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65485.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44827.log.gz 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73728.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%/var/log/ceph/ceph-client.admin.65485.log: -- replaced with /var/log/ceph/ceph-client.admin.40798.log.gz 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65485.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58383.log 2024-04-26T21:22:21.626 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73728.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73728.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52801.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62706.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58383.log: /var/log/ceph/ceph-client.admin.52801.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58383.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52801.log.gz/var/log/ceph/ceph-client.admin.62706.log: 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67154.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62706.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67154.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68036.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43839.log: -- /var/log/ceph/ceph-client.admin.54430.log 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-04-26T21:22:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63653.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54430.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68036.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60153.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56917.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63653.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.60153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60153.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65695.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56917.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56917.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73984.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63737.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65695.log: /var/log/ceph/ceph-client.admin.73984.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65695.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65839.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63737.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73984.log.gz 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63737.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45484.log 2024-04-26T21:22:21.628 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69318.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65839.log: /var/log/ceph/ceph-client.admin.45484.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65839.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63571.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45484.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47841.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63571.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.69318.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63571.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71955.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77698.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71955.log: /var/log/ceph/ceph-client.admin.47841.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47841.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71955.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54634.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77698.log: /var/log/ceph/ceph-client.admin.44217.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77698.log.gz 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78790.log 2024-04-26T21:22:21.629 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56641.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54634.log.gz/var/log/ceph/ceph-client.admin.78790.log: 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44385.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4%/var/log/ceph/ceph-client.admin.56641.log: -- replaced with /var/log/ceph/ceph-client.admin.78790.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75687.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56641.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54676.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-client.admin.75687.log: -- replaced with /var/log/ceph/ceph-client.admin.44385.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58519.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%/var/log/ceph/ceph-client.admin.54676.log: -- replaced with /var/log/ceph/ceph-client.admin.75687.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43706.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54676.log.gz 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2024-04-26T21:22:21.630 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58519.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.43706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43706.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80523.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80523.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70081.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48503.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63951.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40630.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70081.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57575.log 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63951.log.gz 2024-04-26T21:22:21.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55214.log 2024-04-26T21:22:21.652 INFO:teuthology.orchestra.run.smithi184.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2%/var/log/ceph/ceph-client.admin.57575.log: -- replaced with /var/log/ceph/ceph-client.admin.40630.log.gz 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48768.log 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57575.log.gz/var/log/ceph/ceph-client.admin.55214.log: 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55214.log.gz 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59246.log 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61034.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48565.log 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59246.log: -- /var/log/ceph/ceph-client.admin.55256.log 2024-04-26T21:22:21.806 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59246.log.gz 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64119.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55256.log.gz 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1%/var/log/ceph/ceph-client.admin.64119.log: -- replaced with /var/log/ceph/ceph-client.admin.48565.log.gz 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64119.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72938.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52302.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45568.log: /var/log/ceph/ceph-client.admin.72938.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72938.log.gz 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75975.log 2024-04-26T21:22:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52302.log: /var/log/ceph/ceph-client.admin.52134.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52302.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75488.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54388.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75975.log: /var/log/ceph/ceph-client.admin.75488.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75975.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75488.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63143.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77280.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63143.log: /var/log/ceph/ceph-client.admin.54388.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54388.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63143.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75530.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49014.log 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77280.log: /var/log/ceph/ceph-client.admin.75530.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77280.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75530.log.gz 2024-04-26T21:22:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63227.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49014.log: /var/log/ceph/ceph-client.admin.48930.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49014.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58131.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58131.log: /var/log/ceph/ceph-client.admin.63227.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63227.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58131.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59965.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49366.log 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44785.log: /var/log/ceph/ceph-client.admin.59965.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59965.log.gz 2024-04-26T21:22:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71167.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60157.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.71167.log: /var/log/ceph/ceph-client.admin.49366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71167.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49366.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62933.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.60157.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.68840.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.0.60157.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62933.log: /var/log/ceph/ceph-client.admin.68840.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62933.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68840.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56301.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49056.log: /var/log/ceph/ceph-client.admin.56301.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56301.log.gz 2024-04-26T21:22:21.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58173.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42910.log: /var/log/ceph/ceph-client.admin.58173.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58173.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80565.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74501.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80565.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80565.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.55653.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55298.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74501.log: /var/log/ceph/ceph-client.admin.55653.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74501.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57989.log 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55653.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55298.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55298.log.gz 2024-04-26T21:22:21.811 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.59375.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57989.log: /var/log/ceph/ceph-client.admin.59375.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57989.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41265.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50588.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59375.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76101.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68672.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76101.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41265.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76101.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69485.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57460.log 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68672.log: /var/log/ceph/ceph-client.admin.69485.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68672.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69485.log.gz 2024-04-26T21:22:21.812 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72595.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75933.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57460.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73940.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72595.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75933.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77059.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73940.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59331.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77059.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71333.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70470.log 2024-04-26T21:22:21.813 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59331.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56078.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70470.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71333.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70470.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70513.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56078.log: /var/log/ceph/ceph-client.admin.56120.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56078.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72511.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56120.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70513.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56120.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34821.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70204.log 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34821.log: /var/log/ceph/ceph-client.admin.72511.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72511.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34821.log.gz 2024-04-26T21:22:21.814 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67070.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70204.log: /var/log/ceph/ceph-client.admin.55028.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70204.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41863.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-client.admin.41863.log: -- replaced with /var/log/ceph/ceph-client.admin.67070.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52446.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41863.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.57905.log: gzip -5 --verbose 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.40356.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79346.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52446.log: /var/log/ceph/ceph-client.admin.40356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52446.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80103.log 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-04-26T21:22:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51908.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79346.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80103.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80103.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74035.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51908.log: /var/log/ceph/ceph-client.admin.42562.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51908.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57701.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66007.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74035.log: /var/log/ceph/ceph-client.admin.57701.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74035.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57701.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76807.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76807.log: /var/log/ceph/ceph-client.admin.66007.log: 88.8% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66007.log.gz 2024-04-26T21:22:21.816 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76807.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43264.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78910.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50458.log: /var/log/ceph/ceph-client.0.43264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43264.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65923.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62748.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78910.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69359.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65923.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63185.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69359.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62748.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69359.log.gz 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39926.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62370.log 2024-04-26T21:22:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63185.log: /var/log/ceph/ceph-client.admin.39926.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63185.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69653.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39926.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46604.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69653.log: 89.3% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69653.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62370.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71249.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72721.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46604.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76975.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72721.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71249.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72721.log.gz 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62496.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-04-26T21:22:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62496.log: /var/log/ceph/ceph-client.admin.76975.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76975.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62496.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46318.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56515.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46444.log: /var/log/ceph/ceph-client.admin.46318.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46318.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51317.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56515.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48111.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.51317.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51317.log.gz 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58570.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74375.log 2024-04-26T21:22:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48111.log: /var/log/ceph/ceph-client.admin.58570.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48111.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58570.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78748.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69779.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74375.log: /var/log/ceph/ceph-client.admin.78748.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74375.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78748.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78952.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68940.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69779.log: /var/log/ceph/ceph-client.admin.78952.log: 88.3% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69779.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78952.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46816.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78622.log 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68940.log: /var/log/ceph/ceph-client.admin.46816.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68940.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46816.log.gz 2024-04-26T21:22:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53226.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78622.log: /var/log/ceph/ceph-client.admin.41274.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78622.log.gz 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67364.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54472.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53226.log: /var/log/ceph/ceph-client.admin.67364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53226.log.gz 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72888.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67364.log.gz 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64605.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54472.log: /var/log/ceph/ceph-client.admin.72888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72888.log.gz 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42436.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47421.log 2024-04-26T21:22:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64605.log: /var/log/ceph/ceph-client.admin.42436.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64605.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42436.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49534.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51662.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47421.log: /var/log/ceph/ceph-client.admin.49534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47421.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49534.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80061.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62024.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51662.log: /var/log/ceph/ceph-client.admin.80061.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51662.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80061.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73896.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70162.log 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73896.log: /var/log/ceph/ceph-client.admin.62024.log: 88.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62024.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73896.log.gz 2024-04-26T21:22:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75855.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70162.log: /var/log/ceph/ceph-client.admin.49156.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70162.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50853.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.75855.log: /var/log/ceph/ceph-client.admin.50853.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50853.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75855.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63821.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46524.log: /var/log/ceph/ceph-client.admin.41270.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74333.log 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41270.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63821.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-04-26T21:22:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69276.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59857.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74333.log: /var/log/ceph/ceph-client.admin.69276.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74333.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69276.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59857.log: /var/log/ceph/ceph-client.admin.54944.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59857.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63695.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53957.log: /var/log/ceph/ceph-client.admin.77101.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.63695.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63695.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78706.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53747.log 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77101.log.gz 2024-04-26T21:22:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40103.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78706.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55695.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53747.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53270.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55695.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70661.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68594.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70661.log: /var/log/ceph/ceph-client.admin.53270.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53270.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70661.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73285.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68594.log: /var/log/ceph/ceph-client.admin.73285.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68594.log.gz 2024-04-26T21:22:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73285.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62228.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45064.log: /var/log/ceph/ceph-client.admin.45526.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60403.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73854.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60403.log: 88.9% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60403.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62228.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51824.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73854.log: /var/log/ceph/ceph-client.admin.51966.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73854.log.gz 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77360.log 2024-04-26T21:22:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.51824.log: -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51824.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61676.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76723.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61676.log: /var/log/ceph/ceph-client.admin.77360.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77360.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61676.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61616.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76723.log: -- /var/log/ceph/ceph-client.admin.60277.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76723.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72991.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61616.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78664.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60277.log.gz 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76678.log 2024-04-26T21:22:21.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78664.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72991.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71798.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78664.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70001.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76678.log: /var/log/ceph/ceph-client.admin.71798.log: 88.3% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71798.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76678.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50727.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67784.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70001.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50727.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6%/var/log/ceph/ceph-client.admin.67784.log: -- replaced with /var/log/ceph/ceph-client.admin.50727.log.gz 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67784.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63101.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74885.log 2024-04-26T21:22:21.828 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.63101.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63101.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74585.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.74885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74885.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76285.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74585.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74585.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52260.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76285.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58215.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52260.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61262.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58215.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58215.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61262.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61262.log.gz 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71651.log 2024-04-26T21:22:21.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47733.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71291.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71651.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70787.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71291.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71291.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74207.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.70787.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70787.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65139.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.74207.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74207.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42352.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65139.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65139.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54023.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42352.log.gz 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65965.log 2024-04-26T21:22:21.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54023.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54023.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68426.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65965.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65965.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68426.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68426.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40546.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80930.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51399.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80930.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80930.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59162.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49408.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68798.log 2024-04-26T21:22:21.831 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59162.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59162.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68798.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66860.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45610.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66860.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66860.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80187.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43881.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73201.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.80187.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80187.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79036.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.73201.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73201.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55070.log 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79036.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79036.log.gz 2024-04-26T21:22:21.832 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59289.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55070.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55070.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76059.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59289.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59289.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72201.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76059.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59755.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72201.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72201.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.59755.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59755.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60722.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50033.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62580.log 2024-04-26T21:22:21.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60722.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60722.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62580.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62580.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49679.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42730.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55611.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47249.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.55611.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55611.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41599.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47775.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48441.log 2024-04-26T21:22:21.834 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41599.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41599.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56725.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48441.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48441.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66178.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56725.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56725.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66178.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65259.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65259.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65259.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79472.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.52008.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76620.log 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79472.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79472.log.gz 2024-04-26T21:22:21.835 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67868.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.76620.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76620.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66902.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67868.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67868.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72123.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66902.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66902.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46066.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72123.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72123.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53915.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46066.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46066.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64971.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53915.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53915.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61076.log 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64971.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64971.log.gz 2024-04-26T21:22:21.836 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78065.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61076.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61076.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60992.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78065.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78065.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61118.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60992.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60992.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53365.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61118.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61118.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45352.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53365.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78257.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45352.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51578.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.78257.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78257.log.gz 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69401.log 2024-04-26T21:22:21.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51578.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51578.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51321.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.69401.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69401.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68898.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.51321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51321.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47337.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.68898.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68898.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53537.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47337.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47337.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72081.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.53537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40272.log 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.72081.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72081.log.gz 2024-04-26T21:22:21.838 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62664.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40272.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40272.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.62664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62664.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47463.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65055.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65055.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65055.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60950.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41601.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56809.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60950.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60950.log.gz 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58780.log 2024-04-26T21:22:21.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.56809.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56809.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71543.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58780.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58780.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64077.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.71543.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71543.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66732.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.64077.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64077.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77143.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.66732.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66732.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77610.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77143.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77143.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61220.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77610.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77610.log.gz 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74881.log 2024-04-26T21:22:21.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.61220.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61220.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67238.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.74881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74881.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41598.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.67238.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67238.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41598.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41598.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51063.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.60908.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51063.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51063.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65097.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44487.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77939.log 2024-04-26T21:22:21.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.65097.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65097.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79977.log 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.77939.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77939.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58089.log 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.79977.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79977.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44427.log 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.58089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58089.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51740.log 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44427.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44427.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54718.log 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51740.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51740.log.gz 2024-04-26T21:22:21.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.54718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54718.log.gz 2024-04-26T21:22:21.874 INFO:teuthology.orchestra.run.smithi184.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T21:22:21.940 INFO:teuthology.orchestra.run.smithi184.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T21:22:22.033 INFO:teuthology.orchestra.run.smithi184.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T21:22:22.033 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T21:22:22.095 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T21:22:22.130 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T21:22:22.164 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T21:22:22.214 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T21:22:22.253 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T21:22:22.290 INFO:teuthology.orchestra.run.smithi184.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T21:22:22.292 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-04-26T21:22:22.292 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m0.984s 2024-04-26T21:22:22.292 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m5.291s 2024-04-26T21:22:22.292 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.201s 2024-04-26T21:22:22.332 INFO:teuthology.orchestra.run.smithi040.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T21:22:22.375 INFO:teuthology.orchestra.run.smithi040.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T21:22:22.381 INFO:teuthology.orchestra.run.smithi040.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T21:22:22.457 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T21:22:22.459 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-26T21:22:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m1.187s 2024-04-26T21:22:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m6.431s 2024-04-26T21:22:22.459 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.682s 2024-04-26T21:22:22.460 INFO:tasks.ceph:Archiving logs... 2024-04-26T21:22:22.460 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi040/log 2024-04-26T21:22:22.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:22:23.853 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi184/log 2024-04-26T21:22:23.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:22:24.680 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:22:24.698 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:22:24.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:22:24.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:22:24.783 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:22:24.792 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:22:24.793 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:22:24.795 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:22:24.810 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:22:24.812 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +1159us[+1158us] +/- 41ms 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1012us[-1012us] +/- 58ms 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -732us[ -732us] +/- 74ms 2024-04-26T21:22:24.815 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +1154us[+1173us] +/- 41ms 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -999us[ -980us] +/- 58ms 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -721us[ -702us] +/- 74ms 2024-04-26T21:22:24.817 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:22:24.817 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:22:24.827 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:22:24.828 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:22:24.838 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:22:24.949 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:22:25.020 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:22:25.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:22:25.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:22:25.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:22:25.080 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:22:25.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:22:25.099 INFO:teuthology.task.internal:Duration was 1014.515799 seconds 2024-04-26T21:22:25.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:22:25.108 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:22:25.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:22:25.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:22:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:22:25.159 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:22:25.491 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:22:25.491 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-26T21:22:25.491 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:22:25.514 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-04-26T21:22:25.514 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:22:25.559 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:22:25.559 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:22:25.562 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:22:25.626 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:22:25.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:22:25.630 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:22:25.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:22:25.781 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:22:25.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:22:25.806 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:22:25.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:22:25.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:22:25.850 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:22:25.870 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-26T21:22:25.891 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-04-26T21:22:25.903 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:22:25.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:22:25.938 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:22:25.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:22:25.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:22:25.968 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:22:25.969 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi040 2024-04-26T21:22:25.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:22:26.013 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675205/remote/smithi184 2024-04-26T21:22:26.013 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:22:26.046 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:22:26.046 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:22:26.050 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:22:26.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:22:26.105 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:22:26.105 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:22:26.115 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:22:26.115 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:22:26.117 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:22:26.129 INFO:teuthology.orchestra.run.smithi040.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:22 /home/ubuntu/cephtest 2024-04-26T21:22:26.149 INFO:teuthology.orchestra.run.smithi184.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:22 /home/ubuntu/cephtest 2024-04-26T21:22:26.150 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:22:26.162 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:22:26.184 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits} duration: 1014.5157988071442 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:22:26.184 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:22:26.296 INFO:teuthology.run:pass